Agenda for the meeting. The only item on the agenda of this meeting is a discussion of clustering software for the new orange and teal clusters.

Ed has pared down the clustering software options and developed a page listing their attributes and cost.

The 3 candidates are codine, PBS and LSF.
Codine is a German product (they are starting a US Branch of support).
PBS - free source - but you pay for support.
LSF is a commercial product.

OS issues.
LSF is the only product that states it currently supports Solaris 2.7 (Hamp has installed 2.7 on the new orange suns).
With PBS we can pay them (how much - is not clear) to hurry the porting of the code to 2.7. The PBS page indicates that the product has been ported to 2.5 (could find no mention of 2.6). This may present a problem as 2.6 has been around a while and a failure to port to that OS could indicate that this product is not being maintained.
Codine will have support for 2.7 in "the next couple of weeks".

Comparing the functionality of the 3 packages.

• They all do batch.
  
• They all can run parallel programs using MPI (Condor could only use PVM).
  
• They all do checkpointing. The difference is that PSM and Codine do the checkpointing at the system level - so the ability to checkpoint will depend on whether the OS (irix, solaris, or AIX) itself provides for checkpointing. The group wasn't sure what the various OSs provided for (Tom S felt there was an AIX library for checkpointing and several other members felt that Irix provided for checkpointing).
  LSF provides checkpointing through linking with libraries - this is nice is that it removes the OS requirement - the difficulty lies with the various packages users use on the SP. Checkpointing would not be available within those packages unless the packages were recompiled (!) - not very likely.
• Process Migration - PBS is the only package that doesn't do process migration. This feature would have been a requirement had we decided to include the unixlab machines into the cluster. Since we are not - the ability to migrate processes is not that big of a deal and won't count "against" PBS. If we want to idle a machine - we can just do what is done (and works well on) the sp - that is configure the cluster management software to stop dispatching processes to the CPU of interest.
• They all do load balancing.
• They all allow for run time limits.
• They all have configurable schedulers.
• They all have GUI interfaces.
• Only PBS is public domain - LSF and Codine are commercial.

Pricing Issues
PBS is far away the cheapest option - so the bias of the group was to go that route unless there was a reason NOT to.
The cost of the software had implications beyond ITC - we have many departments either building clusters or interested in building them - and we'd like to be able to suggest a low cost/free solution to them.
Between the two commercial packages - LSF appears to be a bit more desirable as that software is bundled into Enterprise type packages offered by Sun and SGI.
Ed and Hamp seemed to have gotten a bit of conflicting info regarding the option of buying unbundled LSF from Sun and SGI. In any case, if PBS doesn't pan out - we made need to investgate purchase options for LSF.

The winner is PBS!
The only difficulty with PBS is that it is unclear whether it has been ported to Solaris 2.6 (their web page only indicates availability on 2.5).
Ed will contact PBS and see if it is available on 2.6 (and then mail us) - if it is - Hamp will remove 2.7 from the cluster machines - install Solaris 2.6 - install PBS and then we can begin testing.

Meeting adjourned at 11 am, next meeting at: June 28 10:00 AM

Go to: ITC Research Computing Committee Home Page

by Dawn Adelsberger-Mangan