Our clusters use the Linux Operating system as their core together with some add-on tools provided by the San Diego Super Computing Center. Together the complete package as known as ROCKS. Since UNIX is the core of the operating system part of the process of wielding the power of the cluster is learning how use the various UNIX provided, and other commands to manipulate your data.
To help in this process
Our clusters are Linux based, so to get the maximum use of them you'll need to become familiar with the UNIX command line, which is a powerful text based interface which lets you type in commands to the computer for evaluation. This can be a bit of a barrier to getting your work done, but with perseverance you can learn enough UNIX to be at least able to use the cluster. While we can answer some quick, "how do I do this", or "why is this" type questions, in the long term it's better that you bite the bullet and get this done. Here are some resources that we have run across
- O'Reilly and Assocates publishes very good books that range from beginner to reference. In particular, Learning the UNIX Operating System is very good.
- Computer Science offers CS 108: UNIX, 1 hour course that is meant for beginners. If you don't take the class you might sit in, or get the book Linux: The Textbook which is pricey, but also very good.
- Several people have taken courses offered by Perl Source Informatics but they can be pricey.
You may also find loads of stuff using Google, and hopefully we will write our own version. It's just the case that right now we need more work on infrastructure, so spending time writing up own on version of things is counterproductive. If someone is willing to buy some of these reference texts, and donate them to use, we are willing to serve as the library. We will also be willing to track interest in the Perl Source Informatics courses as they do have bulk purchasing pages.
Initially when you use the cluster you can use some of our pre-canned scripts. After awhile you will probably be interested in write your own, or perhaps you will be forced to do so. While you can continue to use our scripts as templates for your own you might also want to get some books on scripting, and programming. First, you can get quite decent scripting done just by learning more about the shell we use which is the BASH shell. O'Reilly and Associates ahs a book on BASH which is called Learning the BASH Shell. This would also be useful for people wanting just to learn more about the powerful things they can do on the command line, or who want more information about the things they can do to customize their environment. They also have books on Perl, and Python for people interested in using BioPerl, or BioPtyhon. Computer Science also has Perl, Python, C++, and possibly other programming classes listed under the CS 105 class headinghttp://x.cygwin.com/. You will also want to read our documentation on the SGE Batch system which you will use to submit jobs to the cluster.Labels parameters
While it would be nice to provide our own spin on how to use UNIX, and the power of the UNIX operating system, right now our time is better spent elsewhere. To get the most use of of our clusters, or any other UNIX system you should learn a little bit about how UNIX works, and how to make minimal use of the bash shell. Our recommendation is to invest in books publishd by O'Reilly and Associates. ORA has very good beginner's books. One that seems good is Learning the UNIX Operating System . As you progress in your use of UNIX, you may also want to pick books up on bash, shell scripting, perl scripting, and/or python scripting (as needed).
To use 3rd party software we have added, you will use the module command.
Here is a A Little UNIX Dictionary which will let you use many of the buzzwords that you here.