Submitting a simple job using launcher_creator.py (GVA14)

Let's submit a simple test project to Lonestar.

1. First, copy over this simple profiling script to your local homedir:

cp $BI/bin/nodeprobe.sh .

Feel free to look inside it to see what it's doing. You will note that it makes use of the system's RAM disk, /dev/shm. You might want to try using /dev/shm for files if your applications are disk I/O bound.

2. Now let's run three commands in parallel on one node. Create a file called "commands" with any three commands you'd like, such as these

date
ls -l
./nodeprobe.sh

3. Now, use launcher_creator.py to wrap a job submission script around that commands file.

launcher_creator.py -n testjob -j commands -t 00:10:00 -a DNAdenovo -q development

4. Use qsub to submit your batch job script

5. Use qstat to examine your job in the queue

6. When your job is done, examine the ".o<JID>" file - notice how cores are allocated on a single node, one per command.

• • If the commands file had 13 commands, 12 would have been executed on one node and the 13th on a second node.
  • If you need to limit commands to, say, 4 per node, check out the -w and -N options to launcher_creator.py.  This may be helpful if you have a multi-threaded application, like a mapper, that you want to run multiple instances of on each node.