launcher_creator GVA2021 -- unfinished

does not appear to work with stampede2. incomplete tutorial

Overview:

This "mini tutorial" is designed in the vein of the idev, ssh, and scp tutorials to give the most basic information for using this script. Friday's tutorial and presentation will provide more of the theory of how this script works, and the TACC resources that are being accessed.

Objectives:

  1. Explain what a commands file is, what should go in it, and how it will behave.
  2. Explain most important options of launcher_creator.py script. 
  3. Use sbatch to submit a job.
  4. Give information of how to check stats of a job.

Creating a commands file:

What it is:

List of individual commands to be executed on a compute node. 

What goes in it:

These are the commands that you initially have typed in the interactive idev sessions, but here they will be executed remotely without you watching them run.

How it will behave:

Each line will be executed 1 at a time as if you had typed it directly onto the command line on a compute node. If you specify multiple "tasks" using the launcher_creator.py script, multiple lines will execute at the same time. 

Using launcher_creator.py:

Explanation:

The launcher_creator.py script is a small python script developed by the BioITeam to help put together launcher files 

Command line options most likely used:

Short optionLong optionRequiredDescription

-n

name

Yes

The name of the job.

-t

time

Yes

Time allotment for job, format must be hh:mm:ss.

-j

job

j and/or b must be used

Filename of list of commands to be distributed to nodes.

-q

queue

Default: Development

The queue to submit to, like 'normal' or 'largemem', etc.

-a

allocation

If you are on multiple allocations

The allocation you want to charge the run to.

-w

wayness

Kind of 

Optional The number of jobs in a job list you want to give to each node. (Default is 12 for Lonestar, 16 for Stampede.)

-N

number of nodes


Optional Specifies a certain number of nodes to use. You probably don't need this option, as the launcher calculates how many nodes you need based on the job list (or Bash command string) you submit. It sometimes comes in handy when writing pipelines.

Submitting a job: