Overview

idev (short for interactive development) sessions on TACC are a method of accessing one of the compute nodes directly rather than sending a slurm job off using the sbatch command. In this course, it will let us view the text that would normally print to the screen. We will further benefit from TACC setting up a reservation for us that will allow us to skip the queue and immediately launch an interactive session so we spend our class time working rather than waiting.

Learning Objectives

Learn the importance of launching an idev session
Launch an idev instance
Check if you are currently in an idev session.

Tutorial

Importance of using idev sessions

As mentioned in the introduction tutorial, idev sessions are important to get off the head node and not negatively impact all other TACC users. As mentioned in the bowtie2 tutorial failure to do so has actually caused TACC to crash in previous years, and none of us (you included) want to be part of that happening again. As such most tutorials will contain a warning box similar to this and a link back to this page to ensure you are staying on an idev node and not being a bad TACC citizen.

Launching an iDev session with a reservation

As we discussed in our first tutorial the head node is a space shared by all and we don't like stepping on each others toes. While the launcher_creator.py helper script makes working with the compute nodes much easier, they still take time to initiate a run (waiting in the queue) and if you have errors in your commands your job will fail and you will lose your place in line. An idev (or interactive development) session is a way to move off the head node and onto a single compute node, but work interactively to see if your commands actually work, give you much quicker feedback, and if everything goes as you hope, your data. idev sessions are much more limited in duration and in general its not necessary to see every line a program spits out once you are familiar with the type of data you will get. Additionally, we are going to use a priority access reservation set up special for the summer school that you normally would not have access to but should guarantee immediate starting of your idev session.

Your idev command line will contain 3 flags: -m, -r -A. Using the `idev -h` command, can you figure out what these 3 flags mean and what you told the system you wanted to do?

Click here to see if you are correct...

From the OPTIONS: section of the idev help output:

-m minutes sets time in minutes (default: 30)

-r reservation_name requests use of a specific reservation

-A account_name sets account name (default: -A none)

So you will request an idev node using the reservation named Genomic_Variant_Analysis, and asked that it be charged to the account named UT-2015-05-18.

The question becomes 'how long do we need to request the idev node for'? As the reservation only runs between 9am and 12pm each day this week, you need to request the enough minutes to set it to end a few minutes before 12:00 so the session will start. If you set it to end after 12:00 it won't actually launch.

Copy and paste the following command replacing the trailing 3 ? marks with the number of minutes you want your idev session to last:

Starting an idev session

idev -r Genomic_Variant_Analysis -A UT-2015-05-18 -m ???

Expected output.

We found an ACTIVE reservation request for you, named Genomic_Variant_Analysis. 
Do you want to use it for your interactive session? 
Enter y/n [default y]:

If for any reason you don't see the above message let me know on zoom. As mentioned above, trying to complete most of the tutorials on the head node will cause problems for you or everyone.

After you answer yes by hitting the 'y' key, you should see the following line followed by some additional depending on if you are on multiple different project allocations.

->Reservation      : --reservation=Genomic_Variant_Analysis (ACTIVE)


# Some of you may see a new prompt stating something like the following: 


We need a project to charge for interactive use. 
We will be using a dummy job submission to determine your project(s). 
We will store your (selected) project $HOME/.idevrc file. 
Please select the NUMBER of the project you want to charge.\n 
1 OTHER_PROJECTS
2 UT-2015-05-18


Please type the NUMBER(default=1) and hit return:

If you see the text prompting for a number MAKE SURE you select the number associated with the 'UT-2015-05-18' project.

Everyone will then see something like the following:

# job status: PD # job status: R 
# --> Job is now running on masternode= nid00032...OK 
# --> Sleeping for 7 seconds...OK 
# --> Checking to make sure your job has initialized an env for you....OK 
# --> Creating interactive terminal session (login) on master node nid00032.

If the above takes more than 60 seconds without giving you your prompt back, again get my attention and be ready to share your screen so we can figure out what is going on.

Verifying you are in an iDev session

It is best to check if you are on an idev session before executing a command that you expect to be taxing for TACC to execute.

The simplest way is to use the following command

showq -u

If you are on an idev node you should see an "ACTIVE JOB" with a name including a job name beginning with 'idv'. Further you will see a countdown of how much longer the session will be active.

Launching an iDev session without a reservation

The same command used above can be used, omitting the '-r' flag. The output then becomes much more simple as at most you will be prompted for a choice of what allocation to use.

Starting an idev session

idev -A UT-2015-05-18 -m 30

30 minutes is a good amount of time to select, nearly all of the tutorials can be completed in that time. You can use the 'logout' command to exit an idev node if you finish 1 tutorial and are about to start another and notice the showq -u command only has 5 or 10 minutes left on it.

Here is a link back to the GVA2020 home page, but remember you can use your browser navigation to go back to the page you just came from.