IGV Tutorial -- GVA2020

IGV Tutorial -- GVA2020

Overview

The Integrative Genomics Viewer (IGV) from the Broad Center allows you to view several types of data files involved in any NGS analysis that employs a reference genome, including how reads from a dataset are mapped, gene annotations, and predicted genetic variants.

Learning Objectives

  1. Create a custom genome database (usually used for microbial genomes).

  2. Load a pre-existing genome assembly (usually used for the genomes of model organisms and higher Eukaryotes).

  3. Load output from mapping reads to a reference genome.

  4. Load output from calling genetic variants.

  5. Navigate the view of the genome and interpret the display of this data.

Theory

Because NGS datasets are very large, it is often impossible or inefficient to read them entirely into a computer's memory when searching for a specific piece of data. In order to more quickly retrieve the data we are interested in analyzing or viewing, most programs have a way of treating these data files as databases. Database indexes enable one to rapidly pull specific subsets of the data from them.

The Integrative Genomics Viewer is a program for reading several types of indexed database information, including mapped reads and variant calls, and displaying them on a reference genome. It is invaluable as a tool for viewing and interpreting the "raw data" of many NGS data analysis pipelines.

With all that being said, the goal of visualizing the data with IGV is not to look at every read, or even every base in the reference genome (which is actually the smaller of the 2 possibilities!). IGV works best to dig deeper on something that you already are interested in either because of what gene it is, or is something that seems confusing (ie trying to figure out why mapping quality declines for a particular variant), or are trying to familiarize yourself with the concepts of what is going on. 

Installing IGV

This is done on your local computer not on TACC. IGV can not be installed on TACC which should make some sense to you as IGV is a program designed to let you visualize information and we know TACC doesn't allow GUIs.

There are multiple ways to launch IGV on a local computer. For this course I recommend in a separate a web browser window/tab going to: http://www.broadinstitute.org/software/igv/download and selecting the appropriate operating system. Mac users will need to unzip the file and launch the application. Window's users will need to download, choose an installation location, agree to some licenses, navigate to your installation location and then launch the program. Believe it or not, this is a significantly improved process compared to actions that used to be required.

Viewing E. coli data in IGV

Data files

You can start this tutorial two ways:

  1. If you have completed the Mapping tutorial and the SNV calling tutorial, then you should use those files for part 1 of this tutorial. You can proceed with either one alone, or with both.

Prepare a GFF feature file for the reference sequence

IGV likes its reference genome files in GFF (Gene Feature Format). Unfortunately, our old friend bp_seqconvert.pl doesn't deal with GFF. So, we're going to show you another tool for sequence format conversion called Readseq. We've already installed it into the $BI/bin directory so you don't have to, but here we provide the steps that can be used to install it in a local directory.

To use it you need to first download the file readseq.jar linked from here.

To get this onto TACC easily, use the wget command:
wget https://sourceforge.net/projects/readseq/files/readseq/2.1.19/readseq.jar

After that, you simply need to know where you downloaded it. As it is an executable $WORK or $HOME would be good places for it if you were going to use it on TACC, think about good places and naming systems to put in place in other contexts. In tomorrows final tutorial there will be a section about making java calls easier.

Readseq is written in java which makes it a little more complicated to use, but the general command to run the software is one of these (note that you do need to include the entire path, not just the "readseq.jar" command name):

java -jar /corral-repl/utexas/BioITeam/bin/readseq.jar java -cp /corral-repl/utexas/BioITeam/bin/readseq.jar run

This should return the help for Readseq.

Explaining the long invocation

You are actually using the command java and telling it where to find a "jar" file of java code to run. The -jar and -cp options run it in different ways. It is important to learn that java executables (.jar files) always require specifying the full path to the executable. In tomorrows final lecture we'll cover how you can work around this so you can build your own shortcuts and not have to remember where all your .jar files are stored (can be particularly difficult if you store them in different places (like some in your $HOME/local/bin directory, and some in various BioITeam directories.



To do the conversion that we want, use this command:

cds mkdir GVA_IGV_Tutorial cd GVA_IGV_Tutorial java -cp /corral-repl/utexas/BioITeam/bin/readseq.jar run $SCRATCH/GVA_bowtie2_mapping/NC_012967.1.gbk -f GFF -o NC_012967.1.gbk.gff

It's a bit hard to figure out because, unlike most conventions, it takes the unnamed arguments before the optional flag arguments, there is no example command, and you have to switch -jar to -cp. Search online for usage examples when you can't figure something out from the help. Take a look at the contents of the original Genbank file and the new GFF file and try to get a handle on what is going on in this conversion using commands like head and tail.

Copy files to your local computer

Again, since IGV is an interactive graphical viewer program that we'll be running on our local computer, we need to get the files we want to visualize onto your desktop machine.

The files we need include:

  • Indexed reference FASTA files

  • GFF reference sequence feature files

  • Sorted and indexed mapped read BAM files

  • VCF result files

  • Depending on your analysis you may need or can use additional files but those won't be discussed here



Rather than transferring each file individually, from multiple different directories, the easiest (and most common, and best practice) thing to do is:

  1. create a new directory (suggest you include a keyword like export)

  2. copy all the files you want to transfer into the new directory

  3. compress the directory to speed up the transfer

In the case of this tutorial, since many of the tutorial output files had the same names (but resided in different directories) we need to be sure to give them unique destination names when you copy them into the new directory together. Additionally, to ensure you don't overwrite files that you want (here is another reminder about there being no undo command in linux) you can (and is good practice to) use the -n or -i option with the cp command. On ls5 the -n command will not allow you to overwrite files, while the -i command will prompt you before overwriting anything. It is worth noting that there is a slight difference in different versions of linux/bash that with these command options and they may not work on all systems so double check with the help flag before using it. 

This is probably the largest code box in the entire course, note that some of these lines likely extend beyond the right side of the window and you may need to scroll to the right to see the entire command
cds mkdir GVA_IGV_export cp -i $SCRATCH/GVA_IGV_Tutorial/NC_012967.1.gbk.gff GVA_IGV_export # this is the new file you just created above cp -i $SCRATCH/GVA_bowtie2_mapping/NC_012967.1.fasta GVA_IGV_export cp -i $SCRATCH/GVA_samtools_tutorial/NC_012967.1.fasta.fai GVA_IGV_export cp -i $SCRATCH/GVA_samtools_tutorial/SRR030257.vcf GVA_IGV_export cp -i $SCRATCH/GVA_samtools_tutorial/SRR030257.sorted.bam GVA_IGV_export/bowtie2.sorted.bam cp -i $SCRATCH/GVA_samtools_tutorial/SRR030257.sorted.bam.bai GVA_IGV_export/bowtie2.sorted.bam.bai tar -czvf GVA_IGV_export.tar.gz GVA_IGV_export

Now, copy the entire compressed IGV directory back to your local Desktop machine. Remember there is a separate tutorial that covers scp file transfers in more detail that can be found here. In this case, you would replace README in that tutorial with GVA_IGV_export.tar.gz and need to determine the full path to that file using the pwd command.

In the Remote terminal (on the Right)

pwd

Expect this to be just the $SCRATCH directory, but remember your local computer won't know what $SCRATCH means.

In the Left terminal (on the Left). Remember to replace the text in the <> brackets with your own information:

scp -r <username>@ls5.tacc.utexas.edu:<full_path_to_file>/GVA_IGV_export.tar.gz . # enter your password/tacc token

Once you have transferred the compressed file, you will need to expand the file on your local machine so that you can access the individual files in IGV. 

You can use the tar --help option to try to figure out how to extract the contents, but I suggest the the following command:
tar -xzvf GVA_IGV_export.tar.gz





For the remainder of the tutorial, work on your local machine. NOT TACC!

Load genome into IGV