Annotating Variants: Introduction
As we've already seen, determining the presence or absence of a variant from NGS data is not trivial. It is software dependent and has inherent trade-offs between sensitivity and specificity. Inevitably, the number of putative variants in a real data set is very large; for example the first samples from the 1000 Genomes project typically found 0.1% variants (3 million variants), approximately 10% of which had never been previously observed (300,000 novel variants per individual.) False positive discovery rates are also typically very high at this stage.
...
Because these tools draw in information from may disparate sources, they can be very difficult to install, configure, use, and maintain. For example, the vcf
files from the 1000 Genomes project are arranged in a deep ftp tree by date of data generation. Large genome centers spend significant resources managing these tools.
Pre-packaged programs
Annovar - one of the most powerful yet simple to run variant annotators available
Annovar is a variant annotator. Given a vcf file from an unknown sample and a host of existing data about genes, other known SNPs, gene variants, etc., Annovar will place the discovered variants in context.
...
This next exercise will give you some idea of how Annovar works; we've taken the liberty of writing the bash script annovar_pipe.sh
around the existing summarize_annovar.pl
wrapper (a wrapper within a wrapper - a common trick) to even further simplify the process for this course.
Exercise:
First we want to move to a new location on $SCRATCH
Code Block |
---|
language | bash |
---|
title | Solution to make a new directory on scratch. You should know this answer. |
---|
collapse | true |
---|
|
cds
mkdir BDIBGVA_Annovar
cd BDIBGVA_Annovar |
Next, look at the code for our annovar_pipe.sh
command.
Expand |
---|
title | Click here for a hint |
---|
|
The which command is used to give the location of a program or script that is in your $PATH. Code Block |
---|
language | bash |
---|
title | Here is an easy one-liner to cat the contents of a script (note ` is a back-tick, not apostrophe) |
---|
collapse | true |
---|
| cat `which annovar_pipe.sh` # the command within the `` is evaluated first and the output placed within the `` marks to be evaluated by the outer command
|
This script simply does a format conversion and then calls summarize_annovar.pl . Which begs the question what does summarize_annovar.pl do?
Expand |
---|
title | Figure out what summarize_annovar.pl is doing using the back tick trick you just learned |
---|
| Again note the ` characters are "backtick", not apostrophes Code Block |
---|
title | Print out the text of the perl script summarize_annovar.pl |
---|
| more `which summarize_annovar.pl`
|
The more command was used to make it slightly easier to read and to remind you that there are multiple options of how to look a things (less, head, tail, etc), eventually you will develop your own habits but the more you are exposed to the fewer instances of angry eureka moments you will have. When working with unknown files or file types, cat should typically be avoided unless you run a wc -l command else you risk crashing your terminal window. |
This script is written in perl (a programing language that while powerful, is not very favored among most computational biologists who typically prefer: Python, R, or Bash shell scripts) as a method of standardizing input to call a series of external commands. Such wrappers (or wrappers within wrappers) make your life much easier. breseq itself has wrappers for using bowtie2 and samtools which you ran separately on other data. As you increase your understanding of scripts and the command line by looking at what others have done you may begin to make your own wrappers and small scripts, but such wrappers are a great example of the BioITeam and other community resources can provide you with. While we I have told you no one will ever care as much about your data as you do, quality of life issues regarding repetitive treatment of similar inputs and outputs may be easily solved by someone else. |
...
Code Block |
---|
language | bash |
---|
title | Click here to check your work |
---|
collapse | true |
---|
|
# Use only 1 of the following copy commands
# if you have already done the trios tutorial
cp $SCRATCH/BDIBGVA_Human_tutorialtrios/raw_files/N*.vcf .
# if you have not done the trios tutorial
cp $BI/ngs_course/human_variation/N*.vcf .
ls *.vcf | perl -n -e 'chomp; $_=~/(NA\d+).*(sam|GATK)/; print "annovar_pipe.sh $_ >$1.$2.log 2>&1 &\n";' > commands
|
Note |
---|
title | A helpful note to make your life better. |
---|
|
"commands" files are 1 way to both create a record of what it is that you are doing as well as an easy way to execute multiple commands simultaneously. In a previous tutorial (the advanced breseq tutorial) we showed you how to do this by adding an & as the last character of the line to force the command to execute in the background. Tomorrow we will go over the more common method of submitting "commands" files to the que to run as jobs rather than being spoiled with interacting with everything on idev nodes. In computational biology this instructor has often found that once I learn to do something a particular way, it takes an incredible amount of inertia to change how I do something even when i know it would help to do so. In an effort to learn from me making things harder on myself than need be, I urge you to STRONGLY consider not naming every list of commands you want to run "commands" but rather something that is actually descriptive, even "commands_DATE" would be helpful. As you start submitting your own jobs you will quickly be able to build bad habits, try to be aware of and avoid this one if possible. |
Expand |
---|
title | investigate the commands file to to determine what the piping command actually did (click for answer) |
---|
|
Code Block |
---|
language | bash |
---|
title | Think about the commands that let you look at what the contents of a file are |
---|
collapse | true |
---|
| cat commands |
|
...
Warning |
---|
title | Make sure you are on an IDEV node using the showq -u command |
---|
|
Code Block |
---|
language | bash |
---|
title | How to request an idev node if needed |
---|
collapse | true |
---|
| # if doing this tutorial on WednesdayThursday:
idev -m 180 -r CCBB_5.24.17PM_Day_3 -A UT-2015-05-18
# if doing this tutorial on ThursdayFriday:
idev -m 180 -r CCBB_5.25.17PM_Day_4 -A UT-2015-05-18 |
Code Block |
---|
title | Create the submission script and submit |
---|
| chmod +x commands
./commands |
|
The easiest way to check if this is working is to use ls and see an explosion of new files. This will take quite a bit of time to complete running. As such, we have ALREADY pre-computed these outputs so you can begin evaluating the results.
Code Block |
---|
language | bash |
---|
title | Use this code block to copy the completed results |
---|
|
cds
cd BDIBGVA_Annovar
mkdir provided_results
cd provided_results
cp $BI/ngs_course/human_variation/*chrom20.samtools* . |
Later, when your results have completed running (likely >90 minutes as annovar not configured to use multiple processes, compare your results to these provided results).
For the scavengar scavenger hunts below you should also copy over the GATK results.
Code Block |
---|
language | bash |
---|
title | Use this code block to copy the completed results |
---|
|
cp $BI/ngs_course/human_variation/*chrom20.GATK* . |
ANNOVAR output
Annovar does a ton of work in assessing variants for us (though if you were going for clinical interpretation, you still have a long way to go - compare this to RUNES or CarpeNovo). It provides all these output files:
...
The exome_summary.csv
is probably the most useful files because it brings together nearly all the useful information. Here are the fields in that file (see these docs for more information, or the Annovar filter descriptions page here):
Func | exonic, splicing, ncRNA, UTR5, UTR3, intronic, upstream, downstream, intergenic |
Gene | The common gene name |
ExonicFunc | frameshift insertion/deletion/block subst, stopgain, stoploss, nonframeshift ins/del/block stubst., nonsynonymous SNV, synonymous SNV, or Unknown |
AAChange (in gene coordinates) |
|
|
Conserved (i.e. SNP is in a conserved region) | based on the UCSC 46-way conservation model |
SegDup (snp is in a segmental dup. region) |
|
|
ESP5400_ALL | Alternate Allele Frequency in 3510 NHLBI ESP European American Samples |
1000g2010nov_ALL | Alternative Allele Frequency in 1000 genomes pilot project 2012 Feb release (minor allele could be reference or alternative allele). |
dbSNP132 | The id# in dbSNP if it exists |
AVSIFT | The AVSIFT score of how deleterious the variant might be |
LJB_PhyloP | Conservation score provided by dbNSFP which is re-scaled from original phylop score. The new score ranges from 0-1 with larger scores signifying higher conservation. A recommended cutoff threshold is 0.95. If the score > 0.95, the prediction is "conservative". if the score <0.95, the prediction is "non-conservative". |
LJB_PhyloP_Pred |
|
|
LJB_SIFT | SIFT takes a query sequence and uses multiple alignment information to predict tolerated and deleterious substitutions for every position of the query sequence. Positions with normalized probabilities less than 0.05 are predicted to be deleterious, those greater than or equal to 0.05 are predicted to be tolerated. |
LJB_SIFT_Pred |
|
|
LJB_PolyPhen2 | Functional prediction score for non-syn variants from Polyphen2 provided by dbNSFP (higher score represents functionally more deleterious). A score greater than 0.85 corresponds to prediciton of "probably damaging". The prediciton is "possbily damaging" if score is between 0.85 and 0.15, and "benign" if score is below 0.15. |
LJB_PolyPhen2_Pred |
|
|
LJB_LRT | Functional prediction score for non-syn variants from LRT provided by dbNSFP (higher score represents functionally more deleterious. It ranges from 0 to 1. This score needs to be combined with other information prediction. If a threshold has to be picked up under some situation, 0.995 can be used as starting point. |
LJB_LRT_Pred |
|
|
LRT_MutationTaster | Functional prediction score for non-syn variants from Mutation Taster provided by dbNSFP (higher score represents functionally more deleterious). The score ranges from 0 to 1. Similar to LRT, the prediction is not entirely depending on the score alone. But if a threshold has to be picked, 0.5 is the recommended as the starting point. |
LRT_MutationTaster_Pred |
|
|
LJB_GERP++ | higher scores are more deleterious |
Chr |
| | |
|
Ref | Reference base |
Obs | Observed base-pair or variant |
SNP Quality value |
|
|
(ALL the VCF info is here!!) |
| | ...
Everything after the "LJB_GERP++" field in exome_summary came from the original VCF file, so this file REALLY contains everything you need to go on to functional analysis! This is one of the many reasons I like Annovar.
Scavenger hunts!
Expand |
---|
title | Find the gene with two frameshift deletions in NA12878. See what you can come up with as answer on your own and then click here for an answer and an expansion of how to generate more meaningful representations of that data |
---|
|
The final answer is "DEFB126" Code Block |
---|
| grep "frameshift" NA12878.chrom20.GATK.vcf.exome_summary.csv # this will print all the lines which contains the text "frameshift"
# From the output you can key into the first few columns having the information you are interested in: location-classification, gene, mutation type each separated by commas. This should lead you to think about adding the awk command to print only some columns.
grep "frameshift" NA12878.chrom20.GATK.vcf.exome_summary.csv | awk -F"," '{print $2"\t"$3}' # the -F"," syntax forces it to split on commas
# you will likely notice this data is easier to visualize, and in this case you can probably see what gene is represented multiple times, but why stop there ... lets add the uniq -c command to the pipes to have linux count for us
grep "frameshift" NA12878.chrom20.GATK.vcf.exome_summary.csv | awk -F"," '{print $2"\t"$3}' | uniq -c
# for the number of mutations we have this is sufficient, but for increased numbers of mutations where you may be interested in displaying them in a particular order. This can be done by adding the sort command
grep "frameshift" NA12878.chrom20.GATK.vcf.exome_summary.csv | awk -F"," '{print $2"\t"$3}' | uniq -c | sort -r # the -r option on the sort command sorts in reverse order |
|
...
Expand |
---|
title | Thinking back to our first example where we looked for a gene with 2 frameshift deletions, can you think of a way to look at all 3 patients' GATK analysis results at once, and look for the total number of mutations detected (sorted by gene) displayed in descending order? |
---|
|
The final form of the command that you had was:grep "frameshift" NA12878.chrom20.GATK.vcf.exome_summary.csv | awk -F"," '{print $2"\t"$3}' | uniq -c | sort -r Code Block |
---|
language | bash |
---|
title | Try to come up with a solution on your own before checking your answer here |
---|
collapse | true |
---|
| cat *GATK.vcf.exome_summary.csv | grep "exonic" | awk -F"," '{print $2,"\t"$3}' | sort | uniq -c | sort -nr |
Some questions for thought: - Why might this type of analysis be useful?
- What other greps would make this more useful?
|
Other variant annotators:
Notes
Variants consist of single base base changes, insertions and deletions, and larger scale structural changes. "Larger scale" is usually defined relative to the capabilities of the technology; for example, a "small indel" usually means "detectable within a single sequence read". In 2009, sequence reads were about 50 bp, by 2011 they were 100 bp, and today the standard hiSEQ run is 150bp and miSEQ runs can be 600bp.
Return to GVA2017GVA2019