Content Comparison

...

Try to modify the previous code block to run in a new directory called BDIB_Annovar with from the .vcf files from the 3 individuals for both samtools and gatk that we looked at yesterday. Hint: you copied these files into your $SCRATCH/BDIB_Human_tutorial/raw_files directory yesterday.

launcher_creator.py -l annovar.sge -n annovar -t 00:30:00 -j commands -A UT-2015-05-18
qsub annovar.sge

We have ALREADY pre-computed these outputs (although Annovar will run pretty quickly on data from only chr20). 

more `which annovar_pipe.sh`

more `which summarize_annovar.pl`

ANNOVAR output

Annovar does a ton of work in assessing variants for us (though if you were going for clinical interpretation, you still have a long way to go - compare this to RUNES or CarpeNovo).  It provides all these output files:

cds
mkdir BDIB_Annovar
cd BDIB_Annovar
cp $SCRATCH/BDIB_Human_tutorial/raw_files/N*.vcf .
ls *.vcf | perl -n -e 'chomp; $_=~/(NA\d+).*(sam|GATK)/; print "annovar_pipe.sh $_ >$1.$2.log 2>&1\n";' > commands

launcher_creator.py -l annovar.sge -n annovar -t 00:30:00 -j commands -A UT-2015-05-18
qsub annovar.sge

We have ALREADY pre-computed these outputs (although Annovar will run pretty quickly on data from only chr20). 

more `which annovar_pipe.sh`

more `which summarize_annovar.pl`

ANNOVAR output

Annovar does a ton of work in assessing variants for us (though if you were going for clinical interpretation, you still have a long way to go - compare this to RUNES or CarpeNovo).  It provides all these output files:

NA12878.chrom20.samtools.vcf.hg19_esp5400_all_droppedexome_summary.csv
NA12878.chrom20.samtools.vcf.hg19exonic_esp5400variant_all_filteredfunction
NA12878.chrom20.samtools.vcf.hg19genome_genomicSuperDupssummary.csv
NA12878.chrom20.samtools.vcf.hg19_ljb_allALL.sites.2010_11_dropped
NA12878.chrom20.samtools.vcf.hg19_ljb_allALL.sites.2010_11_filtered
NA12878.chrom20.samtools.vcf.hg19_phastConsElements46wayavsift_dropped
NA12878.chrom20.samtools.vcf.hg19_snp132avsift_droppedfiltered
NA12878.chrom20.samtools.vcf.hg19_snp132_esp5400_all_dropped
NA12878.chrom20.samtools.vcf.hg19_esp5400_all_filtered
NA12878.chrom20.samtools.vcf.loghg19_genomicSuperDups
NA12878.chrom20.samtools.vcf.variant_function

I find the exome_summary.csv to be one of the most useful files because it brings together nearly all the useful information.  Here are the fields in that file (see these docs for more information, or the Annovar filter descriptions page here):

...

Alternative Allele Frequency in 1000 genomes pilot project 2012 Feb release (minor allele could be reference or alternative allele).

...

Functional prediction score for non-syn variants from Polyphen2 provided by dbNSFP  (higher score represents functionally more deleterious). A score greater than 0.85 corresponds to prediciton of "probably damaging". The prediciton is "possbily damaging" if score is between 0.85 and 0.15, and "benign" if score is below 0.15.

...

hg19_ljb_all_dropped
NA12878.chrom20.samtools.vcf.hg19_ljb_all_filtered
NA12878.chrom20.samtools.vcf.hg19_phastConsElements46way
NA12878.chrom20.samtools.vcf.hg19_snp132_dropped
NA12878.chrom20.samtools.vcf.hg19_snp132_filtered
NA12878.chrom20.samtools.vcf.log
NA12878.chrom20.samtools.vcf.variant_function

I find the exome_summary.csv to be one of the most useful files because it brings together nearly all the useful information.  Here are the fields in that file (see these docs for more information, or the Annovar filter descriptions page here):

Everything after the "LJB_GERP++" field in exome_summary came from the original VCF file, so this file REALLY contains everything you need to go on to functional analysis!  This is one of the many reasons I like Annovar.

Scavenger hunts!

Find the gene with two frameshift deletions in NA12878

Test "genetic drift" vs. "functional selection" - e.g. is the distribution of variants different among non-coding regions, synonymous changes in coding regions, and non-synonymous changes in coding regions?

...

Compare the output of these three commands:

...

Everything after the "LJB_GERP++" field in exome_summary came from the original VCF file, so this file REALLY contains everything you need to go on to functional analysis!  This is one of the many reasons I like Annovar.

Scavenger hunts!

grep "frameshift" NA12878.chrom20.GATK.vcf.exome_summary.csv  # this will print all the lines which contains the text "frameshift"
# From the output you can key into the first few columns having the information you are interested in: location-classification, gene, mutation type each separated by commas. This should lead you to think about adding the awk command to print only some columns.
 
grep "frameshift" NA12878.chrom20.GATK.vcf.exome_summary.csv | awk -F"," '{print $2"\t"$3}'  # the -F"," syntax forces it to split on commas
# you will likely notice this data is easier to visualize, and in this case you can probably see what gene is represented multiple times, but why stop there ... lets add the uniq -c command to the pipes to have linux count for us
 
grep "frameshift" NA12878.chrom20.GATK.vcf.exome_summary.csv | awk -F"," '{print $2"\t"$3}' | uniq -c 
# for the number of mutations we have this is sufficient, but for increased numbers of mutations where you may be interested in displaying them in a particular order. This can be done by adding the sort command

grep "frameshift" NA12878.chrom20.GATK.vcf.exome_summary.csv | awk -F"," '{print $2"\t"$3}' | uniq -c | sort -r  # the -r option on the sort command  sorts in reverse order

grep "frameshift" NA12878.chrom20.GATK.vcf.exome_summary.csv | awk -F"," '{print $2"\t"$3}' | uniq -c | sort -r
grep "frameshift" NA12878.chrom20.samtools.vcf.exome_summary.csv | awk -F"," '{print $2"\t"$3}' | uniq -c | sort -r

grep intergenic NA12878.chrom20.samtools.vcf.genome_summary.csv | awk 'BEGIN {FS=","} {print $26"\t"$27}' | sort | uniq -c | sort -n -r | head -20
grep exonic NA12878.chrom20.samtools.vcf.genome_summary.csv | grep -w synonymous | awk 'BEGIN {FS=","} {print $25"\t"$26}' | sort | uniq -c | sort -n -r | head -20
grep exonic NA12878.chrom20.samtools.vcf.genome_summary.csv | grep -w nonsynonymous | awk 'BEGIN {FS=","} {print $25"\t"$26}' | sort | uniq -c | sort -n -r | head -20

grep "frameshift" NA12878.chrom20.GATK.vcf.exome_summary.csv | awk -F"," '{print $2"\t"$3}' | uniq -c | sort -r

cat *GATK.vcf.exome_summary.csv | grep "exonic" | awk -F"," '{print $2,"\t"$3}' | sort | uniq -c | sort -n -rc | headsort -20

nr

Other variant annotators:

...