Introduction

Throughout the course we have focused on samll data sets, a limited number of samples, and in some cases even purposefully capped the total number of reads you have access to. This has been done for the purpose of time and letting you see the results tick by rather than simply having you come in for 30 minutes, submit a job, and wait an hour (or 6) before it starts running,  and have it take another 10 hours to run. The reality is while you will sometimes work with a test sample or a small pilot project, Big Data in Biology means LOTS of data and lots of data means needing to not just identify variants in 1 sample, but to identify commonality across different systems. here we introduce you to bedtools. A program designed to make comparisons across differnt file types generaterated from different samples or using different parameters of a given pipeline.

Objectives

1. Become familiar with how to use bedtools
2. Understand when and how bedtools is useful

Comparing the results of different mappers using bedtools

Often you want to compare the results of variant calling on different samples or using different pipelines. Bedtools is a suite of utility programs that work on a variety of file formats, one of which is conveniently VCF format. It provides many ways of slicing, dicing, and comparing the information in VCF files. In this tutorial we will use it on .vcf files generated with samtools after mapping with each of 3 different read mappers (bowtie2, bwa and bowtie) to determine what predictions are the same and which are different from the variant calling on reads mapped with different programs. You could do this with your own personally generated data from the SRR030257 fastq files we used in the mapping tutorials. For now, create a new directory named BDIB_bedtools on scratch and copy the respective VCF files to it from $BI/gva_course/bedtools/:

mkdir $SCRATCH/BDIB_bedtools 
cd $SCRATCH/BDIB_bedtools
cp  -i $BI/gva_course/bedtools/*.vcf .

Remember the above command is simply 1 possible solution there are multiple ways you could have done this, most commonly recurrsively copying the entire directly, or copying all the files rather than just the vcd files. Useing the subcommands bedtools intersect and bedtools subtract we can find equal and different predictions between mappers. Try to figure out how to to do this on your own first. Hint:

bedtools intersect -a bowtie2.vcf -b bwa.vcf > common_bowtie2_bwa.vcf

bedtools subtract -a bowtie2.vcf -b common_bowtie2_bwa.vcf > unique_bowtie2.vcf
bedtools subtract -a bwa.vcf -b common_bowtie2_bwa.vcf > unique_bwa.vcf

#### may need to be added to the top