Versions Compared

Old Version 5

changes.mady.by.user Deatherage, Daniel E

Saved on

compared with

New Version Current

changes.mady.by.user Deatherage, Daniel E

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

Introduction

Throughout the course we have focused on samll data sets, a limited number of samples, and in some cases even purposefully capped the total number of reads you have access to. This has been done for the purpose of time and letting you see the results tick by rather than simply having you come in for 30 minutes, submit a job, and wait an hour (or 6) before it starts running,  and have it take another 10 hours to run. The reality is while you will sometimes work with a test sample or a small pilot project, Big Data in Biology means LOTS of data and lots of data means needing to not just identify variants in 1 sample, but to identify commonality across different systems. here we introduce you to bedtools. A program designed to make comparisons across differnt file types generaterated from different samples or using different parameters of a given pipeline.

A note on version control

As mentioned in our samtools tutorial, different versions of software behave differently. Once again we have a situation where the BioITeam bedtools is available by default (version 2.20.1), and a different version is available on lonestar (version 2.25.0). A KEY addition to version 2.21 (aka the version after that which is available to you by default through the BioITeam as of this writing) was the ability of bedtools to simultaneously scan multiple files at once rather than having to sequencially scan pairs of files. Using the old version, to identify the common variants of files A, B, C and D, bedtools would have to be invoked a minimum of 3 times:

...

Warning
titleBased on what was just said, what do you think the first 2 things you should do are?
Expand
titleNeed a hint?
  1. Different versions exist
  2. Can be very computationally intensive
Expand
titleThink you know? Check your guess...

Because different versions exist, you need to make sure that you are using the right version:

Code Block
languagebash
titleCheck that you remember how to check if you have access to a command, what version that command is, load a different version, and verify that is the version being used
collapsetrue
which bedtools  # check if you have access to bedtools
bedtools --version  # determine what version of bedtools you are using
 
# This will tell you that the bedtools you are using is in the $BI/bin directory and that you are using version v2.20.1-2-g6f0be00
 
module load bedtools  # load the TACC version of bedtools, remember module spider module avail and module list can help you find exactly what module you are looking for
which bedtools
bedtools --version
 
# This will now tell you that you are using version v2.25.0 which is located in the /opt/apps/bedtools/2.25.0/bin dreictory.

 

Because it is computationally intensive, you need to make sure you are not on the head node

Code Block
languagebash
titleCheck that you remember how to get an idev node (remember to change your reservation to day 3)
collapsetrue
idev  -m 180 -r CCBB_Bio_Summer_School_2016_Day3 -A UT-2015-05-18 -N 2 -n 8

Surely now that you have executed the above commands nothing could possibly go wrong right?

Expand
titleHopefully you recognize the bold italic underlined sentence above as a warning that something can (and is likely about to go wrong), but if not ... something is about to go wrong, so try to figure out what that might be, and then click here to see if you are right
 If you executed the above commands in the order they are listed from the head node (say because its the first tutorial you ran today and you are not already on a n idev node) you will find that bedtools has reverted to the BioITeam location and version because when you access an idev node it reloads your .bashrc file. Always make sure you are loading tools when and where you need them, and verify their accuracy rather than assuming it. module load bedtools

 

Objectives

  1. Become familiar with how to use bedtools intersect and subtract
  2. Understand when and how bedtools is useful

 

Comparing the results of different mappers using bedtools

Often you want to compare the results of variant calling on different samples or using different pipelines. Bedtools is a suite of utility programs that work on a variety of file formats, one of which is conveniently VCF format. It provides many ways of slicing, dicing, and comparing the information in VCF files. In this tutorial we will use it on .vcf files generated with samtools after mapping with each of 3 different read mappers (bowtie2, bwa and bowtie) to determine what predictions are the same and which are different from the variant calling on reads mapped with different programs. 

Getting some data to work with

You could do this with your own personally generated data from the SRR030257 fastq files we used in the mapping tutorials. For now, create a new directory named BDIB_bedtools on scratch and copy the respective VCF files to it from $BI/gva_course/bedtools/:

...

Remember the above command is simply 1 possible solution there are multiple ways you could have done this, most commonly recursively copying the entire directly, or copying all the files rather than just the vcd files. 

Common among all (intersect)

One of if not the most useful links that will be provided in this course is this link to the bedtools intersect page. Not only does it give a very nice graphical interface of what intersect is doing right at the top, but it provides amazing explanations for how to build a good analysis command. If the graphical image is say not what your ideal situation is (i.e. you are interested in seeing where 100% of the samples overlap, not where 1 sample over laps with any of the other samples) there is a command for that as well. Using the link provided, and the bedtools intersect -h command, see if you can identify the regions that were found by all 3 mappers.

...

Expand
titleThe above is 1 possible answer. Did you come up with something different? here is what i thought was important to go into the command line

the use of -f 1.0 ensures that only variants in 100% of all the samples are listed

the -header to keep the header information moving forward.

Differential comparison (subtract)

Imagine a data set comprised of "disease state" and "normal state" individuals where the disease state is caused by any of some number of dominant (yet rare) mutations. Knowing that mutation is dominant, means that variants detected in the "normal state" are not of interest. How would one remove the variants identified in the "normal state" from the "diseased state"? How about bedtools subtract? Use the bedtools subtract -h command to see if you can put together a command.

...

Code Block
languagebash
titleNext let's pretend that the bowtie.vcf file represents the intersection of the normal group
collapsetrue
bedtools subtract -a diseased.vcf -b bowtie.vcf -f 1.0 -header > causitive.vcf

 

Going further

Hopefully you see how these tools can be useful (if not ask). These are but 2 of the bedtools commands, take a few minutes to look through the other bedtools subcommands that are available using bedtools -h, and  bedtools other_command -h

...