fastp - GVA2022
- 1 Overview
- 2 Learning objectives:
- 3 Installing fastp
- 4 Trimming adapter sequences
- 4.1 Example generic command
- 4.2 Trimming a single sample
- 4.3 Trim all the samples from the multiqc tutorial
- 4.3.1 Get some data
- 4.4 Trim the fastq files
- 4.4.2 Submitting as a job
- 4.4.2.1 Modify your slurm file
- 4.4.2.2 submit the job to run on the que
- 4.4.3 Running on idev
- 4.4.4 Comparing different run options
- 4.5 Evaluating the output
- 5 Optional next steps:
Overview
As mentioned in the introduction tutorial as well as the read processing tutorial, read processing can make a huge impact on downstream work. While cutadapt which was introduced in the read processing tutorial is great for quick evaluation or dealing with a single bad sample, it is not as robust as some other trimmers in particular when it comes to removing sequence that you know shouldn't be present but may exist in odd orientations (such as adapter sequences from the library preparation). This tutorial is adapted from the 2021 trimmomatic tutorial which sought to do the same basic things as fastp: get rid of adapter sequences first and foremost, ideally even before fastQC so you can make any quality or length based improvements on actual data not artifacts. The #1 biggest reason why fastp is now the instructor's preferred trimming program is this box taken from the trimmomatic tutorial:
A note on the adapter file used here
The adapter file listed here is likely the correct one to use for standard library preps that have been generated in the last few years, but may not be appropriate for all library preps (such as single end sequencing adapters, nextera based preps, and certainly not appropriate for PacBio generated data). Look to both the trimmomatic documentation and your experimental procedures at the bench to figure out if the adapter file is sufficient or if you need to create your own.
The more collaborative your work is, the less confidence you will have in picking the correct adapter file with trimmoatic, and while thanks to conda installations it can be pretty easy to test multiple different ones, fastp does all the guess work for you, and can generate some interesting graphs itself.
Learning objectives:
Install fastp
Remove adapter sequences from some plasmids and evaluate effect on read quality, or assembly.
Installing fastp
fastp's home page can be found on github and has links to the paper discussing the program, installation instructions for conda, and information on each of the different options available to the program. This is far above the quality the average programs will have as most will not have a user manual (or not nearly so detailed), may not have been updated since originally published (or may not have been published), etc. It having been updated since the publication is one thing that makes it such a good tool as the more who use it the more likely problems are found, and having a group who is going to actively improve the program will significantly increase its longevity.
Example command for creating a new environment
conda create -n GVA-ReadPreProcessing -c bioconda -c conda-forge fastp fastqc multiqc
Trimming adapter sequences
Example generic command
Example command for trimming illumina paired end adapters
fastp -i <READ1> -I <READ2> -o <TRIM1> -O <TRIM2> --threads # --detect_adapter_for_pe -j <LOG.json> -h <LOG.html>Breaking down the parts of the above command:
Part | Purpose | replace with/note |
|---|---|---|
fastp | tell the computer you are using the fastp prgram |
|
-i <READ1> | fastq file read1 you are trying to trim | actual name of fastq file |
-I <READ2> | fastq file read2 you are trying to trim | actual name of paired fastq file |
-o <TRIM1> | output file of trimmed fastq file of read 1 | desired name of trimmed fastq file |
-O <TRIM2> | output file of trimmed fastq file of read 2 | desired name of paired trimmed fastq file |
--threads # | use more processors, make command run faster | number of additional processors (68 max on stampede2) |
--detect_adapter_for_pe | automatically detect adapter sequence based on paired end reads, and remove them |
|
| json file with information about how the trim was accomplished. can be helpful for looking at multiple samples similar to multiqc analysis | name of json file you want to use |
| html file with infomration similar to the json file, but with graphs | name of html file you want to use |
All of the above has been put together from the help fastp --help command.
Trimming a single sample
Get some data
set up directories and copy files
mkdir -p $SCRATCH/GVA_fastp_1sample/Trim_Reads $SCRATCH/GVA_fastp_1sample/Raw_Reads
cd $SCRATCH/GVA_fastp_1sample
cp $BI/gva_course/plasmid_qc/E1-7* Raw_ReadsThe ls command should show you 2 gzipped fastq files. You may notice that here that we used a wildcard in the middle of our copy path for the first time. This is done so that you can grab both R1 and R2 easily without having to type out the full command. Double tab will help tell you when you have a sufficiently specific base name to only get the files you are after.
Trim the fastq files
The following command can be run on the head node. Like with FastQC if we are dealing with less than say 1-2Million reads, it is reasonable to run the command on the head node unless we have 100s of samples in which case submitting to the queue will be faster as the files can be trimmed all at once rather than 1 at a time. Use what you have learned in the class to determine if you think this command should be run on the head node. (this was covered in more detail in the first part of the evaluating and processing read quality tutorial.)
Example command for trimming illumina paired end adapters
fastp -i Raw_Reads/E1-7_S187_L001_R1_001.fastq.gz -I Raw_Reads/E1-7_S187_L001_R2_001.fastq.gz -o Trim_Reads/E1-7_S187_L001_R1_001.trim.fastq.gz -O Trim_Reads/E1-7_S187_L001_R2_001.trim.fastq.gz -w 4 --detect_adapter_for_pe Evaluating the output
Using everything you have learned so far in the class, can you answer the following questions?