...
This next exercise will give you some idea of how Annovar works; we've taken the liberty of writing the bash script annovar_pipe.sh around the existing summarize_annovar.pl wrapper (a wrapper within a wrapper - a common trick) to even further simplify the process for this course.
Exercise:
First we want to move to a new location on $SCRATCH
| Code Block |
|---|
| language | bash |
|---|
| title | Solution to make a new directory on scratch. You should know this answer. |
|---|
| collapse | true |
|---|
|
cds
mkdir BDIB_Annovar
cd BDIB_Annovar |
Next, look at the code for our annovar_pipe.sh command.
| Expand |
|---|
| title | Click here for a hint |
|---|
|
The which command is used to give the location of a program or script that is in your $PATH. | Code Block |
|---|
| language | bash |
|---|
| title | Here is an easy one-liner to cat the contents of a script (note ` is a back-tick, not apostrophe) |
|---|
| collapse | true |
|---|
| cat `which annovar_pipe.sh` # the command within the `` is evaluated first and the output placed within the `` marks to be evaluated by the outer command
|
This script simply does a format conversion and then calls summarize_annovar.pl. Which begs the question what does summarize_annovar.pl do? | Expand |
|---|
| title | Figure out what summarize_annovar.pl is doing using the back tick trick you just learned |
|---|
| Again note the ` characters are "backtick", not apostrophes | Code Block |
|---|
| title | Print out the text of the perl script summarize_annovar.pl |
|---|
| more `which summarize_annovar.pl`
|
The more command was used to make it slightly easier to read and to remind you that there are multiple options of how to look a things (less, head, tail, etc), eventually you will develop your own habits but the more you are exposed to the fewer instances of angry eureka moments you will have. When working with unknown files or file types, cat should typically be avoided unless you run a wc -l command else you risk crashing your terminal window. |
This script is written in perl (a programing language that while powerful, is not very favored among most computational biologists who typically prefer: Python, R, or Bash shell scripts) as a method of standardizing input to call a series of external commands. Such wrappers (or wrappers within wrappers) make your life much easier. breseq itself has wrappers for using bowtie2 and samtools which you ran separately on other data. As you increase your understanding of scripts and the command line by looking at what others have done you may begin to make your own wrappers and small scripts, but such wrappers are a great example of the BioITeam and other community resources can provide you with. While we have told you no one will ever care as much about your data as you do, quality of life issues regarding repetitive treatment of similar inputs and outputs may be easily solved by someone else. |
Now let's run it on the .vcf files from the 3 individuals (NA12878, NA12891, and NA12892) from both the samtools and gatk output in the $BI/ngs_course/human_variation/ directory. (You may recognize these as the same individuals that we worked with on the Trios tutorial. Throughout the class we've been teaching you how to create a commands file using nano, but here we provide a more complex example of how you can generate a commands file. As you become more proficient with the command line, it is likely you will use various piping techniques to generate commands file. The following calls Perl to custom-create the 6 command lines needed and put them straight into a commands file:
| Code Blockwarning |
|---|
| title | Create the "Do not run this command yet |
|---|
|
| Code Block |
|---|
| title | Create the "commands" file to run annovar on six vcf files - use Perl to extract the sample name and mapper so the log files have meaningful, but shorter, names |
|---|
| ls $BI/ngs_course/human_variation/N*.vcf | \
perl -n -e 'chomp; $_=~/(NA\d+).*(sam|GATK)/; print "annovar_pipe.sh $_ >$1.$2.log 2>&1\n";' > commands
|
|
| Code Block |
|---|
| language | bash |
|---|
| title | Click here to check your work |
|---|
| collapse | true |
|---|
|
cds
mkdir BDIB_Annovar
cd BDIB_Annovar
# Use only 1 of the following copy commands
# if you have already done the trios tutorial
cp $SCRATCH/BDIB_Human_tutorial/raw_files/N*.vcf .
# if you have not done the trios tutorial
cp $BI/ngs_course/human_variation/N*.vcf .
ls *.vcf | perl -n -e 'chomp; $_=~/(NA\d+).*(sam|GATK)/; print "annovar_pipe.sh $_ >$1.$2.log 2>&1 &\n";' > commands
|
| Note |
|---|
| title | A helpful note to make your life better. |
|---|
|
| "commands" files are 1 way to both create a record of what it is that you are doing as well as an easy way to execute multiple commands simultaneously. In a previous tutorial (the advanced breseq tutorial, or the 2nd breseq example from the first day) we showed you how to do this by adding an & as the last character of the line to force the command to execute in the background. Tomorrow we will go over the more common method of submitting "commands" files to the que to run as jobs rather than being spoiled with interacting with everything on idev nodes. In computational biology this instructor has often found that once I learn to do something a particular way, it takes an incredible amount of inertia to change how I do something even when i know it would help to do so. In an effort to learn from me making things harder on myself than need be, I urge you to STRONGLY consider not naming every list of commands you want to run "commands" but rather something that is actually descriptive, even "commands_DATE" would be helpful. As you start submitting your own jobs you will quickly be able to build bad habits, try to be aware of and avoid this one if possible. |
...
| Warning |
|---|
| title | Make sure you are on an IDEV node using the showq -u command |
|---|
|
| Code Block |
|---|
| language | bash |
|---|
| title | How to request an idev node if needed |
|---|
| collapse | true |
|---|
| bash |
|---|
| title | How to request an idev node if needed |
|---|
| collapse | true |
|---|
| # if doing this tutorial on Wednesday:
idev -m 180 -r CCBB_5.24.17PM -A UT-2015-05-18
# if doing this tutorial on Thursday:
idev -m 180 -r CCBB_5.25.17PMidev -m 120 -r CCBB_Bio_Summer_School_2016_Day3 -A UT-2015-05-18 -N 2 -n 818 |
| Code Block |
|---|
| title | Create the submission script and submit |
|---|
| chmod +x commands
./commands |
|
...
Variants consist of single base base changes, insertions and deletions, and larger scale structural changes. "Larger scale" is usually defined relative to the capabilities of the technology; for example, a "small indel" usually means "detectable within a single sequence read". In 2009, sequence reads were about 50 bp but in , by 2011 they were 100 bp, and today the standard hiSEQ run is 150bp.
Return to GVA2017