...
| Code Block |
|---|
| language | bash |
|---|
| title | Click here for help with copying the files using a wildcard after making a new directory |
|---|
| collapse | true |
|---|
|
mkdir $SCRATCH/GVA_multiqc
cp $BI/gva_course/plasmid+qc/* $SCRATCH/GVA_multiqc
cd $SCRATCH/GVA_multiqc |
| Code Block |
|---|
| language | bash |
|---|
| title | You may remember from the first tutorial on read preprocessing that fastqc is a module you can load |
|---|
| collapse | true |
|---|
|
module load fastqc |
...
We are going to construct a single commands file with 544 lines that will we can give to the launcher_creator.py script to launch all commands without having to know the name of any single file. To do so we will use the bash 'for' command.
...
- A list of something to deal with 1 at a time. Followed by a ';'
- for f in *.gz; in the following example
- Something to do with each item in the list. this must start with the word 'do'
- do echo "fastqc -o fastqc_output $f & "; in the following example
- The word "done" so bash knows to stop looking for more commands.
- done in the following example, but we add a final redirect (>) so rather than printing to the screen the output goes to a file (fastqc_commands in this case)
...
| Code Block |
|---|
| language | bash |
|---|
| title | Putting it all together |
|---|
| collapse | true |
|---|
|
for f in *.gz; do echo "fastqc -o fastqc_output $f &" ;done > fastqc_commands |
...
| Code Block |
|---|
| language | bash |
|---|
| title | Do the analysissubmit the job to run on the que |
|---|
|
mkdir fastqc_output
chmod + |
...
launcher_creator.py -N 5 -n "fastqc" -t 00:30:00 -a "UT-2015-05-18" -j fastqc_commands
sbatch fastqc.slurm |
Run MultiQC tool on all fastQC output
Unfortunately, multiqc is not available as a module on lonestar, and it actually ends up being more complicated to access in the BioITeam than it is to install it on your own. Finally, MultiQC provides us with a great opportunity to install a program for ourselves using pip (something that is fairly common among published scripts) and deal with the issue that pip by default tries to install things with administrator privileges which is fine for your personal laptop, but you don't have admin rights on TACC.
From the multiqc web page https://multiqc.info/ we see that the 'quick install' command is listed as pip install multiqc. The trick to getting pip installation commands to work on lonestar is to add --user after install, but before multiqc.
| Code Block |
|---|
| language | bash |
|---|
| title | pip installation of multiqc |
|---|
|
cd $HOME
pip install --user multiqc |
You should see multiple different dependancies installing one after another and when it is done you should have it installed in a potentially a new folder .local/bin/ run the following command to verify the installation worked as expected. If you see a help file that is similar to others you have seen in class it worked, if you get an error message let me know.
| Code Block |
|---|
| language | bash |
|---|
| title | Testing installation |
|---|
|
~/.local/bin/multiqc -h |
Hopefully by now the job we submitted with all the fastqc commands has finished. Use the showq -u command to check. Assuming it is, run the following command (on the headnode is fine).
| Code Block |
|---|
| language | bash |
|---|
| title | Testing installation |
|---|
|
cd $SCRATCH/GVA_multiqc
~/.local/bin/multiqc . |
Evaluate MultiQC report
As the multiqc_report.html file is a html file, you will need to transfer it back to your laptop to view it. If you do not know how to do this by this point in the course get my attention and we will go through it together as it is a skill you will vitally need going forward.
Optional Exercise
Using information gained in the MultiQC report, modify the bash loop used for the fastQC commands to improve the raw reads.
...