Page tree

Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

Code Block
languagebash
titleStart an idev session
idev -p developmentnormal -m 120180 -A UT-2015-05-18 -N 1 -n 68 --reservation=BIO_DATA_week_1

Then stage the sample datasets and references we will use.

Code Block
languagebash
titleGet the alignment exercises files
mkdir -p $SCRATCH/core_ngs/references/fasta
mkdir -p $SCRATCH/core_ngs/alignment/fastq
cp $CORENGS/references/*.fa     $SCRATCH/core_ngs/references/fasta/
cp $CORENGS/alignment/*fastq.gz $SCRATCH/core_ngs/alignment/fastq/
cd $SCRATCH/core_ngs/alignment/fastq

...

Expand
titleMake sure you're in a idev session

Make sure you're in an idev session. If you're in an idev session, the hostname command will display a name like c455-021.stampede2.tacc.utexas.edu. But if you're on a login node the hostname will be something like login3.stampede2.tacc.utexas.edu.

If you're on a login node, start an idev session like this:

Code Block
languagebash
titleStart an idev session
idev -p developmentnormal -m 120 -A UT-2015-05-18 -N 1 -n 68 --reservation=BIO_DATA_week_1
Code Block
languagebash
module load biocontainers  # takes a while
module load bwa
bwa

...

First we need to make sure that we don't look at fields in the SAM header lines. We're going to end up with a series of pipe operations, and the best way to make sure you're on track is to enter them one at a time piping to head:

...

Expand
titleAnswer

The expression above returns 612,968. There were 1,184,360 records total, so the percentage is:

Code Block
languagebash
titleCalculate alignment rate
awk 'BEGIN{print 612968/1184360}'

or about 51%. Not great.

Note we perform this calculation in awk's BEGIN block, which is always executed, instead of the body block, which is only executed for lines of input. And here we call awk without piping it any input. See Linux fundamentals: cut,sort,uniq,grep,awk

Exercise: What might we try in order to improve the alignment rate?

...

Make sure you're in an idev session. If you're in an idev session, the hostname command will display a name like c455-021.stampede2.tacc.utexas.edu. But if you're on a login node the hostname will be something like login3.stampede2.tacc.utexas.edu.

If you're on a login node, start an idev session like this:

Expand
titleMake sure you're in a idev session
Code Block
language
Code Block
languagebash
titleStart an idev session
idev -p developmentnormal -m 120 -A UT-2015-05-18 -N 1 -n 68 --reservation=BIO_DATA_week_1
Code Block
languagebash
# If not already loaded
module load biocontainers  # takes a while

module load samtools
samtools

...

Expand
titleCatch up (if needed)
Code Block
languagebash
# Copy over the Yeast data if needed
mkdir -p $SCRATCH/core_ngs/alignment/fastq
cp $CORENGS/alignment/Sample_Yeast*.gz $SCRATCH/core_ngs/alignment/fastq/

# Copy a pre-built sacCer3 reference if you didn't build one already
mkdir -p $SCRATCH/core_ngs/references
rsync -avrP $CORENGS/references/ $SCRATCH/core_ngs/references/


Code Block
languagebash
titleRun multiple alignments using the TACC batch system
# Make sure you're not in an idev session by looking at the hostname
hostname
# If the hostname looks like "c455-004.stampede2.tacc.utexas.edu", exit the idev session

# Make a new alignment directory for running these scripts
mkdir -p $SCRATCH/core_ngs/alignment/bwa_script
cd $SCRATCH/core_ngs/alignment/bwa_script
ln -s -f ../fastq

# Copy the alignment commands file and submit the batch job
cp $CORENGS/tacc/aln_script.cmds .
launcher_creator.py -j aln_script.cmds -n aln_script -t 02:00:00 -w 4 -a UT-2015-05-18 -q normal
sbatch --reservation=BIO_DATA_week_1 aln_script.slurm 
showq -u

...