Versions Compared

Old Version 105

changes.mady.by.user Anna M Battenhouse

Saved on

compared with

New Version 106

changes.mady.by.user Anna M Battenhouse

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

Tip
titleReservations

Use our summer school reservation (CoreNGS-Tue) when submitting batch jobs to get higher priority on the ls6 normal queue today:

sbatch --reservation=CoreNGSday2 CoreNGS-Tue <batch_file>.slurm
idev -m 180 -N 1 -A OTH21164 -r CoreNGSday2CoreNGS-Tue

Note that the reservation name (CoreNGS-Tue) is different from the TACC allocation/project for this class, which is OTH21164.

...

Code Block
languagebash
Command 1 of 16 (4 per node) ran on node c302c303-005.ls6.tacc.utexas.edu core 0
Command 10 of 16 (4 per node) ran on node c305c304-005.ls6.tacc.utexas.edu core 9
Command 11 of 16 (4 per node) ran on node c305c304-005.ls6.tacc.utexas.edu core 10
Command 12 of 16 (4 per node) ran on node c305c304-005.ls6.tacc.utexas.edu core 11
Command 13 of 16 (4 per node) ran on node c305c304-006.ls6.tacc.utexas.edu core 12
Command 14 of 16 (4 per node) ran on node c305c304-006.ls6.tacc.utexas.edu core 13
Command 15 of 16 (4 per node) ran on node c305c304-006.ls6.tacc.utexas.edu core 14
Command 16 of 16 (4 per node) ran on node c305c304-006.ls6.tacc.utexas.edu core 15
Command 2 of 16 (4 per node) ran on node c302c303-005.ls6.tacc.utexas.edu core 1
Command 3 of 16 (4 per node) ran on node c302c303-005.ls6.tacc.utexas.edu core 2
Command 4 of 16 (4 per node) ran on node c302c303-005.ls6.tacc.utexas.edu core 3
Command 5 of 16 (4 per node) ran on node c302c303-006.ls6.tacc.utexas.edu core 4
Command 6 of 16 (4 per node) ran on node c302c303-006.ls6.tacc.utexas.edu core 5
Command 7 of 16 (4 per node) ran on node c302c303-006.ls6.tacc.utexas.edu core 6
Command 8 of 16 (4 per node) ran on node c302c303-006.ls6.tacc.utexas.edu core 7
Command 9 of 16 (4 per node) ran on node c305c304-005.ls6.tacc.utexas.edu core 8

...

We've already touched on the need to redirect standard output and standard error for each task. Just remember that funny redirection syntax:

Code Block
languagebash
titleRedirect both standard output and standard error to a file
my_program input_file1 output_file1 > file1.log 2>&1

...

For example, you might have a script called align_bwa.sh (a bash script) or align_bowtie2.py (written in python Python) that performs multiple steps needed during the alignment process:

...

The BioITeam maintains a set of such scripts in the /work/projects/BioITeam/common/script directory. Take a look at some of them after you feel more comfortable with initial NGS processing steps. They can be executed by anyone with a TACC account.

...

Code Block
languagebash
titleRelative path exercise
# navigate through the symbolic link in your Home directory
cd ~scratch/core_ngs/slurm/simple 
ls ../wayness
ls ../..
ls -l ~/.bashrc

And read more about Absolute and relative pathname syntax

Interactive sessions (idev)

...

Code Block
languagebash
titleStart an idev session
idev -m 3060 -N 1 -A OTH21164 -p normal -r CoreNGSday2CoreNGS-Tue

Notes:

  • -p normal requests nodes on the normal queue
    • this is the default for our reservation, while the development queue is the normal default
  • -m 60 asks for a 60 minute session
  • -A OTH21164 specifies the TACC allocation/project to use
  • -N 1 asks for 1 node
  • --reservation=CoreNGSday2CoreNGS-Tue gives us priority access to TACC nodes for the class. You normally won't use this argumentoption.

When you ask for an idev session, you'll see output as shown below. Note that the process may repeat the "job status:  PD" (pending) step while it waits for an available node.

...