What and why is Shell Scripting?
A shell is a program that takes your commands from the keyboard and gives them to the operating system. Most Linux systems utilize Bourne Again SHell (bash), but there are several additional shell programs on a typical Linux system such as ksh, tcsh, and zsh. The simplest way to check which shell your machine has is to type any random letters and hit enter. For example, Lonestar in TACC uses bash.
login1$ ffkakldk -bash: ffkakldk: command not found
Let’s assume that you go to a restaurant. The rule of the restaurant is to order one by one: order a drink and get it, then order an appetizer and get it, then order dishes and so forth. What a stupid ordering system is! To make matters worse, even if you want to order the exact same meal you ate before, you must go through the tedious process again. Shell scripting addresses this problem. A shell script is series of commands written in a plain text file. Instead of entering commands one by one, you can store the sequence of commands to text file and tell the shell to execute this text file. When you want to repeatedly execute the series of command lines for multiple datasets, the shell script can automate your task and save lots of time.
Exercise 1 - Hello world
Below is a simple shell script that takes one argument (the text to print after "Hello") and echos it.
- The first line tells the shell which program to use to execute this file (here, the bash program).
- The 2nd line sets the shell variable TEXT to the first command line argument.
- The 3rd line defaults the value of TEXT to the string "Shell World" if no command line argument is provided.
- Remaining lines echo some text, substituting the value passed in on the command line.
#!/bin/bash
TEXT=$1
: ${TEXT:="Shell World"}
echo "-------------------"
echo "Hello, $TEXT!"
echo "-------------------"
Open your favorite text editor, enter these lines, and save as hello.sh (note the file extension for shell scripts is .sh). Then open a Terminal window and change into the directory where the script was saved. For example:
cd /Desktop
The script can be run, with or without command line arguments, by explicitly invoking bash as follows:
user$ bash hello.sh ------------------- Hello, Shell World! ------------------- user$ bash hello.sh Goddess ------------------- Hello, Goddess! -------------------
There is a shortcut, though. Since we have the line at the top of this file that names the program that should run it, we should be able to execute the script just by typing in its pathname like this (where ./ means current directory):
user$ ./hello.sh -bash: ./hello.sh: Permission denied
But there''s a complication. Welcome to the world of Unix permissions! The script file must be marked executable for this to work. To see what the current permissions are:
user$ ls -la hello.sh -rw-rw-r-- 1 user group 122 May 18 01:16 hello.sh
This says that anyone can read the file, the owner (you) or anyone in your group can modify it (write permission), but no one can execute it. We use the chmod program to allow anyone to execute the script:
username$ chmod +r hello.sh username$ ls -la hello.sh -rwxrwxr-x 1 user group 122 May 18 01:16 hello.sh
Now hello.sh can be invoked directly:
username$ ./hello.sh "Expert scripter" ------------------- Hello, Expert scripter! -------------------
Note that when we supplied the text "Expert scripter", we put it in quotes, which group the two words into one argument to the script. Without the quotes, the word "Expert" would be seen by the script as argument 1 and "scripter" would be seen as argument 2 (which our script ignores).
BWA alignment script
The first real script you will likely find yourself wanting is one that performs a standard set of alignment tasks such as mapping, bam file creation and statistics reporting. The script we want for the bwa aligner would do the following:
- Aligns a fastq file to a pre-made reference genome
- Extracts alignments from bwa's proprietary binary .sai file to a .sam file
- Converts the .sam file into a .bam file using samtools
- Sort and index the .bam file so that it can be viewed in IGV.
- Count the number of aligned and unaligned reads, and calculate the mapping rate.
