Core NGS Tools -- Resource lists

A healthy taste of resources available, specifically for this course - not a comprehensive catalog.

Community Resources

• SEQAnwers forum - many NGS sequencing questions answered here
  • A funny SEQAnwers post about biologists starting to analyze NGS data: http://seqanswers.com/forums/showthread.php?t=4589
• UCSC Genome Browser - visualize and download NGS data (see more below)
• Galaxy website for online sequencing data analysis
• Broad Institute Integrated Genomcs Viewer (IGV) - especially good for bam files
• 2012 Next-Gen Sequence Analysis Workshop (Michigan State University) has similar tutorials to our course
  • also includes introductions to using the Amazon EC2 where you can "rent" Linux machines (useful if you don't have access to TACC)
  • Python, R, ChIP-Seq, etc.maintained mRNA-seq protocols from this workshop here

Sequencing Technologies

• Paper comparing NGS technologies (Liu et al., 2012)
• Broad Center GA Boot Camp

• Technology videos

  • Illumina (Solexa) Genome Analyzer and HiSeq

  • Life Technologies SOLiD

  • Roche/454

  • Pacific Biosciences

Fastq analysis/manipulation

• Wikipedia FASTQ format page
• Illumina library construction on GSAF user wiki - useful for contaminant detection or adapter removal
• FastQC from Babraham Bioinformatics – http://www.bioinformatics.babraham.ac.uk/projects/fastqc/
  
  • produces nice quality report for fastq files
• cutadapt – https://code.google.com/p/cutadapt/
  • An excellent command line tool for adapter sequence removal
  • Good support for trimming paired-end datasets
  • Available at TACC only at Anna's /work/01063/abattenh/local/bin/cutadapt
  • documentation at https://cutadapt.readthedocs.org/en/stable/
• trimmomatic – http://www.usadellab.org/cms/?page=trimmomatic
  • Supports trimming paired-end datasets. I haven't used it but it seems to be popular.
• fastx toolkit – http://hannonlab.cshl.edu/fastx_toolkit/
  • Command line tools for fastq analysis and manipulation
  • Good for hard clipping. Available at TACC.
  • Documentation at: http://hannonlab.cshl.edu/fastx_toolkit/commandline.html

Alignment and aligners

• Comparison of different aligners
  • by Heng Li, developer of BWA and MAQ
  • by Nils Homer, developer of BFAST
• Aligners
  • bwa - fast, sensitive and easy to use
  • bowtie2 - fast, sensitive, configurable, easy to use
  • bowtie (http://bowtie-bio.sourceforge.net/index.shtml) - very fast, not very sensitive
  • BFAST wiki & manual - slow and relatively complicated, but tunable sensitivity
• File formats
  • fastq format
  • The SAM (Sequence Alignment Map) format specification (pdf)

Alignment analysis

• SAM (Sequence Alignment Map) format specification (pdf)
• sam/bam tools
  • samtools - sam/bam conversion, flag filtering, bam sort/index
  • Picard - sam/bam utilities that are read-group aware
  • Translate SAM file flags - type in a decimal number to see which flags are set
• SAMstat - produces detailed graphical statistics for sam/bam files.
• BEDTools - region overlap, merge, coverage & much more, w/bed, bam, vcf, gff support
  • BEDTools manual on readthedocs.org

Aligners and SAM/BAM manipulation

• bwa (Burrows-Wheeler Aligner) by Heng Li – http://bio-bwa.sourceforge.net/
• bowtie2 – http://bowtie-bio.sourceforge.net/bowtie2/manual.shtml
• samtools – http://samtools.sourceforge.net/
  • also by Heng Li
• Picard toolkit – http://broadinstitute.github.io/picard/
• bedtools – http://bedtools.readthedocs.org/en/latest/
  
  • Swiss army knife for all manner of common BED/BAM file manipulation such as:
    • region filtering
    • generation of per-base genome-wide signal (bedGraph)
    • intersecting peaks with annotation files
    • extracting fasta corresponding to regions
• Anna has some TACC-aware alignment scripts you might find useful
  • bwa alignment
    • /work/01063/abattenh/code/script/align/align_bwa_illumina.sh
  • bowtie2 alignment
    • /work/01063/abattenh/code/script/align/align_bowtie2_illumina.sh
  • merging sorted BAM files (read-group aware)
    
    • /work/01063/abattenh/code/script/align/merge_sorted_bams.sh
  • email or come talk to me if you have questions or problems

UCSC Genome Browser

• Main UCSC Genome Browser web site
• Beta Test browser site - most up-to-date datasets and features; can be buggy
• Visualize mapped data at UCSC genome browser on this wiki
• File formats - BED format especially is widely used
• Table browser - Browse and download data in different formats
• ENCODE data downloads at UCSC - useful for getting data to work with

File formats and conversion

• SAM format specification – http://samtools.github.io/hts-specs/SAMv1.pdf
  • crucial for performing format conversions, of which ChIP-seq analysis can have many
• Genome browser file formats – http://genome.ucsc.edu/FAQ/FAQformat.html
  
  • BED, bedGraph, narrowPeak and many more
• SRA (Sequence Read Archive) from NCBI
  • overview on this wiki
  • SRA search home page
  • SRA Toolkit
    • NCBI documentation
    • SRA toolkit downloads
• UCSC file format conversion scripts - useful for getting to/from wig and bed to corresponding binary formats.
  • Make sure you download the correct script for your operating system!
  • A directory containing these tools can be found on stampede at /work/01063/abattenh/local/UCSC_utilities
• Mason program for simulating second-generation sequencing reads

Transcriptome analysis

• The Tuxedo pipeline: RNAseq with tophat/cufflinks
  • RNAseq analysis protocol article in Nature Protocols
  • tophat - exon-aware sequence alignment (uses bowtie2/bowtie )
  • cufflinks - transcript assembly, differential expression & regulation
    • cufflinks resource bundles for selected organisms (gff annotations, pre-built bowtie2 references, etc.)

Variant calling

• The 1000 Genomes project - catalog of human genetic variants
• Tools
  • Broad institute GATK - complex but powerful; used by 1000 Genomes
• File formats
  • VCF (Variant Call Format) v4.0 - developed by 1000 Genomes project
    • list of tools and format specs