A healthy taste of resources available, specifically for this course - not a comprehensive catalog.
Linux
- Linux fundamentals on this wiki
Online tutorials:
- Ryan's Linux Tutorial: http://ryanstutorials.net/linuxtutorial/
- Unix and Perl for Biologists: http://korflab.ucdavis.edu/unix_and_Perl/
Community Resources
- SEQAnwers forum - many NGS sequencing questions answered here
- A funny SEQAnwers post about biologists starting to analyze NGS data: http://seqanswers.com/forums/showthread.php?t=4589
- UCSC Genome Browser - visualize and download NGS data (see more below)
- Galaxy website for online sequencing data analysis
- Broad Institute Integrated Genomcs Viewer (IGV) - especially good for bam files (requires login after quick registration)
- 2012 Next-Gen Sequence Analysis Workshop (Michigan State University) has similar tutorials to our course
- also includes introductions to using the Amazon EC2 where you can "rent" Linux machines (useful if you don't have access to TACC)
- Python, R, ChIP-Seq, etc. maintained mRNA-seq protocols from this workshop here
Sequencing Technologies
- Paper comparing NGS technologies (Liu et al., 2012)
Technology videos
Fastq analysis/manipulation
- Wikipedia FASTQ format page
- Illumina library construction on GSAF user wiki - useful for contaminant detection or adapter removal
- FastQC from Babraham Bioinformatics – http://www.bioinformatics.babraham.ac.uk/projects/fastqc/
- produces nice quality report for fastq files
- cutadapt – https://code.google.com/p/cutadapt/
- An excellent command line tool for adapter sequence removal
- Available at TACC only at Anna's /work/01063/abattenh/local/bin/cutadapt
- documentation at https://cutadapt.readthedocs.org/en/stable/
- Good support for trimming paired-end datasets
- Anna has an script that handles the details of paired-end read trimming:
- /work/01063/abattenh/code/script/align/trim_adapters.sh
- trimmomatic – http://www.usadellab.org/cms/?page=trimmomatic
- Supports trimming paired-end datasets. I haven't used it but it seems to be popular.
- fastx toolkit – http://hannonlab.cshl.edu/fastx_toolkit/
- Command line tools for fastq analysis and manipulation
- Good for hard clipping. Available at TACC.
- Documentation at: http://hannonlab.cshl.edu/fastx_toolkit/commandline.html
Alignment and aligners
- Comparison of different aligners
- by Heng Li, developer of BWA and MAQ
- by Nils Homer, developer of BFAST
- File formats
- Aligners
- bwa (Burrows-Wheeler Aligner) by Heng Li – http://bio-bwa.sourceforge.net/
- fast, sensitive and easy to use
- bowtie2 – http://bowtie-bio.sourceforge.net/bowtie2/manual.shtml
- fast, sensitive and extremely configurable
- bwa (Burrows-Wheeler Aligner) by Heng Li – http://bio-bwa.sourceforge.net/
- Anna has some TACC-aware alignment scripts you might find useful
- bwa alignment
- /work/01063/abattenh/code/script/align/align_bwa_illumina.sh
- bowtie2 alignment
- /work/01063/abattenh/code/script/align/align_bowtie2_illumina.sh
- merging sorted BAM files (read-group aware)
- /work/01063/abattenh/code/script/align/merge_sorted_bams.sh
- email or come talk to me if you have questions or problems
- bwa alignment
Reference genomes
- Gencode – https://www.gencodegenes.org/
- reference genomes, transcriptomes and high-quality annotations for human and mouse
- https://www.gencodegenes.org/releases/current.html
- UCSC downloads – http://hgdownload.cse.ucsc.edu/downloads.html
- reference genomes, transcriptomes and high-quality annotations for many eukaryotes
- Ensembl downloads – ftp://ftp.ensembl.org/pub/
- reference genomes, transcriptomes and high-quality annotations for many eukaryotes
- NCBI GenBank Nucleotide collection – https://www.ncbi.nlm.nih.gov/nuccore/
- for prokaryotic genomes
- Reference genome vocabulary – https://software.broadinstitute.org/gatk/documentation/article?id=7857
- excellent introduction to the types of genome references and the vocabulary used to describe them
- aimed at higher eukaryotes but vocabulary useful nonethele
- excellent introduction to the types of genome references and the vocabulary used to describe them
- GATK blog describing ALT contigs in GRCh38 – https://software.broadinstitute.org/gatk/blog?id=8180
- Support for mapping to ALT contigs containing variants
- bwa mem + bwakit by Heng-Li – https://github.com/lh3/bwa/blob/master/README-alt.md
Alignment analysis
- SAM (Sequence Alignment Map) format specification (pdf)
- Translate SAM file flags web calculator: http://broadinstitute.github.io/picard/explain-flags.html
- type in a decimal number to see which flags are set
- Translate SAM file flags web calculator: http://broadinstitute.github.io/picard/explain-flags.html
- samtools – http://samtools.sourceforge.net/ by Heng Li
- sam/bam conversion, flag filtering, sorting, indexing, duplicate filtering
- Picard toolkit – http://broadinstitute.github.io/picard/
- sam/bam utilities that are read-group aware
- especially MarkDuplicates for flagging duplicate alignments
- SAMStat - http://samstat.sourceforge.net/
- produces detailed graphical statistics for sam/bam files.
- bedtools – http://bedtools.readthedocs.org/en/latest/
- Swiss army knife for all manner of common bed, bam, vcf, gff file manipulation such as:
- intersecting bam or bed with annotation files
- bedtools intersect
- merging overlapping regions
- bedtools merge
- generation of per-base genome-wide signal in bedGraph format
- bedtools coverage
- bedtools multicov
- extracting fasta corresponding to regions
- bedtools getfasta
- intersecting bam or bed with annotation files
- Available in the TACC module system
- Swiss army knife for all manner of common bed, bam, vcf, gff file manipulation such as:
File formats and conversion
- SAM format specification – http://samtools.github.io/hts-specs/SAMv1.pdf
- crucial for performing format conversions, of which ChIP-seq analysis can have many
- Genome browser file formats – http://genome.ucsc.edu/FAQ/FAQformat.html
- BED, bedGraph, narrowPeak and many more
- SRA (Sequence Read Archive) from NCBI
- overview on this wiki
- SRA search home page
- SRA Toolkit
- UCSC file format conversion scripts - useful for getting to/from wig and bed to corresponding binary formats.
- Make sure you download the correct script for your operating system!
- A directory containing these tools can be found on stampede at /work/01063/abattenh/local/UCSC_utilities
- Mason program for simulating second-generation sequencing reads
UCSC Genome Browser
- Main UCSC Genome Browser web site
- File formats - BED format especially is widely used
- Table browser - Browse and download data in different formats
- ENCODE data downloads at UCSC - useful for getting data to work with
- Beta Test browser site - most up-to-date datasets and features; can be buggy
- Visualize mapped data at UCSC genome browser on this wiki
Transcriptome analysis
- The Tuxedo pipeline: RNAseq with tophat/cufflinks
- RNAseq analysis protocol article in Nature Protocols
- TopHat - http://ccb.jhu.edu/software/tophat/index.shtml
- exon-aware sequence alignment (uses bowtie2/bowtie )
- resource bundles for selected organisms (gff annotations, pre-built bowtie2 references, etc.)
- cuffquant, cuffnorm, cufflinks – http://cole-trapnell-lab.github.io/cufflinks/manual/
- transcript quantification, normalization, differential expression
Variant calling
- The 1000 Genomes project - catalog of human genetic variants
- Tools
- Broad institute GATK - complex but powerful; used by 1000 Genomes
- File formats
- VCF (Variant Call Format) v4.0 - developed by 1000 Genomes project
Genome Annotation
- GREAT: an analysis tool that takes bed files as input and outputs enriched genes, GO-terms, motifs, etc.
- for human, mouse, zebrafish
- MEME-suite: a motif identification and discovery tool. Works with most species.
- takes fasta files as input, so filter your bam/bed files to get the regions of interest, then convert over using bamtofastq in bedtools.
