FASTQ, SAM, BAM, and CRAM: A Practical Field Guide to Sequencing File Formats

A major, recurring bottleneck in high-throughput genomics workflows is the management and integration of biological data files. It is common for a newly deployed pipeline to fail at a late stage because of mismatched chromosome naming conventions, unindexed coordinate indices, or unrecognized quality score encodings. This friction is not simply a minor inconvenience: it directly drives up storage costs, increases computation times, and risks silent data corruption if researchers do not precisely understand what metadata survives at each stage of a pipeline.

To build reproducible, efficient pipelines, researchers must move beyond treating these formats as generic black boxes. This article serves as a definitive, technically precise field guide to the four primary bioinformatics file formats used in modern genomics: FASTQ, SAM, BAM, and CRAM.

The Sequencing File Pipeline

Data formats evolve sequentially throughout a sequencing and alignment pipeline, transitioning from raw, unmapped reads to highly compressed, reference-aligned archives:

The Sequencing File Format Pipeline

1. FASTQ: The Raw Starting Point

The FASTQ format is the standard entry point for almost all downstream secondary analysis. It represents raw, unmapped biological sequence reads generated directly by a sequencing instrument.

What It Stores:

Every single read in a FASTQ file is represented by a strict, four-line record:

Line 1 (Identifier): Begins with a @ character, containing unique instrument and run metadata (such as flowcell coordinates and tile IDs).
Line 2 (Sequence): The raw nucleotide bases (A, C, G, T, N).
Line 3 (Separator): Begins with a + character, occasionally repeating the run identifier.
Line 4 (Quality Scores): The per-base Phred quality scores ($Q$), encoded as ASCII characters.

The Phred score represents the logarithmic probability of a base-calling error ($P_e$), calculated using the standard formula:

$$Q = -10 \log_{10}(P_e)$$

In modern Illumina and PacBio datasets, these are typically encoded using the Sanger/Phred+33 format, where the ASCII character is offset by 33, ensuring that high-quality scores map to highly readable, standard text characters.

2. SAM: The Transition to Alignment

Once raw reads are processed, the next phase in sequencing pipelines is alignment, where an algorithm (such as BWA-MEM or Bowtie2) maps each read to its corresponding coordinate on a reference genome. The SAM (Sequence Alignment/Map) format is the standard, human-readable text output of this alignment step.

What It Adds:

A SAM file consists of two primary parts: a header section (lines beginning with @, documenting reference chromosome names, lengths, and the specific alignment command used) and the alignment records. Every alignment record contains 11 mandatory, tab-separated fields alongside optional, custom tags:

QNAME: Query template name (the read identifier).
FLAG: A bitwise flag representing alignment characteristics (such as whether the read is paired, mapped in a proper pair, or reverse-complemented).
RNAME: Reference sequence name (the chromosome).
POS: 1-based leftmost mapping position.
MAPQ: Mapping quality (Phred-scaled probability that the alignment is incorrect).
CIGAR: Concise Idiosyncratic Gapped Alignment Report (detailing insertions, deletions, matches, and soft-clips, such as 100M or 50M2I48M).
RNEXT: Reference name of the mate read (for paired-end sequencing).
PNEXT: 1-based leftmost position of the mate.
TLEN: Observed template length.
SEQ: Segment sequence (the read bases).
QUAL: ASCII-encoded Phred quality scores.

The SAM format is highly detailed, but its human-readable text structure makes it exceptionally large and computationally slow to parse directly.

3. BAM: The Compressed, Binary Standard

The BAM format is simply the compressed, binary equivalent of a SAM file, designed to serve as the active working standard for almost all downstream genomic tools (such as GATK or Samtools).

How It Relates to SAM:

BGZF Compression: BAM files are compressed using Blocked GNU Zip Format (BGZF), a specialized variant of gzip. BGZF divides the file into self-contained, 64KB blocks, allowing index utilities to quickly decompress and access specific genomic coordinates without reading the entire multi-gigabyte file from the beginning.
Lossless Conversion: Converting a SAM file to a BAM file is completely lossless. No biological sequence data, alignment flags, or quality scores are lost during this conversion.
Coordinate Indexing: By sorting a BAM file by chromosome coordinates and generating a companion index file (such as a .bai file using samtools index), downstream tools can query specific genomic regions in milliseconds, making it the practical standard for active variant calling.

4. CRAM: Reference-Based Archiving

As sequencing scale has escalated, even compressed BAM files present a major data storage bottleneck. To address this, the CRAM format was introduced by Fritz et al. (2011) at the European Bioinformatics Institute (EBI) as a reference-based compression alternative. Today, the CRAM format specification is maintained alongside the SAM/BAM specification in the same samtools/hts-specs repository. While the Global Alliance for Genomics and Health (GA4GH) promotes its use and works on genomic standards broadly, it is not the original author of the format.

Reference-Based Compression:

Unlike BAM, which stores the complete nucleotide sequence for every single read, CRAM achieves massive storage savings by saving only the differences between a read and a designated reference genome:

The Mechanism: If a read matches the reference genome perfectly, the CRAM record only stores the reference coordinate and length, completely omitting the redundant nucleotide bases. If a mismatch exists (such as a single nucleotide variant), the CRAM file records only the specific base change at that coordinate.
The Tradeoffs: This reference-dependent design introduces a critical computational constraint. To read or process a CRAM file, you must have access to the exact, identical reference genome file used during the original compression step. If the reference fasta is missing or modified, the CRAM file becomes completely unparseable, creating a strict dependency that requires robust, version-controlled reference registries (such as the CRAM reference cache).

Comparing the Formats: What Survives?

The following comparison matrix summarizes which data survives as you transition through the pipeline and illustrates how to choose sequencing formats for different use cases:

Feature / Metric	FASTQ	SAM	BAM	CRAM
Data Representation	Raw reads + Quality scores.	Aligned reads + Metadata.	Aligned reads + Metadata.	Diff-aligned reads + Metadata.
File Structure	4-line text record.	Tab-separated text.	BGZF-compressed binary.	Reference-compressed binary.
Human-Readable?	Yes	Yes	No	No
Reference Dependent?	No	No	No	Yes (requires identical reference FASTA).
Typical Use Case	Raw sequencer output.	Pipeline intermediate.	Active variant calling and analysis.	Long-term archive and storage.

A Practical Decision Framework

To help researchers choose the appropriate format for their pipelines, we can establish a simple, three-point decision logic:

For Raw sequencers and Quality Control: Always start with FASTQ. This is the raw biological input that preserves the unmapped baseline of your sequence run.
For Active Analysis and Variant Calling: Always sort and index into BAM. Virtually all standard downstream annotation and variant calling tools (such as GATK HaplotypeCaller) expect indexed BAM files to perform coordinate-based queries.
For Long-Term Archiving and Storage: Once analysis is complete, convert your BAM files into CRAM using samtools view -C. This dramatically reduces your institutional storage costs while preserving full genomic fidelity, provided you maintain a secure, version-controlled reference genome cache.

References and Authoritative Specifications

For researchers seeking to inspect the low-level mathematical structures, field flags, or compression specifications of these formats, the following peer-reviewed publications and repository specifications serve as primary, verified references:

The SAM/BAM/CRAM Specifications: Maintained by the HTS-specs committee and the Global Alliance for Genomics and Health (GA4GH). github.com/samtools/hts-specs
The Original SAM/BAM Publication: Heng Li et al. (2009). “The Sequence Alignment/Map format and SAMtools.” Bioinformatics, 25(16), 2078-2079. doi:10.1093/bioinformatics/btp352
The Original CRAM Publication: Fritz, M.H., Leinonen, R., Cochrane, G., & Birney, E. (2011). “Efficient storage of high throughput DNA sequencing data using reference-based compression.” Genome Research, 21(5), 734-740. doi:10.1101/gr.114819.110
Samtools and HTSlib Documentation: The primary, open-source C library for reading and writing high-throughput sequencing data formats. www.htslib.org
GATK Best Practices: Detailed documentation on file format requirements for standard variant discovery pipelines. software.broadinstitute.org/gatk