How omics data are generated
Every file you analyse is the end of a physical process: a sample was collected, molecules were extracted, converted into something an instrument can read, and measured. Knowing that process tells you where the artefacts in your data come from and which parameters in your pipeline exist to correct them.
The common path
- Study designQuestion, groups, replicates, randomisation, metadata sheet
- Sample collectionBlood, tissue, cells, stool, swab; time to freezing; preservative (EDTA, PAXgene, RNAlater, FFPE)
- ExtractionIsolate DNA, RNA, protein or metabolites; measure quantity and integrity (Qubit, RIN/DV200, BCA)
- ConversionLibrary preparation for sequencing, digestion for proteomics, derivatisation for GC-MS
- MeasurementSequencer, mass spectrometer, array scanner, NMR, microscope
- Raw dataFASTQ / POD5 / BAM, RAW / mzML, IDAT, FID, images
- QC & depositRun metrics, sample QC, upload to GEO/SRA/PRIDE/MetaboLights
Many apparent biological signals come from handling. RNA degrades within minutes at room temperature; hypoxia during surgery changes gene expression; metabolites keep changing in unfrozen blood; formalin fixation damages DNA (C>T artefacts) and fragments RNA. Record collection time, processing delay, freeze–thaw cycles and storage duration for every sample, and keep them balanced across groups.
Sequencing libraries
A sequencing library is a collection of DNA fragments, each flanked by known adapter sequences that let the sequencer capture, amplify and read them. Whatever the starting molecule (genomic DNA, cDNA from RNA, bisulfite-converted DNA, immunoprecipitated chromatin, Tn5-tagged fragments), the result has the same structure:
- Fragmentation: sonication, enzymatic shearing or transposase (tagmentation) produces fragments of ~200–600 bp for short reads. Long-read libraries keep fragments of 10–100 kb or longer.
- End repair, A-tailing and adapter ligation attach adapters; tagmentation (Nextera) fragments and adds adapters in one step.
- PCR amplification adds index sequences and enriches adapter-ligated fragments. PCR creates duplicates and GC bias; PCR-free WGS avoids this when input DNA allows.
- Unique molecular identifiers (UMIs): random barcodes attached before amplification, so reads from the same original molecule can be collapsed. Essential for single-cell counting and ultra-low-frequency variant detection.
- Target enrichment (exomes, panels): hybridisation capture with biotinylated probes or amplicon PCR. Coverage then depends on probe efficiency, which is why exome depth is uneven.
- Library QC: fragment size (Bioanalyzer/TapeStation), concentration (qPCR) and adapter-dimer contamination.
Short-read sequencing by synthesis (Illumina)
Library fragments bind to a flow cell and are copied in place into clusters of ~1,000 identical molecules (bridge amplification or exclusion amplification on patterned flow cells). Then sequencing proceeds in cycles. In each cycle a polymerase adds one fluorescently labelled, reversibly terminated nucleotide to every cluster; a camera images the flow cell; the dye and terminator are cleaved; the next cycle begins. The colour of each cluster in each cycle is the base.
Errors grow along the read because clusters lose synchrony (phasing): some molecules fall a base behind or jump ahead, blurring the signal. That is why base quality drops toward the 3′ end of reads. The instrument converts intensities into base calls and Phred quality scores, stored in FASTQ files:
Long-read sequencing
PacBio HiFi (SMRT)
A single polymerase copies a circularised DNA molecule (SMRTbell) inside a tiny well (zero-mode waveguide) while fluorescent nucleotides are detected in real time. Because the molecule is circular, the polymerase reads it many times; a consensus of these passes gives 15–25 kb reads at ~99.9% accuracy. Polymerase kinetics (pauses) reveal methylation.
Oxford Nanopore
A motor protein ratchets a single DNA or RNA strand through a protein pore in a membrane. Ionic current through the pore depends on the ~5–9 bases inside it at each moment. A neural-network basecaller translates the current trace (the "squiggle") into bases, and modified bases (5mC, 6mA, m6A) change the current, so they are detected directly. Read length is limited only by the molecule.
Arrays
Microarrays measure many known targets at once by hybridising labelled sample to millions of immobilised probes. SNP genotyping arrays (e.g. Illumina Global Screening Array, UK Biobank Axiom) read the two alleles of ~0.6–2 million variants from two colour channels; the ratio gives the genotype (plotted as genotype clusters) and the B-allele frequency also reveals copy number. Methylation arrays (EPIC) read bisulfite-converted DNA at ~935k CpGs. Expression arrays are now largely replaced by RNA-seq, but decades of array data remain in GEO.
Mass spectrometry
Proteomics, metabolomics and lipidomics are mostly measured by mass spectrometry (MS). Molecules are separated by liquid or gas chromatography, ionised (electrospray for LC, electron ionisation for GC, MALDI for imaging), then the mass analyser separates ions by mass-to-charge ratio (m/z):
| Analyser | Principle | Resolution | Typical use |
|---|---|---|---|
| Quadrupole (Q, QqQ) | Oscillating electric fields let only one m/z pass at a time | Low | Targeted quantification (MRM), filtering |
| Time-of-flight (TOF) | Lighter ions fly faster down a flight tube | High, fast | Untargeted metabolomics, timsTOF proteomics with ion mobility |
| Orbitrap | Ions orbit an electrode; frequency of oscillation gives m/z via Fourier transform | Very high | Proteomics (Exploris, Astral), metabolomics |
| FT-ICR | Cyclotron frequency in a strong magnet | Highest | Complex mixtures, top-down |
| Ion mobility (IMS) | Separates ions by shape and size in a gas | Extra dimension | Isomer separation, collision cross section |
In tandem MS (MS/MS), a selected ion is fragmented by collision with gas and the fragments are measured. Fragment patterns identify peptide sequences and small-molecule structures. Raw files are vendor-specific (.raw, .d, .wiff) and are converted to open mzML for most open-source tools.
NMR, imaging and other readouts
- NMR spectroscopy measures how atomic nuclei (¹H, ¹³C) in a magnetic field absorb radio-frequency energy; chemical shifts identify functional groups. Quantitative and reproducible, used in large cohort metabolomics.
- Imaging-based spatial omics (MERFISH, Xenium, CosMx) hybridise fluorescent probes in repeated rounds; each transcript's colour sequence across rounds forms a barcode decoded to a gene. Protein imaging (CODEX, IMC) uses barcoded or metal-tagged antibodies.
- Flow and mass cytometry (CyTOF) measure ~20–50 proteins per cell in millions of cells.
- Affinity proteomics (Olink, SomaScan) converts protein binding into DNA barcodes or aptamer signals read by sequencing or arrays.
Quality metrics at the source
| Data type | Metric | What good looks like |
|---|---|---|
| Any sequencing | % bases ≥ Q30; adapter content; duplication | >80–90% Q30 on modern Illumina; low adapter dimers |
| WGS | Mean depth, % genome ≥ 20×, insert size, contamination (VerifyBamID) | ~30× germline; contamination <2% |
| RNA-seq | RIN/DV200, % rRNA, % uniquely mapped, 3′ bias | RIN ≥ 7 for poly(A); >70–80% unique mapping |
| scRNA-seq | Cells recovered, median genes/cell, % reads in cells, % mitochondrial | Knee plot with clear separation of cells from empty droplets |
| ATAC-seq | TSS enrichment, fragment-size periodicity, FRiP | TSS enrichment > ~6–7 (ENCODE); visible nucleosome ladder |
| Proteomics | IDs per run, missed cleavages, mass error, RT stability | Stable IDs in QC standard (e.g. HeLa digest) across runs |
| Metabolomics | Pooled-QC CV, internal standard RT/intensity drift, blanks | CV < 20–30% in pooled QCs |