# MutationSieve algorithm review

Review date: 2026-08-01. This is a source-design review, not a performance
comparison. The cited methods were examined for reusable principles; no
third-party caller source was copied or vendored.

## Decision rule

MutationSieve adopts an idea only when it can be implemented transparently,
tested deterministically, bounded in memory/time, and interpreted without an
unavailable assay-specific training set. Features that need learned weights,
physical validation material, or a substantially different assembly engine are
kept as explicit future work rather than approximated and overstated.

## 2026 comparator decision

There is no defensible single "best" caller for every assay, variant class,
depth, tumour purity, and sequencing platform. The current primary literature
supports a deliberately diverse comparison set:

- **Lancet2 is the priority indel comparator.** Its authors report the strongest
  indel performance among the compared tools on their four matched
  tumour/normal cell-line benchmark, using joint local assembly, partial-order
  multiple-sequence alignment, read realignment, and explainable learned
  scoring. That is a strong benchmark-specific result, not a universal ranking.
- **DeepSomatic is the priority learned comparator.** It supplies current
  technology-specific neural models and public somatic benchmark results, but
  its performance still depends on whether the available model matches the
  specimen, library, platform, and analysis mode.
- **Mutect2 and Strelka2 remain established assembly/model baselines**, while
  **LoFreq is retained as a quality-aware low-frequency baseline**. Together
  they exercise different error and haplotype assumptions, which is more
  informative than comparing only closely related callers.
- **Agreement is useful as triangulation, but never as ground truth.** A 2025
  whole-exome benchmark found that prespecified voting ensembles outperformed
  its best individual caller, with different ensembles for SNVs and indels.
  MutationSieve therefore keeps caller outputs separate and supports a
  prespecified cross-caller comparison; it does not turn a majority vote into
  a truth label or silently union calls.
- **The assay and decision task can reverse a ranking.** A 2025/2026 cfDNA
  benchmark found different leading methods for discovery versus fixed-
  precision genotyping and for intermediate-depth versus ultra-deep data.
  That result reinforces assay-specific calibration, explicit depth/VAF
  strata, matched-normal/PoN evidence, and no universal-best claim.

The executable external-validation protocol therefore accepts Mutect2,
Strelka2, LoFreq, DeepSomatic, and Lancet2 as operator-declared labels while
pinning the declared executable/artifact bytes and expected version output.
Those observations do not prove engine identity. The protocol does not treat
agreement as truth, and it does not report a ranking until the tools have
actually run against the same reference, callable/confident regions, sample
pairing, truth set, and prespecified evaluator policy under an externally
controlled lockbox.

## Adopted principles

| Source method | Useful principle | MutationSieve implementation | Important difference |
|---|---|---|---|
| GATK Mutect2 / HaplotypeCaller | Evaluate reads in local haplotype context instead of trusting a single pileup column. | CIGAR-led permissive discovery is followed by bounded Smith-Waterman scoring against normalized local REF and ALT haplotypes. | MutationSieve does **not** perform de-novo assembly-graph discovery or PairHMM genotyping. Unrepresented alleles can still be missed. |
| LoFreq | Treat observations as non-identical Bernoulli trials using their sequencing and mapping qualities, with a Poisson-binomial tail. | Every admitted REF, ALT, or OTHER primary unit contributes to a fixed Q0--Q93 effective-error histogram. The exact Poisson-binomial tail is used within declared work limits; a worst-quality binomial upper bound is used beyond them. | The base and mapping error probabilities are combined conservatively by a union bound. No uncalibrated one-third substitution assumption is made, and base qualities are never repurposed as indel-error probabilities. |
| Strelka2 | Model indel error empirically, use tiered haplotype evidence, and handle the matched normal as part of the somatic problem. | Indel/complex significance requires a compatible empirical panel entry; local haplotype evidence is retained; the matched normal is always opened, compatibility checked, and recounted at fixed candidate loci. | MutationSieve has no fitted indel mixture model or normal-contamination model. Missing calibration therefore caps results at exploratory review/no-call instead of substituting generic parameters. |
| Lancet / Lancet2 | Joint tumor-normal local sequence context and repeat-aware assembly are valuable for difficult indels. | Repeat-aware normalization, physical CIGAR-junction recount, matched-normal evidence, homopolymer/complexity annotations, and conservative ambiguous-read classification are implemented. | There is no colored de Bruijn graph, k-mer optimization, or graph-path enumeration in this release. |
| Octopus | Haplotype-aware models can unify SNVs, indels, complex replacements, and different experimental designs while keeping evidence inspectable. | Normalized SNV, MNV, INS, DEL, and bounded COMPLEX records share one evidence/output contract; exact evidence and limitations remain inspectable. | A transparent rule/rank system is used, not Octopus's Bayesian genotyping models. MutationSieve deliberately emits no calibrated genotype or posterior probability. |
| DeepVariant / DeepSomatic | Learned image/tensor classifiers can improve accuracy when trained and tested on representative technologies and specimen types. | No learned classifier is used. The release records structured evidence that could support a future, separately governed model study. | Importing a pretrained score would not establish validity for this panel, specimen preparation, VAF range, or platform. `train` therefore remains fail-closed. |
| fgbio molecular consensus | UMI evidence must be formed from real molecular families with explicit consensus likelihoods, then realigned before variant analysis. | Explicit configured UMI tags can define the primary unit; family completeness and consensus are audited separately from coordinate-family shadows. | MutationSieve does not reproduce fgbio's likelihood model, duplex protocol handling, or consensus-read realignment and must not treat coordinate families as molecules. |
| BAMSurgeon | Realistic spike-ins should preserve empirical alignment/error context and avoid incompatible haplotype edits. | The hand-constructed aligned-BAM regression fixture rejects duplicate/overlapping truth alleles, isolates random streams, protects encoded event bases, records ordinary versus forced control realization, and includes a structurally independent SpikeForge edit-output control. | The fixture is not generated from FASTQ by a pinned production aligner against the full intended reference. It is a deterministic software control, not synthetic-read validation or sensitivity evidence. |
| GA4GH benchmarking and hap.py/vcfeval | Complex variants must be compared by resulting haplotypes inside a precisely defined confident/callable region. | Internal software-control scoring preserves record-exact metrics and a separately labeled bounded MNV/component equivalence check. The public protocol requires a pinned hap.py/vcfeval-style engine as the primary real benchmark. | The built-in bounded equivalence helper is not a general graph comparator and is not presented as one. |

## Sources reviewed

- GATK local assembly and haplotype determination:
  https://gatk.broadinstitute.org/hc/en-us/articles/360036227612
- GATK Mutect2:
  https://gatk.broadinstitute.org/hc/en-us/articles/360036733771-Mutect2
- Strelka2, *Nature Methods* (2018):
  https://www.nature.com/articles/s41592-018-0051-x
- LoFreq, *Nucleic Acids Research* (2012):
  https://academic.oup.com/nar/article/40/22/11189/1152727
- Lancet, *Communications Biology* (2018):
  https://pmc.ncbi.nlm.nih.gov/articles/PMC6123722/
- Lancet2, joint multi-sample local assembly:
  https://pmc.ncbi.nlm.nih.gov/articles/PMC13064521/
- Octopus, *Nature Biotechnology* (2021):
  https://www.nature.com/articles/s41587-021-00861-3
- DeepVariant, *Nature Biotechnology* (2018):
  https://www.nature.com/articles/nbt.4235
- DeepSomatic final paper and public somatic benchmarks:
  https://www.nature.com/articles/s41587-025-02839-x
- DeepSomatic source code:
  https://github.com/google/deepsomatic
- Whole-exome individual-caller and voting-ensemble benchmark:
  https://pmc.ncbi.nlm.nih.gov/articles/PMC11790059/
- Patient-derived cfDNA caller benchmark across depth and use case:
  https://pmc.ncbi.nlm.nih.gov/articles/PMC12852940/
- fgbio molecular-consensus documentation:
  https://fulcrumgenomics.github.io/fgbio/tools/develop/CallMolecularConsensusReads.html
- BAMSurgeon, *Bioinformatics* (2015):
  https://pmc.ncbi.nlm.nih.gov/articles/PMC4856034/
- GA4GH small-variant benchmarking best practices:
  https://pmc.ncbi.nlm.nih.gov/articles/PMC6699627/
- hap.py reference implementation:
  https://github.com/Illumina/hap.py

## Consequence for this release

The review supports the current conservative architecture, but it does not
justify claiming state-of-the-art biological accuracy. The largest algorithmic
gap is discovery by local assembly graph: MutationSieve can normalize and
rescore an indel that CIGAR evidence exposes, including repeat-shifted events,
but cannot discover every hidden haplotype that Mutect2, Lancet, or Octopus may
assemble. The software-verification release must therefore keep automatic
`PASS`/`HIGH` disabled and publish failed hypotheses and no-call territory
alongside any successful software checks.
