# MutationSieve algorithm review Review date: 2026-08-01. This is a source-design review, not a performance comparison. The cited methods were examined for reusable principles; no third-party caller source was copied or vendored. ## Decision rule MutationSieve adopts an idea only when it can be implemented transparently, tested deterministically, bounded in memory/time, and interpreted without an unavailable assay-specific training set. Features that need learned weights, physical validation material, or a substantially different assembly engine are kept as explicit future work rather than approximated and overstated. ## 2026 comparator decision There is no defensible single "best" caller for every assay, variant class, depth, tumour purity, and sequencing platform. The current primary literature supports a deliberately diverse comparison set: - **Lancet2 is the priority indel comparator.** Its authors report the strongest indel performance among the compared tools on their four matched tumour/normal cell-line benchmark, using joint local assembly, partial-order multiple-sequence alignment, read realignment, and explainable learned scoring. That is a strong benchmark-specific result, not a universal ranking. - **DeepSomatic is the priority learned comparator.** It supplies current technology-specific neural models and public somatic benchmark results, but its performance still depends on whether the available model matches the specimen, library, platform, and analysis mode. - **Mutect2 and Strelka2 remain established assembly/model baselines**, while **LoFreq is retained as a quality-aware low-frequency baseline**. Together they exercise different error and haplotype assumptions, which is more informative than comparing only closely related callers. - **Agreement is useful as triangulation, but never as ground truth.** A 2025 whole-exome benchmark found that prespecified voting ensembles outperformed its best individual caller, with different ensembles for SNVs and indels. MutationSieve therefore keeps caller outputs separate and supports a prespecified cross-caller comparison; it does not turn a majority vote into a truth label or silently union calls. - **The assay and decision task can reverse a ranking.** A 2025/2026 cfDNA benchmark found different leading methods for discovery versus fixed- precision genotyping and for intermediate-depth versus ultra-deep data. That result reinforces assay-specific calibration, explicit depth/VAF strata, matched-normal/PoN evidence, and no universal-best claim. The executable external-validation protocol therefore accepts Mutect2, Strelka2, LoFreq, DeepSomatic, and Lancet2 as operator-declared labels while pinning the declared executable/artifact bytes and expected version output. Those observations do not prove engine identity. The protocol does not treat agreement as truth, and it does not report a ranking until the tools have actually run against the same reference, callable/confident regions, sample pairing, truth set, and prespecified evaluator policy under an externally controlled lockbox. ## Adopted principles | Source method | Useful principle | MutationSieve implementation | Important difference | |---|---|---|---| | GATK Mutect2 / HaplotypeCaller | Evaluate reads in local haplotype context instead of trusting a single pileup column. | CIGAR-led permissive discovery is followed by bounded Smith-Waterman scoring against normalized local REF and ALT haplotypes. | MutationSieve does **not** perform de-novo assembly-graph discovery or PairHMM genotyping. Unrepresented alleles can still be missed. | | LoFreq | Treat observations as non-identical Bernoulli trials using their sequencing and mapping qualities, with a Poisson-binomial tail. | Every admitted REF, ALT, or OTHER primary unit contributes to a fixed Q0--Q93 effective-error histogram. The exact Poisson-binomial tail is used within declared work limits; a worst-quality binomial upper bound is used beyond them. | The base and mapping error probabilities are combined conservatively by a union bound. No uncalibrated one-third substitution assumption is made, and base qualities are never repurposed as indel-error probabilities. | | Strelka2 | Model indel error empirically, use tiered haplotype evidence, and handle the matched normal as part of the somatic problem. | Indel/complex significance requires a compatible empirical panel entry; local haplotype evidence is retained; the matched normal is always opened, compatibility checked, and recounted at fixed candidate loci. | MutationSieve has no fitted indel mixture model or normal-contamination model. Missing calibration therefore caps results at exploratory review/no-call instead of substituting generic parameters. | | Lancet / Lancet2 | Joint tumor-normal local sequence context and repeat-aware assembly are valuable for difficult indels. | Repeat-aware normalization, physical CIGAR-junction recount, matched-normal evidence, homopolymer/complexity annotations, and conservative ambiguous-read classification are implemented. | There is no colored de Bruijn graph, k-mer optimization, or graph-path enumeration in this release. | | Octopus | Haplotype-aware models can unify SNVs, indels, complex replacements, and different experimental designs while keeping evidence inspectable. | Normalized SNV, MNV, INS, DEL, and bounded COMPLEX records share one evidence/output contract; exact evidence and limitations remain inspectable. | A transparent rule/rank system is used, not Octopus's Bayesian genotyping models. MutationSieve deliberately emits no calibrated genotype or posterior probability. | | DeepVariant / DeepSomatic | Learned image/tensor classifiers can improve accuracy when trained and tested on representative technologies and specimen types. | No learned classifier is used. The release records structured evidence that could support a future, separately governed model study. | Importing a pretrained score would not establish validity for this panel, specimen preparation, VAF range, or platform. `train` therefore remains fail-closed. | | fgbio molecular consensus | UMI evidence must be formed from real molecular families with explicit consensus likelihoods, then realigned before variant analysis. | Explicit configured UMI tags can define the primary unit; family completeness and consensus are audited separately from coordinate-family shadows. | MutationSieve does not reproduce fgbio's likelihood model, duplex protocol handling, or consensus-read realignment and must not treat coordinate families as molecules. | | BAMSurgeon | Realistic spike-ins should preserve empirical alignment/error context and avoid incompatible haplotype edits. | The hand-constructed aligned-BAM regression fixture rejects duplicate/overlapping truth alleles, isolates random streams, protects encoded event bases, records ordinary versus forced control realization, and includes a structurally independent SpikeForge edit-output control. | The fixture is not generated from FASTQ by a pinned production aligner against the full intended reference. It is a deterministic software control, not synthetic-read validation or sensitivity evidence. | | GA4GH benchmarking and hap.py/vcfeval | Complex variants must be compared by resulting haplotypes inside a precisely defined confident/callable region. | Internal software-control scoring preserves record-exact metrics and a separately labeled bounded MNV/component equivalence check. The public protocol requires a pinned hap.py/vcfeval-style engine as the primary real benchmark. | The built-in bounded equivalence helper is not a general graph comparator and is not presented as one. | ## Sources reviewed - GATK local assembly and haplotype determination: https://gatk.broadinstitute.org/hc/en-us/articles/360036227612 - GATK Mutect2: https://gatk.broadinstitute.org/hc/en-us/articles/360036733771-Mutect2 - Strelka2, *Nature Methods* (2018): https://www.nature.com/articles/s41592-018-0051-x - LoFreq, *Nucleic Acids Research* (2012): https://academic.oup.com/nar/article/40/22/11189/1152727 - Lancet, *Communications Biology* (2018): https://pmc.ncbi.nlm.nih.gov/articles/PMC6123722/ - Lancet2, joint multi-sample local assembly: https://pmc.ncbi.nlm.nih.gov/articles/PMC13064521/ - Octopus, *Nature Biotechnology* (2021): https://www.nature.com/articles/s41587-021-00861-3 - DeepVariant, *Nature Biotechnology* (2018): https://www.nature.com/articles/nbt.4235 - DeepSomatic final paper and public somatic benchmarks: https://www.nature.com/articles/s41587-025-02839-x - DeepSomatic source code: https://github.com/google/deepsomatic - Whole-exome individual-caller and voting-ensemble benchmark: https://pmc.ncbi.nlm.nih.gov/articles/PMC11790059/ - Patient-derived cfDNA caller benchmark across depth and use case: https://pmc.ncbi.nlm.nih.gov/articles/PMC12852940/ - fgbio molecular-consensus documentation: https://fulcrumgenomics.github.io/fgbio/tools/develop/CallMolecularConsensusReads.html - BAMSurgeon, *Bioinformatics* (2015): https://pmc.ncbi.nlm.nih.gov/articles/PMC4856034/ - GA4GH small-variant benchmarking best practices: https://pmc.ncbi.nlm.nih.gov/articles/PMC6699627/ - hap.py reference implementation: https://github.com/Illumina/hap.py ## Consequence for this release The review supports the current conservative architecture, but it does not justify claiming state-of-the-art biological accuracy. The largest algorithmic gap is discovery by local assembly graph: MutationSieve can normalize and rescore an indel that CIGAR evidence exposes, including repeat-shifted events, but cannot discover every hidden haplotype that Mutect2, Lancet, or Octopus may assemble. The software-verification release must therefore keep automatic `PASS`/`HIGH` disabled and publish failed hypotheses and no-call territory alongside any successful software checks.