Metadata-Version: 2.4
Name: mutationsieve
Version: 0.1.0.dev1
Summary: Auditable research-stage SNV and small-indel evidence calling for targeted DNA sequencing
Author: Yang Shao
License-Expression: MIT
Keywords: bioinformatics,variant-calling,snv,indel,targeted-sequencing,somatic,germline,mosaic,illumina,dnbseq
Classifier: Development Status :: 3 - Alpha
Classifier: Intended Audience :: Science/Research
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Programming Language :: Python :: 3.14
Classifier: Topic :: Scientific/Engineering :: Bio-Informatics
Classifier: Typing :: Typed
Requires-Python: >=3.10
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: numpy>=1.24
Requires-Dist: scipy>=1.10
Provides-Extra: htslib
Requires-Dist: pysam>=0.23; extra == "htslib"
Provides-Extra: llm
Requires-Dist: anthropic>=0.40; extra == "llm"
Provides-Extra: dev
Requires-Dist: pytest>=7.4; extra == "dev"
Requires-Dist: build>=1.0; extra == "dev"
Requires-Dist: jsonschema>=4.18; extra == "dev"
Dynamic: license-file

# MutationSieve

MutationSieve is a research-stage, audit-first caller for single-nucleotide
variants and small insertions/deletions in targeted paired-end DNA sequencing.
It consumes an aligned BAM, an exact compact reference bundle, a target
manifest, and—when available—an assay-matched panel of normals and matched
normal. It writes normalized VCF, review and complete-audit TSVs, versioned
JSON, QC, callability, HTML, and a hash-validated completion manifest.

MutationSieve is **not analytically or clinically validated** and must not be used for
automatic patient reporting. `HIGH` and `PASS` are unavailable in this release
because complete-search statistical calibration has not been established.
`REVIEW` means “inspect this hypothesis”; it does not mean “positive.”

## Scope

Implemented scope:

- targeted-DNA SNVs, adjacent substitutions/MNV representation, and
  insertions/deletions up to 50 bp;
- germline, somatic, and mosaic evidence thresholds;
- duplicate-aware paired-fragment counting, coordinate-family shadow counts,
  and explicit UMI-family consensus when configured tags are sufficiently
  complete;
- matched-normal recounting and a strict, versioned panel-of-normals error
  model;
- full-callable-search multiplicity accounting rather than candidate-only
  adjustment;
- CIGAR-led discovery followed by local reference-versus-alternate haplotype
  rescoring;
- batch sharding, verified resume, isolated failures, deterministic aggregate
  artifacts, simulation, benchmark, validation, report, and release checks.

Explicitly out of scope: large/complex indels beyond the configured limit,
fusions, exon copy-number variants, large structural variants, repeat
expansions, RNA calling, transcript consequence, pathogenicity, oncogenicity,
and treatment interpretation.

## Install and verify

Python 3.10 or newer is required.

```bash
python -m venv .venv
# Windows: .venv\Scripts\activate
# macOS/Linux: source .venv/bin/activate
python -m pip install -e ".[dev]"
python -m pytest -q -p no:cacheprovider
mutationsieve --version
```

An unpacked software-verification evidence release also provides
cross-platform smoke-check launchers. They run the test suite, a reduced
benchmark, software-control scoring, and strict evidence-bundle verification:

```bash
./scripts/quick_validate.sh /tmp/mutationsieve-quick-software-verification
# Windows Command Prompt:
# scripts\quick_validate.cmd C:\Temp\mutationsieve-quick-software-verification
```

See `docs/QUICK_VALIDATION.md` for prerequisites, Windows details, and the
important distinction between this smoke path and the frozen release evidence.

The built-in BAM reader is dependency-free and fail-closed for BAM/BGZF.
CRAM requires the optional HTSlib-backed extra and the exact full reference
FASTA used by the CRAM (the compact target bundle alone is not sufficient):

```bash
python -m pip install -e ".[htslib]"

# add this to run/batch/buildpon when inputs are CRAM
--cram-reference-fasta /exact/path/GRCh38.fa
```

## Quick research workflow

Create a compact, assembly-pinned reference bundle from the exact FASTA used
for alignment and a BED file (BED is 0-based, half-open; column 4 may contain a
target ID, and an optional gene label may be placed in column 7 after BED6):

```bash
mutationsieve buildref \
  --fasta GRCh38.fa \
  --targets panel.bed \
  --assembly GRCh38 \
  --outdir reference
```

Build an error panel from independent, assay-matched negative libraries. A
normal's failure to emit a candidate is not treated as a measured zero: the
union of observed alleles is directly recounted in every normal.

```bash
mutationsieve buildpon /path/to/independent_normals \
  --reference-dir reference \
  --out reference/pon.json
```

Run tumor/normal research analysis:

```bash
mutationsieve run tumor.bam \
  --normal-bam normal.bam \
  --sample CASE_001 \
  --assay somatic \
  --reference-dir reference \
  --pon-path reference/pon.json \
  --outdir results
```

Without a compatible PoN, or without a matched normal in somatic mode, the
pipeline can still surface exploratory hypotheses but caps them at `REVIEW`.

## Counting definitions

- **Raw records** are used only to generate permissive candidates.
- **Paired fragments** reconcile overlapping mates and are the ordinary
  numerator and denominator.
- **BAM-coordinate families** group records by library, unclipped 5′
  coordinates, and orientation. They are a shadow audit proxy, not molecules.
- **Explicit UMI families** become primary only when configured tags are
  sufficiently complete and family consensus passes. This software does not
  claim to implement every vendor's duplex/UMI protocol.

Alternate support and reference depth always use the same primary unit. The
reported AF is therefore a support fraction in that unit, not automatically a
tumor fraction or physical-molecule VAF.

## Output contract

For sample `CASE_001`:

| Artifact | Purpose |
|---|---|
| `CASE_001.variants.tsv` | Non-rejected hypotheses for human review |
| `CASE_001.all-candidates.tsv` | Every generated candidate, including rejects and reasons |
| `CASE_001.variants.vcf` | Normalized non-rejected alleles; `QUAL=.` until deployment calibration exists |
| `CASE_001.mutationsieve.json` | Complete evidence, filters, config, identities, and claims boundary |
| `CASE_001.qc.json` | Sample status, QC, and callability summary |
| `CASE_001.callability.bed` | Per-base depth state plus longest same-unit callable REF span at each start |
| `CASE_001.report.html` | Self-contained local reviewer report |
| `CASE_001.manifest.json` | Hash-validated completion marker, written last |

Sample states distinguish `VARIANT_CANDIDATE_PRESENT`, `NO_VARIANT_FLAG`,
`PARTIAL_COVERAGE_NO_CALL`, and `NO_CALL_QC`. `NO_VARIANT_FLAG` means only that
the frozen research rules emitted no flag inside the callable target territory;
its headline fraction is basewise. Length-specific MNV/indel negatives require
the BED `max_callable_ref_span` value to cover the full REF span or insertion
junction. It is not a clinical negative.

## Why the design is conservative

The caller uses permissive candidate generation, exact normalized REF/ALT,
two-pass fragment-consistent recounting, local haplotype rescoring, explicit
hard and soft rules, compatible background evidence, and full-search
multiplicity. A transparent evidence score ranks candidates; it is not a
probability. Known hotspots, if added by a deployment, may annotate but may not
rescue failed evidence or enter a learned score. Optional LLM text is advisory
only and cannot change evidence, score, tier, genotype, or reportability.

`--enable-reportable-tiers` is retained only as a governed compatibility
declaration in this research release. It also requires an explicit
`assay_compatibility_token`, but that is not sufficient to emit PASS/HIGH:
version 0.1 has no frozen, held-out calibration distribution from complete
negative searches, so every otherwise acceptable candidate remains REVIEW.
The observed BAM platform, discovery settings, and primary counting unit form
the strict PoN identity; filenames are never treated as compatibility proof.

For SNVs and MNVs, the exploratory sequencing-error annotation uses the full
REF+ALT+other-allele primary-unit quality distribution. It combines base-call and mapping
uncertainty by a union bound, stores a fixed Q0-Q93 histogram, and evaluates an
exact Poisson-binomial tail when bounded computation permits; larger inputs use
a worst-quality binomial upper bound. No division-by-three substitution model
is assumed. Base qualities are not treated as indel-error probabilities:
INS/DEL/COMPLEX candidates receive no quality-derived significance, and require
an assay-compatible empirical background model even for exploratory ranking.

Execution is explicitly bounded. In addition to total records, candidates,
pending mates, and UMI callability cells, each alignment record is limited by
`max_query_bases_per_record` (default 1,000) and
`max_events_per_record` (default 20,000). Fixed-locus recount is separately
limited to `max_recount_observations` admitted candidate/unit cells (default
1,000,000). Exceeding a limit fails the run instead of silently truncating it
or retaining an unbounded evidence graph.

## Validation boundary

The included benchmark is a software-engineering experiment using
hand-constructed aligned-BAM regression fixtures and independently edited
SpikeForge interoperability controls derived from the shared synthetic
baseline. Because the coupled alignment fields were not jointly produced by a
pinned production aligner against the intended full reference, this is not
synthetic-read validation. It is intended to find implementation defects and
show which event classes remain weak. It cannot establish analytical or
clinical sensitivity, specificity, PPV, NPV, LoB, LoD95, repeatability,
reproducibility, or transportability. Those require frozen assay-specific
resources, physical reference material, independent truth, dilution
replicates, negatives, lots/runs/operators/sites, prespecified gates,
confidence intervals, and an unopened lockbox.

Positive MutationSieve-generated software controls may declare a minimum of
three analyzable truth-bearing primary units per allele. The simulator records
ordinary Bernoulli selections, ordinary selections that helped satisfy that
floor, and conditionally added units separately. The benchmark aggregates
those counts in `control_generation_accounting`; therefore its
representation-equivalent recovery fraction is a deterministic control-
recovery check and must not be relabeled as analytical or clinical
sensitivity. The SpikeForge control uses a separately implemented edit engine,
resolved from the sibling checkout during development or from the complete
hash-verified frozen source snapshot in an unpacked evidence release. The
control runs in a fresh isolated Python process from a private byte-verified
copy; every loaded `spikeforge.*` module is bound to the recorded source
identity. The bundled release fallback must additionally match an approved
versioned source fingerprint; a live sibling development checkout is recorded
but intentionally not allowlisted so a future version can be evaluated before
release approval. This is not independence of the experiment: the baseline
aligned BAM and compact reference remain shared. Its native `realism` audit
fields and stable gate ID are retained for interface compatibility, but the
human interpretation is limited to structural edit-output integrity and
distribution checks, not realistic alignment or biological performance.

See `docs/RECONCILED_REQUIREMENTS.md`, `docs/SCIENTIFIC_AUDIT.md`,
`docs/ALGORITHM_REVIEW.md`, `docs/VALIDATION_PLAN.md`, and
`docs/EXTERNAL_VALIDATION.md` for the full combined requirements,
primary-method comparison, executable hash-pinned external-caller/evaluator
protocol, and claims boundary inherited from the FusionSieve and LGASieve
reviews. Evaluator and caller names in that protocol are unverified operator
declarations; the harness records raw outputs and computes no scientific pass.
