Skip to content

Variant Effect Analysis (M4)

Goals served: adaptive splice prediction + drug target identification

Tier: Active (most mature of the active applications)

Last updated: 2026-04

Problem

Genetic variants can disrupt splicing by abolishing canonical sites or creating cryptic ones. Quantifying variant effect on splicing is essential for (a) interpreting clinical variants of uncertain significance (VUS), (b) mapping per-gene splice vulnerability, and © identifying variant-induced isoforms as potential drug targets. A production-grade variant effect pipeline must handle both strands, multiple consequence types (donor loss, acceptor gain, etc.), and benchmark honestly against clinical ground truth.

User-facing functionality

  • Compute per-variant splice delta scores (ref vs alt) using M1-S v2 or M2-S v2
  • Classify splice consequence (donor/acceptor gain/loss, cryptic sites)
  • Benchmark against ClinVar (splice-filtered and unfiltered) with pathogenic/benign discrimination
  • Benchmark against MutSpliceDB (RNA-seq-validated variants) for consequence-type concordance
  • Batch process variants from YAML, VCF, or custom TSV
  • Generate delta-score plots for single variants with flanking context

Driving examples

Additional plan docs:

src/ surface

  • agentic_spliceai.splice_engine.meta_layer.inference.variant_runner.VariantRunner — ref/alt delta computation
  • agentic_spliceai.splice_engine.meta_layer.inference.splice_event_detector.SpliceEventDetector — consequence classification
  • agentic_spliceai.splice_engine.meta_layer.models.sequence_model — M1-S / M2-S loaded for inference
  • agentic_spliceai.splice_engine.features.dense_feature_extractor — locus-level features at ref/alt (note: multimodal features cancel for SNVs)

Evaluation

  • Datasets:
  • ClinVar splice-filtered (N=2,059; 77% pathogenic prevalence)
  • ClinVar unfiltered (N=11,310; ~89% non-splicing mechanisms)
  • MutSpliceDB (N=434; RNA-seq-validated variants)
  • 13 disease-gene variants (curated, dual-strand)
  • 4 SpliceAI paper cases with RNA-seq-confirmed cryptic positions
  • Baselines: OpenSpliceAI base model, M1-S v2 logit blend, M2-S v2
  • Key metrics: PR-AUC, ROC-AUC, consequence concordance (MutSpliceDB)
  • Results:
  • Variant effect validation — 13 variants + 4 RNA-seq cases
  • M4 benchmark sweep — full Phase 2 benchmark report
  • MutSpliceDB analysis — detailed breakdown

Key findings (Phase 2):

  • Base, M1-S v2, M2-S v2 all reach PR-AUC ≈ 0.92 on splice-filtered ClinVar (tie)
  • M2-S v2 wins consequence concordance by +23 pts on MutSpliceDB (68% vs 45%)
  • Locus-level multimodal features cancel for SNVs in Δ computation — meta-layer value is in locus classification, not pathogenicity ranking
  • Cryptic site positions match RNA-seq within 2bp (MYBPC3, FAM229B)

Maturity tier and signals

Current tier: Active

Signals supporting the tier:

  • Most mature active application by benchmark rigor
  • Phase 1A (delta), 1B (consequences), and 2 (benchmarks) all complete
  • Both strands validated; dual-strand fix applied
  • Reproducible benchmark sweep via pod orchestration
  • Stable args across the 4 driver scripts

Graduation signals

To advance to Mature, the application needs:

  • Phase 8.3 clinical pathogenicity head (stacks splice Δ with gnomAD AF, LOEUF, AlphaMissense — see ROADMAP.md)
  • CLI entry point (agentic-spliceai-variant — proposed, not implemented)
  • Inference-path tests
  • SpliceVarDB cross-validation for OOD generalization

Known limitations

  • Locus-level multimodal features do not differentiate SNVs at the same position; meta layer gains are in classification, not Δ ranking
  • Base-model PR-AUC ceiling on ClinVar Δ-ranking (~0.92) — further gains require variant-level features (Phase 8.3)
  • Unfiltered ClinVar floors at PR-AUC ~0.72 because ~89% of pathogenic variants are non-splicing
  • Cross-strand coordinate handling requires careful attention (see negative strand tutorial)
  • Bootstrap confidence intervals needed for smaller benchmarks (MutSpliceDB)