Evaluating the added value of sequence models in noncoding RNA assays: a baseline-aware framework

Abstract

Sequence models are evaluated against non-coding RNA assay read-outs, but prediction alone does not show what their features add to an explicit baseline. We estimated model-added-last changes in AUROC for lncRNA labels and cross-validated Spearman correlation for miRNA repression, fixing members and chromosome folds across available sources. In cross-cell CRISPRi (n = 2961), AlphaGenome increased AUROC from 0.6426 to 0.7551; adding K562 accessibility changed these values to 0.7282 and 0.7534. Corrected-source Cas13 increments ranged from -0.0161 to +0.0130 (n = 871). Separate post-audit comparisons tested fixed sequence composition, broader regularization and TargetScan aggregate identity. With expanded regularization, composition improved four baselines but reduced Cas13 AUROC by 0.0365. RNA-FM's increment was -0.0014 for CRISPRi and +0.000044 for HEK293FT after composition adjustment; small signs changed with regularization. On the same 1227 HeLa members, replacing a seed-family aggregate with an identity-supported mature-miRNA aggregate increased baseline Spearman from 0.4140 to 0.4799. RNA-FM then added 0.0170, or 0.0014 after composition adjustment. These source-row aggregates lack experimental transcript/site alignment. The results support testing baseline content, identity and regularization before interpreting residual model increments. Negative or near-zero increments do not establish absent information or equivalence. Resampling summaries remain conditional descriptions or non-joint sensitivities; calibrated confirmation, justified practical-size thresholds and pretraining-overlap verification were unavailable.

Lifecycle

• biorxiv v1 2026-10-25 source ↗

Discussion

No qualifying discussion yet.