lekss361/gendesign
1
0
Fork 0
Code Issues 57 Pull requests Projects Releases Packages Wiki Activity Actions

fix(backtest): binomial significance gate on §9.6 verdict (anti false-promote) #1050

Closed
bot-backend wants to merge 1 commit from fix/backtest-verdict-significance-gate into main
pull from: fix/backtest-verdict-significance-gate
merge into: lekss361:main
Conversation 0 Commits 1 Files changed 2 +212 -13

1 commit

Author SHA1 Message Date
Light1YT
5530f472b3 fix(backtest): add binomial significance gate to §9.6 verdict 
The OOS verdict flagged a variant 'candidate to promote' on hit-rate >= 0.5+margin
+ lag_stable alone. On thin data this over-claims: Source A Almon-ADL scored 6/10
(0.60) lag-stable and was flagged as signal, but P(X>=6|10,0.5)~=0.377 -- a coin
flip. Live ground-truth confirmed no signal (full-sample R2~=0.003, wrong sign).

Add exact stdlib-only one-sided binomial _binom_sf_ge + _VERDICT_ALPHA=0.05 and
require P(X>=hits|n_test,0.5) < alpha in both verdict() and cross_source_verdict()
on top of the effect-size margin. hits recovered exactly as round(hit_rate*n_test)
(n_test==scored invariant; no evaluator shape change). Verdict text now states
n_test + the binomial p on pass and fail. Evaluator/estimator math and the
read-only SELECT discipline untouched. Refs #978.
 2026-06-04 16:24:08 +05:00