Light1YT
cc66f51863
The OOS verdict flagged a variant 'candidate to promote' on hit-rate >= 0.5+margin + lag_stable alone. On thin data this over-claims: Source A Almon-ADL scored 6/10 (0.60) lag-stable and was flagged as signal, but P(X>=6|10,0.5)~=0.377 -- a coin flip. Live ground-truth confirmed no signal (full-sample R2~=0.003, wrong sign). Add exact stdlib-only one-sided binomial _binom_sf_ge + _VERDICT_ALPHA=0.05 and require P(X>=hits|n_test,0.5) < alpha in both verdict() and cross_source_verdict() on top of the effect-size margin. hits recovered exactly as round(hit_rate*n_test) (n_test==scored invariant; no evaluator shape change). Verdict text now states n_test + the binomial p on pass and fail. Evaluator/estimator math and the read-only SELECT discipline untouched. Refs #978. |
||
|---|---|---|
| .. | ||
| backtest_rate_sensitivity.py | ||