feat(tradein/estimator): hedonic year+area correction on expected_sold (#2002) #2004

Merged

lekss361 merged 2 commits from feat/tradein-hedonic-year-area-correction into main

2026-06-27 16:06:08 +00:00

lekss361 commented

2026-06-27 16:05:57 +00:00

Owner

Summary

A hedonic correction on the trade-in expected_sold point — fixes systematic mis-estimation by building era + unit size (the engine underestimated newer buildings and mishandled large lots). The strongest measurable lever from the #2002 elite/accuracy research (year/floor/area are on ground-truth deals; finish/class are not).

factor = clamp( exp(b0 + b_year·(year−2000)/20 + b_larea·ln(area)), 0.75, 1.30 )
expected_sold_{per_m2,price} *= factor

Coefficients fit on n=2366 prod deals (held-out OLS of log(actual_sold/expected_sold) ~ year + ln(area)). Gated behind estimate_hedonic_correction_enabled (default ON; OFF ⇒ exact legacy).

Order: ratio → hedonic factor → le_asking re-clamp → calibrated PI range. The PI range (#2001) wraps the corrected point; the le_asking invariant is re-asserted after the factor (the hedonic could otherwise push expected_sold above asking — overpay risk for trade-in).

Measured effect (frozen backtest gate, regenerated hermetically)

segment	bias before → after	MAPE before → after
эконом	+17.2 → +4.3	18.0 → 15.2
комфорт	−8.0 → −4.1	16.7 → 11.6
бизнес	−21.5 → −10.2	22.1 → 12.9
элит	−38.6 → −33.2 (n=6 fixture noise; not harmed)	—
overall	−1.2 → −2.9	18.63 → 13.23

range_coverage 80.1 → 82.2; headline/sharpness byte-identical (correction is scoped to expected_sold only). On the larger held-out fit set (n=2366) бизнес bias −22%→−14%, overall median-abs-err 18.5%→16.1%, элит unchanged (no harm).

Why year+area and not +dkp

Tested dkp_ratio (and es_price-gating it for premium): it lifts overall MAPE via the эконом/комфорт bulk but worsens бизнес (bias −14%→−19%, MAPE 15.5→18.9) and элит/премиум — and es_price can't cleanly gate premium (same unidentifiability). year+area is best for the value segments and harmless to the tails.

Test plan

regression gate green (replay == regenerated baseline)
new tests/test_estimator_hedonic.py — 6 cases (factor mid/clamp-both-bounds/target_year=None neutral/le_asking invariant holds/control off)
full tradein suite 2508 passed / 2 skipped (CI-equivalent)
7 ratio-mechanism tests hold hedonic OFF to isolate the orthogonal ratio path — reviewer-verified legitimate (no assertion weakening)
code-reviewer ⚠️→✅ (le_asking interaction fixed, hedonic-ON unit coverage added)
re-clamp impact measured: preserves бизнес gain (~1pp), improves overall MAPE

Follow-up (#2002)

Elit point still data-limited (unit-level finish/class not on deals) → manual-review flag + description-class mining (broader extractor; existing parse_house_class catches only 7%) + forward-logging.

Refs #2002

## Summary A hedonic correction on the trade-in `expected_sold` point — fixes systematic mis-estimation by **building era + unit size** (the engine underestimated newer buildings and mishandled large lots). The strongest *measurable* lever from the #2002 elite/accuracy research (`year`/`floor`/`area` are on ground-truth deals; finish/class are not). ``` factor = clamp( exp(b0 + b_year·(year−2000)/20 + b_larea·ln(area)), 0.75, 1.30 ) expected_sold_{per_m2,price} *= factor ``` Coefficients fit on **n=2366 prod deals** (held-out OLS of `log(actual_sold/expected_sold) ~ year + ln(area)`). Gated behind `estimate_hedonic_correction_enabled` (default ON; OFF ⇒ exact legacy). Order: `ratio → hedonic factor → le_asking re-clamp → calibrated PI range`. The PI range (#2001) wraps the corrected point; the `le_asking` invariant is re-asserted after the factor (the hedonic could otherwise push `expected_sold` above asking — overpay risk for trade-in). ## Measured effect (frozen backtest gate, regenerated hermetically) | segment | bias before → after | MAPE before → after | |---|---|---| | эконом | +17.2 → **+4.3** | 18.0 → 15.2 | | комфорт | −8.0 → −4.1 | 16.7 → 11.6 | | **бизнес** | **−21.5 → −10.2** | **22.1 → 12.9** | | элит | −38.6 → −33.2 *(n=6 fixture noise; not harmed)* | — | | **overall** | −1.2 → −2.9 | **18.63 → 13.23** | range_coverage 80.1 → 82.2; headline/sharpness **byte-identical** (correction is scoped to `expected_sold` only). On the larger held-out fit set (n=2366) бизнес bias −22%→−14%, overall median-abs-err 18.5%→16.1%, **элит unchanged (no harm)**. ## Why year+area and not +dkp Tested `dkp_ratio` (and es_price-gating it for premium): it lifts overall MAPE via the эконом/комфорт bulk but **worsens бизнес** (bias −14%→−19%, MAPE 15.5→18.9) **and элит/премиум** — and es_price can't cleanly gate premium (same unidentifiability). `year+area` is best for the value segments and harmless to the tails. ## Test plan - [x] regression gate green (replay == regenerated baseline) - [x] new `tests/test_estimator_hedonic.py` — 6 cases (factor mid/clamp-both-bounds/`target_year=None` neutral/le_asking invariant holds/control off) - [x] full tradein suite **2508 passed / 2 skipped** (CI-equivalent) - [x] 7 ratio-mechanism tests hold hedonic OFF to isolate the orthogonal ratio path — reviewer-verified legitimate (no assertion weakening) - [x] code-reviewer ⚠️→✅ (le_asking interaction fixed, hedonic-ON unit coverage added) - [x] re-clamp impact measured: preserves бизнес gain (~1pp), improves overall MAPE ## Follow-up (#2002) Elit point still data-limited (unit-level finish/class not on deals) → manual-review flag + description-class mining (broader extractor; existing `parse_house_class` catches only 7%) + forward-logging. Refs #2002

lekss361 added 2 commits 2026-06-27 16:05:57 +00:00

feat(tradein/estimator): hedonic year+area correction on expected_sold (#2002 ) 7d9ecb117f

Held-out fit (n=2366 prod deals, 2026-06-27) of
log(actual_sold/expected_sold) ~ year + ln(area) gives a multiplicative
factor that corrects systematic mis-estimation by building era + unit size
(underestimates newer buildings, mis-handles large units).

factor = exp(b0 + b_year*(year-2000)/20 + b_larea*ln(area)), clamped
[0.75, 1.30]; applied to the expected_sold point BEFORE the calibrated PI
range block (so the range follows the corrected point). Headline/asking is
untouched. Gated behind estimate_hedonic_correction_enabled (OFF => exact
old behavior, proven by the regression gate).

Frozen baseline regenerated: overall expected_sold MAPE 18.63->14.24;
per-segment median_bias_pct moves toward 0 (бизнес -21.5->-9.2,
комфорт -8.0->-3.6, эконом 17.2->4.3); элит mildly better, no harm.

Tests that assert the exact ratio relation (expected_sold == headline x
ratio) hold the orthogonal hedonic layer OFF.

fix(tradein/estimator): hedonic respects le_asking + hedonic unit test (#2002 )

CI / changes (pull_request) Successful in 7s

Details

CI / backend-tests (pull_request) Has been skipped

Details

CI / openapi-codegen-check (pull_request) Has been skipped

Details

CI / frontend-tests (pull_request) Has been skipped

Details

f999011b30

Review found the hedonic factor (≤1.30) was applied AFTER the
estimate_expected_sold_le_asking clamp, so expected_sold could exceed the
asking headline (median) by up to ~26% for new/small lots — an overpay risk
that violated the default-True le_asking invariant.

Fix: re-assert le_asking right after the hedonic block and before the
calibrated-PI range block. Final order: ratio clamp -> hedonic factor ->
le_asking re-clamp -> calibrated PI range (range still wraps the clamped
point). round(median_ppm2) keeps expected_sold_per_m2 an int.

The re-clamp does NOT materially erode the новостройка/бизнес benefit
(бизнес bias -9.15->-10.19, MAPE held 12.87) and actually improves overall
expected_sold MAPE on the frozen fixture (14.24->13.23) by capping
over-corrections that overshot actual sold prices.

Adds tests/test_estimator_hedonic.py (hermetic, no DB): factor magnitude vs
OFF, both clamp boundaries, neutral year term for target_year=None, and the
le_asking invariant under hedonic ON. Baseline regenerated; gate green.

Also clarifies the config comment: the quoted held-out fit metrics
(18.5->16.1 etc., n=2366) are distinct from the 277-deal backtest fixture
(18.63->13.23).