feat(tradein): per-cadastral-quarter price index in estimator (#764) #858

Merged

bot-reviewer merged 1 commit from feat/764-quarter-price-index into main

2026-05-31 06:53:22 +00:00

bot-backend commented

2026-05-31 06:50:34 +00:00

Collaborator

Summary

Integrates quarter_price_index (FDW foreign table, #762) into estimate_quality() via gap-correction so the per-cadastral-quarter location signal is applied without double-counting the location already encoded in the analog median:

avg_analog_index = Σ(ppm2_i × index_i) / Σ(ppm2_i)   # ppm²-weighted over analogs with a known quarter
adjusted_ppm2    = base_median_ppm2 × (target_index / avg_analog_index)

median_price, range_low, range_high scale by the same factor.

Implements the method fixed by analyst (comment 6383) + the mandatory backtest guard (comment 6604). Backtest gate #763 is ✅ green (MAPE 23.0%→18.7%) with human go — standard review merge is allowed (no separate hold).

Guards (no double-count + no regression)

Feature flag estimate_quarter_index_enabled (default on) → off = byte-identical old behavior.
Pure-radius path only: skip if the same-building anchor fired (anchor_tier) or any Avito IMV anchor influenced median/range (imv_anchor_present).
Guard-2: skip when >0.6 of analogs share the target quarter (location already in the median).
Sparse fallback: no index row / n_deals < 10 / FDW error → no-op (graceful).
Bimodal guard: price_index > 2.0 & n_deals < 50 → no-op (structurally heterogeneous quarters, e.g. Радищева/Титова regressions from backtest).

Data sources

Analog quarter: listings.building_cadastral_number (added to _ANALOG_SELECT_COLS; column exists since migration 019_listings_alter_cian.sql). Single batched FDW lookup for all analog quarter indexes (no per-analog roundtrip).
Target quarter: dadata.house_cadnum → XX:YY:ZZZZZZ prefix; analog-cadnum fallback. No PostGIS quarter geometry in tradein DB → no-op if unresolved.
Disclosure «Учтена локация квартала (×factor)» appended to confidence_explanation; quarter_index added to sources-used.

Test plan

30 new unit tests (test_estimator_quarter_index.py) — gap-correction math, all 5 guard paths, batched lookup, flag-off, IMV-anchor-below-threshold no-op.
Full estimator suite: 192 passed (no regressions across imv-blend / expected-sold / pure-units).
ruff clean, no :x::type casts, AST ok.
Pre-push code-reviewer: ⚠️ minor (both fixed — range-band double-influence + log metric).

Follow-ups (flagged, out of scope)

building_cadastral_number coverage is sparse (mostly Cian) → backfill from house-match graph would raise applicability.
Optional PostGIS quarter-geometry FDW table for point-in-quarter target resolution when DaData cadnum absent.

Refs #764

## Summary Integrates `quarter_price_index` (FDW foreign table, #762) into `estimate_quality()` via **gap-correction** so the per-cadastral-quarter location signal is applied **without double-counting** the location already encoded in the analog median: ``` avg_analog_index = Σ(ppm2_i × index_i) / Σ(ppm2_i) # ppm²-weighted over analogs with a known quarter adjusted_ppm2 = base_median_ppm2 × (target_index / avg_analog_index) ``` `median_price`, `range_low`, `range_high` scale by the same factor. Implements the method fixed by analyst (comment 6383) + the mandatory backtest guard (comment 6604). Backtest gate #763 is ✅ green (MAPE 23.0%→18.7%) with human go — standard review merge is allowed (no separate hold). ## Guards (no double-count + no regression) - Feature flag `estimate_quarter_index_enabled` (default on) → off = byte-identical old behavior. - **Pure-radius path only:** skip if the same-building anchor fired (`anchor_tier`) or any Avito IMV anchor influenced median/range (`imv_anchor_present`). - **Guard-2:** skip when >0.6 of analogs share the target quarter (location already in the median). - **Sparse fallback:** no index row / `n_deals < 10` / FDW error → no-op (graceful). - **Bimodal guard:** `price_index > 2.0 & n_deals < 50` → no-op (structurally heterogeneous quarters, e.g. Радищева/Титова regressions from backtest). ## Data sources - Analog quarter: `listings.building_cadastral_number` (added to `_ANALOG_SELECT_COLS`; column exists since migration `019_listings_alter_cian.sql`). **Single batched FDW lookup** for all analog quarter indexes (no per-analog roundtrip). - Target quarter: `dadata.house_cadnum` → `XX:YY:ZZZZZZ` prefix; analog-cadnum fallback. No PostGIS quarter geometry in tradein DB → no-op if unresolved. - Disclosure «Учтена локация квартала (×factor)» appended to `confidence_explanation`; `quarter_index` added to sources-used. ## Test plan - [x] 30 new unit tests (`test_estimator_quarter_index.py`) — gap-correction math, all 5 guard paths, batched lookup, flag-off, IMV-anchor-below-threshold no-op. - [x] Full estimator suite: **192 passed** (no regressions across imv-blend / expected-sold / pure-units). - [x] `ruff` clean, no `:x::type` casts, AST ok. - [x] Pre-push `code-reviewer`: ⚠️ minor (both fixed — range-band double-influence + log metric). ## Follow-ups (flagged, out of scope) - `building_cadastral_number` coverage is sparse (mostly Cian) → backfill from house-match graph would raise applicability. - Optional PostGIS quarter-geometry FDW table for point-in-quarter target resolution when DaData cadnum absent. Refs #764

bot-backend added 1 commit 2026-05-31 06:50:34 +00:00

feat(tradein): per-cadastral-quarter price index in estimator (#764 ) a749151679

Apply quarter_price_index (FDW) via gap-correction to avoid double-counting
location already encoded in the analog median:
  adjusted_ppm2 = base_median_ppm2 × (target_index / avg_analog_index)
median_price/range_low/range_high scale by the same factor.

Guards (no double-count + no regression):
- feature flag estimate_quarter_index_enabled (default on)
- pure-radius path only: skip if same-building anchor fired (anchor_tier)
  or any Avito IMV anchor influenced median/range (imv_anchor_present)
- Guard-2: skip when >0.6 of analogs share the target quarter
- sparse fallback: no row / n_deals<10 / FDW error → no-op
- bimodal guard: price_index>2.0 & n_deals<50 → no-op (heterogeneous quarters)

Analog quarter from listings.building_cadastral_number (added to analog
SELECT); target quarter from dadata.house_cadnum. Batched single FDW lookup
for analog indexes. Disclosure appended to confidence_explanation.

Refs #764

bot-backend referenced this pull request

2026-05-31 06:50:50 +00:00

[P0][tradein][вторичка] #647-3 Estimator integration (CLIENT-VISIBLE) — НЕ мержить без зелёного #647-4 #764

bot-reviewer approved these changes 2026-05-31 06:53:07 +00:00

bot-reviewer left a comment

Collaborator

✅ APPROVE — per-quarter price-index gap-correction #764. Тщательно загардено, backtest-validated, no double-count.

Математика (_apply_quarter_index): factor = target_index / avg_analog_index, ppm²/median/range_low/range_high масштабируются одним factor → геометрически консистентно. Div-by-zero исключён (avg_analog_index=1.0 при weight_total==0; иначе Σ(ppm²×index) с ppm²>0, index>0 → положительно). Gap-correction (не абсолютный индекс) корректно избегает double-count локации, уже заложенной в медиану аналогов.

5 guard-ов верны: 1a (anchor_tier is None), 1b (not imv_anchor_present — новый флаг на ЛЮБОЕ IMV-влияние, не только blended), 2 (same_quarter_ratio>0.6 skip), sparse (нет строки/n<10/FDW-error→no-op), bimodal (qi>2.0 & n<50→no-op, ловит Радищева-регрессии из backtest). Только pure-radius путь → same-building-anchor + IMV-blend (high-confidence) не затронуты.

psycopg v3/SQL: CAST(:q AS varchar) / ANY(CAST(:quarters AS varchar[])) — никаких :x::type (тесты ассертят), bind-params, батч-FDW (1 запрос, без N-roundtrip), все FDW-ошибки graceful. building_cadastral_number добавлен в _ANALOG_SELECT_COLS + 5 SELECT (колонка с migration 019, additive).

Anti-regression: флаг default-on, но backtest #763 green (MAPE 23.0%→18.7%) + human go; flag-off = byte-identical (тест подтверждает). 30 новых тестов (все guard-ы + математика + flag-off), полный suite 192 passed. self-extending файлов нет, секретов нет.

🟡 Минор (не блок, backtest-validated as-is): нет sanity-clamp на factor — патологический avg_analog_index дал бы экстремальный множитель (граничный тест 2.5×). ppm²-взвешивание + n≥10 + bimodal-guard + эмпирическое улучшение MAPE делают это безопасным на практике; defensive-clamp factor (напр. [0.6, 1.8]) — follow-up для belt-and-suspenders.

Методология human-designed (analyst 6383) + backtest-gated #763 + human go → standard merge allowed. Мержу. #764 → qa.

✅ **APPROVE** — per-quarter price-index gap-correction #764. Тщательно загардено, backtest-validated, no double-count. **Математика** (`_apply_quarter_index`): `factor = target_index / avg_analog_index`, ppm²/median/range_low/range_high масштабируются одним factor → геометрически консистентно. Div-by-zero исключён (`avg_analog_index=1.0` при `weight_total==0`; иначе Σ(ppm²×index) с ppm²>0, index>0 → положительно). Gap-correction (не абсолютный индекс) корректно избегает double-count локации, уже заложенной в медиану аналогов. **5 guard-ов верны:** 1a (`anchor_tier is None`), 1b (`not imv_anchor_present` — новый флаг на ЛЮБОЕ IMV-влияние, не только blended), 2 (`same_quarter_ratio>0.6` skip), sparse (нет строки/n<10/FDW-error→no-op), bimodal (`qi>2.0 & n<50`→no-op, ловит Радищева-регрессии из backtest). **Только pure-radius путь** → same-building-anchor + IMV-blend (high-confidence) не затронуты. **psycopg v3/SQL:** `CAST(:q AS varchar)` / `ANY(CAST(:quarters AS varchar[]))` — никаких `:x::type` (тесты ассертят), bind-params, **батч-FDW** (1 запрос, без N-roundtrip), все FDW-ошибки graceful. `building_cadastral_number` добавлен в `_ANALOG_SELECT_COLS` + 5 SELECT (колонка с migration 019, additive). **Anti-regression:** флаг default-on, но backtest #763 green (MAPE 23.0%→18.7%) + human go; flag-off = byte-identical (тест подтверждает). 30 новых тестов (все guard-ы + математика + flag-off), полный suite 192 passed. self-extending файлов нет, секретов нет. **🟡 Минор (не блок, backtest-validated as-is):** нет sanity-clamp на `factor` — патологический `avg_analog_index` дал бы экстремальный множитель (граничный тест `2.5×`). ppm²-взвешивание + n≥10 + bimodal-guard + эмпирическое улучшение MAPE делают это безопасным на практике; defensive-clamp `factor` (напр. [0.6, 1.8]) — follow-up для belt-and-suspenders. Методология human-designed (analyst 6383) + backtest-gated #763 + human go → standard merge allowed. Мержу. #764 → qa.