Commit graph

14 commits

Author SHA1 Message Date
ec9ed56fad feat(tradein/backtest): hermetic estimator regression gate — frozen fixture + baseline (#1966 PR 3/3)
All checks were successful
Deploy Trade-In / changes (push) Successful in 11s
Deploy Trade-In / build-frontend (push) Has been skipped
Deploy Trade-In / build-browser (push) Has been skipped
Deploy Trade-In / test (push) Successful in 1m25s
Deploy Trade-In / build-backend (push) Successful in 55s
Deploy Trade-In / deploy (push) Successful in 54s
2026-06-27 12:55:35 +00:00
57bf8ffb81 feat(tradein/backtest): full-spine prediction + range-coverage/calibration/segment metrics (#1966 PR 2/3)
All checks were successful
Deploy Trade-In / changes (push) Successful in 10s
Deploy Trade-In / build-backend (push) Successful in 53s
Deploy Trade-In / deploy (push) Successful in 48s
Deploy Trade-In / build-frontend (push) Has been skipped
Deploy Trade-In / build-browser (push) Has been skipped
Deploy Trade-In / test (push) Successful in 1m24s
Full-spine backtest harness via _price_from_inputs + range-coverage/calibration/per-segment MAPE. code-reviewer APPROVE. Real prod numbers: overall MAPE 18.4%/bias -1.2%; segments эконом +16% / бизнес -23% / элит -38%; range-coverage 56.6%. Read-only; full suite 2488 passed. Refs #1966.
2026-06-27 11:21:57 +00:00
73fa739e0f fix(backfill): drop nonexistent kadastr_num column from listings SELECT
All checks were successful
CI / backend-tests (pull_request) Has been skipped
CI / frontend-tests (pull_request) Has been skipped
CI / openapi-codegen-check (pull_request) Has been skipped
CI / changes (pull_request) Successful in 6s
2026-06-17 23:47:30 +03:00
d021fc9231 feat(backtest): per-rooms asking→sold correction block (#648 S1) (#650)
All checks were successful
Deploy Trade-In / changes (push) Successful in 6s
Deploy Trade-In / build-frontend (push) Has been skipped
Deploy Trade-In / build-backend (push) Successful in 43s
Deploy Trade-In / deploy (push) Successful in 34s
2026-05-29 13:11:55 +00:00
228d12eab1 feat(backtest): read-only estimator-vs-ДКП accuracy harness (#648) (#649)
All checks were successful
Deploy Trade-In / changes (push) Successful in 6s
Deploy Trade-In / build-frontend (push) Has been skipped
Deploy Trade-In / build-backend (push) Successful in 41s
Deploy Trade-In / deploy (push) Successful in 33s
2026-05-29 12:54:00 +00:00
47ba362303 feat(tradein): приоритизировать DaData house-backfill на дома с листингами (accuracy #2)
ORDER BY has_active_listings DESC, id во всех priority-бакетах + новый
--priority listings (focused sweep: NULL cadnum + есть active listing).
Дневной 100/day budget сначала закрывает дома, которые реально появляются
в оценках → estimator #6 cadastr_exact same-building матчинг загорается на
real estimate targets за дни, а не недели.

Correlated EXISTS бьёт по partial-индексу listings_house_id_fk_idx.
Write-path / EKB-guard / SAVEPOINT / qc_geo rules не тронуты.

code-reviewer:  APPROVE no critical. Tests: 40 passed. Ruff clean.
2026-05-28 15:17:01 +05:00
783a85ad2b feat(tradein): geocode leftover deals via Nominatim fallback (#569 Step 3)
Добавляет scripts/geocode_deals_nominatim.py для ~14% сделок всё ещё
lat IS NULL после house-centroid прохода (Step 2) — улицы без геокодированных
домов, которые centroid-join не покрывает. Переиспользует geocoder.geocode
(cache → Cadastral FDW → Yandex → Nominatim), дедупит по адресу (distinct
улицы определяют число вызовов, не 7k строк), и стампит geocode_tried_at на
КАЖДОЙ попытке (успех И чистый промах) чтобы un-geocodable строки не крутились
по 1 req/s. Resume: lat IS NULL + 30-day staleness, per-address SAVEPOINT,
--limit/--dry-run/--batch/--stale-days.

Pre-push review (code-reviewer):  APPROVE, no critical — проверены SQL-safety,
no-infinite-reloop (mark-tried на miss, no-stamp на raised exc), window-cap,
geom-not-manual, dedup, async. Tests: 17 passed. Ruff clean.
2026-05-28 14:59:24 +05:00
1c6d59e3ee feat(tradein): backfill deals.lat/lon из house street-центроидов (#569)
deals (49,791 ДКП-сделок Росреестра) на 100% NULL geom → estimator
_fetch_deals (ST_DWithin) никогда не матчит, actual_deals всегда пустой
(корень #571 «deals disabled»). Добавляет scripts/geocode_deals_from_houses.py:
строит per-street центроиды из ~8.6k геокодированных houses и проставляет
lat/lon каждой сделке по нормализованному ключу улицы (geom авто-заполняется
триггером deals_set_geom_trg). Zero внешних geocoder-вызовов; --dry-run
проецирует coverage и топ-10 unmatched улиц.

Pre-push review поймал data-corruption баг: _APT_SUFFIX_RE без правого якоря
жрал реальные улицы на кв/корп/оф/пом/стр («Строителей»→'ул.' garbage key →
неверный центроид). Fix: требовать цифру после суффикс-токена + регресс-тест.

Tests: 26 (street-key нормализация вкл. regression, centroid AVG, matched UPDATE,
no_street_match, --dry-run no-op, SAVEPOINT isolation, geom-not-manual). Ruff clean.
2026-05-28 14:25:26 +05:00
27fc8101e5 fix(tradein): DaData backfill — Екатеринбург city context (#606 hotfix)
Dry-run на проде показал, что DaData без явного города мапит короткие
street-адреса в Москву по умолчанию («улица Просторная» → «г Москва, ул
Просторная»). Это бы привело к catastrophic false-positive enrichment'у
если бы скрипт запустили без --dry-run.

Two-layer fix:

1. `_with_city_context(addr)` — pre-pends «Екатеринбург, » если в адресе
   ещё нет упоминания города. Идемпотентен, case-insensitive.

2. `_result_in_ekb(result)` — post-filter. Проверяет `raw.city`
   (структурное поле DaData) с fallback на substring search в
   `canonical_address`. Защищает даже если DaData проигнорирует city-префикс
   и всё равно вернёт другой город (например для homonym streets).

`_is_enriched_result()` теперь требует обе проверки: qc_geo IN (0,1)
AND coords AND city == Екатеринбург. Только так делаем UPDATE coords/cadnum.

Без этого fix'а нельзя запускать backfill — все non-EKB false-positives
перезаписали бы NULL coords мусором.

9 новых тестов: city context (prepend / idempotent / case / empty),
post-filter (raw.city / canonical / Moscow reject / None).
Combined 29 passed.
2026-05-27 19:48:37 +05:00
e332052d0e feat(tradein): DaData /clean/address backfill для houses (PR Q3)
Migration 070 — добавляет houses.house_fias_id + dadata_qc_geo / qc_house /
enriched_at для tracking. Скрипт scripts/backfill_houses_dadata.py
проходит по ~3.8k rows с NULL coords + ~8.6k с NULL cadnum daily 100/day
(DaData demo quota → ~124 дня walltime для полного прохода).

Per-row SAVEPOINT, resume-safe (фильтр dadata_enriched_at IS NULL),
qc_geo IN (0,1) → overwrite coord-payload, иначе только tracking.
--priority coords|cadnum|both, --dry-run, --limit N (default 100).

20 тестов покрывают happy path qc_geo=0, qc_geo=3 city-level skip,
DaData None response, per-row exception isolation, --dry-run, --limit,
--priority filter, и missing-creds fail-fast.
2026-05-27 19:43:07 +05:00
1d8cfb967f feat(tradein): backfill 18k existing listings (PR J) (#596)
All checks were successful
Deploy Trade-In / changes (push) Successful in 5s
Deploy Trade-In / build-frontend (push) Has been skipped
Deploy Trade-In / build-backend (push) Successful in 45s
Deploy Trade-In / deploy (push) Successful in 38s
2026-05-27 10:29:40 +00:00
1ce9e10012 feat(tradein): persistent infra для backfill scripts (PR F) (#592)
All checks were successful
Deploy Trade-In / changes (push) Successful in 5s
Deploy Trade-In / build-frontend (push) Successful in 46s
Deploy Trade-In / build-backend (push) Successful in 1m38s
Deploy Trade-In / deploy (push) Successful in 42s
2026-05-27 06:59:54 +00:00
3c506222e6 feat(tradein): Phase 2-3 of #582 — Yandex backfill + canonical audit (#591)
All checks were successful
Deploy Trade-In / changes (push) Successful in 13s
Deploy Trade-In / build-backend (push) Successful in 23s
Deploy Trade-In / deploy (push) Successful in 36s
Deploy Trade-In / build-frontend (push) Has been skipped
2026-05-27 06:30:54 +00:00
02ad9a1f58 feat(tradein): Phase 1 of #582 — address mismatch audit infrastructure
Stratified 200-house Yandex reverse-geocode audit comparing our DB's
(address, lat/lon) pair against what Yandex.Карты returns for those
coordinates. Лежит фундамент данных для Phases 2-5 (canonical = Yandex
per user decision).

What ships:
- Migration 066: address_mismatch_audit table + FDW foreign table
  gendesign_ekb_districts_geom для stratified sampling.
- Sampling SQL: 8 EKB districts × 25 houses = 200.
- Driver scripts/audit_address_mismatch.py с двумя режимами:
  API (если YANDEX_GEOCODER_API_KEY есть) и Playwright (CAPTCHA-aware,
  персистентный контекст). Resumable по --batch.
- Report SQL: p50/p75/p95 distance, top-20 outliers, per-district breakdown.
- 19 unit-тестов: normalize, distance bind, API parser, resume idempotency,
  captcha → status=blocked.

Devops note: после деплоя миграции — выдать tradein_fdw_reader SELECT на
gendesign.public.ekb_districts_geom перед первым запуском audit.

Refs #582 (Phase 1 of 5). Phase 3 backfill заблокирован до Yandex API
key (#402).
2026-05-25 12:50:30 +05:00