fix(tradein/cian): персист bti_data/valuation house_info в houses + дедуп УК (#2435) #2437

Merged
lekss361 merged 2 commits from feat/tradein-cian-houses-bti-valuation into main 2026-07-04 20:21:33 +00:00

2 commits

Author SHA1 Message Date
bot-backend
db9bec8142 fix(tradein/cian): management_companies dedup — SELECT-before-INSERT, not ON CONFLICT
All checks were successful
CI Trade-In / changes (pull_request) Successful in 8s
CI / changes (pull_request) Successful in 8s
CI Trade-In / frontend-checks (pull_request) Has been skipped
CI / backend-tests (pull_request) Has been skipped
CI / frontend-tests (pull_request) Has been skipped
CI / openapi-codegen-check (pull_request) Has been skipped
CI Trade-In / backend-tests (pull_request) Successful in 51s
ext_id=NULL rows never conflict under Postgres default NULLS DISTINCT, so the
ON CONFLICT (ext_source, name, ext_id) DO UPDATE in _upsert_cian_valuation_management_company
was dead code — every valuation cache-miss for the same building inserted a fresh
duplicate management_companies row instead of deduping by name. Replaced with an
explicit SELECT-then-branch (UPDATE existing / INSERT new); docstring corrected
to stop claiming the constraint dedups it. Found by deep-code-reviewer on #2435.

Refs #2435
2026-07-04 23:16:25 +03:00
bot-backend
37ea0b0f5b fix(tradein/cian): персистить bti_data и valuation house_info в houses (#2435)
Cian secondary-market detail (bti_data) и Valuation Calculator (house_info/
managementCompany/houseId) парсились, но выбрасывались — комментарий "это
задача Stage 6 (houses)" так и не был выполнен.

- cian/detail.py::save_detail_enrichment принимает инжектируемый matcher
  (HouseMatcher, optional) → новый _persist_cian_bti_house резолвит дом через
  match_or_create_house (address/geo листинга, mirror avito/houses.py::
  _persist_house) и пишет BTI-эксклюзивные колонки из 020_houses_alter_cian.sql
  (series_name/entrances/flat_count/is_emergency/heat_supply_type/
  gas_supply_type/overlap_type) через COALESCE(new, existing).
- cian/valuation.py::_save_to_cache получает уже резолвленный house_id (read-only
  match_house_readonly, estimator.py) → новый _persist_cian_valuation_house пишет
  management_company_id (UPSERT management_companies) + cian_internal_house_id
  (filters.houseId) COALESCE(new, existing), плюс houseInfo.items-производные
  поля (год/тип/этажность/газ/отопление/перекрытия/подъезды/квартиры/
  аварийность/детская площадка/лифты) COALESCE(existing, new) — валюация не
  авторитетный источник для них (conflict_resolution.HOUSE_FIELD_PRIORITY).
- Оба пути best-effort: SAVEPOINT (db.begin_nested) изолирует сбой резолва/
  записи дома от основной save-транзакции; house_id=None / matcher=None /
  безномерный адрес (P1 no_house_number) — graceful no-op, без исключений.
- matcher прокинут в реальные call sites: pipeline.py (cian city-sweep +
  full-load — уже был в scope), cian_history_backfill.py, cian_price_history.py,
  admin.py ad-hoc endpoint.

Gap (нет чистого маппинга на существующую колонку houses — не создавали новых
колонок): bti.houseData.{demolishedInMoscowProgramm, heatIndex,
houseOverhaulFundType, lifts (недифференцированный total)}; houseInfo.items
{Мусоропровод, Реновация, Спортивная площадка, Фонд капремонта}.

10 новых тестов (tests/test_cian_bti_house_persist.py), MagicMock db, без
реальной БД — mirror test_snapshot_writer.py / test_extval_house_id_write_path.py.
2026-07-04 22:56:36 +03:00