Commit graph

9 commits

Author SHA1 Message Date
39dd63333e fix(site_finder): SAVEPOINT-isolate DB errors across cluster A (#2464) (#2465)
All checks were successful
Deploy / changes (push) Successful in 9s
Deploy / build-frontend (push) Has been skipped
Deploy / build-backend (push) Successful in 3m29s
Deploy / build-worker (push) Successful in 4m48s
Deploy / deploy (push) Successful in 1m43s
2026-07-07 12:50:11 +00:00
9b6aba42dc fix(site-finder): L2 hidden-supply не обнуляет неизвестный flat_count (#2445-A3) (#2446)
Some checks failed
Deploy / deploy (push) Blocked by required conditions
Deploy / changes (push) Successful in 7s
Deploy / build-frontend (push) Has been skipped
Deploy / build-backend (push) Has been cancelled
Deploy / build-worker (push) Has been cancelled
2026-07-05 18:54:14 +00:00
c8e7cb1c79 perf(objective_lots): inline DISTINCT ON for 3 view-fullscan consumers + cache landing stats (#1953) (#2067)
All checks were successful
Deploy / build-frontend (push) Has been skipped
Deploy / changes (push) Successful in 7s
Deploy / build-backend (push) Successful in 2m8s
Deploy / build-worker (push) Successful in 3m16s
Deploy / deploy (push) Successful in 5m41s
2026-06-28 17:25:41 +00:00
e67cb721bf fix(objective): physflat-dedup current-state consumers + honest n_sold window (#1964)
objective_lots — current-state UPSERT (UNIQUE objective_lot_id, 5 snapshot_date),
но Объектив присваивает ОДНОМУ физлоту (project,corpus,section,floor,lot_number)
несколько lot_id за пере-листинги → таблица раздута ~2.91× (прод: 1.75M квартир-
строк vs 603k физлотов). История — в отдельной objective_lots_history (не трогаем).

STEP 1: миграция 175 — VIEW v_objective_lots_latest (DISTINCT ON physflat-ключ,
последний снапшот). ol.* стабилен (51==51 колонок). DRY-RUN на проде: 603 049
квартир vs 1 753 283 raw.

STEP 2: репойнт current-state консьюмеров objective_lots → v_objective_lots_latest:
- supply_layers._L1_OPEN_SQL (L1 открытое предложение → дефицит-форсайт; прод:
  Юго-Западный комфорт 58 606 → 12 620)
- competitors._SOLD_COUNT_SQL (+ комментарий: COUNT(DISTINCT lot_id) СОХРАНЁН для
  fan-out-защиты маппинга, не COUNT(*) — view гарантирует physflat-дедуп, DISTINCT
  гарантирует mapping-fan-out-safety)
- parcels.py obj_pricing CTE (карточка конкурента units_sold/available)
- special_indices._ARTIFICIAL_DEMAND_SQL
- parcels.py district price block + geo-radius median (sample_size/n)
- concepts._OBJECTIVE_MEDIAN_SQL (гейт n≥10)
- landing KPI3 (% квартир с ценой)
- admin_scrape coverage: `lots` оставлен сырым (ETL-fidelity vs SQLite), добавлен
  `lots_physflat`
#1959 inline-дедуп в market_metrics НЕ рефакторим — добавлен комментарий об общем
physflat-ключе с view.

STEP 3: report_assembler._deal_count теперь = unit_velocity × window_months
(оконные продажи), НЕ кумулятивный n_sold. confidence_engine помечает фактор
«за 6 мес» и гейтит порогами окна (high≥50) — кумулятив (прод EKB ~380 921) делал
гейт бессмысленным и подпись лживой; оконное (~24 876 за 6 мес) честно.

Тесты: +guard'ы (L1/sold-count читают view, deal_count оконный). 963 passed.
2026-06-28 04:27:43 +05:00
be68ba8f54 fix(site-finder): non-overlapping L2/L3 ready_dt windows, stop double-count of 13-24mo supply (#1355)
Some checks failed
CI / changes (push) Successful in 10s
CI / changes (pull_request) Successful in 6s
CI / frontend-tests (push) Has been skipped
CI / openapi-codegen-check (push) Successful in 1m52s
CI / backend-tests (push) Failing after 8m55s
CI / frontend-tests (pull_request) Has been skipped
CI / openapi-codegen-check (pull_request) Successful in 1m47s
CI / backend-tests (pull_request) Failing after 9m0s
2026-06-17 20:29:09 +03:00
d587b9e199 fix(supply_layers): L3 future-supply берёт ПОСЛЕДНИЙ снапшот → потом фильтр (#1212)
Some checks failed
Deploy / build-frontend (push) Blocked by required conditions
Deploy / deploy (push) Blocked by required conditions
Deploy / changes (push) Successful in 10s
Deploy / build-worker (push) Has been cancelled
Deploy / build-backend (push) Has been cancelled
_L3_FUTURE_SQL применял volatile-фильтры (ready_dt > horizon, free_flats
<= threshold) в WHERE ВНУТРИ CTE с DISTINCT ON (obj_id) ORDER BY
snapshot_date DESC. Это значит фильтр применялся ДО DISTINCT ON →
бралась «последний снапшот, ПРОШЕДШИЙ фильтр», а не последний в принципе.

Эффект на проде: объект, когда-то бывший «объявлен, не продаётся»
(free_flats=0/NULL), остаётся в L3 future-supply даже после открытия
продаж — свежий снапшот с free_flats=180 отфильтрован, взят старый
с free_flats=0. Двойной счёт с L1/L2 (open + future одного объекта),
stale flat_count/ready_dt в supply_layers и форсайте.

Patch: разделил фильтры на стабильные (в CTE: region_cd, district_name)
и volatile (во внешнем WHERE после DISTINCT ON: ready_dt, free_flats).
Зеркало паттерна L2-CTE (там volatile уже снаружи). Семантика теперь
матчит docstring: «свежий снапшот, затем фильтр».

62/62 supply_layers тестов зелёные. ruff clean. SQL psycopg v3
(CAST(:x AS interval)) уже корректен.

Closes #1212
2026-06-13 05:53:01 +00:00
681a922d99 feat(forecast): resolve admin district -> micro set in §9.x market/supply/sales filters
All checks were successful
Deploy / changes (push) Successful in 6s
Deploy / build-frontend (push) Has been skipped
Deploy / build-backend (push) Successful in 1m29s
Deploy / build-worker (push) Successful in 2m32s
Deploy / deploy (push) Successful in 1m7s
/analyze passes the official ЕКБ admin district (ekb_districts polygon, e.g.
'Кировский'), but objective_lots/corpus_room_month store informal micro-districts
('Втузгородок','ЖБИ') -> admin name matched 0 rows -> silent empty forecast.

Add resolve_objective_districts() (site_finder/district_resolver.py) mapping an
admin name to its clean micros via ekb_district_alias (note IS NULL), with
None -> EKB-wide fallback and raw-micro pass-through. Wire into the objective_lots
district filters of market_metrics (§9.2 stock+sales), supply_layers L1 (§9.3),
and sales_series Sources A+B (crm shares the micro vocab, prod-verified),
switching the scalar filter to psycopg3-safe = ANY(CAST(:districts AS text[])).
supply_layers L2/L3 keep the admin name (domrf_kn_objects.district_name is admin vocab).

Prod: Кировский/Ленинский/Орджоникидзевский obj_count 0 -> 32/64/31.
Tests mutation-verified non-vacuous. 192 module tests pass; ruff clean. Refs #969 #949.
2026-06-05 07:03:37 +05:00
ed3b302d57 fix(supply-layers): thread dev_group_name into L3 upsert key (#970 CRITICAL)
All checks were successful
Deploy / changes (push) Successful in 6s
Deploy / build-frontend (push) Has been skipped
Deploy / build-backend (push) Successful in 1m22s
Deploy / build-worker (push) Successful in 2m32s
Deploy / deploy (push) Successful in 1m4s
REOPENED. L3 future-supply rows are computed per (district_name, dev_group_name)
but dev_group_name was never a key column — only embedded in method text. With
complex_id/obj_class NULL for L3, every dev_group of a district collapsed to one
upsert key → ~95.6% loss. Ground-truth (Академический, prod): should be 13,808
units / 15 dev_groups / 54 objects; only 1 row / 607 units survived.

Migration 128: ADD COLUMN supply_layers.dev_group_name TEXT + rebuild
uq_supply_layers_logical to (layer, district_name, complex_id, obj_class,
dev_group_name, source, snapshot_date) NULLS NOT DISTINCT (L1/L2 dev_group_name
NULL stays transparent → their dedup unchanged; L3 distinct groups no longer
collapse). Dry-run-verified vs prod catalog (applies clean, ROLLBACK clean).

Worker: SupplyLayerRow gains dev_group_name (L1/L2=None, L3=group); _UPSERT_SQL
adds it to INSERT/VALUES (CAST(:dev_group_name AS text)) + ON CONFLICT (key col,
not in DO UPDATE SET). Service+worker regression tests assert same-district/
different-dev_group → distinct keys (no collapse). 234 supply tests pass.

Deploy applies migration before container restart; collapsed data self-heals on
next supply_layers_refresh. Verification = prod re-measure post-deploy.

Refs #970
2026-06-04 10:06:30 +05:00
900802264a feat(site-finder): supply_layers v2 compute service (#950 EPIC6 step3+4) (#1004)
All checks were successful
Deploy / build-worker (push) Successful in 2m30s
Deploy / changes (push) Successful in 5s
Deploy / build-frontend (push) Has been skipped
Deploy / build-backend (push) Successful in 1m40s
Deploy / deploy (push) Successful in 1m6s
2026-06-02 20:34:26 +00:00