Commit graph

8 commits

Author SHA1 Message Date
8074cb49ef perf(objective_lots): inline-pushdown competitors._SOLD_COUNT_SQL + covering physflat index (#1953)
All checks were successful
CI / changes (pull_request) Successful in 8s
CI / frontend-tests (pull_request) Has been skipped
CI / openapi-codegen-check (pull_request) Successful in 2m14s
CI / backend-tests (pull_request) Successful in 14m23s
Прод EXPLAIN-охота (follow-up эпика #1953):

#1 competitors._SOLD_COUNT_SQL — фикс регрессии #1964. JOIN
v_objective_lots_latest материализовал DISTINCT ON по всей 1.76M-таблице
ДО фильтра маппинга (qual не проталкивается ниже view-DISTINCT ON) →
Parallel Seq Scan + external sort (~213 MB temp). Переписан на
inline-pushdown (зеркало #1964 / market_metrics._STOCK_SQL): фильтр
objective_complex_mapping в CTE → DISTINCT ON по СЫРОЙ objective_lots,
JOIN ON project_name (objective_lots_project_idx). Прод: 7958 → 292 ms,
вывод побайтово идентичен (15/15 obj_id, 0 mismatch). COUNT(DISTINCT
objective_lot_id) сохранён (fan-out-safety).

#2/#6/#5 миграция 176 — покрывающий physflat-индекс. EKB-wide
_STOCK_SQL/_SALES_WINDOW_SQL full-seq-scan + external sort (~140 MB temp),
т.к. partial-индекс 173 не нёс проецируемых колонок. Новый
objective_lots_physflat_covering_idx = тот же ключ + INCLUDE(rooms_int,
area_pd, sales_start_date, is_sold, contract_date, status, class) +
WHERE premise_kind='квартира'; старый _latest_idx дропнут (полностью
замещён). Прод DRY-RUN: Index Only Scan + Unique, без external sort,
5177 → 1595 ms; индекс 252 MB. ANALYZE objective_lots в конце tx.

Деплой вне недельного окна objective-scrape (кратковременный SHARE-lock
при drop+create ~252 MB индекса на 1.75M строк).
2026-06-28 18:45:47 +05:00
e67cb721bf fix(objective): physflat-dedup current-state consumers + honest n_sold window (#1964)
objective_lots — current-state UPSERT (UNIQUE objective_lot_id, 5 snapshot_date),
но Объектив присваивает ОДНОМУ физлоту (project,corpus,section,floor,lot_number)
несколько lot_id за пере-листинги → таблица раздута ~2.91× (прод: 1.75M квартир-
строк vs 603k физлотов). История — в отдельной objective_lots_history (не трогаем).

STEP 1: миграция 175 — VIEW v_objective_lots_latest (DISTINCT ON physflat-ключ,
последний снапшот). ol.* стабилен (51==51 колонок). DRY-RUN на проде: 603 049
квартир vs 1 753 283 raw.

STEP 2: репойнт current-state консьюмеров objective_lots → v_objective_lots_latest:
- supply_layers._L1_OPEN_SQL (L1 открытое предложение → дефицит-форсайт; прод:
  Юго-Западный комфорт 58 606 → 12 620)
- competitors._SOLD_COUNT_SQL (+ комментарий: COUNT(DISTINCT lot_id) СОХРАНЁН для
  fan-out-защиты маппинга, не COUNT(*) — view гарантирует physflat-дедуп, DISTINCT
  гарантирует mapping-fan-out-safety)
- parcels.py obj_pricing CTE (карточка конкурента units_sold/available)
- special_indices._ARTIFICIAL_DEMAND_SQL
- parcels.py district price block + geo-radius median (sample_size/n)
- concepts._OBJECTIVE_MEDIAN_SQL (гейт n≥10)
- landing KPI3 (% квартир с ценой)
- admin_scrape coverage: `lots` оставлен сырым (ETL-fidelity vs SQLite), добавлен
  `lots_physflat`
#1959 inline-дедуп в market_metrics НЕ рефакторим — добавлен комментарий об общем
physflat-ключе с view.

STEP 3: report_assembler._deal_count теперь = unit_velocity × window_months
(оконные продажи), НЕ кумулятивный n_sold. confidence_engine помечает фактор
«за 6 мес» и гейтит порогами окна (high≥50) — кумулятив (прод EKB ~380 921) делал
гейт бессмысленным и подпись лживой; оконное (~24 876 за 6 мес) честно.

Тесты: +guard'ы (L1/sold-count читают view, deal_count оконный). 963 passed.
2026-06-28 04:27:43 +05:00
4846197d54 fix(site-finder): align competitor flats_sold to objective_lots.contract_date (#1926) (#1927)
All checks were successful
Deploy / changes (push) Successful in 6s
Deploy / build-frontend (push) Has been skipped
Deploy / build-backend (push) Successful in 2m12s
Deploy / build-worker (push) Successful in 4m32s
Deploy / deploy (push) Successful in 1m48s
2026-06-26 18:33:51 +00:00
b405ae291d feat(site-finder): additive objective_lots price fallback for competitors (#307 OBJ-3) (#1332)
All checks were successful
Deploy / changes (push) Successful in 7s
Deploy / build-backend (push) Successful in 1m57s
Deploy / build-worker (push) Successful in 2m51s
Deploy / build-frontend (push) Successful in 3m6s
Deploy / deploy (push) Successful in 1m12s
2026-06-14 16:10:02 +00:00
8f3e461959 fix(competitors): _ACTIVE_STATUSES = русские значения как у домрф (#1213)
Some checks failed
Deploy / changes (push) Successful in 7s
Deploy / build-backend (push) Successful in 1m27s
Deploy / build-frontend (push) Has been skipped
Deploy / build-worker (push) Successful in 2m59s
Deploy / deploy (push) Has been cancelled
_ACTIVE_STATUSES = frozenset({"sales", "construction"}) — английский словарь
никогда не совпадал с domrf_kn_objects.site_status, который scraper берёт
СЫРЫМ из siteStatus дом.рф (domrf_kn.py:316). Реальные prod-значения
русские: «Строящиеся»/«Сданные».

Прод-аудит:
- data/sql/105_add_sales_started_flag.sql фильтрует по 'Строящиеся' (~1322 строки).
- partial index 66_indexes_recommend.sql использует те же.
- analytics_queries.py, MarketTab.tsx, CompetitorTable.tsx — все на русских.

Эффект: у ВСЕХ Competitor в POST /parcels/{cad}/competitors is_active=False
и CompetitorsSummary.active_count=0 при любых данных — типизированный
контракт систематически врал.

Patch: _ACTIVE_STATUSES = frozenset({"Строящиеся"}). Заодно обновил два
unit-теста которые кодировали баг (использовали "sales"/"construction"
в моках, тестировали логику против сломанного словаря). Теперь моки
матчат реальную prod-форму.

51/51 competitors-тестов зелёные. ruff clean.

Closes #1213
2026-06-13 05:53:16 +00:00
6a32acb3aa fix(competitors): size-weight avg velocity by flats_total (#949 audit, option B)
All checks were successful
Deploy / changes (push) Successful in 6s
Deploy / build-frontend (push) Has been skipped
Deploy / build-backend (push) Successful in 2m32s
Deploy / build-worker (push) Successful in 3m10s
Deploy / deploy (push) Successful in 1m56s
weighted_avg_velocity was a naive mean despite the name — a 500-flat ЖК weighed
the same as a 20-flat one. Now count-weighted by flats_total (sql.md AVG
principle): Σ(velocity*flats_total)/Σ(flats_total). Competitors with unknown
flats_total are excluded from weights; if sizes are unknown for ALL, graceful
fallback to the simple mean (den>0 guard). Field name + API contract UNCHANGED
(zero consumer ripple — traced: only CompetitorsSummary, no frontend ref).

Tests: equal sizes → weighted==naive (existing 6.0 stays); NEW test with
500-flat@40 + 20-flat@2 → 38.54 (not naive 21.0), proving the weighting.
2026-06-04 14:05:20 +05:00
lekss361
31581cedd2 fix(#112): remove broken status='sold' filter from competitors avg_price query
domrf_kn_flats.status is NULL in ~99.8% of rows, so WHERE status='sold'
always returned 0 rows and avg_price_per_m2 was always None. Drop the
filter; AVG over all rows with price_per_m2 IS NOT NULL is semantically
correct for a complex-level price estimate.

Adds regression test test_competitors_avg_price_populated (Issue #227).
2026-05-16 22:46:10 +03:00
e561df1b55 feat(parcels): competitors endpoint (Forgejo #112 Phase 1 backend) (#191)
Some checks failed
CI / backend (push) Successful in 1m16s
Deploy / changes (push) Successful in 6s
Deploy / build-backend (push) Failing after 12s
Deploy / build-worker (push) Failing after 12s
Deploy / build-frontend (push) Failing after 0s
Deploy / deploy (push) Failing after 0s
CI / frontend (push) Successful in 2m17s
2026-05-15 22:21:05 +00:00