Commit graph

16 commits

Author SHA1 Message Date
e67cb721bf fix(objective): physflat-dedup current-state consumers + honest n_sold window (#1964)
objective_lots — current-state UPSERT (UNIQUE objective_lot_id, 5 snapshot_date),
но Объектив присваивает ОДНОМУ физлоту (project,corpus,section,floor,lot_number)
несколько lot_id за пере-листинги → таблица раздута ~2.91× (прод: 1.75M квартир-
строк vs 603k физлотов). История — в отдельной objective_lots_history (не трогаем).

STEP 1: миграция 175 — VIEW v_objective_lots_latest (DISTINCT ON physflat-ключ,
последний снапшот). ol.* стабилен (51==51 колонок). DRY-RUN на проде: 603 049
квартир vs 1 753 283 raw.

STEP 2: репойнт current-state консьюмеров objective_lots → v_objective_lots_latest:
- supply_layers._L1_OPEN_SQL (L1 открытое предложение → дефицит-форсайт; прод:
  Юго-Западный комфорт 58 606 → 12 620)
- competitors._SOLD_COUNT_SQL (+ комментарий: COUNT(DISTINCT lot_id) СОХРАНЁН для
  fan-out-защиты маппинга, не COUNT(*) — view гарантирует physflat-дедуп, DISTINCT
  гарантирует mapping-fan-out-safety)
- parcels.py obj_pricing CTE (карточка конкурента units_sold/available)
- special_indices._ARTIFICIAL_DEMAND_SQL
- parcels.py district price block + geo-radius median (sample_size/n)
- concepts._OBJECTIVE_MEDIAN_SQL (гейт n≥10)
- landing KPI3 (% квартир с ценой)
- admin_scrape coverage: `lots` оставлен сырым (ETL-fidelity vs SQLite), добавлен
  `lots_physflat`
#1959 inline-дедуп в market_metrics НЕ рефакторим — добавлен комментарий об общем
physflat-ключе с view.

STEP 3: report_assembler._deal_count теперь = unit_velocity × window_months
(оконные продажи), НЕ кумулятивный n_sold. confidence_engine помечает фактор
«за 6 мес» и гейтит порогами окна (high≥50) — кумулятив (прод EKB ~380 921) делал
гейт бессмысленным и подпись лживой; оконное (~24 876 за 6 мес) честно.

Тесты: +guard'ы (L1/sold-count читают view, deal_count оконный). 963 passed.
2026-06-28 04:27:43 +05:00
4846197d54 fix(site-finder): align competitor flats_sold to objective_lots.contract_date (#1926) (#1927)
All checks were successful
Deploy / changes (push) Successful in 6s
Deploy / build-frontend (push) Has been skipped
Deploy / build-backend (push) Successful in 2m12s
Deploy / build-worker (push) Successful in 4m32s
Deploy / deploy (push) Successful in 1m48s
2026-06-26 18:33:51 +00:00
9c52e0b29f feat(etl): housing-class normalization fallback via yandex_realty trigram match (#38) (#1911)
Some checks failed
Deploy / build-worker (push) Blocked by required conditions
Deploy / build-frontend (push) Blocked by required conditions
Deploy / deploy (push) Blocked by required conditions
Deploy / build-backend (push) Blocked by required conditions
Deploy / changes (push) Has been cancelled
housing-class normalization fallback via yandex_realty trigram (#38): migration 169 + ETL + COALESCE consumers. Refs #38
2026-06-26 07:44:25 +00:00
14f3ef2019 fix(week-review): backend-аудит v2 — 82 фиксов (#1660)
All checks were successful
Deploy / build-worker (push) Successful in 2m47s
Deploy / deploy (push) Successful in 1m20s
Deploy / changes (push) Successful in 9s
Deploy / build-frontend (push) Has been skipped
Deploy / build-backend (push) Successful in 1m52s
Co-authored-by: bot-backend <bot-backend@gendsgn.local>
Co-committed-by: bot-backend <bot-backend@gendsgn.local>
2026-06-17 17:13:38 +00:00
479f01ae1a Merge remote-tracking branch 'forgejo/main' into HEAD
Some checks failed
CI / changes (push) Successful in 7s
CI / changes (pull_request) Successful in 8s
CI / frontend-tests (push) Successful in 1m7s
CI / openapi-codegen-check (push) Failing after 2m9s
CI / frontend-tests (pull_request) Successful in 1m4s
CI / openapi-codegen-check (pull_request) Failing after 1m37s
CI / backend-tests (pull_request) Failing after 9m7s
CI / backend-tests (push) Failing after 9m11s
# Conflicts:
#	backend/app/services/site_finder/gate_verdict.py
2026-06-15 21:07:05 +03:00
86e9ea2937 fix(week-review): автофиксы код-ревью — 169 issue (label «week ревью 1»)
Многоагентный аудит + имплементация: один воркер на файл, точечные правки.
Верификация: py_compile (47/47 .py) + tsc --noEmit (0 ошибок). Unit-тесты
не прогонялись (окружение не поднято: rollup native dep / нет pytest-venv).

Полностью исправлено (169): #1336, #1337, #1339, #1340, #1341, #1342, #1343, #1345, #1346, #1348, #1349, #1350, #1351, #1354, #1356, #1358, #1359, #1360, #1362, #1364, #1365, #1366, #1367, #1368, #1369, #1370, #1371, #1372, #1373, #1374, #1375, #1376, #1377, #1378, #1379, #1380, #1381, #1382, #1384, #1385, #1386, #1387, #1388, #1389, #1390, #1391, #1392, #1394, #1395, #1396, #1397, #1399, #1400, #1401, #1402, #1403, #1404, #1408, #1409, #1410, #1411, #1412, #1413, #1414, #1415, #1416, #1417, #1418, #1420, #1423, #1425, #1426, #1427, #1428, #1429, #1430, #1431, #1432, #1433, #1434, #1435, #1437, #1438, #1439, #1440, #1441, #1442, #1443, #1444, #1445, #1446, #1447, #1448, #1449, #1450, #1451, #1452, #1453, #1454, #1455, #1456, #1457, #1458, #1459, #1460, #1461, #1462, #1463, #1464, #1465, #1466, #1467, #1468, #1469, #1471, #1472, #1473, #1474, #1476, #1478, #1479, #1481, #1482, #1483, #1484, #1485, #1487, #1488, #1489, #1490, #1491, #1492, #1493, #1494, #1495, #1496, #1497, #1499, #1500, #1501, #1502, #1504, #1505, #1506, #1507, #1510, #1514, #1515, #1516, #1517, #1518, #1519, #1521, #1522, #1523, #1524, #1525, #1526, #1527, #1528, #1529, #1531, #1532, #1533, #1534, #1535, #1536, #1537, #1538

Частично (9, in-file часть, остаток cross-file): #1361, #1419, #1422, #1424, #1470, #1475, #1477, #1480, #1498
Требуют cross-file (3, не тронуты): #1338, #1363, #1421
Пропущено (1): #1539

Не входило в партию: 22 needs-Leha issue (нужны решения владельца).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-15 20:21:11 +05:00
b405ae291d feat(site-finder): additive objective_lots price fallback for competitors (#307 OBJ-3) (#1332)
All checks were successful
Deploy / changes (push) Successful in 7s
Deploy / build-backend (push) Successful in 1m57s
Deploy / build-worker (push) Successful in 2m51s
Deploy / build-frontend (push) Successful in 3m6s
Deploy / deploy (push) Successful in 1m12s
2026-06-14 16:10:02 +00:00
5507bbc1a4 feat(site-finder): domrf→cad geom-match + parking_ratio конкурентов (#96) (#1327)
Some checks failed
Deploy / build-frontend (push) Blocked by required conditions
Deploy / deploy (push) Blocked by required conditions
Deploy / build-backend (push) Blocked by required conditions
Deploy / build-worker (push) Blocked by required conditions
Deploy / changes (push) Has been cancelled
2026-06-14 14:49:28 +00:00
ddbd924b02 fix(competitors): _AVG_PRICE_SQL — фильтр latest snapshot (#1210)
Some checks failed
Deploy / deploy (push) Blocked by required conditions
Deploy / changes (push) Successful in 5s
Deploy / build-backend (push) Successful in 1m41s
Deploy / build-frontend (push) Has been skipped
Deploy / build-worker (push) Has been cancelled
domrf_kn_flats версионируется (UNIQUE(id, snapshot_date), м.50), scraper
UPSERT per snapshot — то же что для domrf_kn_objects (которое в L3 supply
после #1212 берём только latest). _AVG_PRICE_SQL фильтра snapshot_date НЕ
имел → AVG усреднял ИСТОРИЮ цен (stale на растущем рынке) → UI-поле
Competitor.avg_price_per_m2 + вход _price_similarity получали устаревшую
цену. COUNT '%прод%' множил sold ×N снапшотов → raw_sold/flat_count кратно
завышен → попадал в гард-нейтраль 0.5 или искажал stage_at_horizon как
×N-завышенный sold_pct.

Patch: WHERE f.snapshot_date = (SELECT MAX(snapshot_date) FROM domrf_kn_flats).
Зеркало паттерна best_layouts._SUPPLY_BATCH_SQL и _COMPETITORS_SQL DISTINCT ON
(уже было latest). 51/51 competitors-тестов зелёные.

Closes #1210
2026-06-13 06:17:26 +00:00
8f3e461959 fix(competitors): _ACTIVE_STATUSES = русские значения как у домрф (#1213)
Some checks failed
Deploy / changes (push) Successful in 7s
Deploy / build-backend (push) Successful in 1m27s
Deploy / build-frontend (push) Has been skipped
Deploy / build-worker (push) Successful in 2m59s
Deploy / deploy (push) Has been cancelled
_ACTIVE_STATUSES = frozenset({"sales", "construction"}) — английский словарь
никогда не совпадал с domrf_kn_objects.site_status, который scraper берёт
СЫРЫМ из siteStatus дом.рф (domrf_kn.py:316). Реальные prod-значения
русские: «Строящиеся»/«Сданные».

Прод-аудит:
- data/sql/105_add_sales_started_flag.sql фильтрует по 'Строящиеся' (~1322 строки).
- partial index 66_indexes_recommend.sql использует те же.
- analytics_queries.py, MarketTab.tsx, CompetitorTable.tsx — все на русских.

Эффект: у ВСЕХ Competitor в POST /parcels/{cad}/competitors is_active=False
и CompetitorsSummary.active_count=0 при любых данных — типизированный
контракт систематически врал.

Patch: _ACTIVE_STATUSES = frozenset({"Строящиеся"}). Заодно обновил два
unit-теста которые кодировали баг (использовали "sales"/"construction"
в моках, тестировали логику против сломанного словаря). Теперь моки
матчат реальную prod-форму.

51/51 competitors-тестов зелёные. ruff clean.

Closes #1213
2026-06-13 05:53:16 +00:00
8206a0b067 perf(forecast): per-request memoization cache for §22 cold build (#1129)
All checks were successful
Deploy / changes (push) Successful in 6s
Deploy / build-backend (push) Successful in 1m49s
Deploy / build-frontend (push) Has been skipped
Deploy / build-worker (push) Successful in 2m45s
Deploy / deploy (push) Successful in 1m10s
Cold §22 forecast measured ~215-233s on prod: §9.x layers re-execute the same
horizon/segment-invariant DB loads with identical args hundreds of times per
report (profiled: get_competitors x69, market_metrics x124, get_monthly_macro
x290). Add a per-report ContextVar cache (forecast_cache(), opened once in the
orchestrator) + @cached(key_builder) on the expensive §9.x loaders so each
unique load runs ONCE and reuses the same frozen, read-only instance.

Output is byte-identical (memoized producers are frozen dataclasses / read-only
Pydantic, callers never mutate; cache is per-report, discarded on exit; no-op
outside the report build). No concurrency, no signature changes.

- forecast_request_cache.py: ContextVar cache + cached() decorator (no-op
  outside context, reentrant, _MISS sentinel for cached None)
- @cached on competitors/future_supply/market_metrics/macro_series/
  sales_series/macro_coefficient/demand_normalization/regression loaders
- orchestrator: wrap build_site_finder_report in forecast_cache()
- 58 tests: key discrimination (call-counting regression guard), no-op-outside,
  per-report isolation, reentrancy, frozen-producer canary, amplification proof
  (real get_monthly_macro xN->1)

code-reviewer APPROVE (keys correct, mutation-safe, output identical). 1265
forecast/cache tests green. No new deps. Refs #1129.
2026-06-08 05:26:27 +00:00
d9c2157d02 feat(site_finder): velocity coverage gap-fill via spatial+name fallback (#968 949-A)
All checks were successful
CI / changes (push) Successful in 6s
CI / changes (pull_request) Successful in 5s
Deploy / changes (push) Successful in 6s
Deploy / build-backend (push) Successful in 1m34s
Deploy / build-frontend (push) Has been skipped
Deploy / build-worker (push) Successful in 2m57s
CI / backend-tests (push) Successful in 5m43s
Deploy / deploy (push) Successful in 1m9s
CI / backend-tests (pull_request) Successful in 5m56s
For competitors missing from objective_complex_mapping, bridge to objective
velocity: nearest objective-bearing complex within 200m whose name tolerantly
matches → complex_id → objective_lots.project_name → objective_corpus_room_month.
Prod-measured: 148 → 329 mapped competitors (2.2×), validated by EXPLAIN+execute.

- nearest_cx CTE UNION'd into mapped (gap-fill only; primary 148 byte-identical,
  no double-count — UNION dedups, DISTINCT ON = one complex per competitor)
- candidates restricted to complexes WITH objective_lots.project_name (data-bearing):
  naive nearest-any gave +37; data-bearing nearest gives +181 (the real win)
- empty-comm_name guard avoids LIKE '%%' spatial-only leak
- velocity.py unchanged (its has_mapping coverage-gate is a separate concern);
  parcel.py unchanged (relevance_weight already satisfies DoD)

Blast radius narrow: affects relevance_weight (via stage_at_horizon) for
newly-mapped competitors; §22 market-pulse velocity (velocity.py) untouched.
deep-code-reviewer ⚠️ minor (approved); review items addressed. Part of EPIC #949.
2026-06-06 22:05:15 +05:00
6a32acb3aa fix(competitors): size-weight avg velocity by flats_total (#949 audit, option B)
All checks were successful
Deploy / changes (push) Successful in 6s
Deploy / build-frontend (push) Has been skipped
Deploy / build-backend (push) Successful in 2m32s
Deploy / build-worker (push) Successful in 3m10s
Deploy / deploy (push) Successful in 1m56s
weighted_avg_velocity was a naive mean despite the name — a 500-flat ЖК weighed
the same as a 20-flat one. Now count-weighted by flats_total (sql.md AVG
principle): Σ(velocity*flats_total)/Σ(flats_total). Competitors with unknown
flats_total are excluded from weights; if sizes are unknown for ALL, graceful
fallback to the simple mean (den>0 guard). Field name + API contract UNCHANGED
(zero consumer ripple — traced: only CompetitorsSummary, no frontend ref).

Tests: equal sizes → weighted==naive (existing 6.0 stays); NEW test with
500-flat@40 + 20-flat@2 → 38.54 (not naive 21.0), proving the weighting.
2026-06-04 14:05:20 +05:00
59be55f80e feat(site-finder): per-competitor relevance_weight (#949 PR B) (#1000)
All checks were successful
Deploy / changes (push) Successful in 6s
Deploy / build-worker (push) Successful in 2m34s
Deploy / build-frontend (push) Has been skipped
Deploy / build-backend (push) Successful in 1m26s
Deploy / deploy (push) Successful in 1m10s
2026-06-02 19:45:38 +00:00
lekss361
31581cedd2 fix(#112): remove broken status='sold' filter from competitors avg_price query
domrf_kn_flats.status is NULL in ~99.8% of rows, so WHERE status='sold'
always returned 0 rows and avg_price_per_m2 was always None. Drop the
filter; AVG over all rows with price_per_m2 IS NOT NULL is semantically
correct for a complex-level price estimate.

Adds regression test test_competitors_avg_price_populated (Issue #227).
2026-05-16 22:46:10 +03:00
e561df1b55 feat(parcels): competitors endpoint (Forgejo #112 Phase 1 backend) (#191)
Some checks failed
CI / backend (push) Successful in 1m16s
Deploy / changes (push) Successful in 6s
Deploy / build-backend (push) Failing after 12s
Deploy / build-worker (push) Failing after 12s
Deploy / build-frontend (push) Failing after 0s
Deploy / deploy (push) Failing after 0s
CI / frontend (push) Successful in 2m17s
2026-05-15 22:21:05 +00:00