Code-review #1927 found 2 MAJOR bugs in the initial fix:
1. JOIN fan-out: objective_complex_mapping UNIQUE is (objective_complex_name,
objective_group), NOT domrf_obj_id (68_schema:321) — one obj_id can map to
multiple rows (same name, different group), so COUNT(*) over cm JOIN ol
double-counts lots (GROUP BY collapses output rows but not dups inside the
aggregate). Fixed with COUNT(DISTINCT ol.objective_lot_id).
2. Missing premise_kind: MarketMetrics._STOCK_SQL filters premise_kind='квартира';
the new SQL did not, so sold parking/storage/non-residential lots counted as
sold flats (over-count). Added AND ol.premise_kind = :premise_kind.
3. Aligned the sold predicate to MarketMetrics exactly:
is_sold IS TRUE OR contract_date IS NOT NULL OR LOWER(status)='продан'
(Objective populates these inconsistently — OR mirrors _STOCK_SQL:285-289),
not contract_date alone.
Tests: added static SQL-structure guards (fan-out-safe COUNT(DISTINCT),
premise_kind filter, full predicate) since mock-based API tests bypass real SQL
and cannot detect fan-out; the live-DB fan-out/EXPLAIN check lives in
test_phantom_columns (skipped without TEST_DATABASE_URL). Updated phantom EXPLAIN
to the new SQL.
Full suite: 3530 passed, 51 skipped, 0 failed. Ruff clean.
Audit #1871/#1926: competitor block computed flats_sold from
domrf_kn_flats.status (LIKE '%прод%'/'sold'), populated in only ~0.2% of
rows (99.8% NULL) -> massive undercount (~1600 units in reported case) and
divergence from MarketMetrics, which counts sold via objective_lots.contract_date
(100% populated for sold lots).
Fix: new _SOLD_COUNT_SQL counts sold lots from objective_lots.contract_date
IS NOT NULL, bridged domrf obj_id -> objective_complex_mapping.domrf_obj_id ->
objective_complex_mapping.objective_complex_name == objective_lots.project_name
(same 1:1 mapping pattern as _OBJECTIVE_PRICE_FALLBACK_SQL primary_price CTE).
This is the exact source MarketMetrics._STOCK_SQL uses, so the two paths agree.
Dropped the domrf_kn_flats.status count from _AVG_PRICE_SQL (price-only now).
No schema/Pydantic field change (flats_sold/sold_pct already existed) ->
codegen not needed. psycopg v3 CAST style preserved.
Tests: updated mock DB ordering (+ sold-count execute), added
test_competitors_flats_sold_from_contract_date (120/200 -> 60% sold) and
phantom-column EXPLAIN check for _SOLD_COUNT_SQL.
_ACTIVE_STATUSES = frozenset({"sales", "construction"}) — английский словарь
никогда не совпадал с domrf_kn_objects.site_status, который scraper берёт
СЫРЫМ из siteStatus дом.рф (domrf_kn.py:316). Реальные prod-значения
русские: «Строящиеся»/«Сданные».
Прод-аудит:
- data/sql/105_add_sales_started_flag.sql фильтрует по 'Строящиеся' (~1322 строки).
- partial index 66_indexes_recommend.sql использует те же.
- analytics_queries.py, MarketTab.tsx, CompetitorTable.tsx — все на русских.
Эффект: у ВСЕХ Competitor в POST /parcels/{cad}/competitors is_active=False
и CompetitorsSummary.active_count=0 при любых данных — типизированный
контракт систематически врал.
Patch: _ACTIVE_STATUSES = frozenset({"Строящиеся"}). Заодно обновил два
unit-теста которые кодировали баг (использовали "sales"/"construction"
в моках, тестировали логику против сломанного словаря). Теперь моки
матчат реальную prod-форму.
51/51 competitors-тестов зелёные. ruff clean.
Closes#1213
weighted_avg_velocity was a naive mean despite the name — a 500-flat ЖК weighed
the same as a 20-flat one. Now count-weighted by flats_total (sql.md AVG
principle): Σ(velocity*flats_total)/Σ(flats_total). Competitors with unknown
flats_total are excluded from weights; if sizes are unknown for ALL, graceful
fallback to the simple mean (den>0 guard). Field name + API contract UNCHANGED
(zero consumer ripple — traced: only CompetitorsSummary, no frontend ref).
Tests: equal sizes → weighted==naive (existing 6.0 stays); NEW test with
500-flat@40 + 20-flat@2 → 38.54 (not naive 21.0), proving the weighting.
domrf_kn_flats.status is NULL in ~99.8% of rows, so WHERE status='sold'
always returned 0 rows and avg_price_per_m2 was always None. Drop the
filter; AVG over all rows with price_per_m2 IS NOT NULL is semantically
correct for a complex-level price estimate.
Adds regression test test_competitors_avg_price_populated (Issue #227).
