Add `velocity_by_room: dict[str, float] | None` to `MarketMetrics` — per-bucket
unit velocity (ед./мес) derived from the existing `sold_by_room` ROLLUP data that
`_query_sales_window` already returns. No new SQL required.
Thread per-bucket velocity through `_demand_only_overlay` via the new
`_FORECAST_TO_METRIC_BUCKETS` constant that maps each forecast bucket to its
market_metrics room-bucket keys. "80+ м²" sums "4" + "5+" keys. Fallback to
aggregate `unit_velocity` when `velocity_by_room` is None (thin-data path).
Previously `base_pace` was identical for all 5 room-buckets, so §9.4 norm and §9.2
base_pace cancelled out in pace/max_pace and ranking was driven purely by §9.5
macro_coef (segment steepness proxy). Now §9.2 reflects real per-bucket observed
demand from objective_lots.contract_date data.
Callers of `compute_market_metrics` that don't use `velocity_by_room` are unaffected
(the new field is additive to the frozen dataclass). All existing callers verified —
none construct `MarketMetrics` directly except the one production site.
_price_sensitivity передавал сырое admin-имя ('Кировский') в _elasticity_coef,
который фильтрует objective_corpus_room_month.district по МИКРО-вокабуляру
(Втузгородок, ЖБИ, …) → регрессия получала 0 точек → всегда FALLBACK_ELASTICITY.
§9.2 district-level эластичность молча НЕ считалась в /analyze-пути (только
'Академический' совпадал в обоих вокабулярах случайно).
Fix: вызываем resolve_objective_districts() в _price_sensitivity и передаём
список микро через новый kwarg districts=[…] в _elasticity_coef. Резолвер
None ('не определён' / нет чистых алиасов) → пустой список → EKB-wide
регрессия. _elasticity_coef расширен с back-compat: districts=None →
legacy путь по district_name (другой caller в analytics_queries —
отдельный bug class, вне scope).
5 новых юнит-тестов TestPriceSensitivityDistrictResolution: admin→micros в
SQL bind, None→EKB-wide, regression preserved post-resolve, graceful.
76/76 market_metrics + 156/156 elasticity/sensitivity тестов зелёные.
ruff + psycopg v3 grep clean.
Closes#1211
_SALES_WINDOW_SQL делал GROUP BY ROLLUP (rooms_int), rooms_int nullable
(ETL пишет NULL для «неопределённого типа», sales_series.py:399 явно
обрабатывает None). Проданный лот с rooms_int IS NULL даёт ДВЕ строки
rooms_int IS NULL (NULL-группа + grand-total итог), неразличимые в
Python (оба if r["rooms_int"] is None).
MixedAggregate-план PG16 эмитит grand-total ПЕРВЫМ (среди hash-строк),
NULL-группа после → loop затирает units_total частичным счётом (живой
тест на PG16: 2000 → 200). Эффект: unit_velocity / absorption_rate
занижены, months_of_supply завышен → base_pace в demand_supply_forecast
неверный (recommendation.py:586) → reports/scoring врёт.
Patch:
- SQL: добавить GROUPING(rooms_int) AS is_total (=1 для grand-total).
- Python: ветвить по is_total, NULL-комнатную группу класть в
by_room['unknown'] (отдельный бакет), аккумулировать через +=
вместо assign (защита от будущих NULL-вариантов).
- Тесты: моки получили "is_total" поле (1 для grand-total, 0 иначе).
71/71 market_metrics тестов зелёные. ruff clean.
Closes#1214
Foundation PR: unified "our projects" source the §25.3 overlap engine (PR2)
will consume. Two origins normalized to OwnProject (class/timing/price/unit-mix):
- current <- domrf_kn_objects filtered by settings.own_developer_ids (numeric
prefix of composite dev_id; empty -> [] graceful, no DB hit, no hardcoded id)
- future <- new manual-entry own_planned_project entity (migration 148)
Adds OWN_DEVELOPER_IDS config (comma-sep -> list[int], default []),
own_planned_project table (range/unit_mix CHECKs via IMMUTABLE helper, generated
geom), /api/v1/own-projects CRUD (created_by from X-Authenticated-User), and
get_own_portfolio(db). Per-source graceful degradation; psycopg-v3 CAST clean.
Does not touch special_indices.py or parcels.py (out of scope).
Refs #1169
demand_index used a fixed clamp01(velocity/50); on a live prod refresh
all 8 ЕКБ districts sold ≥50/mo so it saturated to 1.0 everywhere — zero
discrimination between districts. Redesign to mirror infra_index:
normalize each district's unit_velocity against the city reference (MAX
district velocity per refresh run), so demand always discriminates and
self-calibrates as the market grows (no magic constant to rot).
- normalize_demand(velocity, *, city_reference_velocity), pure + graceful
(None stays None; reference<=0 -> honest 0.0, no ZeroDivisionError)
- refresh_locations now two-pass: collect velocities (one
compute_market_metrics per district, no O(n^2)), derive city reference,
normalize + upsert; SAVEPOINT-per-row and counters preserved
- remove _DEMAND_SATURATION_UPM constant; log city_reference_velocity
- tests: rewrite demand normalization + add end-to-end city-relative
suite incl. discrimination regression guarding the all-1.0 prod bug
Refs #948
Promote district to a first-class `location` entity (ТЗ §8.2), ADDITIVE — no
district->FK refactor. New `location` table keyed by district_name (joinable by
string), carrying 4 normalized [0,1] indices (NULL when no data, never 0):
- infra: _district_poi_score / _city_avg_poi_score (per-district POI aggregate)
- competition: market_metrics.overstock_index (available/stuck competing supply
— orthogonal to demand; NOT sell-through, which is market-heat correlated w/ demand)
- demand: market_metrics.unit_velocity (saturating /50)
- future_supply: future_supply_pressure.index (passthrough, already 0..1)
- data/sql/146_location.sql: idempotent table + UNIQUE(district_name) + range
CHECK + centroid GIST
- services/site_finder/locations.py: compute_location_indices (reuses forecast
per-district fns) + refresh_locations (SAVEPOINT per-row, CAST, ON CONFLICT)
- workers/tasks/location_refresh.py + beat (Mon 07:00 MSK, after supply-layers)
- api/v1/locations.py: read-only GET list + GET by name (analyst+admin via rbac,
frontend pilot-gated)
- tests: 34 (normalization 0..1/null, competition⊥demand orthogonality, idempotent
upsert, read API list/by-name/404)
Part of #948 (Part A insight shipped #1164).
/analyze passes the official ЕКБ admin district (ekb_districts polygon, e.g.
'Кировский'), but objective_lots/corpus_room_month store informal micro-districts
('Втузгородок','ЖБИ') -> admin name matched 0 rows -> silent empty forecast.
Add resolve_objective_districts() (site_finder/district_resolver.py) mapping an
admin name to its clean micros via ekb_district_alias (note IS NULL), with
None -> EKB-wide fallback and raw-micro pass-through. Wire into the objective_lots
district filters of market_metrics (§9.2 stock+sales), supply_layers L1 (§9.3),
and sales_series Sources A+B (crm shares the micro vocab, prod-verified),
switching the scalar filter to psycopg3-safe = ANY(CAST(:districts AS text[])).
supply_layers L2/L3 keep the admin name (domrf_kn_objects.district_name is admin vocab).
Prod: Кировский/Ленинский/Орджоникидзевский obj_count 0 -> 32/64/31.
Tests mutation-verified non-vacuous. 192 module tests pass; ruff clean. Refs #969#949.
FOLLOW-UP (Leha: not a bug, follow-up).
get_macro_series / get_latest_macro defaulted region='rf', but CBR mortgage_*
series live under 'sverdl' in macro_indicator (key_rate under 'rf'). Only caller
passes region explicitly so no active leak, but a future caller omitting region
would silently get None — latent footgun. region now str|None; when None →
_canonical_region (mortgage set sourced from _CBR_SERIES_MAP, single source of
truth). Explicit region always wins → existing callers unchanged. Region-binding
tests added. 159 tests pass.
Documented (NOT code-fixed — upstream data-quality): mortgage_debt /
mortgage_overdue stay empty in macro_indicator because cbr_mortgage_series.period
holds value-like garbage (e.g. '10054588.0') for the Debt series — corruption
from the upstream CBR-XLSX scraper that built domrf.db (imported verbatim by
44_import_anton_db.py), NOT an ingest bug. 123_macro_indicator.sql correctly
skips unparseable periods. Needs a separate scraper re-ingest (idempotent
backfill once period is valid). Refs #945
REOPENED. _SALES_WINDOW_SQL derived "sales in window" from objective_lots_history
snapshots, but history is only ~17 days deep — every currently-sold lot had a
sold-snapshot in the window, so window-sales collapsed into the entire cumulative
sold stock (Автовокзал 6mo: 33,245 vs real ~2,308). Inflated absorption_rate
(~235%/mo with confidence=high), months_of_supply, unit_velocity, liquidity,
demand_concentration → contaminated forecast #950/#952.
Count window sales directly from objective_lots by contract_date in the window
(the real sale date — present on 100% of sold lots: 41,091/41,091). Return
contract of _query_sales_window unchanged (units/area/by-room ROLLUP); downstream
formulas untouched. Removed the now-dead objective_lots_history JOIN/CTE.
Regression test: lots sold outside window (contract_date out of range) not counted
(41,091 cumulative vs 2,308 window → absorption 2.35→0.04). 288 tests green.
Verification = prod compute_market_metrics(Автовокзал) post-deploy. Refs #949
REOPENED. L3 future-supply rows are computed per (district_name, dev_group_name)
but dev_group_name was never a key column — only embedded in method text. With
complex_id/obj_class NULL for L3, every dev_group of a district collapsed to one
upsert key → ~95.6% loss. Ground-truth (Академический, prod): should be 13,808
units / 15 dev_groups / 54 objects; only 1 row / 607 units survived.
Migration 128: ADD COLUMN supply_layers.dev_group_name TEXT + rebuild
uq_supply_layers_logical to (layer, district_name, complex_id, obj_class,
dev_group_name, source, snapshot_date) NULLS NOT DISTINCT (L1/L2 dev_group_name
NULL stays transparent → their dedup unchanged; L3 distinct groups no longer
collapse). Dry-run-verified vs prod catalog (applies clean, ROLLBACK clean).
Worker: SupplyLayerRow gains dev_group_name (L1/L2=None, L3=group); _UPSERT_SQL
adds it to INSERT/VALUES (CAST(:dev_group_name AS text)) + ON CONFLICT (key col,
not in DO UPDATE SET). Service+worker regression tests assert same-district/
different-dev_group → distinct keys (no collapse). 234 supply tests pass.
Deploy applies migration before container restart; collapsed data self-heals on
next supply_layers_refresh. Verification = prod re-measure post-deploy.
Refs #970
Round 1 (commit bcd7dc8) был broken: на 2-bucket входах surplus уходил в free
полностью без учёта capacity → free превышал cap → следующая итерация
clamp'ировала его и наоборот. Infinite oscillation в FastAPI handler.
Round 2 fix per review BLOCK (#282 comment):
- Surplus распределяется пропорционально available capacity (cap - v),
не текущему v. Free никогда не вылетит выше cap.
- free = строго < cap (не <=) — иначе деление на 0 capacity.
- Hard guard `for _ in range(N+1)` — гарантированно завершается.
- Pathological (surplus > total_capacity): возвращаем оригинальный pct_map
+ cap_skipped=True (sum=100 invariant сохранён).
- Hamilton round вынесен в _hamilton_round() helper.
Tests:
- 2-bucket cases (90/10, 70/30, 99/1) expected cap_skipped=True
- test_cap_iteration_count_bounded — все pathological завершаются < 100ms
- All 13 cases verified standalone (3 fast-path + 7 reproduced + 3 pathological)
Раньше _VELOCITY_DIVISORS делил агрегаты mv_layout_velocity (24 мес)
на 4/12 для quarter/year, не меняя реальное окно данных. Теперь
inline SQL из objective_corpus_room_month с CAST(:window_interval AS interval).
velocity_per_month = deals_window / months_in_window (1.0/3.0/12.0).
Разные time_window → разные строки из БД → разный mix/velocity/jk_count.
Closes (epic part) #271 item 1
