Commit graph

10 commits

Author SHA1 Message Date
74f1ffb500 perf(objective): request-path consumers dedup inline, not via whole-table view (#1964)
All checks were successful
CI / changes (pull_request) Successful in 6s
CI / frontend-tests (pull_request) Has been skipped
CI / openapi-codegen-check (pull_request) Successful in 1m49s
CI / backend-tests (pull_request) Successful in 13m58s
deep-review #1964: v_objective_lots_latest has NO premise/district filter inside,
so a consumer's outer WHERE cannot push below DISTINCT ON → the view materializes
the WHOLE table (Parallel Seq Scan + external Sort 1.76M rows, ~55MB spill) on
every query. For REQUEST-PATH consumers inside analyze_parcel this is a ~19x latency
regression vs the pre-#1964 raw-table plan.

DISPROVEN remedy (NOT applied): a full index on the physflat-key does NOT help —
DISTINCT ON selects ol.* (51 cols, width≈945) so index-only-unique is impossible;
the planner ignores the index (seq-scan+sort still cheaper) and even forced it is
~3.9 s. A 142MB index for zero request-path benefit + slower bulk-INSERT during
objective-scrape is wrong. Honors original #1964 decision "no new index".

Prod EXPLAIN (Академический / 3km radius, 2026-06-28):
  consumer            via view      inline (this commit)
  concepts median     5854 ms       1640 ms   (bitmap district + sort)
  parcels district    5854 ms       1640 ms
  parcels geo-median  6443 ms        122 ms   (NestedLoop geo->complex bitmap)
  parcels obj_pricing 5721 ms        441 ms   (project bitmap per nearby ЖК)

FIX: keep v_objective_lots_latest ONLY for batch/background/cached consumers
(supply_layers L1, competitors._SOLD_COUNT_SQL, special_indices [/forecast bg task
30-180s], admin, landing). Revert the 4 request-path consumers inside analyze_parcel
to inline DISTINCT ON (physflat-key, latest snapshot) with the filter pushed INTO the
CTE so the district/spatial/project index applies:
- concepts._OBJECTIVE_MEDIAN_SQL
- parcels.py district price block
- parcels.py geo-radius median (complex_id-scoped)
- parcels.py obj_pricing CTE (project_name-scoped; aggregates over deduped set)

Migration 175 header CORRECTED: accurately states the partial mig-173 index does NOT
serve the view (qual can't push below DISTINCT ON), the full index is disproven/not
added, and which consumers use the view vs inline. No DDL change (still view-only).

Tests: +guards (concepts/obj_pricing dedup inline, not view; obj_pricing physflat
DISTINCT ON; perf-pushdown scope preserved). 965 passed.
2026-06-28 05:00:47 +05:00
e67cb721bf fix(objective): physflat-dedup current-state consumers + honest n_sold window (#1964)
objective_lots — current-state UPSERT (UNIQUE objective_lot_id, 5 snapshot_date),
но Объектив присваивает ОДНОМУ физлоту (project,corpus,section,floor,lot_number)
несколько lot_id за пере-листинги → таблица раздута ~2.91× (прод: 1.75M квартир-
строк vs 603k физлотов). История — в отдельной objective_lots_history (не трогаем).

STEP 1: миграция 175 — VIEW v_objective_lots_latest (DISTINCT ON physflat-ключ,
последний снапшот). ol.* стабилен (51==51 колонок). DRY-RUN на проде: 603 049
квартир vs 1 753 283 raw.

STEP 2: репойнт current-state консьюмеров objective_lots → v_objective_lots_latest:
- supply_layers._L1_OPEN_SQL (L1 открытое предложение → дефицит-форсайт; прод:
  Юго-Западный комфорт 58 606 → 12 620)
- competitors._SOLD_COUNT_SQL (+ комментарий: COUNT(DISTINCT lot_id) СОХРАНЁН для
  fan-out-защиты маппинга, не COUNT(*) — view гарантирует physflat-дедуп, DISTINCT
  гарантирует mapping-fan-out-safety)
- parcels.py obj_pricing CTE (карточка конкурента units_sold/available)
- special_indices._ARTIFICIAL_DEMAND_SQL
- parcels.py district price block + geo-radius median (sample_size/n)
- concepts._OBJECTIVE_MEDIAN_SQL (гейт n≥10)
- landing KPI3 (% квартир с ценой)
- admin_scrape coverage: `lots` оставлен сырым (ETL-fidelity vs SQLite), добавлен
  `lots_physflat`
#1959 inline-дедуп в market_metrics НЕ рефакторим — добавлен комментарий об общем
physflat-ключе с view.

STEP 3: report_assembler._deal_count теперь = unit_velocity × window_months
(оконные продажи), НЕ кумулятивный n_sold. confidence_engine помечает фактор
«за 6 мес» и гейтит порогами окна (high≥50) — кумулятив (прод EKB ~380 921) делал
гейт бессмысленным и подпись лживой; оконное (~24 876 за 6 мес) честно.

Тесты: +guard'ы (L1/sold-count читают view, deal_count оконный). 963 passed.
2026-06-28 04:27:43 +05:00
c55eb4f4d4 feat(concept): фронт-пикер типовых домов (#1965 Stage 3b)
All checks were successful
CI / changes (pull_request) Successful in 7s
CI / frontend-tests (pull_request) Successful in 1m1s
CI / openapi-codegen-check (pull_request) Successful in 1m55s
CI / backend-tests (pull_request) Successful in 14m9s
Финальная часть эпика #1953: пользователь выбирает типовые дома
(тип × этажность × число секций) вместо авто max-FAR раскладки, формируя
building_program из Stage 3a.

Бэкенд:
- GET /api/v1/concepts/house-types — read-only каталог HOUSE_TYPES
  (section_type, label_ru, footprint w×d + sqm, default_floors, housing_class)
  как single source of truth; фронт ничего не хардкодит.
- Схема HouseTypeCatalog / HouseTypeCatalogItem в schemas/concept.py.
- Тесты эндпоинта: полнота каталога + совпадение ключей с available_section_types.

Кодген: api-types.ts перегенерён (dump OpenAPI → openapi-typescript →
project-local prettier 3.9.0); 2-й прогон без диффа.

Фронтенд:
- useHouseTypes() (TanStack useQuery, staleTime Infinity) в concept-api.ts;
  building_program в ConceptInput, placed_count/requested_count в ConceptVariant.
- HouseProgramPicker: toggle «Авто (max-FAR)» (default, program omit → greedy)
  vs «Выбрать дома» (список каталожных типов, count 1-50 / floors 1-40, дефолт
  из каталога; габариты/этажность/класс как подсказка). Смонтирован в
  Section7Concept и на странице /concept.
- Partial-fit заметка в ConceptVariantsResult: при placed<requested честное
  «Разместилось N из M секций — участок вмещает меньше» (нейтрально, не ошибка).
2026-06-28 02:49:44 +05:00
94cf1f6217 feat(concept): house-type catalog + program-driven placement (#1965 Stage 3a)
Extend the concept generator so ConceptInput can carry an optional
building_program (list of typed houses from a catalog). When present,
placement lays out EXACTLY that program — for each item, place `count`
sections of the catalog footprint at the item's floors — instead of the
greedy max-FAR coverage-cap sweep. When absent, the existing greedy
behavior is unchanged (byte-for-byte backward-compatible).

- catalog.py: hardcoded HOUSE_TYPES (panel_econom, monolith_comfort,
  tower_business, lowrise_comfort, townhouse) — sane-default catalog,
  promote to DB later; get_house_type / available_section_types lookups.
- schema: additive BuildingProgramItem {section_type, floors, count} and
  ConceptInput.building_program (default None -> greedy). ConceptVariant
  gains optional placed_count / requested_count (partial-fit signal).
- placement: shared _Placer (collision/STRtree/setback machine extracted
  from greedy sweep, reused — no duplication); place_program +
  place_program_variant; branch in place_all_strategies on
  building_program. Mixed-floor TEAP via exact per-floor-group aggregation
  (GFA = sum(area_i * floors_i), no rounding drift).
- partial fit: when the parcel can't fit all sections, place as many as
  fit and report placed_count < requested_count (no hard-422); zero-fit
  still raises ParcelGeometryError (-> 422).
- API: validate program section_type keys against the catalog (unknown ->
  422) before placement.
- tests: catalog integrity, greedy backward-compat, exact 2-item program +
  TEAP reflection, over-packed partial placement, API program path.
- regenerate frontend api-types.ts (OpenAPI codegen gate stays green).
2026-06-28 01:29:04 +05:00
18d012da1b fix(generative): forward genuine price_source through /recompute (#1965 Stage 2a)
Some checks failed
CI / frontend-tests (pull_request) Has been skipped
CI / changes (pull_request) Successful in 7s
CI / openapi-codegen-check (pull_request) Failing after 1m49s
CI / backend-tests (pull_request) Successful in 13m49s
Code-review follow-up: /recompute hardcoded price_source="objective_district_median"
for any body-supplied market_price_per_sqm, mislabeling the honesty-flag once
Stage 2b forwards a price whose genuine source differs (objective_geo_radius /
district_reference / class_norm from financial_estimate).

- schemas/concept.py: add optional price_source: str | None to MassingProgram.
- api/v1/concepts.py: on FAST path use payload.price_source if provided, else the
  default label (now a module-level constant _DEFAULT_PRERESOLVED_SOURCE). DB-fallback
  and class-norm paths keep their own resolved source unchanged.
- tests: assert body-provided price_source echoes through to financial.price_source
  (not overwritten), and the default label applies when the front omits it.
2026-06-28 00:33:59 +05:00
d38d5d43c5 feat(generative): LIVE financial recompute from massing program (#1965 Stage 2a)
Stage 2a of epic #1953: backend service + endpoint for live economic recompute
driven by the interactive 3D massing (Stage 2b debounced sliders).

- teap.py: add pure synthesize_teap_from_program(total_footprint_sqm, floors,
  site_area_sqm, housing_class, sections) — builds a TEAP from the SCALAR
  aggregate footprint × floors, mirroring synthesize_teap_from_buildability and
  reusing the same shared norm constants (_OFFICE_SHARE_OF_GFA / _EFFICIENCY_BY_CLASS
  / _AVG_APARTMENT_SQM / _PARKING_PER_APARTMENT) — single source of truth.
- schemas/concept.py: add MassingProgram (program contract, optional pre-resolved
  market_price_per_sqm + parcel_centroid_wkt) and MassingRecomputeOutput (teap + financial).
- api/v1/concepts.py: add POST /api/v1/concepts/recompute — synthesize TEAP → run
  the existing pure compute_financial. FAST path uses body market_price_per_sqm
  (no DB); else _lookup_market_price by centroid via run_in_threadpool; else class norm.
- tests: synthesize_teap_from_program (gfa math, parity with compute_teap, class
  efficiency, sections no-op) + endpoint (200, coherent output, price passthrough
  skips DB, DB fallback, class-norm default, floors validation).
2026-06-28 00:27:25 +05:00
b1332d6b55 feat(generative): calibrate finmodel sale price from Objective market data (epic #1881 PR-2)
All checks were successful
CI / frontend-tests (pull_request) Successful in 1m3s
CI / changes (pull_request) Successful in 7s
CI / openapi-codegen-check (pull_request) Successful in 2m0s
CI / backend-tests (pull_request) Successful in 10m6s
Заменяет хардкод-цену продажи жилья (класс-норма) на реальную рыночную
медиану из Objective по локации участка. compute_financial остаётся ЧИСТЫМ
(без БД) — DB-lookup в API-слое, цена прокидывается параметром.

- compute_financial: +optional market_price_per_sqm/price_source; цена =
  рынок если есть, иначе класс-норма. Паркинг/СМР НЕ калибруем (только жильё).
- concepts.py _lookup_market_price(db, lon, lat): центроид → ближайший
  ekb_district (ST_DWithin 5км) → медиана objective_lots.price_per_m2_rub
  (n>=10, фильтр 30k-600k) → fallback ekb_districts.median_price_per_m2 →
  None. try/except → graceful (None, class_norm) при любой ошибке (вне ЕКБ/
  нет гео/SQL) без краха генерации. psycopg3 CAST.
- FinancialModel +price_per_sqm_used/price_is_calibrated/price_source (additive).
- Threading market_price через generate→placement→compute_financial (optional
  kwargs, backward-compat).
- UI/CSV: честный caption источника (рынок Objective / справочник района /
  норматив класса). Старый лживый footnote «не калиброванная модель» → условный.

Prod-verified: калибруется 4 главных ЕКБ-района по name-match (Академический
204k лотов, Ленинский 38k, Кировский, Орджоникидзевский); остальные 5 admin-
районов честно → district_reference. Гео-радиус matching (полное покрытие) —
follow-up.

api-types.ts регенерён авторитетно. mypy strict clean (generative.*), +14
тестов (калибровка/lookup 4 ветки/SQL-ошибки graceful/backward-compat).

Refs #1881
2026-06-23 21:10:25 +05:00
8279b0ee1f fix(week-review): cross-file добивка код-ревью (#1338, #1363, #1419, #1421, #1424, #1480)
- #1338 concepts.py: geometry.generate вынесен в run_in_threadpool (не блокирует event loop)
- #1363 house_imv_backfill.py + scrape_pipeline.py: периодический heartbeat в долгих IMV-фазах (reap_zombies не помечает живой sweep zombie)
- #1419 site-finder-api.ts: placeholderData keepPreviousData для bbox-запроса (KPI не мигает «0/0» при пане/зуме)
- #1421 site-finder-api.ts: area_ha nullable + адаптер не подставляет 0; null-гейт потребителей EntryMap.tsx, ParcelDrawer.tsx
- #1424/#1480 EstimateResult.tsx: блок «Параметры объекта» скрыт при восстановлении по shared-ссылке (нет мусора «0 м²/Студия/Этаж 0 из 0»)

Верификация: tsc --noEmit 0 ошибок; py_compile OK.
2026-06-15 20:29:00 +05:00
3b79533f9f feat(generative): движок концепций Stage 1a/1b/1c (#54 #55 #56)
Some checks failed
CI / changes (push) Successful in 7s
CI / frontend-tests (push) Has been skipped
CI / backend-tests (push) Has been cancelled
Заменяет generative stubs детерминированным end-to-end пайплайном:
- 1a geometry: WGS84-parse, метрическая AEQD-репроекция, setback-буфер, placement grid
- 1b placement: greedy section-fill + STRtree коллизии (full-gap), 3 стратегии, coverage cap
- 1c teap/financial: площади/квартиры/FAR/паркинг; выручка/затраты/маржа + IRR-proxy
- exporters: ezdxf DXF (геометрия) + WeasyPrint ТЭП/фин PDF (lazy import)
generate() заменил generate_stub в route (422 на degenerate). mypy-strict + ruff clean,
31 generative-тест + full suite 3078 passed. ConceptVariant заполняется реальными числами.

Closes #54
Closes #55
Closes #56
2026-06-13 21:32:32 +05:00
lekss361
b95559eac9 init 2026-04-25 13:45:19 +03:00