_compute_confidence docstring claimed "stub до G1/G2/D1/D2" — те эшелоны уже
отгружены, формулировка устарела. Это намеренно отдельная метрика: надёжность
site-finder скоринга (coverage POI/район/рынок), НЕ форсайтный §15
confidence_engine (надёжность прогноза спроса/цены над форсайт-входами,
которых в analyze hot-path нет). Поведение не меняется — резолв документацией.
Closes#1668
Replace bare metro_block placeholder ({"nearest_top3": None}, blocked on a
never-merged 22h metro scraper) with a direct KNN query against osm_poi_ekb
category='metro_stop' (loaded by poi_loader). Returns 3 nearest stations with
name + rounded distance_m; empty list when none nearby (vs None on error),
SAVEPOINT + try/except like adjacent SF-B5 blocks. No frontend consumer yet.
Closes#1667
API отвергал ?horizon=24 (422), хотя ТЗ §12.1 называет 6/12/18/24, а движок
УЖЕ считает 24 на каждом ране: _DEFAULT_HORIZONS=(6,12,18,24) во всех 6 точках
стека (orchestrator/forecast-task/demand_supply_forecast/scenarios/
special_indices/report_assembler), PIPELINE_HORIZON_MONTHS=24.
_hidden_release_fraction клампит h/18→1.0 на 24 (без переполнения),
future_supply._horizon_weight расширяет окно чисто — скрытых ≤18 потолков нет.
Чистое расширение валидатора-enum, не новая математика.
Backend: _ALLOWED_FORECAST_HORIZONS → {6,12,18,24}, Query/docstring/error-msg.
Frontend: HorizonSelector HORIZONS=[6,12,18,24] (тип horizon=number, union не нужен;
прочие потребители data-driven через meta.horizons/forecasts_by_horizon).
Тесты: API принимает 24/отвергает 30; движок-тесты доказывают h=24 осмыслен
(поля посчитаны, demand(24)>demand(18), hidden созрел, индексы в диапазонах).
Closes#944 (Q1 горизонт 24)
`cad_parcels_geom` — VIEW над `cad_parcels` где `geom` nullable (~964 prod-
строк с NULL geom задокументировано в analyze:1504-1507). Без фильтра
`geom IS NOT NULL` в UNION-ветках:
- NULL-строка из ранней ветки (`cad_quarters_geom`) ПЕРЕКРЫВАЕТ валидную
геометрию из поздней (`cad_parcels_geom`) — LIMIT 1 берёт первую попавшуюся.
- ST_Centroid(NULL) → NULL → coord_row.lat = NULL → float(None) → TypeError
→ 500 (вместо честного 404).
Guard `if not coord_row` НЕ ловил эту дыру: RowMapping с None-полями
truthy. Зеркало паттерна из analyze (parcels.py:1418-1438) для обоих
эндпоинтов (`/isochrones` + `/poi-score`):
1. `AND geom IS NOT NULL` во ВСЕХ ветках UNION.
2. Belt-and-suspenders: явная проверка `coord_row.get("lat") is None`
на случай топологически битого polygon'а (NOT-NULL geom, NULL centroid).
26 analyze/isochrones/poi-score юнит-тестов зелёные. ruff clean.
Closes#1201
PR #1195 ввёл CTE alias `overlaps` (PG keyword для time-period operator)
→ каждый POST /analyze падал с syntax error ~10ч до hotfix PR #1197. Mock-БД
в tests/api/v1/test_*.py не поймала: MagicMock на db.execute() не парсит SQL.
Расширяет существующий phantom-column gate (tests/integration/) на
_neighbors_summary и parcel_ird_overlaps SQL:
1. _NEIGHBORS_SUMMARY_SQL вынесен из inline-text в module-level constant
в parcels.py (паттерн как в best_layouts/ird_overlay_lookup/krt_lookup).
Заодно сконвертирован на canonical CAST(:wkt AS text) — backend.md rule.
2. EXPLAIN-тесты через phantom_check_session (SSH-туннель к prod-PG) —
skip без TEST_DATABASE_URL, run в спец CI job.
3. Compile-time guards (всегда работают, без БД):
- CTE-alias не должен быть PG-keyword (overlaps/user/current_date/select/where)
- SQL не должен содержать :bind::type (psycopg v3 antipattern)
Поймал бы:
- #1195 (CTE alias overlaps) — оба EXPLAIN-тест + compile-time keyword guard
- ❌:type recurring class — compile-time parametrized guard
- Phantom column / typo PostGIS function — EXPLAIN parses + plans
Refactor backward-compatible: _neighbors_summary использует ту же константу,
behavior идентичен. 26 unit-тестов parcels analyze продолжают зелёные.
Refs #1198
`OVERLAPS` — это PostgreSQL keyword (binary operator для time-periods —
`(start1,end1) OVERLAPS (start2,end2)`). Парсер ругается на `overlaps AS (...)`
в WITH-блоке с
ERROR: syntax error at or near "overlaps"
→ exception → транзакция в aborted state → весь analyze пайплайн ниже
получает InFailedSqlTransaction (ird_overlays, persist, ird block) → фронт
видит data_available=False, neighbors=[]. Воспроизводится на КАЖДОМ
/analyze с момента мерджа PR #1195.
Mock-БД в pytest tests/api/v1/ синтаксис не проверяет — отсюда регрессия
прошла CI. Прямой psql-replay на проде:
WITH neighbors AS (...), overlaps AS (...) → syntax error
WITH neighbors AS (...), overlap_rows AS (...) → ok
Минимальная переименование CTE alias (4 строки: 2 в SQL + 1 в SELECT
column alias + 1 в Python `row["overlap_rows"]`). Семантика идентична PR #1195.
Memory: future-proofing — добавить интеграционный тест с реальной PG для
analyze hot-path (отдельный issue, не блокер этого hotfix).
Refs #1196
Phase B продолжает hot-path latency cut'ы analyze_parcel поверх Phase A (PR #1194):
1. get_air_quality_cached в weather_cache (третий TTL-слот рядом с forecast/climate):
- hit-TTL 1h (hourly bucket Open-Meteo), negative-TTL 5min.
- timeout 5s → 2s.
- single-flight под per-cache threading.Lock.
- Снимает 0.26с на cache-hit и до 5с при DNS-fail (тот же антипаттерн,
что был у forecast/climate до Phase A).
2. _neighbors_summary: 2 SQL → 1 statement.
- WITH neighbors AS (... LIMIT 30), overlaps AS (... LIMIT 5)
- + COALESCE(json_agg(row_to_json(...)), '[]'::json) в основном SELECT.
- Один round-trip вместо двух (~-47 ms на каждый analyze).
- Семантика идентична: те же ST_DWithin(geom, point, 100m) для neighbors,
ST_Intersects + ST_Area для overlaps. Anti-fakes guards (_COST_PER_M2_MIN/MAX,
float() try/except, >50 m² overlap) сохранены.
Формат возвращаемых dict не меняется (frontend контракт). 5 новых юнит-тестов в
test_weather_cache.py (TestAirQualityCache: hot-hit, negative, empty-hourly,
separate-from-weather, ttl-expires). 7 файлов API-тестов обновили patch-цель
_fetch_air_quality_sync → get_air_quality_cached.
Refs #1130
Два внешних HTTP-вызова в analyze (_fetch_weather_sync + _fetch_seasonal_weather_sync)
сидели без кэша. На проде в private network с restricted egress DNS до *.open-meteo.com
фейлит и каждый replay висит полный timeout (5s + 15s = до 20с лишних на analyze).
Профиль cProfile одного replay: 2.38с в httpx + 1.73с в SSL recv.
Вынес логику в app/services/weather_cache.py:
- TTL hit: forecast 6h, climate normals 7d. Negative-cache: 5min (DNS-fail не повторяет
timeout на каждый analyze, восстановление подхватывается через ~5 минут).
- Ключ — округлённые до 0.01° (~1 км) координаты.
- Single-flight: per-cache threading.Lock + check-then-fetch-then-store.
- Тайм-ауты сокращены: forecast 5s→2s, climate 15s→3s.
Контракт для caller'а не изменился: None по-прежнему допустим (env_ok-флаг в
_build_environmental_score). 7 файлов API-тестов обновили patch-цели на новые имена
get_weather_cached / get_seasonal_weather_cached. 11 новых юнит-тестов в
tests/services/test_weather_cache.py (single-flight через threading.Barrier(16), TTL
expire через monkeypatch _now, изоляция forecast vs climate).
Refs #1130
render_report_docx (python-docx) mirrors report_md/report_pdf section order &
content, reuses report_pdf pure helpers (DRY), graceful on thin/empty report.
Widen /forecast/export format Literal to include docx → Word attachment.
Add python-docx dep + regenerate uv.lock (uv sync --frozen passes). Part of #959.
render_report_telegram_summary (pure, no new deps, DRY-reuses report_pdf str
helpers) + `tg` format on GET /{cad}/forecast/export → inline text/plain snippet
(no attachment, copy-paste-ready). md/json unchanged; no-run 404, bad format 422.
Graceful on thin/empty reports. Part of EPIC #959.
render_report_markdown (pure, no new runtime deps) reuses report_pdf's str
helpers (DRY), + GET /{cad_num}/forecast/export?format=md|json. No forecast run
→ 404; graceful on thin/empty reports; GFM-safe table escaping. PDF/XLSX already
existed; this adds the cheapest no-dep formats. Part of EPIC #959.
#992: attach response_model=AnalyzeResponse to POST /{cad_num}/analyze. Model
uses extra="allow" so any result_payload key not explicitly modeled is preserved
in the 200 response (no silent drops that would break Site Finder), and ALL
fields are Optional so the #93 202 fetch-stub serializes without a 500.
#994: add GET /{cad_num}/runs (light summary list, empty 200 not 404) and
GET /runs/{run_id} (full row incl result, 404 if missing), backed by
list_runs_for/get_run in analysis_runs.repository (psycopg v3 CAST(:x AS type)).
Routes ordered before /{parcel_id} so /runs/{run_id} is not shadowed.
Closes#992. Closes#994. Refs #961.
Add nullable lat/lon (EPSG:4326, 6 dp) to /analyze competitors[] and
pipeline_24mo.top_objects[] so the frontend can plot Leaflet markers. Coords
come from domrf_kn_objects.latitude/longitude (same source as distance_m).
Purely additive: no existing field/shape changed. Frontend map layers follow
in a separate PR. Part of EPIC #958 (958-B4).
POST /analyze 500'd (TypeError: float(None)) for ~964 cad_parcels_geom rows
that have a meta row but geom IS NULL: the resolution query returned the row
(no NULL filter), geom_wkt became NULL → ST_Centroid(NULL) → NULL lat/lon →
float(centroid_row["lat"]) crashed (the `if centroid_row else` guard checked
row existence, not NULL values). A user entering any such "random parcel" hit
a 500.
Fix:
- Add `AND geom IS NOT NULL` to all three geometry-resolution queries (initial
UNION, inline-refetch UNION, geom_row WKT subquery) so a NULL-geometry parcel
is treated as "no usable geometry" → falls into the #93 graceful fallback
(enqueue NSPD on-demand fetch → 202 polling), exactly like a parcel absent
from the DB. The fetch then populates real geometry and analyze succeeds.
- Defensive: centroid_lat/lon now check the VALUE is not None (not just row
existence) before float(), falling back to EKB center.
Verified: badformat→400, not-in-DB→202(fetching), valid parcel→200 unaffected.
39 analyze/by-bbox tests green. Refs #944.
LIVE BUG (from #994, already merged): the analysis_runs persist call-site in
analyze_parcel extracted district via result_payload["district"]["district_name"]
— but a district dict without that key (real data: partial district lookup)
raised KeyError OUTSIDE the best-effort SAVEPOINT (extraction is at the call
site, not inside persist_analysis_run) → 500 on the LIVE /analyze endpoint.
Fix: .get("district_name") → None instead of raising. Caught by reviving the
analyze test suite (below).
Test-infra (the suite was un-runnable, which is why the bug shipped):
- Lazy-import WeasyPrint in layout_tz_pdf.py + trade_in_pdf.py (matches the
existing report_pdf.py / snapshot_pdf.py pattern). The eager top-level imports
made `app.main` (→ parcels/trade_in routers) un-importable on hosts without
WeasyPrint native libs (e.g. macOS dev), breaking pytest COLLECTION of the
whole api/v1 suite. WeasyPrint is still imported when a PDF is actually rendered.
- tests/conftest.py: autouse fixture clears app.dependency_overrides after each
test (anti-leak — a leaked get_db override caused real-DB connection attempts).
- test_parcel_by_bbox.py: rewrite get_db mocking from patch("...get_db") (a no-op
— FastAPI Depends holds the original ref) to app.dependency_overrides[get_db],
add explicit X-Authenticated-User header (RBAC gate), patch latest_run_dates,
+ a new test asserting last_analysis_date from a latest run (#994). 5/5 green.
NOTE: reviving collectability exposes PRE-EXISTING rot in other api/v1 suites
(analyze/admin/best_layouts: RBAC-header + stale-assertion/mock drift) — those
are NOT regressions from this PR (they were uncollectable before) and are
tracked separately for a deliberate suite-rehab + CI test-gate effort.
Refs #994.
Persist each successful POST /analyze into analysis_runs (migration 127):
best-effort SAVEPOINT-wrapped INSERT in a thin analysis_runs repository, then
explicit db.commit() (get_db has no commit-on-success; SAVEPOINT RELEASE alone
does not persist — without the commit the row rolls back on request teardown,
a silent no-op caught in code review). A persist failure never breaks the
response or poisons the session. Result serialized via jsonable_encoder→
json.dumps (handles embedded Pydantic models/dates); confidence/status
normalized to satisfy the 127 CHECK constraints (unknown→NULL/'complete').
Populate by-bbox last_analysis_date from v_analysis_runs_latest via a single
batch query (no N+1), replacing the #307 placeholder. The read is best-effort
wrapped too — a view-missing deploy window or future drift must not 500 the
map (falls back to last_analysis_date=None).
Additive only — analyze response shape unchanged. Tests: by-bbox suite patches
latest_run_dates (mock-db returns same rows for any query) + new test asserts
ISO last_analysis_date from a latest run. Analyze mock-suite unaffected
(graceful side_effect overflow + best-effort persist absorb the extra INSERT).
Incidental: ruff-format fixed one pre-existing f-string spacing (line ~2391)
the format hook flags on touch.
Closes#994. Refs #961.
When Objective mapping coverage falls below 50% of competitors in the radius,
fall back to rosreestr_deals JOIN on the parcel's cadastral quarter. Audit shows
237/237 EKB quarters (100%) have rosreestr data for the last 12 months, compared
to <20% Objective coverage before bulk mapping.
- velocity.py: add _compute_rosreestr_fallback(), _OBJECTIVE_COVERAGE_MIN_RATIO
constant, velocity_source field on VelocityResult (objective/rosreestr_fallback/none)
- parcels.py: extract cad_quarter from cad_num, pass to compute_velocity
- site-finder.ts: add velocity_source field to Velocity interface
- VelocityBlock.tsx: badge "Источник: квартальные сделки" when rosreestr_fallback
Epic #271 item #17
- v_bucket_success_score HAVING >= 30 → >= 15 so sparse districts
(Кировский etc.) are no longer silently excluded from the block
- Backend: SUCCESS_REC_MIN_DEALS=15, SUCCESS_REC_STRONG_DEALS=30;
data_confidence='weak' (15-29 deals) or 'strong' (≥30) in response
- TS type ParcelSuccessRecommendation: add data_confidence literal
- SuccessRecommendationBlock: amber badge for weak, all hex → CSS tokens,
emoji star removed (ui-ux.md), empty-state text min 30→15
ekb_districts.median_price_per_m2 was populated from a 24-month window.
LEFT JOIN mv_quarter_price_per_m2 on the cad quarter and COALESCE(median_12m,
fallback) so the API returns the 12-month median where available.
Frontend OverviewTab adds a "(12 мес)" hint next to the value.
Closes#271 item 7
TASK A (#29 G2): add parcel_meta to analyze response
- New ParcelMeta Pydantic schema in app/schemas/parcel.py
- SELECT from cad_parcels WHERE cad_num=:c in analyze_parcel() (step 9f)
- Returns permitted_use_established_by_document, land_record_category_type,
land_record_subtype, cost_value; None when row absent
- Tests: test_analyze_parcel_meta.py (found + not-found cases)
TASK B (#232 G3): cad_zouit fallback in _get_zouit_overlaps
- When nspd_quarter_dumps has zouit_count==0, fall back to ST_Intersects
query on cad_zouit (3483 rows, GIST indexed)
- Overlaps tagged with source='cad_zouit'; format compatible with NSPD path
- gate_verdict.py: BLOCKER_TYPE_ZONE_KEYWORDS tuple for keyword-based
classification (охранная зона / трубопровод / электр / газ -> blocker;
СЗЗ -> warning); NSPD subcategory path preserved backward-compat
- Tests: 6 new test cases in test_gate_verdict.py covering cad_zouit path
and backward-compat for NSPD subcategory path
Updated db.execute call sequence in test_analyze_*.py (index shift +1 at pos 10).
