Commit graph

440 commits

Author SHA1 Message Date
9dbfb03a3d fix(scrapers): disambiguate EMISS yearly vs Q1 in dedup key (#1606)
Some checks failed
CI / changes (push) Successful in 7s
CI / changes (pull_request) Successful in 7s
CI / backend-tests (push) Has been skipped
CI / frontend-tests (push) Has been skipped
CI / openapi-codegen-check (push) Has been skipped
CI / frontend-tests (pull_request) Has been skipped
CI / openapi-codegen-check (pull_request) Successful in 1m38s
CI / backend-tests (pull_request) Failing after 8m41s
_YEAR_PERIODS ('год') → month=1 collided with Q1 ('I квартал') → month=1
for the same region/year: both produced obs_date=YYYY-01-01, so the second
series in the XML silently overwrote the first in the in-memory by_key dict.

Fix: add _emiss_period_granularity() and extend the dedup key from
(region, obs_date) to (region, obs_date, granularity). 'year' and 'quarter'
are now distinct slots, so both rows survive the parser and reach the upsert.

Test: test_yearly_and_q1_both_survive_dedup verifies len==2 with both
values present; test_period_granularity covers all classification branches.
2026-06-17 20:39:52 +03:00
20efc3bf64 merge: resolve conflicts with main (#1660) into week-review-finish
Some checks failed
CI / backend-tests (pull_request) Failing after 9m11s
CI / backend-tests (push) Failing after 9m22s
CI / openapi-codegen-check (push) Successful in 2m14s
CI / changes (pull_request) Successful in 6s
CI / changes (push) Successful in 8s
CI / frontend-tests (pull_request) Successful in 1m13s
CI / frontend-tests (push) Successful in 1m2s
CI / openapi-codegen-check (pull_request) Successful in 1m47s
5 conflicts resolved:
- redaction.py: merged INN checksum + _PHONE_LOCAL_RE + _Repl alias from main
- recommendation.py: kept commercial_sell_through_pct rename (#1635)
- report_pdf.py: kept _scenario_deficit_cell + int|None return type (#1590)
- parcel.py: kept main's detailed docstring (functionally identical)
- alembic/env.py: kept HEAD (app.models import in __init__ already covers all)
2026-06-17 20:21:06 +03:00
14f3ef2019 fix(week-review): backend-аудит v2 — 82 фиксов (#1660)
All checks were successful
Deploy / build-worker (push) Successful in 2m47s
Deploy / deploy (push) Successful in 1m20s
Deploy / changes (push) Successful in 9s
Deploy / build-frontend (push) Has been skipped
Deploy / build-backend (push) Successful in 1m52s
Co-authored-by: bot-backend <bot-backend@gendsgn.local>
Co-committed-by: bot-backend <bot-backend@gendsgn.local>
2026-06-17 17:13:38 +00:00
50db2b82aa Merge pull request 'fix(site-finder): nearest_top3 metro from osm_poi_ekb (#1667)' (#1670) from fix/1667-metro-block into main
Some checks failed
Deploy / build-backend (push) Has been cancelled
Deploy / build-worker (push) Has been cancelled
Deploy / build-frontend (push) Has been cancelled
Deploy / changes (push) Successful in 9s
Deploy / deploy (push) Has been cancelled
2026-06-17 17:13:17 +00:00
e4b80938a0 Merge pull request 'docs(site-finder): analyze confidence by-design distinct from §15 (#1668)' (#1671) from fix/1668-analyze-confidence into main
Some checks are pending
Deploy / changes (push) Waiting to run
Deploy / build-backend (push) Blocked by required conditions
Deploy / build-worker (push) Blocked by required conditions
Deploy / build-frontend (push) Blocked by required conditions
Deploy / deploy (push) Blocked by required conditions
2026-06-17 17:13:07 +00:00
184e429526 Merge pull request 'fix(workers): schedule mv_layout_velocity refresh (#1666)' (#1669) from fix/1666-layout-velocity-beat into main
Some checks are pending
Deploy / changes (push) Waiting to run
Deploy / build-backend (push) Blocked by required conditions
Deploy / build-worker (push) Blocked by required conditions
Deploy / build-frontend (push) Blocked by required conditions
Deploy / deploy (push) Blocked by required conditions
2026-06-17 17:12:54 +00:00
829f046852 feat(scripts): Potrace floor-plan vectorization spike (#299) (#1663)
Some checks failed
Deploy / build-backend (push) Blocked by required conditions
Deploy / build-worker (push) Blocked by required conditions
Deploy / build-frontend (push) Blocked by required conditions
Deploy / deploy (push) Blocked by required conditions
Deploy / changes (push) Has been cancelled
Co-authored-by: bot-backend <bot-backend@gendsgn.local>
Co-committed-by: bot-backend <bot-backend@gendsgn.local>
2026-06-17 17:12:48 +00:00
d3a53ae475 docs(site-finder): clarify analyze confidence is by-design distinct from §15 (#1668)
All checks were successful
CI / changes (push) Successful in 8s
CI / changes (pull_request) Successful in 6s
CI / frontend-tests (push) Has been skipped
CI / openapi-codegen-check (push) Successful in 1m38s
CI / frontend-tests (pull_request) Has been skipped
CI / openapi-codegen-check (pull_request) Successful in 1m40s
CI / backend-tests (push) Successful in 8m42s
CI / backend-tests (pull_request) Successful in 8m40s
_compute_confidence docstring claimed "stub до G1/G2/D1/D2" — те эшелоны уже
отгружены, формулировка устарела. Это намеренно отдельная метрика: надёжность
site-finder скоринга (coverage POI/район/рынок), НЕ форсайтный §15
confidence_engine (надёжность прогноза спроса/цены над форсайт-входами,
которых в analyze hot-path нет). Поведение не меняется — резолв документацией.

Closes #1668
2026-06-17 18:47:07 +05:00
7072d7803e fix(site-finder): compute nearest_top3 metro from osm_poi_ekb (#1667)
All checks were successful
CI / frontend-tests (push) Has been skipped
CI / openapi-codegen-check (push) Successful in 1m39s
CI / backend-tests (pull_request) Successful in 8m41s
CI / backend-tests (push) Successful in 8m40s
CI / frontend-tests (pull_request) Has been skipped
CI / changes (pull_request) Successful in 7s
CI / changes (push) Successful in 7s
CI / openapi-codegen-check (pull_request) Successful in 1m41s
Replace bare metro_block placeholder ({"nearest_top3": None}, blocked on a
never-merged 22h metro scraper) with a direct KNN query against osm_poi_ekb
category='metro_stop' (loaded by poi_loader). Returns 3 nearest stations with
name + rounded distance_m; empty list when none nearby (vs None on error),
SAVEPOINT + try/except like adjacent SF-B5 blocks. No frontend consumer yet.

Closes #1667
2026-06-17 18:44:29 +05:00
7276edc885 fix(workers): schedule mv_layout_velocity refresh in Celery beat (#1666)
All checks were successful
CI / backend-tests (push) Successful in 8m49s
CI / backend-tests (pull_request) Successful in 8m52s
CI / changes (push) Successful in 6s
CI / frontend-tests (push) Has been skipped
CI / changes (pull_request) Successful in 8s
CI / frontend-tests (pull_request) Has been skipped
CI / openapi-codegen-check (push) Successful in 1m57s
CI / openapi-codegen-check (pull_request) Successful in 1m55s
mv_layout_velocity (powers best_layouts, #113) refreshed only via manual
refresh_layout_velocity() → prod data silently went stale. Add Celery task
mirroring refresh_quarter_price_index, register in celery_app include, add
weekly beat entry (Sun 03:00 MSK, outside the monday heavy-refresh cluster).

Closes #1666
2026-06-17 18:40:01 +05:00
764db61734 fix(exporters): annotate scenario deficit horizon in docx/pptx (#1590 follow-up)
All checks were successful
CI / changes (pull_request) Successful in 7s
CI / changes (push) Successful in 7s
CI / frontend-tests (push) Successful in 1m1s
CI / frontend-tests (pull_request) Successful in 1m17s
CI / openapi-codegen-check (push) Successful in 2m19s
CI / backend-tests (pull_request) Successful in 9m11s
CI / backend-tests (push) Successful in 9m15s
CI / openapi-codegen-check (pull_request) Successful in 1m43s
report_docx и report_pptx рендерили bare _scenario_deficit_index под
захардкоженной шапкой «(12 мес)» — при fallback на другой горизонт подпись
лгала. Переключил оба на _scenario_deficit_cell (как в report_md): per-cell
аннотация «(гор. N мес)» при N != 12. Закрывает code-review major по #1590.
2026-06-17 14:35:48 +05:00
c9ba595abe refactor(forecasting): rename commercial_share_pct → commercial_sell_through_pct (#1635)
All checks were successful
CI / changes (pull_request) Successful in 6s
CI / frontend-tests (push) Successful in 1m40s
CI / openapi-codegen-check (push) Successful in 1m38s
CI / backend-tests (pull_request) Successful in 9m51s
CI / changes (push) Successful in 7s
CI / frontend-tests (pull_request) Successful in 53s
CI / openapi-codegen-check (pull_request) Successful in 3m10s
CI / backend-tests (push) Successful in 9m52s
Внутренний recommendation→product_scoring контракт-ключ был мислейблом: величина —
темп распродажи нежилого (sell-through, прокси ликвидности/спроса), а НЕ доля нежилого
в объёме застройки. Переименован ключ + исправлены reason/docstring/комментарии у
потребителя _score_commercial. Числовая логика не изменена. Ключ внутренний (нет
frontend/schema/openapi-потребителей) → rename контракт-безопасен. pytest 171 passed.

Closes #1635
2026-06-17 13:32:03 +05:00
249182b678 fix(week-review): дожим ревью — #1569, #1590, #1642 + feat #801 MSW-preview
All checks were successful
CI / frontend-tests (pull_request) Successful in 1m2s
CI / openapi-codegen-check (push) Successful in 2m13s
CI / openapi-codegen-check (pull_request) Successful in 1m45s
CI / backend-tests (push) Successful in 9m29s
CI / backend-tests (pull_request) Successful in 9m29s
CI / changes (push) Successful in 8s
CI / changes (pull_request) Successful in 8s
CI / frontend-tests (push) Successful in 1m7s
Финиш-волна по cross-file/partial остаткам аудита + a11y-харнесс.

Полностью (4): #1569 alembic регистрирует все ORM-модели (+убран фантомный Parcel) ·
#1590 «Индекс дефицита» больше не лжёт на fallback-горизонте (report_pdf+report_md) ·
#1642 thumbs mtime-freshness (нет устаревшей миниатюры) ·
#801 /__preview/estimate mock-render для axe/lighthouse (env-gated bypass).

Частично (остаток cross-file/домен, открыты): #1593 #1635 #1640.
needs-leha (открыт): #1650 (нужна новая миграция вьюхи supply-слоёв).

Verify: tsc --noEmit 0; py_compile OK.

Closes #1569
Closes #1590
Closes #1642
Closes #801
2026-06-17 11:24:34 +05:00
be70cacf18 test: выровнять weather_cache/nspd_geo тесты под #1377/#1336 (CI red→green)
All checks were successful
CI / changes (push) Successful in 7s
CI / changes (pull_request) Successful in 9s
CI / frontend-tests (push) Successful in 1m4s
CI / frontend-tests (pull_request) Successful in 57s
CI / openapi-codegen-check (push) Successful in 2m7s
CI / openapi-codegen-check (pull_request) Successful in 1m41s
CI / backend-tests (push) Successful in 8m59s
CI / backend-tests (pull_request) Successful in 8m58s
backend-tests CI упал на 4 стейл-тестах — код автофиксов обновили, тесты нет:
- test_weather_cache: фикстура _make_air_response отдавала hourly-форму, а код
  (#1377) теперь читает current-bucket → current.time пуст → None. Фикстура → current.
- test_nspd_geo: 2 теста стерегли ошибочный ST_Transform(3857→4326), который #1336
  убрал (геометрия уже WGS84, иначе Null Island). Ассерты через _strip_sql_comments
  (как сиблинг parcel-тест) → ST_SetSRID(...,4326) без ST_Transform. Переименованы.

Локально: 38 passed (оба файла).
2026-06-16 08:35:14 +03:00
479f01ae1a Merge remote-tracking branch 'forgejo/main' into HEAD
Some checks failed
CI / changes (push) Successful in 7s
CI / changes (pull_request) Successful in 8s
CI / frontend-tests (push) Successful in 1m7s
CI / openapi-codegen-check (push) Failing after 2m9s
CI / frontend-tests (pull_request) Successful in 1m4s
CI / openapi-codegen-check (pull_request) Failing after 1m37s
CI / backend-tests (pull_request) Failing after 9m7s
CI / backend-tests (push) Failing after 9m11s
# Conflicts:
#	backend/app/services/site_finder/gate_verdict.py
2026-06-15 21:07:05 +03:00
8279b0ee1f fix(week-review): cross-file добивка код-ревью (#1338, #1363, #1419, #1421, #1424, #1480)
- #1338 concepts.py: geometry.generate вынесен в run_in_threadpool (не блокирует event loop)
- #1363 house_imv_backfill.py + scrape_pipeline.py: периодический heartbeat в долгих IMV-фазах (reap_zombies не помечает живой sweep zombie)
- #1419 site-finder-api.ts: placeholderData keepPreviousData для bbox-запроса (KPI не мигает «0/0» при пане/зуме)
- #1421 site-finder-api.ts: area_ha nullable + адаптер не подставляет 0; null-гейт потребителей EntryMap.tsx, ParcelDrawer.tsx
- #1424/#1480 EstimateResult.tsx: блок «Параметры объекта» скрыт при восстановлении по shared-ссылке (нет мусора «0 м²/Студия/Этаж 0 из 0»)

Верификация: tsc --noEmit 0 ошибок; py_compile OK.
2026-06-15 20:29:00 +05:00
86e9ea2937 fix(week-review): автофиксы код-ревью — 169 issue (label «week ревью 1»)
Многоагентный аудит + имплементация: один воркер на файл, точечные правки.
Верификация: py_compile (47/47 .py) + tsc --noEmit (0 ошибок). Unit-тесты
не прогонялись (окружение не поднято: rollup native dep / нет pytest-venv).

Полностью исправлено (169): #1336, #1337, #1339, #1340, #1341, #1342, #1343, #1345, #1346, #1348, #1349, #1350, #1351, #1354, #1356, #1358, #1359, #1360, #1362, #1364, #1365, #1366, #1367, #1368, #1369, #1370, #1371, #1372, #1373, #1374, #1375, #1376, #1377, #1378, #1379, #1380, #1381, #1382, #1384, #1385, #1386, #1387, #1388, #1389, #1390, #1391, #1392, #1394, #1395, #1396, #1397, #1399, #1400, #1401, #1402, #1403, #1404, #1408, #1409, #1410, #1411, #1412, #1413, #1414, #1415, #1416, #1417, #1418, #1420, #1423, #1425, #1426, #1427, #1428, #1429, #1430, #1431, #1432, #1433, #1434, #1435, #1437, #1438, #1439, #1440, #1441, #1442, #1443, #1444, #1445, #1446, #1447, #1448, #1449, #1450, #1451, #1452, #1453, #1454, #1455, #1456, #1457, #1458, #1459, #1460, #1461, #1462, #1463, #1464, #1465, #1466, #1467, #1468, #1469, #1471, #1472, #1473, #1474, #1476, #1478, #1479, #1481, #1482, #1483, #1484, #1485, #1487, #1488, #1489, #1490, #1491, #1492, #1493, #1494, #1495, #1496, #1497, #1499, #1500, #1501, #1502, #1504, #1505, #1506, #1507, #1510, #1514, #1515, #1516, #1517, #1518, #1519, #1521, #1522, #1523, #1524, #1525, #1526, #1527, #1528, #1529, #1531, #1532, #1533, #1534, #1535, #1536, #1537, #1538

Частично (9, in-file часть, остаток cross-file): #1361, #1419, #1422, #1424, #1470, #1475, #1477, #1480, #1498
Требуют cross-file (3, не тронуты): #1338, #1363, #1421
Пропущено (1): #1539

Не входило в партию: 22 needs-Leha issue (нужны решения владельца).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-15 20:21:11 +05:00
cab19020a9 perf(site-finder): pushdown obj_pricing CTE to nearby competitors (#70/#307) (#1335)
All checks were successful
Deploy / changes (push) Successful in 5s
Deploy / build-frontend (push) Has been skipped
Deploy / build-backend (push) Successful in 1m40s
Deploy / build-worker (push) Successful in 2m38s
Deploy / deploy (push) Successful in 1m12s
2026-06-14 16:58:43 +00:00
b405ae291d feat(site-finder): additive objective_lots price fallback for competitors (#307 OBJ-3) (#1332)
All checks were successful
Deploy / changes (push) Successful in 7s
Deploy / build-backend (push) Successful in 1m57s
Deploy / build-worker (push) Successful in 2m51s
Deploy / build-frontend (push) Successful in 3m6s
Deploy / deploy (push) Successful in 1m12s
2026-06-14 16:10:02 +00:00
72e0b050f9 feat(site-finder): POI saturation per capita по районам ЕКБ (#42) (#1329)
All checks were successful
Deploy / changes (push) Successful in 6s
Deploy / build-frontend (push) Has been skipped
Deploy / build-backend (push) Successful in 1m38s
Deploy / build-worker (push) Successful in 2m52s
Deploy / deploy (push) Successful in 1m12s
2026-06-14 15:05:22 +00:00
e5e0ccb283 feat(site-finder): сетевые обременения из ЗОУИТ (#1070, Refs #103) (#1328)
All checks were successful
Deploy / changes (push) Successful in 6s
Deploy / build-frontend (push) Has been skipped
Deploy / build-backend (push) Successful in 1m36s
Deploy / build-worker (push) Successful in 2m40s
Deploy / deploy (push) Successful in 1m12s
2026-06-14 14:49:32 +00:00
5507bbc1a4 feat(site-finder): domrf→cad geom-match + parking_ratio конкурентов (#96) (#1327)
Some checks failed
Deploy / build-frontend (push) Blocked by required conditions
Deploy / deploy (push) Blocked by required conditions
Deploy / build-backend (push) Blocked by required conditions
Deploy / build-worker (push) Blocked by required conditions
Deploy / changes (push) Has been cancelled
2026-06-14 14:49:28 +00:00
2c22c3f7ea feat(site-finder): per-building помещения/машино-места + parking_ratio (on-demand MVP, #96)
All checks were successful
CI / frontend-tests (push) Has been skipped
CI / changes (pull_request) Successful in 6s
CI / changes (push) Successful in 8s
CI / frontend-tests (pull_request) Has been skipped
CI / openapi-codegen-check (push) Successful in 1m59s
CI / openapi-codegen-check (pull_request) Successful in 1m58s
CI / backend-tests (push) Successful in 8m48s
CI / backend-tests (pull_request) Successful in 8m48s
Deploy / changes (push) Successful in 6s
Deploy / build-frontend (push) Has been skipped
Deploy / build-backend (push) Successful in 4m58s
Deploy / build-worker (push) Successful in 5m46s
Deploy / deploy (push) Successful in 1m30s
Оживляет мёртвый foundation list_objects_in_building (#168 Q3-deferred). Fix реального
блокера: NSPDOptions не парсил objdocId (camelCase из NSPD search) → objdoc_id всегда None,
метод нельзя было вызвать. AliasChoices(objdoc_id, objdocId) + registers.

premises_lookup.get_building_premises(cad_num) — резолв objdoc → list_objects_in_building,
graceful (WAF/сеть/not-found → None). parking_ratio = машино-места/помещения (None при 0
помещений, 0.0 при реальном отсутствии паркинга). Verified live 66:41:0106036:183.

MVP on-demand (без bulk/схемы/prod-записей). parking_ratio готов, но НЕ wired в analyze:
конкуренты приходят из ДОМ.РФ без cad_num → нужен domrf↔cad_buildings geom-match (отдельная
задача). Сервис примет cad_num как только matching появится. 11 тестов, ruff clean.

Refs #96
2026-06-14 18:48:16 +05:00
eb251ba7e7 ci(infra): coverage-gate (#68) + OpenAPI codegen-assert (#69) + uptime monitoring (#75)
All checks were successful
CI / changes (push) Successful in 13s
CI / changes (pull_request) Successful in 12s
CI / frontend-tests (push) Successful in 53s
CI / openapi-codegen-check (push) Successful in 1m58s
CI / frontend-tests (pull_request) Successful in 47s
CI / openapi-codegen-check (pull_request) Successful in 1m50s
CI / backend-tests (push) Successful in 8m54s
CI / backend-tests (pull_request) Successful in 8m45s
Deploy / changes (push) Successful in 6s
Deploy / build-backend (push) Successful in 2m50s
Deploy / build-frontend (push) Successful in 3m11s
Deploy / build-worker (push) Successful in 4m38s
Deploy / deploy (push) Successful in 2m1s
#68: pytest-cov + [tool.coverage] fail_under=65 (baseline 71%), backend-tests --cov + xml.
#69: openapi-codegen-check job (dump app.openapi() → openapi-typescript → git diff).
  Job использует node_modules-pinned openapi-typescript/prettier (НЕ npx --yes latest —
  иначе version-mismatch ложный diff). + regenerated api-types.ts из СВЕЖЕГО main openapi
  (8288 строк, включая #73 /freshness — синхронен).
#75: Uptime Kuma isolated stack + status.gendsgn.ru Caddy + external Telegram watchdog.
#77: no change — build cache достаточен.

Closes #68
Closes #69
Closes #75
Closes #77
2026-06-13 23:28:06 +05:00
d38417d8af feat(admin): freshness endpoint (#73) + README Forgejo refresh (#83) + migrations doc (#72)
All checks were successful
Deploy / build-frontend (push) Has been skipped
Deploy / changes (push) Successful in 5s
Deploy / build-backend (push) Successful in 2m39s
Deploy / build-worker (push) Successful in 7m20s
Deploy / deploy (push) Successful in 2m48s
#73: GET /admin/scrape/freshness — last_success/age/freshness/status для 5 источников
(kn/objective/nspd/nspd_geo/cadastre) + overall_status (worst по critical). Prod-verified
имена таблиц, FILTER/COALESCE корректны, CAST(:d AS interval). +4 теста.
#83: README — GitHub→Forgejo CI/CD, секция 6 subagents, ссылки перемаплены.
#72: docs/Schema_Migrations_vs_Alembic.md — _schema_migrations паттерн (не Alembic),
рекомендация снести пустой alembic-скелет (отдельный PR).

Closes #73
Closes #83
Closes #72
2026-06-13 18:25:32 +00:00
50555821fb feat(site_finder): routing POI decay (#41) + OSM utility distances (#44) + ЗОУИТ geom (#255bk)
Some checks failed
CI / changes (push) Successful in 7s
CI / frontend-tests (push) Has been skipped
CI / changes (pull_request) Successful in 7s
CI / frontend-tests (pull_request) Has been skipped
CI / backend-tests (push) Successful in 7m42s
CI / backend-tests (pull_request) Successful in 7m40s
Deploy / changes (push) Has been cancelled
Deploy / build-backend (push) Has been cancelled
Deploy / build-worker (push) Has been cancelled
Deploy / build-frontend (push) Has been cancelled
Deploy / deploy (push) Has been cancelled
#41: per-category routing-radius decay через reusable ORS /matrix client (foot-walk
время + per-category пороги + decay-curves), opt-in ?decay=routing, graceful straight-line
fallback (analyze не 500 при ORS down) + length-guard durations==destinations.
#44: +5 OSM-категорий (transformer/gas/water/sewerage/heat) via Overpass nwr;
nearest_{water_main,substation,gas,heat}_m в analyze.utilities.
#255bk: ST_AsGeoJSON(CAST(geom AS geometry)) в _get_cad_zouit_overlaps → geom_geojson.

Closes #41
Closes #44
2026-06-13 22:31:56 +05:00
658d724075 feat(analytics): velocity-anomaly alerts (#17) + ARN ДДУ price indicator (#99)
Some checks failed
CI / changes (push) Successful in 8s
CI / frontend-tests (push) Has been skipped
CI / changes (pull_request) Successful in 8s
CI / frontend-tests (pull_request) Has been skipped
CI / backend-tests (push) Successful in 7m37s
CI / backend-tests (pull_request) Successful in 7m42s
Deploy / deploy (push) Blocked by required conditions
Deploy / build-backend (push) Blocked by required conditions
Deploy / build-worker (push) Blocked by required conditions
Deploy / build-frontend (push) Blocked by required conditions
Deploy / changes (push) Has been cancelled
#17: detect_velocity_anomalies + GET /analytics/velocity-alerts — z-score drop-detection
на domrf_kn_sale_graph (double-gate z≤-2 AND drop≤-30%, starvation-guards). Snapshot=richest
(не MAX — prod новейшие частичные), lookback anchored на latest report_month (scrape лаг ~4мес).
Prod: ЖК Центральный Парк -69%, ~14ms.

#99: mv_ddu_price_indicator (миграция 152) + POST /market/ddu-indicator — ARN-mirror
ценовой индекс per quarter×area_bucket из rosreestr_deals (ДДУ регион 66). MVP: subject-level,
period Q, window 2025-Q2+, methods 1/2 (basis/previous, prev_period_value honesty). Q1-2026
headline 1.0185 vs ARN 1.03 (±5%). Method 3 blocked (нет pre-2025-Q2 данных) — задокументировано.

Closes #17
Closes #99
2026-06-13 22:04:49 +05:00
8242e51b61 feat(generative): exporters (dxf/pdf) + generative test suite (#54 #55 #56)
Some checks failed
CI / frontend-tests (push) Has been skipped
CI / changes (push) Successful in 10s
CI / changes (pull_request) Successful in 7s
CI / frontend-tests (pull_request) Has been skipped
CI / backend-tests (push) Successful in 7m39s
CI / backend-tests (pull_request) Successful in 7m39s
Deploy / deploy (push) Has been cancelled
Deploy / build-frontend (push) Has been cancelled
Deploy / build-backend (push) Has been cancelled
Deploy / build-worker (push) Has been cancelled
Deploy / changes (push) Successful in 6s
Дополняет движок: exporters/{dxf,pdf}.py (ezdxf + WeasyPrint lazy) + 5 тест-модулей
(geometry/placement/teap_financial/exporters/api). Не вошли в предыдущий commit
(untracked-директории).
2026-06-13 21:33:20 +05:00
3b79533f9f feat(generative): движок концепций Stage 1a/1b/1c (#54 #55 #56)
Some checks failed
CI / changes (push) Successful in 7s
CI / frontend-tests (push) Has been skipped
CI / backend-tests (push) Has been cancelled
Заменяет generative stubs детерминированным end-to-end пайплайном:
- 1a geometry: WGS84-parse, метрическая AEQD-репроекция, setback-буфер, placement grid
- 1b placement: greedy section-fill + STRtree коллизии (full-gap), 3 стратегии, coverage cap
- 1c teap/financial: площади/квартиры/FAR/паркинг; выручка/затраты/маржа + IRR-proxy
- exporters: ezdxf DXF (геометрия) + WeasyPrint ТЭП/фин PDF (lazy import)
generate() заменил generate_stub в route (422 на degenerate). mypy-strict + ruff clean,
31 generative-тест + full suite 3078 passed. ConceptVariant заполняется реальными числами.

Closes #54
Closes #55
Closes #56
2026-06-13 21:32:32 +05:00
1883271908 fix(cadastre): NSPD 500-retry + per-layer skip (#252) + disable dead pzz-beat (#259) + geom backfill (#200)
Some checks failed
CI / changes (push) Successful in 7s
CI / frontend-tests (push) Has been skipped
CI / changes (pull_request) Successful in 6s
CI / frontend-tests (pull_request) Has been skipped
CI / backend-tests (push) Successful in 6m43s
CI / backend-tests (pull_request) Successful in 6m41s
Deploy / build-backend (push) Blocked by required conditions
Deploy / deploy (push) Blocked by required conditions
Deploy / build-worker (push) Blocked by required conditions
Deploy / build-frontend (push) Blocked by required conditions
Deploy / changes (push) Has been cancelled
#252: NspdBulkServerError на HTTP 5xx + WMS <ServiceException>-body → quarter-level
autoretry; _grid_walk_category флагует layer_failed только если ВСЕ cells 5xx (none ok),
network-errors исключены — пустой/флапающий слой не ложно-скипается. harvest_meta.layer_X_failed.

#259: pzz-sync-monthly beat отключён (PKK6 deprecated, pzz_zones_ekb=0, заменён
zone_regulation_cache+NSPD dumps). logger.exception→warning (стоп GlitchTip BACKEND-1B),
raise сохранён для admin-endpoint.

#200: geom_unavailable флаг (миграция 151) + backfill_parcel_geom task — grid-walk
layer 36368 per quarter лечит 930/964 NULL-geom участков (NSPD search вернул центроиды),
остаток marks unavailable (не ретраить вечно). SAVEPOINT per-cell, idempotent.

Closes #252
Closes #259
Closes #200
2026-06-13 20:45:05 +05:00
84d08940c6 fix(analytics): свежая ипотечная ставка + recommend last_updated + drop dead route (#236 #237 #251)
All checks were successful
Deploy / changes (push) Successful in 6s
Deploy / build-frontend (push) Successful in 3m9s
Deploy / build-backend (push) Successful in 4m12s
Deploy / build-worker (push) Successful in 5m31s
Deploy / deploy (push) Successful in 1m19s
#236: _current_mortgage_rate сортировал cbr_mortgage_series по TEXT-полю period
лексикографически ('Январь'>'Февраль') → возвращал stale-точку. Сортировка по
parsed (year, month) + regex-guard на malformed period-строки. Остаётся на
subsidized weighted series (НЕ domrf market — иная метрика). Verified prod:
Февраль 2026 (было Январь 2026).

#237: scope.last_updated = MAX(snapshot_date) domrf_kn_objects в recommend_mix +
подпись «данные на <дата>» в recommend page.

#251: удалён never-implemented GET /parcels/{parcel_id} (501) + ParcelDetail схема
(frontend bare 1-сегментный route не зовёт).

Closes #236
Closes #237
Closes #251
2026-06-13 15:22:22 +00:00
3cf9fad683 fix(backend): экранировать Excel formula-injection (#1244) + увести chat-чтение с event loop (#1245)
Some checks failed
CI / changes (push) Successful in 6s
CI / frontend-tests (push) Has been skipped
CI / changes (pull_request) Successful in 7s
CI / frontend-tests (pull_request) Has been skipped
CI / backend-tests (push) Successful in 6m38s
CI / backend-tests (pull_request) Successful in 6m37s
Deploy / build-backend (push) Blocked by required conditions
Deploy / build-worker (push) Blocked by required conditions
Deploy / build-frontend (push) Blocked by required conditions
Deploy / deploy (push) Blocked by required conditions
Deploy / changes (push) Has been cancelled
#1244 (security): внешние/скрейпинг-строки (comm_name из DOM.РФ, headline/usp_text)
с ведущим = + - @ \t \r писались как есть → openpyxl сохранял как формулы
(data_type='f'), исполнялись при открытии в Excel/LibreOffice. _sanitize_formula
префиксует такие строки апострофом (OWASP CSV-injection escape); числа/даты/bool
не трогаются. _write_kv labels тоже санитизируются. Подтверждено на openpyxl 3.1.5.

#1245 (concurrency): async ask() вызывал sync get_report_for_chat() (sync SQLAlchemy
тянет крупный JSONB §22-отчёт) напрямую — блокировал event loop, в отличие от
LLM-ветки (run_in_threadpool). Обёрнуто в run_in_threadpool.

Closes #1244
Closes #1245
2026-06-13 18:10:21 +05:00
99c9a130bc feat(forecast): расширить горизонт прогноза до 24 мес (ТЗ §12.1)
All checks were successful
CI / changes (push) Successful in 6s
CI / frontend-tests (push) Successful in 46s
CI / changes (pull_request) Successful in 6s
CI / frontend-tests (pull_request) Successful in 46s
CI / backend-tests (push) Successful in 6m35s
CI / backend-tests (pull_request) Successful in 6m30s
Deploy / changes (push) Successful in 5s
Deploy / build-backend (push) Successful in 2m14s
Deploy / build-frontend (push) Successful in 2m55s
Deploy / build-worker (push) Successful in 4m6s
Deploy / deploy (push) Successful in 1m40s
API отвергал ?horizon=24 (422), хотя ТЗ §12.1 называет 6/12/18/24, а движок
УЖЕ считает 24 на каждом ране: _DEFAULT_HORIZONS=(6,12,18,24) во всех 6 точках
стека (orchestrator/forecast-task/demand_supply_forecast/scenarios/
special_indices/report_assembler), PIPELINE_HORIZON_MONTHS=24.
_hidden_release_fraction клампит h/18→1.0 на 24 (без переполнения),
future_supply._horizon_weight расширяет окно чисто — скрытых ≤18 потолков нет.
Чистое расширение валидатора-enum, не новая математика.

Backend: _ALLOWED_FORECAST_HORIZONS → {6,12,18,24}, Query/docstring/error-msg.
Frontend: HorizonSelector HORIZONS=[6,12,18,24] (тип horizon=number, union не нужен;
прочие потребители data-driven через meta.horizons/forecasts_by_horizon).
Тесты: API принимает 24/отвергает 30; движок-тесты доказывают h=24 осмыслен
(поля посчитаны, demand(24)>demand(18), hidden созрел, индексы в диапазонах).

Closes #944 (Q1 горизонт 24)
2026-06-13 17:27:35 +05:00
b211183940 feat(site-finder): детерминированная атрибуция застройщика в analyze (#1088)
All checks were successful
CI / changes (pull_request) Successful in 6s
CI / changes (push) Successful in 8s
CI / frontend-tests (push) Has been skipped
CI / frontend-tests (pull_request) Has been skipped
CI / backend-tests (push) Successful in 6m33s
CI / backend-tests (pull_request) Successful in 6m33s
Deploy / changes (push) Successful in 6s
Deploy / build-frontend (push) Has been skipped
Deploy / build-backend (push) Successful in 1m44s
Deploy / build-worker (push) Successful in 5m24s
Deploy / deploy (push) Successful in 1m30s
Сервис get_developer_attribution поверх fn_developer_for_parcel (миграция 149):
топ-1 застройщик участка + track-record (РНС/РВЭ + домрф) + nearby_developers.
Дедуп по норм-ИНН (лучший match_method первым). Wire в analyze additive без флага
(чистый DB-резолвер по индексам, graceful → None). Beat-refresh developer_registry
ежемесячно 05:30 МСК после ekburg-permits. Pydantic-схемы + 18 тестов.

Источники: ekburg_construction_permits (РНС, ИНН) ⋈ domrf_kn_objects по
нормализованному ИНН + spatial/quarter fallback. 68% domrf-застройщиков имеют РНС.
EXPLAIN: point-lookup 0.06ms, резолвер ~30ms (functional/GIST индексы в миграции).

Closes #1088
2026-06-13 16:13:45 +05:00
3df163f741 fix(snapshot-pdf): Unicode '—' вместо HTML-entity '&mdash;' в Jinja or-fallback (#1234)
All checks were successful
CI / changes (pull_request) Successful in 6s
Deploy / changes (push) Successful in 9s
CI / frontend-tests (pull_request) Successful in 55s
Deploy / build-backend (push) Successful in 2m20s
Deploy / build-frontend (push) Successful in 3m25s
Deploy / build-worker (push) Successful in 4m16s
Deploy / deploy (push) Successful in 1m22s
CI / backend-tests (pull_request) Successful in 6m43s
2026-06-13 15:04:12 +05:00
eada9a25ac fix(scrapers): izyatie OCR — '6б:' → '66:' для кад-номеров МСК-66 (#1228) 2026-06-13 15:03:59 +05:00
367ed890f1 fix(objective-sync): log seed failure вместо bare except pass (#1226) 2026-06-13 15:03:34 +05:00
49d26b4696 fix(poi_loader): timedelta(days=730) вместо replace(year=-2) → leap-year 29 февраля safe (#1232) 2026-06-13 15:03:08 +05:00
c3c6e2cef1 fix(okn): expand registry coverage + load detail-card name/address (#1159)
okn_objects_sync собирал 428 ОКН из ~798 ЕГРОКН и сохранял raw_props={id}
без названия/адреса — analyze показывал 'ОКН рядом' без указания КАКОГО.

Coverage: расширил _ADDRESS_PASSES с 2 до 11 подстрок (Екатеринбург,
г. Екатеринбург, 7 районов ЕКБ, 2 формы Свердловской обл.). _PASSES =
декартово (address × category). first-wins дедуп сохраняет высшую категорию.

Detail: fetch_okn_detail(source_id) — GET Show/Show?id=, BeautifulSoup
парсит HTML (dl/dt/dd и tr/td), извлекает name, address, protection_category,
object_kind, dating. _enrich_with_detail сливает в raw_props с graceful
failover (cap=1500, exception per-object не валит прогон).

Lookup: parcel_okn_objects возвращает name/address из raw_props->>'name'.
Existing ird_analyze тесты не сломаны.

49 okn-related тестов passed, ruff clean.

Closes #1159
2026-06-13 15:02:50 +05:00
2e428405d2 fix(site-finder): расширить except в parcel_ird_overlaps до DataError (#1095)
parcel_ird_overlaps ловил только (OperationalError, ProgrammingError).
При D9b-wiring в analyze malformed WKT в ST_GeomFromText давал PostGIS
ERROR → SQLAlchemy DataError, который пробивал try/except → analyze
падал вместо graceful-degrade.

- ird_overlay_lookup: + DataError в imports и в except tuple.
- quarter_dump_lookup: симметричное расширение в _get_engineering_*.
- test_ird_overlay_lookup: parametrized test_graceful_when_db_fails
  покрывает 3 класса (Operational/Programming + DataError для WKT).

Closes #1095
2026-06-13 15:02:50 +05:00
d11c8d47e0 fix(workers): per-prefix db.commit() в gknspecial_harvest для durability (#1116)
harvest_gknspecial_zones делал единственный commit в конце всех 8 prefix-
циклов: WorkerLostError (OOM / рестарт хоста) в середине прогона →
~20k записанных строк откатывались, прогресс ноль.

Mirror ird_harvest.py:216 (per-quarter commit): db.commit() после
финального logger.info внутри for-prefix loop. SAVEPOINT-per-row уже
изолирует битые фичи; per-prefix commit добавляет durability между
префиксами. WFS-fail на префиксе → continue → commit не делается;
предыдущие префиксы уже durable.

+2 теста (durability, WFS-fail skip). 29 gknspecial тестов зелёные.

Closes #1116
2026-06-13 15:02:50 +05:00
8a30238564 fix(forecasting): propagate confounded flag DemandSupplyForecast → §15 (#1222)
DemandSupplyForecast.as_dict() не эмитил 'confounded'/'is_confounded_window',
report_assembler._confounded() всегда возвращал False и §15 confounded_window
factor в compute_report_confidence был мёртв: 48-мес окна, пересекающие
2024-07-01 шок никогда не тянули report confidence к 'low' и шок не назывался
в rationale.

Patch: добавлено confounded: bool в DemandSupplyForecast (от §9.5 macro_coef
OR §9.6 rate_sensitivity), exposed в as_dict(). _confounded() уже использовал
.get() defensively — блокер был в producer'е.

+3 теста: contract на real DemandSupplyForecast.as_dict(), end-to-end
assemble_report → confounded_window factor surfaces at level=low, weakest-link
тянет overall к 'low'. 61 report_assembler + 1034 forecasting тестов зелёные.

Closes #1222
2026-06-13 15:02:50 +05:00
703d3905b8 fix(site-finder): normalize supply room_bucket vocabulary to velocity side (#1229)
best_layouts._SUPPLY_BATCH_SQL эмитил {studio,euro-1,euro-2,1,2,3,4+},
а _INLINE_VELOCITY_SQL читает {студия,1,2,3,4+} из
objective_corpus_room_month (prod check: 'euro-*' rows отсутствуют).

Эффект: rooms=2 + area<50 уходили в euro-1/euro-2 supply-стороной →
выпадали из знаменателя bucket '2' → sold_pct_of_supply двушек
завышен, is_oversold ложно True. (rb='euro-*') dead lookups в supply_map.

Patch: убраны euro-* WHEN в supply CASE. SF-08 euro-биннинг отложен
до момента когда velocity-сторона начнёт его отдавать. +2 regression
теста (bucket match, string guard). 35 best_layouts тестов зелёные.

Closes #1229
2026-06-13 15:02:50 +05:00
780dd5640c fix(analytics): _district_market_saturation latest-snapshot filter (#1221)
domrf_kn_objects/sales_agg хранят ~3 snapshot'а на obj_id (weekly UPSERT,
retention=none). JOIN без latest-фильтра считал каждый ЖК N раз:
sold_median тянулся старыми perc → sat_factor занижен, гейт n>=5
проходил уже при 2 ЖК (n=6), sat_n завышен ~×N в scope. Сданные ЖК
просачивались через старые снапшоты со статусом 'Строящиеся'.

Patch: AND a.snapshot_date = (SELECT MAX(snapshot_date) FROM ...) для
обеих таблиц (зеркало #1210/#1212). COUNT(*) → COUNT(DISTINCT a.obj_id)
defensive. 29 recommend_mix + 8 saturation тестов зелёные.

Closes #1221
2026-06-13 15:02:50 +05:00
d694e735cd fix(site-finder): per-row SAVEPOINT в noise_loader (#1231)
Loop UPSERT в sync_noise_sources_to_db без begin_nested: один замкнутый
3-точечный natural=water way [A,B,A] даёт POLYGON((A,B,A)), PostGIS
отвергает (< 4 точек в ring) → outer tx rollback + raise → весь weekly
noise/water/utility sync падает, тот же way отравляет каждый прогон.

- Оборачиваем каждый UPSERT в `with db.begin_nested():` + per-row
  try/except → logger.warning + skipped++ (канон pzz_loader.py:111).
- В _way_to_polygon_wkt проверяем итоговое кольцо ≥ 4 точек (fail-safe).
- Outer except: добавлен logger.exception для видимости.

Closes #1231
2026-06-13 15:02:50 +05:00
285e8f974a fix(sf): ekb_ppt_tep post-фильтры + врезка в analyze (#1136)
All checks were successful
Deploy / changes (push) Successful in 6s
Deploy / build-backend (push) Successful in 2m12s
Deploy / build-frontend (push) Has been skipped
Deploy / build-worker (push) Successful in 4m34s
Deploy / deploy (push) Successful in 1m11s
Post-фильтры парсера (по образцу ppt2018_22823):
- Табл.11: отбрасываем строки без zone_name (преамбульные артефакты
  merged-cells давали 2 фантома при 12 реальных зонах).
- Табл.13: dedup по (phase, composition, zone, area_ha) + фильтры
  короткого-без-зоны и нумерационного шума (~364 строк → ~30-40 реальных).

Analyze-врезка: новый ppt_tep_lookup.parcel_ppt_tep — JOIN
planning_projects ⋈ ekb_ppt_tep по doc_ref↔source_key/doc_full_name
(best-effort, без FK). Wired в build_ird_analyze_block рядом с
planning_projects/krt_requisites — DB-источники, graceful.

Seed-URL остаётся placeholder с # VERIFY (ingest пропускает с WARNING).

Tests: 27 в parser (3 новых), 7 lookup, 22 wiring (2 новых).
All ruff/syntax green.

Closes #1136
2026-06-13 09:32:52 +00:00
a5ff281bad fix(admin-jobs): UPSERT в PUT /admin/jobs/settings — не терять изменения (#1223)
All checks were successful
Deploy / changes (push) Successful in 6s
Deploy / build-frontend (push) Has been skipped
Deploy / build-backend (push) Successful in 1m37s
Deploy / build-worker (push) Successful in 2m46s
Deploy / deploy (push) Successful in 1m19s
UPDATE без проверки rowcount затрагивал 0 строк для несуществующего
job_type; get_one возвращал hardcoded _fallback. PUT отвечал 200,
админ считал настройку сохранённой — но она терялась (новый job_type
до seed-миграции, опечатка в path).

Patch: update() проверяет result.rowcount: при 0 делает INSERT с
переданными колонками (непереданные → дефолты таблицы из м.81).
INSERT использует ON CONFLICT (job_type) DO NOTHING + повторный
UPDATE как защита от гонки. Поведение для существующих строк не
изменилось. 8 новых тестов (6 service + 2 API).

Closes #1223
2026-06-13 08:36:23 +00:00
f5e7fcdad9 fix(workers): cadastre permanent-fail targets_failed + beat cleanup (#1236)
Some checks failed
Deploy / build-frontend (push) Blocked by required conditions
Deploy / deploy (push) Blocked by required conditions
Deploy / changes (push) Successful in 6s
Deploy / build-worker (push) Has been cancelled
Deploy / build-backend (push) Has been cancelled
2026-06-13 08:36:13 +00:00
28c567a34e fix(scoring): correct price_feasibility reason — market rate, not subsidized (#1225)
All checks were successful
Deploy / changes (push) Successful in 6s
Deploy / build-frontend (push) Has been skipped
Deploy / build-backend (push) Successful in 1m35s
Deploy / build-worker (push) Successful in 2m35s
Deploy / deploy (push) Successful in 1m21s
ProductScore.reason для price_feasibility (§14.2) называл основу платежа
'субсид. ставка', но compute_affordability с #981 DoD перешёл на рыночный
прокси key_rate + 4.5 п.п. (~19%, rate_kind='key_rate_proxy'); субсидированный
путь не вызывается. Метка делала рыночный платёж в §22-отчёте похожим на
льготный и противоречила affordability.degraded_reason в том же payload.

Текстовый фикс: 'рыночная ставка key_rate + спред, §7.9' — число платежа
было корректно, advisory-описание теперь совпадает.

Closes #1225
2026-06-13 08:20:30 +00:00
816bb3d3bd fix(workers): re-raise SoftTimeLimitExceeded в nspd_geo, set paused (#1235)
Some checks failed
Deploy / build-backend (push) Blocked by required conditions
Deploy / build-worker (push) Blocked by required conditions
Deploy / build-frontend (push) Blocked by required conditions
Deploy / deploy (push) Blocked by required conditions
Deploy / changes (push) Has been cancelled
2026-06-13 08:20:18 +00:00