Commit graph

1204 commits

Author SHA1 Message Date
904e945035 perf(sql): geography GIST + covering index для горячего analyze-пути SF (157)
All checks were successful
CI / changes (push) Successful in 6s
CI / frontend-tests (push) Has been skipped
CI / openapi-codegen-check (push) Successful in 1m56s
CI / changes (pull_request) Successful in 6s
CI / frontend-tests (pull_request) Has been skipped
CI / backend-tests (push) Successful in 8m54s
CI / openapi-codegen-check (pull_request) Successful in 1m51s
CI / backend-tests (pull_request) Successful in 8m36s
EXPLAIN ANALYZE на проде выявил Seq Scan на каждый analyze из-за geom::geography
предикатов (плоский geom-GIST не применим). 4 индекса:
- osm_poi_ekb/osm_noise_sources_ekb/cad_buildings geography GIST:
  POI 36->0.2мс, noise 92->3.8мс (4x/analyze), соседи 314->0.3мс
- objective_lots(project_name) INCLUDE(price,area,is_sold): competitor 493->355мс
Lock SHARE (writes-only, читателей analyze не блокирует), build ~2.2с (как 153).
Idempotent (IF NOT EXISTS).

Refs #70
2026-06-14 21:32:41 +05:00
b405ae291d feat(site-finder): additive objective_lots price fallback for competitors (#307 OBJ-3) (#1332)
All checks were successful
Deploy / changes (push) Successful in 7s
Deploy / build-backend (push) Successful in 1m57s
Deploy / build-worker (push) Successful in 2m51s
Deploy / build-frontend (push) Successful in 3m6s
Deploy / deploy (push) Successful in 1m12s
2026-06-14 16:10:02 +00:00
f9dd903f4d feat(sql): conservative trgm backfill objective_complex_mapping +23 (#307 OBJ-2) (#1331)
All checks were successful
Deploy / build-backend (push) Successful in 29s
Deploy / build-worker (push) Successful in 29s
Deploy / deploy (push) Successful in 1m1s
Deploy / changes (push) Successful in 6s
Deploy / build-frontend (push) Has been skipped
2026-06-14 15:51:38 +00:00
f376d8ce05 fix(tradein-browser): non-fatal launch + background proxy retry (crash-loop 3141x) (#1330)
All checks were successful
Deploy Trade-In / deploy (push) Successful in 1m18s
Deploy Trade-In / changes (push) Successful in 7s
Deploy Trade-In / test (push) Has been skipped
Deploy Trade-In / build-frontend (push) Has been skipped
Deploy Trade-In / build-backend (push) Has been skipped
Deploy Trade-In / build-browser (push) Successful in 2m2s
2026-06-14 15:18:58 +00:00
72e0b050f9 feat(site-finder): POI saturation per capita по районам ЕКБ (#42) (#1329)
All checks were successful
Deploy / changes (push) Successful in 6s
Deploy / build-frontend (push) Has been skipped
Deploy / build-backend (push) Successful in 1m38s
Deploy / build-worker (push) Successful in 2m52s
Deploy / deploy (push) Successful in 1m12s
2026-06-14 15:05:22 +00:00
e5e0ccb283 feat(site-finder): сетевые обременения из ЗОУИТ (#1070, Refs #103) (#1328)
All checks were successful
Deploy / changes (push) Successful in 6s
Deploy / build-frontend (push) Has been skipped
Deploy / build-backend (push) Successful in 1m36s
Deploy / build-worker (push) Successful in 2m40s
Deploy / deploy (push) Successful in 1m12s
2026-06-14 14:49:32 +00:00
5507bbc1a4 feat(site-finder): domrf→cad geom-match + parking_ratio конкурентов (#96) (#1327)
Some checks failed
Deploy / build-frontend (push) Blocked by required conditions
Deploy / deploy (push) Blocked by required conditions
Deploy / build-backend (push) Blocked by required conditions
Deploy / build-worker (push) Blocked by required conditions
Deploy / changes (push) Has been cancelled
2026-06-14 14:49:28 +00:00
2c22c3f7ea feat(site-finder): per-building помещения/машино-места + parking_ratio (on-demand MVP, #96)
All checks were successful
CI / frontend-tests (push) Has been skipped
CI / changes (pull_request) Successful in 6s
CI / changes (push) Successful in 8s
CI / frontend-tests (pull_request) Has been skipped
CI / openapi-codegen-check (push) Successful in 1m59s
CI / openapi-codegen-check (pull_request) Successful in 1m58s
CI / backend-tests (push) Successful in 8m48s
CI / backend-tests (pull_request) Successful in 8m48s
Deploy / changes (push) Successful in 6s
Deploy / build-frontend (push) Has been skipped
Deploy / build-backend (push) Successful in 4m58s
Deploy / build-worker (push) Successful in 5m46s
Deploy / deploy (push) Successful in 1m30s
Оживляет мёртвый foundation list_objects_in_building (#168 Q3-deferred). Fix реального
блокера: NSPDOptions не парсил objdocId (camelCase из NSPD search) → objdoc_id всегда None,
метод нельзя было вызвать. AliasChoices(objdoc_id, objdocId) + registers.

premises_lookup.get_building_premises(cad_num) — резолв objdoc → list_objects_in_building,
graceful (WAF/сеть/not-found → None). parking_ratio = машино-места/помещения (None при 0
помещений, 0.0 при реальном отсутствии паркинга). Verified live 66:41:0106036:183.

MVP on-demand (без bulk/схемы/prod-записей). parking_ratio готов, но НЕ wired в analyze:
конкуренты приходят из ДОМ.РФ без cad_num → нужен domrf↔cad_buildings geom-match (отдельная
задача). Сервис примет cad_num как только matching появится. 11 тестов, ruff clean.

Refs #96
2026-06-14 18:48:16 +05:00
11aa011d11 perf(db): covering-index objective_lots district-price (#70) + 152-ФЗ/perf audit docs (#70 #79)
All checks were successful
CI / changes (push) Successful in 7s
CI / changes (pull_request) Successful in 6s
CI / frontend-tests (push) Has been skipped
CI / frontend-tests (pull_request) Has been skipped
CI / openapi-codegen-check (push) Successful in 2m3s
CI / openapi-codegen-check (pull_request) Successful in 2m7s
CI / backend-tests (push) Successful in 8m55s
CI / backend-tests (pull_request) Successful in 8m58s
Deploy / changes (push) Successful in 6s
Deploy / build-frontend (push) Has been skipped
Deploy / build-backend (push) Successful in 29s
Deploy / build-worker (push) Successful in 29s
Deploy / deploy (push) Successful in 1m10s
#70: EXPLAIN top-5 hot-path на prod — analyze district-price block по objective_lots
(1.12M) делал Bitmap Heap Scan ~41k блоков (cold 5.3s/warm 474ms), пробивая p95 ≤3с.
Partial covering-index (district, price_per_m2_rub) WHERE price_per_m2_rub IS NOT NULL →
Index Only Scan (verified scratch: 87ms, Heap Fetches 0). Требует VACUUM ANALYZE
objective_lots post-deploy (stale VM иначе глушит выигрыш). database-expert  verified.

#79: 152-ФЗ PII grep-аудит — plain-PII в логах НЕ найдено; Sentry scrub / PRINZIP-хеширование
/ LLM §19-redaction подтверждены. Остаточный риск: pilot_requests хранит сырые ПДн без
retention (рекомендация в docs). httpx DEBUG off (ops-инвариант).

Closes #70
Closes #79
2026-06-14 00:09:09 +05:00
d030b04cf2 docs(infra): secrets rotation policy + procedure (#78)
All checks were successful
CI / changes (push) Successful in 7s
CI / backend-tests (push) Has been skipped
CI / frontend-tests (push) Has been skipped
CI / openapi-codegen-check (push) Has been skipped
CI / changes (pull_request) Successful in 6s
CI / backend-tests (pull_request) Has been skipped
CI / frontend-tests (pull_request) Has been skipped
CI / openapi-codegen-check (pull_request) Has been skipped
Compliance-doc: реестр классов секретов (CI/CD + prod runtime), процедуры ротации
per-class с downtime-эффектом, периодичность, quarterly audit-чеклист. OIDC verdict:
future (Forgejo 10.0.3 без Actions OIDC token provider; deploy = прямой SSH, не cloud
federation). Флагает sshkey.txt (PR #391) достижимый в git-истории — deploy-key
скомпрометирован, нужна ротация. Значения — в vault meta/00_credentials.md (не в репо).

Closes #78
2026-06-13 23:48:14 +05:00
eb251ba7e7 ci(infra): coverage-gate (#68) + OpenAPI codegen-assert (#69) + uptime monitoring (#75)
All checks were successful
CI / changes (push) Successful in 13s
CI / changes (pull_request) Successful in 12s
CI / frontend-tests (push) Successful in 53s
CI / openapi-codegen-check (push) Successful in 1m58s
CI / frontend-tests (pull_request) Successful in 47s
CI / openapi-codegen-check (pull_request) Successful in 1m50s
CI / backend-tests (push) Successful in 8m54s
CI / backend-tests (pull_request) Successful in 8m45s
Deploy / changes (push) Successful in 6s
Deploy / build-backend (push) Successful in 2m50s
Deploy / build-frontend (push) Successful in 3m11s
Deploy / build-worker (push) Successful in 4m38s
Deploy / deploy (push) Successful in 2m1s
#68: pytest-cov + [tool.coverage] fail_under=65 (baseline 71%), backend-tests --cov + xml.
#69: openapi-codegen-check job (dump app.openapi() → openapi-typescript → git diff).
  Job использует node_modules-pinned openapi-typescript/prettier (НЕ npx --yes latest —
  иначе version-mismatch ложный diff). + regenerated api-types.ts из СВЕЖЕГО main openapi
  (8288 строк, включая #73 /freshness — синхронен).
#75: Uptime Kuma isolated stack + status.gendsgn.ru Caddy + external Telegram watchdog.
#77: no change — build cache достаточен.

Closes #68
Closes #69
Closes #75
Closes #77
2026-06-13 23:28:06 +05:00
d38417d8af feat(admin): freshness endpoint (#73) + README Forgejo refresh (#83) + migrations doc (#72)
All checks were successful
Deploy / build-frontend (push) Has been skipped
Deploy / changes (push) Successful in 5s
Deploy / build-backend (push) Successful in 2m39s
Deploy / build-worker (push) Successful in 7m20s
Deploy / deploy (push) Successful in 2m48s
#73: GET /admin/scrape/freshness — last_success/age/freshness/status для 5 источников
(kn/objective/nspd/nspd_geo/cadastre) + overall_status (worst по critical). Prod-verified
имена таблиц, FILTER/COALESCE корректны, CAST(:d AS interval). +4 теста.
#83: README — GitHub→Forgejo CI/CD, секция 6 subagents, ссылки перемаплены.
#72: docs/Schema_Migrations_vs_Alembic.md — _schema_migrations паттерн (не Alembic),
рекомендация снести пустой alembic-скелет (отдельный PR).

Closes #73
Closes #83
Closes #72
2026-06-13 18:25:32 +00:00
8107c23f18 feat(frontend): mobile responsiveness (#66) + data-sources страница (#80)
Some checks failed
CI / changes (push) Successful in 8s
CI / backend-tests (push) Has been skipped
CI / frontend-tests (push) Successful in 1m3s
CI / changes (pull_request) Successful in 6s
CI / backend-tests (pull_request) Has been skipped
CI / frontend-tests (pull_request) Successful in 50s
Deploy / changes (push) Has been cancelled
Deploy / build-frontend (push) Has been cancelled
Deploy / deploy (push) Has been cancelled
Deploy / build-backend (push) Has been cancelled
Deploy / build-worker (push) Has been cancelled
#66: responsive CSS-layer в globals.css (3-контекста <768/768-1280/>1280, не Tailwind
sm/md/lg) + ResizeObserver в ChartShell (единственный echarts импортёр → все charts
рефлоу) + class-применение (grid-collapse, nav horizontal-scroll, map-stack) на
site-finder/analytics. overflow-x guard.
#80: /data-sources attribution-страница (OSM ODbL/ДОМ.РФ/Объектив/Open-Meteo/Росреестр/
НСПД, server component, tokens) + footer-link.

Closes #66
Closes #80
2026-06-13 23:12:31 +05:00
0b20d98b45 feat(site-finder): ЗОУИТ visual layer на Leaflet карте (#255)
All checks were successful
Deploy / changes (push) Successful in 6s
Deploy / build-backend (push) Has been skipped
Deploy / build-worker (push) Has been skipped
Deploy / build-frontend (push) Successful in 2m49s
Deploy / deploy (push) Successful in 1m6s
ZouitLayer (GeoJSON-полигоны из geom_geojson, severity blocker/warning/info по keyword,
popup тип+имя) + ZouitLayerControlPanel (collapsible toggle per-severity, счётчик mappable)
+ SiteMap integration (default blocker+warning ON, info OFF, фон под POI). Graceful: нет
geom_geojson → слой/панель скрыты, текстовый NspdZouitOverlapsBlock остаётся. 10 vitest.

Closes #255
2026-06-13 17:41:08 +00:00
50555821fb feat(site_finder): routing POI decay (#41) + OSM utility distances (#44) + ЗОУИТ geom (#255bk)
Some checks failed
CI / changes (push) Successful in 7s
CI / frontend-tests (push) Has been skipped
CI / changes (pull_request) Successful in 7s
CI / frontend-tests (pull_request) Has been skipped
CI / backend-tests (push) Successful in 7m42s
CI / backend-tests (pull_request) Successful in 7m40s
Deploy / changes (push) Has been cancelled
Deploy / build-backend (push) Has been cancelled
Deploy / build-worker (push) Has been cancelled
Deploy / build-frontend (push) Has been cancelled
Deploy / deploy (push) Has been cancelled
#41: per-category routing-radius decay через reusable ORS /matrix client (foot-walk
время + per-category пороги + decay-curves), opt-in ?decay=routing, graceful straight-line
fallback (analyze не 500 при ORS down) + length-guard durations==destinations.
#44: +5 OSM-категорий (transformer/gas/water/sewerage/heat) via Overpass nwr;
nearest_{water_main,substation,gas,heat}_m в analyze.utilities.
#255bk: ST_AsGeoJSON(CAST(geom AS geometry)) в _get_cad_zouit_overlaps → geom_geojson.

Closes #41
Closes #44
2026-06-13 22:31:56 +05:00
adc6dde015 feat(site-finder): multi-cad сравнение (#50) + analytics поиск/cross-links (#65)
All checks were successful
Deploy / changes (push) Successful in 6s
Deploy / build-backend (push) Has been skipped
Deploy / build-worker (push) Has been skipped
Deploy / build-frontend (push) Successful in 3m59s
Deploy / deploy (push) Successful in 1m7s
#50: /site-finder/compare — shortlist (localStorage, SSR-guard, cap 2-5) + CompareTable
(per-row winner-highlight, reuse useParcelAnalyzeQuery per cad, hook-order stable в .map,
cache-shared). Cross-link с site-finder.

#65: debounced search (300ms, client-side filter top-15) в DeveloperLeaderboard +
cross-link объект→Site Finder analyze. Existing developers compare не дублирован.

Closes #50
Closes #65
2026-06-13 17:14:15 +00:00
658d724075 feat(analytics): velocity-anomaly alerts (#17) + ARN ДДУ price indicator (#99)
Some checks failed
CI / changes (push) Successful in 8s
CI / frontend-tests (push) Has been skipped
CI / changes (pull_request) Successful in 8s
CI / frontend-tests (pull_request) Has been skipped
CI / backend-tests (push) Successful in 7m37s
CI / backend-tests (pull_request) Successful in 7m42s
Deploy / deploy (push) Blocked by required conditions
Deploy / build-backend (push) Blocked by required conditions
Deploy / build-worker (push) Blocked by required conditions
Deploy / build-frontend (push) Blocked by required conditions
Deploy / changes (push) Has been cancelled
#17: detect_velocity_anomalies + GET /analytics/velocity-alerts — z-score drop-detection
на domrf_kn_sale_graph (double-gate z≤-2 AND drop≤-30%, starvation-guards). Snapshot=richest
(не MAX — prod новейшие частичные), lookback anchored на latest report_month (scrape лаг ~4мес).
Prod: ЖК Центральный Парк -69%, ~14ms.

#99: mv_ddu_price_indicator (миграция 152) + POST /market/ddu-indicator — ARN-mirror
ценовой индекс per quarter×area_bucket из rosreestr_deals (ДДУ регион 66). MVP: subject-level,
period Q, window 2025-Q2+, methods 1/2 (basis/previous, prev_period_value honesty). Q1-2026
headline 1.0185 vs ARN 1.03 (±5%). Method 3 blocked (нет pre-2025-Q2 данных) — задокументировано.

Closes #17
Closes #99
2026-06-13 22:04:49 +05:00
3bbbf25412 feat(concept): concept-компоненты (DrawMap/ParamsForm/VariantsResult/ResultMap/ExportButtons) (#57)
All checks were successful
Deploy / changes (push) Successful in 6s
Deploy / build-backend (push) Has been skipped
Deploy / build-worker (push) Has been skipped
Deploy / build-frontend (push) Successful in 2m45s
Deploy / deploy (push) Successful in 1m3s
Дополняет UI: 5 компонентов которые импортит page.tsx (Leaflet draw, форма ConceptInput,
табы вариантов, placement-карта, экспорт). Не вошли в предыдущий commit (untracked-dir).
ConceptResultMap: Leaflet stroke #D1D5DB (CSS var не резолвится в SVG).
2026-06-13 16:42:19 +00:00
750d34d5cb feat(concept): Concept UI Stage 1d — ввод участка + варианты (#57)
Заменяет concept/page.tsx TODO-stubs полным generative-флоу: draw-полигон или
кадастр→geom ввод участка, форма параметров ConceptInput, POST /concepts через
useMutation, табы вариантов с placement-картой + ТЭП KPI + финмодель + экспорт
(GeoJSON/CSV client-side, DXF/PDF на /concepts/export graceful 404). Типы зеркалят
concept.py; переиспользует site-finder Leaflet + analytics Section/KpiCard паттерны.
tsc + lint + build clean.

Closes #57
2026-06-13 16:42:19 +00:00
8242e51b61 feat(generative): exporters (dxf/pdf) + generative test suite (#54 #55 #56)
Some checks failed
CI / frontend-tests (push) Has been skipped
CI / changes (push) Successful in 10s
CI / changes (pull_request) Successful in 7s
CI / frontend-tests (pull_request) Has been skipped
CI / backend-tests (push) Successful in 7m39s
CI / backend-tests (pull_request) Successful in 7m39s
Deploy / deploy (push) Has been cancelled
Deploy / build-frontend (push) Has been cancelled
Deploy / build-backend (push) Has been cancelled
Deploy / build-worker (push) Has been cancelled
Deploy / changes (push) Successful in 6s
Дополняет движок: exporters/{dxf,pdf}.py (ezdxf + WeasyPrint lazy) + 5 тест-модулей
(geometry/placement/teap_financial/exporters/api). Не вошли в предыдущий commit
(untracked-директории).
2026-06-13 21:33:20 +05:00
3b79533f9f feat(generative): движок концепций Stage 1a/1b/1c (#54 #55 #56)
Some checks failed
CI / changes (push) Successful in 7s
CI / frontend-tests (push) Has been skipped
CI / backend-tests (push) Has been cancelled
Заменяет generative stubs детерминированным end-to-end пайплайном:
- 1a geometry: WGS84-parse, метрическая AEQD-репроекция, setback-буфер, placement grid
- 1b placement: greedy section-fill + STRtree коллизии (full-gap), 3 стратегии, coverage cap
- 1c teap/financial: площади/квартиры/FAR/паркинг; выручка/затраты/маржа + IRR-proxy
- exporters: ezdxf DXF (геометрия) + WeasyPrint ТЭП/фин PDF (lazy import)
generate() заменил generate_stub в route (422 на degenerate). mypy-strict + ruff clean,
31 generative-тест + full suite 3078 passed. ConceptVariant заполняется реальными числами.

Closes #54
Closes #55
Closes #56
2026-06-13 21:32:32 +05:00
d737307022 feat(site-finder): НСПД verify deep-links + landing GenDesign vs НСПД таблица (#97 #100)
All checks were successful
Deploy / changes (push) Successful in 5s
Deploy / build-frontend (push) Successful in 4m29s
Deploy / build-backend (push) Has been skipped
Deploy / build-worker (push) Has been skipped
Deploy / deploy (push) Successful in 1m5s
#97: nspdLinks helper (nspdMapUrl/pkkSearchUrl, encodeURIComponent) + NspdVerifyLink
компонент (var(--accent), rel=noopener) → одноклик-сверка с НСПД/ПКК в Section1
(кадастр/ЕГРН) + Section2 (инж.сети) карточках analyze.

#100: landing — сравнительная секция «GenDesign и НСПД» (7-row таблица value-prop:
цены ДДУ/остатки/скорость/инвест-балл/прогноз/квартирография/экспорт сверх бесплатной
гос-карты) + footer deep-links.

Closes #97
Closes #100
2026-06-13 15:53:02 +00:00
1883271908 fix(cadastre): NSPD 500-retry + per-layer skip (#252) + disable dead pzz-beat (#259) + geom backfill (#200)
Some checks failed
CI / changes (push) Successful in 7s
CI / frontend-tests (push) Has been skipped
CI / changes (pull_request) Successful in 6s
CI / frontend-tests (pull_request) Has been skipped
CI / backend-tests (push) Successful in 6m43s
CI / backend-tests (pull_request) Successful in 6m41s
Deploy / build-backend (push) Blocked by required conditions
Deploy / deploy (push) Blocked by required conditions
Deploy / build-worker (push) Blocked by required conditions
Deploy / build-frontend (push) Blocked by required conditions
Deploy / changes (push) Has been cancelled
#252: NspdBulkServerError на HTTP 5xx + WMS <ServiceException>-body → quarter-level
autoretry; _grid_walk_category флагует layer_failed только если ВСЕ cells 5xx (none ok),
network-errors исключены — пустой/флапающий слой не ложно-скипается. harvest_meta.layer_X_failed.

#259: pzz-sync-monthly beat отключён (PKK6 deprecated, pzz_zones_ekb=0, заменён
zone_regulation_cache+NSPD dumps). logger.exception→warning (стоп GlitchTip BACKEND-1B),
raise сохранён для admin-endpoint.

#200: geom_unavailable флаг (миграция 151) + backfill_parcel_geom task — grid-walk
layer 36368 per quarter лечит 930/964 NULL-geom участков (NSPD search вернул центроиды),
остаток marks unavailable (не ретраить вечно). SAVEPOINT per-cell, idempotent.

Closes #252
Closes #259
Closes #200
2026-06-13 20:45:05 +05:00
84d08940c6 fix(analytics): свежая ипотечная ставка + recommend last_updated + drop dead route (#236 #237 #251)
All checks were successful
Deploy / changes (push) Successful in 6s
Deploy / build-frontend (push) Successful in 3m9s
Deploy / build-backend (push) Successful in 4m12s
Deploy / build-worker (push) Successful in 5m31s
Deploy / deploy (push) Successful in 1m19s
#236: _current_mortgage_rate сортировал cbr_mortgage_series по TEXT-полю period
лексикографически ('Январь'>'Февраль') → возвращал stale-точку. Сортировка по
parsed (year, month) + regex-guard на malformed period-строки. Остаётся на
subsidized weighted series (НЕ domrf market — иная метрика). Verified prod:
Февраль 2026 (было Январь 2026).

#237: scope.last_updated = MAX(snapshot_date) domrf_kn_objects в recommend_mix +
подпись «данные на <дата>» в recommend page.

#251: удалён never-implemented GET /parcels/{parcel_id} (501) + ParcelDetail схема
(frontend bare 1-сегментный route не зовёт).

Closes #236
Closes #237
Closes #251
2026-06-13 15:22:22 +00:00
6883d14177 fix(ops): repair broken main-DB backup + harden auth scripts/docs (#71 #427 #429 #428)
Some checks failed
CI / changes (push) Successful in 7s
CI / backend-tests (push) Has been skipped
CI / frontend-tests (push) Has been skipped
CI / changes (pull_request) Successful in 6s
CI / backend-tests (pull_request) Has been skipped
CI / frontend-tests (pull_request) Has been skipped
Deploy / build-backend (push) Blocked by required conditions
Deploy / build-worker (push) Blocked by required conditions
Deploy / build-frontend (push) Blocked by required conditions
Deploy / deploy (push) Blocked by required conditions
Deploy / changes (push) Has been cancelled
#71 (CRITICAL): backup.sh committed 100644 → git reset --hard on deploy
re-asserts non-exec mode → raw-path cron fails Permission denied (last good
dump 2026-05-27, no S3). Commit 100755 + chmod ops/*.sh in deploy.yml +
size sanity-check (never prune good dumps for a truncated one) + keep-N
retention + optional S3 (redacted /etc/default template). Modeled on the
working backup-tradein-db.sh.

#427: widen basic_auth username regex ^[a-z][a-z0-9_.-]{1,62}$ + literal-escape
  dotted names in grep probes.
#429: replace list_users.sh false-positive grep with anchored awk over basic_auth block.
#428: PILOT_ACCESS.md support email → pilot@gendsgn.ru.

Closes #71
Closes #427
Closes #429
Closes #428
2026-06-13 20:13:04 +05:00
d1bc59a5dd chore(objective): fuzzy backfill objective_complex_mapping +115 dev-corroborated (#307 OBJ-2)
All checks were successful
CI / changes (push) Successful in 7s
CI / frontend-tests (push) Has been skipped
CI / changes (pull_request) Successful in 6s
CI / frontend-tests (pull_request) Has been skipped
CI / backend-tests (push) Successful in 6m31s
CI / backend-tests (pull_request) Successful in 6m29s
Deploy / changes (push) Successful in 5s
Deploy / build-frontend (push) Has been skipped
Deploy / build-backend (push) Successful in 2m16s
Deploy / build-worker (push) Successful in 3m34s
Deploy / deploy (push) Successful in 1m28s
P0 goldmine: objective_lots (1.12M, 81% цен) связывается с domrf для velocity/
layout-аналитики. Mapping было 148 → +115 (107 high + 8 mid) = 263.

Fuzzy-match objective(project_name) ↔ domrf_kn_objects(comm_name) LATEST snapshot
region_cd=66 через pg_trgm на НОРМАЛИЗОВАННЫХ именах (strip ЖК-boilerplate с
\m\M word-boundary). Правило приёмки dev-corroborated: nsim≥0.62 AND dsim≥0.5 —
precision over recall для P0-цен (generic-имена с разным застройщиком, напр.
Солнечный/Прекрасный/Стрелки dsim=0.0, отклонены). match_method='auto_fuzzy_v2',
is_reviewed=false (для human-review топ-кандидатов), confidence-tier в note.
Idempotent ON CONFLICT (objective_complex_name, objective_group). REFRESH
CONCURRENTLY mv_layout_velocity после COMMIT.

Refs #307
2026-06-13 18:50:57 +05:00
414601c1d2 test(analytics): vitest для ObjectSaleChart union-months alignment (#1246)
All checks were successful
CI / changes (push) Successful in 7s
CI / backend-tests (push) Has been skipped
CI / changes (pull_request) Successful in 6s
Deploy / build-worker (push) Has been skipped
Deploy / deploy (push) Successful in 1m6s
CI / backend-tests (pull_request) Has been skipped
CI / frontend-tests (push) Successful in 56s
CI / frontend-tests (pull_request) Successful in 58s
Deploy / changes (push) Successful in 8s
Deploy / build-backend (push) Has been skipped
Deploy / build-frontend (push) Successful in 2m37s
union axis sorted, per-month alignment с nulls, регрессия-guard (хвост позднего
типа не клипается), happy-path unchanged, empty-state. 5 кейсов.
2026-06-13 18:20:41 +05:00
e2c8c0e97b fix(analytics): выровнять ObjectSaleChart серии по union месяцев всех типов (#1246)
Some checks failed
CI / changes (push) Successful in 7s
CI / backend-tests (push) Has been skipped
CI / frontend-tests (push) Has been cancelled
ObjectSaleChart строил ось X из месяцев ТОЛЬКО первого типа и привязывал серии
позиционно. domrf_kn_sale_graph фетчится per-type независимо от DOM.РФ → типы
могут нести разные наборы месяцев — позиционная привязка тогда смещает бары/
линию цены и молча клипает лишние точки. Текущие prod-данные выровнены (457/457
объектов, идентичные месяцы), поэтому live-поломки нет, но код полагался на
coincidence, не invariant. Ось из union всех месяцев + маппинг значений каждого
типа по месяцу через Map (null где отсутствует), как PrinzipVelocityChart. +vitest.

Closes #1246
2026-06-13 18:20:25 +05:00
16e505326d perf(site-finder): honour AbortSignal в poll-цикле useParcelAnalyzeQuery (#1248)
Some checks failed
Deploy / deploy (push) Blocked by required conditions
Deploy / changes (push) Successful in 6s
Deploy / build-backend (push) Has been skipped
Deploy / build-worker (push) Has been skipped
Deploy / build-frontend (push) Has been cancelled
queryFn игнорировал TanStack v5 AbortSignal: first POST /analyze, poll GET
/fetch-status и re-POST не передавали signal, цикл спал на non-abortable
setTimeout без проверки signal.aborted. При unmount / смене cad|horizon цикл
жил до 2-мин кэпа, дёргая GET /fetch-status каждые 2с + POST /analyze на каждый
ready — зомби-поллинг. Проброс signal во все 3 fetch, sleep через shared
abortableSleep (extracted из useSiteAnalysis #1242), throw AbortError перед
итерацией. +3 vitest (abort + happy-path).

Closes #1248
2026-06-13 13:19:50 +00:00
3cf9fad683 fix(backend): экранировать Excel formula-injection (#1244) + увести chat-чтение с event loop (#1245)
Some checks failed
CI / changes (push) Successful in 6s
CI / frontend-tests (push) Has been skipped
CI / changes (pull_request) Successful in 7s
CI / frontend-tests (pull_request) Has been skipped
CI / backend-tests (push) Successful in 6m38s
CI / backend-tests (pull_request) Successful in 6m37s
Deploy / build-backend (push) Blocked by required conditions
Deploy / build-worker (push) Blocked by required conditions
Deploy / build-frontend (push) Blocked by required conditions
Deploy / deploy (push) Blocked by required conditions
Deploy / changes (push) Has been cancelled
#1244 (security): внешние/скрейпинг-строки (comm_name из DOM.РФ, headline/usp_text)
с ведущим = + - @ \t \r писались как есть → openpyxl сохранял как формулы
(data_type='f'), исполнялись при открытии в Excel/LibreOffice. _sanitize_formula
префиксует такие строки апострофом (OWASP CSV-injection escape); числа/даты/bool
не трогаются. _write_kv labels тоже санитизируются. Подтверждено на openpyxl 3.1.5.

#1245 (concurrency): async ask() вызывал sync get_report_for_chat() (sync SQLAlchemy
тянет крупный JSONB §22-отчёт) напрямую — блокировал event loop, в отличие от
LLM-ветки (run_in_threadpool). Обёрнуто в run_in_threadpool.

Closes #1244
Closes #1245
2026-06-13 18:10:21 +05:00
99c9a130bc feat(forecast): расширить горизонт прогноза до 24 мес (ТЗ §12.1)
All checks were successful
CI / changes (push) Successful in 6s
CI / frontend-tests (push) Successful in 46s
CI / changes (pull_request) Successful in 6s
CI / frontend-tests (pull_request) Successful in 46s
CI / backend-tests (push) Successful in 6m35s
CI / backend-tests (pull_request) Successful in 6m30s
Deploy / changes (push) Successful in 5s
Deploy / build-backend (push) Successful in 2m14s
Deploy / build-frontend (push) Successful in 2m55s
Deploy / build-worker (push) Successful in 4m6s
Deploy / deploy (push) Successful in 1m40s
API отвергал ?horizon=24 (422), хотя ТЗ §12.1 называет 6/12/18/24, а движок
УЖЕ считает 24 на каждом ране: _DEFAULT_HORIZONS=(6,12,18,24) во всех 6 точках
стека (orchestrator/forecast-task/demand_supply_forecast/scenarios/
special_indices/report_assembler), PIPELINE_HORIZON_MONTHS=24.
_hidden_release_fraction клампит h/18→1.0 на 24 (без переполнения),
future_supply._horizon_weight расширяет окно чисто — скрытых ≤18 потолков нет.
Чистое расширение валидатора-enum, не новая математика.

Backend: _ALLOWED_FORECAST_HORIZONS → {6,12,18,24}, Query/docstring/error-msg.
Frontend: HorizonSelector HORIZONS=[6,12,18,24] (тип horizon=number, union не нужен;
прочие потребители data-driven через meta.horizons/forecasts_by_horizon).
Тесты: API принимает 24/отвергает 30; движок-тесты доказывают h=24 осмыслен
(поля посчитаны, demand(24)>demand(18), hidden созрел, индексы в диапазонах).

Closes #944 (Q1 горизонт 24)
2026-06-13 17:27:35 +05:00
4454244ac2 chore(tradein-db): drop duplicate listings_geom_active_idx (#730)
All checks were successful
Deploy Trade-In / changes (push) Successful in 6s
Deploy Trade-In / build-frontend (push) Has been skipped
Deploy Trade-In / build-browser (push) Has been skipped
Deploy Trade-In / test (push) Successful in 33s
Deploy Trade-In / build-backend (push) Successful in 24s
Deploy Trade-In / deploy (push) Successful in 39s
После того как 097 снял предикат WHERE is_active с listings_geom_active_idx,
тот стал точным дублем оригинального full listings_geom_idx (002) — оба
GIST(geom) без предиката. listings_geom_idx покрывает нагрузку (active-запросы
добавляют WHERE is_active поверх того же GIST-скана). Освобождает ~3.2 МБ.

Остальное из аудита #730 (5 dup/redundant дропов + 5 is_active де-партиалов)
уже применено миграцией 097 на prod (verify read-only подтвердил: все ABSENT
/ без предиката). Это follow-up на единственный дубль — побочный эффект 097.

Closes #730
2026-06-13 11:53:13 +00:00
6444dc0ebd fix(tradein-frontend): остановить retry-storm на 401 — refetchOnMount:false + backoff (#800)
Some checks failed
CI / backend-tests (push) Has been skipped
CI / frontend-tests (push) Has been skipped
CI / changes (pull_request) Successful in 6s
CI / backend-tests (pull_request) Has been skipped
CI / frontend-tests (pull_request) Has been skipped
CI / changes (push) Successful in 7s
Deploy Trade-In / build-backend (push) Blocked by required conditions
Deploy Trade-In / build-frontend (push) Blocked by required conditions
Deploy Trade-In / build-browser (push) Blocked by required conditions
Deploy Trade-In / test (push) Blocked by required conditions
Deploy Trade-In / deploy (push) Blocked by required conditions
Deploy Trade-In / changes (push) Has been cancelled
На стойком 401/протухшей сессии фронт повторял /me + /trade-in/quota в
бесконечном цикле (184+ запросов): errored query + дефолтный
refetchOnMount=true → каждый новый observer (useMe дёргают RouteGuard,
Topbar, UserMenu, useBrand) ре-фетчил при mount = re-subscribe storm.
Глобально выключаем refetchOnMount/onReconnect в QueryClient + экспоненциальный
retryDelay (cap 30s); те же гарды на useQuota. retry-предикат 4xx→false уже был.
Graceful session-screen (RouteGuard/NoAccessScreen) уже на месте.

Closes #800
2026-06-13 16:47:58 +05:00
e1442d3b1d feat(tradein-scheduler): nightly newbuilding-enrichment schedule (#973)
All checks were successful
CI / changes (pull_request) Successful in 6s
CI / changes (push) Successful in 7s
CI / backend-tests (pull_request) Has been skipped
CI / frontend-tests (pull_request) Has been skipped
CI / backend-tests (push) Has been skipped
CI / frontend-tests (push) Has been skipped
Deploy Trade-In / test (push) Successful in 33s
Deploy Trade-In / build-backend (push) Successful in 1m3s
Deploy Trade-In / changes (push) Successful in 5s
Deploy Trade-In / build-frontend (push) Has been skipped
Deploy Trade-In / build-browser (push) Has been skipped
Deploy Trade-In / deploy (push) Successful in 40s
Регистрирует отдельный in-app scheduler-источник для cian_newbuilding
enrichment-backfill (houses_price_dynamics / house_reliability_checks /
house_reviews), который раньше бежал только инлайн в Cian full-sweep.

- run_newbuilding_enrich() wrapper (heartbeat → backfill → mark_done/failed),
  делегирует backfill_newbuilding_enrichment (#972, уже в main).
- trigger_newbuilding_enrich_run() + dispatch-ветка в scheduler_loop
  (зеркалит trigger_sber_index_pull_run): _claim_run guard, async task,
  zombie-reap наследуется; idempotency наследуется (skip уже обогащённых,
  per-house SAVEPOINT, bounded per-fire limit дренирует backlog 306 домов).
- seed-миграция 103: source='newbuilding_enrich', окно 00:00-01:00 UTC
  (03:00-04:00 МСК, до 01:00+ UTC sweep-блока), limit=25, next_run_at на завтра.

Засеяно enabled=FALSE (dormant): на prod весь external-HTTP scraping намеренно
на паузе. Расписание+триггер готовы, но не стартуют до намеренного возобновления
(UPDATE ... SET enabled=true). Не активирую одиночный источник при общей паузе.

Closes #973
2026-06-13 16:24:03 +05:00
b211183940 feat(site-finder): детерминированная атрибуция застройщика в analyze (#1088)
All checks were successful
CI / changes (pull_request) Successful in 6s
CI / changes (push) Successful in 8s
CI / frontend-tests (push) Has been skipped
CI / frontend-tests (pull_request) Has been skipped
CI / backend-tests (push) Successful in 6m33s
CI / backend-tests (pull_request) Successful in 6m33s
Deploy / changes (push) Successful in 6s
Deploy / build-frontend (push) Has been skipped
Deploy / build-backend (push) Successful in 1m44s
Deploy / build-worker (push) Successful in 5m24s
Deploy / deploy (push) Successful in 1m30s
Сервис get_developer_attribution поверх fn_developer_for_parcel (миграция 149):
топ-1 застройщик участка + track-record (РНС/РВЭ + домрф) + nearby_developers.
Дедуп по норм-ИНН (лучший match_method первым). Wire в analyze additive без флага
(чистый DB-резолвер по индексам, graceful → None). Beat-refresh developer_registry
ежемесячно 05:30 МСК после ekburg-permits. Pydantic-схемы + 18 тестов.

Источники: ekburg_construction_permits (РНС, ИНН) ⋈ domrf_kn_objects по
нормализованному ИНН + spatial/quarter fallback. 68% domrf-застройщиков имеют РНС.
EXPLAIN: point-lookup 0.06ms, резолвер ~30ms (functional/GIST индексы в миграции).

Closes #1088
2026-06-13 16:13:45 +05:00
3df163f741 fix(snapshot-pdf): Unicode '—' вместо HTML-entity '&mdash;' в Jinja or-fallback (#1234)
All checks were successful
CI / changes (pull_request) Successful in 6s
Deploy / changes (push) Successful in 9s
CI / frontend-tests (pull_request) Successful in 55s
Deploy / build-backend (push) Successful in 2m20s
Deploy / build-frontend (push) Successful in 3m25s
Deploy / build-worker (push) Successful in 4m16s
Deploy / deploy (push) Successful in 1m22s
CI / backend-tests (pull_request) Successful in 6m43s
2026-06-13 15:04:12 +05:00
eada9a25ac fix(scrapers): izyatie OCR — '6б:' → '66:' для кад-номеров МСК-66 (#1228) 2026-06-13 15:03:59 +05:00
367ed890f1 fix(objective-sync): log seed failure вместо bare except pass (#1226) 2026-06-13 15:03:34 +05:00
d8a2ad7854 fix(auth): отобразить роль analyst в UserMenu, расширить Role union в useMe (#1237) 2026-06-13 15:03:15 +05:00
49d26b4696 fix(poi_loader): timedelta(days=730) вместо replace(year=-2) → leap-year 29 февраля safe (#1232) 2026-06-13 15:03:08 +05:00
c3c6e2cef1 fix(okn): expand registry coverage + load detail-card name/address (#1159)
okn_objects_sync собирал 428 ОКН из ~798 ЕГРОКН и сохранял raw_props={id}
без названия/адреса — analyze показывал 'ОКН рядом' без указания КАКОГО.

Coverage: расширил _ADDRESS_PASSES с 2 до 11 подстрок (Екатеринбург,
г. Екатеринбург, 7 районов ЕКБ, 2 формы Свердловской обл.). _PASSES =
декартово (address × category). first-wins дедуп сохраняет высшую категорию.

Detail: fetch_okn_detail(source_id) — GET Show/Show?id=, BeautifulSoup
парсит HTML (dl/dt/dd и tr/td), извлекает name, address, protection_category,
object_kind, dating. _enrich_with_detail сливает в raw_props с graceful
failover (cap=1500, exception per-object не валит прогон).

Lookup: parcel_okn_objects возвращает name/address из raw_props->>'name'.
Existing ird_analyze тесты не сломаны.

49 okn-related тестов passed, ruff clean.

Closes #1159
2026-06-13 15:02:50 +05:00
2e428405d2 fix(site-finder): расширить except в parcel_ird_overlaps до DataError (#1095)
parcel_ird_overlaps ловил только (OperationalError, ProgrammingError).
При D9b-wiring в analyze malformed WKT в ST_GeomFromText давал PostGIS
ERROR → SQLAlchemy DataError, который пробивал try/except → analyze
падал вместо graceful-degrade.

- ird_overlay_lookup: + DataError в imports и в except tuple.
- quarter_dump_lookup: симметричное расширение в _get_engineering_*.
- test_ird_overlay_lookup: parametrized test_graceful_when_db_fails
  покрывает 3 класса (Operational/Programming + DataError для WKT).

Closes #1095
2026-06-13 15:02:50 +05:00
d11c8d47e0 fix(workers): per-prefix db.commit() в gknspecial_harvest для durability (#1116)
harvest_gknspecial_zones делал единственный commit в конце всех 8 prefix-
циклов: WorkerLostError (OOM / рестарт хоста) в середине прогона →
~20k записанных строк откатывались, прогресс ноль.

Mirror ird_harvest.py:216 (per-quarter commit): db.commit() после
финального logger.info внутри for-prefix loop. SAVEPOINT-per-row уже
изолирует битые фичи; per-prefix commit добавляет durability между
префиксами. WFS-fail на префиксе → continue → commit не делается;
предыдущие префиксы уже durable.

+2 теста (durability, WFS-fail skip). 29 gknspecial тестов зелёные.

Closes #1116
2026-06-13 15:02:50 +05:00
8a30238564 fix(forecasting): propagate confounded flag DemandSupplyForecast → §15 (#1222)
DemandSupplyForecast.as_dict() не эмитил 'confounded'/'is_confounded_window',
report_assembler._confounded() всегда возвращал False и §15 confounded_window
factor в compute_report_confidence был мёртв: 48-мес окна, пересекающие
2024-07-01 шок никогда не тянули report confidence к 'low' и шок не назывался
в rationale.

Patch: добавлено confounded: bool в DemandSupplyForecast (от §9.5 macro_coef
OR §9.6 rate_sensitivity), exposed в as_dict(). _confounded() уже использовал
.get() defensively — блокер был в producer'е.

+3 теста: contract на real DemandSupplyForecast.as_dict(), end-to-end
assemble_report → confounded_window factor surfaces at level=low, weakest-link
тянет overall к 'low'. 61 report_assembler + 1034 forecasting тестов зелёные.

Closes #1222
2026-06-13 15:02:50 +05:00
fbafb1bf68 fix(landing): mirror Pydantic schema в PilotRequestModal validation (#1240)
Backend PilotRequestInput требует name min_length=2 и др., но modal
проверял только !form.name.trim(). 1-char name ('Я') проходил, backend
возвращал 422, apiFetch клал raw FastAPI JSON в Error.message, modal
рендерил его буквально для public lead form.

Patch: PILOT_LIMITS + PILOT_EMAIL_RE константы (синхр с backend),
per-field onBlur+live валидация с RU-сообщениями, minLength/maxLength
+ aria-invalid/aria-describedby, submit disabled при ошибках,
formatSubmitError → 4xx/5xx → human text. 26 vitest кейсов.

Closes #1240
2026-06-13 15:02:50 +05:00
703d3905b8 fix(site-finder): normalize supply room_bucket vocabulary to velocity side (#1229)
best_layouts._SUPPLY_BATCH_SQL эмитил {studio,euro-1,euro-2,1,2,3,4+},
а _INLINE_VELOCITY_SQL читает {студия,1,2,3,4+} из
objective_corpus_room_month (prod check: 'euro-*' rows отсутствуют).

Эффект: rooms=2 + area<50 уходили в euro-1/euro-2 supply-стороной →
выпадали из знаменателя bucket '2' → sold_pct_of_supply двушек
завышен, is_oversold ложно True. (rb='euro-*') dead lookups в supply_map.

Patch: убраны euro-* WHEN в supply CASE. SF-08 euro-биннинг отложен
до момента когда velocity-сторона начнёт его отдавать. +2 regression
теста (bucket match, string guard). 35 best_layouts тестов зелёные.

Closes #1229
2026-06-13 15:02:50 +05:00
780dd5640c fix(analytics): _district_market_saturation latest-snapshot filter (#1221)
domrf_kn_objects/sales_agg хранят ~3 snapshot'а на obj_id (weekly UPSERT,
retention=none). JOIN без latest-фильтра считал каждый ЖК N раз:
sold_median тянулся старыми perc → sat_factor занижен, гейт n>=5
проходил уже при 2 ЖК (n=6), sat_n завышен ~×N в scope. Сданные ЖК
просачивались через старые снапшоты со статусом 'Строящиеся'.

Patch: AND a.snapshot_date = (SELECT MAX(snapshot_date) FROM ...) для
обеих таблиц (зеркало #1210/#1212). COUNT(*) → COUNT(DISTINCT a.obj_id)
defensive. 29 recommend_mix + 8 saturation тестов зелёные.

Closes #1221
2026-06-13 15:02:50 +05:00
d694e735cd fix(site-finder): per-row SAVEPOINT в noise_loader (#1231)
Loop UPSERT в sync_noise_sources_to_db без begin_nested: один замкнутый
3-точечный natural=water way [A,B,A] даёт POLYGON((A,B,A)), PostGIS
отвергает (< 4 точек в ring) → outer tx rollback + raise → весь weekly
noise/water/utility sync падает, тот же way отравляет каждый прогон.

- Оборачиваем каждый UPSERT в `with db.begin_nested():` + per-row
  try/except → logger.warning + skipped++ (канон pzz_loader.py:111).
- В _way_to_polygon_wkt проверяем итоговое кольцо ≥ 4 точек (fail-safe).
- Outer except: добавлен logger.exception для видимости.

Closes #1231
2026-06-13 15:02:50 +05:00
285e8f974a fix(sf): ekb_ppt_tep post-фильтры + врезка в analyze (#1136)
All checks were successful
Deploy / changes (push) Successful in 6s
Deploy / build-backend (push) Successful in 2m12s
Deploy / build-frontend (push) Has been skipped
Deploy / build-worker (push) Successful in 4m34s
Deploy / deploy (push) Successful in 1m11s
Post-фильтры парсера (по образцу ppt2018_22823):
- Табл.11: отбрасываем строки без zone_name (преамбульные артефакты
  merged-cells давали 2 фантома при 12 реальных зонах).
- Табл.13: dedup по (phase, composition, zone, area_ha) + фильтры
  короткого-без-зоны и нумерационного шума (~364 строк → ~30-40 реальных).

Analyze-врезка: новый ppt_tep_lookup.parcel_ppt_tep — JOIN
planning_projects ⋈ ekb_ppt_tep по doc_ref↔source_key/doc_full_name
(best-effort, без FK). Wired в build_ird_analyze_block рядом с
planning_projects/krt_requisites — DB-источники, graceful.

Seed-URL остаётся placeholder с # VERIFY (ingest пропускает с WARNING).

Tests: 27 в parser (3 новых), 7 lookup, 22 wiring (2 новых).
All ruff/syntax green.

Closes #1136
2026-06-13 09:32:52 +00:00
a5ff281bad fix(admin-jobs): UPSERT в PUT /admin/jobs/settings — не терять изменения (#1223)
All checks were successful
Deploy / changes (push) Successful in 6s
Deploy / build-frontend (push) Has been skipped
Deploy / build-backend (push) Successful in 1m37s
Deploy / build-worker (push) Successful in 2m46s
Deploy / deploy (push) Successful in 1m19s
UPDATE без проверки rowcount затрагивал 0 строк для несуществующего
job_type; get_one возвращал hardcoded _fallback. PUT отвечал 200,
админ считал настройку сохранённой — но она терялась (новый job_type
до seed-миграции, опечатка в path).

Patch: update() проверяет result.rowcount: при 0 делает INSERT с
переданными колонками (непереданные → дефолты таблицы из м.81).
INSERT использует ON CONFLICT (job_type) DO NOTHING + повторный
UPDATE как защита от гонки. Поведение для существующих строк не
изменилось. 8 новых тестов (6 service + 2 API).

Closes #1223
2026-06-13 08:36:23 +00:00