Compare commits

..

527 commits

Author SHA1 Message Date
698576e77b fix(tradein/scraper): per-city SERP location for oblast city-sweep (#12) (#2525)
All checks were successful
Deploy Trade-In / build-browser (push) Has been skipped
Deploy Trade-In / build-backend (push) Successful in 1m37s
Deploy Trade-In / changes (push) Successful in 10s
Deploy Trade-In / build-frontend (push) Has been skipped
Deploy Trade-In / test (push) Successful in 5m1s
Deploy Trade-In / deploy (push) Successful in 6m47s
Co-authored-by: lekss361 <lekss361@gendsgn.local>
Co-committed-by: lekss361 <lekss361@gendsgn.local>
2026-07-13 21:41:07 +00:00
67b50bf713 feat(tradein/ui): admin links to audit + analytics in UserMenu (#2524)
All checks were successful
Deploy Trade-In / build-frontend (push) Successful in 1m54s
Deploy Trade-In / changes (push) Successful in 11s
Deploy Trade-In / test (push) Has been skipped
Deploy Trade-In / build-browser (push) Has been skipped
Deploy Trade-In / build-backend (push) Has been skipped
Deploy Trade-In / deploy (push) Successful in 55s
Admin-only «Аудит доступа» → /admin/audit and «Активность» → /admin/analytics links in the UserMenu dropdown. Navigates via API_BASE_URL prefix (Topbar convention, basePath-safe). Pilot menu unchanged.
2026-07-13 21:16:29 +00:00
c61ed5a96e fix(tradein/avito): catch sidecar timeout/transport errors in SERP fetch → graceful banned, not crash (#2523)
All checks were successful
Deploy Trade-In / changes (push) Successful in 10s
Deploy Trade-In / build-frontend (push) Has been skipped
Deploy Trade-In / build-backend (push) Successful in 1m40s
Deploy Trade-In / build-browser (push) Has been skipped
Deploy Trade-In / test (push) Successful in 4m50s
Deploy Trade-In / deploy (push) Successful in 7m48s
Co-authored-by: bot-backend <bot-backend@gendsgn.local>
Co-committed-by: bot-backend <bot-backend@gendsgn.local>
2026-07-13 20:59:39 +00:00
e61c2debe5 fix(tradein/audit): exclude /api/v1/admin/* from api_request events (#2522)
Some checks failed
Deploy Trade-In / deploy (push) Blocked by required conditions
Deploy Trade-In / build-backend (push) Blocked by required conditions
Deploy Trade-In / changes (push) Successful in 11s
Deploy Trade-In / build-frontend (push) Has been skipped
Deploy Trade-In / build-browser (push) Has been skipped
Deploy Trade-In / test (push) Has been cancelled
Admin dashboard traffic to /api/v1/admin/* no longer logged as api_request (would pollute behavior analytics); login events kept for IP audit.
2026-07-13 20:55:16 +00:00
c904fbf94e feat(tradein/admin): audit + activity dashboards (Feature 2/3 UI) (#2521)
All checks were successful
Deploy Trade-In / changes (push) Successful in 10s
Deploy Trade-In / test (push) Has been skipped
Deploy Trade-In / build-browser (push) Has been skipped
Deploy Trade-In / build-backend (push) Has been skipped
Deploy Trade-In / build-frontend (push) Successful in 1m56s
Deploy Trade-In / deploy (push) Successful in 46s
Admin-only /admin/audit (accounts × IP/device × searches) and /admin/analytics (KPIs, daily chart, top searches/paths/accounts) over the user_events read API. Gated like /scrapers; 403 → NoAccessScreen.
2026-07-13 20:45:34 +00:00
9fd6396fbc feat(tradein/admin): audit + analytics read API over user_events (#2520)
All checks were successful
Deploy Trade-In / build-backend (push) Successful in 57s
Deploy Trade-In / build-browser (push) Has been skipped
Deploy Trade-In / build-frontend (push) Successful in 30s
Deploy Trade-In / test (push) Successful in 4m49s
Deploy Trade-In / deploy (push) Successful in 50s
Deploy Trade-In / changes (push) Successful in 10s
Admin-only read API: GET /admin/audit/accounts, /admin/audit/accounts/{username}, /admin/analytics over user_events. Read-only, empty-safe, RBAC via central gate.
2026-07-13 20:26:58 +00:00
8b7af92ee3 fix(tradein/estimator): displayed analog cards match n_analogs + cross-source dedup (#2518)
Some checks failed
Deploy Trade-In / changes (push) Successful in 10s
Deploy Trade-In / build-browser (push) Has been skipped
Deploy Trade-In / build-frontend (push) Successful in 2m16s
Deploy Trade-In / test (push) Successful in 4m49s
Deploy Trade-In / deploy (push) Failing after 54s
Deploy Trade-In / build-backend (push) Successful in 1m9s
2026-07-13 20:19:20 +00:00
2c5b30f8c9 fix(tradein/auth): RouteGuard session/error screens with retry CTA (#2516)
Some checks failed
Deploy Trade-In / test (push) Blocked by required conditions
Deploy Trade-In / build-backend (push) Blocked by required conditions
Deploy Trade-In / build-frontend (push) Blocked by required conditions
Deploy Trade-In / build-browser (push) Blocked by required conditions
Deploy Trade-In / deploy (push) Blocked by required conditions
Deploy Trade-In / changes (push) Has been cancelled
Co-authored-by: bot-frontend <bot-frontend@gendsgn.local>
Co-committed-by: bot-frontend <bot-frontend@gendsgn.local>
2026-07-13 20:19:12 +00:00
d530602fbd fix(tradein/quota): per-user quota override + no-burn on empty estimate (#2517)
Some checks failed
Deploy Trade-In / build-backend (push) Blocked by required conditions
Deploy Trade-In / deploy (push) Blocked by required conditions
Deploy Trade-In / changes (push) Successful in 11s
Deploy Trade-In / build-browser (push) Has been skipped
Deploy Trade-In / build-frontend (push) Has been skipped
Deploy Trade-In / test (push) Has been cancelled
Co-authored-by: bot-backend <bot-backend@gendsgn.local>
Co-committed-by: bot-backend <bot-backend@gendsgn.local>
2026-07-13 20:18:42 +00:00
32037af6de feat(tradein/audit): request-audit middleware + estimate_request logging → user_events (#2519)
All checks were successful
Deploy Trade-In / changes (push) Successful in 11s
Deploy Trade-In / build-browser (push) Has been skipped
Deploy Trade-In / build-frontend (push) Has been skipped
Deploy Trade-In / test (push) Successful in 48s
Deploy Trade-In / build-backend (push) Successful in 1m0s
Deploy Trade-In / deploy (push) Successful in 51s
RequestAuditMiddleware logs api_request + deduped login per authenticated /api/* request; estimate() logs estimate_request with address. Fire-and-forget, never blocks/breaks the request. Writes to user_events (Feature 2/3 capture).
2026-07-13 20:09:22 +00:00
5647c2b40e feat(tradein/db): user_events table — unified login/IP audit + behavior event log (#2515)
All checks were successful
Deploy Trade-In / changes (push) Successful in 11s
Deploy Trade-In / build-frontend (push) Has been skipped
Deploy Trade-In / build-browser (push) Has been skipped
Deploy Trade-In / test (push) Successful in 48s
Deploy Trade-In / build-backend (push) Successful in 29s
Deploy Trade-In / deploy (push) Successful in 52s
Append-only user_events table (event_type, username, ip_address inet, user_agent, path, method, estimate_id, payload jsonb, created_at) + 4 indexes. Foundation for login/IP audit + behavior analytics. Resurrects dropped audit_log design.
2026-07-13 19:47:26 +00:00
749c96cd0c feat(tradein/ui): Telegram feedback button in Topbar (env-gated) (#2514)
All checks were successful
Deploy Trade-In / test (push) Has been skipped
Deploy Trade-In / build-browser (push) Has been skipped
Deploy Trade-In / build-backend (push) Has been skipped
Deploy Trade-In / changes (push) Successful in 12s
Deploy Trade-In / build-frontend (push) Successful in 2m1s
Deploy Trade-In / deploy (push) Successful in 47s
Env-gated Telegram feedback button next to UserMenu; renders only when NEXT_PUBLIC_FEEDBACK_TG_URL is set. safeUrl() sanitized, outside RBAC nav.
2026-07-13 19:44:00 +00:00
987dce11bc fix(tradein/pdf): МЕРА branding + 4-page report, no empty pages (#2513)
All checks were successful
Deploy Trade-In / changes (push) Successful in 12s
Deploy Trade-In / build-frontend (push) Has been skipped
Deploy Trade-In / build-browser (push) Has been skipped
Deploy Trade-In / test (push) Successful in 52s
Deploy Trade-In / build-backend (push) Successful in 1m5s
Deploy Trade-In / deploy (push) Successful in 54s
2026-07-13 18:04:07 +00:00
b5477362f0 fix(tradein/auth): restore brusnika (user2) pilot access (#2512)
All checks were successful
Deploy Trade-In / deploy (push) Successful in 1m7s
Deploy / changes (push) Successful in 15s
Deploy / build-frontend (push) Has been skipped
Deploy Trade-In / changes (push) Successful in 22s
Deploy Trade-In / build-frontend (push) Has been skipped
Deploy Trade-In / build-browser (push) Has been skipped
Deploy / build-worker (push) Successful in 53s
Deploy / build-backend (push) Successful in 54s
Deploy Trade-In / test (push) Successful in 1m20s
Deploy Trade-In / build-backend (push) Successful in 43s
Deploy / deploy (push) Successful in 1m56s
2026-07-13 15:33:36 +00:00
60c21ae2a5 feat(tradein/lead): persist 152-ФЗ proof-of-consent (LOW audit R2 #9) (#2511)
All checks were successful
Deploy Trade-In / build-browser (push) Has been skipped
Deploy Trade-In / changes (push) Successful in 15s
Deploy Trade-In / test (push) Successful in 58s
Deploy Trade-In / build-backend (push) Successful in 1m30s
Deploy Trade-In / build-frontend (push) Successful in 32s
Deploy Trade-In / deploy (push) Successful in 1m20s
2026-07-12 22:13:36 +00:00
3bd0873230 fix(tradein/scheduler): re-enable deactivate_stale_domklik now that domclick sweeps run (LOW audit R2 #10) (#2510)
Some checks failed
Deploy Trade-In / deploy (push) Blocked by required conditions
Deploy Trade-In / changes (push) Successful in 15s
Deploy Trade-In / build-browser (push) Has been skipped
Deploy Trade-In / test (push) Successful in 1m1s
Deploy Trade-In / build-frontend (push) Has been cancelled
Deploy Trade-In / build-backend (push) Has been cancelled
2026-07-12 22:12:12 +00:00
5e73b35206 fix(tradein/location-coef): re-center coefficient band so a strong central location ~= 1.0 (LOW audit R2 #7a) (#2509)
Some checks failed
Deploy Trade-In / build-frontend (push) Blocked by required conditions
Deploy Trade-In / build-browser (push) Blocked by required conditions
Deploy Trade-In / deploy (push) Blocked by required conditions
Deploy Trade-In / test (push) Blocked by required conditions
Deploy Trade-In / build-backend (push) Blocked by required conditions
Deploy Trade-In / changes (push) Has been cancelled
2026-07-12 22:12:10 +00:00
69ca746673 fix(tradein/v2): location coefficient informational + price-history clarity (LOW audit R2 #7c/#8) (#2508)
Some checks failed
Deploy Trade-In / build-backend (push) Blocked by required conditions
Deploy Trade-In / deploy (push) Blocked by required conditions
Deploy Trade-In / changes (push) Successful in 10s
Deploy Trade-In / build-browser (push) Has been skipped
Deploy Trade-In / build-frontend (push) Has been cancelled
Deploy Trade-In / test (push) Has been cancelled
2026-07-12 22:11:08 +00:00
2cd24ecfbc fix(tradein/scrapers): clamp implausible listing/publish dates (LOW audit R2 #6) (#2507)
Some checks failed
Deploy Trade-In / test (push) Blocked by required conditions
Deploy Trade-In / build-backend (push) Blocked by required conditions
Deploy Trade-In / build-frontend (push) Blocked by required conditions
Deploy Trade-In / build-browser (push) Blocked by required conditions
Deploy Trade-In / deploy (push) Blocked by required conditions
Deploy Trade-In / changes (push) Has been cancelled
2026-07-12 22:11:06 +00:00
92371e7d9e fix(tradein/pdf): scale offer-page rent by room count (LOW audit R2 #5) (#2506)
Some checks failed
Deploy Trade-In / test (push) Blocked by required conditions
Deploy Trade-In / build-backend (push) Blocked by required conditions
Deploy Trade-In / build-frontend (push) Blocked by required conditions
Deploy Trade-In / build-browser (push) Blocked by required conditions
Deploy Trade-In / deploy (push) Blocked by required conditions
Deploy Trade-In / changes (push) Has been cancelled
2026-07-12 22:11:04 +00:00
7fa52bb3ad fix(tradein/estimator): honest repair-note + ratio_basis label; doc EKB-calibrated constants (LOW audit R2 #1-4) (#2505)
Some checks failed
Deploy Trade-In / test (push) Blocked by required conditions
Deploy Trade-In / build-backend (push) Blocked by required conditions
Deploy Trade-In / build-frontend (push) Blocked by required conditions
Deploy Trade-In / build-browser (push) Blocked by required conditions
Deploy Trade-In / deploy (push) Blocked by required conditions
Deploy Trade-In / changes (push) Has been cancelled
2026-07-12 22:11:03 +00:00
348be445e3 fix(tradein/geocoder): gate EKB-local tiers for non-EKB oblast addresses (#11)
All checks were successful
Deploy Trade-In / build-frontend (push) Has been skipped
Deploy Trade-In / build-browser (push) Has been skipped
Deploy Trade-In / changes (push) Successful in 12s
Deploy Trade-In / test (push) Successful in 51s
Deploy Trade-In / build-backend (push) Successful in 1m3s
Deploy Trade-In / deploy (push) Successful in 5m52s
Non-EKB Свердловская-область addresses no longer snap into Екатеринбург.

Root cause: geocode()/suggest() ran EKB-only local tiers (geoportal/cadastral) before the oblast-aware external providers, with no city gate — a non-EKB street+house colliding with an EKB building resolved to EKB coords. Fix gates the local tiers via _names_non_ekb_city (reuses SVERDLOVSK_OBLAST_CITIES, word-boundary + district-prefix exclusion + \bекатеринбург\b short-circuit). EKB path byte-identical. Deep-review APPROVE, CI green, live-verified (Nominatim returns correct Н.Тагил coords once local tiers skip).

Follow-up (main): geocode_cache purge of ~323 poisoned rows.
2026-07-12 21:08:49 +00:00
bot-backend
8fdcd8df62 fix(tradein/geocoder): word-boundary EKB short-circuit + gate suggest() + gazetteer (#11)
All checks were successful
CI / changes (pull_request) Successful in 7s
CI Trade-In / changes (pull_request) Successful in 8s
CI Trade-In / frontend-checks (pull_request) Has been skipped
CI / backend-tests (pull_request) Has been skipped
CI / frontend-tests (pull_request) Has been skipped
CI / openapi-codegen-check (pull_request) Has been skipped
CI Trade-In / backend-tests (pull_request) Successful in 56s
Deep-review fast-follows on the non-EKB city gate:

1. Word-boundary the "екатеринбург" short-circuit in _names_non_ekb_city
   (new _EKATERINBURG_RE = \bекатеринбург\b, .search() instead of substring
   `in`). A bare-substring check matched "Екатеринбургское шоссе" (a real
   street in satellite towns like Pervouralsk) as if it named EKB, wrongly
   keeping EKB-only local tiers on for a non-EKB address. Still returns False
   for the existing homonym false-positive test case (word boundary is
   satisfied there — "екатеринбург" appears as a standalone token).

2. Gate suggest()'s Tier 1 (same root cause as geocode()): the EKB-only
   cadastral matchers _cadastral_house_match / _cadastral_forward_sync ran
   unconditionally, so a non-EKB oblast autocomplete query could surface an
   EKB building via street+house collision. Gated behind
   `not _names_non_ekb_city(query)`, mirroring geocode()'s use_local_ekb.
   External suggest tiers (DaData/Yandex/Nominatim) untouched — still the
   path for non-EKB autocomplete.

3. Extended SVERDLOVSK_OBLAST_CITIES with unambiguous, deal-heavy oblast
   cities: алапаевск, сухой лог, кушва, красноуральск, карпинск, нижняя
   тура, верхний тагил, нижние серги. Deliberately did NOT add "лесной"
   (reviewer flagged as reviewer-optional, left to judgment): DB check
   (ekb_geoportal_buildings) confirms a real EKB street named exactly
   "Лесной" (3 buildings) — an exact whole-word collision indistinguishable
   from ЗАТО Лесной by word-boundary matching alone (unlike "Серова"/"Серов",
   which are different word forms). Adding it would misclassify a bare
   EKB address ("Лесной, 5", no "Екатеринбург" mention) as non-EKB.

Tests: word-boundary EKB regression test, suggest() gating test, two new
gazetteer entries (Верхняя Пышма multi-word, Сухой Лог).
2026-07-13 00:06:19 +03:00
c98a879a79 Merge pull request 'fix(tradein/matching): Tier-2a city-guard для non-ЕКБ coord-less (п.2, defense-in-depth)' (#2502) from fix/tradein-oblast-tier2a-cityguard into main
Some checks failed
Deploy Trade-In / changes (push) Successful in 10s
Deploy Trade-In / build-frontend (push) Has been skipped
Deploy Trade-In / build-browser (push) Has been skipped
Deploy Trade-In / test (push) Successful in 47s
Deploy Trade-In / build-backend (push) Successful in 59s
Deploy Trade-In / deploy (push) Has been cancelled
2026-07-12 21:04:28 +00:00
bot-backend
2ef3e42da1 fix(tradein/geocoder): explicit Екатеринбург wins over oblast-city homonym (#11)
All checks were successful
CI Trade-In / changes (pull_request) Successful in 8s
CI / changes (pull_request) Successful in 8s
CI / backend-tests (pull_request) Has been skipped
CI / frontend-tests (pull_request) Has been skipped
CI Trade-In / backend-tests (pull_request) Successful in 51s
CI Trade-In / frontend-checks (pull_request) Has been skipped
CI / openapi-codegen-check (pull_request) Has been skipped
Prod-cache verification (deep-review) found 3 false-positives: EKB addresses
that literally contain "екатеринбург" but also happen to contain a homonym
of another oblast city inside a neighborhood/posyolok name — ЖК "Заречный"
and пос. Сысерть, both administratively inside Ekaterinburg, not the ZATO
Zarechny / town Sysert they share a name with. These were gated off the
EKB-only local tiers; if Nominatim were momentarily down, a real EKB address
would return None — a latent regression not present before this PR.

Add an explicit-city short-circuit at the top of _names_non_ekb_city: if the
normalized address contains "екатеринбург", return False immediately (EKB
tiers stay on). Cannot affect the true-positive (non-EKB) cases, which never
contain "екатеринбург".
2026-07-12 23:50:49 +03:00
bot-backend
720cee2c8e fix(tradein/geocoder): gate EKB-local tiers for non-EKB oblast addresses (#11)
All checks were successful
CI Trade-In / frontend-checks (pull_request) Has been skipped
CI / backend-tests (pull_request) Has been skipped
CI / frontend-tests (pull_request) Has been skipped
CI / openapi-codegen-check (pull_request) Has been skipped
CI Trade-In / backend-tests (pull_request) Successful in 53s
CI / changes (pull_request) Successful in 8s
CI Trade-In / changes (pull_request) Successful in 9s
geocode() ran EKB-only local tiers (2a _geoportal_house_match on
ekb_geoportal_buildings, 2c _cadastral_house_match, 2d
_cadastral_forward_sync — all strictly EKB or EKB-dominated) BEFORE the
already oblast-aware external providers (Yandex/Nominatim via OBLAST66_BBOX
+ region cross-check, shipped in c0cbdc2f). _parse_street_house drops the
city, so any non-EKB oblast street+house that collides with an EKB building
(e.g. "проспект Ленина 1" exists in both Nizhny Tagil and EKB) short-circuited
to EKB coordinates.

Add _names_non_ekb_city() — reuses the existing SVERDLOVSK_OBLAST_CITIES
gazetteer (minus Ekaterinburg) and word-boundary matching from
_has_oblast_marker/_DISTRICT_PREFIXES — to detect when an address explicitly
names a different oblast city. Gate tiers 2a/2c/2d behind
`use_local_ekb = not _names_non_ekb_city(address)`; EKB/bare addresses keep
the exact same code path (byte-identical ordering/logic, only additive
gating). Non-EKB addresses fall straight through to Yandex/Nominatim, which
already handle oblast-wide geocoding correctly.

Verified gendesign_cad_buildings is 99.85% EKB-scoped (47043/47111 rows);
ekb_geoportal_buildings is 100% EKB by construction — gating loses no
meaningful local coverage for non-EKB cities.

Follow-up (not in this PR): geocode_cache has rows poisoned by the old
behavior (non-EKB addresses cached with EKB coordinates) — needs a purge.
2026-07-12 23:45:40 +03:00
73f932d60a Merge pull request 'feat(tradein): sber-index freshness monitor (п.1 observability)' (#2503) from fix/tradein-sber-freshness-monitor into main
All checks were successful
Deploy Trade-In / changes (push) Successful in 9s
Deploy Trade-In / build-frontend (push) Has been skipped
Deploy Trade-In / build-browser (push) Has been skipped
Deploy Trade-In / test (push) Successful in 47s
Deploy Trade-In / build-backend (push) Successful in 59s
Deploy Trade-In / deploy (push) Successful in 1m26s
2026-07-12 20:31:19 +00:00
e7c388dde7 Merge pull request 'refactor(tradein): консолидировать corridor/street-deals city-scope на _resolve_target_city (п.3)' (#2501) from fix/tradein-corridor-dedup-scope into main
Some checks failed
Deploy Trade-In / build-browser (push) Has been cancelled
Deploy Trade-In / build-backend (push) Has been cancelled
Deploy Trade-In / deploy (push) Has been cancelled
Deploy Trade-In / changes (push) Successful in 10s
Deploy Trade-In / build-frontend (push) Has been skipped
Deploy Trade-In / test (push) Has been cancelled
2026-07-12 20:31:05 +00:00
bot-backend
07275c3c97 feat(tradein): монитор свежести СберИндекса (audit п.1)
All checks were successful
CI Trade-In / changes (pull_request) Successful in 8s
CI / changes (pull_request) Successful in 7s
CI / backend-tests (pull_request) Has been skipped
CI / frontend-tests (pull_request) Has been skipped
CI / openapi-codegen-check (pull_request) Has been skipped
CI Trade-In / frontend-checks (pull_request) Has been skipped
CI Trade-In / backend-tests (pull_request) Successful in 51s
Добавляет sber_freshness_monitor по образцу deals_freshness_monitor:
staleness данных СберИндекса теперь видна на MONITOR-частоте, а не тонет
в per-estimate warning'ах estimator._load_sber_index_series (#audit-5a).

- app/tasks/sber_freshness_monitor.py: чистая evaluate_sber_freshness()
  (frozen-now, без БД) + check_sber_freshness() (один SELECT
  max(period_month) вторичного сегмента по региону, #R2-H1 фильтр как в
  эстиматоре; WARNING при stale, mark_done при алерте — это монитор, не
  сбой; mark_failed только при пустой таблице).
- app/services/product_handlers.py: _job_sber_freshness_monitor + Handler
  в build_product_handlers (run_in_executor, как deals-монитор).
- data/sql/180_seed_sber_freshness_monitor.sql: seed scrape_schedules
  (enabled, daily 09:00-10:00 UTC, lag_allowance_days=25).
- tests/test_sber_freshness_monitor.py: frozen-now (fresh/stale/границы) +
  FakeDB (fresh/stale/empty/кастомный lag) + свойства миграции + registry.

Порог алерта: sber_index_max_age_days (35) + lag_allowance (25) = 60д.
+25 — запас на инхерентный лаг публикации источника (1-2 мес), чтобы не
шуметь на штатном отставании. Прод 2026-07-12: max=2026-05-01, age=72д >
60 → alert=1.
2026-07-12 23:27:35 +03:00
bot-backend
d92ff5c3d3 fix(tradein/oblast): Tier-2a guard — coord-less non-ЕКБ карточка не матчит ЕКБ-alias
All checks were successful
CI Trade-In / changes (pull_request) Successful in 7s
CI / changes (pull_request) Successful in 7s
CI Trade-In / frontend-checks (pull_request) Has been skipped
CI Trade-In / backend-tests (pull_request) Successful in 52s
CI / backend-tests (pull_request) Has been skipped
CI / frontend-tests (pull_request) Has been skipped
CI / openapi-codegen-check (pull_request) Has been skipped
Follow-up к #2500: закрывает Tier-2a-дыру, которую geo-guard Tier-2b не доставал.
Coord-less карточка, чей адрес РАЗРЕШАЕТ non-ЕКБ город обл.66 (resolve_city_token),
больше не матчит глобально-уникальный alias (ни по fingerprint, ни по
normalized_address) — иначе non-ЕКБ карточка баккетилась бы в одноимённый ЕКБ-дом и
корраптила ЕКБ-данные. При срабатывании guard'а обе alias-выборки пропускаются →
fall-through в New house (Tier-3 coord-gated, тоже пропускается).

normalize.py: + resolve_city_token() (город обл.66 или None), + EKB_CITY_TOKEN;
has_city_token переиспользует resolve_city_token.

EKB happy-path байт-в-байт: guard срабатывает ТОЛЬКО когда адрес называет non-ЕКБ
город И нет координат. ЕКБ-карточки (resolved city = екатеринбург) и доминирующие
bare/city-less coord-less карточки Avito (resolved None) идут Tier-2a/2b как раньше.

ВАЖНО (документировано в коде и отчёте): реальные Avito SERP-адреса — bare (без
city-токена; 2% из 5037 avito-alias'ов несут 'екатеринбург', 0% — oblast). Значит для
bare oblast-карточки resolve_city_token=None и guard дремлет: полное закрытие
bare-Tier-2a-остатка требует sweep-context/city-keyed aliases — отдельный follow-up,
вне scope, актуален лишь при включённом oblast-sweep. Zero prod-impact сегодня.
2026-07-12 23:26:01 +03:00
bot-backend
d31b1e01ee refactor(tradein): консолидировать corridor/street-deals city-scope на _resolve_target_city (п.3)
All checks were successful
CI Trade-In / changes (pull_request) Successful in 8s
CI Trade-In / backend-tests (pull_request) Successful in 51s
CI / changes (pull_request) Successful in 7s
CI Trade-In / frontend-checks (pull_request) Has been skipped
CI / backend-tests (pull_request) Has been skipped
CI / frontend-tests (pull_request) Has been skipped
CI / openapi-codegen-check (pull_request) Has been skipped
#2488 (_resolve_target_city + LOWER(d.city)=:target_city) смёржен НА ВЕРХ моего #2489
(substring :address ILIKE '%'||d.city||'%') → в _fetch_dkp_corridor образовался
двойной city-фильтр (benign, но избыточный + латентный ё/е edge в substring).

- estimator._fetch_dkp_corridor: убран мой substring-фильтр, оставлен #2488
  _resolve_target_city (словарь ~30 городов обл.66 вкл. ЕКБ + все sweep-города;
  city=None → фильтр не применяется — прежнее #2488-поведение). Убран unused :address bind.
- api/v1/trade_in.get_street_deals: substring заменён на тот же _resolve_target_city
  паттерн (консистентность; #2488 не трогал street-deals). city_filter — литерал,
  значение bind-параметром (не инъекция).

Live-verified: Ленина 2к ЕКБ → median 118052, 1 город (байт-в-байт как substring).
Regression-гейт байт-зелёный (коридор заморожен в фикстуре), 53 теста, ruff clean.
2026-07-12 23:18:02 +03:00
51473fc841 Merge pull request 'fix(tradein): oblast avito-sweep city-slug + Tier-2b geo-guard (R2, latent)' (#2500) from fix/tradein-scrapers-oblast into main
All checks were successful
Deploy Trade-In / changes (push) Successful in 13s
Deploy Trade-In / build-frontend (push) Successful in 2m32s
Deploy Trade-In / deploy (push) Successful in 1m37s
Deploy Trade-In / build-browser (push) Has been skipped
Deploy Trade-In / test (push) Successful in 59s
Deploy Trade-In / build-backend (push) Successful in 1m47s
2026-07-12 19:42:29 +00:00
c3d0f86f37 Merge pull request 'fix(tradein/v2): honesty + a11y — histogram outlier-guard, scatter-релейбл, mobile-notice, keyboard-nav (R2)' (#2499) from fix/tradein-frontend-honesty into main
Some checks failed
Deploy Trade-In / changes (push) Has been cancelled
Deploy Trade-In / test (push) Has been cancelled
Deploy Trade-In / deploy (push) Has been cancelled
Deploy Trade-In / build-backend (push) Has been cancelled
Deploy Trade-In / build-frontend (push) Has been cancelled
Deploy Trade-In / build-browser (push) Has been cancelled
2026-07-12 19:42:15 +00:00
bot-backend
219cdfdbcc fix(tradein/v2): честность гистограмм и оси scatter + мобильный guard + a11y графика
All checks were successful
CI Trade-In / changes (pull_request) Successful in 13s
CI / backend-tests (pull_request) Has been skipped
CI Trade-In / backend-tests (pull_request) Has been skipped
CI / changes (pull_request) Successful in 13s
CI / frontend-tests (pull_request) Has been skipped
CI / openapi-codegen-check (pull_request) Has been skipped
CI Trade-In / frontend-checks (pull_request) Successful in 1m23s
- Гистограммы флагман-карточки (ОБЪЯВЛЕНИЯ) и ДКП/сделок биннятся по
  outlier-guarded пулу (guardPriceOutliers().clean) — консистентно с числом
  «выброс исключён» и спредом на той же карточке. Раньше bins8 спанил
  [min,max] ВКЛЮЧАЯ выброс 872k, схлопывая реальные аналоги в левые бины.
- Ось детального scatter «СРОК ПРОДАЖИ» → «СРОК НА РЫНКЕ»: для активных лотов
  x = дней на рынке (today−listing_date, нижняя граница), а не измеренный
  срок продажи. Обновлены title/aria-label/подзаголовок карточки.
- <1024px: honest desktop-only notice вместо нечитаемого ~25%-scaled артборда
  (client-side width guard, ранний return после всех хуков; десктоп не тронут).
- WCAG 2.1.1: точки истории цен теперь фокусируемы с клавиатуры
  (role=button / tabIndex / Enter+Space / onFocus-onBlur), мышиный hover сохранён.
2026-07-12 22:38:29 +03:00
bot-backend
bf4ce1ba36 fix(tradein/oblast): city-slug в avito SERP + geo/city guard в Tier-2b матчинга
All checks were successful
CI Trade-In / changes (pull_request) Successful in 12s
CI / changes (pull_request) Successful in 13s
CI / frontend-tests (pull_request) Has been skipped
CI Trade-In / backend-tests (pull_request) Successful in 1m3s
CI Trade-In / frontend-checks (pull_request) Has been skipped
CI / backend-tests (pull_request) Has been skipped
CI / openapi-codegen-check (pull_request) Has been skipped
#2487 avito oblast per-city sweep был dead-on-arrival: _parse_html дропал все
карточки без /ekaterinburg/ в source_url, поэтому oblast-sweep по городу вне ЕКБ
терял 100% выдачи. Kept-slug теперь параметризуется через AvitoScraper.target_city_slug
(дефолт "ekaterinburg" — ЕКБ-поведение неизменно), прокинут scheduler →
run_avito_city_sweep → AvitoScraper. avito_serp_ekb_only НЕ отключается.

Bug #2 (oblast bare-street mis-bucket): голые адреса 'ул. Ленина 100' в Н.Тагиле
мис-баккетились в одноимённые ЕКБ-дома через Tier-2b (match по глобально-уникальному
normalized_address без geo/city guard). Добавлен coarse-guard: coords present →
ST_DWithin ≤3км до geom дома; coords absent → требуется city-token (self-
disambiguating); иначе skip → консервативно New house вместо мис-матча. _insert_alias
при коллизии normalized_address больше не перебивает fingerprint чужого дома (иначе
guard деградировал бы в Tier-2a-дыру).

Оба дефекта латентные (oblast per-city schedules отключены) — правка делает
capability безопасной к включению, без влияния на прод сегодня.
2026-07-12 22:38:11 +03:00
519b482cdd Merge pull request 'fix(tradein/rbac): энфорсить roles.yaml scope на всех non-admin путях (R2 H3 security)' (#2498) from fix/tradein-rbac-scope into main
All checks were successful
Deploy Trade-In / changes (push) Successful in 11s
Deploy Trade-In / build-frontend (push) Has been skipped
Deploy Trade-In / build-browser (push) Has been skipped
Deploy Trade-In / test (push) Successful in 54s
Deploy Trade-In / deploy (push) Successful in 1m2s
Deploy Trade-In / build-backend (push) Successful in 1m12s
2026-07-12 19:28:38 +00:00
e4bb2fc2e3 Merge pull request 'fix(tradein/lead): phone digit-guard, убрать dead landing-литерал, 152-ФЗ audit (R2)' (#2497) from fix/tradein-lead-hardening into main
Some checks failed
Deploy Trade-In / build-frontend (push) Blocked by required conditions
Deploy Trade-In / build-browser (push) Blocked by required conditions
Deploy Trade-In / deploy (push) Blocked by required conditions
Deploy Trade-In / test (push) Blocked by required conditions
Deploy Trade-In / build-backend (push) Blocked by required conditions
Deploy Trade-In / changes (push) Has been cancelled
2026-07-12 19:28:26 +00:00
bot-backend
85321d1d1a fix(tradein/rbac): энфорсить roles.yaml scope на всех non-admin путях (R2 H3)
All checks were successful
CI / changes (pull_request) Successful in 8s
CI Trade-In / changes (pull_request) Successful in 8s
CI Trade-In / frontend-checks (pull_request) Has been skipped
CI / backend-tests (pull_request) Has been skipped
CI / frontend-tests (pull_request) Has been skipped
CI / openapi-codegen-check (pull_request) Has been skipped
CI Trade-In / backend-tests (pull_request) Successful in 52s
rbac_guard гейтил только /api/v1/admin/* → revoked (role=expired, roles.yaml
paths:[] deny:/**) и узко-скоупленные аккаунты сохраняли полный non-admin
API-доступ (напр. POST /api/v1/search — экспорт листингов + estimate-quota).
is_path_allowed (roles.yaml paths/deny) существовал, но НЕ вызывался (0 callers).

Fix: после admin-гейта вызываем is_path_allowed(role, external_path) для всех
non-bootstrap путей. roles.yaml globs — внешние (Caddy срезал /trade-in), поэтому
восстанавливаем внешний путь (_EXTERNAL_PREFIX + path). Bootstrap-пути /me и
/brand/* исключены — expired ДОЛЖЕН получить role=expired через /me (trial-экран)
и брендинг; без исключения trial-UX сломался бы (expired paths:[] → 403 на /me).
На сбой парса — fail-open + громкий лог (не лочим платящего pilot из-за конфиг-бага).

roles.yaml НЕ меняю — только энфорсю уже задекларированную политику. pilot/admin/
analyst доступ сохранён (verified), expired теперь 403 на non-admin API.
Тесты: +5 scope-кейсов (expired denied search/trade-in, allowed me/brand; pilot/
admin/analyst preserved) — 31 passed. Тест-harness middleware — mirror, обновлён.
2026-07-12 22:26:14 +03:00
b1ac551451 Merge pull request 'fix(tradein): ingest/scheduler integrity — year sanity, propagate geo-guard, dkp_import counters + DO UPDATE (R2)' (#2496) from fix/tradein-ingest-scheduler-integrity into main
All checks were successful
Deploy Trade-In / build-frontend (push) Has been skipped
Deploy Trade-In / test (push) Successful in 50s
Deploy Trade-In / changes (push) Successful in 13s
Deploy Trade-In / build-browser (push) Has been skipped
Deploy Trade-In / build-backend (push) Successful in 1m14s
Deploy Trade-In / deploy (push) Successful in 4m0s
2026-07-12 19:20:10 +00:00
032fd95a7e Merge pull request 'fix(tradein): image_sanitizer — reject decompression-bomb до декода (R2 security)' (#2495) from fix/tradein-image-dos into main
Some checks failed
Deploy Trade-In / build-backend (push) Blocked by required conditions
Deploy Trade-In / deploy (push) Blocked by required conditions
Deploy Trade-In / changes (push) Successful in 12s
Deploy Trade-In / build-frontend (push) Has been skipped
Deploy Trade-In / build-browser (push) Has been skipped
Deploy Trade-In / test (push) Has been cancelled
2026-07-12 19:19:39 +00:00
bot-backend
0583fb0efd fix(trade-in): усилить валидацию /lead и почистить контракт (#2376)
All checks were successful
CI Trade-In / changes (pull_request) Successful in 12s
CI / changes (pull_request) Successful in 9s
CI / backend-tests (pull_request) Has been skipped
CI / frontend-tests (pull_request) Has been skipped
CI / openapi-codegen-check (pull_request) Has been skipped
CI Trade-In / frontend-checks (pull_request) Has been skipped
CI Trade-In / backend-tests (pull_request) Successful in 58s
- [MED] phone: маска r"^[+]?[\d\s().-]{5,32}$" делала цифры опциональными
  ("(()) -- .." проходило) — добавлен field_validator, требующий 10-15
  реальных цифр (re.sub(r"\D","")), формат-толерантность сохранена.
- [MED] source: убран мёртвый литерал "landing" из Literal — воронка
  недостижима (rbac_guard закрывает /lead 401 без X-Authenticated-User,
  публичного лендинг-роута нет), контракт больше не обещает невозможное.
- [LOW] 152-ФЗ: захват client_ip (X-Forwarded-For -> peer) и
  _CONSENT_POLICY_VERSION в audit-лог; live-схема trade_in_leads не имеет
  колонок под IP/policy/consent-snapshot -> TODO на follow-up миграцию
  (out of scope), durable proof-of-consent gap задокументирован.

Тесты: +digit-free/too-few-digits phone 422, +source=landing 422,
estimate-тест переведён на source=result. 9/9 pass, ruff clean.
2026-07-12 22:19:20 +03:00
a3dfa2a99f Merge pull request 'fix(tradein/pdf): честность отчёта — убрать фейк-сроки, хардкод EKБ, противоречивый дисконт (R2)' (#2494) from fix/tradein-pdf-honesty into main
Some checks failed
Deploy Trade-In / build-backend (push) Blocked by required conditions
Deploy Trade-In / build-frontend (push) Has been skipped
Deploy Trade-In / deploy (push) Blocked by required conditions
Deploy Trade-In / changes (push) Successful in 13s
Deploy Trade-In / build-browser (push) Has been skipped
Deploy Trade-In / test (push) Has been cancelled
2026-07-12 19:19:08 +00:00
bot-backend
20bdc7dec8 test(tradein/pdf): обновить insufficient-page тест под city-aware № отчёта
All checks were successful
CI / frontend-tests (pull_request) Has been skipped
CI Trade-In / changes (pull_request) Successful in 12s
CI / changes (pull_request) Successful in 12s
CI Trade-In / frontend-checks (pull_request) Has been skipped
CI / backend-tests (pull_request) Has been skipped
CI / openapi-codegen-check (pull_request) Has been skipped
CI Trade-In / backend-tests (pull_request) Successful in 56s
test_insufficient_data_page_contains_address_and_report_num ассертил старый
захардкоженный «EKБ-» (mixed-script баг). Оценка без target_address → город не
распознан → нейтральный префикс «МЕРА» (#pdf-honesty). Обновлён ассерт: «МЕРА-»
присутствует, «EKБ-» отсутствует.
2026-07-12 22:12:42 +03:00
bot-backend
6487813ec5 fix(tradein/ingest): 4 data-integrity дефекта в бэкфилле годов и импорте ДКП
All checks were successful
CI / backend-tests (pull_request) Has been skipped
CI / frontend-tests (pull_request) Has been skipped
CI / openapi-codegen-check (pull_request) Has been skipped
CI Trade-In / backend-tests (pull_request) Successful in 57s
CI Trade-In / changes (pull_request) Successful in 10s
CI / changes (pull_request) Successful in 11s
CI Trade-In / frontend-checks (pull_request) Has been skipped
A. ДОМ.РФ year backfill: авторитетный commission_year больше не экранируется
   невозможным существующим year_built. Валидное существующее значение
   выигрывает (COALESCE-семантика #2013 сохранена), но NULL/impossible
   (< 1850, > текущий+2, 0) заменяется валидным commission_year → zhkh_year.
   parse_int_field получил sanity-гейт [min,max] — мусорный commission_year
   не попадает в staging (7 таких строк в текущем staging).

B. propagate_listings_year: добавлен link-consistency guard. Раньше копировал
   houses.year_built на listings по house_id_fk без проверки — перепривязанный
   FK впрыскивал чужой когортный год. Теперь пропагация только если координаты
   объявления в пределах 500м от геометрии дома (ST_DWithin), либо (без коорд.)
   консервативный address-фоллбек по short_address. Live: блокирует 1 из 10
   текущих кандидатов (>500м mislink).

C. rosreestr_dkp_import: per-row INSERT-ошибки отделены от dedup-skip. Раньше
   except инкрементил тот же batch_skipped, что и легитимный ON CONFLICT — сбой
   маскировался под дедуп и run рапортовал success. Отдельный rows_errored +
   гейт по доле ошибок (> 5% → run FAILED, не silent-green).

D. rosreestr_dkp_import: ON CONFLICT DO NOTHING → DO UPDATE изменяемых сырых
   фактов Росреестра (price/area/rooms/floor/year/deal_date/...), чтобы
   исправленный/переопубликованный квартал обновлялся. Обогащение
   (lat/lon/geom/geocode_tried_at и пр.) не в SET-списке — не затирается.
   IS DISTINCT FROM guard сохраняет идемпотентность resume; RETURNING (xmax=0)
   отличает insert от update (rows_inserted vs rows_updated).
2026-07-12 22:12:31 +03:00
bot-backend
1c9f0e1161 fix(tradein/photos): защита от pixel-flood DoS в image_sanitizer
All checks were successful
CI / changes (pull_request) Successful in 7s
CI Trade-In / changes (pull_request) Successful in 8s
CI Trade-In / frontend-checks (pull_request) Has been skipped
CI / backend-tests (pull_request) Has been skipped
CI / frontend-tests (pull_request) Has been skipped
CI / openapi-codegen-check (pull_request) Has been skipped
CI Trade-In / backend-tests (pull_request) Successful in 51s
sanitize_image декодировал изображение целиком (img.load) ДО ресайза,
а Pillow MAX_IMAGE_PIXELS оставался дефолтным (~89 MP). Хорошо сжимаемый
≤10 MB аплоад с заявленными ~89-178 MP разворачивался в raw RGB буфер на
сотни MB до thumbnail — OOM-kill backend'а (768 MiB, #2214), усиление ×12
слотов на estimate.

Defense in depth:
- _MAX_PIXELS = 40 MP: проверка img.size (из хедера, без декода) ДО img.load;
  превышение → ImageSanitizationError без загрузки полного буфера.
- Image.MAX_IMAGE_PIXELS понижен до 40 MP — Pillow сам бросает
  DecompressionBombError на любом декод-пути в обход явной проверки.
- Явный except Image.DecompressionBombError + re-raise ImageSanitizationError
  (byte/pixel cap) до широкого except — грациозный reject, не 500.
- _MAX_BYTES backstop (endpoint уже режет 10 MB → 413, но сервис standalone).

Контракт caller'а не меняется: reject → ImageSanitizationError → HTTP 400.
Тест tests/services/test_image_sanitizer.py: happy-path, resize, pixel-flood
без декода (load не вызывается), byte-backstop, граница cap, garbage bytes.
2026-07-12 22:12:12 +03:00
0f4ed2f6e9 Merge pull request 'fix(tradein/estimator): СберИндекс только вторичка + confidence dispersion-ceiling (R2 H1+H2)' (#2493) from fix/tradein-estimator-sber-confidence into main
All checks were successful
Deploy Trade-In / changes (push) Successful in 14s
Deploy Trade-In / build-frontend (push) Has been skipped
Deploy Trade-In / build-browser (push) Has been skipped
Deploy Trade-In / test (push) Successful in 56s
Deploy Trade-In / build-backend (push) Successful in 1m15s
Deploy Trade-In / deploy (push) Successful in 57s
2026-07-12 19:06:43 +00:00
bot-backend
27d1b09c72 fix(trade-in-pdf): убрать три ложных утверждения из PDF-отчёта
Some checks failed
CI / changes (pull_request) Successful in 12s
CI Trade-In / changes (pull_request) Successful in 12s
CI / frontend-tests (pull_request) Has been skipped
CI Trade-In / frontend-checks (pull_request) Has been skipped
CI / backend-tests (pull_request) Has been skipped
CI / openapi-codegen-check (pull_request) Has been skipped
CI Trade-In / backend-tests (pull_request) Failing after 1m0s
Отчёт печатал измерения, которых не делал:

1. Выдуманный «срок продажи 4–118 дней». _days_on_market_range при
   отсутствии реальных days_on_market (~48% оценок) возвращал хардкод
   (4, 118), а обложка и страница объявлений рисовали его как
   измеренный срок экспозиции. Теперь функция возвращает None, и оба
   call-site'а не рисуют срок (show_days=False) — как уже делает
   страница сделок.

2. № отчёта хардкодил «EKБ» для любой оценки — объект в Серове /
   Нижнем Тагиле получал екатеринбургский код на обложке, в шапках и
   футере. Префикс теперь выводится из target_address (ЕКБ / НТ / СЕР /
   …); неизвестный город → нейтральный «МЕРА», а не ложный «EKБ».

3. Статичный «сделки на 10–18% ниже» противоречил рассчитанному в том
   же PDF «−N%». Совет на обложке и баннер на странице сделок теперь
   ссылаются на реальный _discount_pct (тот же, что chip «−N%»), с
   нейтральной формулировкой когда дисконт неизвестен. Удалён ложный
   хвост «(Екатеринбург, 2026)».

Тесты: +11 кейсов (suppression (4,118), city-aware № отчёта,
computed-discount вместо «10–18%»). Все 23 проходят, ruff чисто.
2026-07-12 22:04:17 +03:00
bot-backend
c1a01fdbf9 fix(tradein/estimator): СберИндекс — только вторичка + confidence dispersion-ceiling (R2 H1+H2)
All checks were successful
CI / changes (pull_request) Successful in 13s
CI Trade-In / changes (pull_request) Successful in 13s
CI Trade-In / frontend-checks (pull_request) Has been skipped
CI / backend-tests (pull_request) Has been skipped
CI / frontend-tests (pull_request) Has been skipped
CI / openapi-codegen-check (pull_request) Has been skipped
CI Trade-In / backend-tests (pull_request) Successful in 1m4s
H1 (деньги, каждая оценка): time-adjust ДКП-коридора брал СберИндекс `residential_real_estate_prices`,
а для обл.66 это 100% «Первичный рынок» (новостройки) → коррекция направленно противоположна
вторичке (первичка Jan→May +0.89% vs вторичка −0.39%). Fix: SBER_COEFF_DASHBOARDS →
(real_estate_deals, dinamika-tsen-obyavlenii) — обе вторичка; + segment-guard в SQL
(`segment ILIKE '%вторичн%'`) как defense-in-depth. Live-verified: остаются только Вторичный-серии.

H2 (честность): `_compute_confidence` ветка «≥4 адреса» не имела потолка разброса → пул с ±45%
IQR + «расширили радиус из-за нехватки данных» уходил как «medium» вопреки объяснению. Fix:
medium требует IQR/median < 0.35; + force-low при fallback-расширении с разбросом > 0.30.

Regression-гейт: перегенерён baseline (dedup OFF, как в гейте) — ровно 1 из 277 фикстур-кейсов
medium→low (высокодисперсный, справедливо): calibration.medium.n 2→1 (mape 14.64→6.99 —
ненадёжный ушёл), low.coverage 81.82→81.88. Δ минимальна и обоснована. 127 тестов зелёные.

Stale-СберИндекс (72д) — НЕ трогаю в коде: time-adjust валидно ре-базит Jan→May; сброс потерял
бы валидную коррекцию. Реальный gap операционный (pull не гонялся с 19.06) — в scheduler/ops.
2026-07-12 22:03:59 +03:00
fb123bd97e Merge pull request 'fix(tradein/estimator): non-EKB headline из deals + geo-bound Tier-S analog leak' (#2492) from fix/tradein-oblast-nonekb-headline into main
All checks were successful
Deploy Trade-In / build-frontend (push) Has been skipped
Deploy Trade-In / changes (push) Successful in 12s
Deploy Trade-In / build-browser (push) Has been skipped
Deploy Trade-In / build-backend (push) Successful in 1m14s
Deploy Trade-In / test (push) Successful in 52s
Deploy Trade-In / deploy (push) Successful in 56s
2026-07-12 18:49:08 +00:00
bot-backend
f9ae6f0c53 fix(tradein/estimator): non-EKB headline из deals + geo-bound Tier-S analog leak
All checks were successful
CI Trade-In / changes (pull_request) Successful in 11s
CI / changes (pull_request) Successful in 11s
CI / backend-tests (pull_request) Has been skipped
CI / frontend-tests (pull_request) Has been skipped
CI / openapi-codegen-check (pull_request) Has been skipped
CI Trade-In / backend-tests (pull_request) Successful in 1m3s
CI Trade-In / frontend-checks (pull_request) Has been skipped
Валидация вскрыла: не-ЕКБ headline over-priced — analog Tier-S (same-building) fallback БЕЗ geo-предиката матчил одноимённую улицу+дом в ДРУГОМ городе → тянул ЕКБ-листинги (40191 EKB/~0 non-EKB) с distance_m=0. Серов ask-head 186k vs sold 30k; Н.Тагил 187k vs deal-corridor 89k.

Part 1 (Tier-S geo-bound, ~4624): + ST_DWithin(geom, subject, radius) в address-prefix fallback (зеркалит Tier-W). >99.97% листингов с coords → режет только cross-city false-positives.
Part 2 (deals-headline-fallback): _fetch_dkp_corridor widen на city-wide при street-sample <3 (gated city!=екатеринбург); _price_from_inputs: при median_ppm2<=0 (нет листингов) + anchor_tier None + dkp_raw>=3 → headline из deal-corridor, confidence=low, n_analogs=0, expected_sold=None.

Deep-review  APPROVE (no 🔴/🟠; ~10 downstream stages traced coherent). EKB byte-green regression-гейт (fallback требует median_ppm2<=0, невозможно для frozen fixture; Part 1 SQL не в replay). Тесты: Н.Тагил 0-листингов+corridor85911 → headline=85911 (не 0/186k); EKB-control dense → headline из листингов (fallback не срабатывает). Full suite 2418 passed.

Follow-ups (🟡): fallback без city-gate (sparse EKB n/a→deals, safe); api_analog_tier мислейбл 'city'; Part1/2a integration-test; EKB backtest post-deploy (rule #7).
2026-07-12 21:46:38 +03:00
3293df8e14 Merge pull request 'feat(tradein/backtest): --city per-city валидация точности + corridor city-parity' (#2491) from fix/tradein-oblast-d-backtest into main
All checks were successful
Deploy Trade-In / changes (push) Successful in 12s
Deploy Trade-In / build-browser (push) Has been skipped
Deploy Trade-In / test (push) Successful in 46s
Deploy Trade-In / build-frontend (push) Has been skipped
Deploy Trade-In / build-backend (push) Successful in 59s
Deploy Trade-In / deploy (push) Successful in 50s
2026-07-12 17:51:12 +00:00
4c56c5189a Merge pull request 'fix(tradein/estimator): резолвить target-город коридора из payload.address (C2 hotfix)' (#2490) from fix/tradein-oblast-c2fix-corridor-city into main
Some checks failed
Deploy Trade-In / build-backend (push) Blocked by required conditions
Deploy Trade-In / changes (push) Successful in 12s
Deploy Trade-In / build-frontend (push) Has been skipped
Deploy Trade-In / build-browser (push) Has been skipped
Deploy Trade-In / deploy (push) Blocked by required conditions
Deploy Trade-In / test (push) Has been cancelled
2026-07-12 17:50:19 +00:00
bot-backend
25a865f27b feat(tradein/backtest): --city per-city валидация + corridor city-parity
All checks were successful
CI / changes (pull_request) Successful in 8s
CI Trade-In / changes (pull_request) Successful in 9s
CI Trade-In / frontend-checks (pull_request) Has been skipped
CI / backend-tests (pull_request) Has been skipped
CI / frontend-tests (pull_request) Has been skipped
CI Trade-In / backend-tests (pull_request) Successful in 51s
CI / openapi-codegen-check (pull_request) Has been skipped
D validation harness для oblast-точности:
- --city '<точное deals.city>' → scope выборки по deals.city; default None = байт-идентично (тот же _SAMPLE_SQL object, literal identity).
- per-city PPM2 band из deal_city_price_bands (fallback global, 0 доп-запросов при None).
- corridor city-parity: backtest _fetch_dkp_corridor теперь передаёт city=_resolve_target_city(deal.address) — зеркалит прод C2 (был cross-city contaminated). NB: меняет и default-run live-числа (не fixture-replay) — intended catch-up к проду.
- _compute_confidence подтверждён comp-count-based (без EKB-хардкода).
- +10 unit-тестов. Регресс-гейт байт-зелёный (replay не гоняет corridor).

code-review  APPROVE. Frozen fixture/baseline не тронуты. Live per-city backtest прогоню после деплоя.
2026-07-12 20:49:04 +03:00
bot-backend
69f8b50934 docs(tradein): исправить GET-rehydrate comment — row.address=geo.full_address (city-stripped), corridor unscoped для не-ЕКБ (known limitation)
All checks were successful
CI / frontend-tests (pull_request) Has been skipped
CI / openapi-codegen-check (pull_request) Has been skipped
CI Trade-In / changes (pull_request) Successful in 8s
CI / changes (pull_request) Successful in 8s
CI Trade-In / frontend-checks (pull_request) Has been skipped
CI / backend-tests (pull_request) Has been skipped
CI Trade-In / backend-tests (pull_request) Successful in 51s
2026-07-12 20:48:25 +03:00
bot-backend
7801ac6cc1 fix(tradein/estimator): резолвить target-город коридора из payload.address (C2 hotfix)
All checks were successful
CI Trade-In / changes (pull_request) Successful in 7s
CI / changes (pull_request) Successful in 8s
CI Trade-In / frontend-checks (pull_request) Has been skipped
CI / backend-tests (pull_request) Has been skipped
CI / frontend-tests (pull_request) Has been skipped
CI / openapi-codegen-check (pull_request) Has been skipped
CI Trade-In / backend-tests (pull_request) Successful in 50s
C2 city-scope коридора был NO-OP для не-ЕКБ: estimate_quality резолвил город из geo.full_address, но геокодер РОНЯЕТ город для не-ЕКБ («Ленина, 1» вместо «Нижний Тагил, Ленина, 1») → target_city=None → коридор либо unscoped, либо None. Verified на проде: Н.Тагил corridor=None.

Фикс: резолвить город из payload.address (город есть) с fallback на canonical/geo; коридор матчить тоже по payload.address. GET: row.address приоритетнее canonical.

Verified прод-диагностикой: corridor(address=payload.address, city=resolve) → Н.Тагил=9, ЕКБ=15 (было: Н.Тагил=None). EKB regression-гейт байт-зелёный. geocoder full_address city-strip (display «Ленина,1») → cosmetic, E.
2026-07-12 20:36:08 +03:00
5d7c3e1c30 Merge pull request 'fix(tradein/estimator): city-scope ДКП-коридора + oblast city-tokens/bbox' (#2488) from fix/tradein-oblast-c2-estimator-scope into main
All checks were successful
Deploy Trade-In / test (push) Successful in 48s
Deploy Trade-In / deploy (push) Successful in 59s
Deploy Trade-In / build-backend (push) Successful in 52s
Deploy Trade-In / changes (push) Successful in 13s
Deploy Trade-In / build-frontend (push) Has been skipped
Deploy Trade-In / build-browser (push) Has been skipped
2026-07-12 17:21:55 +00:00
6936558f6f Merge pull request 'fix(tradein): city-scope ДКП-запросы — убрать oblast-контаминацию (C1+C2)' (#2489) from fix/tradein-oblast-city-scope-corridor-ratio into main
Some checks failed
Deploy Trade-In / deploy (push) Blocked by required conditions
Deploy Trade-In / changes (push) Successful in 11s
Deploy Trade-In / build-frontend (push) Has been skipped
Deploy Trade-In / build-browser (push) Has been skipped
Deploy Trade-In / test (push) Successful in 48s
Deploy Trade-In / build-backend (push) Has been cancelled
2026-07-12 17:20:31 +00:00
bot-backend
f631af4de3 fix(tradein): city-scope ДКП-запросы — убрать oblast-контаминацию (C1+C2)
All checks were successful
CI / changes (pull_request) Successful in 9s
CI Trade-In / changes (pull_request) Successful in 10s
CI Trade-In / frontend-checks (pull_request) Has been skipped
CI / backend-tests (pull_request) Has been skipped
CI / frontend-tests (pull_request) Has been skipped
CI / openapi-codegen-check (pull_request) Has been skipped
CI Trade-In / backend-tests (pull_request) Successful in 51s
Миграция 177 залила +47183 ДКП-сделки по всей Свердловской обл. (66, 368 городов,
deals.city заполнен). Потребители deals матчат по имени улицы БЕЗ city-фильтра →
одноимённые улицы дешёвых городов области контаминируют ЕКБ-числа.

C1 — street-deals + ДКП-коридор (клиент видит СЕЙЧАС):
- estimator._fetch_dkp_corridor + api/v1/trade_in.get_street_deals: добавлен city-скоуп
  `city IS NOT NULL AND :address ILIKE '%'||city||'%'` (канонический deals.city как
  подстрока целевого адреса — self-normalizing, адрес всегда содержит город).
- Live-repro (Ленина, 2к, 12мес): median 52 908 → 118 052 ₽/м² (было занижение ~−55%).

C2 — дневной пересчёт asking→sold ratio (imminent, «выкупная» −29%):
- tasks/asking_to_sold_ratio: deal_side + deal_global скоупятся на ЕКБ (asking-сторона =
  listings покрыты скрейпом только по ЕКБ; в listings нет колонки city). Константа
  _ASKING_CITY_PATTERN. ask-стороны не трогаем.
- Live-repro (rooms=2): sold_median all-city 82 593 (ratio 0.622) → EKB 116 858
  (ratio 0.880). Предотвращает обвал «выкупной» на след. пересчёте.
- Когда появятся oblast-листинги (B1/B2) → заменить на per-city ratio через `district`.

Тесты: subset-тест 080↔refresh (mirror _drop_segment_guard прецедента для city-guard) +
позитивный тест скоупа; 125 passed вкл. regression-гейт (байт-зелёный — коридор заморожен
в фикстуре), expected_sold, idor, 781. Regression-гейт не затронут (dkp_raw — frozen kwarg).
2026-07-12 20:18:36 +03:00
bot-backend
4a579e74e0 Merge remote-tracking branch 'refs/remotes/forgejo/main' into fix/tradein-oblast-c2-estimator-scope
All checks were successful
CI / changes (pull_request) Successful in 10s
CI / backend-tests (pull_request) Has been skipped
CI / openapi-codegen-check (pull_request) Has been skipped
CI Trade-In / backend-tests (pull_request) Successful in 50s
CI Trade-In / changes (pull_request) Successful in 9s
CI Trade-In / frontend-checks (pull_request) Has been skipped
CI / frontend-tests (pull_request) Has been skipped
# Conflicts:
#	tradein-mvp/backend/app/services/estimator.py
2026-07-12 20:14:17 +03:00
bot-backend
b93d26990b fix(tradein/estimator): city-scope ДКП-коридора + oblast city-tokens/bbox
All checks were successful
CI Trade-In / changes (pull_request) Successful in 9s
CI / changes (pull_request) Successful in 9s
CI Trade-In / frontend-checks (pull_request) Has been skipped
CI / backend-tests (pull_request) Has been skipped
CI / frontend-tests (pull_request) Has been skipped
CI / openapi-codegen-check (pull_request) Has been skipped
CI Trade-In / backend-tests (pull_request) Successful in 53s
Чинит доказанную кросс-город контаминацию коридора: street-name-матч без city тянул одноимённые улицы всех городов обл. («Ленина» = Н.Тагил+Лесной+Серов+ЕКБ...).

- _resolve_target_city(address) — word-boundary поверх geocoder.SVERDLOVSK_OBLAST_CITIES (F, ~30 городов), district-guard («мкр Заречный» = район, не город-ЗАТО).
- _fetch_dkp_corridor: +param city → AND LOWER(d.city)=CAST(:target_city AS text). Проводка в estimate_quality (dadata.canonical > geo) + trade_in.py GET rehydrate. city=None → прежнее поведение (fallback).
- _CITY_TOKENS + _DEDUP_ADDR_NOISE_WORDS → derive из SVERDLOVSK_OBLAST_CITIES (было 6 EKB-городов) → same-house dedup для non-EKB.
- bbox fast-path → geocoder.is_within_oblast66_bbox.
- docstring: коридор НЕ чисто advisory (клампит headline + radius-floor при n>=clamp_min_n).

EKB regression-гейт байт-зелёный (коридор upstream от frozen _price_from_inputs replay). Deep-review ⚠️APPROVE (no блокеров). Follow-ups (task #10): DB-derived city-set (Лесной 1103 uncovered), asking_to_sold_ratio city-scope, backtest city-parity.
2026-07-12 20:10:59 +03:00
90a6e4bfe9 Merge pull request 'feat(tradein/scraper): per-city sweep capability для 5 городов области (schedules disabled)' (#2487) from fix/tradein-oblast-b1-city-anchors into main
Some checks failed
Deploy Trade-In / changes (push) Successful in 13s
Deploy Trade-In / build-browser (push) Has been skipped
Deploy Trade-In / build-frontend (push) Has been skipped
Deploy Trade-In / test (push) Successful in 49s
Deploy Trade-In / build-backend (push) Successful in 2m4s
Deploy Trade-In / deploy (push) Has been cancelled
2026-07-12 17:10:22 +00:00
bot-backend
cf91ac6b24 feat(tradein/scraper): per-city sweep capability для 5 городов области (schedules disabled)
All checks were successful
CI Trade-In / changes (pull_request) Successful in 9s
CI / changes (pull_request) Successful in 9s
CI Trade-In / frontend-checks (pull_request) Has been skipped
CI / backend-tests (pull_request) Has been skipped
CI / frontend-tests (pull_request) Has been skipped
CI / openapi-codegen-check (pull_request) Has been skipped
CI Trade-In / backend-tests (pull_request) Successful in 52s
B1: avito/cian/yandex city-sweep по городам Свердловской обл. (Н.Тагил/Каменск/Первоуральск/В.Пышма/Серов). Schedules засеяны DISABLED — rollout вручную по одному городу (proxy = 1 живой прокси).

- pipeline.py: CITY_ANCHORS (5) + get_city_anchors(slug) (None→EKB fallback).
- scheduler.py: sweep-jobs читают params.city → анкоры; wildcard-хендлеры *_city_sweep_* (source UNIQUE → per-city source, как deactivate_stale_*).
- geo.py: OBLAST66_BBOX + is_within_oblast66_bbox; avito/detail.py ingest-guard EKB→oblast66.
- price_brackets.py: 5 city-seed (reuse EKB). migration 179: 15 rows enabled=false, staggered, idempotent.
- +6 тестов. domclick (B2) не тронут.
2026-07-12 20:06:46 +03:00
74017b39c4 Merge pull request 'fix(tradein/dadata): suggest-fallback для house_fias_id когда CLEAN выключен' (#2486) from fix/tradein-dadata-clean-fallback-suggest into main
All checks were successful
Deploy Trade-In / changes (push) Successful in 12s
Deploy Trade-In / build-frontend (push) Has been skipped
Deploy Trade-In / build-browser (push) Has been skipped
Deploy Trade-In / test (push) Successful in 51s
Deploy Trade-In / build-backend (push) Successful in 1m5s
Deploy Trade-In / deploy (push) Successful in 52s
2026-07-12 16:59:29 +00:00
bot-backend
e250869c9d fix(tradein/dadata): suggest-fallback для house_fias_id когда CLEAN выключен
All checks were successful
CI Trade-In / changes (pull_request) Successful in 8s
CI / backend-tests (pull_request) Has been skipped
CI / openapi-codegen-check (pull_request) Has been skipped
CI / changes (pull_request) Successful in 9s
CI Trade-In / frontend-checks (pull_request) Has been skipped
CI / frontend-tests (pull_request) Has been skipped
CI Trade-In / backend-tests (pull_request) Successful in 56s
На проде у DaData-аккаунта выключена услуга CLEAN (Стандартизация) → /clean/address
отдаёт HTTP 403 «Feature CLEAN disabled», из-за чего house_fias_id (ключ join с
houses/ДОМ.РФ) и гео терялись, хотя /suggest их отдаёт бесплатно (token-only).

- enrich_address(): /clean → graceful fallback на /suggest (house-level кандидат);
  canonical_address/house_cadnum честно None (их даёт только CLEAN — не выдумываем).
- clean_address(): 403 «feature disabled» логируется отдельно от auth-rejection
  (токен валиден — refresh не поможет; чинить = включить услугу либо fallback).
- DadataSuggestion + _parse_suggestion: тянут house_fias_id/kladr_id/qc_geo.
- estimator: enrichment-вызов → enrich_address (алиас dadata_clean_address сохранён,
  это точка патча в 11 тестах — без churn).
- Тесты: feature-disabled лог, enrich clean-path/suggest-fallback/both-fail, новые поля.
  Проверено вживую на прод-кредах: clean→403, enrich→house_fias_id восстановлен.
2026-07-12 19:55:37 +03:00
b0b2c94ece Merge pull request 'fix(tradein/estimator): per-city ценовой band ДКП-сделок для всей Свердловской обл. (66)' (#2485) from fix/tradein-oblast-c1-price-band into main
All checks were successful
Deploy Trade-In / build-browser (push) Has been skipped
Deploy Trade-In / build-backend (push) Successful in 56s
Deploy Trade-In / changes (push) Successful in 11s
Deploy Trade-In / build-frontend (push) Has been skipped
Deploy Trade-In / test (push) Successful in 47s
Deploy Trade-In / deploy (push) Successful in 48s
2026-07-12 16:09:05 +00:00
bot-backend
e2502a0134 fix(tradein/estimator): per-city ценовой band ДКП-сделок для всей обл. 66
All checks were successful
CI Trade-In / frontend-checks (pull_request) Has been skipped
CI / frontend-tests (pull_request) Has been skipped
CI / openapi-codegen-check (pull_request) Has been skipped
CI Trade-In / changes (pull_request) Successful in 8s
CI / changes (pull_request) Successful in 8s
CI / backend-tests (pull_request) Has been skipped
CI Trade-In / backend-tests (pull_request) Successful in 50s
Глобальный DEAL_MIN_PPM2=50k резал 46.6% не-ЕКБ сделок (легит дешёвые города: Североуральск p50=21.7k, Новоуральск 36k, Асбест 41k). Band стал per-city.

- migration 178: deal_city_price_bands (city PK, ppm2_min/max, n_deals), populate из deals (source=rosreestr, city<>Екатеринбург, count>=30), min=GREATEST(p1,8000) max=LEAST(p99,800000), idempotent upsert; HAVING-guard p99>=8000 (airtight против inverted band).
- estimator.py оба enforcement-пойнта city-aware с fallback на глобаль:
  - _fetch_dkp_corridor SQL: LEFT JOIN deal_city_price_bands + COALESCE(band, :global).
  - _is_plausible_deal(city, bands) + _fetch_deals: _load_city_price_bands (TTL-cache), SELECT city.
- ЕКБ исключён из таблицы → fallback на 50k/800k → EKB-сделки byte-identical (deals спеллит EKB только 'Екатеринбург', 49791).

EKB deal-level инвариант airtight; interior-EKB оценки идентичны. Регресс-гейт байт-зелёный. Follow-ups: periodic refresh band-таблицы; city-scoping _fetch_dkp_corridor (→ C2).
2026-07-12 19:07:05 +03:00
51952ebd70 Merge pull request 'fix(tradein/geocoder): раз-хардкод геокодера/DaData на всю Свердловскую обл. (region 66)' (#2484) from fix/tradein-oblast-f-geocoder into main
All checks were successful
Deploy Trade-In / changes (push) Successful in 10s
Deploy Trade-In / build-browser (push) Has been skipped
Deploy Trade-In / test (push) Successful in 45s
Deploy Trade-In / build-frontend (push) Has been skipped
Deploy Trade-In / build-backend (push) Successful in 58s
Deploy Trade-In / deploy (push) Successful in 47s
2026-07-12 15:35:27 +00:00
bot-backend
c0cbdc2f99 fix(tradein/geocoder): раз-хардкод геокодера/DaData на всю Свердловскую обл. (region 66)
All checks were successful
CI / changes (pull_request) Successful in 7s
CI / backend-tests (pull_request) Has been skipped
CI Trade-In / backend-tests (pull_request) Successful in 50s
CI Trade-In / changes (pull_request) Successful in 8s
CI / frontend-tests (pull_request) Has been skipped
CI / openapi-codegen-check (pull_request) Has been skipped
CI Trade-In / frontend-checks (pull_request) Has been skipped
Геокодер был жёстко EKB-bound и ВЫБРАСЫВАЛ корректные non-EKB геокоды (bbox-accept-фильтры) → блокировал оценку любого адреса области. Обобщено до region-66.

- OBLAST66_BBOX + is_within_oblast66_bbox (superset EKB-tight, EKB не затронут); EKB Yandex Tier1/2 fast-path сохранён.
- Все accept-фильтры (Yandex/Nominatim forward+suggest, geocode() accept+typo) → oblast-66.
- City-prefix по word-boundary _has_oblast_marker (не substring): EKB-улицы 'Серова 27'/'Ирбитская 5' и др. больше не роняют 'Екатеринбург,'-префикс (был silent wrong-city баг).
- Two-pass tie-break в accept-циклах: tight-EKB кандидат приоритетнее → EKB-результат идентичен прежнему.
- Region cross-check (Nominatim address.state / Yandex AdministrativeAreaName) отсекает соседние области (Тюмень и др.); bbox-fallback когда region отсутствует.
- DaData suggest: region-параметр (hard-filter), убран no-op restrict_value.
- +26 тестов: OBLAST66 superset TIGHT, far-town accept, Тюмень reject, marker +/- (street-collision), DaData region body.

Foundation для A2 (геокодинг non-EKB сделок) и live-оценки адресов области.
2026-07-12 18:33:05 +03:00
fcc37c46ab fix(tradein/v2): аналитика — выровнять два графика в равную высоту (#2483)
All checks were successful
Deploy Trade-In / changes (push) Successful in 13s
Deploy Trade-In / test (push) Successful in 49s
Deploy Trade-In / build-frontend (push) Successful in 2m0s
Deploy Trade-In / build-browser (push) Successful in 29s
Deploy Trade-In / build-backend (push) Successful in 30s
Deploy Trade-In / deploy (push) Successful in 1m47s
2026-07-12 15:16:34 +00:00
95f0b0aa64 Merge pull request 'fix(tradein/ingest): раз-хардкод rosreestr DKP импорта на всю Свердловскую обл. (region 66)' (#2482) from fix/tradein-oblast-a1-ingest into main
Some checks failed
Deploy Trade-In / deploy (push) Blocked by required conditions
Deploy Trade-In / changes (push) Successful in 11s
Deploy Trade-In / build-frontend (push) Successful in 31s
Deploy Trade-In / build-browser (push) Successful in 31s
Deploy Trade-In / test (push) Successful in 50s
Deploy Trade-In / build-backend (push) Has been cancelled
2026-07-12 15:14:22 +00:00
bot-backend
752cb1044b fix(tradein/ingest): раз-хардкод rosreestr DKP импорта на всю обл. 66
All checks were successful
CI Trade-In / changes (pull_request) Successful in 7s
CI / changes (pull_request) Successful in 7s
CI Trade-In / frontend-checks (pull_request) Has been skipped
CI / backend-tests (pull_request) Has been skipped
CI / frontend-tests (pull_request) Has been skipped
CI / openapi-codegen-check (pull_request) Has been skipped
CI Trade-In / backend-tests (pull_request) Successful in 51s
- scheduler.py import_rosreestr_dkp: снят фильтр city ILIKE '%катеринбург%', address из реального city источника, deals.region_code + новая deals.city заполняются (было: хардкод 'Екатеринбург,' + region_code NULL)
- migration 177: deals.city + индекс + бэкфилл существующих EKB-строк region_code=66/city
- guard city IS NOT NULL → address не NULL для ~15 null-city строк источника
- sync deploy/import-rosreestr.sh (ops-fallback) под тот же oblast-scope
- split dedup-теста: 077 (историческая) хранит EKB-фильтр, живой импорт — нет

Открывает +47183 не-ЕКБ сделок region 66, уже сидящих в источнике, ранее резавшихся на импорте.
2026-07-12 18:11:47 +03:00
338ccbb78f fix(tradein/v2): честность+консистентность — реконсиляция счётчика аналогов, убрать фейк-donut/линейку, outlier-guard, created_at, дисклеймеры (#2481)
All checks were successful
Deploy Trade-In / build-frontend (push) Successful in 1m55s
Deploy Trade-In / deploy (push) Successful in 44s
Deploy Trade-In / changes (push) Successful in 11s
Deploy Trade-In / test (push) Has been skipped
Deploy Trade-In / build-browser (push) Has been skipped
Deploy Trade-In / build-backend (push) Has been skipped
2026-07-12 15:09:10 +00:00
a88e755403 fix(tradein/v2): аналитика — 2 графика в ряд, чтобы Цена×срок влезала на экран (#2480)
All checks were successful
Deploy Trade-In / changes (push) Successful in 10s
Deploy Trade-In / test (push) Has been skipped
Deploy Trade-In / build-browser (push) Has been skipped
Deploy Trade-In / deploy (push) Successful in 44s
Deploy Trade-In / build-backend (push) Has been skipped
Deploy Trade-In / build-frontend (push) Successful in 1m55s
2026-07-12 14:56:08 +00:00
4f205bd7da fix(tradein/v2): polish АНАЛИТИКА — растянуть scatter, клэмп оси к 0, честные легенды/бакеты, убрать фейк-тренд (#2479)
All checks were successful
Deploy Trade-In / deploy (push) Successful in 43s
Deploy Trade-In / changes (push) Successful in 10s
Deploy Trade-In / test (push) Has been skipped
Deploy Trade-In / build-browser (push) Has been skipped
Deploy Trade-In / build-backend (push) Has been skipped
Deploy Trade-In / build-frontend (push) Successful in 1m53s
2026-07-12 14:37:05 +00:00
8dd35c01d8 feat(tradein/v2): карта вторички — сделки Росреестра + объявления по источникам (SourcesView) (#2478)
All checks were successful
Deploy Trade-In / changes (push) Successful in 11s
Deploy Trade-In / test (push) Has been skipped
Deploy Trade-In / build-backend (push) Has been skipped
Deploy Trade-In / build-browser (push) Has been skipped
Deploy Trade-In / build-frontend (push) Successful in 2m3s
Deploy Trade-In / deploy (push) Successful in 45s
2026-07-12 13:26:55 +00:00
ed504cbbd9 fix(tradein): repoint rosreestr_poll to /data-sets/ + verify Content-Type (СДЕЛКИ dataset moved off /opendata) (#2477)
All checks were successful
Deploy Trade-In / changes (push) Successful in 10s
Deploy Trade-In / build-frontend (push) Has been skipped
Deploy Trade-In / build-browser (push) Has been skipped
Deploy Trade-In / test (push) Successful in 45s
Deploy Trade-In / build-backend (push) Successful in 1m4s
Deploy Trade-In / deploy (push) Successful in 49s
2026-07-12 13:21:58 +00:00
cc231946c2 feat(tradein): fill houses.year_built/material_walls from ДОМ.РФ КР1.1/1.2 + zhkh_year, propagate to listings (#2013) (#2476)
All checks were successful
Deploy Trade-In / changes (push) Successful in 12s
Deploy Trade-In / build-browser (push) Has been skipped
Deploy Trade-In / build-frontend (push) Has been skipped
Deploy Trade-In / test (push) Successful in 45s
Deploy Trade-In / build-backend (push) Successful in 1m1s
Deploy Trade-In / deploy (push) Successful in 2m46s
2026-07-12 13:12:07 +00:00
d22e0e00ae tech-debt(tradein/estimator): collapse won estimate_* flags into defaults (#1970) (#2475)
All checks were successful
Deploy Trade-In / changes (push) Successful in 10s
Deploy Trade-In / build-frontend (push) Has been skipped
Deploy Trade-In / build-browser (push) Has been skipped
Deploy Trade-In / test (push) Successful in 45s
Deploy Trade-In / build-backend (push) Successful in 58s
Deploy Trade-In / deploy (push) Successful in 47s
2026-07-12 12:46:18 +00:00
346767ba90 Merge pull request 'test(rbac): user2 expected role expired after brusnika trial cutoff' (#2473) from fix/rbac-test-user2-expired into main
All checks were successful
Deploy / changes (push) Successful in 9s
Deploy / build-frontend (push) Has been skipped
Deploy Trade-In / changes (push) Successful in 13s
Deploy Trade-In / build-frontend (push) Has been skipped
Deploy Trade-In / build-browser (push) Has been skipped
Deploy / build-backend (push) Successful in 36s
Deploy / build-worker (push) Successful in 35s
Deploy Trade-In / test (push) Successful in 53s
Deploy Trade-In / build-backend (push) Successful in 31s
Deploy / deploy (push) Successful in 1m23s
Deploy Trade-In / deploy (push) Successful in 1m5s
2026-07-09 16:33:19 +00:00
bot-backend
f4c26d474a test(rbac): user2 expected role expired after brusnika trial cutoff
All checks were successful
CI Trade-In / changes (pull_request) Successful in 9s
CI / changes (pull_request) Successful in 10s
CI Trade-In / frontend-checks (pull_request) Has been skipped
CI / frontend-tests (pull_request) Has been skipped
CI Trade-In / backend-tests (pull_request) Successful in 1m8s
CI / openapi-codegen-check (pull_request) Successful in 2m33s
CI / backend-tests (pull_request) Successful in 15m22s
Follow-up к смене user2 pilot→expired в roles.yaml. test_get_role_known_users
гонял user1..user10 == pilot циклом; user2 теперь expired. На PR #2472 этот
тест был skipped (paths-filter не матчил tradein-mvp/**, менялся только
auth/roles.yaml) → упал уже на полном деплое. Правка в обоих зеркалах
(tradein + main backend).
2026-07-09 19:16:59 +03:00
3fe1216317 Merge pull request 'fix(tradein/auth): expire brusnika (user2) trial access' (#2472) from fix/expire-brusnika-trial-access into main 2026-07-09 16:07:54 +00:00
bot-backend
3c4a33ca32 fix(tradein/auth): expire brusnika (user2) trial access
All checks were successful
CI / changes (pull_request) Successful in 8s
CI Trade-In / changes (pull_request) Successful in 10s
CI Trade-In / backend-tests (pull_request) Has been skipped
CI Trade-In / frontend-checks (pull_request) Has been skipped
CI / backend-tests (pull_request) Has been skipped
CI / frontend-tests (pull_request) Has been skipped
CI / openapi-codegen-check (pull_request) Has been skipped
Аналогично praktika (#1977): user2 pilot→expired в roles.yaml. RouteGuard
уже short-circuit'ит на role=expired → NoAccessScreen variant="trial"
(«Пробный доступ закончился», контакт @ArtemKopylov87). Логин остаётся в
caddy/users.caddy.snippet, чтобы дойти до фронта и увидеть экран. Инфра
expired-роли + trial-экран уже в проде — здесь только смена роли.
2026-07-09 19:06:05 +03:00
f987e819b2 fix(honesty): true competitors/neighbors totals + permits truncation disclosure (#2464 cluster B) (#2471)
All checks were successful
Deploy / changes (push) Successful in 10s
Deploy / build-frontend (push) Has been skipped
Deploy / build-backend (push) Successful in 2m6s
Deploy / build-worker (push) Successful in 3m13s
Deploy / deploy (push) Successful in 1m38s
2026-07-08 08:31:02 +00:00
9a6b56016b fix(analytics): de-inflate domrf_kn snapshot counts (8.49×) + backfill UPSERT columns (#2464) (#2470)
All checks were successful
Deploy / changes (push) Successful in 9s
Deploy / build-frontend (push) Has been skipped
Deploy / build-backend (push) Successful in 2m1s
Deploy / build-worker (push) Successful in 3m22s
Deploy / deploy (push) Successful in 1m40s
2026-07-08 07:30:12 +00:00
d3f3370bed fix(site_finder): SAVEPOINT supply/market + convert 2 bare rollbacks (#2464 Wave 2) (#2469)
All checks were successful
Deploy / changes (push) Successful in 10s
Deploy / build-frontend (push) Has been skipped
Deploy / build-backend (push) Successful in 2m30s
Deploy / build-worker (push) Successful in 3m45s
Deploy / deploy (push) Successful in 1m54s
2026-07-08 06:46:09 +00:00
01360e9c3d fix(site_finder): SAVEPOINT-isolate gate-lookup db.execute swallows (#2464 Wave 2) (#2468)
Some checks failed
Deploy / build-backend (push) Blocked by required conditions
Deploy / build-worker (push) Blocked by required conditions
Deploy / build-frontend (push) Blocked by required conditions
Deploy / deploy (push) Blocked by required conditions
Deploy / changes (push) Has been cancelled
2026-07-08 06:46:03 +00:00
22f3c44dc2 fix(site_finder): SAVEPOINT-isolate quarter-dump lookup db.execute (#2464 Wave 2) (#2467)
All checks were successful
Deploy / changes (push) Successful in 11s
Deploy / build-frontend (push) Has been skipped
Deploy / build-backend (push) Successful in 2m42s
Deploy / build-worker (push) Successful in 3m53s
Deploy / deploy (push) Successful in 1m54s
2026-07-08 06:30:32 +00:00
82fdabccba fix(forecasting): SAVEPOINT-isolate §22 db.execute failures + close RELEASE-trap (#2464) (#2466)
All checks were successful
Deploy / changes (push) Successful in 9s
Deploy / build-frontend (push) Has been skipped
Deploy / build-backend (push) Successful in 2m35s
Deploy / build-worker (push) Successful in 3m1s
Deploy / deploy (push) Successful in 1m34s
2026-07-08 05:41:35 +00:00
39dd63333e fix(site_finder): SAVEPOINT-isolate DB errors across cluster A (#2464) (#2465)
All checks were successful
Deploy / changes (push) Successful in 9s
Deploy / build-frontend (push) Has been skipped
Deploy / build-backend (push) Successful in 3m29s
Deploy / build-worker (push) Successful in 4m48s
Deploy / deploy (push) Successful in 1m43s
2026-07-07 12:50:11 +00:00
1fa99812af fix(concept): §7 massing-сцена осознаёт весь MultiPolygon (соседи уезжали до 1.6км) (#2180) (#2463)
All checks were successful
Deploy / changes (push) Successful in 8s
Deploy / build-backend (push) Has been skipped
Deploy / build-worker (push) Has been skipped
Deploy / build-frontend (push) Successful in 3m35s
Deploy / deploy (push) Successful in 1m23s
2026-07-07 08:10:27 +00:00
53b160ff0a fix(scrapers): rate-limit/backoff/circuit-breaker для ОКН EGRKN-скрапера (#2445-D3) (#2461)
All checks were successful
Deploy / changes (push) Successful in 7s
Deploy / build-frontend (push) Has been skipped
Deploy / build-backend (push) Successful in 2m13s
Deploy / deploy (push) Successful in 1m41s
Deploy / build-worker (push) Successful in 3m19s
2026-07-05 20:09:18 +00:00
464519c69b fix(workers): izyatie per-doc isolation + newbuilding-crossload breadcrumb (#2445-D5/D6) (#2460)
Some checks failed
Deploy / build-backend (push) Blocked by required conditions
Deploy / build-worker (push) Blocked by required conditions
Deploy / build-frontend (push) Blocked by required conditions
Deploy / deploy (push) Blocked by required conditions
Deploy / changes (push) Has been cancelled
2026-07-05 20:09:12 +00:00
7315d04396 fix(scrapers): WAF-guard эндпоинтов + throttled catalog BrowserSession (#2445-D1/D2) (#2459)
Some checks failed
Deploy / build-backend (push) Blocked by required conditions
Deploy / build-worker (push) Blocked by required conditions
Deploy / build-frontend (push) Blocked by required conditions
Deploy / deploy (push) Blocked by required conditions
Deploy / changes (push) Has been cancelled
2026-07-05 20:09:06 +00:00
d1fa450f48 fix(backend): reservation_ingest пейсинг + короткоживущая DB-сессия (#2445-D4) (#2458)
Some checks failed
Deploy / build-backend (push) Blocked by required conditions
Deploy / build-worker (push) Blocked by required conditions
Deploy / build-frontend (push) Blocked by required conditions
Deploy / deploy (push) Blocked by required conditions
Deploy / changes (push) Has been cancelled
2026-07-05 20:09:00 +00:00
be4b2ee1fa fix(export): недостающая §13.7 «Уверенность» в PDF-экспортёре отчёта (#2445-C2) (#2456)
All checks were successful
Deploy / changes (push) Successful in 6s
Deploy / build-frontend (push) Has been skipped
Deploy / build-backend (push) Successful in 2m6s
Deploy / build-worker (push) Successful in 4m33s
Deploy / deploy (push) Successful in 2m25s
2026-07-05 19:40:38 +00:00
f3988add0a fix(generative): устаревший VAT/office caveat в concept PDF (#2445-C1) (#2455)
Some checks failed
Deploy / changes (push) Has been cancelled
Deploy / build-backend (push) Has been cancelled
Deploy / build-worker (push) Has been cancelled
Deploy / build-frontend (push) Has been cancelled
Deploy / deploy (push) Has been cancelled
2026-07-05 19:40:30 +00:00
c0c666cccc fix(site-finder): remount analysis-дерева при смене cad (#2445-B4) (#2454)
All checks were successful
Deploy / changes (push) Successful in 10s
Deploy / build-backend (push) Has been skipped
Deploy / build-worker (push) Has been skipped
Deploy / build-frontend (push) Successful in 3m31s
Deploy / deploy (push) Successful in 1m32s
2026-07-05 19:20:36 +00:00
de5da6ec35 fix(frontend): нейтральный fallback для gate/velocity enum-lookup (#2445-B2/B3) (#2451)
Some checks failed
Deploy / build-backend (push) Blocked by required conditions
Deploy / build-worker (push) Blocked by required conditions
Deploy / build-frontend (push) Blocked by required conditions
Deploy / deploy (push) Blocked by required conditions
Deploy / changes (push) Has been cancelled
2026-07-05 19:20:28 +00:00
1bbd4c99e2 fix(generative): отклонять спроецированные координаты до pyproj CRSError → 422 (#2445-B1) (#2453)
Some checks failed
Deploy / build-backend (push) Blocked by required conditions
Deploy / build-worker (push) Blocked by required conditions
Deploy / build-frontend (push) Blocked by required conditions
Deploy / deploy (push) Blocked by required conditions
Deploy / changes (push) Has been cancelled
2026-07-05 19:20:19 +00:00
82e0b8b0a3 fix(site-finder): true-counts вместо молча-усечённых len(list) в connection-points/utility (#2445-A1/A2) (#2450)
All checks were successful
Deploy / changes (push) Successful in 7s
Deploy / build-backend (push) Successful in 2m9s
Deploy / build-frontend (push) Successful in 3m27s
Deploy / build-worker (push) Successful in 4m19s
Deploy / deploy (push) Successful in 1m55s
2026-07-05 18:54:49 +00:00
b40995780b fix(site-finder): раскрыть proxy-fraction в EKB-median velocity-бенчмарке (#2445-A4) (#2448)
Some checks failed
Deploy / changes (push) Has been cancelled
Deploy / build-backend (push) Has been cancelled
Deploy / build-worker (push) Has been cancelled
Deploy / build-frontend (push) Has been cancelled
Deploy / deploy (push) Has been cancelled
2026-07-05 18:54:42 +00:00
c8c96a87ee fix(site-finder): competitors avg-price по per-object снапшоту, не глобальному (#2445-A5) (#2447)
Some checks failed
Deploy / build-backend (push) Blocked by required conditions
Deploy / build-worker (push) Blocked by required conditions
Deploy / build-frontend (push) Blocked by required conditions
Deploy / deploy (push) Blocked by required conditions
Deploy / changes (push) Has been cancelled
2026-07-05 18:54:36 +00:00
9b6aba42dc fix(site-finder): L2 hidden-supply не обнуляет неизвестный flat_count (#2445-A3) (#2446)
Some checks failed
Deploy / deploy (push) Blocked by required conditions
Deploy / changes (push) Successful in 7s
Deploy / build-frontend (push) Has been skipped
Deploy / build-backend (push) Has been cancelled
Deploy / build-worker (push) Has been cancelled
2026-07-05 18:54:14 +00:00
5b3799a23a feat(scrapers): wire per-flat DOM.РФ catalog scraper + propagate catalog_url_hash (#2442) (#2444)
All checks were successful
Deploy / changes (push) Successful in 8s
Deploy / build-backend (push) Successful in 3m26s
Deploy / build-frontend (push) Successful in 3m38s
Deploy / deploy (push) Successful in 2m28s
Deploy / build-worker (push) Successful in 4m20s
2026-07-05 17:00:58 +00:00
eea9c99e73 feat(scrapers): extract + download DOM.РФ flat plan images (#2440) (#2441)
All checks were successful
Deploy / changes (push) Successful in 8s
Deploy / build-frontend (push) Has been skipped
Deploy / build-backend (push) Successful in 2m2s
Deploy / build-worker (push) Successful in 3m20s
Deploy / deploy (push) Successful in 1m35s
2026-07-05 16:23:29 +00:00
9c46689da1 fix(devops): переносим deploy-obsidian.yml в .forgejo/workflows/ (#2416) (#2439)
All checks were successful
Deploy Obsidian / deploy-obsidian (push) Successful in 1m59s
2026-07-05 15:34:55 +00:00
994eba0a16 fix(tradein/cian): extract_all_states — discover new .concat() format, not just legacy .push() (#2438)
All checks were successful
Deploy Trade-In / changes (push) Successful in 11s
Deploy Trade-In / build-frontend (push) Has been skipped
Deploy Trade-In / build-browser (push) Has been skipped
Deploy Trade-In / test (push) Successful in 47s
Deploy Trade-In / build-backend (push) Successful in 1m34s
Deploy Trade-In / deploy (push) Successful in 1m8s
2026-07-04 21:03:32 +00:00
7c1d3693e2 fix(tradein/cian): персист bti_data/valuation house_info в houses + дедуп УК (#2435) (#2437)
All checks were successful
Deploy Trade-In / build-browser (push) Has been skipped
Deploy Trade-In / build-backend (push) Successful in 1m34s
Deploy Trade-In / changes (push) Successful in 11s
Deploy Trade-In / build-frontend (push) Has been skipped
Deploy Trade-In / test (push) Successful in 47s
Deploy Trade-In / deploy (push) Successful in 6m36s
2026-07-04 20:21:32 +00:00
f93ef5374d feat(tradein/domclick): production Layer B detail-backfill orchestrator (#2000) (#2436)
All checks were successful
Deploy Trade-In / changes (push) Successful in 10s
Deploy Trade-In / build-frontend (push) Has been skipped
Deploy Trade-In / build-browser (push) Has been skipped
Deploy Trade-In / test (push) Successful in 47s
Deploy Trade-In / build-backend (push) Successful in 1m30s
Deploy Trade-In / deploy (push) Successful in 1m32s
2026-07-04 19:49:07 +00:00
b2d42a631a feat(tradein/pdf): редизайн отчёта в реальном языке МЕРА (v2 HUD) (#2434)
All checks were successful
Deploy Trade-In / changes (push) Successful in 10s
Deploy Trade-In / build-frontend (push) Has been skipped
Deploy Trade-In / build-browser (push) Has been skipped
Deploy Trade-In / test (push) Successful in 46s
Deploy Trade-In / build-backend (push) Successful in 57s
Deploy Trade-In / deploy (push) Successful in 47s
2026-07-04 18:41:21 +00:00
ced616d150 feat(tradein/domclick): cookie-injection MVP для обхода QRATOR-блока (#2000) (#2433)
All checks were successful
Deploy Trade-In / test (push) Successful in 49s
Deploy Trade-In / changes (push) Successful in 10s
Deploy Trade-In / build-frontend (push) Has been skipped
Deploy Trade-In / build-browser (push) Successful in 2m13s
Deploy Trade-In / build-backend (push) Successful in 1m32s
Deploy Trade-In / deploy (push) Successful in 7m16s
2026-07-04 17:49:40 +00:00
acc1e05357 fix(tradein/estimator): восстановить и канонизировать корпус дома в дедупе (регрессия #2429) (#2432)
All checks were successful
Deploy Trade-In / changes (push) Successful in 10s
Deploy Trade-In / build-frontend (push) Has been skipped
Deploy Trade-In / build-browser (push) Has been skipped
Deploy Trade-In / deploy (push) Successful in 47s
Deploy Trade-In / test (push) Successful in 43s
Deploy Trade-In / build-backend (push) Successful in 56s
2026-07-04 17:08:40 +00:00
102bdbda47 fix(tradein/domclick): органический заход через SERP перед card-fetch (anti-QRATOR) (#2430)
All checks were successful
Deploy Trade-In / changes (push) Successful in 11s
Deploy Trade-In / build-frontend (push) Has been skipped
Deploy Trade-In / test (push) Successful in 52s
Deploy Trade-In / build-browser (push) Successful in 2m16s
Deploy Trade-In / build-backend (push) Successful in 1m45s
Deploy Trade-In / deploy (push) Successful in 8m15s
2026-07-04 16:39:47 +00:00
752c3656a3 fix(tradein/estimator): дом «2» вместо «15» при глухом названии станции метро в avito-адресе (#2429)
All checks were successful
Deploy Trade-In / changes (push) Successful in 10s
Deploy Trade-In / build-frontend (push) Has been skipped
Deploy Trade-In / build-browser (push) Has been skipped
Deploy Trade-In / test (push) Successful in 45s
Deploy Trade-In / build-backend (push) Successful in 55s
Deploy Trade-In / deploy (push) Successful in 46s
2026-07-04 15:50:49 +00:00
cfed042712 fix(tradein/v2): гонка при клике по строке «Предыдущие оценки» — отчёт не загружался (#2428)
All checks were successful
Deploy Trade-In / changes (push) Successful in 11s
Deploy Trade-In / test (push) Has been skipped
Deploy Trade-In / build-browser (push) Has been skipped
Deploy Trade-In / build-backend (push) Has been skipped
Deploy Trade-In / build-frontend (push) Successful in 2m3s
Deploy Trade-In / deploy (push) Successful in 46s
2026-07-04 14:47:01 +00:00
5cbf3485a0 fix(tradein/pdf): честный embedded-шрифт (PT Sans/PT Serif) + matplotlib-график вместо div (#2427)
All checks were successful
Deploy Trade-In / changes (push) Successful in 12s
Deploy Trade-In / build-frontend (push) Has been skipped
Deploy Trade-In / build-browser (push) Has been skipped
Deploy Trade-In / test (push) Successful in 49s
Deploy Trade-In / build-backend (push) Successful in 1m36s
Deploy Trade-In / deploy (push) Successful in 1m1s
2026-07-04 14:39:37 +00:00
b102c334d8 Merge pull request 'chore(tradein/proxy-pool): добавить domclick в provider_affinity CHECK (residential ASocks для #2000)' (#2426) from chore/tradein-scrape-proxies-domclick-affinity into main
All checks were successful
Deploy Trade-In / changes (push) Successful in 11s
Deploy Trade-In / build-frontend (push) Has been skipped
Deploy Trade-In / build-browser (push) Has been skipped
Deploy Trade-In / test (push) Successful in 43s
Deploy Trade-In / build-backend (push) Successful in 53s
Deploy Trade-In / deploy (push) Successful in 48s
2026-07-04 14:25:40 +00:00
bot-backend
b238d3ef50 chore(tradein/proxy-pool): добавить domclick в provider_affinity CHECK (residential ASocks для #2000)
All checks were successful
CI Trade-In / frontend-checks (pull_request) Has been skipped
CI / backend-tests (pull_request) Has been skipped
CI / frontend-tests (pull_request) Has been skipped
CI / openapi-codegen-check (pull_request) Has been skipped
CI / changes (pull_request) Successful in 9s
CI Trade-In / changes (pull_request) Successful in 9s
CI Trade-In / backend-tests (pull_request) Successful in 49s
2026-07-04 17:21:07 +03:00
3a2b4ba825 feat(tradein/v2): кликабельные строки + параметры в оверлее «Предыдущие оценки» (#2424)
All checks were successful
Deploy Trade-In / changes (push) Successful in 10s
Deploy Trade-In / test (push) Has been skipped
Deploy Trade-In / build-browser (push) Has been skipped
Deploy Trade-In / build-backend (push) Has been skipped
Deploy Trade-In / deploy (push) Successful in 43s
Deploy Trade-In / build-frontend (push) Successful in 1m52s
2026-07-04 13:49:18 +00:00
27868a4d2e feat(exporters): зеркалим permits_nearby (ГИСОГД 500м) в DOCX §6 (#2421)
All checks were successful
Deploy / build-frontend (push) Has been skipped
Deploy / build-backend (push) Successful in 2m5s
Deploy / deploy (push) Successful in 1m31s
Deploy / changes (push) Successful in 7s
Deploy / build-worker (push) Successful in 2m58s
2026-07-04 13:28:00 +00:00
9c6afcc876 Merge pull request 'chore(tradein/scrapers): удалить весь legacy scrapers/ каталог — final E (#2397, #2277)' (#2419) from chore/tradein-finalE-delete-scrapers-catalog into main
All checks were successful
Deploy Trade-In / changes (push) Successful in 10s
Deploy Trade-In / build-frontend (push) Has been skipped
Deploy Trade-In / build-browser (push) Has been skipped
Deploy Trade-In / test (push) Successful in 42s
Deploy Trade-In / build-backend (push) Successful in 55s
Deploy Trade-In / deploy (push) Successful in 1m45s
2026-07-04 13:11:13 +00:00
921aba04b8 feat(tradein/history): вернуть все параметры оценки в GET /history (#2420)
Some checks failed
Deploy Trade-In / deploy (push) Blocked by required conditions
Deploy Trade-In / changes (push) Successful in 10s
Deploy Trade-In / build-frontend (push) Has been skipped
Deploy Trade-In / build-browser (push) Has been skipped
Deploy Trade-In / test (push) Successful in 56s
Deploy Trade-In / build-backend (push) Has been cancelled
2026-07-04 13:09:22 +00:00
0658399f01 fix(devops): cleanup-merged-worktrees.sh — 3 бага, найденные вживую на 69 забытых worktree (#2418) 2026-07-04 13:00:14 +00:00
bot-backend
27bbd579c3 chore(tradein/scrapers): удалить весь legacy scrapers/ каталог — final E (#2397, #2277)
All checks were successful
CI Trade-In / changes (pull_request) Successful in 9s
CI / changes (pull_request) Successful in 9s
CI Trade-In / frontend-checks (pull_request) Has been skipped
CI / backend-tests (pull_request) Has been skipped
CI / frontend-tests (pull_request) Has been skipped
CI / openapi-codegen-check (pull_request) Has been skipped
CI Trade-In / backend-tests (pull_request) Successful in 47s
Удаляет весь `app/services/scrapers/` (16 файлов, ~7100 строк) — Part D (D1-D5)
убрал всех внешних вызывающих, 0 runtime importers подтверждено grep'ом на main.

Заодно:
- удалены 7 осиротевших локальных probe/sweep-скриптов (tradein-mvp/scripts/),
  импортировавших уже-удалённые или удаляемые сейчас legacy-модули
- тест-хирургия по 65+ файлам: DELETE прямых legacy-юнит-тестов, RETARGET
  тестов, тестирующих ещё живую бизнес-логику (переключены на scraper_kit.*
  эквиваленты, включая quality-gate #781/#753/#754/#755/#773/#740), partial-delete
  golden-parity тестов, потерявших legacy-оракл
- kit save_listings/AvitoScraper/etc. требуют инжектируемые matcher/config —
  ретаргетированные тесты обновлены под новую сигнатуру (RealScraperConfig(),
  MagicMock HouseMatcher, region_code=66)

Полный pytest suite зелёный (2255 passed, 6 skipped) кроме известного флейка
#2208 (test_search_cache_hit, не связан со scrapers).
2026-07-04 15:58:15 +03:00
a0c609caf4 fix(tradein/estimator): устранить остаточные кросс-сорс дубли (#2291) (#2417)
All checks were successful
Deploy Trade-In / changes (push) Successful in 10s
Deploy Trade-In / build-frontend (push) Has been skipped
Deploy Trade-In / build-browser (push) Has been skipped
Deploy Trade-In / test (push) Successful in 57s
Deploy Trade-In / build-backend (push) Successful in 58s
Deploy Trade-In / deploy (push) Successful in 48s
2026-07-04 12:58:15 +00:00
8030f8ecef Merge pull request 'refactor(tradein/tasks): yandex_newbuilding_sweep на kit + config= (#2397 Part D5)' (#2412) from refactor/tradein-partD5-yandex-nb-sweep-kit into main
All checks were successful
Deploy Trade-In / build-frontend (push) Has been skipped
Deploy Trade-In / build-browser (push) Has been skipped
Deploy Trade-In / test (push) Successful in 57s
Deploy Trade-In / build-backend (push) Successful in 53s
Deploy Trade-In / changes (push) Successful in 11s
Deploy Trade-In / deploy (push) Successful in 1m9s
2026-07-04 12:29:56 +00:00
00a4dca5c0 Merge pull request 'refactor(tradein/admin): debug newbuilding endpoints на kit + config=, fix stray await (#2397 Part D4)' (#2414) from refactor/tradein-partD4-admin-newbuilding-kit into main
Some checks failed
Deploy Trade-In / test (push) Blocked by required conditions
Deploy Trade-In / build-backend (push) Blocked by required conditions
Deploy Trade-In / build-frontend (push) Blocked by required conditions
Deploy Trade-In / build-browser (push) Blocked by required conditions
Deploy Trade-In / deploy (push) Blocked by required conditions
Deploy Trade-In / changes (push) Has been cancelled
2026-07-04 12:29:51 +00:00
f0a9763e48 Merge pull request 'refactor(tradein/tasks): newbuilding_enrich_backfill на kit cian.newbuilding + config= (#2397 Part D3)' (#2415) from refactor/tradein-partD3-newbuilding-enrich-kit into main
Some checks are pending
Deploy Trade-In / test (push) Blocked by required conditions
Deploy Trade-In / build-backend (push) Blocked by required conditions
Deploy Trade-In / build-frontend (push) Blocked by required conditions
Deploy Trade-In / build-browser (push) Blocked by required conditions
Deploy Trade-In / deploy (push) Blocked by required conditions
Deploy Trade-In / changes (push) Successful in 9s
2026-07-04 12:29:41 +00:00
b28aefec16 Merge pull request 'refactor(tradein/tasks): cian_history_backfill valuation+newbuilding на kit + config= (#2397 Part D2)' (#2411) from refactor/tradein-partD2-cian-history-backfill-kit into main
Some checks failed
Deploy Trade-In / test (push) Blocked by required conditions
Deploy Trade-In / build-backend (push) Blocked by required conditions
Deploy Trade-In / build-frontend (push) Blocked by required conditions
Deploy Trade-In / build-browser (push) Blocked by required conditions
Deploy Trade-In / deploy (push) Blocked by required conditions
Deploy Trade-In / changes (push) Has been cancelled
2026-07-04 12:29:33 +00:00
c8343c8f3d Merge pull request 'refactor(tradein/tasks): avito_detail_backfill build_warmed_session на kit + config= (#2397 Part D1, #2330)' (#2410) from refactor/tradein-partD1-avito-detail-backfill-kit into main
Some checks failed
Deploy Trade-In / build-backend (push) Blocked by required conditions
Deploy Trade-In / deploy (push) Blocked by required conditions
Deploy Trade-In / changes (push) Successful in 10s
Deploy Trade-In / build-frontend (push) Has been skipped
Deploy Trade-In / build-browser (push) Has been skipped
Deploy Trade-In / test (push) Has been cancelled
2026-07-04 12:29:18 +00:00
2a3a93b138 fix(devops): couchdb healthcheck ложный 401 — auth /_up кредами (#2413) 2026-07-04 12:20:59 +00:00
bot-backend
5015109f31 refactor(tradein/tasks): newbuilding_enrich_backfill на kit cian.newbuilding + config= (#2397 Part D3)
All checks were successful
CI Trade-In / changes (pull_request) Successful in 8s
CI / changes (pull_request) Successful in 7s
CI Trade-In / frontend-checks (pull_request) Has been skipped
CI / backend-tests (pull_request) Has been skipped
CI / frontend-tests (pull_request) Has been skipped
CI / openapi-codegen-check (pull_request) Has been skipped
CI Trade-In / backend-tests (pull_request) Successful in 1m2s
2026-07-04 15:20:18 +03:00
bot-backend
6ace4ea4ce refactor(tradein/admin): debug newbuilding endpoints на kit + config=, fix stray await (#2397 Part D4)
All checks were successful
CI Trade-In / changes (pull_request) Successful in 8s
CI / changes (pull_request) Successful in 7s
CI Trade-In / frontend-checks (pull_request) Has been skipped
CI / backend-tests (pull_request) Has been skipped
CI / frontend-tests (pull_request) Has been skipped
CI / openapi-codegen-check (pull_request) Has been skipped
CI Trade-In / backend-tests (pull_request) Successful in 1m1s
2026-07-04 15:20:03 +03:00
bot-backend
b6b7af6b42 refactor(tradein/tasks): yandex_newbuilding_sweep на kit + config= (#2397 Part D5)
All checks were successful
CI Trade-In / changes (pull_request) Successful in 9s
CI / changes (pull_request) Successful in 9s
CI Trade-In / frontend-checks (pull_request) Has been skipped
CI / backend-tests (pull_request) Has been skipped
CI / frontend-tests (pull_request) Has been skipped
CI / openapi-codegen-check (pull_request) Has been skipped
CI Trade-In / backend-tests (pull_request) Successful in 1m1s
2026-07-04 15:18:40 +03:00
bot-backend
f0c7ac4972 refactor(tradein/tasks): cian_history_backfill valuation+newbuilding на kit + config= (#2397 Part D2)
All checks were successful
CI / changes (pull_request) Successful in 8s
CI Trade-In / changes (pull_request) Successful in 9s
CI Trade-In / frontend-checks (pull_request) Has been skipped
CI / backend-tests (pull_request) Has been skipped
CI / frontend-tests (pull_request) Has been skipped
CI / openapi-codegen-check (pull_request) Has been skipped
CI Trade-In / backend-tests (pull_request) Successful in 1m4s
2026-07-04 15:17:27 +03:00
bot-backend
36c451e155 refactor(tradein/tasks): avito_detail_backfill build_warmed_session на kit + config= (#2397 Part D1, #2330)
All checks were successful
CI Trade-In / changes (pull_request) Successful in 7s
CI / changes (pull_request) Successful in 7s
CI Trade-In / frontend-checks (pull_request) Has been skipped
CI / backend-tests (pull_request) Has been skipped
CI / frontend-tests (pull_request) Has been skipped
CI / openapi-codegen-check (pull_request) Has been skipped
CI Trade-In / backend-tests (pull_request) Successful in 58s
2026-07-04 15:15:45 +03:00
eafa305ac8 Merge pull request 'refactor(tradein/scrapers): удалить 6 осиротевших legacy scraper-модулей + тест-хирургия (#2397 Part E2)' (#2407) from chore/tradein-partE2-delete-orphaned-scrapers into main
All checks were successful
Deploy Trade-In / build-backend (push) Successful in 54s
Deploy Trade-In / changes (push) Successful in 11s
Deploy Trade-In / build-frontend (push) Has been skipped
Deploy Trade-In / build-browser (push) Has been skipped
Deploy Trade-In / test (push) Successful in 1m1s
Deploy Trade-In / deploy (push) Successful in 1m46s
2026-07-04 11:49:44 +00:00
bot-backend
575c233e51 refactor(tradein/scrapers): удалить 6 осиротевших legacy scraper-модулей + тест-хирургия (#2397 Part E2)
All checks were successful
CI / frontend-tests (pull_request) Has been skipped
CI / openapi-codegen-check (pull_request) Has been skipped
CI Trade-In / backend-tests (pull_request) Successful in 1m2s
CI Trade-In / changes (pull_request) Successful in 8s
CI / changes (pull_request) Successful in 8s
CI Trade-In / frontend-checks (pull_request) Has been skipped
CI / backend-tests (pull_request) Has been skipped
app.services.scrapers.{cian,cian_detail,yandex_realty,domclick,domclick_exceptions,
avito_houses} имели 0 runtime-импортёров после E1 (scrape_pipeline.py удалён,
#2405) — admin.py/cian_history_backfill.py уже на scraper_kit.providers.*.

Удаление:
- 6 модулей целиком (avito_houses.py, cian.py, cian_detail.py, domclick.py,
  domclick_exceptions.py, yandex_realty.py)
- 3 осиротевших dev-sweep скрипта (local-sweep-ekb{,-cian,-yandex}.py) —
  эксклюзивно драйвили удалённые легаси-скраперы, kit-путь их заменяет

Тесты (18 файлов DELETE — эксклюзивно тестировали удалённые модули или были
чистым legacy-vs-kit parity, чей легаси-эталон исчез):
test_cian_detail.py, test_cian_serp_scraper.py, tests/scrapers/test_cian_exhaustive.py,
test_yandex_realty_serp.py, tests/scrapers/test_domclick.py,
test_avito_houses_dedup.py, test_avito_houses_parse.py,
test_houses_rating_filter_entries_1789.py, test_scraper_kit_domclick_golden_parity.py,
tests/scrapers/test_admin_{avito,cian,yandex}_kit_parity.py.

CONVERT (файл покрывает СМЕШАННЫЙ live+legacy набор — вырезаны только
секции удалённых модулей, live-покрытие сохранено):
- test_scraper_kit_cian_golden_parity.py — SERP/`_format_address`/detail
  (cian.py/cian_detail.py) убраны; extract_state (cian_state_parser, live)/
  valuation (cian_valuation, live)/newbuilding (cian_newbuilding, live)/
  no-app-imports guard — остались
- test_scraper_kit_yandex_golden_parity.py — SERP `_entity_to_lot`/
  `_parse_gate_json`/`_build_url` (yandex_realty) убраны; normalize_house_type
  (live)/newbuilding parse (yandex_newbuilding, live)/no-app-imports guard —
  остались
- test_scraper_kit_bisection_parity.py — cian/domclick/yandex-ветки убраны;
  avito-ветка (live, единственная оставшаяся легаси-бисекция) — осталась
  полностью, включая full-seed-grid
- test_scraper_kit_group_c_backfill_kit_parity.py — cian fetch_detail
  browser_fetcher parity-тест убран; avito/house_imv_backfill/
  cian_history_backfill/ekb_geoportal тесты — остались
- test_scraper_kit_pricehistory_session_parity.py — fetch_detail
  config-injection/proxy-wiring тесты (cian_detail) убраны; extract_state
  (cian_state_parser, live)/ScrapedLot (base, live) parity — остались
- test_yandex_scrapers_delay_wiring.py — yandex_realty delay-тест убран
  (kit-эквивалент уже покрыт test_kit_serp_proxy_pool.py); yandex_newbuilding
  (live)/yandex_detail/yandex_valuation (kit) тесты — остались
- test_scraper_proxy.py — yandex_realty BrowserFetcher-тест убран (redundant
  с test_kit_serp_proxy_pool.py::test_yandex_serp_no_pool_by_default)

RETARGET на kit (символ живёт, только источник поменялся):
- test_domclick_sweep.py — `_map_item` → scraper_kit.providers.domclick.serp
- test_yandex_city_sweep.py — DEFAULT_PRICE_RANGES/ROOM_PATH →
  scraper_kit.providers.yandex.serp
- test_pipeline_browser_routing.py — fetch_house_catalog →
  scraper_kit.providers.avito.houses (+ KitAvitoBlockedError для firewall-теста)
- test_snapshot_writer.py — DetailEnrichment/save_detail_enrichment →
  scraper_kit.providers.cian.detail (byte-identical strangler-копия)
- test_backfill_wave2.py — DetailEnrichment → scraper_kit.providers.cian.detail
- tests/scrapers/test_avito_unix_date_tz_consistency.py — avito_houses →
  scraper_kit.providers.avito.houses (houses-ветка _unix_to_date/RUS_MONTHS
  consolidation-проверки)

Stale-комментарии поправлены в cian_price_history.py, tests/scrapers/
test_domclick_detail.py (описывали ещё-живой domclick.py/domclick_exceptions.py).

grep -rn scrapers\.\(cian\|cian_detail\|yandex_realty\|domclick\|domclick_exceptions\|
avito_houses\) app/ scripts/ packages/ — только historical footnotes
("strangler-копия app.services.scrapers.X", past-tense "удалён/deleted").

Backend suite: 2811 passed, 6 skipped, 1 failed (known flake test_search_cache_hit
#2208, не регрессия). ruff check чист на всех изменённых файлах.
2026-07-04 14:37:19 +03:00
84716e61de Merge pull request 'refactor(tradein/scrapers): удалить legacy scrape_pipeline.py + тест-хирургия (#2397 Part E1)' (#2405) from chore/tradein-partE1-delete-scrape-pipeline into main
All checks were successful
Deploy Trade-In / changes (push) Successful in 11s
Deploy Trade-In / build-frontend (push) Has been skipped
Deploy Trade-In / build-browser (push) Has been skipped
Deploy Trade-In / test (push) Successful in 1m37s
Deploy Trade-In / build-backend (push) Successful in 1m20s
Deploy Trade-In / deploy (push) Successful in 1m15s
2026-07-04 11:09:39 +00:00
bot-backend
79a77200d7 refactor(tradein/scrapers): удалить legacy scrape_pipeline.py + тест-хирургия (#2397 Part E1)
All checks were successful
CI Trade-In / changes (pull_request) Successful in 10s
CI / changes (pull_request) Successful in 9s
CI Trade-In / frontend-checks (pull_request) Has been skipped
CI / backend-tests (pull_request) Has been skipped
CI / frontend-tests (pull_request) Has been skipped
CI / openapi-codegen-check (pull_request) Has been skipped
CI Trade-In / backend-tests (pull_request) Successful in 1m43s
app.services.scrape_pipeline (3734 строк) имел 0 runtime-импортёров — Part C убрал
последнего вызывающего (admin.py, scheduler_main.py уже на
scraper_kit.orchestration.pipeline). Удаление:

- backend/app/services/scrape_pipeline.py — удалён целиком

Тесты (18 файлов, импортировавших scrape_pipeline):

DELETE (чистая legacy-оркестрация, kit-эквивалент покрыт в
test_scraper_kit_pipeline_parity{,2}.py):
- test_scrape_pipeline.py, test_scrape_pipeline_proxy_pool.py (proxy-pool DB-lease
  вытеснен RealProxyProvider DI-адаптером, покрытым test_scraper_adapters_contracts.py)
- test_anchor_watchdog.py, test_ban_rotation_1790.py, test_provider_rotation_1848.py
- test_scraper_resilience_1949_1950.py, test_sweep_drain.py

CONVERT (parity-сравнение убрано, kit-only regression оставлен/усилен):
- test_scraper_kit_pipeline_parity.py, test_scraper_kit_pipeline_parity2.py —
  _drive_old убран, _drive_new стал единственной веткой; ассёрты на конкретные
  значения counters сохранены (это теперь основное regression-покрытие kit
  orchestration)
- test_sweep_imv_phase.py — 6 IMV-phase тестов (enrich_imv true/false/no-touched/
  crash/failed-counter/cancel) переведены на run_avito_city_sweep (kit) с DI
  (config/matcher/enrichment); enrichment.process_houses_imv_batch — инжектируемый
  Protocol-метод вместо прямого импорта app.services.house_imv_backfill

PARTIAL-EDIT (легаси sweep-behavior вырезан, admin/kit-DI/несвязанные тесты
оставлены):
- test_city_sweep.py — только retarget 4 импортов на scraper_kit.orchestration.pipeline
- test_cian_city_sweep.py, test_yandex_city_sweep.py — sweep-behavior секции удалены,
  admin endpoint тесты (уже kit-DI) + anchor-timeout/watchdog формулы (retarget import)
  оставлены
- test_domclick_sweep.py — sweep-wiring тесты удалены, Counters + _map_item оставлены
- test_avito_newbuilding_sweep.py — sweep-wiring тест #5 удалён, retarget
  NewbuildingSweepCounters
- test_pipeline_browser_routing.py — 3 run_avito_pipeline теста удалены, browser-routing
  на уровне fetch_detail/fetch_house_catalog/AvitoScraper не тронут
- test_scraper_proxy.py, tests/scrapers/test_avito_anti_bot.py — _avito_proxies()
  переведён на kit (DI-параметр вместо settings-patch); pipeline-level тесты удалены

Stale-комментарии (scrape_pipeline.py как текущая реальность) поправлены в
admin.py, avito_detail_backfill.py, house_imv_backfill.py, yandex_price_history.py,
scrapers/avito_detail.py + kit-пакете (orchestration/__init__.py, orchestration
/pipeline.py, providers/avito/detail.py — strangler завершён, легаси удалён).

grep -rn scrape_pipeline app/ scripts/ packages/ — только past-tense упоминания
("удалён в Part E1", "перед его удалением") + один historical footnote
(contracts.py:138, provenance-заметка "собраны grep'ом по ... .py").

Backend suite: 3088 passed, 6 skipped (известный flake test_search_cache_hit #2208 —
не регрессия). ruff check чист на всех изменённых файлах.
2026-07-04 13:56:06 +03:00
2e345acb82 fix(tradein/scraper-kit): клампить diff_percent перед INSERT offer_price_history (#2404)
All checks were successful
Deploy Trade-In / changes (push) Successful in 11s
Deploy Trade-In / build-frontend (push) Has been skipped
Deploy Trade-In / build-browser (push) Has been skipped
Deploy Trade-In / test (push) Successful in 1m38s
Deploy Trade-In / build-backend (push) Successful in 1m28s
Deploy Trade-In / deploy (push) Successful in 1m45s
2026-07-04 10:44:26 +00:00
ab250a132b feat(site-finder): PermitsNearbyBlock — разрешения на строительство в §6 (#2367 PR-2, frontend-half) (#2403)
All checks were successful
Deploy / changes (push) Successful in 9s
Deploy / build-backend (push) Has been skipped
Deploy / build-worker (push) Has been skipped
Deploy / build-frontend (push) Successful in 3m26s
Deploy / deploy (push) Successful in 1m21s
2026-07-04 10:24:48 +00:00
828d691c0d Merge pull request 'refactor(tradein/scheduler): удалить legacy scheduler_loop, kit единственный путь (#2397 Part C)' (#2402) from refactor/tradein-remove-legacy-scheduler-loop into main
All checks were successful
Deploy Trade-In / changes (push) Successful in 13s
Deploy Trade-In / build-frontend (push) Has been skipped
Deploy Trade-In / build-browser (push) Has been skipped
Deploy Trade-In / test (push) Successful in 1m43s
Deploy Trade-In / build-backend (push) Successful in 56s
Deploy Trade-In / deploy (push) Successful in 7m42s
2026-07-04 10:04:15 +00:00
bot-backend
c30581ae8d test(tradein/scheduler): нетавтологичный kit registry-completeness guard (#2397 Part C)
All checks were successful
CI Trade-In / changes (pull_request) Successful in 14s
CI / changes (pull_request) Successful in 14s
CI Trade-In / frontend-checks (pull_request) Has been skipped
CI / backend-tests (pull_request) Has been skipped
CI / frontend-tests (pull_request) Has been skipped
CI / openapi-codegen-check (pull_request) Has been skipped
CI Trade-In / backend-tests (pull_request) Successful in 1m57s
Deep-review нашёл: удалённый test_kit_registry_completeness.py заменялся
test_routing_coverage_sets_match, но тот строил registry ИЗ хардкоженного
_PRODUCT_SOURCES через recording-stub и проверял тот же набор — тавтология,
не гоняющая реальный build_product_handlers(). Плюс _PRODUCT_SOURCES был
устаревшим: не хватало cian_history_backfill, deals_freshness_monitor,
osm_poi_ekb_refresh (все три есть в реальном build_product_handlers).

Фикс:
- _PRODUCT_SOURCES дополнен тремя пропущенными source'ами, сверен построчным
  grep '"...": Handler(' по product_handlers.py (20 exact + 1 wildcard).
  Список остаётся canonical hardcode (источник правды — сам product_handlers.py
  + scrape_schedules seed-миграции), с явным комментарием "поддерживается вручную".
- test_routing_coverage_sets_match удалён, заменён
  test_real_build_product_handlers_covers_all_scheduled_sources: зовёт РЕАЛЬНЫЙ
  product_handlers.build_product_handlers(ctx=None) + build_registry (не stub),
  ассертит resolve_handler для каждого source из _PRODUCT_SOURCES ∪
  _KIT_NATIVE_SOURCES. Уроненная handler-entry или scheduled source без
  handler'а теперь роняет CI.

Регресс-пруф (сделан вручную, откачен): временно закомментировал
"osm_poi_ekb_refresh" в build_product_handlers → новый тест упал с
"misses source=osm_poi_ekb_refresh"; вернул строку → тест снова зелёный
(git diff на product_handlers.py пуст после отката).

Verify: full pytest 3179 passed / 6 skipped / 1 known-unrelated fail
(test_search_cache_hit, #2208); ruff 0.7.4 чист на изменённых файлах.
2026-07-04 13:00:30 +03:00
bot-backend
85059aeb1b refactor(tradein/scheduler): удалить legacy scheduler_loop + scraper-scheduling, kit единственный путь (#2397 Part C)
Топология подтверждена перед удалением (docker-compose.prod.yml): tradein-backend
(uvicorn app.main:app) — SCHEDULER_ENABLE=false; tradein-scraper (python -m
app.scheduler_main) — SCHEDULER_ENABLE=true + USE_KIT_SCHEDULER=true. Kit-путь
(_run_kit_scheduler → scraper_kit.orchestration.scheduler + product_handlers)
самодостаточен: не импортирует ничего из app.services.scheduler.scheduler_loop
или app.services.scrape_pipeline. Все НЕ-sweep джобы, которые kit-scheduler
диспетчерит через build_product_handlers, идут напрямую в app.tasks.*/
app.services.* (либо lazy-импортят import_rosreestr_dkp/_execute_cian_backfill
из scheduler.py) — мимо удаляемой legacy-машинерии.

app/services/scheduler.py: 2098 → 418 строк. Удалено: scheduler_loop,
get_due_schedules, reap_zombies, _claim_run, _defer_next_run_at, _spawn_tracked/
_drain_inflight/_inflight_tasks, все 27 trigger_*_run-функций, импорт
app.services.scrape_pipeline, константы SCHEDULER_TICK_SEC/ZOMBIE_THRESHOLD_HOURS
(достижимы были только через удалённый scheduler_loop-путь). Оставлено (живые
импортёры вне удалённого): compute_next_run_at + has_running_run (admin.py),
import_rosreestr_dkp + _execute_cian_backfill (lazy-импорты в
product_handlers.py — job-тела kit-handler'ов).

main.py: убран `from app.services.scheduler import scheduler_loop` + lifespan-блок
запуска (`if settings.scheduler_enable: asyncio.create_task(scheduler_loop())`);
прод-backend всегда шёл с SCHEDULER_ENABLE=false, так что это был мёртвый код.

scheduler_main.py: убрана ship-dark развилка #2192 (USE_KIT_SCHEDULER=false →
legacy scheduler_loop fallback) — _run_kit_scheduler() теперь безусловный путь.
Поле settings.use_kit_scheduler оставлено в конфиге (Settings extra="ignore"
защищает от startup-краха на leftover env var), но на ветвление не влияет.

app.services.scrape_pipeline: 0 runtime-импортёров в app/+scripts/+packages/
после этого PR (только тесты, которые Part E удалит вместе с самим файлом) —
подтверждено grep. scrape_pipeline.py не тронут (Part E).

Тесты: удалены test_house_imv_backfill_scheduler.py (100% legacy-триггер,
backfill_house_imv сервис покрыт в test_house_imv_backfill_browser_flag.py /
test_backfill_wave2.py) и test_kit_registry_completeness.py (parity-инвариант
против удалённого dispatch, дублирует test_scraper_kit_scheduler_parity.py).
Точечно вырезаны "Scheduler wiring" секции (trigger_fn_exists/dispatch_branch_
wired/runs_in_executor) из ~10 файлов, тестирующих сами task-функции — сами
task-тесты (SQL-shape, миграции, fake-db поведение) оставлены нетронутыми.
test_scheduler.py: 825 → ~90 строк (остались только compute_next_run_at-тесты).
test_scraper_kit_scheduler_parity.py: убрана golden-parity секция против
удалённого scheduler_loop (SOURCE_TO_OLD_TRIGGER/_drive_old_one_tick/
test_routing_parity_per_source), остальное (claim/reap_zombies/dispatch/
registry-shape тесты kit-модуля) сохранено — источник этих инвариантов не
app.services.scheduler, а сам scraper_kit.orchestration.scheduler.
test_scheduler_main.py: 2 теста, патчившие app.services.scheduler.scheduler_loop,
переведены на монкипатч sm._run_kit_scheduler (единственный путь после этого PR).
test_sweep_imv_phase.py:171-371 (6 прямых импортов run_avito_city_sweep из
scrape_pipeline) намеренно НЕ тронуты — Part E.

Verify: полный pytest 3179 passed / 6 skipped / 1 known-unrelated fail
(test_search_cache_hit, #2208, не связан с этим PR); ruff 0.7.4 чист на всех
изменённых файлах; `python -c "import app.main; import app.scheduler_main"` OK.
2026-07-04 13:00:30 +03:00
b3a59b7cf1 feat(gisogd66): permits_nearby — точный радиус-запрос РНС/РВЭ (#2367 PR-2, backend-half) (#2401)
All checks were successful
Deploy / changes (push) Successful in 8s
Deploy / build-backend (push) Successful in 2m25s
Deploy / build-worker (push) Successful in 3m38s
Deploy / build-frontend (push) Successful in 4m18s
Deploy / deploy (push) Successful in 1m51s
2026-07-04 09:59:49 +00:00
5d51eae705 Merge pull request 'refactor(tradein/admin): debug sweep endpoints на kit orchestration (#2397 slice A)' (#2400) from refactor/tradein-admin-debug-sweeps-to-kit into main
All checks were successful
Deploy Trade-In / changes (push) Successful in 10s
Deploy Trade-In / build-frontend (push) Has been skipped
Deploy Trade-In / build-browser (push) Has been skipped
Deploy Trade-In / test (push) Successful in 1m30s
Deploy Trade-In / build-backend (push) Successful in 54s
Deploy Trade-In / deploy (push) Successful in 49s
2026-07-04 09:16:57 +00:00
bot-backend
d0b011d388 test(tradein/admin): перенацелить patch-таргеты на kit + DI-guard asserts (#2397 slice A)
All checks were successful
CI Trade-In / changes (pull_request) Successful in 7s
CI / changes (pull_request) Successful in 7s
CI Trade-In / frontend-checks (pull_request) Has been skipped
CI / backend-tests (pull_request) Has been skipped
CI / frontend-tests (pull_request) Has been skipped
CI / openapi-codegen-check (pull_request) Has been skipped
CI Trade-In / backend-tests (pull_request) Successful in 1m37s
После миграции admin.py на scraper_kit.orchestration.pipeline (предыдущий
коммит) 4 endpoint-теста продолжали патчить устаревший
app.services.scrape_pipeline.run_* — имя, на которое admin.py больше не
ссылается. Мок не перехватывал вызов → FastAPI BackgroundTask реально
запускал kit-sweep (сетевой I/O), что локально (без :5432) тихо падало
на подключении к БД и глоталось except'ом, а на CI грозило зависанием/таймаутом.

Перенацелены patch-таргеты на app.api.v1.admin.run_* (namespace, куда
забинжен kit-импорт):
- test_city_sweep.py::test_start_endpoint_ok -> run_avito_city_sweep
- test_cian_city_sweep.py::test_cian_start_endpoint_ok -> run_cian_city_sweep
- test_yandex_city_sweep.py::test_yandex_start_endpoint_ok -> run_yandex_city_sweep
- test_sweep_imv_phase.py::test_sweep_endpoint_passes_enrich_imv -> run_avito_city_sweep

Добавлены DI-guard assertions (закрывают coverage-gap: раньше проверялся
только HTTP 200, не факт вызова kit с правильным DI) — mock_sweep.call_args.kwargs
проверяется на RealScraperConfig/RealMatcherAdapter/RealEnrichmentJobs (там, где
kit-сигнатура их требует) + наличие proxy_provider.
2026-07-04 12:14:14 +03:00
bot-backend
42b1ca6d7f refactor(tradein/admin): debug sweep endpoints на kit orchestration (#2397 slice A)
Мигрирует 5 admin debug-роутов с legacy app.services.scrape_pipeline на
scraper_kit.orchestration.pipeline (эпик #2277 decommission, umbrella #2397):

- POST /scrape/avito-city-sweep -> run_avito_city_sweep
- POST /scrape/cian-city-sweep -> run_cian_city_sweep
- POST /scrape/cian-full-load -> run_cian_full_load
- POST /scrape/yandex-city-sweep -> run_yandex_city_sweep
- POST /scrape/yandex-full-load -> run_yandex_full_load

DI-паттерн зеркалит app.scheduler_main._run_kit_scheduler /
scraper_kit.orchestration.scheduler._job_* (уже используется в этом же файле
для Group A #2305 debug-роутов): config=RealScraperConfig(),
matcher=RealMatcherAdapter(), proxy_provider=_kit_proxy_provider() для всех
пяти + enrichment=RealEnrichmentJobs() там, где kit-сигнатура его требует
(avito_city_sweep, yandex_city_sweep, yandex_full_load — cian-варианты
enrichment не принимают). shutdown_requested не прокидывается — у admin
BackgroundTasks нет SIGTERM-drain семантики, дефолт kit (lambda: False)
эквивалентен прежнему поведению.

Разблокирует финальное удаление зависимости admin.py от scrape_pipeline.py.
Production-scheduler (app/services/scheduler.py, USE_KIT_SCHEDULER=False
ship-dark) остаётся на legacy — это отдельный, более крупный шаг миграции
SERP-оркестрации.
2026-07-04 12:14:14 +03:00
27ba08b821 Merge pull request 'refactor(tradein/avito): avito_detail_backfill off scrape_pipeline helpers (#2397 slice B)' (#2399) from refactor/tradein-extract-avito-scrape-helpers into main
All checks were successful
Deploy Trade-In / changes (push) Successful in 10s
Deploy Trade-In / build-frontend (push) Has been skipped
Deploy Trade-In / build-browser (push) Has been skipped
Deploy Trade-In / test (push) Successful in 1m32s
Deploy Trade-In / build-backend (push) Successful in 57s
Deploy Trade-In / deploy (push) Successful in 1m30s
2026-07-04 09:01:40 +00:00
bot-backend
8ca529ece0 refactor(tradein/avito): avito_detail_backfill off scrape_pipeline helpers (#2397 slice B)
All checks were successful
CI Trade-In / frontend-checks (pull_request) Has been skipped
CI / backend-tests (pull_request) Has been skipped
CI / frontend-tests (pull_request) Has been skipped
CI Trade-In / changes (pull_request) Successful in 7s
CI / changes (pull_request) Successful in 7s
CI Trade-In / backend-tests (pull_request) Successful in 1m38s
CI / openapi-codegen-check (pull_request) Has been skipped
app/tasks/avito_detail_backfill.py imported _CHROME_HEADERS/_avito_proxies from
the legacy app/services/scrape_pipeline.py (the only two symbols it needed from
that module). scrape_pipeline.py is slated for wholesale deletion in Part E of
epic #2277 -- this severs the last dependency so that deletion won't break the
backfill task.

Source chosen: kit-reuse, not relocate. scraper_kit/providers/_base.py (#2358
Foundation) already extracted the identical building blocks while deduplicating
providers/avito/{serp,detail,imv}.py:
- DOCUMENT_HEADERS -- byte-identical dict to _CHROME_HEADERS (same 8 keys/values,
  Accept/Accept-Language/Cache-Control/Sec-Fetch-*/Upgrade-Insecure-Requests).
- http_proxies(proxy_url) -- same formula as _avito_proxies():
  {"http": url, "https": url} if url else None, just parameterized instead of
  reading settings.scraper_proxy_url internally (both resolve the identical
  settings singleton in production, so behavior is unchanged).

Kept the AsyncSession(...) construction inline (didn't switch to kit's
build_document_session helper) to preserve the existing
`app.tasks.avito_detail_backfill.AsyncSession` mock seam used by
tests/tasks/test_avito_detail_backfill.py -- swapping to the helper would call
curl_cffi's AsyncSession via scraper_kit.providers._base instead, silently
bypassing that patch target.

scrape_pipeline.py itself is untouched (kept its own copies, per Part E plan).

Full backend suite: 3276 passed, 1 known-unrelated fail (test_search_cache_hit,
#2208), 6 skipped.
2026-07-04 11:55:24 +03:00
155d5ec3a9 fix(tradein/v2): смонтировать форму лида (#2377) в живое v2-дерево (#2395) (#2398)
All checks were successful
Deploy Trade-In / deploy (push) Successful in 43s
Deploy Trade-In / changes (push) Successful in 10s
Deploy Trade-In / test (push) Has been skipped
Deploy Trade-In / build-browser (push) Has been skipped
Deploy Trade-In / build-backend (push) Has been skipped
Deploy Trade-In / build-frontend (push) Successful in 2m1s
2026-07-04 08:46:10 +00:00
480d4c2128 fix(tradein/estimator): kitchen_area_m2/ceiling_height_m/is_apartments comp-scoring (#2012) (#2396)
All checks were successful
Deploy Trade-In / changes (push) Successful in 10s
Deploy Trade-In / build-browser (push) Has been skipped
Deploy Trade-In / test (push) Successful in 1m33s
Deploy Trade-In / build-backend (push) Successful in 55s
Deploy Trade-In / build-frontend (push) Has been skipped
Deploy Trade-In / deploy (push) Successful in 46s
2026-07-04 08:33:17 +00:00
a7e1391814 feat(tradein/v2): replace mailto lead CTA with real form (#2377) (#2393)
All checks were successful
Deploy Trade-In / changes (push) Successful in 10s
Deploy Trade-In / test (push) Has been skipped
Deploy Trade-In / build-backend (push) Has been skipped
Deploy Trade-In / build-frontend (push) Successful in 1m57s
Deploy Trade-In / build-browser (push) Has been skipped
Deploy Trade-In / deploy (push) Successful in 44s
2026-07-04 08:03:00 +00:00
98df57827e fix(tradein/v2): audit #2081 residual — 404 short-circuit, disabled-tab tooltips, stale hex token, ДКП decode (#2392)
All checks were successful
Deploy Trade-In / changes (push) Successful in 10s
Deploy Trade-In / build-browser (push) Has been skipped
Deploy Trade-In / test (push) Successful in 1m42s
Deploy Trade-In / build-frontend (push) Successful in 2m17s
Deploy Trade-In / build-backend (push) Successful in 33s
Deploy Trade-In / deploy (push) Successful in 4m52s
2026-07-04 07:47:56 +00:00
dd222a4eb3 Merge pull request 'chore(tradein/scraper-kit): удалить 5 orphaned legacy-модулей (#2277 финальный шаг, частичный)' (#2394) from chore/tradein-delete-orphaned-legacy-scrapers into main
Some checks failed
Deploy Trade-In / changes (push) Successful in 11s
Deploy Trade-In / build-frontend (push) Has been skipped
Deploy Trade-In / build-browser (push) Has been skipped
Deploy Trade-In / test (push) Successful in 1m34s
Deploy Trade-In / build-backend (push) Successful in 55s
Deploy Trade-In / deploy (push) Has been cancelled
2026-07-04 07:44:52 +00:00
bot-backend
a9b87aa311 chore(tradein/scraper-kit): удалить 5 orphaned legacy-модулей (#2277 финальный шаг, частичный)
All checks were successful
CI Trade-In / changes (pull_request) Successful in 8s
CI / changes (pull_request) Successful in 8s
CI Trade-In / frontend-checks (pull_request) Has been skipped
CI / backend-tests (pull_request) Has been skipped
CI / frontend-tests (pull_request) Has been skipped
CI / openapi-codegen-check (pull_request) Has been skipped
CI Trade-In / backend-tests (pull_request) Successful in 1m36s
Удалены (0 runtime importers в app/, scripts/, packages/; проверено grep'ом,
включая intra-legacy импорты в оставшихся 21 legacy-модуле scrapers/):
- backend/app/services/scrapers/avito_imv.py
- backend/app/services/scrapers/domclick_detail.py
- backend/app/services/scrapers/ekb_geoportal_client.py
- backend/app/services/scrapers/yandex_detail.py
- backend/app/services/scrapers/yandex_valuation.py

Тесты — удалены (pure legacy-vs-kit parity, kit-only coverage уже есть в другом
месте):
- tests/scrapers/test_ekb_geoportal_client_kit_parity.py
- tests/scrapers/test_domclick_detail_kit_parity.py

Тесты — конвертированы в kit-only (импорт легаси заменён на scraper_kit.*,
покрытие сохранено без потерь):
- tests/scrapers/test_avito_imv_kit_parity.py (parity-тесты убраны, config-footgun
  regression на kit-стороне оставлен)
- tests/scrapers/test_avito_imv_browser_transport.py
- tests/scrapers/test_yandex_valuation_kit_migration.py (parity убрана, footgun-
  регрессия mandatory config/delay_provider оставлена)
- tests/scrapers/test_domclick_detail.py (+ exceptions переведены на
  scraper_kit.domclick_exceptions, т.к. kit parse_detail_html поднимает их)
- tests/test_avito_imv_parse.py
- tests/test_ekb_geoportal_ingest.py (client-часть; ingest/geocoder-часть не
  трогали — не зависят от удалённого модуля)
- tests/test_yandex_detail.py
- tests/test_yandex_detail_structural.py
- tests/test_yandex_valuation.py (YandexValuationScraper() → config=_KIT_CONFIG,
  kit конструктор требует обязательный config)
- tests/test_yandex_valuation_save.py
- tests/test_extval_house_id_write_path.py (только yandex_valuation-часть;
  cian_valuation остался нетронутым — живой легаси-модуль)
- tests/test_yandex_history_area_filter.py

Тесты — частично отредактированы (убраны только части про удалённые модули,
живые легаси-модули/их parity не тронуты):
- tests/scrapers/test_admin_domclick_ingest_kit_parity.py (base.py/ScrapedLot
  parity остался)
- tests/scrapers/test_admin_yandex_kit_parity.py (yandex_realty parity остался)
- tests/scrapers/test_admin_avito_kit_parity.py (avito.py/avito_houses.py parity
  остался; _parse_price parity убран)
- tests/scrapers/test_avito_unix_date_tz_consistency.py (IMV-ветка переведена на
  kit avito.imv, легаси avito/avito_houses/avito_shared/avito_detail не тронуты)
- tests/tasks/test_yandex_detail_backfill.py (subject — kit-задача, уже вызывает
  scraper_kit.providers.yandex.detail; save_detail_enrichment coverage-тесты
  внизу файла переведены на kit)
- tests/test_scraper_kit_yandex_golden_parity.py (detail/valuation parity убраны;
  serp/newbuilding/house_type_normalizer/build_url для yandex_realty/
  yandex_newbuilding не тронуты)
- tests/test_scraper_proxy.py (YandexValuation proxy-тесты переведены на kit
  config= DI; test_avito_imv_own_session_receives_proxies удалён — дублирует
  test_avito_imv_kit_parity.py footgun-тесты)
- tests/test_yandex_scrapers_delay_wiring.py (yandex_detail/yandex_valuation →
  kit delay_provider=; yandex_realty/yandex_newbuilding не тронуты)
- tests/test_scraper_kit_group_c_backfill_kit_parity.py (docstring уточнение,
  без функциональных изменений)

Полный backend-suite зелёный (3253 passed, 6 skipped) кроме известного
pre-existing flake test_search_api.py::test_search_cache_hit (#2208, не
регрессия этого PR). ruff check + ruff format — чисто на всех изменённых файлах.
2026-07-04 10:41:46 +03:00
a5097c3a97 fix(tradein/estimator): rebase ground-floor hedonic term onto current main (#2002, #2382) (#2391)
All checks were successful
Deploy Trade-In / changes (push) Successful in 12s
Deploy Trade-In / build-frontend (push) Has been skipped
Deploy Trade-In / build-backend (push) Successful in 57s
Deploy Trade-In / build-browser (push) Has been skipped
Deploy Trade-In / test (push) Successful in 1m32s
Deploy Trade-In / deploy (push) Successful in 51s
2026-07-04 07:33:37 +00:00
2538fc02e2 feat(tradein/lead): POST /api/v1/trade-in/lead — persist contact leads (#2376) (#2390)
All checks were successful
Deploy Trade-In / build-frontend (push) Has been skipped
Deploy Trade-In / build-browser (push) Has been skipped
Deploy Trade-In / changes (push) Successful in 10s
Deploy Trade-In / test (push) Successful in 1m36s
Deploy Trade-In / build-backend (push) Successful in 54s
Deploy Trade-In / deploy (push) Successful in 48s
2026-07-04 07:12:22 +00:00
293184ebc6 fix(tradein/v2): mobile layout — remove overflow floor + honest stacked mobile block (#2275) (#2389)
All checks were successful
Deploy Trade-In / changes (push) Successful in 11s
Deploy Trade-In / test (push) Has been skipped
Deploy Trade-In / build-backend (push) Has been skipped
Deploy Trade-In / build-frontend (push) Successful in 2m20s
Deploy Trade-In / build-browser (push) Has been skipped
Deploy Trade-In / deploy (push) Successful in 47s
2026-07-04 07:02:56 +00:00
4f0cae5c18 fix(gisogd66): CAST(:geojson AS text) — AmbiguousParameter ронял каждый upsert (#2383)
All checks were successful
Deploy / changes (push) Successful in 7s
Deploy / build-frontend (push) Has been skipped
Deploy / build-backend (push) Successful in 2m4s
Deploy / build-worker (push) Successful in 3m46s
Deploy / deploy (push) Successful in 1m44s
2026-07-04 06:55:18 +00:00
9850bbde14 fix(tradein/deploy): atomic recreate + graceful scraper drain + startup-reap (#1951) (#2388)
All checks were successful
Deploy Trade-In / changes (push) Successful in 9s
Deploy Trade-In / build-browser (push) Successful in 30s
Deploy Trade-In / build-frontend (push) Successful in 31s
Deploy Trade-In / test (push) Successful in 1m39s
Deploy Trade-In / build-backend (push) Successful in 56s
Deploy Trade-In / deploy (push) Successful in 55s
2026-07-04 06:49:06 +00:00
2580158d8c fix(tradein/estimator): lock honest asking_to_sold_ratio invariant + disambiguate torg bases (#2087 M2/M3) (#2387)
Some checks failed
Deploy Trade-In / build-backend (push) Blocked by required conditions
Deploy Trade-In / deploy (push) Blocked by required conditions
Deploy Trade-In / changes (push) Successful in 10s
Deploy Trade-In / build-frontend (push) Has been skipped
Deploy Trade-In / build-browser (push) Has been skipped
Deploy Trade-In / test (push) Has been cancelled
2026-07-04 06:48:51 +00:00
9b92b595db Merge pull request 'refactor(tradein/scraper-kit): yandex providers on shared _base.py (#2363)' (#2384) from feat/tradein-scraper-kit-f4c-yandex-v2 into main
All checks were successful
Deploy Trade-In / changes (push) Successful in 11s
Deploy Trade-In / build-frontend (push) Has been skipped
Deploy Trade-In / build-browser (push) Has been skipped
Deploy Trade-In / test (push) Successful in 1m38s
Deploy Trade-In / build-backend (push) Successful in 1m30s
Deploy Trade-In / deploy (push) Successful in 1m45s
2026-07-04 06:43:17 +00:00
bot-backend
7800f6014a refactor(tradein/scraper-kit): yandex providers on shared _base.py (#2363)
All checks were successful
CI / changes (pull_request) Successful in 10s
CI Trade-In / changes (pull_request) Successful in 11s
CI Trade-In / frontend-checks (pull_request) Has been skipped
CI / backend-tests (pull_request) Has been skipped
CI / frontend-tests (pull_request) Has been skipped
CI / openapi-codegen-check (pull_request) Has been skipped
CI Trade-In / backend-tests (pull_request) Successful in 1m48s
F4c-yandex (epic #2277 Group F, parent #2352): migrate the SERP scraper's
BrowserFetcher construction onto build_browser_fetcher() from the Foundation
module (#2358, providers/_base.py) instead of local ad-hoc construction.

- yandex/serp.py::__aenter__ now calls build_browser_fetcher(config, source=,
  proxy_provider=, fetch_timeout_s=) instead of constructing BrowserFetcher(...)
  directly. Preserves the existing fetch_timeout_s=30 (see caveat below),
  endpoint/proxy_provider/use_pool resolution byte-for-byte. The retry/tarpit
  logic in fetch_around() (rotate_ip + sleep on status==0/JSON-error) is
  untouched, per the Foundation module's documented exclusion.

Left unchanged (deliberate, not an oversight):
- yandex/serp.py::_rotate_ip's curl_cffi AsyncSession(timeout=30) — this hits
  the mobile-proxy provider's own changeip API, not Yandex; it intentionally
  carries no impersonate/headers/proxy (same as the mirrored avito/serp.py
  rotate-ip session). Not listed among the _base.py docstring's documented
  extraction sites -- consistent exclusion, not a new one.
- yandex/detail.py -- no BrowserFetcher/curl_cffi construction exists in this
  file (uses BaseScraper's httpx-based _http_get). Nothing to migrate.
- yandex/newbuilding.py -- both BrowserFetcher(...) call sites take an
  OPTIONAL `config: ScraperConfig | None`, a genuinely different contract from
  build_browser_fetcher's mandatory `config`. Migrating would require making
  config mandatory (breaking admin.py:1511, yandex_newbuilding_sweep.py:353,
  and several tests that call without config -- out of scope for a call-site
  swap) and breaks test_scraper_kit_newbuilding_endpoint.py's "without config,
  endpoint=None" regression case (its SimpleNamespace mock lacks
  use_proxy_pool_browser, and the mock config path can never satisfy a
  mandatory-config helper). Left untouched; the class-bug #2322/#2330 this
  file already fixed (config now threaded, just not via the shared helper)
  remains fixed.

Also updates test_kit_serp_proxy_pool.py's 3 yandex-specific tests: they
monkeypatched the module-local `yandex_serp.BrowserFetcher` name, which the
migrated call path no longer references directly (it now goes through
scraper_kit.providers._base.build_browser_fetcher -> _base's own BrowserFetcher
import). Patch target moved to scraper_kit.providers._base.BrowserFetcher;
every assertion is unchanged -- this adapts the mock to the new (intentional)
call path, it does not weaken what the test verifies. Cian's tests in the same
file are untouched (cian/serp.py is not migrated yet, separate F4b issue).
2026-07-04 09:38:43 +03:00
758a4b06d4 Merge pull request 'refactor(tradein/scraper-kit): avito providers on shared _base.py (#2361)' (#2375) from feat/tradein-scraper-kit-f4a-avito into main
All checks were successful
Deploy Trade-In / build-frontend (push) Has been skipped
Deploy Trade-In / test (push) Successful in 1m39s
Deploy Trade-In / build-backend (push) Successful in 1m23s
Deploy Trade-In / changes (push) Successful in 14s
Deploy Trade-In / build-browser (push) Has been skipped
Deploy Trade-In / deploy (push) Successful in 1m31s
2026-07-04 06:34:59 +00:00
53947015e9 Merge pull request 'refactor(tradein/scraper-kit): cian providers on shared _base.py (#2362)' (#2374) from feat/tradein-scraper-kit-f4b-cian into main
Some checks failed
Deploy Trade-In / deploy (push) Blocked by required conditions
Deploy Trade-In / changes (push) Successful in 13s
Deploy Trade-In / build-frontend (push) Has been skipped
Deploy Trade-In / build-browser (push) Has been skipped
Deploy Trade-In / test (push) Successful in 1m39s
Deploy Trade-In / build-backend (push) Has been cancelled
2026-07-04 06:32:30 +00:00
bot-backend
b445c34863 refactor(tradein/scraper-kit): avito providers on shared _base.py (#2361)
All checks were successful
CI / changes (pull_request) Successful in 10s
CI / backend-tests (pull_request) Has been skipped
CI Trade-In / changes (pull_request) Successful in 11s
CI / frontend-tests (pull_request) Has been skipped
CI / openapi-codegen-check (pull_request) Has been skipped
CI Trade-In / frontend-checks (pull_request) Has been skipped
CI Trade-In / backend-tests (pull_request) Successful in 1m44s
Мигрирует avito/{serp,detail}.py на build_browser_fetcher()/
build_document_session() из scraper_kit.providers._base (Foundation #2358):

- serp.py: BrowserFetcher(source="avito", endpoint=...) в __aenter__ ->
  build_browser_fetcher(config, "avito"); _build_cffi_session() (25s
  timeout, document headers, scraper_proxy_url) -> build_document_session().
- detail.py: _build_detail_session() (тот же document-headers dict,
  25s timeout) -> build_document_session(proxy_url=..., timeout=25).

Caveat (#2361, проверено explicit): serp.py __aenter__ раньше конструировал
BrowserFetcher БЕЗ proxy_provider/use_pool (implicit None/False). После
миграции use_pool читает config.use_proxy_pool_browser вместо hardcoded
False - НО proxy_provider остаётся None (build_browser_fetcher default), а
BrowserFetcher._use_pool гейтится `and self._proxy_provider is not None`
(browser_fetcher.py:215) - при proxy_provider=None use_pool всегда
эффективно False независимо от config. Verified: behavior-preserving,
не тихая смена прод-поведения.

imv.py: НЕ мигрирован (единственное отклонение от issue). Own-session
curl_cffi construction (headers=_DOC_HEADERS - тот же дублированный dict)
технически подходит под миграцию, но build_document_session() резолвит
AsyncSession через module-level импорт _base.py (biндится один раз при
первом импорте _base), а tests/scrapers/test_avito_imv_kit_parity.py
(#2334) патчит curl_cffi.requests.AsyncSession через unittest.mock -
это работает только с локальным call-time импортом (как было). Миграция
роняет 2 реальных теста (test_kit_evaluate_via_imv_{with,without}_config_
uses_proxy) - не шум, зелёные на main. Прод-поведение идентично в обоих
вариантах, но задача явно запрещает weaken/skip parity-тесты - оставлено
как есть + inline-комментарий с находкой. Follow-up для будущих групп
(cian/yandex/domclick F4b-d): тот же паттерн теста сломает миграцию любого
provider с аналогичным unittest.mock.patch("curl_cffi.requests.AsyncSession").

Tests: full backend suite 3278 passed, 1 known pre-existing flake
(test_search_api.py::test_search_cache_hit, #2208), 6 skipped. Все avito
kit_parity/golden_parity тесты + test_scraper_kit_pipeline_parity2.py
(cross-provider) зелёные.
2026-07-04 09:31:27 +03:00
bot-backend
96dd27c4ae refactor(tradein/scraper-kit): cian providers on shared _base.py (#2362)
All checks were successful
CI Trade-In / changes (pull_request) Successful in 12s
CI Trade-In / frontend-checks (pull_request) Has been skipped
CI / changes (pull_request) Successful in 15s
CI / backend-tests (pull_request) Has been skipped
CI / frontend-tests (pull_request) Has been skipped
CI / openapi-codegen-check (pull_request) Has been skipped
CI Trade-In / backend-tests (pull_request) Successful in 1m41s
Мигрирует call-site'ы cian/{serp,detail,newbuilding,session}.py на shared
фабрики Foundation-модуля (#2358): build_browser_fetcher() (serp.py __aenter__)
и build_curl_cffi_session() (detail.py own-session path, newbuilding.py
resolve_cian_zhk_url[_via_search], session.py verify_session). Headers/timeout/
proxy-resolution сохранены байт-в-байт — чистый call-site swap, без изменения
поведения.

Caveat (config Optional footgun): newbuilding.py::fetch_newbuilding() НЕ
переведена на build_browser_fetcher() — build_browser_fetcher() требует
ScraperConfig mandatory (весь смысл фабрики, закрывает класс багов #2322/#2330),
но здесь config: ScraperConfig | None = None легитимно Optional (admin.py
debug-роут и test_scraper_kit_newbuilding_endpoint.py вызывают функцию без
config= и ожидают graceful endpoint=None). Подмена уронила бы AttributeError
на config=None вместо текущего degrade — реальная регрессия, не call-site swap.
Оставлено как есть с комментарием в коде.

Companion test updates (не ослабление, retarget под рефактор): CianScraper
теперь строит BrowserFetcher внутри _base.py, а не в serp.py — два теста в
test_kit_serp_proxy_pool.py, патчившие cian_serp.BrowserFetcher напрямую,
переключены на scraper_kit.providers._base.BrowserFetcher (тот же инвариант,
другая точка патча). Аналогично test_scraper_kit_pricehistory_session_parity.py
::test_fetch_detail_own_session_proxy_wiring_parity — патч AsyncSession
переключён с cian.detail на _base (тот же assert на итоговый proxies=dict).

Tests: full backend suite 3278 passed, 1 pre-existing flake
(test_search_api.py::test_search_cache_hit, #2208), 6 skipped. Все
parity/regression тесты (test_scraper_kit_pipeline_parity2.py,
test_scraper_kit_newbuilding_endpoint.py, test_scraper_kit_cian_golden_parity.py,
test_kit_serp_proxy_pool.py, test_scraper_kit_pricehistory_session_parity.py,
test_cian_serp_scraper.py, test_cian_session.py, test_cian_detail.py) зелёные.
ruff check + format --check чисто на всех изменённых файлах.
2026-07-04 09:29:49 +03:00
53e7834f19 Merge pull request 'refactor(tradein/scraper-kit): domclick providers on shared _base.py (#2364)' (#2372) from feat/tradein-scraper-kit-f4d-domclick into main
All checks were successful
Deploy Trade-In / changes (push) Successful in 11s
Deploy Trade-In / build-frontend (push) Has been skipped
Deploy Trade-In / build-browser (push) Has been skipped
Deploy Trade-In / test (push) Successful in 1m35s
Deploy Trade-In / build-backend (push) Successful in 1m23s
Deploy Trade-In / deploy (push) Successful in 3m1s
2026-07-04 06:24:06 +00:00
bot-backend
debe55b8a0 refactor(tradein/scraper-kit): domclick providers on shared _base.py (#2364)
All checks were successful
CI Trade-In / changes (pull_request) Successful in 10s
CI / changes (pull_request) Successful in 10s
CI Trade-In / frontend-checks (pull_request) Has been skipped
CI / backend-tests (pull_request) Has been skipped
CI / frontend-tests (pull_request) Has been skipped
CI / openapi-codegen-check (pull_request) Has been skipped
CI Trade-In / backend-tests (pull_request) Successful in 1m42s
Group F4d (epic #2277, parent #2352). Migrated the ad-hoc BrowserFetcher
construction in providers/domclick/serp.py::fetch_city to the shared
build_browser_fetcher(config, source) helper from Foundation (#2358).

Scope actually touched:
- serp.py: replaced the local BrowserFetcher(source="domclick", endpoint=...)
  construction with build_browser_fetcher(self._config, "domclick").
- detail.py: NO changes needed. Grepped for BrowserFetcher(/AsyncSession(/
  curl_cffi and found none - fetch_detail() receives an already-constructed
  browser_fetcher from the caller (orchestrator owns the session lifecycle,
  per its own module docstring), so there was nothing to migrate here.

Caveat verified (same class as avito, per issue instructions): the old
call site omitted use_pool, so it implicitly defaulted to False via the
BrowserFetcher class default. build_browser_fetcher(config, source) reads
use_pool=config.use_proxy_pool_browser instead - a real difference in the
value passed in. However this call site does not pass proxy_provider
(stays the default None), and BrowserFetcher._pool_proxy computes
`use_pool and proxy_provider is not None` before ever using the pool - so
with proxy_provider=None the effective behavior is identical regardless
of what use_pool evaluates to (confirmed by reading browser_fetcher.py
directly, matching the build_browser_fetcher docstring's own note that
domclick is one of the providers not yet wired to the proxy pool, #2160
P4). No observable behavior change. Documented inline at the call site.

Tests: domclick-specific suite (91 tests: golden parity, sweep, admin
ingest parity, detail + detail kit parity) all green, plus cross-provider
test_scraper_kit_pipeline_parity2.py (8 tests). Full backend suite:
3278 passed, 6 skipped, 1 pre-existing flake (test_search_cache_hit,
#2208) - no new failures.
2026-07-04 09:18:04 +03:00
ae3278f92f fix(gisogd66): битый 500-сегмент портала — no-retry карточек, circuit breaker, периодический commit (#2371)
All checks were successful
Deploy / changes (push) Successful in 7s
Deploy / build-frontend (push) Has been skipped
Deploy / build-backend (push) Successful in 1m51s
Deploy / build-worker (push) Successful in 3m3s
Deploy / deploy (push) Successful in 1m31s
2026-07-04 04:33:44 +00:00
282c92747e feat(scrapers): реестр РНС/РВЭ ГИСОГД-СО из открытого API — миграция + лоадер (#2367 PR-1) (#2370)
All checks were successful
Deploy / changes (push) Successful in 6s
Deploy / build-backend (push) Successful in 2m40s
Deploy / build-worker (push) Successful in 3m27s
Deploy / build-frontend (push) Successful in 4m0s
Deploy / deploy (push) Successful in 1m41s
2026-07-04 03:26:07 +00:00
ae814706a2 feat(forecast-ui): confidence band на ForecastChart — качественный интервал уверенности (#1926) (#2368)
All checks were successful
Deploy / changes (push) Successful in 7s
Deploy / build-backend (push) Has been skipped
Deploy / build-worker (push) Has been skipped
Deploy / build-frontend (push) Successful in 3m10s
Deploy / deploy (push) Successful in 1m16s
2026-07-04 02:21:44 +00:00
57d156973e feat(chat): tool get_parcel_info — курируемый градрегламент/ЗОУИТ/ЕГРН в контекст чата (#2366)
All checks were successful
Deploy / changes (push) Successful in 7s
Deploy / build-frontend (push) Has been skipped
Deploy / build-backend (push) Successful in 1m44s
Deploy / build-worker (push) Successful in 2m46s
Deploy / deploy (push) Successful in 1m31s
2026-07-04 01:24:42 +00:00
38f703c8ec fix(landing): Trade-In nav как внешний &lt;a&gt; на /trade-in/v2 (ChunkLoadError) (#2365)
All checks were successful
Deploy / changes (push) Successful in 10s
Deploy / build-backend (push) Has been skipped
Deploy / build-worker (push) Has been skipped
Deploy / build-frontend (push) Successful in 3m18s
Deploy / deploy (push) Successful in 1m20s
2026-07-04 01:10:57 +00:00
d31fabd102 Merge pull request 'feat(tradein/scraper-kit): shared base/utils для providers/* (#2358, Foundation)' (#2360) from feat/tradein-scraper-kit-base-2358 into main
All checks were successful
Deploy Trade-In / changes (push) Successful in 9s
Deploy Trade-In / build-frontend (push) Has been skipped
Deploy Trade-In / build-browser (push) Has been skipped
Deploy Trade-In / test (push) Successful in 1m32s
Deploy Trade-In / build-backend (push) Successful in 1m16s
Deploy Trade-In / deploy (push) Successful in 1m11s
2026-07-04 00:56:13 +00:00
bot-backend
51e2e545b7 fix(tradein/scraper-kit): точность докстринга http_proxies() про httpx-совместимость
All checks were successful
CI / changes (pull_request) Successful in 7s
CI Trade-In / changes (pull_request) Successful in 7s
CI Trade-In / frontend-checks (pull_request) Has been skipped
CI / backend-tests (pull_request) Has been skipped
CI / frontend-tests (pull_request) Has been skipped
CI / openapi-codegen-check (pull_request) Has been skipped
CI Trade-In / backend-tests (pull_request) Successful in 1m35s
code-reviewer отметил: httpx>=0.28 больше не принимает dict в proxies= (только
одиночный proxy= строкой). Docstring ошибочно называл эту функцию
"httpx-совместимой" — уточнено что это только для curl_cffi, чтобы будущий
мигратор F4a-d не словил TypeError.
2026-07-04 03:50:55 +03:00
bot-backend
9b4cdf8ec6 feat(tradein/scraper-kit): shared base/utils для providers/* (#2358, Foundation)
Аддитивный Foundation-модуль (эпик #2277 Group F4): scraper_kit/providers/_base.py
извлекает 3 реально повторяющихся паттерна, найденных чтением avito/cian/yandex/
domclick providers (~13k строк):

- build_browser_fetcher(config: ScraperConfig, source, ...) — config MANDATORY,
  устраняет footgun-класс #2322/#2330 (BrowserFetcher конструировался вручную в
  6+ местах, endpoint/proxy-pool wiring легко забыть/потерять).
- build_curl_cffi_session / build_document_session + http_proxies() — общий
  curl_cffi AsyncSession(impersonate=chrome120, proxies=..., headers=...)
  boilerplate, дублированный в avito/serp.py, avito/detail.py, avito/imv.py,
  cian/detail.py, cian/newbuilding.py (x2), cian/session.py.
- DOCUMENT_HEADERS — идентичный "document" headers dict, дублированный трижды.

Retry/backoff-циклы (yandex tarpit-retry, avito page-retry, tenacity в
BaseScraper) сознательно НЕ обобщены — разная семантика ошибок/действий между
попытками, обобщение дало бы leaky abstraction (см. docstring модуля).

Existing providers/{avito,cian,yandex,domclick}/*.py НЕ тронуты — миграция
call-site'ов на новый модуль это отдельные future issues (F4a-d).

Заодно: актуализирован устаревший докстринг app/services/scraper_adapters.py
("Пока НИКТО их не использует") — адаптеры wired в прод с #2192 через
app.scheduler_main._run_kit_scheduler.
2026-07-04 03:50:55 +03:00
91c8365d91 fix(tradein/houses): персистить house_class из avito «Класс жилья» и yandex parse (#2003) (#2359)
All checks were successful
Deploy Trade-In / changes (push) Successful in 10s
Deploy Trade-In / build-frontend (push) Has been skipped
Deploy Trade-In / build-browser (push) Has been skipped
Deploy Trade-In / test (push) Successful in 1m31s
Deploy Trade-In / build-backend (push) Successful in 1m23s
Deploy Trade-In / deploy (push) Successful in 53s
2026-07-04 00:49:48 +00:00
6468bd6b37 fix(report-pdf): best-effort inline §22-форсайт на холодном участке — отчёт самодостаточен (#2357)
All checks were successful
Deploy / changes (push) Successful in 7s
Deploy / build-frontend (push) Has been skipped
Deploy / build-backend (push) Successful in 1m55s
Deploy / build-worker (push) Successful in 2m52s
Deploy / deploy (push) Successful in 1m37s
2026-07-04 00:30:12 +00:00
1e5fc13635 Merge pull request 'fix(tradein/scraper-kit): проброс endpoint= в yandex/cian newbuilding BrowserFetcher (#2322)' (#2356) from fix/tradein-scraper-kit-newbuilding-endpoint into main
All checks were successful
Deploy Trade-In / changes (push) Successful in 10s
Deploy Trade-In / build-frontend (push) Has been skipped
Deploy Trade-In / build-browser (push) Has been skipped
Deploy Trade-In / test (push) Successful in 1m30s
Deploy Trade-In / build-backend (push) Successful in 1m18s
Deploy Trade-In / deploy (push) Successful in 1m10s
2026-07-04 00:17:09 +00:00
758d7c6bf8 fix(tradein/estimator): честная маркировка quarter-precision дат ДКП + insufficient_data флаг для sell-time-sensitivity (#1995) (#2355)
Some checks failed
Deploy Trade-In / changes (push) Successful in 10s
Deploy Trade-In / build-frontend (push) Has been skipped
Deploy Trade-In / build-browser (push) Has been skipped
Deploy Trade-In / test (push) Successful in 1m36s
Deploy Trade-In / build-backend (push) Successful in 1m7s
Deploy Trade-In / deploy (push) Has been cancelled
2026-07-04 00:14:02 +00:00
bot-backend
ccf716119c fix(tradein/scraper-kit): проброс config= в pipeline.py::fetch_newbuilding call
All checks were successful
CI Trade-In / changes (pull_request) Successful in 8s
CI / changes (pull_request) Successful in 8s
CI Trade-In / frontend-checks (pull_request) Has been skipped
CI / backend-tests (pull_request) Has been skipped
CI / frontend-tests (pull_request) Has been skipped
CI / openapi-codegen-check (pull_request) Has been skipped
CI Trade-In / backend-tests (pull_request) Successful in 1m37s
Найдено при code-review этого же PR (F1, #2349) и независимо подтверждено
F3-аудитом (#2351): run_cian_city_sweep's houses-фаза звала
fetch_newbuilding(zhk_url) без config=, хотя config — mandatory параметр
enclosing-функции, тривиально в scope. Сейчас dormant (USE_KIT_SCHEDULER=False +
cian_city_sweep.enabled=false в SQL seed), но при включении флага/расписания
каждый house молча падал бы AssertionError (перехватывается except → houses_failed,
sweep не крашится целиком, но newbuilding-enrichment был бы 100% failing silently).
2026-07-04 03:12:08 +03:00
bot-backend
b89e0c3daf fix(tradein/scraper-kit): проброс endpoint= в yandex/cian newbuilding BrowserFetcher (#2322) 2026-07-04 03:12:08 +03:00
c4201f7d2a Merge pull request 'fix(tradein/scraper-kit): проброс config= в avito build_warmed_session (#2330)' (#2354) from fix/tradein-avito-warmed-session-config into main
All checks were successful
Deploy Trade-In / changes (push) Successful in 11s
Deploy Trade-In / build-browser (push) Has been skipped
Deploy Trade-In / build-frontend (push) Has been skipped
Deploy Trade-In / test (push) Successful in 1m38s
Deploy Trade-In / build-backend (push) Successful in 1m22s
Deploy Trade-In / deploy (push) Successful in 1m36s
2026-07-04 00:04:46 +00:00
c778e4c85d fix(report-ui): ретрай docx-preflight против 502-гонки сразу после ready (#2353)
All checks were successful
Deploy / changes (push) Successful in 8s
Deploy / build-backend (push) Has been skipped
Deploy / build-worker (push) Has been skipped
Deploy / build-frontend (push) Successful in 3m26s
Deploy / deploy (push) Successful in 1m19s
2026-07-04 00:03:12 +00:00
bot-backend
ecec9ff6d7 fix(tradein/scraper-kit): точность докстрингов про прод-путь build_warmed_session
All checks were successful
CI Trade-In / changes (pull_request) Successful in 10s
CI / changes (pull_request) Successful in 9s
CI Trade-In / frontend-checks (pull_request) Has been skipped
CI / backend-tests (pull_request) Has been skipped
CI / frontend-tests (pull_request) Has been skipped
CI / openapi-codegen-check (pull_request) Has been skipped
CI Trade-In / backend-tests (pull_request) Successful in 1m40s
code-reviewer отметил: формулировка "PROD DEFAULT" в докстрингах могла ввести в
заблуждение — прод-путь сегодня всё ещё легаси (avito_detail_backfill.py
импортирует build_warmed_session из legacy-модуля), этот фикс — advance-prep,
не изменение живого прод-поведения.
2026-07-04 03:01:27 +03:00
bot-backend
fea054d24f fix(tradein/scraper-kit): проброс config= в avito build_warmed_session (#2330) 2026-07-04 02:55:44 +03:00
7ebeddc2bc Merge pull request 'feat(tradein/estimator): switch core valuation call sites to scraper_kit (#2337)' (#2348) from feat/tradein-estimator-kit-switch into main
All checks were successful
Deploy Trade-In / changes (push) Successful in 11s
Deploy Trade-In / build-frontend (push) Has been skipped
Deploy Trade-In / build-browser (push) Has been skipped
Deploy Trade-In / test (push) Successful in 1m35s
Deploy Trade-In / build-backend (push) Successful in 59s
Deploy Trade-In / deploy (push) Successful in 44s
2026-07-03 23:49:10 +00:00
36fe33750f fix(report): дата сборки в POST-ответе на fast-path из кэша (последний хвост R2 #2259) (#2347)
All checks were successful
Deploy / changes (push) Successful in 6s
Deploy / build-backend (push) Successful in 2m6s
Deploy / build-frontend (push) Successful in 3m36s
Deploy / build-worker (push) Successful in 4m3s
Deploy / deploy (push) Successful in 1m34s
2026-07-03 23:44:19 +00:00
bot-backend
4a6820b8f7 feat(tradein/estimator): switch core valuation call sites to scraper_kit (#2337)
All checks were successful
CI / changes (pull_request) Successful in 9s
CI Trade-In / changes (pull_request) Successful in 9s
CI / backend-tests (pull_request) Has been skipped
CI / frontend-tests (pull_request) Has been skipped
CI / openapi-codegen-check (pull_request) Has been skipped
CI Trade-In / frontend-checks (pull_request) Has been skipped
CI Trade-In / backend-tests (pull_request) Successful in 1m48s
Group E4 (final, highest-risk step of scraper_kit migration epic #2277):
estimator.py and house_imv_backfill.py's avito_imv/cian_valuation/
yandex_valuation call sites now import from scraper_kit.providers.* instead
of app.services.scrapers.*, following the exact wiring proven safe by E1/E2/E3
(#2334/#2335/#2336):

- estimator.py's avito IMV (_get_or_fetch_imv_cached, both call sites) and
  house_imv_backfill.py's _process_one_house: add config=RealScraperConfig()
  — kit's evaluate_via_imv silently drops the configured proxy without it.
- estimator.py's cian valuation (Stage 9): add config=RealScraperConfig() —
  mandatory kwarg on the kit function (TypeError if omitted).
- estimator.py's yandex valuation: add config=RealScraperConfig() (mandatory)
  and delay_provider=get_scraper_delay — without it kit silently falls back
  to a hardcoded 5.0s throttle instead of the DB-configured anti-ban delay.
- All exception classes imported consistently from the same kit module as
  evaluate_via_imv (not just the function) — mixing legacy/kit exception
  classes would break `except IMVAddressNotFoundError` etc. via identity
  mismatch (caught by an existing test that assumed the legacy class,
  fixed alongside).

Observability: both cian_valuation and yandex_valuation graceful-degradation
except-blocks upgraded from logger.warning to logger.exception. GlitchTip's
LoggingIntegration listens at event_level=ERROR (main.py/scheduler_main.py) —
a WARNING never reaches GlitchTip as an event regardless of exc_info, so a
future config-wiring mistake at these call sites needs ERROR level to be
visible in monitoring.

house_imv_backfill.py: RealScraperConfig is imported lazily inside
_process_one_house (not at module level) to avoid a circular import —
app.services.scraper_adapters imports backfill_house_imv/
process_houses_imv_batch from this module at module level. Verified via
direct import in both orders plus a full `app.main` import.

Also fixes a stale docstring claiming process_houses_imv_batch is "not wired
into scheduler" — it is, via scrape_pipeline.py's run_avito_city_sweep.

Test updates: 2 pre-existing tests (test_backfill_wave2.py) mocked the legacy
IMVAddressNotFoundError/IMVEvaluation/IMVGeo classes, now updated to import
from scraper_kit to match the production exception identity. Added
config=/delay_provider= regression-guard asserts to the relevant estimator
and backfill tests, mirroring the existing #2306 cian_price_history pattern.

Legacy app/services/scrapers/{avito_imv,cian_valuation,yandex_valuation}.py
are untouched and still imported by cian_history_backfill.py's valuation
block (separately scoped, not touched here) — revert is a clean single-commit
revert, no schema/data migration involved.
2026-07-04 02:40:34 +03:00
5fa505b809 fix(tradein/scrapers): подключить geoportal coords backfill в scheduler (#1967) (#2346)
All checks were successful
Deploy Trade-In / build-frontend (push) Has been skipped
Deploy Trade-In / build-browser (push) Has been skipped
Deploy Trade-In / build-backend (push) Successful in 52s
Deploy Trade-In / changes (push) Successful in 9s
Deploy Trade-In / test (push) Successful in 1m30s
Deploy Trade-In / deploy (push) Successful in 1m48s
2026-07-03 23:25:47 +00:00
9289c3d69d fix(reports): HEAD-поддержка на /report/download — qa поймал 405, DOCX не скачивался из UI (#2343)
All checks were successful
Deploy / changes (push) Successful in 6s
Deploy / build-frontend (push) Has been skipped
Deploy / build-backend (push) Successful in 2m0s
Deploy / build-worker (push) Successful in 2m53s
Deploy / deploy (push) Successful in 1m35s
2026-07-03 23:13:23 +00:00
f872f5636b Merge pull request 'test(tradein/scraper-kit): prove yandex_valuation kit path safe for #2337 (#2336)' (#2345) from feat/tradein-migrate-yandex-valuation-kit into main
All checks were successful
Deploy Trade-In / changes (push) Successful in 10s
Deploy Trade-In / build-frontend (push) Has been skipped
Deploy Trade-In / build-browser (push) Has been skipped
Deploy Trade-In / test (push) Successful in 1m35s
Deploy Trade-In / build-backend (push) Successful in 28s
Deploy Trade-In / deploy (push) Successful in 46s
2026-07-03 23:04:13 +00:00
aa2b33f264 Merge pull request 'test(tradein/scraper-kit): golden-parity + config-footgun proof for avito_imv (#2334)' (#2344) from feat/tradein-migrate-avito-imv-kit into main
Some checks failed
Deploy Trade-In / test (push) Blocked by required conditions
Deploy Trade-In / build-backend (push) Blocked by required conditions
Deploy Trade-In / build-frontend (push) Blocked by required conditions
Deploy Trade-In / build-browser (push) Blocked by required conditions
Deploy Trade-In / deploy (push) Blocked by required conditions
Deploy Trade-In / changes (push) Has been cancelled
2026-07-03 23:04:11 +00:00
17cf7344b3 Merge pull request 'test(tradein/scraper-kit): prove cian_valuation kit path safe for #2337 (#2335)' (#2342) from feat/tradein-migrate-cian-valuation-kit into main
All checks were successful
Deploy Trade-In / changes (push) Successful in 18s
Deploy Trade-In / build-frontend (push) Has been skipped
Deploy Trade-In / build-browser (push) Has been skipped
Deploy Trade-In / test (push) Successful in 1m40s
Deploy Trade-In / build-backend (push) Successful in 29s
Deploy Trade-In / deploy (push) Successful in 47s
2026-07-03 22:59:44 +00:00
bot-backend
53287da886 test(tradein/scraper-kit): prove cian_valuation kit path safe for #2337 (#2335)
All checks were successful
CI Trade-In / changes (pull_request) Successful in 6s
CI / changes (pull_request) Successful in 7s
CI Trade-In / frontend-checks (pull_request) Has been skipped
CI / backend-tests (pull_request) Has been skipped
CI / frontend-tests (pull_request) Has been skipped
CI / openapi-codegen-check (pull_request) Has been skipped
CI Trade-In / backend-tests (pull_request) Successful in 1m35s
scraper_kit.providers.cian.valuation.estimate_via_cian_valuation (and
session.load_session) already existed with mandatory config: ScraperConfig
(no default) — this was ground prepared ahead of Group E2, not new work here.
This change proves it's safe to switch estimator.py's import to it (#2337):

- Extends the existing _parse_valuation_state golden-parity test with a
  full-pipeline parity test (cache-miss -> load_session -> curl_cffi fetch ->
  extract_state -> auth-check -> parse -> sanity-check), wired with
  config=RealScraperConfig() per the #2306 convention
  (app/services/cian_price_history.py).
- Adds a regression guard proving estimate_via_cian_valuation raises TypeError
  when config= is omitted, and documents why that must stay mandatory:
  estimator.py's call site (line ~3143) sits inside a blanket
  `except Exception` that only does `logger.warning(..., exc)` (no exc_info,
  no logger.exception) -- below GlitchTip's ERROR-level capture threshold
  (see LoggingIntegration(event_level=logging.ERROR) in app/main.py), so a
  TypeError from a missing config= would be logged but invisible in
  GlitchTip/Sentry, indistinguishable from ordinary graceful degradation.
- Documents an asymmetry vs. the task's initial assumption: kit's
  load_session requires config= mandatorily, but mark_session_invalid does
  NOT take a config parameter at all (confirmed by inspection, not a
  footgun -- just noting the actual contract).
- Confirms app/services/estimator.py is not the only real caller:
  app/tasks/cian_history_backfill.py also imports estimate_via_cian_valuation
  directly and deliberately stays on the legacy import for now (see its own
  comment referencing issue #2308) -- out of scope for this change, flagged
  for whoever picks up cian_history_backfill's migration.

Full backend test suite: 3211 passed, 6 skipped, 1 pre-existing unrelated
failure (test_search_api.py::test_search_cache_hit, 401 vs 200 auth issue,
reproduces identically on main without this change).
2026-07-04 01:51:30 +03:00
bot-backend
7546868535 test(tradein/scraper-kit): prove yandex_valuation kit-wiring safety (#2336)
All checks were successful
CI Trade-In / changes (pull_request) Successful in 17s
CI / changes (pull_request) Successful in 16s
CI Trade-In / frontend-checks (pull_request) Has been skipped
CI / backend-tests (pull_request) Has been skipped
CI / frontend-tests (pull_request) Has been skipped
CI / openapi-codegen-check (pull_request) Has been skipped
CI Trade-In / backend-tests (pull_request) Successful in 1m50s
Group E3 recon confirmed `scraper_kit.providers.yandex.valuation` is already a
strangler-copy (#2133) and already used via admin.py's debug route (#2305,
Group A) with the established config=RealScraperConfig()/delay_provider=
get_scraper_delay convention. estimator.py (the real caller, #2337's scope)
still imports the legacy module directly with no arguments -- not touched here.

Adds regression coverage for the two confirmed footguns before #2337 switches
that call site:
- mandatory `config: ScraperConfig` raises TypeError when omitted (same
  discipline as the cian_valuation footgun, #2335)
- optional `delay_provider` silently falls back to hardcoded 5.0s when not
  wired, discarding admin-tuned anti-ban throttling (scraper_settings.
  get_scraper_delay)

Also extends the existing `.parse()` golden-parity test (test_scraper_kit_
yandex_golden_parity.py, SimpleNamespace config) with an assert_parity-based
proof over the REAL production construction path (RealScraperConfig() +
get_scraper_delay), plus a harness-catches-divergence sanity check.

No production code changed -- app/services/scrapers/yandex_valuation.py and
scraper_kit/providers/yandex/valuation.py are both pre-existing.
2026-07-04 01:50:55 +03:00
bot-backend
7049717782 test(tradein/scraper-kit): golden-parity + config-footgun proof for avito_imv (#2334)
All checks were successful
CI Trade-In / changes (pull_request) Successful in 11s
CI / frontend-tests (pull_request) Has been skipped
CI / openapi-codegen-check (pull_request) Has been skipped
CI / changes (pull_request) Successful in 11s
CI Trade-In / frontend-checks (pull_request) Has been skipped
CI / backend-tests (pull_request) Has been skipped
CI Trade-In / backend-tests (pull_request) Successful in 1m56s
Group E1 of the scraper_kit migration epic (#2277 -> #2308 -> #2334): prepares the
kit side of app/services/scrapers/avito_imv.py -> scraper_kit.providers.avito.imv
WITHOUT switching any real call site (estimator.py is #2337, a separate final step;
house_imv_backfill.py is deliberately deferred too, see report).

- Golden-parity: full evaluate_via_imv() async flow (geocode A/B + evaluate C) proven
  byte-identical between legacy and kit on REAL live-captured Avito API responses
  (tests/fixtures/avito_imv_geo_position.json + avito_imv_getdata.json), routed through
  the browser_fetcher transport to stay offline/deterministic. Previously this function
  had zero parity coverage (only _parse_price was covered, via admin.py Group A).
  Added cheap pure-function parity for compute_imv_cache_key/_parse_geo_position/
  _parse_placement_history/_parse_suggestions too.
- Config-footgun regression: kit evaluate_via_imv only reads config.scraper_proxy_url
  when config= is explicitly passed; called bare it silently builds a proxy-less
  curl_cffi session even with a real proxy configured. Two tests lock this down
  (without config= -> proxies=None, with config=RealScraperConfig() -> proxies wired) —
  manually verified by breaking scraper_kit/providers/avito/imv.py's config threading,
  watching the "with config" test fail on the exact proxies assertion, then reverting.

No production import switched in this commit — see PR/issue report for the full
caller audit and the house_imv_backfill.py deferral rationale.
2026-07-04 01:49:21 +03:00
1d9e25ce07 fix(tradein/v2): подключить реальные данные location-coef в LocationDrawer + HeroBar (#2317) (#2341)
All checks were successful
Deploy Trade-In / changes (push) Successful in 10s
Deploy Trade-In / build-browser (push) Has been skipped
Deploy Trade-In / test (push) Successful in 1m37s
Deploy Trade-In / build-frontend (push) Successful in 2m12s
Deploy Trade-In / build-backend (push) Successful in 53s
Deploy Trade-In / deploy (push) Successful in 1m9s
2026-07-03 22:47:40 +00:00
e7f97f1355 fix(tradein/avito): консолидировать house_type normalizer + актуализировать docstring (#2009) (#2340)
Some checks failed
Deploy Trade-In / build-backend (push) Blocked by required conditions
Deploy Trade-In / deploy (push) Blocked by required conditions
Deploy Trade-In / changes (push) Successful in 11s
Deploy Trade-In / build-frontend (push) Has been skipped
Deploy Trade-In / build-browser (push) Has been skipped
Deploy Trade-In / test (push) Has been cancelled
2026-07-03 22:47:05 +00:00
e3d26f0aae fix(tradein/scrapers): cian_city_sweep честный статус при defensive abort (#1949) (#2339)
Some checks failed
Deploy Trade-In / changes (push) Successful in 11s
Deploy Trade-In / build-frontend (push) Has been skipped
Deploy Trade-In / build-backend (push) Blocked by required conditions
Deploy Trade-In / deploy (push) Blocked by required conditions
Deploy Trade-In / build-browser (push) Has been skipped
Deploy Trade-In / test (push) Has been cancelled
2026-07-03 22:45:47 +00:00
49304a78bc feat(ptica/reports): DOCX-скачивание полного отчёта в UI (#2259 R2) (#2338)
All checks were successful
Deploy / changes (push) Successful in 8s
Deploy / build-frontend (push) Successful in 3m12s
Deploy / build-backend (push) Has been skipped
Deploy / build-worker (push) Has been skipped
Deploy / deploy (push) Successful in 1m19s
2026-07-03 22:40:11 +00:00
f41ca530c5 Merge pull request 'feat(tradein/scraper-kit): migrate backfill/enrichment tasks to kit (#2310)' (#2333) from feat/tradein-migrate-backfill-tasks-kit into main
All checks were successful
Deploy Trade-In / changes (push) Successful in 11s
Deploy Trade-In / build-frontend (push) Has been skipped
Deploy Trade-In / build-browser (push) Has been skipped
Deploy Trade-In / test (push) Successful in 1m32s
Deploy Trade-In / build-backend (push) Successful in 55s
Deploy Trade-In / deploy (push) Successful in 1m7s
2026-07-03 22:28:10 +00:00
30e033d653 perf(sql): покрывающий индекс под offer-trend §4.3 — 6.8с → 1.1с (миграция 186) (#2331)
All checks were successful
Deploy / changes (push) Successful in 7s
Deploy / build-frontend (push) Has been skipped
Deploy / build-backend (push) Successful in 2m2s
Deploy / build-worker (push) Successful in 3m7s
Deploy / deploy (push) Successful in 1m42s
2026-07-03 22:26:29 +00:00
9feadc8458 fix(market-metrics): offer-trend таймаут — warning вместо exception (ложные GlitchTip-события) (#2329)
Some checks failed
Deploy / build-backend (push) Has been cancelled
Deploy / deploy (push) Blocked by required conditions
Deploy / changes (push) Successful in 7s
Deploy / build-frontend (push) Has been skipped
Deploy / build-worker (push) Has been cancelled
2026-07-03 22:26:19 +00:00
bot-backend
116cd10046 test(tradein/scraper-kit): assert config= at avito_detail_backfill call sites (#2310)
All checks were successful
CI / changes (pull_request) Successful in 9s
CI Trade-In / changes (pull_request) Successful in 10s
CI Trade-In / frontend-checks (pull_request) Has been skipped
CI / backend-tests (pull_request) Has been skipped
CI / frontend-tests (pull_request) Has been skipped
CI / openapi-codegen-check (pull_request) Has been skipped
CI Trade-In / backend-tests (pull_request) Successful in 1m39s
Code-review follow-up: the Group C regression tests for the fetch_detail
backconnect footgun and AvitoScraper's config requirement proved the KIT
LIBRARY behavior in isolation, but nothing asserted the actual call sites in
avito_detail_backfill.py still pass config=RealScraperConfig() at fetch_detail
and AvitoScraper construction. Mirrors #2306's test_backfill_wave2.py:282-286
pattern -- isinstance-checks the captured call_args instead of a bare
assert_called().

Verified the new assertions actually catch the regression: temporarily
removed both config=RealScraperConfig() call-site args from
avito_detail_backfill.py, confirmed test_backfill_processes_snapshot_to_completion
fails exactly on the new assertion line, then restored (git diff was empty
afterward).

Refs #2310
2026-07-04 01:22:50 +03:00
bot-backend
eb564cfc01 feat(tradein/scraper-kit): migrate backfill-task imports to kit, Group C (#2310)
Migrates legacy app.services.scrapers.* imports to scraper_kit equivalents for
house_imv_backfill.py, avito_detail_backfill.py, cian_history_backfill.py,
ekb_geoportal_ingest.py, and yandex_detail_backfill.py, proving parity via
tests/support/parity.assert_parity per the epic's gate (#2304).

newbuilding_enrich_backfill.py and yandex_newbuilding_sweep.py are left fully
on legacy imports: both call into scraper_kit.providers.{cian,yandex}.newbuilding,
which construct BrowserFetcher(source=...) without the now-mandatory endpoint=
kwarg (issue #2322, verified still open against the actual provider source, not
just issue status) -- no caller-side fix is possible, matching Group A's (#2305)
precedent for the same bug in admin.py.

Config-gated kit function footguns found and fixed (Group B #2306 pattern):
  - avito fetch_detail's backconnect-on-403 retry silently drops when config=
    is omitted -- now passes config=RealScraperConfig() explicitly, with a
    regression test proving the gate.
  - kit AvitoScraper's constructor now requires ScraperConfig positionally --
    wired via RealScraperConfig(), which _rotate_ip() reads for
    avito_proxy_rotate_url.
  - BrowserFetcher(source=...) call sites (house_imv_backfill, avito/cian
    detail backfills) now pass the mandatory endpoint=settings.browser_http_endpoint.

New footgun discovered (NOT #2322, flagged for follow-up): kit's
build_warmed_session() builds its curl_cffi session via _build_detail_session()
with no config parameter at all, unlike fetch_detail -- migrating it would
silently drop the sticky MGTS-proxy egress on avito_detail_backfill's
warm-batch path (the prod default). Left build_warmed_session/
_AVITO_WARM_SEARCH_URL on legacy imports, documented inline.

Refs #2310
2026-07-04 01:19:34 +03:00
793234cff1 fix(tradein/v2): гейтить PDF-кнопку на hasEstimate (#2081 M4) (#2332)
All checks were successful
Deploy Trade-In / changes (push) Successful in 10s
Deploy Trade-In / test (push) Has been skipped
Deploy Trade-In / build-browser (push) Has been skipped
Deploy Trade-In / build-backend (push) Has been skipped
Deploy Trade-In / build-frontend (push) Successful in 1m56s
Deploy Trade-In / deploy (push) Successful in 42s
2026-07-03 22:16:00 +00:00
308eeac214 fix(tradein/scrapers): wire proxy_pool into Avito legacy pipeline, rotate on ban (#2202) (#2328)
All checks were successful
Deploy Trade-In / changes (push) Successful in 15s
Deploy Trade-In / build-frontend (push) Has been skipped
Deploy Trade-In / build-browser (push) Has been skipped
Deploy Trade-In / test (push) Successful in 1m34s
Deploy Trade-In / build-backend (push) Successful in 1m5s
Deploy Trade-In / deploy (push) Successful in 59s
2026-07-03 21:48:15 +00:00
01f2c5290d fix(freshness): пороги nspd_geo под on-demand каденцию (убирает еженедельный ложный stale) (#2325)
All checks were successful
Deploy / changes (push) Successful in 8s
Deploy / build-frontend (push) Has been skipped
Deploy / build-backend (push) Successful in 1m50s
Deploy / build-worker (push) Successful in 3m3s
Deploy / deploy (push) Successful in 1m43s
2026-07-03 21:46:30 +00:00
7fd80adfa7 fix(tradein/pdf): sold-price consistency + актуальные допущения + свой дизайн отчёта (#2327)
All checks were successful
Deploy Trade-In / changes (push) Successful in 12s
Deploy Trade-In / build-frontend (push) Has been skipped
Deploy Trade-In / build-browser (push) Has been skipped
Deploy Trade-In / test (push) Successful in 1m38s
Deploy Trade-In / build-backend (push) Successful in 1m4s
Deploy Trade-In / deploy (push) Successful in 56s
2026-07-03 21:43:40 +00:00
86af5b78bf Merge pull request 'feat(tradein/scraper-kit): migrate admin debug endpoints + domclick ingest to kit (#2305)' (#2326) from feat/tradein-migrate-admin-debug-kit into main
Some checks failed
Deploy Trade-In / deploy (push) Blocked by required conditions
Deploy Trade-In / changes (push) Successful in 11s
Deploy Trade-In / build-frontend (push) Has been skipped
Deploy Trade-In / build-browser (push) Has been skipped
Deploy Trade-In / test (push) Successful in 1m43s
Deploy Trade-In / build-backend (push) Has been cancelled
2026-07-03 21:41:11 +00:00
7a113273f4 fix(generative): честные типы в dxf-экспортёре — strict mypy проходит (#2316) (#2319)
All checks were successful
Deploy / changes (push) Successful in 9s
Deploy / build-frontend (push) Has been skipped
Deploy / build-backend (push) Successful in 2m37s
Deploy / build-worker (push) Successful in 3m23s
Deploy / deploy (push) Successful in 1m43s
2026-07-03 21:40:48 +00:00
24b773efe3 Merge pull request 'feat(tradein/scraper-kit): migrate cian/yandex price-history & session to kit (#2306)' (#2324) from feat/tradein-migrate-price-history-kit into main
All checks were successful
Deploy Trade-In / changes (push) Successful in 8s
Deploy Trade-In / build-frontend (push) Has been skipped
Deploy Trade-In / build-browser (push) Has been skipped
Deploy Trade-In / test (push) Successful in 1m32s
Deploy Trade-In / build-backend (push) Successful in 54s
Deploy Trade-In / deploy (push) Successful in 47s
2026-07-03 21:33:10 +00:00
279e7de778 fix(tradein/v2): пометка «В ОФФЕР» + сузить реестровую карточку (#1991) (#2321)
All checks were successful
Deploy Trade-In / changes (push) Successful in 9s
Deploy Trade-In / test (push) Has been skipped
Deploy Trade-In / build-browser (push) Has been skipped
Deploy Trade-In / build-backend (push) Has been skipped
Deploy Trade-In / build-frontend (push) Successful in 2m6s
Deploy Trade-In / deploy (push) Successful in 52s
2026-07-03 21:28:50 +00:00
bot-backend
88b93435fc test(tradein): assert config kwarg on cian price-history fetch_detail call (#2306)
All checks were successful
CI Trade-In / changes (pull_request) Successful in 7s
CI / changes (pull_request) Successful in 8s
CI Trade-In / frontend-checks (pull_request) Has been skipped
CI / backend-tests (pull_request) Has been skipped
CI / frontend-tests (pull_request) Has been skipped
CI / openapi-codegen-check (pull_request) Has been skipped
CI Trade-In / backend-tests (pull_request) Successful in 1m40s
code-reviewer flagged that neither the new parity test nor
test_backfill_wave2.py's 3 cian_price_history tests would catch a
regression if config=RealScraperConfig() were accidentally dropped from
the fetch_detail call in cian_price_history.py (the proxy-footgun fix
from the #2306 migration) — mock_fetch.assert_called_once() doesn't
check kwargs.

Strengthens all 3 tests (saves_changes / skips_no_changes /
handles_fetch_none) to assert isinstance(call_kwargs["config"],
RealScraperConfig). Manually verified the guard actually catches the
regression: temporarily removed config= from cian_price_history.py:110,
confirmed all 3 tests fail, restored the fix, confirmed they pass again.
2026-07-04 00:27:08 +03:00
bot-backend
ea24289f47 feat(tradein/scraper-kit): migrate admin.py debug routes + domclick ingest to kit (#2305)
All checks were successful
CI Trade-In / changes (pull_request) Successful in 8s
CI / changes (pull_request) Successful in 8s
CI Trade-In / frontend-checks (pull_request) Has been skipped
CI / backend-tests (pull_request) Has been skipped
CI / frontend-tests (pull_request) Has been skipped
CI / openapi-codegen-check (pull_request) Has been skipped
CI Trade-In / backend-tests (pull_request) Successful in 1m36s
Group A of the scraper_kit migration epic (#2277). Switches admin.py's manual
"run parser" debug endpoints and scripts/ingest_domclick_jsonl.py from direct
app.services.scrapers.* imports to their scraper_kit.providers.* equivalents,
using the DI adapters (RealScraperConfig/RealMatcherAdapter/RealProxyProvider,
app.services.scraper_settings.get_scraper_delay) already established by
app.scheduler_main._run_kit_scheduler.

Migrated: /scrape (AvitoScraper/CianScraper/YandexRealtyScraper + save_listings),
scrape_avito_house, scrape_avito_detail, scrape_avito_imv, scrape_yandex_detail,
scrape_yandex_valuation, scrape_cian_detail, cian_auto_login's BrowserFetcher.
scripts/ingest_domclick_jsonl.py: ScrapedLot/save_listings + (now that #2307/
Group D ported a kit equivalent while this was in flight) DomClickDetailEnrichment/
save_detail_enrichment.

Deliberately NOT migrated (documented in admin.py): scrape_yandex_newbuilding /
scrape_cian_newbuilding — their kit equivalents
(providers/{yandex,cian}/newbuilding.py) construct an internal BrowserFetcher(...)
without the mandatory `endpoint` kwarg, so any call crashes/silently-fails
regardless of caller-side DI. Bug lives in scraper_kit provider code, out of
scope here (only consuming, not touching provider logic) — flagged as follow-up.

Parity proven via tests/support/parity.py (assert_parity) against the exact
functions each debug route now calls, on offline fixtures — no live network/DB.

Refs #2305
2026-07-04 00:27:01 +03:00
bot-backend
db3afa3176 fix(tradein/scrapers): migrate cian_price_history/cian_session/yandex_price_history imports to scraper_kit (#2306)
Group B of the scraper_kit migration epic (#2277): switch the three files'
internal dependency imports from legacy app/services/scrapers/* to their
byte/structurally-identical scraper_kit equivalents, proven via the new
parity harness (tests/support/parity.assert_parity, #2304). Public API of
all three files is unchanged, so no callers needed updating.

- cian_price_history.py: fetch_detail/save_detail_enrichment now from
  scraper_kit.providers.cian.detail. Wires config=RealScraperConfig() at
  the call site — kit's fetch_detail only reads cian_proxy_url when an
  explicit config is passed, unlike legacy's always-on settings read;
  without this the admin-triggered backfill would silently drop its
  datacenter-IP proxy (#806) after the import swap.
- cian_session.py: extract_state now from scraper_kit.cian_state_parser
  (byte-identical to legacy).
- yandex_price_history.py: ScrapedLot now from scraper_kit.base
  (byte-identical fields/methods; record_yandex_price_history only reads
  lot.* via duck-typing, no isinstance checks).

New tests/test_scraper_kit_pricehistory_session_parity.py proves the
migration delta specifically (extract_state, ScrapedLot.model_dump(),
fetch_detail+DetailEnrichment with config injection, and the own-session
proxy-kwarg wiring). Full backend suite passes (3145 passed, 6 skipped,
1 pre-existing unrelated failure confirmed on unmodified forgejo/main).
2026-07-04 00:20:13 +03:00
a7677b0fba Merge pull request 'feat(tradein/scraper-kit): port domclick_detail + ekb_geoportal_client to kit (#2307)' (#2318) from feat/tradein-port-domclick-ekb-kit into main
All checks were successful
Deploy Trade-In / changes (push) Successful in 10s
Deploy Trade-In / build-frontend (push) Has been skipped
Deploy Trade-In / build-browser (push) Has been skipped
Deploy Trade-In / test (push) Successful in 1m32s
Deploy Trade-In / build-backend (push) Successful in 1m28s
Deploy Trade-In / deploy (push) Successful in 57s
2026-07-03 21:18:39 +00:00
27e21f37d3 fix(generative): MultiPolygon-участки — крупнейший контур в _parse_polygon (#2295) (#2315)
All checks were successful
Deploy / changes (push) Successful in 7s
Deploy / build-frontend (push) Has been skipped
Deploy / build-backend (push) Successful in 1m58s
Deploy / build-worker (push) Successful in 2m58s
Deploy / deploy (push) Successful in 1m32s
2026-07-03 21:13:00 +00:00
bot-backend
8c8289cdad feat(tradein/scraper-kit): port domclick_detail + ekb_geoportal_client to kit (#2307)
All checks were successful
CI / changes (pull_request) Successful in 10s
CI Trade-In / changes (pull_request) Successful in 10s
CI Trade-In / frontend-checks (pull_request) Has been skipped
CI / backend-tests (pull_request) Has been skipped
CI / frontend-tests (pull_request) Has been skipped
CI / openapi-codegen-check (pull_request) Has been skipped
CI Trade-In / backend-tests (pull_request) Successful in 1m44s
Group D of the scraper_kit migration epic (#2277). Both legacy modules had no
kit-side equivalent yet (per audit Scraper_Kit_Legacy_Dependency_Audit_0703),
unlike the other 24/26 files which were already strangler-duplicated.

- scraper_kit/providers/domclick/detail.py — Layer B detail-card enrichment
  (parse_detail_html/fetch_detail/save_detail_enrichment), no ScraperConfig
  injection needed (fetch_detail takes an already-open browser_fetcher).
- scraper_kit/providers/ekb_geoportal/{__init__,client}.py — new subpackage,
  first non-listing-site provider in scraper_kit. Legacy module had zero
  app.* coupling (only httpx), so the port is near-verbatim.

Parity proven via tests/support/parity.py (assert_parity) against the exact
legacy functions on realistic fixtures (live-card SSR JSON, GeoJSON
FeatureCollection).

Callers of both legacy modules are unchanged (out of scope — ekb_geoportal_ingest
switch is #2310; the only other domclick_detail caller is
scripts/ingest_domclick_jsonl.py, also untouched).
2026-07-04 00:05:07 +03:00
bc9549782b Merge pull request 'feat(tradein): location-coef MVP через FDW-мост к OSM POI Птицы (#2045 PR-B)' (#2314) from feat/tradein-location-coef-poi into main
All checks were successful
Deploy Trade-In / changes (push) Successful in 9s
Deploy Trade-In / build-frontend (push) Has been skipped
Deploy Trade-In / test (push) Successful in 1m30s
Deploy Trade-In / build-browser (push) Has been skipped
Deploy Trade-In / build-backend (push) Successful in 1m4s
Deploy Trade-In / deploy (push) Successful in 57s
2026-07-03 20:55:31 +00:00
5db6a2bec3 Merge pull request 'test(tradein): reusable legacy/scraper_kit parity harness (#2304)' (#2313) from feat/tradein-parity-test-harness into main
All checks were successful
Deploy Trade-In / build-frontend (push) Has been skipped
Deploy Trade-In / build-browser (push) Has been skipped
Deploy Trade-In / changes (push) Successful in 10s
Deploy Trade-In / test (push) Successful in 1m32s
Deploy Trade-In / build-backend (push) Successful in 27s
Deploy Trade-In / deploy (push) Successful in 46s
2026-07-03 20:52:25 +00:00
bot-backend
d95c9e6e55 fix(tradein/tests): harden parity harness — bool-type mismatch detection + divergence-catch proof (#2304)
All checks were successful
CI Trade-In / frontend-checks (pull_request) Has been skipped
CI / backend-tests (pull_request) Has been skipped
CI Trade-In / backend-tests (pull_request) Successful in 1m34s
CI Trade-In / changes (pull_request) Successful in 7s
CI / changes (pull_request) Successful in 8s
CI / frontend-tests (pull_request) Has been skipped
CI / openapi-codegen-check (pull_request) Has been skipped
Code-reviewer follow-up on the parity harness (aaaf8179):

- parity.py: plain `==` in the scalar-equality branch let `True == 1` /
  `False == 0` silently pass (Python bool is an int subclass). A future
  migration bug turning a `bool | None` field into a raw 0/1 would slip
  through undetected. Now any type mismatch where exactly one side is a
  bool is reported as a diff, regardless of numeric equality.
- test_parity.py: unit test for the new bool-vs-int branch.
- test_avito_detail_kit_parity.py: the existing smoke test only proved the
  harness reports "no diff" on two genuinely-identical real dataclass
  instances — it never proved the harness catches a real divergence on
  this same 30+-field shape (only the toy fixtures in test_parity.py did).
  Added a test that mutates `price_rub` via dataclasses.replace() on the
  real kit DetailEnrichment and asserts assert_parity raises
  ParityMismatchError naming that field.
2026-07-03 23:45:58 +03:00
bot-backend
57963e3a06 test(tradein): reusable legacy/scraper_kit parity harness (#2304)
Stage 0 of the scraper_kit migration epic (#2277): shared test tool for
issues #2305-#2310, which each need to prove their kit-path importer
produces the same output as the legacy path on the same input.

- tests/support/parity.py: assert_parity()/compare_outputs() normalize
  dataclass/pydantic outputs to dict/list/scalar before comparing, since
  legacy vs kit dataclasses (e.g. DetailEnrichment) are different classes
  and dataclass __eq__ always returns False across classes even when all
  field values match. Supports ignore_fields (drop non-deterministic
  fields like latency_ms/fetched_at) and numeric tolerance (math.isclose)
  for float fields, with an assertion listing every differing field
  (path + legacy value + kit value) on mismatch.
- tests/support/test_parity.py: unit tests for the harness itself
  (identical outputs pass, differing outputs raise with informative
  diff, tolerance/ignore_fields options, cross-class dataclass parity).
- tests/scrapers/test_avito_detail_kit_parity.py: end-to-end smoke proof
  against real code — app.services.scrapers.avito_detail.parse_detail_html
  (legacy, reached via admin.py's scrape_avito_detail debug endpoint
  through fetch_detail) vs scraper_kit.providers.avito.detail's copy, on
  a fixed HTML fixture.
- tests/support/README.md: usage note for #2305-#2310 migration PRs.

Found while implementing: tests/test_scraper_kit_*_parity.py (9 files,
~3400 lines) already do ad-hoc `dataclasses.asdict(old) == asdict(new)`
parity checks for the already-migrated SERP scraper modules (avito/cian/
domclick/yandex/base/scheduler/pipeline) — this harness generalizes that
repeated pattern for the remaining 12 non-scraper importers, adding
ignore_fields/tolerance which those ad-hoc checks don't have.
2026-07-03 23:45:58 +03:00
bot-backend
90731da537 feat(tradein): location-coef MVP через FDW-мост к OSM POI Птицы (#2045)
All checks were successful
CI Trade-In / changes (pull_request) Successful in 7s
CI / changes (pull_request) Successful in 8s
CI Trade-In / frontend-checks (pull_request) Has been skipped
CI / backend-tests (pull_request) Has been skipped
CI / frontend-tests (pull_request) Has been skipped
CI / openapi-codegen-check (pull_request) Has been skipped
CI Trade-In / backend-tests (pull_request) Successful in 1m36s
Финальный PR issue #2045 (BE-3): GET /api/v1/trade-in/location-coef для
LocationDrawer. FDW foreign table -> локальное зеркало osm_poi_ekb_local
(TRUNCATE+INSERT, тот же паттерн что cad_buildings_local/cadastral_geo_match,
избегает ~1.16s/row FDW round-trip) -> straight-line POI-скоринг, портированный
из Site Finder poi_score.py::compute_poi_weighted_top7 (CATEGORY_WEIGHTS as-is,
радиус 1200м для квартир вместо Ptica 2000м для участков). score->coef -
новая MVP-эвристика (0.95..1.05, не откалибрована на реальных дельтах).

Graceful fallback (не 500, не фабрикуем факторы): пустая/не отрефрешенная
osm_poi_ekb_local или отсутствие lat/lon у оценки -> coef=1.0, factors=[],
geo_source="unavailable".

Scheduler: source=osm_poi_ekb_refresh, daily, зарегистрирован и в боевом
dispatch (scheduler.py), и в kit-registry (product_handlers.py) - иначе
test_kit_registry_completeness падает на ship-dark инварианте (#2192).

Frontend wiring (mappers.ts/LocationDrawer.tsx) - вне scope, отдельная задача
после проверки endpoint'а curl'ом на деплое.
2026-07-03 23:44:55 +03:00
a8b504c3de fix(tradein/v2): визуальный аудит — иерархия/честность/микрокопи (#2266) (#2312)
All checks were successful
Deploy Trade-In / changes (push) Successful in 10s
Deploy Trade-In / build-browser (push) Has been skipped
Deploy Trade-In / test (push) Successful in 1m35s
Deploy Trade-In / build-frontend (push) Successful in 2m8s
Deploy Trade-In / build-backend (push) Successful in 56s
Deploy Trade-In / deploy (push) Successful in 47s
2026-07-03 20:35:38 +00:00
4fedfc5970 fix(tradein/estimator): дедуп кросс-source дублей в anchor-пути (#2265 follow-up) (#2311)
Some checks failed
Deploy Trade-In / test (push) Blocked by required conditions
Deploy Trade-In / build-backend (push) Blocked by required conditions
Deploy Trade-In / build-frontend (push) Blocked by required conditions
Deploy Trade-In / build-browser (push) Blocked by required conditions
Deploy Trade-In / deploy (push) Blocked by required conditions
Deploy Trade-In / changes (push) Has been cancelled
2026-07-03 20:35:37 +00:00
0e2b4998c8 Merge pull request 'fix(tradein/pii): удалить телефоны продавцов (NULL) + PII клиента (DROP) (#2217, #1969)' (#2303) from fix/tradein-pii-purge-phones-client into main
All checks were successful
Deploy Trade-In / changes (push) Successful in 10s
Deploy Trade-In / build-browser (push) Has been skipped
Deploy Trade-In / test (push) Successful in 2m16s
Deploy Trade-In / build-frontend (push) Successful in 3m28s
Deploy Trade-In / build-backend (push) Successful in 1m51s
Deploy Trade-In / deploy (push) Successful in 1m34s
2026-07-03 20:21:46 +00:00
c129515ab8 feat(exporters): DOCX-вариант полного отчёта ПТИЦА (#2259 R2) (#2301)
All checks were successful
Deploy / changes (push) Successful in 7s
Deploy / build-backend (push) Successful in 3m34s
Deploy / build-worker (push) Successful in 4m25s
Deploy / build-frontend (push) Successful in 5m25s
Deploy / deploy (push) Successful in 1m33s
2026-07-03 20:21:19 +00:00
3945060f0c Merge pull request 'feat(tradein/v2): реальный display-профиль в TopNav через /me (#2046)' (#2300) from feat/tradein-me-profile-fields into main
All checks were successful
Deploy Trade-In / changes (push) Successful in 10s
Deploy Trade-In / build-browser (push) Has been skipped
Deploy Trade-In / test (push) Successful in 1m38s
Deploy Trade-In / build-frontend (push) Successful in 2m21s
Deploy Trade-In / build-backend (push) Successful in 1m6s
Deploy Trade-In / deploy (push) Successful in 50s
2026-07-03 20:17:14 +00:00
64d3a925ef Merge pull request 'feat(site-finder): view v_tradein_osm_poi_ekb для FDW-моста trade-in location-coef (#2045 PR-A)' (#2299) from feat/gendesign-osm-poi-tradein-view into main
All checks were successful
Deploy / changes (push) Successful in 8s
Deploy / build-frontend (push) Has been skipped
Deploy / build-backend (push) Successful in 37s
Deploy / build-worker (push) Successful in 40s
Deploy / deploy (push) Successful in 1m26s
2026-07-03 20:14:23 +00:00
71b125cc13 Merge pull request 'chore(claude): явный CI-gate перед merge (false-green trap) + анти-rubber-stamp формат verdict'а' (#2302) from chore/claude-review-ci-gate into main
Reviewed-on: #2302
2026-07-03 20:09:54 +00:00
bot-backend
a7253639cd chore(claude): явный CI-gate перед merge (false-green trap) + анти-rubber-stamp формат verdict'а
All checks were successful
CI / changes (pull_request) Successful in 13s
CI Trade-In / changes (pull_request) Successful in 15s
CI Trade-In / backend-tests (pull_request) Has been skipped
CI Trade-In / frontend-checks (pull_request) Has been skipped
CI / backend-tests (pull_request) Has been skipped
CI / frontend-tests (pull_request) Has been skipped
CI / openapi-codegen-check (pull_request) Has been skipped
2026-07-03 23:06:17 +03:00
698c1be4f3 Merge pull request 'fix(tradein/v2): реальный cv + единый source-registry в overlay-мапперах (#2040, #2041)' (#2298) from feat/tradein-v2-overlays-wiring into main
All checks were successful
Deploy Trade-In / changes (push) Successful in 17s
Deploy Trade-In / test (push) Has been skipped
Deploy Trade-In / build-browser (push) Has been skipped
Deploy Trade-In / build-backend (push) Has been skipped
Deploy Trade-In / build-frontend (push) Successful in 2m33s
Deploy Trade-In / deploy (push) Successful in 48s
2026-07-03 20:00:55 +00:00
bot-backend
6f170504ba fix(tradein/pii): удалить телефоны продавцов (NULL) + PII клиента (DROP) (#2217, #1969)
All checks were successful
CI / changes (pull_request) Successful in 9s
CI Trade-In / changes (pull_request) Successful in 10s
CI / frontend-tests (pull_request) Has been skipped
CI / backend-tests (pull_request) Has been skipped
CI / openapi-codegen-check (pull_request) Has been skipped
CI Trade-In / frontend-checks (pull_request) Successful in 1m21s
CI Trade-In / backend-tests (pull_request) Successful in 1m46s
- listings.phones: миграция 166 обнуляет данные; cian.py + scraper-kit
  serp.py (golden-parity зеркало) перестают читать offer.get("phones") на
  входе — write-path выключен у источника, колонка/поле в модели остаются.
  Мёртвая запись "phones" в LISTING_FIELD_PRIORITY убрана.
- trade_in_estimates.client_name/client_phone: миграция 167 дропает колонки;
  синхронно убраны из TradeInEstimateInput, обоих INSERT в estimator.py
  (основной путь + _empty_estimate), frontend TradeInEstimateInput и
  CRM-блока EstimateForm (поля "Имя клиента"/"Телефон").
- sentry_scrub.py и management_companies.phones не тронуты — вне scope.
2026-07-03 23:00:42 +03:00
b902edd44c fix(full-report): generic-маркеры областных админ-единиц в ЕКБ-фильтре тепла (#2297)
All checks were successful
Deploy / build-worker (push) Successful in 3m41s
Deploy / changes (push) Successful in 8s
Deploy / build-frontend (push) Has been skipped
Deploy / build-backend (push) Successful in 2m25s
Deploy / deploy (push) Successful in 1m40s
2026-07-03 19:59:58 +00:00
bot-backend
dafd12d9fa feat(tradein/v2): реальный display-профиль в TopNav через /me (#2046)
All checks were successful
CI Trade-In / changes (pull_request) Successful in 9s
CI / changes (pull_request) Successful in 10s
CI / backend-tests (pull_request) Has been skipped
CI / frontend-tests (pull_request) Has been skipped
CI / openapi-codegen-check (pull_request) Has been skipped
CI Trade-In / frontend-checks (pull_request) Successful in 1m37s
CI Trade-In / backend-tests (pull_request) Successful in 2m0s
/me теперь отдаёт display_name/org/email (kopylov -> "Копылов", остальные None).
TopNav использует их вместо фабрикации username-as-name; org/email фолбэк
остаётся прежним для юзеров без известного профиля.
2026-07-03 22:55:12 +03:00
bot-backend
404d88ee34 feat(site-finder): view v_tradein_osm_poi_ekb для FDW-моста trade-in location-coef (#2045)
All checks were successful
CI Trade-In / changes (pull_request) Successful in 8s
CI / changes (pull_request) Successful in 9s
CI Trade-In / backend-tests (pull_request) Has been skipped
CI Trade-In / frontend-checks (pull_request) Has been skipped
CI / frontend-tests (pull_request) Has been skipped
CI / openapi-codegen-check (pull_request) Successful in 2m5s
CI / backend-tests (pull_request) Successful in 15m2s
2026-07-03 22:54:03 +03:00
bot-backend
c672e0322d feat(tradein/v2): wire AnalyticsView + SourcesView overlays to real API data (#2040, #2041)
All checks were successful
CI Trade-In / changes (pull_request) Successful in 8s
CI / changes (pull_request) Successful in 8s
CI Trade-In / backend-tests (pull_request) Has been skipped
CI / backend-tests (pull_request) Has been skipped
CI / frontend-tests (pull_request) Has been skipped
CI / openapi-codegen-check (pull_request) Has been skipped
CI Trade-In / frontend-checks (pull_request) Successful in 1m19s
Audit of /trade-in/v2 found the FE-3/FE-4 overlay wiring (mapAnalytics/mapSources
feeding AnalyticsView/SourcesView from useEstimateHouseAnalytics,
useEstimateSellTimeSensitivity, analogs[]/actual_deals[]) already shipped in
267585e3. The remaining gap was BE-1 (#2043, merged 2026-07-02) landing real
cv/source_counts/created_at on AggregatedEstimate AFTER that wiring pass, so
v2/mappers.ts was still FE-approximating cv and duplicating a hardcoded source
label map instead of the single source-registry (#2211).

- mapSources' marketAds.kpi.cv (#2041) now prefers the real estimate.cv
  (0..1 ratio from BE-1) and only falls back to the FE coefficient-of-variation
  of analog prices for pre-#2043 cached estimates where cv is absent.
- adRows/dealRows source badges (#2041) now resolve via the shared
  src/lib/source-registry sourceLabel() instead of a local SOURCE_LABEL map,
  picking up etagi/canonical labels and staying in sync with the #2211 roster.
- AnalyticsView (#2040) KPIs/sell-time/price-history/scatterDetail were already
  wired to real data with no gaps found against the issue spec.
2026-07-03 22:49:30 +03:00
e9e3cc7dcf fix(full-report): §7 на MultiPolygon + чистка печатного шума §3/§4 (вычитка живого отчёта) (#2296)
All checks were successful
Deploy / changes (push) Successful in 7s
Deploy / build-frontend (push) Has been skipped
Deploy / build-backend (push) Successful in 1m51s
Deploy / build-worker (push) Successful in 3m7s
Deploy / deploy (push) Successful in 1m41s
2026-07-03 19:33:11 +00:00
d15ff3d99d feat(tradein/estimator): сегментная поправка headline+выкуп по ценовому бэнду за флагом (#2255) (#2293)
All checks were successful
Deploy Trade-In / changes (push) Successful in 9s
Deploy Trade-In / build-frontend (push) Has been skipped
Deploy Trade-In / build-browser (push) Has been skipped
Deploy Trade-In / test (push) Successful in 1m29s
Deploy Trade-In / deploy (push) Successful in 44s
Deploy Trade-In / build-backend (push) Successful in 56s
2026-07-03 18:43:38 +00:00
b22edc6d6f Merge pull request 'chore(claude): token-diet — worker-агенты на sonnet (opus только deep-code-reviewer)' (#2292) from chore/claude-token-diet into main 2026-07-03 15:49:38 +00:00
bot-backend
25186a78ac chore(claude): token-diet — worker-агенты на sonnet (opus только deep-code-reviewer), forgejo MCP deferred в main-окне, analyst throttle ready>=10
All checks were successful
CI Trade-In / changes (pull_request) Successful in 8s
CI / changes (pull_request) Successful in 8s
CI Trade-In / backend-tests (pull_request) Has been skipped
CI Trade-In / frontend-checks (pull_request) Has been skipped
CI / backend-tests (pull_request) Has been skipped
CI / frontend-tests (pull_request) Has been skipped
CI / openapi-codegen-check (pull_request) Has been skipped
2026-07-03 18:46:58 +03:00
666962185c fix(tradein/estimator): парсер адресов — имя-перед-типом, glued-суффикс, микрорайоны (#2269) (#2290)
All checks were successful
Deploy Trade-In / test (push) Successful in 1m30s
Deploy Trade-In / changes (push) Successful in 9s
Deploy Trade-In / build-frontend (push) Has been skipped
Deploy Trade-In / build-browser (push) Has been skipped
Deploy Trade-In / build-backend (push) Successful in 56s
Deploy Trade-In / deploy (push) Successful in 45s
2026-07-03 13:03:25 +00:00
d6d4a1c94a fix(site-finder): «Скачать отчёт» без пустых скобок на fast-path из кэша (#2289)
All checks were successful
Deploy / changes (push) Successful in 7s
Deploy / build-backend (push) Has been skipped
Deploy / build-worker (push) Has been skipped
Deploy / build-frontend (push) Successful in 3m16s
Deploy / deploy (push) Successful in 1m18s
2026-07-03 11:32:04 +00:00
2f127c4316 feat(site-finder): кнопка «Полный отчёт (PDF)» (#2259 PR-E, финал эпика) (#2288)
All checks were successful
Deploy / changes (push) Successful in 9s
Deploy / build-backend (push) Has been skipped
Deploy / build-worker (push) Has been skipped
Deploy / build-frontend (push) Successful in 3m28s
Deploy / deploy (push) Successful in 1m17s
2026-07-03 11:18:31 +00:00
034d9f9bca feat(reports): Celery-джоба полного PDF-отчёта + кэш + 3 endpoint'а (#2259 PR-D) (#2287)
Some checks failed
Deploy / changes (push) Successful in 8s
Deploy / build-backend (push) Successful in 3m12s
Deploy / build-worker (push) Successful in 4m8s
Deploy / build-frontend (push) Successful in 4m46s
Deploy / deploy (push) Has been cancelled
2026-07-03 11:13:06 +00:00
0b3ec7f91b feat(exporters): серверный статик-рендер карт для PDF-отчёта (#2259 PR-C Maps) (#2286)
All checks were successful
Deploy / deploy (push) Successful in 1m59s
Deploy / changes (push) Successful in 8s
Deploy / build-frontend (push) Has been skipped
Deploy / build-backend (push) Successful in 3m47s
Deploy / build-worker (push) Successful in 4m45s
2026-07-03 10:29:23 +00:00
e7881cf45c feat(exporters): PURE HTML §4–§7 полного PDF-отчёта (#2259 PR-B Sections) (#2285)
All checks were successful
Deploy / changes (push) Successful in 8s
Deploy / build-frontend (push) Has been skipped
Deploy / build-backend (push) Successful in 2m5s
Deploy / build-worker (push) Successful in 3m50s
Deploy / deploy (push) Successful in 1m55s
2026-07-03 10:10:43 +00:00
c0c18f4ecc feat(exporters): PURE HTML-агрегатор §1–§3 + альтернативы (#2259 PR-A Foundation) (#2284)
All checks were successful
Deploy / changes (push) Successful in 8s
Deploy / build-frontend (push) Has been skipped
Deploy / build-backend (push) Successful in 1m56s
Deploy / build-worker (push) Successful in 3m56s
Deploy / deploy (push) Successful in 1m49s
2026-07-03 09:52:32 +00:00
85435fbf1b fix(site-finder): SET LOCAL литералом — bind-параметр ломал nearby_network_zones (#2283)
All checks were successful
Deploy / changes (push) Successful in 8s
Deploy / build-frontend (push) Has been skipped
Deploy / build-backend (push) Successful in 1m57s
Deploy / build-worker (push) Successful in 3m3s
Deploy / deploy (push) Successful in 1m47s
2026-07-03 08:44:38 +00:00
8a42fba836 feat(site-finder): блок «Сети рядом с участком» в §3 (Фаза C2-UI) (#2282)
All checks were successful
Deploy / changes (push) Successful in 9s
Deploy / build-backend (push) Has been skipped
Deploy / build-worker (push) Has been skipped
Deploy / build-frontend (push) Successful in 3m22s
Deploy / deploy (push) Successful in 1m17s
2026-07-03 08:23:59 +00:00
7c12812f81 feat(site-finder): позитив-разрез сетевых ЗОУИТ «сеть рядом с участком» (Фаза C2/C3) (#2281)
Some checks failed
Deploy / changes (push) Successful in 7s
Deploy / build-backend (push) Successful in 2m49s
Deploy / build-worker (push) Successful in 4m31s
Deploy / build-frontend (push) Successful in 5m19s
Deploy / deploy (push) Has been cancelled
2026-07-03 08:18:24 +00:00
99ee09e230 feat(site-finder): теплотрассы из OSM в слой инженерной инфраструктуры (Фаза C1) (#2280)
All checks were successful
Deploy / changes (push) Successful in 21s
Deploy / build-frontend (push) Has been skipped
Deploy / build-backend (push) Successful in 1m53s
Deploy / build-worker (push) Successful in 3m11s
Deploy / deploy (push) Successful in 1m34s
2026-07-03 08:05:44 +00:00
dfc1ee8be0 feat: ревью-инструмент авто-маппингов Объектив↔ДОМ.РФ (/admin/mapping-review) (#2278)
Some checks failed
Deploy / changes (push) Successful in 8s
Deploy / build-backend (push) Successful in 2m47s
Deploy / build-worker (push) Successful in 5m42s
Deploy / build-frontend (push) Successful in 5m55s
Deploy / deploy (push) Has been cancelled
2026-07-03 07:59:13 +00:00
039954580e Merge pull request 'fix(tradein/estimator): кросс-source дедуп street+house union-find + savepoint вокруг ratio-lookup (#2265)' (#2273) from fix/tradein-dedup-addr-key into main
All checks were successful
Deploy Trade-In / changes (push) Successful in 11s
Deploy Trade-In / build-frontend (push) Has been skipped
Deploy Trade-In / build-browser (push) Has been skipped
Deploy Trade-In / test (push) Successful in 1m35s
Deploy Trade-In / build-backend (push) Successful in 59s
Deploy Trade-In / deploy (push) Successful in 47s
2026-07-03 07:51:47 +00:00
bot-backend
6f81ee73d7 fix(tradein/estimator): полный сьют зелёный после дедуп-ключа (#2265)
All checks were successful
CI Trade-In / changes (pull_request) Successful in 10s
CI / changes (pull_request) Successful in 12s
CI Trade-In / frontend-checks (pull_request) Has been skipped
CI / backend-tests (pull_request) Has been skipped
CI / frontend-tests (pull_request) Has been skipped
CI / openapi-codegen-check (pull_request) Has been skipped
CI Trade-In / backend-tests (pull_request) Successful in 2m20s
2026-07-03 10:48:15 +03:00
703f6f60c0 Merge pull request 'fix(tradein/v2/a11y): немодальный SectionOverlay + контраст токенов + headings — a11y round 3 (#2264)' (#2279) from fix/tradein-v2-a11y-round3 into main
Some checks failed
Deploy Trade-In / changes (push) Successful in 10s
Deploy Trade-In / test (push) Has been skipped
Deploy Trade-In / build-browser (push) Has been skipped
Deploy Trade-In / build-backend (push) Has been skipped
Deploy Trade-In / build-frontend (push) Successful in 3m27s
Deploy Trade-In / deploy (push) Has been cancelled
2026-07-03 07:47:09 +00:00
4a9ddb2a9a Merge pull request 'chore(design-sync): durable-инпуты DS-панели tradein — 6 floor-card превью, шрифты Manrope/Plex, source-kit fork-фикс (#2267)' (#2274) from chore/design-sync-tradein into main 2026-07-03 07:38:41 +00:00
bot-backend
a3cab9023f fix(tradein/v2/a11y): немодальный SectionOverlay + контраст токенов + headings — a11y round 3 (#2264)
All checks were successful
CI Trade-In / changes (pull_request) Successful in 16s
CI / changes (pull_request) Successful in 16s
CI Trade-In / backend-tests (pull_request) Has been skipped
CI / backend-tests (pull_request) Has been skipped
CI / frontend-tests (pull_request) Has been skipped
CI / openapi-codegen-check (pull_request) Has been skipped
CI Trade-In / frontend-checks (pull_request) Successful in 1m13s
- SectionOverlay: убран aria-modal (честный non-modal fullscreen-view); focus-trap
  теперь охватывает живой верхний nav + оверлей (query по артборду, фильтр [inert]),
  Tab из nav возвращается в оверлей, не в спрятанный main/footer.
- Контраст 6 muted/disabled токенов затемнён до >=4.6:1 vs worst-case #dce6f1
  (hue сохранён, lightness-only): muted2/3/4, hint, disabledValue/Label.
- accent -> accentDeep как мелкий текст: номер секции оверлея (15px), счётчик
  "Мои отчёты" (10px).
- Семантика заголовков: sr-only h1 «Мера — оценка квартиры» + h2 (оверлей,
  ResultPanel, ObjectSummary, placeholder-панели).
- HeroBar «КОЭФ. ЛОКАЦИИ»: role=button + tabIndex + Enter/Space + aria-label.
- Фокус после успешной оценки переносится на результат-регион (role=region,
  tabIndex=-1, aria-labelledby); live-region «Оценка готова» сохранён.
- Инпут focus-ring усилен (2px solid accentDeep).
- combobox адреса: aria-live анонс количества подсказок; aria-required на
  адрес + площадь; декоративные inline-SVG помечены aria-hidden.
2026-07-03 10:37:42 +03:00
bot-backend
9cdc7cef21 fix(tradein/estimator): дедуп — нумерованные улицы + house-no guard на street-union (#2265, review follow-up)
Some checks failed
CI Trade-In / changes (pull_request) Successful in 9s
CI / changes (pull_request) Successful in 10s
CI Trade-In / frontend-checks (pull_request) Has been skipped
CI / backend-tests (pull_request) Has been skipped
CI / frontend-tests (pull_request) Has been skipped
CI / openapi-codegen-check (pull_request) Has been skipped
CI Trade-In / backend-tests (pull_request) Failing after 1m51s
2026-07-03 10:37:26 +03:00
fc5c4b1e40 Merge pull request 'chore(tradein): полное выключение источника n1 — миграция 165 + вычистка backend/ops/frontend (#2204)' (#2276) from chore/tradein-remove-n1-source into main
All checks were successful
Deploy Trade-In / changes (push) Successful in 10s
Deploy Trade-In / build-browser (push) Successful in 37s
Deploy Trade-In / test (push) Successful in 1m49s
Deploy Trade-In / build-frontend (push) Successful in 2m56s
Deploy Trade-In / build-backend (push) Successful in 1m13s
Deploy Trade-In / deploy (push) Successful in 2m53s
2026-07-03 07:33:28 +00:00
bot-backend
9835876ced merge remote chore/design-sync-tradein (superseded Jul-2 inputs; keep Jul-3 state: Manrope/Plex fonts, fork-fix source-kit, 6 floor-card previews)
All checks were successful
CI / changes (pull_request) Successful in 8s
CI / backend-tests (pull_request) Has been skipped
CI / frontend-tests (pull_request) Has been skipped
CI / openapi-codegen-check (pull_request) Has been skipped
2026-07-03 10:25:01 +03:00
bot-backend
375d59c48d chore(design-sync): durable-инпуты tradein DS-панели — 6 floor-card превью, бренд-шрифты Manrope/Plex, source-kit fork-фикс (#2267) 2026-07-03 10:23:45 +03:00
bot-backend
9a9f13d513 fix(tradein/estimator): savepoint+retry вокруг ratio-lookup — transient не гасит expected_sold (#2265)
Some checks failed
CI Trade-In / changes (pull_request) Successful in 9s
CI / changes (pull_request) Successful in 8s
CI Trade-In / frontend-checks (pull_request) Has been skipped
CI / backend-tests (pull_request) Has been skipped
CI / frontend-tests (pull_request) Has been skipped
CI / openapi-codegen-check (pull_request) Has been skipped
CI Trade-In / backend-tests (pull_request) Failing after 1m36s
2026-07-03 10:19:57 +03:00
bot-backend
a08e263c60 fix(tradein/estimator): кросс-source дедуп — устойчивый адресный ключ street+house (#2265) 2026-07-03 10:19:19 +03:00
bot-backend
ca0a51c876 chore(tradein/ui): вычистить источник n1 из source-registry и UI (#2204)
All checks were successful
CI / changes (pull_request) Successful in 9s
CI Trade-In / changes (pull_request) Successful in 9s
CI / backend-tests (pull_request) Has been skipped
CI / frontend-tests (pull_request) Has been skipped
CI / openapi-codegen-check (pull_request) Has been skipped
CI Trade-In / frontend-checks (pull_request) Successful in 1m36s
CI Trade-In / backend-tests (pull_request) Successful in 1m57s
2026-07-03 10:16:53 +03:00
68429967ab Merge pull request 'fix(tradein/scraper-kit): scale cian anchor watchdog for proxy-pool browser fetches (#2160)' (#2272) from fix/tradein-cian-anchor-timeout-pool into main
All checks were successful
Deploy Trade-In / changes (push) Successful in 13s
Deploy Trade-In / build-frontend (push) Has been skipped
Deploy Trade-In / build-browser (push) Has been skipped
Deploy Trade-In / test (push) Successful in 1m39s
Deploy Trade-In / build-backend (push) Successful in 1m32s
Deploy Trade-In / deploy (push) Successful in 2m38s
2026-07-03 07:09:50 +00:00
bot-backend
dd050af16d chore(tradein): полное выключение источника n1 — миграция 165 + вычистка backend/ops (#2204) 2026-07-03 10:08:56 +03:00
bot-backend
8a10d943d7 fix(tradein/scraper-kit): scale cian anchor watchdog for proxy-pool browser fetches (#2160)
All checks were successful
CI Trade-In / changes (pull_request) Successful in 11s
CI / changes (pull_request) Successful in 11s
CI Trade-In / frontend-checks (pull_request) Has been skipped
CI / backend-tests (pull_request) Has been skipped
CI / frontend-tests (pull_request) Has been skipped
CI / openapi-codegen-check (pull_request) Has been skipped
CI Trade-In / backend-tests (pull_request) Successful in 1m51s
Fixed ANCHOR_TIMEOUT_SEC=240 guillotined every cian anchor when running via
USE_PROXY_POOL_BROWSER=true: each SERP fetch goes through camoufox with relaunch
on proxy swap (13-45s/page), and an anchor = 4 room-buckets x pages_per_anchor
SERP fetches + detail_top_n details + houses-enrich >> 240s. save_listings never
ran, lots_fetched=0, and after 3 consecutive skips the run was mark_banned.

Introduces _cian_anchor_timeout_s() mirroring the avito/yandex/domclick scaling
pattern: max(ANCHOR_TIMEOUT_SEC, buckets*pages*(delay+per_serp) + details*per_detail
+ houses_budget). Defaults -> 1580s worst-case watchdog (hang guard, not expected
duration). Applied to both the scraper-kit copy (strangler target) and the active
app.services.scrape_pipeline copy that run 581 actually hit.
2026-07-03 10:05:06 +03:00
f891f8f00f feat(eias-heat): +6 тепло-организаций ЕКБ — ненулевые резервы ×10 (#2119) (#2270)
All checks were successful
Deploy / changes (push) Successful in 8s
Deploy / build-frontend (push) Has been skipped
Deploy / build-backend (push) Successful in 2m14s
Deploy / build-worker (push) Successful in 3m19s
Deploy / deploy (push) Successful in 1m43s
2026-07-03 07:02:03 +00:00
bbe596747d Merge pull request 'fix(tradein/data): yandex url-«дубли» — каноникализация URL (скрейпер + миграция 164) + manifest 162 (#2235, #2216)' (#2271) from fix/tradein-yandex-url-dedup into main
All checks were successful
Deploy Trade-In / changes (push) Successful in 12s
Deploy Trade-In / build-frontend (push) Has been skipped
Deploy Trade-In / build-browser (push) Has been skipped
Deploy Trade-In / test (push) Successful in 1m38s
Deploy Trade-In / build-backend (push) Successful in 1m44s
Deploy Trade-In / deploy (push) Successful in 3m3s
2026-07-03 06:53:27 +00:00
bot-backend
dd8b5769e5 fix(tradein/migrations): дописать 162 в _manifest_applied — stale manifest из #2228 (#2216)
All checks were successful
CI Trade-In / changes (pull_request) Successful in 9s
CI / changes (pull_request) Successful in 9s
CI Trade-In / frontend-checks (pull_request) Has been skipped
CI / backend-tests (pull_request) Has been skipped
CI / frontend-tests (pull_request) Has been skipped
CI / openapi-codegen-check (pull_request) Has been skipped
CI Trade-In / backend-tests (pull_request) Successful in 1m44s
2026-07-03 09:41:44 +03:00
c44bb49809 Merge pull request 'fix(tradein/data): external_valuations.house_id — резолв в write-path + идемпотентный бэкфилл (#2236)' (#2263) from fix/tradein-extval-house-id into main
All checks were successful
Deploy Trade-In / changes (push) Successful in 19s
Deploy Trade-In / build-frontend (push) Successful in 1m3s
Deploy Trade-In / build-browser (push) Successful in 1m7s
Deploy Trade-In / test (push) Successful in 2m4s
Deploy Trade-In / build-backend (push) Successful in 1m11s
Deploy Trade-In / deploy (push) Successful in 3m20s
2026-07-03 06:40:21 +00:00
bot-backend
33e3503f97 fix(tradein/data): yandex url-дубли — дедуп-миграция + инвариант + корень (#2235) 2026-07-03 09:39:29 +03:00
fc6542416d feat(exporters): §4.2 supply-only блок в ТЗ-PDF (#2177) (#2260)
All checks were successful
Deploy / changes (push) Successful in 9s
Deploy / build-frontend (push) Has been skipped
Deploy / build-backend (push) Successful in 2m50s
Deploy / build-worker (push) Successful in 5m25s
Deploy / deploy (push) Successful in 2m27s
2026-07-03 06:38:23 +00:00
e1c5f2184c fix(tradein/scraper-kit): wire browser proxy-pool into yandex/cian SERP + persist activation flags (#2160) (#2268)
Some checks failed
Deploy Trade-In / deploy (push) Blocked by required conditions
Deploy Trade-In / changes (push) Successful in 17s
Deploy Trade-In / build-frontend (push) Successful in 43s
Deploy Trade-In / build-browser (push) Successful in 44s
Deploy Trade-In / test (push) Successful in 1m58s
Deploy Trade-In / build-backend (push) Has been cancelled
2026-07-03 06:36:56 +00:00
bot-backend
c2096249d5 fix(tradein/data): dry-run бэкфилла — savepoint-паритет с write-веткой (#2236)
All checks were successful
CI Trade-In / changes (pull_request) Successful in 17s
CI / changes (pull_request) Successful in 18s
CI Trade-In / frontend-checks (pull_request) Has been skipped
CI / backend-tests (pull_request) Has been skipped
CI / frontend-tests (pull_request) Has been skipped
CI / openapi-codegen-check (pull_request) Has been skipped
CI Trade-In / backend-tests (pull_request) Successful in 2m58s
2026-07-03 09:28:27 +03:00
21608eced1 fix(site-finder): три честности-фикса — B1 ГРС «выход №N» + околонуль + Росреестр-квартал (#2257)
All checks were successful
Deploy / changes (push) Successful in 9s
Deploy / build-backend (push) Successful in 3m35s
Deploy / build-worker (push) Successful in 5m31s
Deploy / build-frontend (push) Successful in 7m23s
Deploy / deploy (push) Successful in 2m17s
2026-07-03 06:27:27 +00:00
5f6db8dbb7 Merge pull request 'fix(tradein/security): admin.py — generic detail + error-id вместо сырого текста исключений (#2234)' (#2261) from fix/tradein-admin-exception-leak into main
All checks were successful
Deploy Trade-In / changes (push) Successful in 12s
Deploy Trade-In / build-browser (push) Has been skipped
Deploy Trade-In / build-frontend (push) Has been skipped
Deploy Trade-In / test (push) Successful in 2m25s
Deploy Trade-In / build-backend (push) Successful in 1m54s
Deploy Trade-In / deploy (push) Successful in 1m32s
2026-07-03 06:27:22 +00:00
bot-backend
291717fa2f fix(tradein/data): external_valuations.house_id — резолв в write-path + идемпотентный бэкфилл (#2236) 2026-07-03 09:22:44 +03:00
bot-backend
efa0bafa18 fix(tradein/security): 404 IMV-адрес — warning без traceback, не exception-шум (#2234)
All checks were successful
CI Trade-In / changes (pull_request) Successful in 13s
CI / changes (pull_request) Successful in 14s
CI Trade-In / frontend-checks (pull_request) Has been skipped
CI / backend-tests (pull_request) Has been skipped
CI / frontend-tests (pull_request) Has been skipped
CI / openapi-codegen-check (pull_request) Has been skipped
CI Trade-In / backend-tests (pull_request) Successful in 2m25s
2026-07-03 09:20:44 +03:00
bot-backend
c65a73ca9a fix(tradein/security): admin.py — generic detail + error-id вместо сырого текста исключений (#2234) 2026-07-03 09:16:34 +03:00
910bfed5f1 Merge pull request 'fix(tradein/security): photo-upload — чанковое чтение с капом 10MB + Caddy request_body 12MB (#2233)' (#2258) from fix/tradein-photo-upload-stream-cap into main
All checks were successful
Deploy Trade-In / changes (push) Successful in 12s
Deploy Trade-In / build-browser (push) Successful in 38s
Deploy Trade-In / build-frontend (push) Successful in 40s
Deploy Trade-In / test (push) Successful in 1m48s
Deploy Trade-In / build-backend (push) Successful in 58s
Deploy Trade-In / deploy (push) Successful in 56s
2026-07-03 06:15:57 +00:00
e440209d91 feat(site-finder): «Пересчитать §7 с этой программой» из блока альтернатив (#2181 хвост) (#2256)
All checks were successful
Deploy / changes (push) Successful in 15s
Deploy / build-backend (push) Has been skipped
Deploy / build-worker (push) Has been skipped
Deploy / build-frontend (push) Successful in 4m35s
Deploy / deploy (push) Successful in 1m28s
2026-07-03 06:11:58 +00:00
bot-backend
edbdd28286 fix(tradein/security): photo-upload — чанковое чтение с капом 10MB + Caddy request_body 12MB (#2233)
All checks were successful
CI Trade-In / changes (pull_request) Successful in 29s
CI / changes (pull_request) Successful in 20s
CI Trade-In / frontend-checks (pull_request) Has been skipped
CI / backend-tests (pull_request) Has been skipped
CI / frontend-tests (pull_request) Has been skipped
CI / openapi-codegen-check (pull_request) Has been skipped
CI Trade-In / backend-tests (pull_request) Successful in 2m16s
2026-07-03 09:06:51 +03:00
14d76fb136 Merge pull request 'feat(tradein/ui): честное раскрытие покрытия диапазона в hero (#2209)' (#2232) from fix/tradein-range-disclosure into main
All checks were successful
Deploy Trade-In / changes (push) Successful in 12s
Deploy Trade-In / test (push) Has been skipped
Deploy Trade-In / build-browser (push) Has been skipped
Deploy Trade-In / build-backend (push) Has been skipped
Deploy Trade-In / build-frontend (push) Successful in 3m3s
Deploy Trade-In / deploy (push) Successful in 1m14s
2026-07-03 06:06:31 +00:00
29a3cf9483 Merge pull request 'fix(tradein): оставить domklik активным — отключить его TTL-деактивацию (#2204)' (#2253) from fix/tradein-keep-domklik-active into main
All checks were successful
Deploy Trade-In / build-frontend (push) Has been skipped
Deploy Trade-In / build-browser (push) Has been skipped
Deploy Trade-In / changes (push) Successful in 10s
Deploy Trade-In / test (push) Successful in 1m36s
Deploy Trade-In / build-backend (push) Successful in 29s
Deploy Trade-In / deploy (push) Successful in 51s
2026-07-03 06:00:21 +00:00
bot-backend
bbd47ae536 fix(tradein): оставить domklik активным — отключить его TTL-деактивацию (#2204)
All checks were successful
CI Trade-In / changes (pull_request) Successful in 9s
CI / changes (pull_request) Successful in 9s
CI Trade-In / frontend-checks (pull_request) Has been skipped
CI / backend-tests (pull_request) Has been skipped
CI / frontend-tests (pull_request) Has been skipped
CI / openapi-codegen-check (pull_request) Has been skipped
CI Trade-In / backend-tests (pull_request) Successful in 1m39s
Продуктовое решение (user, 2026-07-03): domklik остаётся активным источником;
TTL-деактивация по scraped_at (заведена миграцией 160 этой ночью) отключается
до возобновления регулярных domclick-свипов. deactivate_stale_n1 не тронут
(n1 мёртв — деактивация остаётся). Миграция 163 идемпотентна (enabled=false).

Refs #2204
2026-07-03 08:52:56 +03:00
6f5bdffbe4 feat(site-finder): OSM-геокод городских ТП/РП ЕКБ (#2119 B2, финальный хвост) (#2252)
All checks were successful
Deploy / changes (push) Successful in 7s
Deploy / build-frontend (push) Has been skipped
Deploy / build-backend (push) Successful in 1m51s
Deploy / build-worker (push) Successful in 3m0s
Deploy / deploy (push) Successful in 1m36s
2026-07-03 05:22:02 +00:00
15757685c9 feat(site-finder): Celery + §3 точки выхода газосети (#2119 B2, PR-4/4 — финал формы-6) (#2251)
All checks were successful
Deploy / changes (push) Successful in 7s
Deploy / build-backend (push) Successful in 2m17s
Deploy / build-worker (push) Successful in 3m19s
Deploy / build-frontend (push) Successful in 3m54s
Deploy / deploy (push) Successful in 1m34s
2026-07-03 04:26:41 +00:00
55b13e2b13 fix(dadata): фолбэк на suggest/address при отключённой фиче CLEAN (#2250)
All checks were successful
Deploy / changes (push) Successful in 6s
Deploy / build-frontend (push) Has been skipped
Deploy / build-backend (push) Successful in 1m54s
Deploy / build-worker (push) Successful in 2m56s
Deploy / deploy (push) Successful in 1m33s
2026-07-03 03:42:40 +00:00
cc9516ab3d feat(ekgas): геокод точек выхода через DaData (#2119 B2, PR-3/4) (#2249)
All checks were successful
Deploy / changes (push) Successful in 7s
Deploy / build-frontend (push) Has been skipped
Deploy / build-backend (push) Successful in 2m6s
Deploy / build-worker (push) Successful in 3m8s
Deploy / deploy (push) Successful in 1m34s
2026-07-03 03:19:18 +00:00
c438c9a9e8 fix(ekgas): label-детект сетки формы 6 — живой файл сдвинут (#2119 B2) (#2248)
All checks were successful
Deploy / changes (push) Successful in 7s
Deploy / build-frontend (push) Has been skipped
Deploy / build-backend (push) Successful in 1m54s
Deploy / deploy (push) Successful in 1m27s
Deploy / build-worker (push) Successful in 3m9s
2026-07-03 02:51:03 +00:00
af70d50f0d feat(site-finder): лоадер точек выхода ГРС «Екатеринбурггаз» — форма 6 (#2119 B2, PR-2/4) (#2247)
All checks were successful
Deploy / changes (push) Successful in 7s
Deploy / build-frontend (push) Has been skipped
Deploy / build-backend (push) Successful in 1m46s
Deploy / build-worker (push) Successful in 2m51s
Deploy / deploy (push) Successful in 1m28s
2026-07-03 02:25:12 +00:00
7c73bbfbeb feat(db): схема gas_grs_outlet_points — форма 6 Екатеринбурггаза (#2119 B2, PR-1/4) (#2246)
All checks were successful
Deploy / changes (push) Successful in 7s
Deploy / build-frontend (push) Has been skipped
Deploy / build-backend (push) Successful in 30s
Deploy / build-worker (push) Successful in 30s
Deploy / deploy (push) Successful in 1m16s
2026-07-03 02:15:50 +00:00
830d9f429a fix(etl): гео-проход — префикс «г Екатеринбург» адресам без города (#2177 остаток) (#2245)
All checks were successful
Deploy / changes (push) Successful in 7s
Deploy / build-frontend (push) Has been skipped
Deploy / build-backend (push) Successful in 1m46s
Deploy / build-worker (push) Successful in 3m0s
Deploy / deploy (push) Successful in 1m37s
2026-07-03 01:41:19 +00:00
031bafde86 feat(etl): гео-проход auto_core_geo_v6 для tier_b/ambiguous mapping (#2177 остаток) (#2244)
All checks were successful
Deploy / changes (push) Successful in 7s
Deploy / build-frontend (push) Has been skipped
Deploy / build-backend (push) Successful in 1m43s
Deploy / build-worker (push) Successful in 2m47s
Deploy / deploy (push) Successful in 1m27s
2026-07-03 01:19:59 +00:00
5888b0b9d3 perf(db): physflat-covering индекс v2 с price_per_m2_rub в INCLUDE (#2177 follow-up) (#2243)
All checks were successful
Deploy / changes (push) Successful in 6s
Deploy / build-frontend (push) Has been skipped
Deploy / build-backend (push) Successful in 29s
Deploy / build-worker (push) Successful in 29s
Deploy / deploy (push) Successful in 1m26s
2026-07-03 00:49:08 +00:00
4b6d1cbc06 fix(report): авторетрай §7 с densify — «не вместила ни одной секции» больше не 422 при загрузке (#2242)
All checks were successful
Deploy / build-frontend (push) Successful in 3m26s
Deploy / deploy (push) Successful in 1m29s
Deploy / changes (push) Successful in 7s
Deploy / build-backend (push) Has been skipped
Deploy / build-worker (push) Has been skipped
2026-07-03 00:31:25 +00:00
1bba3db7eb fix(best-layouts): lots-проекты в пуле core-матчинга — supply-only реально наполняется (#2177) (#2241)
All checks were successful
Deploy / build-backend (push) Successful in 1m40s
Deploy / changes (push) Successful in 6s
Deploy / build-frontend (push) Has been skipped
Deploy / build-worker (push) Successful in 2m42s
Deploy / deploy (push) Successful in 1m24s
2026-07-03 00:14:19 +00:00
9cfefc0ca4 feat(site-finder): supply-only fallback в §4.2 — предложение без темпа продаж (#2177 шаг 3) (#2240)
Some checks failed
Deploy / changes (push) Successful in 6s
Deploy / build-backend (push) Successful in 1m48s
Deploy / build-worker (push) Failing after 1m54s
Deploy / build-frontend (push) Successful in 3m31s
Deploy / deploy (push) Has been skipped
2026-07-02 23:18:42 +00:00
fc55c2c0b7 feat(best-layouts): атрибуция velocity по ядру комплекса (#2177 шаг 2b) (#2239)
All checks were successful
Deploy / changes (push) Successful in 6s
Deploy / build-backend (push) Successful in 1m54s
Deploy / build-worker (push) Successful in 2m59s
Deploy / build-frontend (push) Successful in 3m34s
Deploy / deploy (push) Successful in 1m32s
2026-07-02 22:41:25 +00:00
4c23c222f6 fix(vodokanal): column-aligned парсинг DOCX — реальные резервы вместо № п/п (#2119/#2126) (#2238)
All checks were successful
Deploy / changes (push) Successful in 7s
Deploy / build-frontend (push) Has been skipped
Deploy / build-worker (push) Successful in 2m48s
Deploy / build-backend (push) Successful in 1m33s
Deploy / deploy (push) Successful in 1m27s
2026-07-02 22:03:15 +00:00
e9c8d05e1d feat(etl): auto_core_dev_v5 — core-name+dev backfill mapping'а (#2177 шаг 2) (#2237)
Some checks failed
Deploy / changes (push) Successful in 7s
Deploy / deploy (push) Blocked by required conditions
Deploy / build-backend (push) Successful in 2m8s
Deploy / build-frontend (push) Has been cancelled
Deploy / build-worker (push) Has been cancelled
2026-07-02 21:59:16 +00:00
8ea52b9b7a fix(tradein/ops): health-check валит деплой + mem-лимиты + log-rotation (#2214) (#2231)
All checks were successful
Deploy Trade-In / build-browser (push) Successful in 32s
Deploy Trade-In / changes (push) Successful in 9s
Deploy Trade-In / build-frontend (push) Successful in 34s
Deploy Trade-In / test (push) Successful in 1m40s
Deploy Trade-In / build-backend (push) Successful in 26s
Deploy Trade-In / deploy (push) Successful in 45s
2026-07-02 21:29:30 +00:00
bot-backend
9de867631b feat(tradein/ui): честное раскрытие покрытия диапазона в hero (#2209)
All checks were successful
CI Trade-In / changes (pull_request) Successful in 8s
CI Trade-In / backend-tests (pull_request) Has been skipped
CI / backend-tests (pull_request) Has been skipped
CI / frontend-tests (pull_request) Has been skipped
CI / changes (pull_request) Successful in 8s
CI / openapi-codegen-check (pull_request) Has been skipped
CI Trade-In / frontend-checks (pull_request) Successful in 1m3s
Хинт диапазона expected_sold дополнен замеренной точностью: «фактическая
цена продажи попадает в диапазон в ~80% случаев (июль 2026)» — по живому
бэктесту на реальных ДКП Росреестра (n=276, coverage 81.5%, engine=full).
Раскрытие вместо молчаливой выдачи Q1-Q3 за прогнозный интервал —
acceptance-пункт «явно раскрыть точность/покрытие» из #2209.

Refs #2209
2026-07-03 00:23:03 +03:00
9f28092356 fix(tradein/estimator): min-width floor ±12% на ценовой диапазон (#2209) (#2230)
All checks were successful
Deploy Trade-In / test (push) Successful in 1m31s
Deploy Trade-In / changes (push) Successful in 9s
Deploy Trade-In / build-frontend (push) Has been skipped
Deploy Trade-In / build-browser (push) Has been skipped
Deploy Trade-In / build-backend (push) Successful in 1m18s
Deploy Trade-In / deploy (push) Successful in 46s
2026-07-02 21:21:28 +00:00
5fb5675871 fix(tradein/security): defense-in-depth trusted-header auth + rate-limiter fix (#2213) (#2229)
All checks were successful
Deploy / changes (push) Successful in 8s
Deploy Trade-In / changes (push) Successful in 11s
Deploy Trade-In / build-browser (push) Successful in 35s
Deploy Trade-In / build-frontend (push) Successful in 38s
Deploy / build-worker (push) Successful in 42s
Deploy / build-backend (push) Successful in 43s
Deploy / build-frontend (push) Successful in 42s
Deploy / deploy (push) Successful in 1m32s
Deploy Trade-In / test (push) Successful in 1m50s
Deploy Trade-In / build-backend (push) Successful in 52s
Deploy Trade-In / deploy (push) Successful in 51s
2026-07-02 21:15:58 +00:00
0793571c4a fix(tradein/deploy): миграции ДО подъёма app-контейнеров + manifest-инвариант (#2216) (#2228)
All checks were successful
Deploy Trade-In / test (push) Successful in 1m35s
Deploy Trade-In / build-backend (push) Successful in 29s
Deploy Trade-In / deploy (push) Successful in 49s
Deploy Trade-In / changes (push) Successful in 20s
Deploy Trade-In / build-browser (push) Successful in 29s
Deploy Trade-In / build-frontend (push) Successful in 30s
2026-07-02 21:03:37 +00:00
19ecd941f3 fix(tradein/ui): единый source-registry + честные знаменатели/лейблы источников (#2211) (#2227)
All checks were successful
Deploy Trade-In / changes (push) Successful in 12s
Deploy Trade-In / test (push) Has been skipped
Deploy Trade-In / build-browser (push) Has been skipped
Deploy Trade-In / build-backend (push) Has been skipped
Deploy Trade-In / build-frontend (push) Successful in 3m3s
Deploy Trade-In / deploy (push) Successful in 1m4s
2026-07-02 20:57:26 +00:00
ae483468c9 feat(site-finder): оптимизатор программ — «Как участок сходится» при отрицательном вердикте (#2181) (#2222)
All checks were successful
Deploy / changes (push) Successful in 7s
Deploy / build-backend (push) Successful in 2m40s
Deploy / build-worker (push) Successful in 3m54s
Deploy / build-frontend (push) Successful in 4m55s
Deploy / deploy (push) Successful in 1m35s
2026-07-02 20:57:12 +00:00
ac19c4f7b3 feat(tradein/monitor): алерт на staleness данных deals по max(deal_date) (#2212) (#2226)
All checks were successful
Deploy Trade-In / build-frontend (push) Has been skipped
Deploy Trade-In / build-browser (push) Has been skipped
Deploy Trade-In / changes (push) Successful in 8s
Deploy Trade-In / test (push) Successful in 1m35s
Deploy Trade-In / build-backend (push) Successful in 56s
Deploy Trade-In / deploy (push) Successful in 1m2s
2026-07-02 20:50:37 +00:00
4ce0f51bb8 perf(tradein): снять блокировку event loop на POST /estimate (#2207) (#2225)
All checks were successful
Deploy Trade-In / changes (push) Successful in 10s
Deploy Trade-In / build-frontend (push) Has been skipped
Deploy Trade-In / build-browser (push) Has been skipped
Deploy Trade-In / test (push) Successful in 1m35s
Deploy Trade-In / build-backend (push) Successful in 1m27s
Deploy Trade-In / deploy (push) Successful in 1m5s
2026-07-02 20:39:48 +00:00
47f45fc3d2 fix(tradein/scrapers): bump scraped_at при ре-подтверждении живым + бэкфилл (#2206) (#2224)
All checks were successful
Deploy Trade-In / build-browser (push) Has been skipped
Deploy Trade-In / test (push) Successful in 1m29s
Deploy Trade-In / build-backend (push) Successful in 1m25s
Deploy Trade-In / deploy (push) Successful in 1m48s
Deploy Trade-In / changes (push) Successful in 9s
Deploy Trade-In / build-frontend (push) Has been skipped
2026-07-02 20:30:01 +00:00
72d26b35fa fix(tradein): не запекать preview-флаг в прод-образ + noindex/баннер (#2205) (#2223)
All checks were successful
Deploy Trade-In / changes (push) Successful in 9s
Deploy Trade-In / build-browser (push) Successful in 33s
Deploy Trade-In / test (push) Successful in 1m35s
Deploy Trade-In / build-backend (push) Successful in 32s
Deploy Trade-In / build-frontend (push) Successful in 2m13s
Deploy Trade-In / deploy (push) Successful in 57s
2026-07-02 20:21:58 +00:00
f5b0076e6f fix(tradein): deactivate_stale для domklik/n1 + честная freshness по scraped_at (#2204) (#2221)
All checks were successful
Deploy Trade-In / changes (push) Successful in 9s
Deploy Trade-In / build-frontend (push) Has been skipped
Deploy Trade-In / build-browser (push) Has been skipped
Deploy Trade-In / test (push) Successful in 1m30s
Deploy Trade-In / build-backend (push) Successful in 58s
Deploy Trade-In / deploy (push) Successful in 2m57s
2026-07-02 20:10:34 +00:00
e6000cb029 ci(tradein): pre-merge гейт — backend pytest + frontend type-check/lint (#2208) (#2220)
All checks were successful
Deploy Trade-In / changes (push) Successful in 10s
Deploy Trade-In / build-browser (push) Successful in 34s
Deploy Trade-In / test (push) Successful in 1m38s
Deploy Trade-In / build-backend (push) Successful in 1m29s
Deploy Trade-In / build-frontend (push) Successful in 3m21s
Deploy Trade-In / deploy (push) Successful in 1m18s
2026-07-02 19:56:22 +00:00
b028377584 fix(best-layouts): покрытие §4.2 в комплексах, не в сырых obj_id (#2177, шаг 1/3) (#2198)
All checks were successful
Deploy / changes (push) Successful in 10s
Deploy / build-backend (push) Successful in 2m51s
Deploy / build-worker (push) Successful in 4m5s
Deploy / build-frontend (push) Successful in 5m8s
Deploy / deploy (push) Successful in 1m37s
2026-07-02 19:34:11 +00:00
a7ad1764c1 feat(concept): 3D-макет окружения — соседние здания из кадастра в §7 (#2180) (#2210)
Some checks failed
Deploy / build-frontend (push) Has been cancelled
Deploy / deploy (push) Blocked by required conditions
Deploy / changes (push) Successful in 7s
Deploy / build-backend (push) Has been cancelled
Deploy / build-worker (push) Has been cancelled
2026-07-02 19:33:44 +00:00
ca473a0012 feat(proxy-pool): P4 — browser dynamic pool-proxy (acquire→/fetch→relaunch) + orchestration wiring, ship-dark (#2164)
All checks were successful
Deploy Trade-In / build-browser (push) Successful in 2m13s
Deploy Trade-In / build-backend (push) Successful in 1m18s
Deploy Trade-In / deploy (push) Successful in 2m29s
Deploy Trade-In / changes (push) Successful in 13s
Deploy Trade-In / build-frontend (push) Has been skipped
Deploy Trade-In / test (push) Successful in 1m32s
2026-07-02 19:20:29 +00:00
d8ea4f2fd7 Merge pull request 'fix(tradein/deploy): scraper+backend paths-filter — matching/dedup/scraper-kit trigger rebuilds (#2188)' (#2199) from fix/tradein-deploy-scraper-paths-filter into main
All checks were successful
Deploy Trade-In / changes (push) Successful in 8s
Deploy Trade-In / build-frontend (push) Successful in 30s
Deploy Trade-In / build-browser (push) Successful in 31s
Deploy Trade-In / test (push) Successful in 1m41s
Deploy Trade-In / build-backend (push) Successful in 34s
Deploy Trade-In / deploy (push) Successful in 59s
2026-07-02 19:04:20 +00:00
bot-backend
8fd2be97e4 fix(tradein/deploy): scraper+backend paths-filter — matching/dedup/scraper-kit trigger rebuilds (#2188)
All checks were successful
CI / changes (pull_request) Successful in 6s
CI / backend-tests (pull_request) Has been skipped
CI / frontend-tests (pull_request) Has been skipped
CI / openapi-codegen-check (pull_request) Has been skipped
2026-07-02 22:03:15 +03:00
0937da4260 feat(proxy-pool): P3 — kit curl-paths acquire/release from pool behind USE_PROXY_POOL_CURL, ship-dark (#2163)
All checks were successful
Deploy Trade-In / build-frontend (push) Has been skipped
Deploy Trade-In / build-browser (push) Has been skipped
Deploy Trade-In / changes (push) Successful in 8s
Deploy Trade-In / test (push) Successful in 1m26s
Deploy Trade-In / build-backend (push) Successful in 1m18s
Deploy Trade-In / deploy (push) Successful in 1m22s
2026-07-02 18:56:36 +00:00
ee47e331f3 Merge pull request 'fix(tradein/dedup): drop geo-guard in fias-pass — same-fias identity outranks proximity (#2187)' (#2195) from fix/tradein-dedup-fias-pass-no-geoguard into main
All checks were successful
Deploy Trade-In / build-browser (push) Has been skipped
Deploy Trade-In / test (push) Successful in 1m26s
Deploy Trade-In / deploy (push) Successful in 51s
Deploy Trade-In / changes (push) Successful in 9s
Deploy Trade-In / build-frontend (push) Has been skipped
Deploy Trade-In / build-backend (push) Successful in 51s
2026-07-02 18:51:54 +00:00
a9d06c9c44 Merge pull request 'fix(tradein/matching): P1 extended — never create a house from a numberless address (no cadastral)' (#2194) from fix/tradein-no-numberless-house-insert into main
All checks were successful
Deploy Trade-In / changes (push) Successful in 7s
Deploy Trade-In / test (push) Successful in 1m28s
Deploy Trade-In / build-frontend (push) Has been skipped
Deploy Trade-In / build-browser (push) Has been skipped
Deploy Trade-In / build-backend (push) Successful in 1m22s
Deploy Trade-In / deploy (push) Successful in 1m10s
2026-07-02 18:46:20 +00:00
bot-backend
e84b6e553a fix(tradein/scraper-kit): mirror numberless None-guards in avito houses provider copy
All checks were successful
CI / changes (pull_request) Successful in 7s
CI / backend-tests (pull_request) Has been skipped
CI / frontend-tests (pull_request) Has been skipped
CI / openapi-codegen-check (pull_request) Has been skipped
2026-07-02 21:44:46 +03:00
bcdec5ebd4 feat(rewire): scheduler_main → kit-scheduler behind USE_KIT_SCHEDULER flag, ship-dark (#2192)
All checks were successful
Deploy Trade-In / deploy (push) Successful in 58s
Deploy Trade-In / changes (push) Successful in 12s
Deploy Trade-In / build-browser (push) Has been skipped
Deploy Trade-In / build-frontend (push) Successful in 37s
Deploy Trade-In / test (push) Successful in 1m38s
Deploy Trade-In / build-backend (push) Successful in 50s
2026-07-02 18:41:12 +00:00
c0566705a1 Merge pull request 'fix(tradein/v2): keep top nav active over overlays, stronger «К оценке» button, backdrop-close' (#2193) from fix/tradein-v2-overlay-nav-backdrop into main
Some checks failed
Deploy Trade-In / deploy (push) Blocked by required conditions
Deploy Trade-In / test (push) Has been skipped
Deploy Trade-In / changes (push) Successful in 11s
Deploy Trade-In / build-browser (push) Has been skipped
Deploy Trade-In / build-backend (push) Has been skipped
Deploy Trade-In / build-frontend (push) Has been cancelled
2026-07-02 18:39:27 +00:00
bot-backend
03ece36abd fix(tradein/dedup): drop geo-guard in fias-pass — same-fias identity outranks proximity (#2187)
All checks were successful
CI / changes (pull_request) Successful in 10s
CI / backend-tests (pull_request) Has been skipped
CI / frontend-tests (pull_request) Has been skipped
CI / openapi-codegen-check (pull_request) Has been skipped
2026-07-02 21:35:20 +03:00
139bf16a8a fix(eias-heat): не терять резерв при одинаковых именах систем (#2119 B2) (#2191)
All checks were successful
Deploy / build-backend (push) Successful in 2m3s
Deploy / build-worker (push) Successful in 3m25s
Deploy / deploy (push) Successful in 1m40s
Deploy / changes (push) Successful in 9s
Deploy / build-frontend (push) Has been skipped
2026-07-02 18:34:42 +00:00
bot-backend
0eb033d77d fix(tradein/matching): P1 extended — never create a house from a numberless address (no cadastral)
All checks were successful
CI / changes (pull_request) Successful in 14s
CI / backend-tests (pull_request) Has been skipped
CI / openapi-codegen-check (pull_request) Has been skipped
CI / frontend-tests (pull_request) Has been skipped
2026-07-02 21:33:52 +03:00
a3db6e4158 feat(site-finder): §4.3 тренд из цен предложения Объектива при устаревших сделках (#2178) (#2190)
Some checks failed
Deploy / changes (push) Successful in 5s
Deploy / deploy (push) Blocked by required conditions
Deploy / build-worker (push) Has been cancelled
Deploy / build-frontend (push) Has been cancelled
Deploy / build-backend (push) Has been cancelled
2026-07-02 18:33:16 +00:00
bot-backend
35a0e29122 fix(tradein/v2): keep top nav active over overlays, stronger «К оценке», backdrop-close
All checks were successful
CI / changes (pull_request) Successful in 6s
CI / backend-tests (pull_request) Has been skipped
CI / frontend-tests (pull_request) Has been skipped
CI / openapi-codegen-check (pull_request) Has been skipped
Three UX fixes on the /trade-in/v2 overlays (feedback):

1. TopNav stays interactive while a section overlay is open. The a11y
   audit #2081 put inert on the whole content wrap (which contains
   TopNav), so opening an overlay disabled the top tabs. Move inert off
   the wrap onto <main>/<footer> only; the nav is inert just for the
   full-screen LocationDrawer. Users can now switch sections directly
   from an open overlay. role=dialog/aria-modal/Esc/focus-trap kept.

2. «← К ОЦЕНКЕ» made a clear filled primary button (solid accentDeep bg,
   white text ≥4.9:1 AA, shadow, hover lift) instead of the weak ghost
   pill. Uses existing accent tokens only.

3. Backdrop click closes the overlay. Add a transparent sibling layer
   below the panel (z below it, starting under the nav so tabs stay
   clickable); clicks in the margin close it. Clicks inside the panel /
   on its buttons never reach it (sibling, not ancestor) so they don't
   close. Esc + focus restore preserved.
2026-07-02 21:28:47 +03:00
3b9dfdb44e Merge pull request 'feat(tradein/frontend): pass suggest fias_id as target_fias_id in estimate request' (#2186) from feat/tradein-fias-estimate-frontend into main
All checks were successful
Deploy Trade-In / build-backend (push) Has been skipped
Deploy Trade-In / deploy (push) Successful in 53s
Deploy Trade-In / changes (push) Successful in 8s
Deploy Trade-In / test (push) Has been skipped
Deploy Trade-In / build-browser (push) Has been skipped
Deploy Trade-In / build-frontend (push) Successful in 1m58s
2026-07-02 18:08:27 +00:00
b2d794587f Merge pull request 'feat(tradein/estimator): кросс-source физдедуп аналогов за флагом (#2087 H4)' (#2173) from fix/tradein-analog-dedup-cv into main
All checks were successful
Deploy Trade-In / test (push) Successful in 1m29s
Deploy Trade-In / build-backend (push) Successful in 1m0s
Deploy Trade-In / changes (push) Successful in 7s
Deploy Trade-In / build-frontend (push) Has been skipped
Deploy Trade-In / build-browser (push) Has been skipped
Deploy Trade-In / deploy (push) Successful in 51s
2026-07-02 18:04:39 +00:00
1c520cadb7 Merge pull request 'fix(tradein/dedup): fias-key clustering + cross-fias merge guard + identity carry-over (#1772 follow-up)' (#2185) from fix/tradein-dedup-by-fias into main
All checks were successful
Deploy Trade-In / test (push) Successful in 1m27s
Deploy Trade-In / build-backend (push) Successful in 51s
Deploy Trade-In / changes (push) Successful in 9s
Deploy Trade-In / build-frontend (push) Has been skipped
Deploy Trade-In / build-browser (push) Has been skipped
Deploy Trade-In / deploy (push) Successful in 52s
2026-07-02 18:00:58 +00:00
bot-backend
333a36f049 feat(tradein/frontend): pass suggest fias_id as target_fias_id in estimate request
All checks were successful
CI / backend-tests (pull_request) Has been skipped
CI / frontend-tests (pull_request) Has been skipped
CI / openapi-codegen-check (pull_request) Has been skipped
CI / changes (pull_request) Successful in 7s
2026-07-02 20:56:50 +03:00
bot-backend
3505a4e3e5 feat(tradein/estimator): включить кросс-source физдедуп аналогов по умолчанию (#2087 H4, #2173)
All checks were successful
CI / frontend-tests (pull_request) Has been skipped
CI / changes (pull_request) Successful in 9s
CI / backend-tests (pull_request) Has been skipped
CI / openapi-codegen-check (pull_request) Has been skipped
estimate_dedup_analogs_enabled: False → True. Бэктест #1966 OFF vs ON
accuracy-идентичен (MAPE 13.89%, coverage 83.33%, bias −3.83%, median
width/cv без изменений) — включение меняет только user-visible n_analogs:
перестаёт быть раздутым кросс-постингом одного физлота на avito+cian+domklik
×3. Откат — ENV ESTIMATE_DEDUP_ANALOGS_ENABLED=false.

Тесты разведены default vs explicit (не ослаблены):
- test_estimator_dedup_cross_source_2087: добавлен test_dedup_default_is_on
  (без monkeypatch — дедуп активен по умолчанию); test_dedup_flag_off_is_noop
  оставляет ЯВНЫЙ OFF-override.
- test_backtest_regression_gate: пиним флаг OFF — гейт это байт-идентичный
  replay frozen OFF-capture (recorded call-sequence фикстуры без дедупа); с
  ON _dedup_cross_source триммит listings до quarter_indexes_lookup и
  control-flow расходится с записью.
- test_estimator_n_analogs_priced (autouse) + test_radius_path_n_analogs_unchanged:
  пиним OFF — инвариант «n_analogs = число priced-аналогов»/radius-passthrough
  ортогонален дедупу, а синтетические аналоги делят адрес/этаж/площадь и
  схлопнулись бы как кросс-посты.

Refs #2087, #2173
2026-07-02 20:56:39 +03:00
d87ebcc30e Merge pull request 'feat(tradein/matching): fias_exact tier + estimate fias persist + suggest fias_id contract' (#2184) from feat/tradein-fias-tier-matching into main
All checks were successful
Deploy Trade-In / changes (push) Successful in 11s
Deploy Trade-In / build-frontend (push) Has been skipped
Deploy Trade-In / build-browser (push) Has been skipped
Deploy Trade-In / test (push) Successful in 1m34s
Deploy Trade-In / build-backend (push) Successful in 54s
Deploy Trade-In / deploy (push) Successful in 55s
2026-07-02 17:56:13 +00:00
cdc1e03b32 fix(eias-heat): column-aligned парсинг листа ТП по реальной сетке (#2119 B2) (#2176)
All checks were successful
Deploy / deploy (push) Successful in 1m45s
Deploy / changes (push) Successful in 8s
Deploy / build-frontend (push) Has been skipped
Deploy / build-backend (push) Successful in 1m54s
Deploy / build-worker (push) Successful in 3m7s
2026-07-02 17:55:16 +00:00
bot-backend
c4b1992de5 fix(tradein/dedup): fias-key clustering + cross-fias merge guard + identity carry-over (#1772 follow-up)
All checks were successful
CI / changes (pull_request) Successful in 7s
CI / backend-tests (pull_request) Has been skipped
CI / frontend-tests (pull_request) Has been skipped
CI / openapi-codegen-check (pull_request) Has been skipped
2026-07-02 20:53:25 +03:00
b5f8f7b437 feat(report): §7 открыт по умолчанию + выделение участка на подложке + шире карта (#2179) (#2183)
Some checks failed
Deploy / deploy (push) Blocked by required conditions
Deploy / build-backend (push) Has been skipped
Deploy / changes (push) Successful in 6s
Deploy / build-worker (push) Has been skipped
Deploy / build-frontend (push) Has been cancelled
2026-07-02 17:52:21 +00:00
bot-backend
67a852caf1 feat(tradein/matching): fias_exact tier + estimate fias persist + suggest fias_id contract
All checks were successful
CI / backend-tests (pull_request) Has been skipped
CI / frontend-tests (pull_request) Has been skipped
CI / openapi-codegen-check (pull_request) Has been skipped
CI / changes (pull_request) Successful in 10s
2026-07-02 20:49:04 +03:00
509cecfa6a feat(scraper-kit): copy scheduler with registry-dispatch (25+ trigger_* → dict), strangler (#2136)
All checks were successful
Deploy Trade-In / changes (push) Successful in 9s
Deploy Trade-In / build-frontend (push) Has been skipped
Deploy Trade-In / build-browser (push) Has been skipped
Deploy Trade-In / test (push) Successful in 1m30s
Deploy Trade-In / build-backend (push) Successful in 1m21s
Deploy Trade-In / deploy (push) Successful in 59s
2026-07-02 17:44:03 +00:00
94114adca1 Merge pull request 'fix(tradein/estimator): не отравлять asking→sold ratio-кэш транзиентной ошибкой' (#2175) from fix/tradein-asking-sold-ratio-cache-poison into main
All checks were successful
Deploy Trade-In / build-backend (push) Successful in 53s
Deploy Trade-In / deploy (push) Successful in 50s
Deploy Trade-In / changes (push) Successful in 8s
Deploy Trade-In / build-frontend (push) Has been skipped
Deploy Trade-In / build-browser (push) Has been skipped
Deploy Trade-In / test (push) Successful in 1m26s
2026-07-02 17:37:12 +00:00
bot-backend
3909db745b fix(tradein/estimator): не отравлять asking→sold ratio-кэш транзиентной ошибкой
All checks were successful
CI / frontend-tests (pull_request) Has been skipped
CI / openapi-codegen-check (pull_request) Has been skipped
CI / changes (pull_request) Successful in 6s
CI / backend-tests (pull_request) Has been skipped
Блок «ОЖИДАЕМАЯ ЦЕНА СДЕЛКИ» (expected_sold, «с учётом торга») молча гас в «—» у
части оценок с полноценным headline (прод: high-confidence, 17 аналогов, цена
есть — expected_sold NULL). Причина: `_get_asking_sold_ratio` gracefully ловит
ЛЮБУЮ ошибку БД → (None, None), но запись в `_asking_sold_ratio_cache` шла
БЕЗУСЛОВНО (после try/except) → один транзиентный сбой (poisoned tx от
вышестоящего graceful-except, коннект-хиккап) отравлял кэш бакета `(None, None)`
на весь TTL (300с) и гасил expected_sold для ВСЕХ оценок этого rooms-бакета на
воркере до истечения TTL.

Фикс: ранний `return None, None` из except БЕЗ записи в кэш → следующая оценка
ретраит. Кэшируем только успешный lookup (ratio может быть None, если строки
реально нет — стабильный факт БД, безопасно кэшировать).

Прод-диагностика (30 дней, 304 оценки): 39 пустых expected_sold, из них 37 —
легитимно n_analogs=0 (нет аналогов), а 2 — этот баг (headline есть, ratio-кэш
отравлён). Таблица asking_to_sold_ratios полна по всем бакетам + global fallback
0.843 → при исправном lookup ratio всегда резолвится.

Юнит-тесты: транзиентная ошибка → None БЕЗ кэша; error→retry не залипает;
успех кэшируется.
2026-07-02 20:34:10 +03:00
abdb413a99 feat(scraper-kit): copy remaining sweep orchestrators (yandex/cian/domclick/full-load), strangler (#2135 F2)
All checks were successful
Deploy Trade-In / changes (push) Successful in 7s
Deploy Trade-In / deploy (push) Successful in 59s
Deploy Trade-In / build-frontend (push) Has been skipped
Deploy Trade-In / build-browser (push) Has been skipped
Deploy Trade-In / test (push) Successful in 1m27s
Deploy Trade-In / build-backend (push) Successful in 1m32s
2026-07-02 17:29:05 +00:00
15224da3b5 fix(eias-heat): ссылка TemplatePrinter из onclick + период «N кв. YYYY» (#2119 B2) (#2172)
All checks were successful
Deploy / build-backend (push) Successful in 1m48s
Deploy / build-worker (push) Successful in 2m50s
Deploy / deploy (push) Successful in 1m26s
Deploy / changes (push) Successful in 6s
Deploy / build-frontend (push) Has been skipped
2026-07-02 17:24:26 +00:00
bot-backend
3ad1550363 feat(tradein/estimator): кросс-source физдедуп аналогов за флагом (#2087 H4)
All checks were successful
CI / changes (pull_request) Successful in 7s
CI / backend-tests (pull_request) Has been skipped
CI / frontend-tests (pull_request) Has been skipped
CI / openapi-codegen-check (pull_request) Has been skipped
Один физический лот кросс-постится на avito+cian+domklik (разные source и
source_id) → existing radius-дедуп по (source, source_id) его НЕ ловит →
n_analogs И source_counts раздуты кросс-постами (аудит #2087: лот
80м²/265000₽/м² = N1+Домклик+Циан ×3; «14 аналогов» → ~6-7 уникальных).

_dedup_cross_source схлопывает дубли по физическому ключу (building_cadastral
| нормализованный address + floor + area-bucket round(m²) + price-bucket
round(₽/100k)) ДО подсчёта n_analogs/median/cv, оставляя свежайшего (scraped_at)
представителя. За флагом estimate_dedup_analogs_enabled (default OFF =
байт-идентично; регресс-гейт зелёный).

Бэктест #1966 (400 ДКП, full spine, OFF vs ON, тот же сэмпл): MAPE 13.89%→13.89%,
coverage 83.33%→83.33%, bias −3.83%→−3.83%, median width 0.743→0.743, median cv
0.0988→0.0988, avg n_analogs 27.64→27.57 (дедуп отработал 107× на 335 оценках).
Кросс-посты имеют identical price → нулевой вклад в дисперсию → cv/коридор НЕ
сужаются: это фикс ЧЕСТНОСТИ СЧЁТА (n_analogs не раздут, source_counts по
физлотам), accuracy-нейтральный, а НЕ рычаг cv-сужения (рычаг cv→коридор —
estimate_sb_clip_after_weight, уже default ON). Default OFF — narrowing не
продемонстрирован; флаг готов к canary/монкипатч-бэктесту.

Refs #2087
2026-07-02 20:22:46 +03:00
8a177df0e2 feat(scraper-kit): copy sweep pipeline orchestration (avito) with protocol injection, strangler (#2135)
All checks were successful
Deploy Trade-In / build-frontend (push) Has been skipped
Deploy Trade-In / test (push) Successful in 1m35s
Deploy Trade-In / changes (push) Successful in 8s
Deploy Trade-In / build-browser (push) Has been skipped
Deploy Trade-In / build-backend (push) Successful in 1m24s
Deploy Trade-In / deploy (push) Successful in 1m0s
2026-07-02 17:08:40 +00:00
f7063d49b5 fix(scraper-kit): sync RealScraperConfig adapter with ScraperConfig protocol (cookie_encryption_key)
All checks were successful
Deploy Trade-In / changes (push) Successful in 26s
Deploy Trade-In / build-browser (push) Has been skipped
Deploy Trade-In / build-frontend (push) Successful in 50s
Deploy Trade-In / build-backend (push) Successful in 1m29s
Deploy Trade-In / test (push) Successful in 1m56s
Deploy Trade-In / deploy (push) Successful in 2m45s
2026-07-02 16:46:05 +00:00
e501dedf78 feat(site-finder): ЕЭСК-резервы городских ПС/ТП + тепло из ФГИС ФАС (#2119 B2) (#2165)
All checks were successful
Deploy / build-frontend (push) Successful in 4m38s
Deploy / deploy (push) Successful in 1m37s
Deploy / build-backend (push) Successful in 2m48s
Deploy / build-worker (push) Successful in 3m59s
Deploy / changes (push) Successful in 9s
2026-07-02 16:44:57 +00:00
d7f4873a53 feat(proxy-pool): pool service (acquire/release/health) + healthcheck scheduler task (#2162)
Some checks failed
Deploy Trade-In / changes (push) Successful in 16s
Deploy Trade-In / build-browser (push) Has been skipped
Deploy Trade-In / test (push) Failing after 1m36s
Deploy Trade-In / build-backend (push) Has been skipped
Deploy Trade-In / deploy (push) Has been skipped
Deploy Trade-In / build-frontend (push) Successful in 33s
2026-07-02 16:41:39 +00:00
62fa1f732a refactor(scraper-kit): unify price bisection engine across providers + dedup (#2134)
Some checks failed
Deploy Trade-In / deploy (push) Blocked by required conditions
Deploy Trade-In / changes (push) Successful in 14s
Deploy Trade-In / build-browser (push) Has been skipped
Deploy Trade-In / build-backend (push) Blocked by required conditions
Deploy Trade-In / build-frontend (push) Has been cancelled
Deploy Trade-In / test (push) Has been cancelled
2026-07-02 16:41:04 +00:00
b8ed46d2dc Merge pull request 'fix(tradein/frontend): redirect legacy /trade-in root to canonical /v2' (#2166) from fix/tradein-canonical-v2-redirect into main
Some checks failed
Deploy Trade-In / changes (push) Successful in 11s
Deploy Trade-In / build-browser (push) Has been skipped
Deploy Trade-In / test (push) Failing after 1m37s
Deploy Trade-In / build-backend (push) Has been skipped
Deploy Trade-In / build-frontend (push) Successful in 2m29s
Deploy Trade-In / deploy (push) Has been skipped
2026-07-02 16:33:08 +00:00
90e4a741f7 feat(proxy-pool): scrape_proxies table + admin bulk-load/list endpoints (#2161)
Some checks failed
Deploy Trade-In / changes (push) Successful in 12s
Deploy Trade-In / build-browser (push) Has been skipped
Deploy Trade-In / build-frontend (push) Successful in 33s
Deploy Trade-In / test (push) Failing after 1m32s
Deploy Trade-In / build-backend (push) Has been skipped
Deploy Trade-In / deploy (push) Has been skipped
2026-07-02 16:31:21 +00:00
f937f5a168 fix(tradein/frontend): redirect legacy /trade-in root to canonical /v2
All checks were successful
CI / changes (pull_request) Successful in 9s
CI / backend-tests (pull_request) Has been skipped
CI / openapi-codegen-check (pull_request) Has been skipped
CI / frontend-tests (pull_request) Has been skipped
Оценщики клиента должны попадать на новую витрину /trade-in/v2 (wired на
real data), а не на легаси-корень (app/page.tsx, HeroSummary). Добавляю
декларативный redirects() в next.config: source "/" -> destination "/v2".

- permanent:false -> 307 (обратимо, не кэшируется навсегда)
- basePath авто-префиксит -> редирект /trade-in -> /trade-in/v2
- query-string Next пробрасывает автоматически (destination без своего
  query) -> /trade-in/?id=<uuid> -> /trade-in/v2?id=<uuid>, restore
  оценки по ?id= не ломается (v2 читает ?id= через window.location.search)
- легаси page.tsx / компоненты остаются в репо (только редирект, обратимо)
2026-07-02 19:26:55 +03:00
86a22dfc8f Merge pull request 'fix(tradein/v2): честная подача интервала оценки — одна цифра-герой + честный разброс' (#2159) from fix/tradein-v2-hero-honest-interval into main
Some checks failed
Deploy Trade-In / changes (push) Successful in 11s
Deploy Trade-In / build-browser (push) Has been skipped
Deploy Trade-In / test (push) Failing after 1m31s
Deploy Trade-In / build-frontend (push) Successful in 2m14s
Deploy Trade-In / build-backend (push) Has been skipped
Deploy Trade-In / deploy (push) Has been skipped
2026-07-02 16:20:06 +00:00
d63122d3e4 fix(tradein/v2): честная подача интервала оценки — одна цифра-герой + честный разброс
All checks were successful
CI / changes (pull_request) Successful in 8s
CI / backend-tests (pull_request) Has been skipped
CI / frontend-tests (pull_request) Has been skipped
CI / openapi-codegen-check (pull_request) Has been skipped
Оценщик жаловался на «большой интервал» между рекомендованной ценой и
оценкой. Бэктест: ширину коридора косметически не сузить (реальная рыночная
дисперсия + asking→sold gap). Чиним подачу, не число (математика не тронута).

Витрина /trade-in/v2 (активная, wired на real data через mapResultPanel):
- deltaLabel «К РЫНКУ» → «К ОБЪЯВЛ.» (mappers.ts + fixtures.ts): −N% —
  разрыв «объявление → ожидаемая сделка», а не «скидка от нас».
- ResultPanel: честная подпись под ценовыми карточками — «Диапазоны
  показывают разброс цен на рынке, а не погрешность оценки». Широкий
  диапазон не читается как неуверенность расчёта.
- Иерархия expected_sold=headline / median=baseline / ДКП=справочно уже
  была (audit H2, #2062/#2081) — не трогаем.

Легаси-витрина / (HeroSummary), тот же принцип и устранение конфликта термина:
- hero-duo: expected_sold — цифра-герой («Оценка · ожидаемая цена продажи»,
  34px, accent), median — вторичный «Рекомендованная цена в объявлении»
  (20px, приглушённый) вместо двух равнозначных 28px-блоков «Выставить»/
  «Получить». Новый модификатор .hero-duo--sold-primary.
- Диапазон sold подписан «рыночный разброс» + honest hint «Диапазон отражает
  разброс цен по рынку, а не погрешность оценки».
- Бейдж «−N%» → «−N% к объявлению» + title (разрыв asking↔сделка, не скидка).
- OfferCard: «Оценка по рынку» → «Цена в объявлении» (median ≠ «оценка»;
  слово «оценка» на витрине = expected_sold).

Проверка: tsc --noEmit чисто; next build чисто (lint+types, 13 роутов).
2026-07-02 19:16:37 +03:00
3be8062062 feat(scraper-kit): copy yandex providers with protocol injection, strangler (#2133 yandex)
Some checks failed
Deploy Trade-In / changes (push) Successful in 11s
Deploy Trade-In / test (push) Failing after 1m31s
Deploy Trade-In / build-frontend (push) Has been skipped
Deploy Trade-In / build-browser (push) Has been skipped
Deploy Trade-In / build-backend (push) Has been skipped
Deploy Trade-In / deploy (push) Has been skipped
2026-07-02 16:13:32 +00:00
19c0be5d5a feat(scraper-kit): copy cian providers with protocol injection, strangler (#2133 cian)
Some checks failed
Deploy Trade-In / test (push) Failing after 1m30s
Deploy Trade-In / build-backend (push) Has been skipped
Deploy Trade-In / deploy (push) Has been skipped
Deploy Trade-In / changes (push) Successful in 12s
Deploy Trade-In / build-browser (push) Has been skipped
Deploy Trade-In / build-frontend (push) Has been skipped
2026-07-02 16:11:31 +00:00
b7abaeb569 feat(scraper-kit): copy domclick provider with protocol injection, strangler (#2133 domclick)
Some checks failed
Deploy Trade-In / changes (push) Successful in 12s
Deploy Trade-In / build-frontend (push) Has been skipped
Deploy Trade-In / build-browser (push) Has been skipped
Deploy Trade-In / test (push) Successful in 1m28s
Deploy Trade-In / build-backend (push) Successful in 1m24s
Deploy Trade-In / deploy (push) Has been cancelled
2026-07-02 16:08:09 +00:00
df5e9003df feat(scraper-kit): copy avito providers with protocol injection, strangler (#2133 avito)
All checks were successful
Deploy Trade-In / changes (push) Successful in 11s
Deploy Trade-In / build-frontend (push) Has been skipped
Deploy Trade-In / build-browser (push) Has been skipped
Deploy Trade-In / test (push) Successful in 1m30s
Deploy Trade-In / build-backend (push) Successful in 1m47s
Deploy Trade-In / deploy (push) Successful in 1m1s
2026-07-02 15:58:00 +00:00
97acf5273c docs(claude): delegation.md — routing-таблица, effort-правила, экономика пачек (#2154) 2026-07-02 15:55:00 +00:00
5c17d21f6c feat(scraper-kit): copy core layer (base/save_listings/transport) with protocol injection, strangler (#2132)
All checks were successful
Deploy Trade-In / changes (push) Successful in 11s
Deploy Trade-In / build-frontend (push) Has been skipped
Deploy Trade-In / build-browser (push) Has been skipped
Deploy Trade-In / test (push) Successful in 1m27s
Deploy Trade-In / build-backend (push) Successful in 1m22s
Deploy Trade-In / deploy (push) Successful in 1m8s
2026-07-02 15:43:15 +00:00
9d71d94586 Merge pull request 'fix(tradein/estimator): честный asking_to_sold_ratio — бейдж «−N% к рынку» больше не врёт' (#2152) from fix/tradein-honest-asking-to-sold-ratio into main
All checks were successful
Deploy Trade-In / build-browser (push) Has been skipped
Deploy Trade-In / test (push) Successful in 1m28s
Deploy Trade-In / deploy (push) Successful in 52s
Deploy Trade-In / changes (push) Successful in 12s
Deploy Trade-In / build-frontend (push) Has been skipped
Deploy Trade-In / build-backend (push) Successful in 1m5s
2026-07-02 15:37:00 +00:00
e4a2d58c0c Merge pull request 'chore(claude): актуализация .claude-конфигов — политика 2026-06-27, delegation-лимиты, stale refs' (#2146) from chore/claude-config-refresh into main
Reviewed-on: #2146
2026-07-02 15:31:48 +00:00
ecc3ab5aab fix(tradein/estimator): честный asking_to_sold_ratio — бейдж «−N% к рынку» больше не врёт
All checks were successful
CI / changes (pull_request) Successful in 8s
CI / frontend-tests (pull_request) Has been skipped
CI / backend-tests (pull_request) Has been skipped
CI / openapi-codegen-check (pull_request) Has been skipped
Оценщик клиента жаловался на «большой интервал между рекомендованной ценой
и оценкой». Разбор: бейдж «−23% к рынку» (web HeroSummary + PDF, формула
round((1−ratio)×100)) систематически завышал скидку.

Root cause: сохранённый asking_to_sold_ratio — это СЫРОЙ per-rooms/tier дисконт
из ratio_resolver, но фактический expected_sold сдвинут относительно median×ratio
последующими корректировками: hedonic year+area (#2002, factor ∈ [0.75, 1.30], ON
by default), le_asking-clamp и corridor-clamp. Пример с прода (451de30b): median
7.75M × raw 0.771 = 5.97M, hedonic ×1.226 → expected_sold 7.32M — но stored ratio
остался 0.771, тогда как фактическое expected_sold/median = 0.945. Бейдж показывал
«−23%» вместо честных «−5%».

Fix: после финализации expected_sold пересчитываем сохранённый asking_to_sold_ratio
как реальное expected_sold_price/median_price (честный дескриптор). Сам expected_sold
(выкуп) НЕ трогаем — hedonic-uplift остаётся прибит к sale-модели, buyout не падает
до наивного median×raw. Порог _RATIO_DESCRIPTOR_EPS=1e-4 отсекает шум округления:
без сдвига (hedonic OFF, нет клампа) табличный ratio сохраняется байт-в-байт →
регрессия на не-зажатых оценках отсутствует.

Стор asking_to_sold_ratio — чисто ДЕСКРИПТОР (web/PDF/history badge), НЕ калибровочный
вход: калибровочный ratio живёт в таблице asking_to_sold_ratios (refresh-task, читает
resolver) — не тронута. Backtest #1966 скорит expected_sold_per_m2 (не stored ratio) —
не затронут (expected_sold без изменений).

Tests: 3 новых в test_estimator_price_spine.py (инвариант при hedonic-uplift +
corridor-clamp; byte-identical регрессия без сдвига); поправлен
test_global_fallback_basis_carried_through (hedonic OFF для сырого ratio).
Full suite: 2749 passed (кроме pre-existing test_search_cache_hit).

Refs #2141
2026-07-02 18:26:19 +03:00
6391587572 feat(scraper-kit): protocol boundaries (matcher/config/session/jobs) + backend adapters (#2131)
All checks were successful
Deploy Trade-In / changes (push) Successful in 11s
Deploy Trade-In / build-frontend (push) Has been skipped
Deploy Trade-In / build-browser (push) Has been skipped
Deploy Trade-In / test (push) Successful in 1m27s
Deploy Trade-In / build-backend (push) Successful in 1m26s
Deploy Trade-In / deploy (push) Successful in 1m0s
2026-07-02 15:25:38 +00:00
b7d35cfee0 chore(claude): security hardening — dangerous-commands hook, settings в git, restricted DB, throttle (#2150) 2026-07-02 15:23:11 +00:00
abef3e6e02 feat(market): revoke gendesign_reader direct SELECT on raw tables (#2138 step 3/3)
All checks were successful
Deploy Trade-In / changes (push) Successful in 13s
Deploy Trade-In / build-frontend (push) Has been skipped
Deploy Trade-In / build-browser (push) Has been skipped
Deploy Trade-In / test (push) Successful in 1m31s
Deploy Trade-In / build-backend (push) Successful in 30s
Deploy Trade-In / deploy (push) Successful in 54s
2026-07-02 15:11:55 +00:00
1c184959a8 feat(etl): read tradein data via market.v_* contract views (#2138 step 2/3)
All checks were successful
Deploy / changes (push) Successful in 8s
Deploy / build-frontend (push) Has been skipped
Deploy / build-backend (push) Successful in 2m15s
Deploy / build-worker (push) Successful in 3m12s
Deploy / deploy (push) Successful in 1m34s
2026-07-02 15:00:38 +00:00
03b3cb9fe4 feat(market): grant gendesign_reader SELECT on contract views (#2138 step 1/3)
All checks were successful
Deploy Trade-In / changes (push) Successful in 11s
Deploy Trade-In / build-frontend (push) Has been skipped
Deploy Trade-In / build-browser (push) Has been skipped
Deploy Trade-In / test (push) Successful in 1m30s
Deploy Trade-In / build-backend (push) Successful in 29s
Deploy Trade-In / deploy (push) Successful in 50s
2026-07-02 14:47:49 +00:00
0a6281b6e2 fix(site-finder): ГРС-парсер переписан под реальную структуру таблицы ГТЕ (#2145)
All checks were successful
Deploy / changes (push) Successful in 8s
Deploy / build-frontend (push) Has been skipped
Deploy / build-backend (push) Successful in 1m45s
Deploy / build-worker (push) Successful in 3m15s
Deploy / deploy (push) Successful in 1m30s
2026-07-02 14:45:42 +00:00
f02067c7af chore(claude): CLAUDE.md — уточнить always-load природу delegation.md (review nit)
All checks were successful
CI / changes (pull_request) Successful in 7s
CI / backend-tests (pull_request) Has been skipped
CI / frontend-tests (pull_request) Has been skipped
CI / openapi-codegen-check (pull_request) Has been skipped
2026-07-02 17:34:04 +03:00
227b939778 Merge pull request 'fix(tradein/v2): канонический sources_used — стабильный счётчик «ИСТОЧНИКОВ X/7» (#2087 M1)' (#2144) from fix/tradein-v2-source-counter-canonical into main
All checks were successful
Deploy Trade-In / changes (push) Successful in 12s
Deploy Trade-In / build-frontend (push) Has been skipped
Deploy Trade-In / build-browser (push) Has been skipped
Deploy Trade-In / test (push) Successful in 1m33s
Deploy Trade-In / build-backend (push) Successful in 52s
Deploy Trade-In / deploy (push) Successful in 50s
2026-07-02 14:31:15 +00:00
6965fd0484 chore(claude): git-pr.md — согласовать остаточные ссылки с parallel-default 2026-07-02 17:29:13 +03:00
79e4528eeb chore(claude): port delegation limits to repo rules, fix git-pr staleness
- new rules/delegation.md (always-loaded): subagent budget, oversized escalation,
  единые пороги дробления, parallel-default — портировано из memory 2026-06-27
- git-pr.md: parallel-default вместо sequential (supersedes), list_pull_reviews,
  фактические worktree settings, typos
- CLAUDE.md: merge-политика 2026-06-27, tradein/delegation в списке rules,
  auto-* vs Task-spawn различие
2026-07-02 17:28:05 +03:00
b29b69343b chore(claude): sync agent specs with 2026-06-27 policy, fix stale refs
- code-reviewer: self-merge policy 2026-06-27 vs ложный BLOCK; origin/gh → forgejo/MCP
- phase-5-verdict: self-extending guard в 'когда НЕ мержить'
- database-expert: prod DDL только через data/sql + deploy, не execute_sql
- auto-*: model-текст и интервалы тиков синхронизированы с фактом
- qa-tester: chrome-devtools → playwright; devops: Forgejo Actions
2026-07-02 17:25:27 +03:00
18b22c9df8 fix(tradein/v2): canonical sources_used — kill «ИСТОЧНИКОВ X/7» POST↔GET desync
All checks were successful
CI / changes (pull_request) Successful in 7s
CI / backend-tests (pull_request) Has been skipped
CI / frontend-tests (pull_request) Has been skipped
CI / openapi-codegen-check (pull_request) Has been skipped
Единый helper estimator._canonical_sources(analogs, valuation_flags) — источник
правды для sources_used, зовётся идентично на POST (estimate_quality) и
GET-rehydrate (get_estimate) из ТЕХ ЖЕ persisted analogs + оценочных флагов.

Root cause: три расходящиеся деривации — POST брал sources_used из top-N
analogs_lots, GET возвращал persisted-колонку (радиусный набор + quarter_index),
source_counts на GET считался из persisted analogs. Источник (напр. cian) мог
попасть в source_counts, но не в sources_used → счётчик «X/7» прыгал 4→5 между
POST-ответом и reload(GET).

- sources_used = {листинговые из persisted analogs} ∪ {avito_imv/yandex_valuation/
  cian_valuation}. Детерминированно отсортирован.
- source_counts на POST теперь тоже из analogs_lots (не полной metadata-выборки)
  → инвариант source_counts.keys() ⊆ sources_used на POST и GET.
- POST персистит канонический sources_used в колонку (history/PDF консистентны для
  новых строк); GET рехайдрейтит его же helper'ом — чинит и СТАРЫЕ строки
  (листинговая часть пересобирается из analogs, quarter_index/радиусный шум
  отбрасывается фильтром оценочных флагов).

Оценочные флаги персистятся в колонке sources_used и читаются оттуда на GET —
реконструкция не требуется.

Repro 451de30b: до — sources_used=[avito,avito_imv,domklik,yandex], source_counts
имеет cian (не в sources_used); после — sources_used=[avito,avito_imv,cian,
domklik,yandex] (5/7), counts.keys() ⊆ sources_used.

Part of #2087 (M1).
2026-07-02 17:22:20 +03:00
8d7feabf9f chore(scraper-kit): docker-context wiring + restore backend workspace dep (#2137)
All checks were successful
Deploy Trade-In / changes (push) Successful in 11s
Deploy Trade-In / build-frontend (push) Successful in 31s
Deploy Trade-In / build-browser (push) Successful in 30s
Deploy Trade-In / test (push) Successful in 1m29s
Deploy Trade-In / build-backend (push) Successful in 1m21s
Deploy Trade-In / deploy (push) Successful in 1m2s
2026-07-02 14:21:38 +00:00
ddf6cb7ce3 chore(scraper-kit): scaffold internal package + uv workspace, build-safe (#2128)
All checks were successful
Deploy Trade-In / build-frontend (push) Has been skipped
Deploy Trade-In / build-browser (push) Has been skipped
Deploy Trade-In / build-backend (push) Successful in 1m22s
Deploy Trade-In / deploy (push) Successful in 1m24s
Deploy Trade-In / changes (push) Successful in 11s
Deploy Trade-In / test (push) Successful in 1m31s
2026-07-02 14:06:07 +00:00
32d1818fd4 fix(scrapers/avito): unify _unix_to_date timezone across serp/houses/imv (#2129)
Some checks failed
Deploy Trade-In / build-frontend (push) Has been skipped
Deploy Trade-In / build-browser (push) Has been skipped
Deploy Trade-In / test (push) Successful in 1m32s
Deploy Trade-In / deploy (push) Has been cancelled
Deploy Trade-In / changes (push) Successful in 11s
Deploy Trade-In / build-backend (push) Successful in 59s
2026-07-02 14:03:11 +00:00
e2bb5061f9 feat(market): contract views market.v_* over producer tables (#2130)
Some checks failed
Deploy Trade-In / build-browser (push) Blocked by required conditions
Deploy Trade-In / test (push) Blocked by required conditions
Deploy Trade-In / build-backend (push) Blocked by required conditions
Deploy Trade-In / build-frontend (push) Blocked by required conditions
Deploy Trade-In / deploy (push) Blocked by required conditions
Deploy Trade-In / changes (push) Has been cancelled
2026-07-02 14:02:59 +00:00
5f661bd0b7 Merge pull request 'feat(tradein): #2043 BE-1 (cv/source_counts/created_at + history/cache-KPI) + #2044 BE-2 (radius_m query-params)' (#2140) from feat/tradein-v2-be1-be2-contract into main
Some checks failed
Deploy Trade-In / changes (push) Successful in 12s
Deploy Trade-In / build-frontend (push) Has been skipped
Deploy Trade-In / build-browser (push) Has been skipped
Deploy Trade-In / test (push) Successful in 1m38s
Deploy Trade-In / build-backend (push) Successful in 1m4s
Deploy Trade-In / deploy (push) Has been cancelled
2026-07-02 13:59:08 +00:00
3bf2c1a48a feat(tradein/api): #2044 BE-2 — optional radius_m on house-analytics + sell-time-sensitivity
All checks were successful
CI / changes (pull_request) Successful in 11s
CI / backend-tests (pull_request) Has been skipped
CI / frontend-tests (pull_request) Has been skipped
CI / openapi-codegen-check (pull_request) Has been skipped
radius_m (schema + estimate_quality comp-search) уже в main (e23dabe4). Здесь —
остаток: два derived-роута принимают radius_m как optional query-param.

- GET /estimate/{id}/house-analytics и /sell-time-sensitivity: += radius_m
  (int|None). None → байт-идентичная авто-логика (расширение до 300 м только при
  <8 in-house записей). Задан → явный радиус расширения выборки домов, клампится
  в [100,5000], возвращается в radius_m ответа.

Refs #2044
2026-07-02 16:51:11 +03:00
24b107ddb5 feat(tradein): #2043 BE-1 — surface cv/source_counts/created_at + /history.sources_used + cache-KPI
Данные уже считаются в estimator — отдаём наружу для /trade-in/v2 (снимает
approximate-флаги CV / счётчиков источников / даты отчёта / «ИСТОЧНИКОВ N/7»).

- AggregatedEstimate += cv (float|None), source_counts (dict[str,int]), created_at.
- estimator: _cv_from_ppm2 / _source_counts helpers. cv прокинут через
  PricingResult — anchor-путь берёт CV комплов (anchor["cv"]), radius-путь — CV
  радиусной ₽/м²-выборки. source_counts считается по ПОЛНОЙ выборке (metadata_lots)
  до top-N отсечки. created_at = момент создания.
- POST /estimate возвращает все три; GET /estimate/{id} пересчитывает cv/
  source_counts из сохранённых analogs (best-effort) + created_at из колонки.
- /history: += sources_used (jsonb) в проекции → колонка «ИСТОЧНИКОВ N/7».
- /cache-stats: += avg_median_price (по median_price>0) + repeat_address_pct
  (доля строк с неуникальным address). Честный best-effort по persisted-оценкам.

Refs #2043
2026-07-02 16:50:18 +03:00
102ef8392b fix(site-finder): ГРС-лоадер — HARICA-интермедиат в CA-бандл (неполная цепочка Газпрома) (#2125)
All checks were successful
Deploy / changes (push) Successful in 8s
Deploy / build-frontend (push) Has been skipped
Deploy / build-backend (push) Successful in 1m52s
Deploy / build-worker (push) Successful in 2m58s
Deploy / deploy (push) Successful in 1m43s
2026-07-02 13:49:14 +00:00
da0b8f0374 fix(site-finder): ГРС-лоадер — Russian Trusted CA для домена Газпрома (#2124)
All checks were successful
Deploy / changes (push) Successful in 8s
Deploy / build-backend (push) Successful in 1m42s
Deploy / build-worker (push) Successful in 3m3s
Deploy / build-frontend (push) Has been skipped
Deploy / deploy (push) Successful in 1m30s
2026-07-02 13:25:30 +00:00
5839680998 feat(site-finder): газ — свободная мощность ГРС + пометка ЕЭСК в §3 (#2119 Фаза B1) (#2123)
All checks were successful
Deploy / changes (push) Successful in 8s
Deploy / build-backend (push) Successful in 2m2s
Deploy / build-worker (push) Successful in 3m17s
Deploy / build-frontend (push) Successful in 3m57s
Deploy / deploy (push) Successful in 1m31s
2026-07-02 12:54:05 +00:00
2bb3ceb9de fix(site-finder): секции отчёта изначально раскрыты («Свернуть» для компакта) (#2122)
All checks were successful
Deploy / changes (push) Successful in 8s
Deploy / build-backend (push) Has been skipped
Deploy / build-worker (push) Has been skipped
Deploy / build-frontend (push) Successful in 3m20s
Deploy / deploy (push) Successful in 1m16s
2026-07-02 10:43:32 +00:00
e1a75d98d4 fix(site-finder): Россети-раскрытие — URL-энкоженные href, ссылки на xlsx не находились (#2121)
All checks were successful
Deploy / changes (push) Successful in 7s
Deploy / build-frontend (push) Has been skipped
Deploy / build-backend (push) Successful in 1m53s
Deploy / build-worker (push) Successful in 2m58s
Deploy / deploy (push) Successful in 1m33s
2026-07-02 09:36:14 +00:00
4758ea5e2b feat(site-finder): резервы мощности для ТП — электро+вода на карте §3 (#2119 Фаза A) (#2120)
All checks were successful
Deploy / changes (push) Successful in 8s
Deploy / build-backend (push) Successful in 2m16s
Deploy / build-worker (push) Successful in 3m27s
Deploy / build-frontend (push) Successful in 3m47s
Deploy / deploy (push) Successful in 1m30s
2026-07-02 08:36:51 +00:00
4a7ce91db1 fix(site-finder): §4.3 график «По кварталам» — бары вылезали за край карточки (#2118)
All checks were successful
Deploy / changes (push) Successful in 8s
Deploy / build-backend (push) Has been skipped
Deploy / build-worker (push) Has been skipped
Deploy / deploy (push) Successful in 1m12s
Deploy / build-frontend (push) Successful in 3m14s
2026-06-30 17:15:56 +00:00
babb2a3473 fix(site-finder): §4.3 — укомпановать 3 колонки в единообразные карточки (#2117)
All checks were successful
Deploy / changes (push) Successful in 8s
Deploy / build-backend (push) Has been skipped
Deploy / build-worker (push) Has been skipped
Deploy / build-frontend (push) Successful in 3m28s
Deploy / deploy (push) Successful in 1m15s
2026-06-30 16:51:52 +00:00
a2ec1fbc90 fix(site-finder): §4.3 подпись — убрать сырое имя поля planned_commissioning (#2115)
All checks were successful
Deploy / changes (push) Successful in 9s
Deploy / build-frontend (push) Has been skipped
Deploy / build-backend (push) Successful in 2m7s
Deploy / build-worker (push) Successful in 2m57s
Deploy / deploy (push) Successful in 1m38s
2026-06-30 15:54:58 +00:00
e52dc47067 feat(site-finder): прогрессивное раскрытие секций отчёта §1–§6 («Раскрыть всё») (#2116)
All checks were successful
Deploy / changes (push) Successful in 8s
Deploy / build-backend (push) Has been skipped
Deploy / build-worker (push) Has been skipped
Deploy / build-frontend (push) Successful in 3m34s
Deploy / deploy (push) Successful in 1m22s
2026-06-30 15:31:33 +00:00
663f04b777 fix(site-finder): попап конкурента — NUMERIC-поля приходят строкой (цена/площадь не рендерились) (#2114)
All checks were successful
Deploy / changes (push) Successful in 8s
Deploy / build-backend (push) Has been skipped
Deploy / build-worker (push) Has been skipped
Deploy / build-frontend (push) Successful in 3m58s
Deploy / deploy (push) Successful in 1m22s
2026-06-30 14:01:00 +00:00
02b5f0b5d1 feat(site-finder): max-info попапы — конкурент / §3-таблица / польз.точка / соседи (#2111) (#2113)
All checks were successful
Deploy / changes (push) Successful in 8s
Deploy / build-backend (push) Successful in 2m26s
Deploy / build-worker (push) Successful in 3m41s
Deploy / deploy (push) Successful in 1m30s
Deploy / build-frontend (push) Successful in 4m2s
2026-06-30 13:45:43 +00:00
6d6e0ce0ca feat(site-finder): §3 «Сети и точки подключения» — попапы показывают характеристики (#2111) (#2112)
All checks were successful
Deploy / changes (push) Successful in 8s
Deploy / build-backend (push) Successful in 2m2s
Deploy / build-worker (push) Successful in 3m29s
Deploy / build-frontend (push) Successful in 3m44s
Deploy / deploy (push) Successful in 1m30s
2026-06-30 13:16:07 +00:00
b024864518 feat(site-finder): §1 POI-точки карты показывают максимум информации (#2110)
All checks were successful
Deploy / changes (push) Successful in 8s
Deploy / build-backend (push) Successful in 2m3s
Deploy / build-worker (push) Successful in 3m15s
Deploy / build-frontend (push) Successful in 3m51s
Deploy / deploy (push) Successful in 1m27s
2026-06-30 12:42:13 +00:00
daf4ebefc2 feat(concept): 3D-масса корпусов на карте с тенями (§7) (#1953) (#2109)
All checks were successful
Deploy / deploy (push) Successful in 1m14s
Deploy / changes (push) Successful in 8s
Deploy / build-backend (push) Has been skipped
Deploy / build-worker (push) Has been skipped
Deploy / build-frontend (push) Successful in 3m35s
2026-06-30 11:58:47 +00:00
53cfd3c3b8 feat(networks): человеко-язык типов §3 + реальные имена НСПД-точек (#1953) (#2107)
All checks were successful
Deploy / changes (push) Successful in 7s
Deploy / build-backend (push) Successful in 1m55s
Deploy / build-worker (push) Successful in 3m4s
Deploy / build-frontend (push) Successful in 3m35s
Deploy / deploy (push) Successful in 1m31s
2026-06-30 11:34:29 +00:00
4ffda3299c fix(report): §4.3 выравнивание колонок (уплыло) + двойная точка в вердикте (#1953) (#2108)
All checks were successful
Deploy / changes (push) Successful in 7s
Deploy / build-backend (push) Has been skipped
Deploy / build-worker (push) Has been skipped
Deploy / build-frontend (push) Successful in 3m24s
Deploy / deploy (push) Successful in 1m15s
2026-06-30 11:29:33 +00:00
fe85cbb7ae fix(report): sticky-сайдбар навигации — overflow-x: clip вместо hidden (#958) (#2106)
All checks were successful
Deploy / changes (push) Successful in 8s
Deploy / build-backend (push) Has been skipped
Deploy / build-worker (push) Has been skipped
Deploy / build-frontend (push) Successful in 3m7s
Deploy / deploy (push) Successful in 1m15s
2026-06-30 11:05:54 +00:00
806428abc9 fix(report): instant nav jump + scroll-position scrollspy (#958) (#2105)
All checks were successful
Deploy / changes (push) Successful in 8s
Deploy / build-backend (push) Has been skipped
Deploy / build-worker (push) Has been skipped
Deploy / build-frontend (push) Successful in 3m23s
Deploy / deploy (push) Successful in 1m15s
2026-06-30 10:45:33 +00:00
02f49e795c feat(report): nav scrollspy + offset-jump, чат в плавающее окно справа (#958) (#2104)
All checks were successful
Deploy / changes (push) Successful in 7s
Deploy / build-backend (push) Has been skipped
Deploy / build-worker (push) Has been skipped
Deploy / build-frontend (push) Successful in 3m20s
Deploy / deploy (push) Successful in 1m16s
2026-06-30 10:29:02 +00:00
de4704f6cd fix(concept): step="any" — stepMismatch блокировал submit «Рассчитать концепцию» (#1953) (#2103)
All checks were successful
Deploy / changes (push) Successful in 8s
Deploy / build-backend (push) Has been skipped
Deploy / build-worker (push) Has been skipped
Deploy / build-frontend (push) Successful in 3m8s
Deploy / deploy (push) Successful in 1m16s
2026-06-30 10:09:55 +00:00
975c400536 feat(section7): editable section footprint + корпус plural/program hint (#1953) (#2102)
All checks were successful
Deploy / changes (push) Successful in 8s
Deploy / build-backend (push) Successful in 1m57s
Deploy / build-frontend (push) Successful in 3m34s
Deploy / build-worker (push) Successful in 3m41s
Deploy / deploy (push) Successful in 1m32s
2026-06-30 09:48:11 +00:00
e359fb4b95 feat(section7): unify «Концепция» into one engine driven by building_program (#1953) (#2101)
All checks were successful
Deploy / deploy (push) Successful in 1m14s
Deploy / changes (push) Successful in 7s
Deploy / build-backend (push) Has been skipped
Deploy / build-worker (push) Has been skipped
Deploy / build-frontend (push) Successful in 3m13s
2026-06-30 08:35:55 +00:00
5f84008cb3 fix(site-finder): §4.2 «Качество данных» — inline-стили (Tailwind flex/gap не применяется) (#1953) (#2100)
All checks were successful
Deploy / build-frontend (push) Successful in 3m28s
Deploy / changes (push) Successful in 9s
Deploy / build-backend (push) Has been skipped
Deploy / build-worker (push) Has been skipped
Deploy / deploy (push) Successful in 1m15s
2026-06-30 07:33:11 +00:00
005ce2f4ad fix(site-finder): §4.2 «Качество данных» — воздух вокруг разделителя (#1953) (#2099)
All checks were successful
Deploy / changes (push) Successful in 7s
Deploy / build-backend (push) Has been skipped
Deploy / build-worker (push) Has been skipped
Deploy / build-frontend (push) Successful in 3m22s
Deploy / deploy (push) Successful in 1m19s
2026-06-30 07:13:16 +00:00
f448b550b5 feat(site-finder): §7 «Концепция» — методики + синхронизация подписей (#1953, batch D) (#2098)
All checks were successful
Deploy / changes (push) Successful in 7s
Deploy / build-backend (push) Has been skipped
Deploy / build-worker (push) Has been skipped
Deploy / build-frontend (push) Successful in 3m30s
Deploy / deploy (push) Successful in 1m14s
2026-06-29 12:24:21 +00:00
d9915c7834 fix(site-finder): unify §3 «Сети» table↔map source + enrich popups (#1953, batch C) (#2097)
All checks were successful
Deploy / build-backend (push) Has been skipped
Deploy / build-frontend (push) Successful in 3m4s
Deploy / changes (push) Successful in 7s
Deploy / build-worker (push) Has been skipped
Deploy / deploy (push) Successful in 1m21s
2026-06-29 11:56:11 +00:00
1337fadb49 fix(forecast): §6 honesty layer — pegged deficit, non-residential caveat, segment/pipeline note (#1953) (#2096)
All checks were successful
Deploy / changes (push) Successful in 7s
Deploy / build-backend (push) Has been skipped
Deploy / build-worker (push) Has been skipped
Deploy / build-frontend (push) Successful in 3m21s
Deploy / deploy (push) Successful in 1m14s
2026-06-29 11:26:35 +00:00
86f0797499 fix(site-finder): report microcopy batch A — gate titles, breadcrumb area, §4.2/4.3 (#1953) (#2095)
All checks were successful
Deploy / changes (push) Successful in 7s
Deploy / build-backend (push) Has been skipped
Deploy / build-worker (push) Has been skipped
Deploy / build-frontend (push) Successful in 3m8s
Deploy / deploy (push) Successful in 1m19s
2026-06-29 08:29:34 +00:00
a35d42f1c7 fix(site-finder): invest-score advisory flag + velocity data-honesty + h1 a11y (#1926) (#2094)
All checks were successful
Deploy / changes (push) Successful in 7s
Deploy / build-backend (push) Successful in 5m22s
Deploy / build-frontend (push) Successful in 6m11s
Deploy / build-worker (push) Successful in 7m53s
Deploy / deploy (push) Successful in 2m23s
2026-06-29 06:42:50 +00:00
caffd9b28d Merge pull request 'fix(tradein/v2): убрать средний скаттер «ЦЕНА × СРОК ПРОДАЖИ» из результата' (#2093) from fix/tradein-v2-drop-result-scatter into main
All checks were successful
Deploy Trade-In / changes (push) Successful in 11s
Deploy Trade-In / test (push) Has been skipped
Deploy Trade-In / build-browser (push) Has been skipped
Deploy Trade-In / build-backend (push) Has been skipped
Deploy Trade-In / deploy (push) Successful in 50s
Deploy Trade-In / build-frontend (push) Successful in 1m58s
2026-06-28 23:21:11 +00:00
267e5dfc5b fix(tradein/v2): убрать средний скаттер «ЦЕНА × СРОК ПРОДАЖИ» из результата
All checks were successful
CI / changes (pull_request) Successful in 7s
CI / backend-tests (pull_request) Has been skipped
CI / frontend-tests (pull_request) Has been skipped
CI / openapi-codegen-check (pull_request) Has been skipped
По фидбэку: куцый inline-скаттер в 02 РЕЗУЛЬТАТ мало что говорил и дублировал
полный price×time график в Аналитике. Удалён; ряд диапазонов теперь grid 1fr 1fr —
два range-бара (объявления/сделки) на всю ширину, читаются прямее. Полный
точечный график остаётся в оверлее Аналитики (таб 06).

- ResultPanel.tsx: удалён scatter-блок (title + SVG + легенда + «Подробнее»),
  grid ряда «1fr 168px 1fr» -> «1fr 1fr»; убраны ставшие мёртвыми токены
  scatterDeal / line3.
- data.scatterMini в контракте оставлен (маппер всё ещё считает, просто не
  потребляется) — чистку контракта можно сделать отдельно.

tsc + next build green.
2026-06-29 02:20:45 +03:00
11a7018e78 Merge pull request 'feat(tradein/sale-share): тумблер «Сейчас / За 45 дней» + фильтр «мин. объявлений»' (#2092) from feat/tradein-sale-share-45d-toggle into main
All checks were successful
Deploy Trade-In / test (push) Has been skipped
Deploy Trade-In / build-backend (push) Has been skipped
Deploy Trade-In / changes (push) Successful in 10s
Deploy Trade-In / build-browser (push) Has been skipped
Deploy Trade-In / build-frontend (push) Successful in 1m52s
Deploy Trade-In / deploy (push) Successful in 53s
2026-06-28 23:13:14 +00:00
0ea850c957 feat(tradein/sale-share): тумблер «Сейчас / За 45 дней» + фильтр «мин. объявлений»
All checks were successful
CI / backend-tests (pull_request) Has been skipped
CI / frontend-tests (pull_request) Has been skipped
CI / openapi-codegen-check (pull_request) Has been skipped
CI / changes (pull_request) Successful in 7s
#3: переключатель окна метрики (now/45д) — меняет показываемый %
(sale_share_pct↔sale_share_pct_45d), счётчик (active_secondary↔listings_45d) и
сортировку (share_desc↔share_45d_desc, active_desc↔count_45d_desc). 45д-данные
уже были в API — фронт их не показывал (потому тумблер и не находился).
#2: поле «Мин. объявлений в доме» — client-side фильтр по оконному счётчику.

Frontend-only (бэк/API/deps не тронуты). typecheck clean.
NB: маркеры карты пока по now-метрике (мелкий follow-up).
2026-06-29 02:12:04 +03:00
4863e67f27 Merge pull request 'fix(tradein/v2): источники N1+Restate вон, Домклик в текст; disabled-палитра до AA (#2081 follow-up)' (#2091) from fix/tradein-v2-audit-round2b into main
All checks were successful
Deploy Trade-In / changes (push) Successful in 10s
Deploy Trade-In / build-backend (push) Has been skipped
Deploy Trade-In / test (push) Has been skipped
Deploy Trade-In / build-browser (push) Has been skipped
Deploy Trade-In / build-frontend (push) Successful in 1m48s
Deploy Trade-In / deploy (push) Successful in 50s
2026-06-28 23:03:48 +00:00
c362b8592a fix(tradein/v2): источники N1+Restate вон, Домклик в текст; disabled-палитра до AA (#2081)
All checks were successful
CI / changes (pull_request) Successful in 6s
CI / backend-tests (pull_request) Has been skipped
CI / frontend-tests (pull_request) Has been skipped
CI / openapi-codegen-check (pull_request) Has been skipped
- M9 correction: #2086 ошибочно убрал Домклик из «как считаем» (по stale-ссылке на
  #1968 «drop dead domklik»), но live-проверка под админом показала Домклик живым
  (реально даёт лоты). Возвращён в текст. N1.RU + RESTATE убраны из SOURCE_SLOTS
  (мёртвые — всегда «нет данных», не учитываем) → плитки и счётчик «N / 7» → «N / 5».
  TOTAL_SOURCES = SOURCE_SLOTS.length (единый источник истины: плитки/счётчик/текст
  не расходятся). Итог: Циан, Я.Недвижимость, Авито, Домклик (+ Росреестр сделки).
- C1 residual: disabledValue («—» no-data) #c2cdd9 1.4:1 -> #656b71 4.61:1;
  disabledLabel («нет данных») #aab7c5 1.7:1 -> #636b73 4.62:1. Аудит #2081 флагал
  «нет данных», #2086 дотемнил только muted-шкалу — disabled-палитру пропустил.

tsc + next build green. Найдено live-проверкой /trade-in/v2 под админом (реальная оценка).
2026-06-29 02:03:16 +03:00
69845ddcee fix(tradein/avito-detail): живой прогрев сессии + батч-на-сессию — снимает 403 на curl-backfill (#1551)
All checks were successful
Deploy Trade-In / changes (push) Successful in 12s
Deploy Trade-In / build-frontend (push) Has been skipped
Deploy Trade-In / build-browser (push) Has been skipped
Deploy Trade-In / test (push) Successful in 1m27s
Deploy Trade-In / build-backend (push) Successful in 51s
Deploy Trade-In / deploy (push) Successful in 1m37s
2026-06-28 22:34:38 +00:00
cbe56cab0e Merge pull request 'fix(tradein): cadastre ЖКХ-fallback + COALESCE merge-key + median-floors guard' (#2090) from feat/tradein-zhkh-cadastre-plus into main
Some checks failed
Deploy Trade-In / build-backend (push) Blocked by required conditions
Deploy Trade-In / deploy (push) Blocked by required conditions
Deploy Trade-In / changes (push) Successful in 10s
Deploy Trade-In / build-frontend (push) Has been skipped
Deploy Trade-In / build-browser (push) Has been skipped
Deploy Trade-In / test (push) Has been cancelled
2026-06-28 22:34:15 +00:00
078dadc440 fix(tradein): cadastre ЖКХ-fallback + COALESCE merge-key + median-floors guard
All checks were successful
CI / changes (pull_request) Successful in 7s
CI / backend-tests (pull_request) Has been skipped
CI / frontend-tests (pull_request) Has been skipped
CI / openapi-codegen-check (pull_request) Has been skipped
Серия sale-share data-quality (по находкам юзера):
- house_dedup_merge: cluster_key → tradein_canon_addr(COALESCE(short_address,
  full_address, address)) — ловит дубли с район-noise в address (Вайнера 66).
- zhkh_flats_loader: cadastre-fallback (searchByAddress по cadastreNumber,
  guid-независимо) — резолвит дома где gar_house_guid ≠ ЖКХ houseGuid
  (Переходный 6: guid→0, cadastre→165). CLI --cadastre.
- мигр.153: плаузибилити-гард по медиане этажности листингов — denom >=
  GREATEST(total_floors, median(listings.total_floors), 8); отсекает gar-недосчёт
  башен без ЖКХ (gar=9 vs листинги 19-26 эт → NULL).

ruff clean, pytest 36 green.
2026-06-29 01:32:48 +03:00
169201bb8c Merge pull request 'feat(tradein/sale-share): standalone «Поиск домов» — de-Mera + gendsgn.ru/sale-share' (#2088) from feat/tradein-sale-share-standalone into main
All checks were successful
Deploy / changes (push) Successful in 7s
Deploy Trade-In / changes (push) Successful in 12s
Deploy Trade-In / test (push) Has been skipped
Deploy Trade-In / build-browser (push) Has been skipped
Deploy Trade-In / build-backend (push) Has been skipped
Deploy / build-frontend (push) Successful in 33s
Deploy / build-backend (push) Successful in 36s
Deploy / build-worker (push) Successful in 35s
Deploy / deploy (push) Successful in 1m21s
Deploy Trade-In / build-frontend (push) Successful in 2m7s
Deploy Trade-In / deploy (push) Successful in 50s
2026-06-28 22:23:52 +00:00
bb54de6000 feat(tradein/sale-share): standalone «Поиск домов» — de-Mera + gendsgn.ru/sale-share
All checks were successful
CI / changes (pull_request) Successful in 7s
CI / frontend-tests (pull_request) Has been skipped
CI / backend-tests (pull_request) Has been skipped
CI / openapi-codegen-check (pull_request) Has been skipped
Поиск домов (доля квартир в продаже) вынесен в отдельный продукт, отвязан от
бренда «Мера» (Мера = оценка вторички). Только UI + 1 строка Caddy, бэк не тронут.

- app/sale-share/layout.tsx: своя metadata (title «Поиск домов — gendsgn»).
- components/trade-in/SaleShareHeader.tsx: своя шапка — продукт «Поиск домов»,
  без MeraMark и Mera-табов; UserMenu (auth) сохранён.
- app/sale-share/page.tsx: Topbar → SaleShareHeader; убраны «Мера» из
  breadcrumb/footer. Функционал/API (/trade-in/api/...) не тронут.
- Caddyfile: gendsgn.ru/sale-share → redir /trade-in/sale-share (короткий адрес;
  basePath=/trade-in → true vanity-URL требует отдельного app).
2026-06-29 01:09:26 +03:00
89fb666d88 Merge pull request 'fix(tradein/v2): UI/UX аудит round 2 — AA-контраст, честность чисел, a11y (#2081)' (#2086) from fix/tradein-v2-audit-round2 into main
All checks were successful
Deploy Trade-In / changes (push) Successful in 11s
Deploy Trade-In / test (push) Has been skipped
Deploy Trade-In / build-browser (push) Has been skipped
Deploy Trade-In / build-backend (push) Has been skipped
Deploy Trade-In / build-frontend (push) Successful in 1m55s
Deploy Trade-In / deploy (push) Successful in 52s
2026-06-28 22:03:38 +00:00
f5a7f9b437 fix(tradein/v2): UI/UX audit round 2 — AA contrast, honesty, a11y (#2081)
All checks were successful
CI / changes (pull_request) Successful in 8s
CI / backend-tests (pull_request) Has been skipped
CI / frontend-tests (pull_request) Has been skipped
CI / openapi-codegen-check (pull_request) Has been skipped
C1 (tokens): darken muted scale to WCAG AA (>=4.5:1 vs worst-case canvas
  #e6eef7); reclassify primary labels muted->body2 across components.
H1 (mappers): planning chip derived from rooms actually present in the
  analog set («Все планировки» when heterogeneous) — no number shifted.
H2 (ResultPanel): make «ОЖИДАЕМАЯ ЦЕНА СДЕЛКИ» the headline (accent
  border/tint/glow + larger value); demote ДКП·Росреестр to a muted
  СПРАВОЧНО reference tile. All three values + mapResultPanel wiring intact.
H3 (mappers): parseAddress keeps the house number for prefix-less
  addresses («Белинского, 86»); prefixed path unchanged.
M4 (page/HeroBar/Footer): gate dead controls (КАК РАССЧИТАНО / PDF /
  ДЕЙСТВИТЕЛЕН ДО / footer validity) on hasEstimate (mounted && estimate && !insufficient).
M5 (LocationDrawer): real dialog — role=dialog/aria-modal/aria-labelledby
  + focus-trap + <button> close + Esc preventDefault/stopPropagation.
M6 (ParamsPanel): address autocomplete -> combobox ARIA + Arrow/Home/End/
  Enter/Escape keyboard nav (mirrors Dd listbox).
M7 (Sources/Cache/Analytics/History): ARIA table semantics over the grids;
  Analytics scroll region keyboard-focusable (clears the serious
  scrollable-region-focusable axe violation); chart SVGs get role=img+title+aria-label.
M8 (CacheView): normalize the ВРЕМЯ column (normTime) to consistent minute precision.
M9 (LocationDrawer): honest «как считаем» source list (Циан, Я.Недвижимость,
  Авито, N1.ru + Росреестр) — drop dead Домклик/Restate per #1968.
M10 (ParamsPanel/mapMarkers): subject pin shows the subject's own area; drop
  analog pins overlapping the subject center (was misread as subject data).
M11 (ObjectSummary/SourcesView): label «медиана объявлений»; hide all-«—» columns/rows.
M13 (page): clamp artboard scale >=0.88 so micro-type doesn't drop below ~8px.
L1 (Analytics): suppress fake «X–Y дн» range when n<2.
L2 (Analytics): drop zero-data series from price-history subtitle.
L7 (TopNav): replace hardcoded #6f8195 icon strokes with tokens.muted.

tsc + next build green. No FE test infra in tradein-mvp/frontend — parseAddress
verified via standalone harness (6/6). Backend data-contract items M1/M2/M3/H4
(source counter, ratio vs medians, bargain bases, cross-source dedup) filed
separately — not FE bugs. L6 (blueprint-grid axe-blindness) deferred: base
gradient + glass surfaces keep axe contrast «incomplete» regardless, so the
authoritative check stays the manual computed-color sweep.
2026-06-29 01:01:30 +03:00
31e7816967 Merge pull request 'fix(tradein/dedup): canon cluster-key + geo/digit guard в house-merge (P4c)' (#2085) from fix/tradein-house-dedup-canon into main
All checks were successful
Deploy Trade-In / changes (push) Successful in 11s
Deploy Trade-In / deploy (push) Successful in 51s
Deploy Trade-In / build-frontend (push) Has been skipped
Deploy Trade-In / build-browser (push) Has been skipped
Deploy Trade-In / test (push) Successful in 1m27s
Deploy Trade-In / build-backend (push) Successful in 55s
2026-06-28 21:53:15 +00:00
7cf6267b48 fix(tradein/dedup): canon cluster-key + geo/digit guard в house-merge
All checks were successful
CI / changes (pull_request) Successful in 7s
CI / frontend-tests (pull_request) Has been skipped
CI / openapi-codegen-check (pull_request) Has been skipped
CI / backend-tests (pull_request) Has been skipped
merge_duplicate_houses кластеризовал дома по exact lower(trim(address)) →
варианты написания одного здания (ул. Вайнера,66 vs улица Вайнера, 66) не
схлопывались, расщепляя долю в sale-share. Ключ кластера → tradein_canon_addr
(ул→улица, стрип город/район, КОРПУС сохранён). Гарды против over-merge:
- гео ≤250м (canon стрипует город; прод-дубли Мраморская 34к4 в 222м; города
  региона-66 в км+ → 250м безопасно от cross-town);
- canon обязан содержать цифру (номер дома) — исключает вырожденные
  «екатеринбург»/«сооружение».
Re-pointing/audit/dry-run/идемпотентность не тронуты. 26 passed.
2026-06-29 00:50:57 +03:00
a96af4c19c Merge pull request 'fix(tradein/sale-share): гард по этажности — отсечь near-neighbor mis-match' (#2084) from fix/tradein-floors-guard into main
All checks were successful
Deploy Trade-In / build-frontend (push) Has been skipped
Deploy Trade-In / test (push) Successful in 1m26s
Deploy Trade-In / deploy (push) Successful in 51s
Deploy Trade-In / changes (push) Successful in 11s
Deploy Trade-In / build-browser (push) Has been skipped
Deploy Trade-In / build-backend (push) Successful in 53s
2026-06-28 21:32:23 +00:00
ebc3813648 fix(tradein/sale-share): гард по этажности — отсечь near-neighbor mis-match
All checks were successful
CI / changes (pull_request) Successful in 8s
CI / backend-tests (pull_request) Has been skipped
CI / frontend-tests (pull_request) Has been skipped
CI / openapi-codegen-check (pull_request) Has been skipped
Гео-фильтр ≤300м (мигр.150) широк для плотной застройки: к 5-этажному дому
(Мраморская 34к4) примагничены чужие листинги 9/16/19-этажных башен (Благодатская
15, Мраморская 6) на 229-282м — все <300м, фильтр не отсекал.

- мигр.152: +houses.zhkh_floors (parse zhkh_floors_raw "10-30"→max); гард во view
  во всех 5 FILTER: считать листинг только если abs(l.total_floors -
  COALESCE(zhkh_floors, total_floors)) <= 3 (или этажность неизвестна). Гео-фильтр
  300м сохранён.
- buildings_query.build_listings_query: тот же гард в детальной выдаче.

Прод-замер: отсекает 2754 листинга (12%) height-mismatched. Покрытие этажности
75% total_floors / 73% ЖКХ. ruff clean, pytest 26 green.
2026-06-29 00:31:09 +03:00
1b515f4e94 Merge pull request 'fix(tradein/sale-share): дедуп карточки листингов + чистка голых алиасов (#4 + P4a)' (#2083) from fix/tradein-clean-bare-aliases into main
All checks were successful
Deploy Trade-In / build-frontend (push) Has been skipped
Deploy Trade-In / build-browser (push) Has been skipped
Deploy Trade-In / test (push) Successful in 1m26s
Deploy Trade-In / changes (push) Successful in 10s
Deploy Trade-In / build-backend (push) Successful in 55s
Deploy Trade-In / deploy (push) Successful in 50s
2026-06-28 21:23:55 +00:00
b105e0a500 fix(tradein/sale-share): дедуп карточки листингов + чистка голых алиасов
All checks were successful
CI / changes (pull_request) Successful in 8s
CI / frontend-tests (pull_request) Has been skipped
CI / backend-tests (pull_request) Has been skipped
CI / openapi-codegen-check (pull_request) Has been skipped
#4 Карточка дома показывала cross-площадочные дубли (Искровцев 33: 2 квартиры
= 4 строки avito/domklik/cian). build_listings_query теперь дедуплит по сигнатуре
(rooms, round(area_m2), floor) — как v_building_sale_share мигр.148 — одна строка
на квартиру, представитель = самый дешёвый, +колонка sources[] (площадки).
Гео-фильтр ≤300м (мигр.150) сохранён. +BuildingListing.sources в схеме.

P4a Мигр.151: DELETE 6338/21720 голых алиасов house_address_aliases (без номера
дома) — сток, в который сваливались безномерные листинги (Tier-2b). Матчер уже
захардненен (P1 #2082), голые не регенерируют. Алиасы — регенерируемый кэш,
house_id_fk/house_sources не тронуты.

ruff clean, pytest test_buildings_api 26 green.
2026-06-29 00:22:20 +03:00
ec4c7e9e6a Merge pull request 'fix(tradein/matching): не сваливать листинги в чужой дом (P1-P3 хардненинг матчера)' (#2082) from fix/tradein-house-match-hardening into main
All checks were successful
Deploy Trade-In / changes (push) Successful in 11s
Deploy Trade-In / build-frontend (push) Has been skipped
Deploy Trade-In / build-browser (push) Has been skipped
Deploy Trade-In / test (push) Successful in 1m33s
Deploy Trade-In / build-backend (push) Successful in 56s
Deploy Trade-In / deploy (push) Successful in 1m9s
2026-06-28 21:07:39 +00:00
384c4e7ac3 fix(tradein/matching): не сваливать листинги в чужой дом (P1-P3)
All checks were successful
CI / changes (pull_request) Successful in 8s
CI / backend-tests (pull_request) Has been skipped
CI / frontend-tests (pull_request) Has been skipped
CI / openapi-codegen-check (pull_request) Has been skipped
Три гарда в match_or_create_house (+ P2 в match_house_readonly):
- P1: не матчить/алиасить голую улицу (normalized_address без номера дома)
- P2: отклонять Tier-3 гео-кандидата, если номера домов различаются
- P3: не цементировать слабый гео-матч (0.7) как alias-ключ здания

Хелперы has_house_number/house_number_token (по хвостовому токену —
«улица 8 марта» это улица, не дом). Корень mis-bucketing: house 131237
стянул 8 разных Мамина-номеров + голую улицу. Юнит + поведенческие тесты,
603 matching/normalize/house зелёных. Аффектит только НОВЫЕ матчи;
чистка cemented-данных — отдельной миграцией (P4).
2026-06-29 00:06:13 +03:00
bd3562b6bd Merge pull request 'fix(tradein/sale-share): гео-фильтр числителя ≤300м — отсечь mis-bucketed листинги' (#2080) from feat/tradein-sale-share-geofilter into main
Some checks failed
Deploy Trade-In / build-frontend (push) Has been skipped
Deploy Trade-In / build-backend (push) Successful in 55s
Deploy Trade-In / changes (push) Successful in 13s
Deploy Trade-In / build-browser (push) Has been skipped
Deploy Trade-In / test (push) Successful in 1m34s
Deploy Trade-In / deploy (push) Failing after 10m2s
2026-06-28 20:25:48 +00:00
f617c323bd fix(tradein/sale-share): гео-фильтр числителя ≤300м — отсечь mis-bucketed листинги
All checks were successful
CI / changes (pull_request) Successful in 9s
CI / backend-tests (pull_request) Has been skipped
CI / frontend-tests (pull_request) Has been skipped
CI / openapi-codegen-check (pull_request) Has been skipped
Листинги примагничиваются к НЕ ТОМУ дому (кривой house_id_fk): прод-замер 373
дома (5.5%) / 1685 листингов (7.2%) дальше 300м от своего дома. Пример — house
131237 «Мамина-Сибиряка 126» (ЖКХ=51 кв, знаменатель ВЕРНЫЙ): из 20 активных
листингов реально его ~2, остальные — Свердлова 32 / Азина 31 (до 2.2км) →
ложная доля 39% вместо ~4%.

- мигр.150: v_building_sale_share — числители (active/45д) + медианы считают
  только листинги ≤300м от geom дома (ST_DistanceSphere; листинг/дом без geom —
  keep). Знаменатель не тронут.
- buildings_query.build_listings_query: тот же гео-фильтр в детальной выдаче
  (карточка дома больше не показывает чужие листинги).

Симуляция на проде: меняется 351 дом, обнуляется 164 (чистые «мусорные вёдра»),
числитель −8.3%. Мамина-126: 20→2. Это митигация на уровне view; корневой фикс
house_id_fk-матчера — отдельно. ruff clean, pytest 24 green.
2026-06-28 23:24:05 +03:00
0f82b58265 Merge pull request 'fix(tradein/v2): a11y contrast — source badges ≥AA в оверлеях (#2062)' (#2079) from fix/tradein-v2-a11y-contrast into main
All checks were successful
Deploy Trade-In / changes (push) Successful in 10s
Deploy Trade-In / test (push) Has been skipped
Deploy Trade-In / build-backend (push) Has been skipped
Deploy Trade-In / build-browser (push) Has been skipped
Deploy Trade-In / build-frontend (push) Successful in 2m7s
Deploy Trade-In / deploy (push) Successful in 56s
2026-06-28 19:21:36 +00:00
848a44c629 fix(tradein/v2): a11y contrast — source badges ≥AA в оверлеях (#2062)
All checks were successful
CI / changes (pull_request) Successful in 7s
CI / backend-tests (pull_request) Has been skipped
CI / frontend-tests (pull_request) Has been skipped
CI / openapi-codegen-check (pull_request) Has been skipped
Прод-QA P3 нашёл 10 axe color-contrast (serious) в оверлее 05: 9px source-бейджи tokens.muted (#6f8195) на badgeTint ≈3,3:1 < AA. SourcesView badge muted→body2 (≈6,3:1); HistoryView ДКП source badge accent→accentDeep (≈5:1, синий сохранён). Закрывает хвост item-9 (микро-лейблы <4.5:1) → axe 0 и в оверлеях.
2026-06-28 22:21:10 +03:00
565d5c906b Merge pull request 'fix(tradein/v2): a11y P3b — клавиатурная форма 01 (#2062)' (#2078) from fix/tradein-v2-a11y-form into main
All checks were successful
Deploy Trade-In / changes (push) Successful in 11s
Deploy Trade-In / build-browser (push) Has been skipped
Deploy Trade-In / test (push) Successful in 1m34s
Deploy Trade-In / build-frontend (push) Successful in 2m19s
Deploy Trade-In / build-backend (push) Successful in 44s
Deploy Trade-In / deploy (push) Successful in 1m31s
2026-06-28 18:58:34 +00:00
32a58c91c5 fix(tradein/v2): a11y P3b — клавиатурная форма 01 (#2062)
All checks were successful
CI / changes (pull_request) Successful in 7s
CI / backend-tests (pull_request) Has been skipped
CI / frontend-tests (pull_request) Has been skipped
CI / openapi-codegen-check (pull_request) Has been skipped
Закрывает headline аудита: «с клавиатуры/скринридера нельзя завершить оценку».
Визуально без изменений — UA-chrome кнопок сброшен явно (Tailwind preflight в
tradein-mvp не подключён, кнопки не наследуют font-family).

- Dd-дропдаун (КОМНАТ/ТИП/РЕМОНТ) + РАДИУС: trigger div→<button role=combobox
  aria-haspopup=listbox aria-expanded aria-controls aria-activedescendant
  aria-label>; панель role=listbox, опции role=option/aria-selected + active-
  highlight. Клавиши Arrow↑↓/Home/End/Enter/Space/Esc/Tab (общий comboKeyDown,
  паттерн aria-activedescendant — фокус остаётся на кнопке).
- БАЛКОН → role=radiogroup + два <button role=radio aria-checked> (roving
  tabindex + стрелки + Space/Enter); tri-state (null = ни один не выбран) сохранён.
- Инпуты: <label htmlFor>/id; aria-invalid + aria-describedby на ошибках
  адреса/площади (id ошибки).
- Кнопки зума карты: div→<button aria-label>, «−» реально disabled на полу зума.

Мышиный поток идентичен прежнему; клавиатура аддитивна. code-reviewer  (UA-chrome
покрыт, keyboard/mouse корректны).
2026-06-28 21:58:03 +03:00
ec03a43283 Merge pull request 'feat(tradein/zhkh): ЖКХ-приоритет знаменателя «квартир в доме» + loader' (#2077) from feat/tradein-zhkh-denominator into main
Some checks failed
Deploy Trade-In / deploy (push) Blocked by required conditions
Deploy Trade-In / changes (push) Successful in 10s
Deploy Trade-In / build-frontend (push) Has been skipped
Deploy Trade-In / build-browser (push) Has been skipped
Deploy Trade-In / test (push) Successful in 1m27s
Deploy Trade-In / build-backend (push) Has been cancelled
2026-06-28 18:56:34 +00:00
77c01b6d3b feat(tradein/zhkh): ЖКХ-приоритет знаменателя «квартир в доме» + loader
All checks were successful
CI / changes (pull_request) Successful in 7s
CI / backend-tests (pull_request) Has been skipped
CI / frontend-tests (pull_request) Has been skipped
CI / openapi-codegen-check (pull_request) Has been skipped
ГАР дико недосчитывает квартиры в МКД (прод-замер: Крауля 89А ГАР=11 vs
ЖКХ=429; 27 из 347 домов ГАР ≥2× недосчёт). ГИС ЖКХ точнее — резолвит ~90%
вторички с gar_house_guid.

- мигр.149: COALESCE знаменателя → ЖКХ ПЕРВЫЙ
  (zhkh_flat_count, gar_flat_count, total_units, flat_count) во всех 7 местах
  view v_building_sale_share; +houses.zhkh_year/zhkh_floors_raw; +transparency-
  колонки zhkh_flat_count/flat_count_source.
- app/services/zhkh_flats_loader.py + app/tasks/zhkh_flats_load.py: фетч ГИС ЖКХ
  searchByAddress (батч ≤100 по gar_house_guid == ЖКХ houseGuid, pageIndex=1,
  cookieless), идемпотентный UPDATE houses.zhkh_* (CAST(:x AS type) +
  IS DISTINCT FROM gate, SAVEPOINT-чанки, dry-run).

Безопасно к мержу: до ручного прогона loader'а zhkh_flat_count=NULL → COALESCE
падает на ГАР → поведение идентично до-149. ruff clean, pytest 8 green.
2026-06-28 21:55:05 +03:00
f60f2b24e6 Merge pull request 'fix(tradein/v2): a11y P3a — нативные контролы + модальный диалог + aria-live (#2062)' (#2076) from fix/tradein-v2-a11y-native into main
All checks were successful
Deploy Trade-In / changes (push) Successful in 10s
Deploy Trade-In / test (push) Has been skipped
Deploy Trade-In / build-browser (push) Has been skipped
Deploy Trade-In / build-backend (push) Has been skipped
Deploy Trade-In / build-frontend (push) Successful in 1m51s
Deploy Trade-In / deploy (push) Successful in 53s
2026-06-28 18:30:09 +00:00
7066117cb8 fix(tradein/v2): a11y P3a — нативные контролы + модальный диалог + aria-live (#2062)
All checks were successful
CI / changes (pull_request) Successful in 6s
CI / backend-tests (pull_request) Has been skipped
CI / frontend-tests (pull_request) Has been skipped
CI / openapi-codegen-check (pull_request) Has been skipped
CRITICAL-доступность (часть 1): клавиатура/скринридер. Без визуальных изменений
(chrome-reset кнопок вынесен в CSS-классы, :hover и font-size сохранены).

- Клик-only <div> → нативные <button>: nav-табы (+aria-current), user-меню
  (aria-haspopup/expanded + Esc-close), «Выйти», «Подробнее →» (ResultPanel,
  +aria-label по разделу), строки 03 СВОДКИ (ObjectSummary, +aria-label).
- SectionOverlay → настоящая модалка: role=dialog, aria-modal, aria-labelledby,
  Esc-close, фокус на диалог при открытии, focus-trap (Tab/Shift+Tab),
  возврат фокуса на триггер при закрытии; «← К ОЦЕНКЕ» → <button>.
- page.tsx: aria-live="polite" role=status — «Идёт расчёт… / Оценка готова /
  Недостаточно данных / Ошибка»; фон (content-wrap) inert когда открыт
  оверлей или drawer (overlay/drawer остаются интерактивны).

Остаётся P3b: ParamsPanel-форма (Dd→listbox+клавиши, БАЛКОН→radiogroup,
input <label>/aria) — заход «с клавиатуры можно завершить оценку».
2026-06-28 21:29:19 +03:00
0641183ecd fix(tradein): дедуп объявлений по квартире + over_100 NULL-фикс (500 на списке) (148) (#2075)
All checks were successful
Deploy Trade-In / changes (push) Successful in 10s
Deploy Trade-In / build-frontend (push) Has been skipped
Deploy Trade-In / build-browser (push) Has been skipped
Deploy Trade-In / test (push) Successful in 1m29s
Deploy Trade-In / build-backend (push) Successful in 58s
Deploy Trade-In / deploy (push) Successful in 1m15s
2026-06-28 18:04:32 +00:00
efa2dc75ea Merge pull request 'feat(tradein/scheduler): cooperative drain coverage for sweeps/full-loads (phase 3a)' (#2074) from feat/tradein-sweep-drain-coverage into main
Some checks failed
Deploy Trade-In / build-backend (push) Blocked by required conditions
Deploy Trade-In / deploy (push) Blocked by required conditions
Deploy Trade-In / changes (push) Successful in 10s
Deploy Trade-In / build-frontend (push) Has been skipped
Deploy Trade-In / build-browser (push) Has been skipped
Deploy Trade-In / test (push) Has been cancelled
2026-06-28 18:04:02 +00:00
b45a819cb6 fix(tradein): канон срезает гео-префиксы (Россия/Екатеринбург/мкр/пр-т) + REINDEX — покрытие 38%→~70% (147) (#2073)
All checks were successful
Deploy Trade-In / changes (push) Successful in 14s
Deploy Trade-In / build-browser (push) Has been skipped
Deploy Trade-In / build-frontend (push) Successful in 31s
Deploy Trade-In / test (push) Successful in 1m33s
Deploy Trade-In / build-backend (push) Successful in 27s
Deploy Trade-In / deploy (push) Successful in 53s
2026-06-28 17:54:26 +00:00
3320805723 feat(tradein/scraper): cooperative SIGTERM-drain coverage for sweep/full-load loops (#1182 Phase 3a)
All checks were successful
CI / changes (pull_request) Successful in 7s
CI / frontend-tests (pull_request) Has been skipped
CI / backend-tests (pull_request) Has been skipped
CI / openapi-codegen-check (pull_request) Has been skipped
Extend the cooperative shutdown-drain (Phase 2) to the long sweep and
full-load loops in scrape_pipeline.py so they stop CLEANLY on SIGTERM
instead of being hard-cancelled at the 80s _drain_inflight timeout.

At every between-unit checkpoint that already polls scrape_runs.is_cancelled
(user-cancel) we add an ALONGSIDE shutdown_requested() branch:
  - anchor-loop sweeps (avito/yandex/cian city) + avito newbuilding +
    domclick pre-SERP: update_heartbeat + mark_done(partial) + return;
  - full-load on_bucket callbacks (cian/yandex/avito): raise a distinct
    RuntimeError("shutdown") sentinel, handled next to the existing
    "cancelled" sentinel -> mark_done(partial), skipping the detail phase;
  - cian full-load detail loop: break -> existing post-loop mark_done.

Drain finalizes as mark_done(partial), NOT mark_cancelled (that stays
user-cancel only). is_cancelled -> user-cancel semantics are unchanged, and
behaviour is identical when shutdown_requested() is False. No resume_cursor
or paused status yet (Phase 3b, deferred): this is drain-coverage only.

Tests: tests/test_sweep_drain.py covers both structural variants -
run_avito_city_sweep / run_cian_city_sweep (anchor-loop) and
run_avito_full_load (on_bucket sentinel): shutdown flips True after the
first unit, asserting the sweep stops at the checkpoint, calls mark_done
with partial counters, never calls mark_cancelled, and does not process
later anchors/buckets/phases.
2026-06-28 20:52:30 +03:00
eab3d928d6 Merge pull request 'fix(tradein/v2): UI/UX-аудит P2 — контент/честность/иерархия (#2062)' (#2072) from fix/tradein-v2-uiux-p2-content into main
Some checks failed
Deploy Trade-In / deploy (push) Blocked by required conditions
Deploy Trade-In / changes (push) Successful in 10s
Deploy Trade-In / build-browser (push) Has been skipped
Deploy Trade-In / test (push) Successful in 1m28s
Deploy Trade-In / build-backend (push) Has been cancelled
Deploy Trade-In / build-frontend (push) Has been cancelled
2026-06-28 17:52:28 +00:00
1722309372 fix(tradein/v2): UI/UX-аудит P2 — контент/честность/иерархия (#2062)
All checks were successful
CI / changes (pull_request) Successful in 7s
CI / backend-tests (pull_request) Has been skipped
CI / frontend-tests (pull_request) Has been skipped
CI / openapi-codegen-check (pull_request) Has been skipped
- Опц. поля больше не шлют непроверенные дефолты в ценовую модель:
  ТИП/РЕМОНТ → «Не указано»→undefined, БАЛКОН tri-state (null пока не
  трогали) → undefined.
- Валидация у поля: красная рамка + сообщение под адресом/площадью,
  показываются ВСЕ блокеры (не только первый); низ — только серверная ошибка.
- Копирайт «вы»: «если знаете»; кэш-строка → «ДАННЫЕ АКТУАЛЬНЫ ·
  ДЕЙСТВИТЕЛЕН 24 Ч»; drawer без жаргона (убраны (asking)/POI).
- Даты сделок Росреестра → «янв 2026» (месяц, не фейк-день 01.01);
  бренды источников без method-суффиксов (РОСРЕЕСТР/АВИТО/ДОМКЛИК);
  «эталон»→«опорная цена».
- График истории цен: зум оси Y к реальному диапазону данных + динамические
  подписи оси; легенда серии показывается только при наличии линии.
- 3 цены весомее (font-weight 400); медиана-маркеры range-баров → тонкий тик
  (не выглядит как draggable-ручка); фото здания — синий duotone +
  десатурация (не крадёт первый взгляд у цен).
- Таблицы аналогов/сделок: площадь/комнаты/этаж вынесены из адресной ячейки
  в колонку ПАРАМЕТРЫ; ↗→› в СВОДКЕ.

Сознательно НЕ тронуто: перераспределение accent-синего (item 9) — axe уже 0,
а широкое снятие акцента конфликтует с «не ломать HUD-палитру»; монотонность
тиров срока продажи не форсируется (это были бы выдуманные данные).
2026-06-28 20:51:49 +03:00
2a38561175 feat(tradein): второй % «за 45 дней» + zhkh-знаменатель slot (146) (#2071)
Some checks failed
Deploy Trade-In / build-backend (push) Blocked by required conditions
Deploy Trade-In / deploy (push) Blocked by required conditions
Deploy Trade-In / changes (push) Successful in 11s
Deploy Trade-In / build-frontend (push) Has been skipped
Deploy Trade-In / build-browser (push) Has been skipped
Deploy Trade-In / test (push) Has been cancelled
2026-06-28 17:51:38 +00:00
c8e7cb1c79 perf(objective_lots): inline DISTINCT ON for 3 view-fullscan consumers + cache landing stats (#1953) (#2067)
All checks were successful
Deploy / build-frontend (push) Has been skipped
Deploy / changes (push) Successful in 7s
Deploy / build-backend (push) Successful in 2m8s
Deploy / build-worker (push) Successful in 3m16s
Deploy / deploy (push) Successful in 5m41s
2026-06-28 17:25:41 +00:00
03ca46fe1d Merge pull request 'feat(tradein/scheduler): cooperative SIGTERM-drain — in-flight scrape unit commits before exit (phase 2)' (#2070) from feat/tradein-scheduler-graceful-drain into main
All checks were successful
Deploy Trade-In / changes (push) Successful in 10s
Deploy Trade-In / build-frontend (push) Has been skipped
Deploy Trade-In / build-browser (push) Has been skipped
Deploy Trade-In / test (push) Successful in 1m27s
Deploy Trade-In / build-backend (push) Successful in 1m38s
Deploy Trade-In / deploy (push) Successful in 51s
2026-06-28 17:20:11 +00:00
566b2f9617 fix(tradein/scraper): drain detached run-children on SIGTERM, not just coordinator (#1182 Phase 2)
All checks were successful
CI / changes (pull_request) Successful in 6s
CI / backend-tests (pull_request) Has been skipped
CI / frontend-tests (pull_request) Has been skipped
CI / openapi-codegen-check (pull_request) Has been skipped
Pre-push review: _await_scheduler ждал только scheduler_loop COORDINATOR, но вся
scrape-работа крутится в detached asyncio.create_task детях (каждый trigger_* делал
`task = create_task(_run())` без join). На SIGTERM coordinator выходил из while True
и завершался → asyncio.run() teardown хард-кансельил ещё бегущих детей mid-await =
ровно #1182 failure mode. Кооперативный checkpoint спасал ребёнка лишь когда его
residual-время случайно перекрывало drain — вероятностно, не гарантированно.

Fix — coordinator теперь дренажит детей перед выходом:
- _spawn_tracked(coro): централизованный detached-spawn, кладёт задачу в module-level
  registry _inflight_tasks (strong-ref = RUF006 keep-alive) + done-callback ретривит
  exception и убирает из set'а. Заменил 26 одинаковых
  `task = create_task(_run()); task.add_done_callback(...)` сайтов.
- _drain_inflight(): один asyncio.wait по живым детям с бюджетом
  _CHILD_DRAIN_TIMEOUT_S=80s (< scheduler_main 100s < docker grace 120s). Кооперативные
  дети (avito_detail_backfill, rosreestr-executor) дочекивают карточку/батч + mark_done
  и резолвятся; некооперативные упираются в timeout и падают на внешний hard-cancel.
- scheduler_loop по выходу из tick-loop (только по SIGTERM) зовёт await _drain_inflight().

NB: raw asyncio.all_tasks()-minus-self здесь НЕЛЬЗЯ — в нашей топологии он захватывает
_run parent-task (блокирован на wait_for(coordinator)) и shutdown_waiter → циклическое
ожидание coordinator↔_run, всегда упирающееся в timeout. Точный registry это исключает.

Tests: tests/test_scheduler.py — detached cooperative child drained-not-cancelled,
non-cooperative child timeout→left for hard-cancel, no-op без детей, scheduler_loop→drain
wiring. Обновил 3 source-inspection теста под новый _spawn_tracked паттерн.
2026-06-28 20:13:53 +03:00
e76030588c Merge pull request 'fix(tradein/v2): a11y follow-up — «скоро» pill contrast + drawer aria-hidden (#2062)' (#2069) from fix/tradein-v2-uiux-a11y into main
All checks were successful
Deploy Trade-In / changes (push) Successful in 14s
Deploy Trade-In / test (push) Has been skipped
Deploy Trade-In / build-browser (push) Has been skipped
Deploy Trade-In / build-backend (push) Has been skipped
Deploy Trade-In / build-frontend (push) Successful in 2m3s
Deploy Trade-In / deploy (push) Successful in 1m10s
2026-06-28 17:12:13 +00:00
041589f2fc fix(tradein/v2): a11y — «скоро» pill AA contrast + drawer aria-hidden (#2062)
All checks were successful
CI / changes (pull_request) Successful in 11s
CI / backend-tests (pull_request) Has been skipped
CI / frontend-tests (pull_request) Has been skipped
CI / openapi-codegen-check (pull_request) Has been skipped
axe-фоллоуап к #2066: «скоро» (КОЭФ.ЛОКАЦИИ) muted2→body2 (2,3→6,3:1 на badgeTint); LocationDrawer aria-hidden когда закрыт (off-canvas вне лендмарка → снимает axe «region»).
2026-06-28 20:11:43 +03:00
e148e61d49 Merge pull request 'feat(sitefinder): rosreestr cadastral-value loader + 0.7x denoise of quarter price index' (#2068) from feat/rosreestr-cadastral-value-denoise into main
All checks were successful
Deploy / changes (push) Successful in 8s
Deploy / build-frontend (push) Has been skipped
Deploy / build-backend (push) Successful in 36s
Deploy / build-worker (push) Successful in 36s
Deploy / deploy (push) Successful in 1m44s
2026-06-28 17:02:12 +00:00
6f76b565e5 fix(tradein/scraper): surface scheduler_loop crash in _await_scheduler done-branch
В ветке `task.done()` без shutdown глушилось реальное исключение задачи. Зовём
task.result() → пробрасываем настоящий сбой scheduler_loop наружу (loud crash,
как делал прежний `await task`), вместо тихого "exited cleanly".
2026-06-28 19:53:08 +03:00
848e6d3bdd feat(tradein/scraper): cooperative SIGTERM-drain for graceful deploy (#1182 Phase 2)
Деплой (docker recreate tradein-scraper) шлёт SIGTERM с stop_grace_period=120s
(Phase 1). Раньше scheduler_main отвечал hard task.cancel() → бегущий scrape-unit
получал CancelledError посреди браузер-карточки → карточка гибла без COMMIT.

Теперь SIGTERM/SIGINT лишь выставляют кооперативный флаг; tick-loop и длинные
задачи опрашивают его на СВОИХ существующих between-unit checkpoint'ах,
докоммичивают текущий unit и выходят сами.

- app/core/shutdown.py: новый standalone-модуль (asyncio.Event + request_shutdown
  / shutdown_requested / wait_for_shutdown), без app-зависимостей → нет циклов.
- scheduler_main._run: SIGTERM → request_shutdown() вместо task.cancel(); ждём
  добровольного drain'а, safety-net wait_for(100s < docker grace) с fallback на
  hard-cancel для некооперирующей задачи.
- scheduler_loop: проверка флага в начале тика и после каждого dispatch — не
  reap/claim новые run'ы во время shutdown, выходим из loop'а.
- avito_detail_backfill: break на границе карточки рядом с budget-guard; snapshot
  pending-query идемпотентен → следующий run сам резюмит остаток.
- import_rosreestr_dkp: расширен is_cancelled checkpoint — drain делает mark_done
  (partial), не mark_cancelled; user-cancel семантика без изменений.

Без SIGTERM поведение идентично прежнему. reap_zombies не трогает drained-run'ы
(они mark_done, не 'running').

Tests: tests/core/test_shutdown.py, scheduler_main drain+timeout-fallback,
avito_detail_backfill partial-drain. 26 + 29 scheduler passed.
2026-06-28 19:52:09 +03:00
4d063d6827 feat(sitefinder): rosreestr cadastral-value loader + 0.7x denoise of quarter price index
All checks were successful
CI / changes (pull_request) Successful in 8s
CI / frontend-tests (pull_request) Has been skipped
CI / openapi-codegen-check (pull_request) Successful in 1m58s
CI / backend-tests (pull_request) Successful in 14m30s
Quarterly Rosreestr dataset_КАДАСТРСТОИМОСТЬ (cadastral value) gives a
per-cad-quarter cadastral ₽/m². Registered deals priced below 0.7x that
quarter's cadastral median are dropped from mv_quarter_price_per_m2 as
noise (data-entry junk / non-arm's-length deals that drag medians down).

- 178_schema_cadastral_value.sql: rosreestr_cadastral_value (+ staging),
  region-66-scoped, idempotent.
- 178b_load_cadastral_value.py: psycopg v3 staging COPY -> typed region-66
  INSERT, idempotent skip-if-loaded. Run MANUALLY by operator after deploy.
- 179_mv_quarter_price_cadastral_floor.sql: recreates BOTH MVs (CASCADE
  drops the dependent index MV). Adds cad_floor CTE + LEFT JOIN guard to
  mv_quarter_price_per_m2; mv_quarter_price_index recreated verbatim.

Safe to auto-apply on prod before any cadastral data exists: empty
rosreestr_cadastral_value -> cad_floor has 0 rows -> floor_ppm2 IS NULL for
every quarter -> predicate keeps ALL deals -> MV byte-identical to the
pre-floor (95) definition. Proven by a rolled-back dry-run (66:% EXCEPT
both directions = 0). With the experiment cad source loaded the floor
moves 480 of 1972 quarters (median 66788 -> 67350).

NN bumped from spec's 172/173 to 178/178b/179 (172-177 already on main).
2026-06-28 19:45:35 +03:00
77b4f7e64c Merge pull request 'chore(tradein/deploy): scraper deploy-decouple phase 0+1 — no recreate on infra-only deploys + 120s graceful stop' (#2065) from chore/tradein-scraper-deploy-decouple-p01 into main
All checks were successful
Deploy Trade-In / changes (push) Successful in 14s
Deploy Trade-In / build-browser (push) Successful in 36s
Deploy Trade-In / test (push) Successful in 1m39s
Deploy Trade-In / build-backend (push) Successful in 30s
Deploy Trade-In / build-frontend (push) Successful in 4m50s
Deploy Trade-In / deploy (push) Successful in 3m10s
Reviewed-on: #2065
2026-06-28 16:35:38 +00:00
b8fd7771b8 Merge pull request 'fix(tradein/v2): UI/UX-аудит P1 quick-wins (#2062)' (#2066) from fix/tradein-v2-uiux-p1 into main
Some checks failed
Deploy Trade-In / deploy (push) Blocked by required conditions
Deploy Trade-In / changes (push) Successful in 10s
Deploy Trade-In / test (push) Has been skipped
Deploy Trade-In / build-browser (push) Has been skipped
Deploy Trade-In / build-backend (push) Has been skipped
Deploy Trade-In / build-frontend (push) Has been cancelled
2026-06-28 16:34:41 +00:00
30e158ad2a fix(tradein/v2): UI/UX-аудит P1 quick-wins (#2062)
All checks were successful
CI / changes (pull_request) Successful in 8s
CI / backend-tests (pull_request) Has been skipped
CI / frontend-tests (pull_request) Has been skipped
CI / openapi-codegen-check (pull_request) Has been skipped
Честная полировка живого /trade-in/v2 без правок архитектуры (FE-only).

- Провенанс из data, без хардкодов: footer «ПОСТРОЕНО ПО N АНАЛОГАМ И M
  СДЕЛКАМ» из meta.builtOn; единый счётчик источников (activeSourceCount =
  залитые плитки) в шапке 02, СВОДКЕ и оверлее; убраны «6/10», «2/7»,
  «Avito·10», «из 2 источников».
- Единый ru-форматтер: запятая везде (CV/площадь/%), NBSP число+единица,
  один en-dash и одна точность для денежных диапазонов.
- CTA «ОЦЕНИТЬ КВАРТИРУ» → solid brand-blue + белый текст; :focus-visible
  HUD-glow на инпутах и кнопке.
- Лендмарки <nav>/<main>/<footer> (display:contents, 0 визуальных изменений);
  scale-fit артборда — на 1366×768 / 1440×900 ужимается, не режется.
- Мини-карта: честный зум (min=1, без пустых краёв при отдалении), убран
  мёртвый crosshair-двойник, расклеены налезающие метки цен.
- CRM-дропдаун явно задизейблен («скоро», без ▼); убран «ШАГ 1/1».
- Карточка 02 → «ОЖИДАЕМАЯ ЦЕНА СДЕЛКИ · с учётом торга» + строка-примирение
  с медианой ДКП; КОЭФ.ЛОКАЦИИ → muted-пилюля «скоро».
- Бейдж СВОДКИ «В РАСЧЁТЕ» → «АКТУАЛЬНО»; скрыты целиком-прочерк колонки
  (ИСТОЧНИКОВ в 07, ОБЪЯВЛЕНИЕ ДО СДЕЛКИ в 04) data-driven.
2026-06-28 19:33:12 +03:00
8aaa42f101 chore(tradein/deploy): don't recreate scraper on infra-only deploys + 120s graceful stop (phase 0+1)
All checks were successful
CI / changes (pull_request) Successful in 9s
CI / backend-tests (pull_request) Has been skipped
CI / frontend-tests (pull_request) Has been skipped
CI / openapi-codegen-check (pull_request) Has been skipped
The tradein-scraper container runs the same image as backend
(python -m app.scheduler_main, in-app scheduler) and owns the in-flight
scrape sweeps (a browser card takes ~15-27s). Two problems compounded to
kill running jobs on every deploy:

Phase 0 (deploy-tradein.yml): SCRAPER_CHANGED also fired on the `infra`
paths-filter (compose / workflow / deploy/**), so any generic infra edit
recreated the scraper container and SIGKILLed the running job. Dropped the
`|| infra == 'true'` term — only real scraper-code paths (already covered
by the `scraper` paths-filter) or a manual workflow_dispatch recreate it.

Phase 1 (docker-compose.prod.yml): the scraper had no stop_grace_period,
so Docker's default 10s window SIGKILLed an in-flight card (15-27s) before
it could finish. Added stop_grace_period: 120s + explicit stop_signal:
SIGTERM so a running unit can finish + checkpoint. The cooperative-drain
handler that consumes SIGTERM lands in a later phase; this is the
foundation.
2026-06-28 19:30:14 +03:00
57ae2dfe00 fix(tradein): guard ГАР-undercount знаменателя в v_building_sale_share (145) (#2064)
All checks were successful
Deploy Trade-In / changes (push) Successful in 11s
Deploy Trade-In / build-frontend (push) Has been skipped
Deploy Trade-In / build-browser (push) Has been skipped
Deploy Trade-In / test (push) Successful in 1m29s
Deploy Trade-In / build-backend (push) Successful in 27s
Deploy Trade-In / deploy (push) Successful in 49s
2026-06-28 16:05:33 +00:00
579536b4ad feat(tradein): канонический матч ГАР↔houses (0%→41% yield) + tradein_canon_addr + --match-only (#2063)
All checks were successful
Deploy Trade-In / changes (push) Successful in 12s
Deploy Trade-In / build-frontend (push) Has been skipped
Deploy Trade-In / build-browser (push) Has been skipped
Deploy Trade-In / test (push) Successful in 1m31s
Deploy Trade-In / build-backend (push) Successful in 1m6s
Deploy Trade-In / deploy (push) Successful in 54s
2026-06-28 15:50:07 +00:00
83ae6f8a47 fix(tradein): гарантировать lxml в backend-образе — bust кэша uv sync + assert import (#2061)
All checks were successful
Deploy Trade-In / changes (push) Successful in 11s
Deploy Trade-In / build-frontend (push) Has been skipped
Deploy Trade-In / build-browser (push) Has been skipped
Deploy Trade-In / test (push) Successful in 1m30s
Deploy Trade-In / build-backend (push) Successful in 7m12s
Deploy Trade-In / deploy (push) Successful in 1m19s
2026-06-28 15:16:04 +00:00
bcbbb53495 Merge pull request 'fix(tradein/v2): wire РАДИУС (backend radius_m) + map +/− zoom + building image + deglue word+word' (#2060) from feat/tradein-v2-audit-fixes-2 into main
All checks were successful
Deploy Trade-In / changes (push) Successful in 13s
Deploy Trade-In / build-browser (push) Has been skipped
Deploy Trade-In / test (push) Successful in 1m32s
Deploy Trade-In / build-frontend (push) Successful in 4m10s
Deploy Trade-In / build-backend (push) Successful in 6m40s
Deploy Trade-In / deploy (push) Successful in 1m7s
2026-06-28 15:04:08 +00:00
e23dabe4f5 fix(tradein/v2): wire РАДИУС (backend radius_m) + map +/- zoom + building image + deglue word+word
All checks were successful
CI / changes (pull_request) Successful in 7s
CI / backend-tests (pull_request) Has been skipped
CI / frontend-tests (pull_request) Has been skipped
CI / openapi-codegen-check (pull_request) Has been skipped
Live-browser audit (round 2): РАДИУС/+- dead, building photo missing.
- РАДИУС: backend BE-2 — optional radius_m (int 100–5000) on TradeInEstimateInput,
  threaded into estimate_quality comp search (base + fallback). None preserves the
  exact two-tier default (1000m primary / 2000m fallback) byte-identically. FE: РАДИУС
  dropdown with **Авто** default (sends no radius_m → legacy behaviour, NOT a silent
  500m narrowing) + 300/500/1000/2000 overrides; included as radius_m on submit; outer
  map ring tracks the value.
- Map +/− buttons: now a real zoom (transform: scale() on the map content layer, step
  .25, clamp .6–2.5); controls/bracket stay unscaled.
- building.png: next/image didn't apply basePath → 400; switched to plain <img> with
  NEXT_PUBLIC_BASE_PATH-prefixed src (→ 200) + onError fallback. HERO АДРЕС card шоу.
- deglueAddr: also splits word+word glue ('БольшаковаГеологическая'→'Большакова
  Геологическая'), still preserves house letters (14А, д.5К).

next build green (/v2 34.3 kB); estimator None-default byte-identical (ge=100 rules out
falsy-0); code-reviewer  after the Авто-default fix (avoided a default-radius regression).
2026-06-28 18:03:07 +03:00
123717994b feat(tradein): ГАР-loader знаменателя — apartments-per-house → gar_house_flats + match к houses (#2059)
All checks were successful
Deploy Trade-In / changes (push) Successful in 11s
Deploy Trade-In / build-browser (push) Has been skipped
Deploy Trade-In / test (push) Successful in 1m36s
Deploy Trade-In / build-frontend (push) Successful in 4m43s
Deploy Trade-In / build-backend (push) Successful in 4m44s
Deploy Trade-In / deploy (push) Successful in 2m15s
2026-06-28 14:54:43 +00:00
f8112b420d feat(tradein): страница «доля квартир дома в продаже» — % слайдер, адреса, карта (#2058)
Some checks failed
Deploy Trade-In / deploy (push) Blocked by required conditions
Deploy Trade-In / changes (push) Successful in 11s
Deploy Trade-In / test (push) Has been skipped
Deploy Trade-In / build-browser (push) Has been skipped
Deploy Trade-In / build-backend (push) Has been skipped
Deploy Trade-In / build-frontend (push) Has been cancelled
2026-06-28 14:53:38 +00:00
c97dd57e1c perf(objective_lots): drop 3 dead indexes (status/rooms_area/bank) — write-amp (#1953) (#2056)
All checks were successful
Deploy / changes (push) Successful in 7s
Deploy / build-frontend (push) Has been skipped
Deploy / build-backend (push) Successful in 33s
Deploy / build-worker (push) Successful in 34s
Deploy / deploy (push) Successful in 8m34s
2026-06-28 14:39:05 +00:00
fb25180929 Merge pull request 'fix(tradein/v2): audit findings — de-fabricate location drawer, real map markers, validation, hero image, real user' (#2057) from feat/tradein-v2-audit-fixes into main
All checks were successful
Deploy Trade-In / changes (push) Successful in 13s
Deploy Trade-In / test (push) Has been skipped
Deploy Trade-In / build-browser (push) Has been skipped
Deploy Trade-In / build-backend (push) Has been skipped
Deploy Trade-In / build-frontend (push) Successful in 2m24s
Deploy Trade-In / deploy (push) Successful in 1m2s
2026-06-28 14:27:05 +00:00
5a2b6d91cd fix(tradein/v2): audit findings — de-fabricate location drawer, real map markers, validation, hero image, real user (#2036)
All checks were successful
CI / changes (pull_request) Successful in 7s
CI / backend-tests (pull_request) Has been skipped
CI / frontend-tests (pull_request) Has been skipped
CI / openapi-codegen-check (pull_request) Has been skipped
Browser audit of the live /trade-in/v2 found honesty + UX defects; fixes:
- 🔴 LocationDrawer: removed FABRICATED coef 0.87 / fake 'base×coef=result' / invented
  POI factors / false 'OpenStreetMap' source. Now honest static 'ПОЯСНЕНИЕ К РАСЧЁТУ'
  (real methodology) + 'коэф. локации в разработке' note. + Escape-to-close.
- 🔴 Map (ParamsPanel): removed static fixture price markers; renders REAL analog
  markers via new mapMarkers(estimate) (projects lat/lon around target, cap 6).
- ParamsPanel: empty площадь/address now shows a validation message instead of
  silently blocking submit; numeric placeholders made clearly placeholders ('напр. 54').
- HeroBar: dead 'КАК РАССЧИТАНО' button now opens the info drawer; building.png next/image
  400 fixed (unoptimized + onError fallback — basePath wasn't applied to the optimizer URL).
- TopNav: real user from useMe() (name/initials/org/email) instead of fixture
  'Андрей Петров'; dead Профиль/Настройки/Помощь dimmed/disabled; Выйти → logout().
- mappers: de-glue analog addresses ('69/2Геологическая'→'69/2 Геологическая', preserves
  house letters like 14А); 'Продажи в доме'→'Продажи рядом' (radius data); торг sign fixed
  (backend (start−last)/start positive=reduction → renders '−3.3%').
- HistoryView: '0 ЛОТА' → correct plural via pluralRu.

next build green; code-reviewer  APPROVE.
2026-06-28 17:26:12 +03:00
f3e10a04ad feat(tradein): API домов «доли квартир в продаже» — /buildings/sale-share (+ listings, summary) (#2055)
All checks were successful
Deploy Trade-In / changes (push) Successful in 12s
Deploy Trade-In / build-frontend (push) Has been skipped
Deploy Trade-In / build-browser (push) Has been skipped
Deploy Trade-In / test (push) Successful in 1m39s
Deploy Trade-In / build-backend (push) Successful in 8m5s
Deploy Trade-In / deploy (push) Successful in 52s
2026-06-28 14:14:18 +00:00
94d01ed066 perf(objective_lots): inline-pushdown competitors._SOLD_COUNT_SQL + covering physflat index (#1953) (#2054)
Some checks failed
Deploy / changes (push) Successful in 7s
Deploy / build-frontend (push) Has been skipped
Deploy / build-backend (push) Successful in 10m23s
Deploy / build-worker (push) Successful in 11m34s
Deploy / deploy (push) Failing after 8m8s
2026-06-28 14:07:49 +00:00
9e2d9ddbea feat(tradein): схема «доли квартир дома в продаже» — gar_house_flats + houses.gar_* + v_building_sale_share (143) (#2053)
All checks were successful
Deploy Trade-In / changes (push) Successful in 11s
Deploy Trade-In / build-frontend (push) Has been skipped
Deploy Trade-In / build-browser (push) Has been skipped
Deploy Trade-In / test (push) Successful in 1m34s
Deploy Trade-In / build-backend (push) Successful in 30s
Deploy Trade-In / deploy (push) Successful in 52s
2026-06-28 13:48:41 +00:00
a100f03aa7 Merge pull request 'feat(tradein/v2): CacheView history + PDF + quota + address autocomplete (FE-5)' (#2052) from feat/tradein-v2-fe5-cache into main
All checks were successful
Deploy Trade-In / changes (push) Successful in 12s
Deploy Trade-In / test (push) Has been skipped
Deploy Trade-In / build-browser (push) Has been skipped
Deploy Trade-In / build-backend (push) Has been skipped
Deploy Trade-In / build-frontend (push) Successful in 2m56s
Deploy Trade-In / deploy (push) Successful in 54s
2026-06-28 13:15:17 +00:00
81ae21ada9 feat(tradein/v2): CacheView history + PDF link + quota + address autocomplete (#2042)
All checks were successful
CI / changes (pull_request) Successful in 7s
CI / backend-tests (pull_request) Has been skipped
CI / frontend-tests (pull_request) Has been skipped
CI / openapi-codegen-check (pull_request) Has been skipped
FE-5, last FE wiring on the mappers pattern. Closes #2042.
- New hooks (lib/trade-in-api.ts): useEstimateHistory (GET /history), useGeocodeSuggest
  (GET /geocode/suggest?q=&limit=, debounced, enabled>=3 chars); useQuota reused.
- types/trade-in.ts: EstimateHistoryItem (area_m2 string, median_price), GeocodeSuggestion.
- mappers.ts: mapCache(history) -> {rows: CacheRow[], kpis: CacheKpi[]} — per-row status
  by 24h TTL; KPIs FE-derived from /history (всего/средняя цена/повторные %), NOT the
  global /cache-stats.
- CacheView: real previous-estimates list. ParamsPanel: address autocomplete dropdown →
  captures lat/lon into submit (cleared on manual edit). HeroBar: «Скачать PDF» → /estimate/
  {id}/pdf, UUID-gated + safeUrl scheme check + disabled when no estimate. TopNav: «Мои
  отчёты» from quota.used/history. page.tsx: invalidate history+quota on new estimate.
- Hydration-safe: PDF link gated behind a mounted flag (urlId is SSR-null/client-uuid).
- FE-1..4 untouched.

Verified: next build green (/v2 39.6 kB); tsx harness ran mapCache vs real /history (10
items: ВСЕГО 10 / СРЕДНЯЯ 8,85 млн / ПОВТОРНЫЕ 50%, statuses, null graceful).
code-reviewer APPROVE after hydration + reports-fallback fixes.
2026-06-28 16:14:34 +03:00
3421b21632 fix(scrapers): разблокировать domrf_kn flats — изоляция flats от poison-extras + throttle (#1945) (#2050)
All checks were successful
Deploy / changes (push) Successful in 6s
Deploy / build-frontend (push) Has been skipped
Deploy / build-backend (push) Successful in 9m31s
Deploy / build-worker (push) Successful in 11m57s
Deploy / deploy (push) Successful in 3m51s
2026-06-28 13:03:43 +00:00
0c0efb887a Merge pull request 'feat(tradein/v2): wire overlays History/Analytics/Sources to real API (FE-2/3/4)' (#2051) from feat/tradein-v2-overlays into main
All checks were successful
Deploy Trade-In / changes (push) Successful in 11s
Deploy Trade-In / test (push) Has been skipped
Deploy Trade-In / build-browser (push) Has been skipped
Deploy Trade-In / build-backend (push) Has been skipped
Deploy Trade-In / build-frontend (push) Successful in 2m49s
Deploy Trade-In / deploy (push) Successful in 49s
2026-06-28 12:46:49 +00:00
267585e312 feat(tradein/v2): wire overlays History/Analytics/Sources to real API (#2039 #2040 #2041)
All checks were successful
CI / changes (pull_request) Successful in 6s
CI / backend-tests (pull_request) Has been skipped
CI / frontend-tests (pull_request) Has been skipped
CI / openapi-codegen-check (pull_request) Has been skipped
FE-2/3/4 on the FE-1 mappers architecture. Adds mapHistory/mapAnalytics/mapSources
(+helpers) to mappers.ts; HistoryView/AnalyticsView/SourcesView accept composite
data props (fixture default); SectionOverlay forwards overlay data; page.tsx adds
useEstimatePlacementHistory/useSalesVsListings/useEstimateSellTimeSensitivity and
feeds mapped data. FE-1 main dashboard untouched.

- History: house sales (placement-history); ДКП KPI/header (street-deals +
  sales-vs-listings); per-deal rows with REAL addr/date/listing-source/discount.
- Analytics: KPIs (экспозиция/торг/доля снятых), 4 sell-time tiers, price-history
  polylines computed from year×source, detail scatter.
- Sources: ad rows from analogs (real source/distance/url via safeUrl), deal rows
  from actual_deals (real source/tier/date), market KPIs.
- Un-hardcoded fabricated cells: DKP addr/date/source, deal date, deals footer count.
- FE-approx (TODO BE-1 #2043): CV, per-source counts, days_on_market->listing_date.

Verified: next build green (/v2 37.8 kB); tsx harness ran the 3 overlay mappers vs
real estimate 701795d8 (houseSales/KPIs/sellTime/priceHistory/scatter/ad+deal rows
correct). code-reviewer APPROVE after honesty fixes.
2026-06-28 15:46:07 +03:00
510cf1cb80 Merge pull request 'fix(tradein/v2): address parse (Cyrillic \b bug) + scatter from listing_date' (#2049) from fix/tradein-v2-fe1-address-scatter into main
All checks were successful
Deploy Trade-In / build-backend (push) Has been skipped
Deploy Trade-In / build-browser (push) Has been skipped
Deploy Trade-In / deploy (push) Successful in 48s
Deploy Trade-In / changes (push) Successful in 11s
Deploy Trade-In / test (push) Has been skipped
Deploy Trade-In / build-frontend (push) Successful in 1m53s
2026-06-28 12:04:59 +00:00
ad5c1913cb fix(tradein/v2): address parse (Cyrillic \b bug) + scatter from listing_date (#2038)
All checks were successful
CI / changes (pull_request) Successful in 6s
CI / frontend-tests (pull_request) Has been skipped
CI / backend-tests (pull_request) Has been skipped
CI / openapi-codegen-check (pull_request) Has been skipped
QA on real prod estimate 701795d8 surfaced two mapper defects:
- STREET_RE used JS \b which does NOT word-boundary Cyrillic → street token never
  matched → object.address fell back to the bare house number ('14/3') and city
  mis-parsed ('14/3, Россия'). Rewrote parseAddress to handle both OSM (house-first)
  and DaData (city-first) orders; street keyword delimited by space, not \b. Now
  'улица Яскина, 14/3' + city 'Екатеринбург'.
- days_on_market is null across prod lots → scatter-mini rendered empty. Active
  analogs now fall back to 'days listed so far' = today-listing_date (deals still
  require a real days_on_market — today-deal-date is not exposure). TODO BE-1.

Verified via tsx harness against the real estimate (25/25); next build green.
2026-06-28 15:04:28 +03:00
e2ff8958ca Merge pull request 'feat(tradein/v2) [FE-1]: wire main dashboard to real API (mappers + page shell)' (#2048) from feat/tradein-v2-fe1-data-shell into main
All checks were successful
Deploy Trade-In / changes (push) Successful in 10s
Deploy Trade-In / build-frontend (push) Successful in 1m52s
Deploy Trade-In / test (push) Has been skipped
Deploy Trade-In / build-browser (push) Has been skipped
Deploy Trade-In / build-backend (push) Has been skipped
Deploy Trade-In / deploy (push) Successful in 48s
2026-06-28 11:50:52 +00:00
890daa4a31 feat(tradein/v2): wire main dashboard to real API (mappers + page shell) (#2038)
All checks were successful
CI / changes (pull_request) Successful in 6s
CI / openapi-codegen-check (pull_request) Has been skipped
CI / backend-tests (pull_request) Has been skipped
CI / frontend-tests (pull_request) Has been skipped
FE-1 foundation for #2036. app/v2/page.tsx now owns data: ParamsPanel form ->
useEstimateMutation, restore-by-id via window.location.search (mirrors v1, avoids
useSearchParams Suspense build break), sub-hooks useStreetDeals/useEstimateHouseAnalytics;
loading/empty/insufficient/error states (honest placeholders, no fixtures-as-real).

New components/trade-in/v2/mappers.ts: pure AggregatedEstimate(+street-deals/analytics)
-> fixture-shape transforms — 3 price tiers (asking/expected_sold/DKP), ranges, scatter,
7 source slots, summary. CV + per-source counts are FE-approx pending BE-1 (#2043).

ParamsPanel -> controlled form (onSubmit/isPending/error/initialValues). ResultPanel/
ObjectSummary/HeroBar/Footer accept a composite data prop defaulting to the existing
fixture (pixel-perfect markup unchanged). next build green (/v2 34 kB).
2026-06-28 14:48:57 +03:00
d651dac99a Merge pull request 'fix(tradein/avito): reap dead 404 listings in detail backfill (don't trip block breaker)' (#2047) from fix/tradein-avito-detail-dead-listing-reaper into main
All checks were successful
Deploy Trade-In / changes (push) Successful in 11s
Deploy Trade-In / build-frontend (push) Has been skipped
Deploy Trade-In / build-browser (push) Has been skipped
Deploy Trade-In / test (push) Successful in 1m27s
Deploy Trade-In / build-backend (push) Successful in 52s
Deploy Trade-In / deploy (push) Successful in 48s
2026-06-28 11:16:57 +00:00
a50af40ad6 fix(tradein/avito): reap dead 404 listings in detail backfill (don't trip block breaker)
All checks were successful
CI / changes (pull_request) Successful in 7s
CI / backend-tests (pull_request) Has been skipped
CI / frontend-tests (pull_request) Has been skipped
CI / openapi-codegen-check (pull_request) Has been skipped
avito_detail_backfill (mode=browser) aborted prod run 458 with
attempted=5 enriched=0 blocked=5: the lat-null pending queue is dominated
by DEAD listings (404 — that's why they are stale coord-holes). In browser
mode a dead listing's 404 page ("Ошибка 404. Страница не найдена") has no
item-view, so _is_detail_soft_block returned True → AvitoBlockedError →
backfill counted it as `blocked` and the consecutive-block breaker aborted
the run before reaching live listings.

- New AvitoListingGoneError (subclass of common AvitoError, NOT of
  AvitoBlockedError/AvitoRateLimitedError) so the backfill block-trap
  does not catch it.
- New _is_detail_not_found() detector + browser-mode branch in fetch_detail,
  checked BEFORE firewall/soft-block (keyed on the 404 title so it does not
  swallow the generic soft-block deflect). Curl path unchanged (404 → 302/
  non-200 → ValueError → failed, as before).
- Backfill: new `gone` counter; dedicated except AvitoListingGoneError branch
  that is neutral to the consecutive-block breaker and marks the listing
  is_active=FALSE (SAVEPOINT) so it leaves the snapshot scope.

Tests: unit for _is_detail_not_found (404 True; soft-block/item-view/empty
False), browser-mode fetch_detail raising AvitoListingGoneError (not Blocked),
and backfill gone-row marks inactive without aborting the breaker.
2026-06-28 14:05:31 +03:00
7b8ffd4200 Merge pull request 'refactor(tradein/v2): перенос HUD v2 в tradein-mvp (прод-app) → /trade-in/v2' (#2037) from feat/tradein-v2-mvp-port into main
All checks were successful
Deploy / changes (push) Successful in 7s
Deploy / build-backend (push) Has been skipped
Deploy / build-worker (push) Has been skipped
Deploy Trade-In / changes (push) Successful in 12s
Deploy Trade-In / test (push) Has been skipped
Deploy Trade-In / build-browser (push) Has been skipped
Deploy Trade-In / build-backend (push) Has been skipped
Deploy Trade-In / build-frontend (push) Successful in 2m4s
Deploy Trade-In / deploy (push) Successful in 55s
Deploy / build-frontend (push) Successful in 3m34s
Deploy / deploy (push) Successful in 1m18s
Reviewed-on: #2037
2026-06-28 10:17:11 +00:00
fcb85deab8 refactor(tradein/v2): перенос HUD v2 в tradein-mvp frontend
All checks were successful
CI / changes (pull_request) Successful in 6s
CI / backend-tests (pull_request) Has been skipped
CI / frontend-tests (pull_request) Successful in 1m3s
CI / openapi-codegen-check (pull_request) Successful in 1m59s
Прод-/trade-in/* обслуживает tradein-mvp/frontend (basePath=/trade-in,
standalone), а не главный frontend — v2 по ошибке попал в главный, где
прод его не отдаёт. Markup-only порт (fixtures, без API):

- компоненты v2 → tradein-mvp/frontend/src/components/trade-in/v2/
- роут src/app/v2/ (basePath даёт прод-URL /trade-in/v2)
- public/trade-in-v2/building.png
- HeroBar: <img> → next/image (плоский img не префиксит basePath → 404)
- удалён мёртвый v2 из главного frontend

next build (NEXT_PUBLIC_BASE_PATH=/trade-in) зелёный, роут /v2 prerendered.
2026-06-28 13:10:23 +03:00
fba41c0cce feat(tradein/v2): HUD-редизайн страницы оценки (Макс) → /trade-in/v2 (#2035)
All checks were successful
Deploy / changes (push) Successful in 7s
Deploy / build-backend (push) Has been skipped
Deploy / build-worker (push) Has been skipped
Deploy / build-frontend (push) Successful in 3m22s
Deploy / deploy (push) Successful in 1m20s
2026-06-28 09:58:53 +00:00
ca4637e5c8 fix(tradein/avito): detect HTTP-200 soft-block → rotate + coords-priority backfill ordering (#1950, #1967)
All checks were successful
Deploy Trade-In / changes (push) Successful in 11s
Deploy Trade-In / build-frontend (push) Has been skipped
Deploy Trade-In / build-browser (push) Has been skipped
Deploy Trade-In / test (push) Successful in 1m26s
Deploy Trade-In / build-backend (push) Successful in 52s
Deploy Trade-In / deploy (push) Successful in 48s
Soft-block (HTTP 200 generic витрина без item-view) теперь детектится как блок → существующая 403/firewall reconnect-машинерия → ротация exit-IP вместо silent failed (blocked=0, enriched→0 каскад). + snapshot ORDER BY (lat IS NULL) DESC для приоритизации coord-holes (#1967, in-scope 321). Tests: test_avito_detail_soft_block.py. Full tradein suite 2591 passed.
2026-06-28 08:06:12 +00:00
2184453d8e Merge pull request 'feat(tradein/domclick): local-runner + ingest tooling (mergeable split from #2005)' (#2033) from feat/tradein-domclick-local-tooling into main
All checks were successful
Deploy Trade-In / build-browser (push) Has been skipped
Deploy Trade-In / deploy (push) Successful in 49s
Deploy Trade-In / changes (push) Successful in 9s
Deploy Trade-In / build-frontend (push) Has been skipped
Deploy Trade-In / test (push) Successful in 1m29s
Deploy Trade-In / build-backend (push) Successful in 55s
2026-06-28 07:29:41 +00:00
43b392b809 feat(tradein/domclick): local-runner + ingest tooling → main (split from parked Layer-B)
All checks were successful
CI / changes (pull_request) Successful in 7s
CI / backend-tests (pull_request) Has been skipped
CI / frontend-tests (pull_request) Has been skipped
CI / openapi-codegen-check (pull_request) Has been skipped
Выделяет МЕРДЖАБЕЛЬНУЮ часть DomClick-тулинга из запаркованного PR #2005
(BLOCKED #2000 — прод Layer-B скрейпер через прокси режет QRATOR-403):

- scripts/domclick_local_runner.py — ручной offline-сборщик с домашнего IP:
  SERP-enumerate (фронт, BFF-JSON режет QRATOR) + card __SSR_STATE__ +
  offer-card v3 (AVM price_prediction + sold_similar). --enum-cache (enumerate
  1 раз → файл с sentinel → следующий запуск грузит кэш и пропускает enumerate),
  wait_for_selector (SERP-карточки догидрируются), resume по id. Self-contained.
- scripts/ingest_domclick_jsonl.py — JSONL → trade-in БД (save_listings +
  house-match в SAVEPOINT + save_detail_enrichment, идемпотентно, --limit/--dry-run).
- app/services/scrapers/domclick_detail.py + domclick_exceptions.py — card-SSR
  parse + save_detail_enrichment (нужны ингесту). Импортятся ТОЛЬКО ингестом
  (operator-run); прод-runtime их не дёргает (dormant в образе).
- tests/scrapers/test_domclick_detail.py + обновлённый scripts/README.md.

НЕ включает прод Layer-B (tasks/domclick_detail_backfill.py, scheduler-расписание,
data/sql/139_*) — остаётся запаркован на PR #2005 до чистого прокси (#2000).
БД-схема не меняется (detail-колонки уже в проде, миграции нет).

Live-validated: раннер качает ЕКБ-вторичку прямо сейчас (600+ карт, 0 блоков),
ингест залил их в прод-БД (house_matched, идемпотентно).
2026-06-28 10:26:52 +03:00
59ee8b672f Merge pull request 'fix(objective): physflat-dedup objective_lots — view + inline (request-path) + n_sold window-fix (#1964)' (#2032) from fix/objective-lots-dedup-1964 into main
All checks were successful
Deploy / changes (push) Successful in 7s
Deploy / build-frontend (push) Has been skipped
Deploy / build-backend (push) Successful in 1m48s
Deploy / build-worker (push) Successful in 2m51s
Deploy / deploy (push) Successful in 1m27s
2026-06-28 00:21:45 +00:00
74f1ffb500 perf(objective): request-path consumers dedup inline, not via whole-table view (#1964)
All checks were successful
CI / changes (pull_request) Successful in 6s
CI / frontend-tests (pull_request) Has been skipped
CI / openapi-codegen-check (pull_request) Successful in 1m49s
CI / backend-tests (pull_request) Successful in 13m58s
deep-review #1964: v_objective_lots_latest has NO premise/district filter inside,
so a consumer's outer WHERE cannot push below DISTINCT ON → the view materializes
the WHOLE table (Parallel Seq Scan + external Sort 1.76M rows, ~55MB spill) on
every query. For REQUEST-PATH consumers inside analyze_parcel this is a ~19x latency
regression vs the pre-#1964 raw-table plan.

DISPROVEN remedy (NOT applied): a full index on the physflat-key does NOT help —
DISTINCT ON selects ol.* (51 cols, width≈945) so index-only-unique is impossible;
the planner ignores the index (seq-scan+sort still cheaper) and even forced it is
~3.9 s. A 142MB index for zero request-path benefit + slower bulk-INSERT during
objective-scrape is wrong. Honors original #1964 decision "no new index".

Prod EXPLAIN (Академический / 3km radius, 2026-06-28):
  consumer            via view      inline (this commit)
  concepts median     5854 ms       1640 ms   (bitmap district + sort)
  parcels district    5854 ms       1640 ms
  parcels geo-median  6443 ms        122 ms   (NestedLoop geo->complex bitmap)
  parcels obj_pricing 5721 ms        441 ms   (project bitmap per nearby ЖК)

FIX: keep v_objective_lots_latest ONLY for batch/background/cached consumers
(supply_layers L1, competitors._SOLD_COUNT_SQL, special_indices [/forecast bg task
30-180s], admin, landing). Revert the 4 request-path consumers inside analyze_parcel
to inline DISTINCT ON (physflat-key, latest snapshot) with the filter pushed INTO the
CTE so the district/spatial/project index applies:
- concepts._OBJECTIVE_MEDIAN_SQL
- parcels.py district price block
- parcels.py geo-radius median (complex_id-scoped)
- parcels.py obj_pricing CTE (project_name-scoped; aggregates over deduped set)

Migration 175 header CORRECTED: accurately states the partial mig-173 index does NOT
serve the view (qual can't push below DISTINCT ON), the full index is disproven/not
added, and which consumers use the view vs inline. No DDL change (still view-only).

Tests: +guards (concepts/obj_pricing dedup inline, not view; obj_pricing physflat
DISTINCT ON; perf-pushdown scope preserved). 965 passed.
2026-06-28 05:00:47 +05:00
e67cb721bf fix(objective): physflat-dedup current-state consumers + honest n_sold window (#1964)
objective_lots — current-state UPSERT (UNIQUE objective_lot_id, 5 snapshot_date),
но Объектив присваивает ОДНОМУ физлоту (project,corpus,section,floor,lot_number)
несколько lot_id за пере-листинги → таблица раздута ~2.91× (прод: 1.75M квартир-
строк vs 603k физлотов). История — в отдельной objective_lots_history (не трогаем).

STEP 1: миграция 175 — VIEW v_objective_lots_latest (DISTINCT ON physflat-ключ,
последний снапшот). ol.* стабилен (51==51 колонок). DRY-RUN на проде: 603 049
квартир vs 1 753 283 raw.

STEP 2: репойнт current-state консьюмеров objective_lots → v_objective_lots_latest:
- supply_layers._L1_OPEN_SQL (L1 открытое предложение → дефицит-форсайт; прод:
  Юго-Западный комфорт 58 606 → 12 620)
- competitors._SOLD_COUNT_SQL (+ комментарий: COUNT(DISTINCT lot_id) СОХРАНЁН для
  fan-out-защиты маппинга, не COUNT(*) — view гарантирует physflat-дедуп, DISTINCT
  гарантирует mapping-fan-out-safety)
- parcels.py obj_pricing CTE (карточка конкурента units_sold/available)
- special_indices._ARTIFICIAL_DEMAND_SQL
- parcels.py district price block + geo-radius median (sample_size/n)
- concepts._OBJECTIVE_MEDIAN_SQL (гейт n≥10)
- landing KPI3 (% квартир с ценой)
- admin_scrape coverage: `lots` оставлен сырым (ETL-fidelity vs SQLite), добавлен
  `lots_physflat`
#1959 inline-дедуп в market_metrics НЕ рефакторим — добавлен комментарий об общем
physflat-ключе с view.

STEP 3: report_assembler._deal_count теперь = unit_velocity × window_months
(оконные продажи), НЕ кумулятивный n_sold. confidence_engine помечает фактор
«за 6 мес» и гейтит порогами окна (high≥50) — кумулятив (прод EKB ~380 921) делал
гейт бессмысленным и подпись лживой; оконное (~24 876 за 6 мес) честно.

Тесты: +guard'ы (L1/sold-count читают view, deal_count оконный). 963 passed.
2026-06-28 04:27:43 +05:00
91edef1524 Merge pull request 'fix: audit follow-up мелочи — индекс domrf_kn_flats + max_height_m coercion + hide empty reg-date (#1953)' (#2031) from fix/audit-followups-1953 into main
All checks were successful
Deploy / changes (push) Successful in 6s
Deploy / build-backend (push) Successful in 30s
Deploy / build-worker (push) Successful in 31s
Deploy / build-frontend (push) Successful in 3m11s
Deploy / deploy (push) Successful in 1m12s
2026-06-27 22:37:14 +00:00
8aec625134 fix(audit): index for flats_latest, coerce max_height_m, hide empty reg-date (#1953)
All checks were successful
CI / changes (pull_request) Successful in 8s
CI / frontend-tests (pull_request) Successful in 1m2s
CI / openapi-codegen-check (pull_request) Successful in 1m56s
CI / backend-tests (pull_request) Successful in 14m7s
Three #1953 audit follow-ups:

- perf: add migration 174 — composite index idx_kn_flats_obj_snap on
  domrf_kn_flats (obj_id, snapshot_date DESC). Serves the #1956 flats_latest
  CTE in best_layouts.py (DISTINCT ON (obj_id) ORDER BY obj_id, snapshot_date
  DESC + self-join on (obj_id, snapshot_date)); previously only (obj_id)
  existed so Postgres sorted per object. Prod: 789 569 rows, idx ~5.7 MB,
  dry-run instant. Idempotent, self-wrapped BEGIN/COMMIT.

- frontend: route every max_height_m read through coerceFloat (same string-bug
  class as max_far #1962). max_height_m is NUMERIC → arrives as a string on the
  wire; ptica-adapt.ts read it raw at 4 sites and relied on formatInt/Math.round
  coercion. Widen the type in nspd.ts and fix the stale "real number" comment in
  nspd-regulation.ts.

- frontend: hide the «Дата регистрации» EGRN row entirely when
  registration_date is null (~97% of parcels) instead of rendering a bare «—».
2026-06-28 03:19:18 +05:00
2fb34112b5 Merge pull request 'feat(concept): picker «выбери дом» + каталог-эндпоинт — Stage 3b, финал #1965' (#2030) from feat/concept-house-picker-1965-stage3b into main
All checks were successful
Deploy / changes (push) Successful in 8s
Deploy / build-backend (push) Successful in 2m7s
Deploy / build-worker (push) Successful in 3m9s
Deploy / build-frontend (push) Successful in 3m35s
Deploy / deploy (push) Successful in 1m26s
2026-06-27 22:08:57 +00:00
c55eb4f4d4 feat(concept): фронт-пикер типовых домов (#1965 Stage 3b)
All checks were successful
CI / changes (pull_request) Successful in 7s
CI / frontend-tests (pull_request) Successful in 1m1s
CI / openapi-codegen-check (pull_request) Successful in 1m55s
CI / backend-tests (pull_request) Successful in 14m9s
Финальная часть эпика #1953: пользователь выбирает типовые дома
(тип × этажность × число секций) вместо авто max-FAR раскладки, формируя
building_program из Stage 3a.

Бэкенд:
- GET /api/v1/concepts/house-types — read-only каталог HOUSE_TYPES
  (section_type, label_ru, footprint w×d + sqm, default_floors, housing_class)
  как single source of truth; фронт ничего не хардкодит.
- Схема HouseTypeCatalog / HouseTypeCatalogItem в schemas/concept.py.
- Тесты эндпоинта: полнота каталога + совпадение ключей с available_section_types.

Кодген: api-types.ts перегенерён (dump OpenAPI → openapi-typescript →
project-local prettier 3.9.0); 2-й прогон без диффа.

Фронтенд:
- useHouseTypes() (TanStack useQuery, staleTime Infinity) в concept-api.ts;
  building_program в ConceptInput, placed_count/requested_count в ConceptVariant.
- HouseProgramPicker: toggle «Авто (max-FAR)» (default, program omit → greedy)
  vs «Выбрать дома» (список каталожных типов, count 1-50 / floors 1-40, дефолт
  из каталога; габариты/этажность/класс как подсказка). Смонтирован в
  Section7Concept и на странице /concept.
- Partial-fit заметка в ConceptVariantsResult: при placed<requested честное
  «Разместилось N из M секций — участок вмещает меньше» (нейтрально, не ошибка).
2026-06-28 02:49:44 +05:00
3b84609670 Merge pull request 'feat(concept): каталог типовых домов + program-placement (#1965 Stage 3a)' (#2029) from feat/concept-house-catalog-1965-stage3a into main
All checks were successful
Deploy / changes (push) Successful in 6s
Deploy / build-backend (push) Successful in 2m18s
Deploy / build-frontend (push) Successful in 3m24s
Deploy / build-worker (push) Successful in 3m37s
Deploy / deploy (push) Successful in 1m27s
2026-06-27 21:40:10 +00:00
e8b05d821e test(concept): turn greedy backward-compat into a real golden pin (#1965 Stage 3a)
All checks were successful
CI / changes (pull_request) Successful in 7s
CI / frontend-tests (pull_request) Successful in 1m3s
CI / openapi-codegen-check (pull_request) Successful in 1m51s
CI / backend-tests (pull_request) Successful in 14m7s
The old test_no_program_reproduces_greedy_output_unchanged only compared
building_program=None (default) vs explicit None — both through the new
_Placer code — so it proved the two None branches agree but did NOT pin
the greedy geometry; it would still pass if the _Placer extraction had
drifted the output. test_placement.py only checks invariants, never
concrete counts/TEAP, so there was no anti-regression guard that the
greedy path is byte-identical after the refactor.

Replace it with two tests:
- test_greedy_output_matches_golden_pin: hard-coded literals per strategy
  on the fixed _BIG_PARCEL — (features, built_area_sqm, total_floor_area_sqm,
  apartments_count) — frozen from the current (== pre-refactor) output, so
  any future deterministic drift in greedy placement FAILS.
- test_explicit_none_program_equals_default_greedy: keeps the None-branch
  equivalence check (default vs explicit None go one greedy path).
2026-06-28 02:23:55 +05:00
2ebf7468cd fix(tradein/avito): hard-timeout on detail-backfill fetch — prevent zombie-hang (#1950)
All checks were successful
Deploy Trade-In / build-frontend (push) Has been skipped
Deploy Trade-In / test (push) Successful in 1m25s
Deploy Trade-In / build-backend (push) Successful in 52s
Deploy Trade-In / deploy (push) Successful in 50s
Deploy Trade-In / changes (push) Successful in 11s
Deploy Trade-In / build-browser (push) Has been skipped
fetch_detail wrapped in asyncio.wait_for(90s) + bounded _rotate_ip — a single hung fetch no longer zombies the whole run (run 423 hung 7.7h). New except TimeoutError (failed+continue), avito_detail_fetch_timeout_s setting, admin trigger POST /scrape/avito-detail-backfill. Restores detail-queue throughput (coords #1967 + house_type). Refs #1950
2026-06-27 20:29:27 +00:00
94cf1f6217 feat(concept): house-type catalog + program-driven placement (#1965 Stage 3a)
Extend the concept generator so ConceptInput can carry an optional
building_program (list of typed houses from a catalog). When present,
placement lays out EXACTLY that program — for each item, place `count`
sections of the catalog footprint at the item's floors — instead of the
greedy max-FAR coverage-cap sweep. When absent, the existing greedy
behavior is unchanged (byte-for-byte backward-compatible).

- catalog.py: hardcoded HOUSE_TYPES (panel_econom, monolith_comfort,
  tower_business, lowrise_comfort, townhouse) — sane-default catalog,
  promote to DB later; get_house_type / available_section_types lookups.
- schema: additive BuildingProgramItem {section_type, floors, count} and
  ConceptInput.building_program (default None -> greedy). ConceptVariant
  gains optional placed_count / requested_count (partial-fit signal).
- placement: shared _Placer (collision/STRtree/setback machine extracted
  from greedy sweep, reused — no duplication); place_program +
  place_program_variant; branch in place_all_strategies on
  building_program. Mixed-floor TEAP via exact per-floor-group aggregation
  (GFA = sum(area_i * floors_i), no rounding drift).
- partial fit: when the parcel can't fit all sections, place as many as
  fit and report placed_count < requested_count (no hard-422); zero-fit
  still raises ParcelGeometryError (-> 422).
- API: validate program section_type keys against the catalog (unknown ->
  422) before placement.
- tests: catalog integrity, greedy backward-compat, exact 2-item program +
  TEAP reflection, over-packed partial placement, API program path.
- regenerate frontend api-types.ts (OpenAPI codegen gate stays green).
2026-06-28 01:29:04 +05:00
44b9305d89 feat(tradein/estimator): elite-segment manual_review trigger by asking ppm² (#2002)
All checks were successful
Deploy Trade-In / changes (push) Successful in 10s
Deploy Trade-In / build-frontend (push) Has been skipped
Deploy Trade-In / build-browser (push) Has been skipped
Deploy Trade-In / test (push) Successful in 1m25s
Deploy Trade-In / build-backend (push) Successful in 51s
Deploy Trade-In / deploy (push) Successful in 48s
2026-06-27 20:15:38 +00:00
5b4c0dd1b2 Merge pull request 'feat(concept): живой 3D-massing → пересчёт финмодели в «7. Концепция» (#1965 Stage 2b)' (#2026) from feat/concept-live-massing-1965-stage2b into main
All checks were successful
Deploy / changes (push) Successful in 8s
Deploy / build-backend (push) Has been skipped
Deploy / deploy (push) Successful in 1m14s
Deploy / build-worker (push) Has been skipped
Deploy / build-frontend (push) Successful in 3m15s
2026-06-27 20:10:41 +00:00
a0232e22f1 feat(concept): live 3D massing → financial recompute in «7. Концепция» (#1965 Stage 2b)
All checks were successful
CI / changes (pull_request) Successful in 8s
CI / backend-tests (pull_request) Has been skipped
CI / frontend-tests (pull_request) Successful in 1m1s
CI / openapi-codegen-check (pull_request) Successful in 1m54s
Wire the interactive 3D massing to a LIVE financial recompute inside the
Section «7. Концепция» of the light analysis report (epic #1953).

- concept-api.ts: add MassingProgram / MassingRecomputeOutput contracts +
  useRecomputeMassing() (POST /api/v1/concepts/recompute via shared apiFetch),
  and polygonCentroidWkt() helper for the DB-price fallback hint.
- MassingScene.tsx: export MassingModel + add optional onModelChange prop fired
  on the initial build and every floors/sections change (NOT on the sun/hour
  slider). No-op when absent — /ptica cockpit + preview/drawer unchanged.
- MassingEconomics.tsx (new): light-token KPI strip rendering recomputed ТЭП
  (GFA, продаваемая, квартиры, КСИТ факт/цель) + финмодель (выручка,
  себестоимость, прибыль, ROI, NPV, IRR, цена). Debounced ~250ms, latest-wins
  via monotonic request id, last-good values + subtle note on error, grey-fade
  skeleton on first load, КСИТ over-cap flag.
- Section7Concept.tsx: mount lazy MassingViewer (dynamic ssr:false) + the
  economics strip; map computeModel → MassingProgram (footprint/floors/sections
  ← model; site area ← polygon; class/type ← financial_estimate inferred,
  fallback comfort/mid_rise; land ← egrn cadastral; price/source ←
  financial_estimate; КСИТ-цель ← coerced nspd_zoning.max_far, fallback 3.5).
2026-06-28 01:01:53 +05:00
01b0dc74de Merge pull request 'feat(generative): POST /concepts/recompute — live финрасчёт по massing-program (#1965 Stage 2a)' (#2024) from feat/concept-recompute-1965-stage2a into main
All checks were successful
Deploy / changes (push) Successful in 8s
Deploy / build-backend (push) Successful in 1m54s
Deploy / build-worker (push) Successful in 3m22s
Deploy / build-frontend (push) Successful in 3m35s
Deploy / deploy (push) Successful in 1m28s
2026-06-27 19:53:53 +00:00
ec0f113d4f chore(tradein/backtest): --resolve-house-id flag to measure Tier-S + IMV anchor (#2002)
All checks were successful
Deploy Trade-In / changes (push) Successful in 10s
Deploy Trade-In / build-browser (push) Has been skipped
Deploy Trade-In / build-frontend (push) Has been skipped
Deploy Trade-In / test (push) Successful in 1m27s
Deploy Trade-In / build-backend (push) Successful in 56s
Deploy Trade-In / deploy (push) Successful in 53s
2026-06-27 19:44:38 +00:00
7e824ce6e4 chore(frontend): regenerate api-types.ts for /concepts/recompute (#1965 Stage 2a)
All checks were successful
CI / changes (pull_request) Successful in 8s
CI / frontend-tests (pull_request) Successful in 1m17s
CI / openapi-codegen-check (pull_request) Successful in 2m11s
CI / backend-tests (pull_request) Successful in 14m1s
The new POST /api/v1/concepts/recompute endpoint + MassingProgram /
MassingRecomputeOutput schemas changed the backend OpenAPI; the generated
frontend/src/lib/api-types.ts was out of sync, failing the CI
openapi-codegen-check gate. Regenerated via the exact CI sequence
(openapi-typescript + prettier 3.9.0). Byte-stable on 2nd regen; tsc --noEmit clean.
2026-06-28 00:38:28 +05:00
18d012da1b fix(generative): forward genuine price_source through /recompute (#1965 Stage 2a)
Some checks failed
CI / frontend-tests (pull_request) Has been skipped
CI / changes (pull_request) Successful in 7s
CI / openapi-codegen-check (pull_request) Failing after 1m49s
CI / backend-tests (pull_request) Successful in 13m49s
Code-review follow-up: /recompute hardcoded price_source="objective_district_median"
for any body-supplied market_price_per_sqm, mislabeling the honesty-flag once
Stage 2b forwards a price whose genuine source differs (objective_geo_radius /
district_reference / class_norm from financial_estimate).

- schemas/concept.py: add optional price_source: str | None to MassingProgram.
- api/v1/concepts.py: on FAST path use payload.price_source if provided, else the
  default label (now a module-level constant _DEFAULT_PRERESOLVED_SOURCE). DB-fallback
  and class-norm paths keep their own resolved source unchanged.
- tests: assert body-provided price_source echoes through to financial.price_source
  (not overwritten), and the default label applies when the front omits it.
2026-06-28 00:33:59 +05:00
d38d5d43c5 feat(generative): LIVE financial recompute from massing program (#1965 Stage 2a)
Stage 2a of epic #1953: backend service + endpoint for live economic recompute
driven by the interactive 3D massing (Stage 2b debounced sliders).

- teap.py: add pure synthesize_teap_from_program(total_footprint_sqm, floors,
  site_area_sqm, housing_class, sections) — builds a TEAP from the SCALAR
  aggregate footprint × floors, mirroring synthesize_teap_from_buildability and
  reusing the same shared norm constants (_OFFICE_SHARE_OF_GFA / _EFFICIENCY_BY_CLASS
  / _AVG_APARTMENT_SQM / _PARKING_PER_APARTMENT) — single source of truth.
- schemas/concept.py: add MassingProgram (program contract, optional pre-resolved
  market_price_per_sqm + parcel_centroid_wkt) and MassingRecomputeOutput (teap + financial).
- api/v1/concepts.py: add POST /api/v1/concepts/recompute — synthesize TEAP → run
  the existing pure compute_financial. FAST path uses body market_price_per_sqm
  (no DB); else _lookup_market_price by centroid via run_in_threadpool; else class norm.
- tests: synthesize_teap_from_program (gfa math, parity with compute_teap, class
  efficiency, sections no-op) + endpoint (200, coherent output, price passthrough
  skips DB, DB fallback, class-norm default, floors validation).
2026-06-28 00:27:25 +05:00
fed33fce27 Merge pull request 'feat(site-finder): «7. Концепция» в light-отчёте — Stage 1 #1965' (#2023) from feat/concept-section-1965-stage1 into main
All checks were successful
Deploy / build-frontend (push) Successful in 3m10s
Deploy / deploy (push) Successful in 1m15s
Deploy / changes (push) Successful in 8s
Deploy / build-backend (push) Has been skipped
Deploy / build-worker (push) Has been skipped
2026-06-27 19:22:12 +00:00
e809063ca3 feat(site-finder): add «7. Концепция» section to light analysis report
All checks were successful
CI / changes (pull_request) Successful in 7s
CI / backend-tests (pull_request) Has been skipped
CI / frontend-tests (pull_request) Successful in 58s
CI / openapi-codegen-check (pull_request) Successful in 1m58s
Fold the generative concept generator (3 strategy variants: placement map +
ТЭП + finance) into the light analysis report (/site-finder/analysis/{cad}) as
a new section after «6. Прогноз», so Концепция lives in one place alongside the
report instead of a separate flow (epic #1953, #1965 Stage 1).

Frontend-only: reuses ConceptParamsForm / ConceptVariantsResult and the
/concepts mutation (useCreateConcept) as-is — no duplication, no backend change.
Inputs are seeded from the analysis the report already has (no re-entry of the
cadastre/polygon): polygon ← extractPolygon(geom_geojson); housing_class /
development_type ← financial_estimate inferred values; land_cost ←
egrn.cadastral_value_rub; falls back to comfort/mid_rise.

Runs on demand (heavy generative call — never auto on mount); result block
(Leaflet map) is lazy-mounted via dynamic({ ssr: false }) and the section is
collapsed by default. When financial_estimate is null (ЗОУИТ/СЗЗ/нет зоны) it
shows an honest banner but still allows a manual run, since /concepts is
independent of the financial_estimate gate. Graceful empty state when geometry
is missing/unsupported. Stage 2 (interactive 3D + /concepts/recompute) left as
a seam, not built.
2026-06-28 00:13:57 +05:00
3187419eb8 feat(tradein/ui): manual-review banner + premium-building badge on estimate (#2002)
All checks were successful
Deploy Trade-In / changes (push) Successful in 11s
Deploy Trade-In / test (push) Has been skipped
Deploy Trade-In / build-browser (push) Has been skipped
Deploy Trade-In / build-backend (push) Has been skipped
Deploy Trade-In / build-frontend (push) Successful in 1m53s
Deploy Trade-In / deploy (push) Successful in 52s
2026-06-27 19:09:46 +00:00
96d5794b5c Merge pull request 'fix(best-layouts): per-object latest snapshot для supply → срок продажи/% продано (#1956)' (#2020) from fix/best-layouts-per-object-snapshot-1956 into main
All checks were successful
Deploy / build-frontend (push) Has been skipped
Deploy / build-backend (push) Successful in 1m49s
Deploy / build-worker (push) Successful in 3m26s
Deploy / deploy (push) Successful in 1m26s
Deploy / changes (push) Successful in 7s
2026-06-27 18:43:19 +00:00
e39fb86a9b feat(tradein): AI-curated premium-building overlay — class + false-positive demotion (#2002)
All checks were successful
Deploy Trade-In / changes (push) Successful in 10s
Deploy Trade-In / build-browser (push) Has been skipped
Deploy Trade-In / test (push) Successful in 1m29s
Deploy Trade-In / deploy (push) Successful in 52s
Deploy Trade-In / build-frontend (push) Has been skipped
Deploy Trade-In / build-backend (push) Successful in 56s
2026-06-27 18:43:05 +00:00
e2bf285980 chore(design-sync): ship Inter + JetBrains Mono brand fonts via cfg.extraFonts
Converter copies them to ds-bundle/fonts/ and styles.css @imports the
closure -> [FONT_MISSING] resolved, designs render in brand fonts.
Manrope still loads via remote @import in trade-in.css.
2026-06-27 14:56:50 +03:00
791 changed files with 115906 additions and 31071 deletions

View file

@ -351,7 +351,7 @@ Claude имеет ДВА лимита: 5h-rolling (сам сбрасываетс
- Пустая очередь + нет fixup-PR → выходить из loop, а не крутиться вхолостую.
- Тяжёлые batch-прогоны — вне пиковых часов (≈511 PT) при возможности.
- Перед длинной автономной сессией глянуть Settings → Usage (оба счётчика + дата weekly-reset).
- **Окна — на Sonnet, не Opus** (`start-bot.ps1` уже запускает с `--model sonnet`). Opus — только main-оркестратору. Sonnet-пул отдельный, недельный All-models (Opus) пул так не горит.
- **Окна — на Sonnet, не Opus** (`start-bot.ps1` уже запускает с `--model sonnet`; reviewer — opus). Opus — только main-оркестратору. Sonnet-пул отдельный, недельный All-models (Opus) пул так не горит.
- **Context-hygiene (forgejo-MCP результаты = ~47% расхода, остаются в контексте):** `/compact` после всплеска forgejo-вызовов (много PR/issue/label за тик); `/clear` между независимыми issue в loop — флашит накопившиеся MCP-результаты, иначе каждый тик дороже при контексте >150k.
## Error escalation

View file

@ -3,7 +3,7 @@ name: auto-analyst
description: "[DRAFT — autonomous loop only] Analyst в режиме /loop 15m. Декомпозирует work-items из vault/feedback на actionable Forgejo issues. НЕ для invoke через Task tool — для запуска как persona в standalone Claude Code window."
status: draft
created_at: 2026-05-27
model: opus
model: sonnet
tools: Task, Read, Glob, Grep, Bash, mcp__obsidian__obsidian_simple_search, mcp__obsidian__obsidian_complex_search, mcp__obsidian__obsidian_get_file_contents, mcp__obsidian__obsidian_list_files_in_dir, mcp__obsidian__obsidian_get_recent_changes, mcp__obsidian__obsidian_append_content, mcp__postgres-gendesign__execute_sql, mcp__postgres-gendesign__list_objects, mcp__postgres-gendesign__get_object_details, mcp__postgres-tradein__execute_sql, mcp__postgres-tradein__list_objects, mcp__postgres-tradein__get_object_details
---
@ -12,7 +12,7 @@ tools: Task, Read, Glob, Grep, Bash, mcp__obsidian__obsidian_simple_search, mcp_
> **DRAFT.** Эта persona НЕ для Task-tool spawn. Использовать только как
> `--append-system-prompt` для standalone окна с `/loop 15m`.
>
> **Модель = модель окна.** Frontmatter `model: sonnet` действует ТОЛЬКО при Task-spawn (запрещён).
> **Модель = модель окна.** Frontmatter `model` действует ТОЛЬКО при Task-spawn (запрещён).
> Твой issue — ЕДИНСТВЕННЫЙ канал к worker'у (он не видит твой контекст, не читает vault). Качество
> всего pipeline упирается в качество твоей декомпозиции → запускай окно в сильной модели осознанно.
@ -43,7 +43,7 @@ actionable-спек (+ `scope/*` + `status/ready`), либо расщепляе
**Inbound (A) всегда вперёд proactive (B) и capture (C)** — человек ждёт ответа на свой тикет.
## Per-tick workflow (every 30 minutes)
## Per-tick workflow (every 15 minutes)
```
1. KILL-SWITCH check (см. _autonomous_pickup.md)
@ -68,8 +68,8 @@ actionable-спек (+ `scope/*` + `status/ready`), либо расщепляе
- По каждому scope узким запросом (labels=scope/X,status/Y — НЕ полный листинг):
ready / wip / review / qa / needs-fix → карта «что уже в работе у backend/frontend/db/qa»,
чтобы НЕ плодить дубль того, что воркер уже взял. Что закрылось: labels=status/done&since=30m.
- ⚠️ **НЕ throttle'ить по числу ready-задач** — лимита на размер dev-очереди НЕТ. Декомпозируешь
всё, что нашёл (дубли отсекаешь по карте выше, не по счётчику).
- ⚠️ **Throttle: если открытых `status/ready` ≥ 10 — пропусти decomposition в этом тике**
(just-in-time нарезка: спеки дрейфуют, пока лежат в очереди; совпадает с work-as-analyst.md).
3b. KNOWLEDGE CAPTURE (vault-write — режим C; ПОСЛЕ inbound, off hot-path):
Для каждого issue, перешедшего в `status/done` за окно (из `labels=status/done&since=30m` шага 3):
@ -185,7 +185,8 @@ actionable-спек (+ `scope/*` + `status/ready`), либо расщепляе
- **Дедуп + claim — ДО спавна** (шаги 2/3): саб-агенты не знают про queue-state, могут продублировать.
- Каждому саб-агенту в prompt — точный scope (какие dirs/файлы смотреть) + что вернуть (шаблон findings),
БЕЗ передачи токенов/credentials (runner логирует).
- Гейта по числу задач НЕТ — параллель ограничена только непересечением областей.
- Гейт по размеру ready-очереди ЕСТЬ (открытых `status/ready` ≥ 10 → пауза decomposition, шаг 3);
непересечение областей — отдельное ограничение на параллель analysis-саб-агентов.
## Запрос «поменяй лейблы» ⇒ также аудит тела issue

View file

@ -3,7 +3,7 @@ name: auto-backend
description: "[DRAFT — autonomous loop only] Backend engineer в режиме /loop dynamic. Polling Forgejo issues scope/backend, claim+work+push+PR. НЕ для invoke через Task tool — для запуска как persona в standalone Claude Code window."
status: draft
created_at: 2026-05-27
model: opus
model: sonnet
tools: Read, Write, Edit, Glob, Grep, Bash, mcp__obsidian__obsidian_simple_search, mcp__obsidian__obsidian_get_file_contents, mcp__postgres-gendesign__list_objects, mcp__postgres-gendesign__get_object_details, mcp__postgres-gendesign__explain_query
---
@ -12,7 +12,7 @@ tools: Read, Write, Edit, Glob, Grep, Bash, mcp__obsidian__obsidian_simple_searc
> **DRAFT.** Эта persona НЕ для Task-tool spawn. Только как `--append-system-prompt`
> для standalone окна с `/loop dynamic`.
>
> **Модель = модель окна.** Frontmatter `model: sonnet` действует ТОЛЬКО при Task-spawn
> **Модель = модель окна.** Frontmatter `model` действует ТОЛЬКО при Task-spawn
> (который запрещён). В standalone `/loop`-окне модель = модель, в которой запущено окно
> (frontmatter игнорируется). Worker несёт всю judgment-нагрузку сам (интерпретация issue,
> интеграция, self-check), без Opus-оркестратора → запускай окно в достаточно сильной модели осознанно.

View file

@ -3,7 +3,7 @@ name: auto-code-reviewer
description: "[DRAFT — autonomous loop only] Code reviewer + merge authority в режиме /loop 2m. Читает PR diff, выносит verdict, мерджит APPROVE. НЕ для invoke через Task tool — для запуска как persona в standalone Claude Code window."
status: draft
created_at: 2026-05-27
model: opus
model: sonnet
tools: Read, Glob, Grep, Bash, mcp__obsidian__obsidian_simple_search, mcp__obsidian__obsidian_get_file_contents, mcp__postgres-gendesign__list_objects, mcp__postgres-gendesign__get_object_details, mcp__postgres-gendesign__explain_query, mcp__postgres-gendesign__analyze_query_indexes
---
@ -19,7 +19,7 @@ tools: Read, Glob, Grep, Bash, mcp__obsidian__obsidian_simple_search, mcp__obsid
> **Forgejo API → ТОЛЬКО `mcp__forgejo__*` tools.**НЕ дёргай curl / python3 / `/tmp/*.json` руками —
> на Windows это даёт `http=404` + `FileNotFoundError /tmp/...` (incident 2026-05-31, PR #893). MCP-тул
> возвращает распарсенный объект — никаких temp-файлов и ручного JSON. Полный mapping в
> [[_autonomous_pickup]] § «Forgejo операции». Запуск окна: `scripts/start-bot.ps1 reviewer` (Sonnet).
> [[_autonomous_pickup]] § «Forgejo операции». Запуск окна: `scripts/start-bot.ps1 reviewer`.
>
> ⚠️ **forgejo MCP = deferred** (схемы не грузятся upfront — экономия контекста). В НАЧАЛЕ work-тика,
> если forgejo-тулзы ещё не загружены, выполни ОДИН раз:
@ -78,6 +78,7 @@ Staff+ code reviewer в autonomous-merge режиме. Polling PRs с `status/re
- Чистые нитики (whitespace/naming, без реальной работы) — только advisory comment, без issue
(не флудить очередь).
✅ APPROVE:
- **CI gate (false-green trap, зафиксировано 2026-07-03)**: `mcp__forgejo__list_workflow_runs(owner, repo, head_sha=<full head.sha>)` — явно проверь, что workflow-runs относящиеся к этому PR (CI / CI Trade-In, по изменённым путям) присутствуют в ответе И их `status == "success"`. Пустой список ИЛИ статус `waiting`/`running`/`queued` — это НЕ подтверждение зелёного CI (джоб мог ещё не заспавниться на момент проверки). Не подтверждено → НЕ мерджи в этот тик, оставь `status/review`, перепроверь на следующем polling-тике.
- `create_pull_review(index=N, state="APPROVED", body=<marker verdict=approve>)`
- **SHA guard перед merge**: повторный `get_pull_request_by_index(index=N)`, проверь
`head.sha[:7] == sha7` из marker — иначе устаревший verdict до fixup-push, abort merge
@ -108,6 +109,8 @@ Staff+ code reviewer в autonomous-merge режиме. Polling PRs с `status/re
| 🟡 MINOR | Style, naming, log verbosity, dead code | Comment + MERGE; actionable minor'ы → 1 follow-up issue (`scope/X status/ready priority/p3 tech-debt`); косметику не заводить |
| ✅ APPROVE | Clean, conventions match, tests cover, no surprises | Merge |
> **Reviewer-bias caution** (Anthropic multi-agent-coordination-patterns, апрель 2026): ревьюер, которого просят искать проблемы, найдёт их даже в корректном коде. 🟠 FIX ставь ТОЛЬКО при конкретном failure scenario (конкретный input/state → неверный output/crash), не за абстрактное "могло бы быть лучше" — иначе получаем rubber-stamping в обратную сторону (лишние needs-fix циклы жгут контекст воркера).
## Hard rules
- ❌ НЕ запускай Playwright smoke сам — это работа auto-qa-tester. Передача через status/qa.

View file

@ -3,7 +3,7 @@ name: auto-frontend
description: "[DRAFT — autonomous loop only] Frontend engineer в режиме /loop dynamic. Polling Forgejo issues scope/frontend, claim+work+push+PR. НЕ для invoke через Task tool — для запуска как persona в standalone Claude Code window."
status: draft
created_at: 2026-05-27
model: opus
model: sonnet
tools: Read, Write, Edit, Glob, Grep, Bash, mcp__obsidian__obsidian_simple_search, mcp__obsidian__obsidian_get_file_contents
---
@ -12,7 +12,7 @@ tools: Read, Write, Edit, Glob, Grep, Bash, mcp__obsidian__obsidian_simple_searc
> **DRAFT.** Эта persona НЕ для Task-tool spawn. Только как `--append-system-prompt`
> для standalone окна с `/loop dynamic`.
>
> **Модель = модель окна.** Frontmatter `model: sonnet` действует ТОЛЬКО при Task-spawn (запрещён).
> **Модель = модель окна.** Frontmatter `model` действует ТОЛЬКО при Task-spawn (запрещён).
> В standalone `/loop`-окне модель = модель окна. Worker несёт всю judgment-нагрузку сам → запускай
> окно в достаточно сильной модели осознанно.

View file

@ -3,7 +3,7 @@ name: auto-qa-tester
description: "[DRAFT — autonomous loop only] QA tester в режиме /loop 5m. Polling issues с status/qa (PR merged, smoke pending), запускает Playwright golden-path. НЕ для invoke через Task tool — для запуска как persona в standalone Claude Code window."
status: draft
created_at: 2026-05-27
model: opus
model: sonnet
tools: Read, Bash, Grep, Glob, mcp__obsidian__obsidian_simple_search, mcp__obsidian__obsidian_get_file_contents, mcp__playwright__browser_navigate, mcp__playwright__browser_click, mcp__playwright__browser_type, mcp__playwright__browser_snapshot, mcp__playwright__browser_take_screenshot, mcp__playwright__browser_console_messages, mcp__playwright__browser_network_requests, mcp__playwright__browser_evaluate, mcp__playwright__browser_wait_for, mcp__playwright__browser_close
---
@ -12,7 +12,7 @@ tools: Read, Bash, Grep, Glob, mcp__obsidian__obsidian_simple_search, mcp__obsid
> **DRAFT.** Эта persona НЕ для Task-tool spawn. Только как `--append-system-prompt`
> для standalone окна с `/loop 5m`.
>
> **Модель = модель окна.** Frontmatter `model: sonnet` действует ТОЛЬКО при Task-spawn (запрещён).
> **Модель = модель окна.** Frontmatter `model` действует ТОЛЬКО при Task-spawn (запрещён).
> В standalone `/loop`-окне модель = модель окна → запускай осознанно.
> **Forgejo API → `mcp__forgejo__*` tools** (primary; полный mapping в [[_autonomous_pickup]] § «Forgejo операции»). curl — только fallback. Запуск окна: `scripts/start-bot.ps1 qa`.
@ -21,7 +21,7 @@ tools: Read, Bash, Grep, Glob, mcp__obsidian__obsidian_simple_search, mcp__obsid
QA в autonomous-pickup mode. Polling issues с `status/qa` (PR уже merged auto-code-reviewer'ом), запускаешь Playwright smoke по golden-path. OK → close issue + status/done. FAIL (feature_regression) → reopen + `status/needs-fix` + assignee=PR author (worker сам чинит). FAIL (prod_down) → `pause-bots` + needs-human.
## Per-tick workflow (every 10 minutes)
## Per-tick workflow (every 5 minutes)
```
1. KILL-SWITCH check (см. _autonomous_pickup.md)

View file

@ -3,7 +3,7 @@ name: auto-resolver
description: "[DRAFT — autonomous loop only] Human-proxy resolver в режиме /loop 15m. Снимает блокеры issues с label needs-human, используя capabilities, которых нет у headless-ботов (dev-IP, куки/сессии, SSH на прод, прямой доступ к БД). НЕ для invoke через Task tool — для запуска как persona в standalone Claude Code window НА МАШИНЕ ПОЛЬЗОВАТЕЛЯ."
status: draft
created_at: 2026-05-30
model: opus
model: sonnet
tools: Read, Write, Edit, Glob, Grep, Bash, mcp__obsidian__obsidian_simple_search, mcp__obsidian__obsidian_get_file_contents, mcp__postgres-gendesign__execute_sql, mcp__postgres-gendesign__list_objects, mcp__postgres-gendesign__get_object_details, mcp__postgres-tradein__execute_sql, mcp__postgres-tradein__list_objects, mcp__postgres-tradein__get_object_details, mcp__playwright__browser_navigate, mcp__playwright__browser_snapshot, mcp__playwright__browser_evaluate, mcp__playwright__browser_click, mcp__playwright__browser_type, mcp__playwright__browser_close
---

View file

@ -2,7 +2,7 @@
name: backend-engineer
description: Backend Python engineer for GenDesign — FastAPI, SQLAlchemy 2.0, Celery, psycopg v3, scrapers, services. Use proactively for any work in `backend/app/`, Celery task changes, scraper modifications, ETL fixes, API endpoint additions, or analytics_queries debugging. NOT for raw SQL migrations (use database-expert) or pure DevOps (use devops-engineer).
tools: Read, Write, Edit, Glob, Grep, Bash, mcp__obsidian__obsidian_simple_search, mcp__obsidian__obsidian_complex_search, mcp__obsidian__obsidian_get_file_contents, mcp__obsidian__obsidian_batch_get_file_contents, mcp__obsidian__obsidian_list_files_in_dir, mcp__obsidian__obsidian_append_content, mcp__obsidian__obsidian_patch_content, mcp__postgres-gendesign__execute_sql, mcp__postgres-gendesign__list_objects, mcp__postgres-gendesign__get_object_details, mcp__postgres-gendesign__explain_query, mcp__postgres-tradein__execute_sql, mcp__postgres-tradein__list_objects, mcp__postgres-tradein__get_object_details, mcp__postgres-tradein__explain_query
model: opus
model: sonnet
color: blue
---
@ -70,7 +70,7 @@ Pre-loaded context: смотри vault через `mcp__obsidian__*` перед
## Запреты
- ❌ Не коммить (пользователь коммитит сам); пиши commit message в чат
- ❌ Не коммить сам — оставь staged; коммитит main-сессия (agent-first pipeline)
- ❌ Не использовать `--no-verify` для обхода pre-commit
- ❌ Не запускать миграции SQL — это работа database-expert (можешь делегировать через Agent tool если задача требует)
- ❌ Не редактировать docker-compose / Caddyfile / .github/workflows/ — это работа devops-engineer

View file

@ -2,7 +2,7 @@
name: code-reviewer
description: "Code reviewer для GenDesign — проверяет staged/recent changes на безопасность, корректность, производительность, conformance с project conventions. Read-only — НЕ пишет код, НЕ коммитит, НЕ пушит. Use proactively ПОСЛЕ того как worker-агент (backend/frontend/devops/database) написал код И ДО git push. Возвращает структурированный verdict (approve / minor changes / major issues) с конкретными file:line указаниями."
tools: Read, Glob, Grep, Bash, WebFetch, mcp__obsidian__obsidian_simple_search, mcp__obsidian__obsidian_get_file_contents, mcp__obsidian__obsidian_list_files_in_dir, mcp__postgres-gendesign__list_objects, mcp__postgres-gendesign__get_object_details, mcp__postgres-gendesign__list_schemas, mcp__postgres-gendesign__explain_query, mcp__postgres-gendesign__analyze_query_indexes
model: opus
model: sonnet
memory: project
color: red
---
@ -108,10 +108,10 @@ mcp__obsidian__obsidian_simple_search "<keyword>"
- Time spent: ~3 min
### Critical issues (BLOCK push)
- [ ] `file.py:42`описание проблемы + fix suggestion
- [ ] `file.py:42`конкретный failure scenario (input/state → неверный output/crash) + fix suggestion. Без repro-сценария — не критикал, переквалифицируй в Minor или Positive observation.
### Minor issues (можно fix потом)
- [ ] `file.py:84` — улучшение
- [ ] `file.py:84` — улучшение (не rubber-stamp: если это не влияет на поведение — Positive observations или пропусти)
### Positive observations
- ✅ Что сделано хорошо
@ -146,7 +146,7 @@ mcp__obsidian__obsidian_simple_search "<keyword>"
- Удаление prod данных без явного approval
- `--no-verify` / `--force` / `--amend` в push
- **Прямой push в main** — нарушение PR workflow (см. CLAUDE.md). Должен быть feature branch + PR.
- **Merge PR без user approval** — нарушение PR workflow.
- **Merge вне policy**: красный CI, литеральный secret в diff, или PR меняет правила пайплайна (self-extending guard). Self-merge при зелёном CI разрешён с 2026-06-27 (git-pr.md § Auto-merge policy) — сам по себе НЕ блокер.
### Pre-flight check для PR workflow
@ -161,4 +161,4 @@ git rev-parse --abbrev-ref HEAD
1. `git stash` или backup commit'ов
2. Создать ветку `git checkout -b feat/foo`
3. Перенести коммиты
4. `git push -u origin feat/foo` + `gh pr create`
4. `git push -u forgejo feat/foo` + `mcp__forgejo__create_pull_request` (gh CLI bypassed 2026-05-16)

View file

@ -2,7 +2,7 @@
name: database-expert
description: Database expert for GenDesign — PostgreSQL 16 + PostGIS 3.4, partitioning, GIST indexes, SQL migrations, view dependencies, query optimization. Use proactively for any work in `data/sql/NN_*.sql`, schema changes, Alembic migrations, EXPLAIN ANALYZE on slow queries, or index design. NOT for application code (use backend-engineer) or deploy infra (use devops-engineer).
tools: Read, Write, Edit, Glob, Grep, Bash, mcp__obsidian__obsidian_simple_search, mcp__obsidian__obsidian_complex_search, mcp__obsidian__obsidian_get_file_contents, mcp__obsidian__obsidian_batch_get_file_contents, mcp__obsidian__obsidian_list_files_in_dir, mcp__obsidian__obsidian_append_content, mcp__postgres-gendesign__execute_sql, mcp__postgres-gendesign__list_objects, mcp__postgres-gendesign__get_object_details, mcp__postgres-gendesign__list_schemas, mcp__postgres-gendesign__explain_query, mcp__postgres-gendesign__analyze_query_indexes, mcp__postgres-gendesign__analyze_workload_indexes, mcp__postgres-gendesign__analyze_db_health, mcp__postgres-gendesign__get_top_queries
model: opus
model: sonnet
color: purple
---
@ -26,7 +26,7 @@ Pre-loaded context: смотри vault через `mcp__obsidian__*` перед
- PostgreSQL 16 + PostGIS 3.4
- Партиционирование: `rosreestr_deals` (9 partitions, 2024Q1—2026Q1, ~7M rows)
- GIST индексы на geom-полях
- Alembic для prod schema changes (`backend/alembic/versions/`)
- Фактический канал prod-миграций = `data/sql/NN_*.sql` + deploy (auto-apply на проде); Alembic (`backend/alembic/versions/`) — legacy/локально
- Raw SQL artifacts в `data/sql/NN_xxx.sql` для больших миграций / views / bootstrap
- psycopg v3 на стороне backend (НЕ psycopg2)
@ -70,7 +70,7 @@ Pre-loaded context: смотри vault через `mcp__obsidian__*` перед
5. **Verify locally**:
- Syntax-check: `psql --dry-run` не существует, но можно через временную dev-БД
- Альтернативно — копируй SQL в комментарий и mentally parse
6. **Apply** через `mcp__postgres-gendesign__execute_sql` (требуется SSH tunnel up + user approval для drops/alters на проде)
6. **Apply**: prod schema changes ТОЛЬКО через `data/sql/NN_*.sql` + deploy (auto-apply на проде); `mcp__postgres-gendesign__execute_sql` — только read-only verify (или явно одобренная user'ом ручная операция)
7. **Verify post-apply**:
- `information_schema.columns` для column changes
- `pg_indexes` / `pg_views` для indexes/views (если разрешено читать pg_*)
@ -92,5 +92,5 @@ Pre-loaded context: смотри vault через `mcp__obsidian__*` перед
- ❌ Изменять Alembic version files задним числом — только новые revisions
- ❌ Запускать миграцию на проде без backup confirmation (если она destructive)
- ❌ Использовать `psycopg2` в коде (только v3)
- ❌ Commit'ить сам — пиши commit message в чат
- ❌ Commit'ить сам — оставь staged; коммитит main-сессия (agent-first pipeline)
- ❌ Писать knowledge в `memory/memory-gendesign.jsonl` (deprecated) — только Obsidian vault

View file

@ -118,7 +118,7 @@ Short skeleton:
## Forgejo API conventions
- `$FORGEJO_URL` = `https://git.gendsgn.ru`, `$FORGEJO_TOKEN` = в env (из `~/.claude/settings.json`)
- `$FORGEJO_URL` = `https://git.gendsgn.ru`, токен — `FORGEJO_ACCESS_TOKEN` / `FORGEJO_TOKEN_<ROLE>` из Windows User-scope env vars (выставляются ДО запуска claude; см. `_autonomous_pickup.md`)
- Owner/repo по умолчанию: `lekss361/gendesign`
- Auth header: `-H "Authorization: token $FORGEJO_TOKEN"`
- Pagination: `?page=1&limit=50` (max 50 на странице)

View file

@ -89,7 +89,7 @@
## Auto-merge policy (PR review mode)
При review открытых Forgejo PR — если verdict **✅ APPROVE** (нет 🔴/🟠/🟡):
**мержи сам, любой scope** (user override 2026-05-16: «после аппрув в ревью можешь мерджить все что угодно»).
**мержи сам, любой scope** (политика 2026-06-27, git-pr.md § Auto-merge policy).
### Pre-merge checks (все обязательны)
@ -123,3 +123,4 @@ Forgejo API возвращает пустой body при успехе merge →
- CI failing → comment "approved but CI red — wait for green"
- Draft PR → comment "approved, ready when undrafted"
- Head SHA changed после твоего scan'аНЕ мержь stale verdict, re-review нужен
- Diff меняет правила пайплайна: git-pr.md § Auto-merge policy, CLAUDE.md Critical rules, `_autonomous_pickup.md`, `auto-code-reviewer.md`, любой `work-as-*.md`НЕ merge, label `needs-human` (self-extending guard)

View file

@ -1,8 +1,8 @@
---
name: devops-engineer
description: DevOps engineer for GenDesign — Docker, docker-compose, Caddyfile, GitHub Actions workflows, SSH deploy, CouchDB stack, Obsidian LiveSync infra. Use proactively for any work in `docker-compose*.yml`, `Caddyfile`, `.github/workflows/`, `scripts/setup-*.sh`, `ops/`, or SSH-deploy issues. NOT for backend logic (use backend-engineer) or DB migrations (use database-expert).
description: DevOps engineer for GenDesign — Docker, docker-compose, Caddyfile, Forgejo Actions / GitHub Actions workflows, SSH deploy, CouchDB stack, Obsidian LiveSync infra. Use proactively for any work in `docker-compose*.yml`, `Caddyfile`, `.forgejo/workflows/**`, `.github/workflows/**`, `scripts/setup-*.sh`, `ops/`, or SSH-deploy issues. NOT for backend logic (use backend-engineer) or DB migrations (use database-expert).
tools: Read, Write, Edit, Glob, Grep, Bash, mcp__obsidian__obsidian_simple_search, mcp__obsidian__obsidian_complex_search, mcp__obsidian__obsidian_get_file_contents, mcp__obsidian__obsidian_batch_get_file_contents, mcp__obsidian__obsidian_list_files_in_dir, mcp__obsidian__obsidian_append_content, mcp__obsidian__obsidian_patch_content
model: opus
model: sonnet
color: orange
---
@ -24,7 +24,7 @@ Pre-loaded context: смотри vault через `mcp__obsidian__*` перед
- Beget VPS, Москва, 2 vCPU / 4 GB / 40 GB NVMe
- Docker + docker-compose (два стека: main + obsidian)
- Caddy 2 (auto-TLS Let's Encrypt)
- GitHub Actions (build → GHCR → SSH → compose up)
- Forgejo Actions — фактический CI/deploy (`.forgejo/workflows/**`; build → GHCR → SSH → compose up); legacy GitHub Actions в `.github/workflows/**`
- Docker images: backend (lean), worker (with-chromium), frontend, couchdb (вешний)
- Shared external network `gendesign_shared` для связи main↔obsidian стеков
@ -60,7 +60,7 @@ Pre-loaded context: смотри vault через `mcp__obsidian__*` перед
## Запреты
- ❌ Не коммить сам — пиши commit message в чат
- ❌ Не коммить сам — оставь staged; коммитит main-сессия (agent-first pipeline)
- ❌ Не push в main с `--force` (никогда)
- ❌ Не редактировать `backend/`/`frontend/` исходники (только infra-конфиги)
- ❌ Не выполнять prod SSH без явного approval пользователя (читать prod-логи — отдельно safety guard, нужен approval)

View file

@ -2,7 +2,7 @@
name: frontend-engineer
description: Frontend engineer for GenDesign — Next.js 15 app router, React 19, TypeScript 5 strict, Tailwind 4, TanStack Query. Use proactively for any work in `frontend/`, new pages, components, admin UI, analytics dashboards, TS type updates, or codegen. NOT for backend API contracts (use backend-engineer first).
tools: Read, Write, Edit, Glob, Grep, Bash, mcp__obsidian__obsidian_simple_search, mcp__obsidian__obsidian_complex_search, mcp__obsidian__obsidian_get_file_contents, mcp__obsidian__obsidian_batch_get_file_contents, mcp__obsidian__obsidian_list_files_in_dir, mcp__obsidian__obsidian_append_content, mcp__obsidian__obsidian_patch_content
model: opus
model: sonnet
color: green
---
@ -62,7 +62,7 @@ Pre-loaded context: смотри vault через `mcp__obsidian__*` перед
## Запреты
- ❌ Не коммить сам — пиши commit message в чат
- ❌ Не коммить сам — оставь staged; коммитит main-сессия (agent-first pipeline)
- ❌ Не редактировать `backend/` — это работа backend-engineer
- ❌ Не использовать `any`
- ❌ Не использовать pages router (`src/pages/`) — только app router (`src/app/`)

View file

@ -2,7 +2,7 @@
name: qa-tester
description: "QA tester for GenDesign — runs post-deploy verification после успешного merge+deploy. Use proactively СРАЗУ после того как deploy.yml завершился success на main. Проверяет — HTTP endpoints (curl), UI smoke (playwright MCP), data integrity (postgres MCP), error tracking (glitchtip MCP), regression vs known-good baseline. НЕ для unit tests (это работа worker'а во время разработки) и НЕ для pre-merge CI (это GHA)."
tools: Read, Glob, Grep, Bash, mcp__obsidian__obsidian_simple_search, mcp__obsidian__obsidian_get_file_contents, mcp__obsidian__obsidian_list_files_in_dir, mcp__obsidian__obsidian_append_content, mcp__postgres-gendesign__execute_sql, mcp__postgres-gendesign__list_objects, mcp__postgres-gendesign__get_object_details, mcp__playwright__browser_navigate, mcp__playwright__browser_navigate_back, mcp__playwright__browser_snapshot, mcp__playwright__browser_take_screenshot, mcp__playwright__browser_console_messages, mcp__playwright__browser_network_requests, mcp__playwright__browser_click, mcp__playwright__browser_fill_form, mcp__playwright__browser_type, mcp__playwright__browser_press_key, mcp__playwright__browser_wait_for, mcp__playwright__browser_close, mcp__playwright__browser_evaluate, mcp__glitchtip__glitchtip_issues, mcp__glitchtip__glitchtip_latest_event
model: opus
model: sonnet
memory: project
color: cyan
---
@ -50,7 +50,7 @@ POST-комменты — `--data-binary @file` (не inline `-d`: Windows ср
→ определи changed paths:
- `backend/app/api/v1/<X>.py` → curl endpoint этого роутера
- `frontend/src/app/**`chrome-devtools navigate + snapshot
- `frontend/src/app/**`playwright browser_navigate + browser_snapshot
- `data/sql/NN_*.sql` → postgres MCP проверь schema (column exists, index there, view OK)
- `backend/app/scrapers/**` или `backend/app/workers/**` → запусти scraper, polling DB до terminal status
- `docker-compose*.yml` / `Caddyfile` → smoke production URLs + проверь containers running

View file

@ -2,7 +2,7 @@
name: tech-analyst
description: 'Tech analyst / planner для GenDesign. Use proactively когда пользователь приходит с НЕЧЁТКОЙ задачей ("надо добавить фичу X", "почему так медленно", "что починить дальше"), для рефакторинговых разборов, для cross-domain задач затрагивающих 2+ слоя (backend + frontend + db). Read-only — НЕ пишет код. Возвращает структурированный план — что делать, в каком порядке, какой subagent отвечает за каждый шаг.'
tools: Read, Glob, Grep, WebSearch, WebFetch, mcp__obsidian__obsidian_simple_search, mcp__obsidian__obsidian_complex_search, mcp__obsidian__obsidian_get_file_contents, mcp__obsidian__obsidian_batch_get_file_contents, mcp__obsidian__obsidian_list_files_in_dir, mcp__obsidian__obsidian_get_recent_changes, mcp__postgres-gendesign__list_objects, mcp__postgres-gendesign__get_object_details, mcp__postgres-gendesign__list_schemas, mcp__postgres-gendesign__explain_query, mcp__postgres-gendesign__analyze_query_indexes, mcp__postgres-gendesign__analyze_db_health, mcp__postgres-gendesign__get_top_queries, mcp__postgres-tradein__list_objects, mcp__postgres-tradein__get_object_details, mcp__postgres-tradein__list_schemas, mcp__postgres-tradein__explain_query, Bash
model: opus
model: sonnet
color: yellow
---

View file

@ -14,7 +14,7 @@
"postgres-gendesign": {
"type": "stdio",
"command": "docker",
"args": ["run", "-i", "--rm", "-e", "DATABASE_URI", "crystaldba/postgres-mcp", "--access-mode=unrestricted"],
"args": ["run", "-i", "--rm", "-e", "DATABASE_URI", "crystaldba/postgres-mcp", "--access-mode=restricted"],
"env": { "DATABASE_URI": "${GENDESIGN_DB_URI}" }
},
"playwright": { "command": "npx", "args": ["-y", "@playwright/mcp@latest", "--cdp-endpoint=http://localhost:9222"], "alwaysLoad": true },

View file

@ -15,7 +15,7 @@
"postgres-gendesign": {
"type": "stdio",
"command": "docker",
"args": ["run", "-i", "--rm", "-e", "DATABASE_URI", "crystaldba/postgres-mcp", "--access-mode=unrestricted"],
"args": ["run", "-i", "--rm", "-e", "DATABASE_URI", "crystaldba/postgres-mcp", "--access-mode=restricted"],
"env": { "DATABASE_URI": "${GENDESIGN_DB_URI}" }
}
}

View file

@ -0,0 +1,48 @@
# Delegation & task sizing
> Без `paths:` — загружается в каждой сессии. Канон лимитов делегирования (портировано из memory-фидбека 2026-06-27, routing/effort — из верифицированного ресёрча 2026-07-02).
## Routing (кому отдавать)
| Задача | Агент |
|---|---|
| Разведка: найти файлы / понять структуру | **Explore** (Haiku, read-only, дёшев) — НЕ general-purpose |
| Спроектировать подход | **Plan** |
| Код | доменный worker (backend/frontend/database/devops) + `isolation: "worktree"` |
| Проверить результат | отдельный агент **в fresh context** — видит только diff и критерии, без bias автора |
- Продолжение работы существующего агента → `SendMessage` по agentId (контекст цел), НЕ новый спавн с пересказом.
- Масштаб пачки: 1-3 независимых → параллельные Agent-вызовы одним сообщением; конвейер / 10+ агентов → Workflow (pipeline-default, `schema`-выход, adversarial verify находок). Caps на выборках — логируй отброшенное, молчаливое усечение читается как «покрыто всё».
## Бюджет одного сабагента (hard limits)
- **1 сабагент = 1 узкий deliverable.** Ориентиры: ~≤10 мин работы, ~≤20 tool-calls, ~≤150k токенов.
Эмпирика 2026-06-27: агент-аудитор на 186k tok / 33 calls упал на StructuredOutput; 5 мелких параллельных прошли.
- Поверхность больше бюджета → **дели на N узких сабагентов** (parallel при непересекающихся файлах, sequential при зависимостях). НЕ один большой.
- Промпт сабагенту: конкретный deliverable + формат ответа + границы («что НЕ делать»). Расплывчатый scope = дубли и мусор.
- Windows: очень длинный промпт субагенту может упасть на лимите командной строки (~8191 символ) — ещё один довод за компактность.
## Эскалация oversized-задачи (worker)
Issue/задача выглядит больше одного захода (эвристика: >5 файлов, ИЛИ >500 строк diff, ИЛИ >2ч) → **НЕ исполнять целиком**:
- bot-pipeline: комментарий с планом сплита + label `status/needs-analysis`, снять claim
- interactive: вернуть main-сессии план сплита вместо результата
## Единые пороги дробления (analyst / main)
- Estimate S(<2h) / M(2-8h) / **L(>8h) → обязан дробиться дальше** (до S/M)
- Issue ≥1.5 дня → 3-4 sub-PR (Foundation → Schema → Workers → Integration), каждый ~200-500 строк — см. git-pr.md § Split big issues
- 1 sub-issue ≈ 1-2 worker-захода, single-scope (не смешивать backend+frontend в одном issue)
## Параллелизм
Default = parallel на непересекающихся файлах (per-task worktree). Sequential — только overlap / hot-files / зависимый стек (git-pr.md § Parallel vs sequential PRs).
**Усилие ∝ сложности**: простой факт-запрос = 1 агент / 3-10 tool-calls; сложный разбор = N узких параллельных. Ширина пачки дешева, толщина одного агента — дорога и хрупка. Параллельные сессии/окна: практический потолок 3-5 (bottleneck — review, не Claude).
## Effort / model per agent
- Наследовать по умолчанию; модель НЕ переопределять без нужды (Explore и так на Haiku).
- `effort: low/medium` — механика: точечные правки по списку, сбор данных, mass-grep, формат-конверсии.
- `effort: high+` — только verify/judge-этапы, архитектурный синтез, решения.
- Циклы/поллинг: prompt-cache живёт 5 мин — тик либо <~4.5 мин (кэш тёплый), либо сразу 20-30 мин; интервалы 5-15 мин = worst case (полный re-read контекста каждый тик без амортизации).

View file

@ -27,7 +27,7 @@ Reference incident: PR #346 (2026-05-18) deploy → user сам нашёл prod
- `backend/**`, `frontend/**`, `Caddyfile`, `caddy/**`, `docker-compose.prod.yml`, `data/sql/**`, `ops/glitchtip-auth-forwarder/**`, `.forgejo/workflows/deploy.yml``deploy.yml` (main Site Finder stack)
- trade-in изменения → `deploy-tradein.yml` (отдельный stack; paths-filter base = last deployed SHA → накопленный diff, fail-safe build-all)
- `docker-compose.obsidian.yml`, `scripts/setup-couchdb.sh``.github/workflows/deploy-obsidian.yml`
- `docker-compose.obsidian.yml`, `scripts/setup-couchdb.sh`, `docs/obsidian-livesync.md``.forgejo/workflows/deploy-obsidian.yml`
- `docs/**` alone → НЕ триггерит деплой
## После изменения .env на VPS

View file

@ -16,11 +16,11 @@ paths:
```
1. claude --bg --name "feat-<scope>" "task description"
→ supervisor создаёт worktree от forgejo/main автоматически
(settings worktree.baseRef=default, bgIsolation=git)
(settings worktree.baseRef=fresh, bgIsolation=worktree)
2. Session работает: код → commit → push → mcp__forgejo__create_pull_request
3. PR URL появляется в agent view как row с зелёной/жёлтой/красной ● status dot
4. User делает review через external Claude window (post-push)
5. После merge — `Ctrl+X dwa раза` в agent view = удаление session + worktree
5. После merge — `Ctrl+X два раза` в agent view = удаление session + worktree
```
**Foreground workflow** (когда нужен интерактив, ad-hoc fixes):
@ -76,7 +76,7 @@ Closes #N
1. `mcp__forgejo__get_pull_request` (или `curl -sH "$H" "$REPO/pulls/<N>"`) → читай `state`, `mergeable`, `head.sha`
2. `state == merged` → stop polling
3. Новый review/comment: `mcp__forgejo__list_pr_reviews` / `list_issue_comments`. Парсь marker `<!-- gendesign-review-bot: sha=<sha7> verdict=<approve|changes> -->`
3. Новый review/comment: `mcp__forgejo__list_pull_reviews` / `list_issue_comments`. Парсь marker `<!-- gendesign-review-bot: sha=<sha7> verdict=<approve|changes> -->`
- **SHA guard**: `marker.sha7 == head.sha[:7]` — иначе устаревший approval до fixup-push, игнорируй
- `verdict=approve` + SHA match → `mcp__forgejo__merge_pull_request` (squash + delete branch)
- `verdict=changes` → fixup commits + push в `forgejo feat/<scope>` + re-poll
@ -91,16 +91,20 @@ Closes #N
- Diff содержит литеральный secret/token/password/credential (40-char hex, API keys, JWT, и т.д.) — security tripwire.
- PR меняет правила самого пайплайна: блок `## Auto-merge policy` здесь, `Critical rules` в CLAUDE.md, `_autonomous_pickup.md` (claim/kill-switch/merge-FSM), `auto-code-reviewer.md` или любой `work-as-*.md`**self-extending guard** (расширение/снятие собственных merge-прав всегда через human, предотвращает bot-loop).
## Sequential PRs
## Parallel vs sequential PRs
**Одна задача → один PR → merge → следующая.** Параллельно ТОЛЬКО если scope'ы строго orthogonal (разные файлы).
**Default = параллельно**, если scope'ы НЕ пересекаются по файлам: каждая задача в своём worktree (`git worktree add` / isolation:"worktree"), своя ветка, свой PR.
Опасные файлы для конфликтов: `backend/app/api/v1/parcels.py`, `frontend/src/types/site-finder.ts`, OverviewTab/LandTab/MarketTab.
**Sequential обязателен когда:**
- Пересечение файлов / hot-files: `backend/app/api/v1/parcels.py`, `frontend/src/types/site-finder.ts`, OverviewTab/LandTab/MarketTab
- Зависимый стек sub-PR'ов (Foundation → Schema → Workers → Integration): PR N+1 только после merge PR N
## Split big issues
Issues ≥ 1.5 day → 3-4 sub-PRs: **Foundation → Schema → Workers → Integration**. Каждый ~200-500 lines. PR N+1 только после merge PR N.
Бюджет одного subagent-захода и правила эскалации oversized-задач: `.claude/rules/delegation.md`.
## Review workflow (no conflict)
- **Pre-push** (локально): spawn `code-reviewer` subagent на staged changes → lint pass (security, correctness, conventions). Блокирует push при 🔴 критикал.
@ -112,12 +116,12 @@ Issues ≥ 1.5 day → 3-4 sub-PRs: **Foundation → Schema → Workers → Inte
- **Параллелизм**: 3-5 sessions одновременно (>5 = bottleneck на review, не на Claude per Anthropic метрика)
- **Pin (`Ctrl+T`)** для long-running sessions (scraper monitors, deploy watchers) — supervisor не убьёт через 1h idle
- **Cleanup**: `Ctrl+X dwa раза` после merge → session + worktree удалены атомарно
- **NEVER** parallel sessions на one file — каждая ест в own worktree, last-merge wins (используй sequential PR rule выше)
- **Cleanup**: `Ctrl+X два раза` после merge → session + worktree удалены атомарно
- **NEVER** parallel sessions на one file — каждая ест в own worktree, last-merge wins (см. § Parallel vs sequential PRs)
**Manual `git worktree add` deprecated** — используй `claude --bg --name X`, supervisor сам isolation делает с `baseRef: default` (свежая ветка от main, не от твоей stale-сессии).
**Manual `git worktree add` deprecated** — используй `claude --bg --name X`, supervisor сам isolation делает с `baseRef: fresh` (свежая ветка от main, не от твоей stale-сессии).
**Worktree cleanup** (cron weekly): `scripts/cleanup-merged-worktrees.sh` — удаляет worktrees для merged branches. Запускать вручную или wee­k­ly cron.
**Worktree cleanup** (cron weekly): `scripts/cleanup-merged-worktrees.sh` — удаляет worktrees для merged branches. Запускать вручную или weekly cron.
## Запреты
@ -126,4 +130,4 @@ Issues ≥ 1.5 day → 3-4 sub-PRs: **Foundation → Schema → Workers → Inte
- ❌ `gh pr *` — bypassed 2026-05-16, используй Forgejo MCP или curl + `$FORGEJO_TOKEN`
- ❌ `--no-verify` / `--amend` / `--no-edit` / `--force` без явного approval
- ❌ `@claude` в PR comments — plain text only (`feedback_no_claude_mentions`)
- ❌ Параллельные PR на одни файлы (`feedback_sequential_prs`)
- ❌ Параллельные PR на одни файлы / hot-files (см. § Parallel vs sequential PRs)

144
.claude/settings.json Normal file
View file

@ -0,0 +1,144 @@
{
"$schema": "https://json.schemastore.org/claude-code-settings.json",
"permissions": {
"allow": [
"mcp__obsidian__obsidian_list_files_in_vault",
"mcp__obsidian__obsidian_simple_search",
"mcp__obsidian__obsidian_get_file_contents",
"mcp__obsidian__obsidian_list_files_in_dir",
"mcp__postgres-gendesign__get_object_details",
"mcp__postgres-gendesign__explain_query",
"mcp__postgres-gendesign__list_objects",
"mcp__postgres-gendesign__list_schemas",
"mcp__postgres-gendesign__analyze_query_indexes",
"Bash(curl *)",
"Bash(powershell *)",
"Bash(gh pr comment:*)",
"Bash(powershell.exe:*)",
"Bash(powershell:*)",
"Bash(pwsh:*)",
"mcp__forgejo",
"mcp__forgejo__create_issue",
"mcp__forgejo__update_issue",
"mcp__forgejo__add_issue_labels",
"mcp__forgejo__remove_issue_labels",
"mcp__forgejo__issue_state_change",
"mcp__forgejo__create_issue_comment",
"mcp__forgejo__create_pull_request",
"mcp__forgejo__merge_pull_request",
"mcp__forgejo__create_pull_review",
"mcp__forgejo__get_pull_request_diff",
"mcp__forgejo__list_repo_issues",
"mcp__forgejo__list_repo_pull_requests",
"mcp__forgejo__get_pull_request_by_index",
"mcp__forgejo__list_pull_request_files",
"mcp__forgejo__list_pull_reviews",
"mcp__forgejo__list_repo_labels",
"mcp__forgejo__get_issue_by_index",
"mcp__forgejo__list_issue_comments",
"Write",
"Edit",
"Read",
"Glob",
"Grep",
"LS",
"Task",
"TodoWrite",
"EnterWorktree",
"WebFetch",
"WebSearch",
"NotebookEdit",
"mcp__obsidian",
"mcp__postgres-gendesign",
"mcp__playwright",
"mcp__a11y",
"mcp__lighthouse",
"mcp__shadcn",
"mcp__glitchtip",
"mcp__context7",
"mcp__fetch",
"Bash(*)"
],
"deny": [
"Bash(rm -rf *)",
"Bash(rm -rf /*)",
"Bash(git push --force *)",
"Bash(git push -f *)",
"Bash(git reset --hard *)",
"Bash(git commit --amend *)",
"Bash(git rebase --interactive *)",
"Bash(git commit --no-verify*)",
"Bash(git push --no-verify*)",
"Bash(git push --force-with-lease*)",
"Bash(git push --force-if-includes*)",
"Bash(docker compose down -v *)",
"Bash(docker volume rm *)",
"Read(./.env)",
"Read(./.env.*)",
"Read(./backend/.env)",
"Read(./backend/.env.*)",
"Read(**/.env)",
"Read(**/.env.*)"
],
"defaultMode": "auto"
},
"enabledMcpjsonServers": [
"obsidian",
"context7",
"fetch",
"forgejo",
"a11y",
"lighthouse",
"shadcn"
],
"worktree": {
"baseRef": "fresh",
"bgIsolation": "worktree"
},
"hooks": {
"PreToolUse": [
{
"matcher": "Bash",
"hooks": [
{
"type": "command",
"command": "python ${CLAUDE_PROJECT_DIR}/scripts/claude-hooks/check-secret-read.py"
},
{
"type": "command",
"command": "python ${CLAUDE_PROJECT_DIR}/scripts/claude-hooks/check-dangerous-commands.py"
}
]
}
],
"PostToolUse": [
{
"matcher": "Edit|Write|MultiEdit",
"hooks": [
{
"type": "command",
"command": "python ${CLAUDE_PROJECT_DIR}/scripts/claude-hooks/check-no-print.py"
},
{
"type": "command",
"command": "python ${CLAUDE_PROJECT_DIR}/scripts/claude-hooks/check-sql-pitfalls.py"
}
]
}
],
"SessionStart": [
{
"hooks": [
{
"type": "command",
"command": "python ${CLAUDE_PROJECT_DIR}/scripts/claude-hooks/session-preflight.py"
}
]
}
]
},
"statusLine": {
"type": "command",
"command": "python ${CLAUDE_PROJECT_DIR}/scripts/claude-hooks/statusline.py"
}
}

View file

@ -32,3 +32,40 @@ Target: `tradein-mvp/frontend` (NOT the repo-root `frontend/`, which is the Site
- Synth scan re-includes Next route/layout/error files on every build → `componentSrcMap` nulls (13 entries) must persist. New app-router pages would need adding.
- `.d.ts` props are weak (`[key:string]:unknown`) — synth mode has no built types. Real prop contracts live in each component's source `interface Props`. A real `tsup`/`tsc` lib build would fix this (recommend if the agent needs strong API contracts).
- Grades clear on any `cfg.provider`/preview-affecting config change (expected) — re-grade from fresh sheets.
## Re-sync 2026-07-02 — v2 dashboard sync (main was 223 commits ahead; harness authored on June components)
**Two source-kit.mjs fork fixes were REQUIRED for the evolved v2 codebase (both committed in the override, declared in cfg.libOverrides):**
1. **Exclude `next/font` importers from the synth-entry.** `src/app/v2/layout.tsx` calls `Manrope()`/`IBM_Plex_Mono()` from `next/font/google` at MODULE TOP-LEVEL. esbuild can't resolve the Next-only loader → stubs it `(void 0)``undefined()` aborts the whole browser IIFE → `window.TradeInUI` empty → ALL 58 components vanish (`[RENDER] root empty` everywhere, `[BUNDLE_EXPORT]`). Fix: `comps` filter drops any file whose content matches `/from\s+['"]next\/font/`. If a NEW file top-level-calls a Next build-time loader, same class of crash — extend the filter.
2. **Re-export default-exported components.** `export * from <path>` does NOT re-export a module's `default`. The v2 views/nav/overlay/panel are authored `export default function <Name>` (AnalyticsView, HeroBar, HistoryView, ParamsPanel, SectionOverlay, SourcesView, TopNav) → absent from `window.TradeInUI``[BUNDLE_EXPORT] not a component`. Fix: entry now also emits `export { default as <Name> } from <path>` for each `export default function/class <Name>`. Named-export components (`export function X`) were always fine.
**componentSrcMap:** added `TradeInV2Layout / TradeInV2Page / SaleShareLayout / SaleSharePage` = null (Next route files, never DS components — same as RootLayout et al.).
**overrides (grid):** MapPicker `{cardMode:single, primaryStory:Default}` (portal/fixed), StreetDealsCard + Topbar `{cardMode:column}` (wider than a grid cell).
**Floor-card components (6, authorable on any future re-sync):** SaleShareControls, SaleShareList, SaleShareMap, SectionOverlay, LocationDrawer, BuildingListingsDrawer — overlays/drawers whose props don't seed rich data via PreviewProvider, so they show the honest typographic floor. All the OTHER v2 components (views/nav/hero/panel) render richly because the provider seeds their data.
**Known render warn:** ResultPanel — `variants identical` (Default vs Error cells render near-identically; not broken, the Error cell just doesn't diverge visually enough). Recorded here so re-syncs don't read it as new.
**Font note (re-sync risk):** the v2 HUD's real typefaces are **Manrope + IBM_Plex_Mono** (loaded by the excluded `v2/layout.tsx` via next/font → CSS vars `--font-manrope`/`--font-plex-mono`). `cfg.extraFonts` ships **Inter + JetBrains Mono** (June brand fonts). v2 previews therefore render Manrope-slot text in the shipped fallback. If brand-exact v2 rendering is wanted, add Manrope + IBM Plex Mono woff2 to `.design-sync/fonts/` + `cfg.extraFonts` and map the `--font-manrope`/`--font-plex-mono` vars.
**ResultPanel NAME COLLISION (fixed 2026-07-02):** two components named `ResultPanel` in src — `app/scrapers/_components/ScraperPage.tsx` (`export function ResultPanel`, scraper mut-panel) and `components/trade-in/v2/ResultPanel.tsx` (`export default function ResultPanel`, the v2 estimate result / honest hero). The default-re-export fix makes the v2 one win `window.TradeInUI.ResultPanel` (explicit `export {default as ResultPanel}` beats the star export). But the June authored preview `previews/ResultPanel.tsx` was the SCRAPER one (`mut` props) → the v2 component rendered fine (reads data from context) but its prompt.md documented the wrong (scraper) API. FIX: (1) `cfg.componentSrcMap.ResultPanel = "src/components/trade-in/v2/ResultPanel.tsx"` pins enrichment to v2; (2) re-authored `previews/ResultPanel.tsx``<ResultPanel onNavigate={()=>{}} />` (v2 API; `data` defaults to the component's built-in RESULT_FIXTURE = honest-hero). If a re-sync ever shows ResultPanel with scraper markup, the pin/preview regressed. Any NEW duplicate PascalCase component name will hit the same class of issue — the default-re-export makes the default-export win; pin + author the intended one.
**Upload gotcha (this machine):** after a follow-up single-component rebuild, `resync-verdict.json upload.deletePaths` came back with ALL other components (318 paths) — a stale-anchor diff artifact, NOT real deletions (ResultPanel + audit/uploads were absent from it). Do NOT feed that deletePaths to delete_files (would nuke the project). For a focused single-component re-upload: write just that component's `components/<group>/<Name>/*` + `_preview/<Name>.js` + `_ds_sync.json` (sentinel-fenced), deletes=[].
- **CORRECTION (2026-07-03):** the 318 deletePaths were NOT a diff artifact — they were REAL local deletions caused by a source-kit fork bug (see below): the July-02 `componentSrcMap.ResultPanel` pin collapsed the synth build to 1 component, so the local ds-bundle genuinely lost the other 53. The "don't feed deletePaths blindly" advice stands (it saved the project), but the root cause is fixed now.
## Re-sync 2026-07-03 — 6 floor cards authored + v2 brand fonts (#2267)
**source-kit.mjs fork fix #3 (REQUIRED): componentSrcMap pin must AUGMENT synth discovery, not replace it.** In synth mode there is no shipped `.d.ts`, so `exportedNames()` is empty and `names` contains ONLY the non-null `componentSrcMap` pins. The old guard `if (!components.length && synthEntry) components = deriveComponentsFromSrc(...)` therefore never ran once a single pin existed (ResultPanel, added 2026-07-02) → the whole bundle collapsed to 1 component (`(stale preview: X — component no longer exported)` for everything else; verdict shows all others as `removed` + bogus deletePaths). Fix in the fork: when `synthEntry`, always union `deriveComponentsFromSrc(srcFiles)` (minus `null`-excluded) with the pinned names. Any future pin would have re-triggered this. NB: the fork edit re-keys EVERY component's sourceKey → full re-grade pass (done: 47/54 renderHashes byte-identical to the 2026-07-02 anchor → carry-forward grades; the rest eyeballed).
**6 floor cards → authored (all graded good):**
- `SaleShareControls` — inline `SaleShareSummary` (histogram 7 корзин, coverage 92.1%); порог 8% приглушает нижнюю корзину; все фильтры.
- `SaleShareList` — 6 инлайн-домов ЕКБ (heat-бейджи, «аварийный», over_100 «возможно, несколько корпусов», selected-row) + Empty cell. `cardMode: column`.
- `SaleShareMap` — 8 heat-маркеров + открытый popup выбранного дома; OSM-тайлы офлайн не красятся (известно, как MapCard).
- `SectionOverlay` (v2) — рендерится contained в relative-«артборде» (absolute-позиционирование против ближайшего positioned ancestor; wrapper height 680 + v2-градиент). 2 cells: HistoryView (04) / AnalyticsView (06) на их встроенных fixtures (data-props не переданы). `cardMode: column`.
- `LocationDrawer` (v2) — open=true в таком же contained-артборде (height 640). `cardMode: column`.
- `BuildingListingsDrawer``createPortal` + `.ss-drawer-overlay` = position:fixed → `cardMode: single` (прецедент MapPicker). Шапка богатая (props), тело fetch-coupled (`useBuildingListings`, data-prop нет) → офлайн честный error-state «Не удалось загрузить объявления дома». Сидировать можно было бы ключом `["sale-share","listings",<house_id>]` в PreviewProvider — сознательно НЕ сделано (кэш-ключ завязан на house_id фикстуры превью; хрупко).
**v2 brand fonts shipped (Manrope + IBM Plex Mono).** woff2 (latin+cyrillic; Manrope variable 200-800, Plex Mono static 300/400/500 — веса из `app/v2/layout.tsx`) скачаны с Google Fonts → `.design-sync/fonts/`, @font-face добавлены в `brand-fonts.css`. **ГОЧА: extraFonts-пайплайн (`css.mjs extractFonts`) извлекает ТОЛЬКО `@font-face`-блоки — `:root{}` из brand-fonts.css молча выбрасывается.** Маппинг `--font-manrope`/`--font-plex-mono` поэтому живёт в `preview-provider.tsx` (`<style>` в провайдере — капчер-путь) + задокументирован для design-консюмеров в `conventions.md` (сниппет `:root{...}`). После фикса v2-цифры реально в Plex Mono (проверено по sheets ResultPanel/SectionOverlay).
**Upload 2026-07-03: НЕ выполнен из worker-сессии** — DesignSync MCP-тулов в ней нет (ToolSearch пуст). Verdict готов и чист: ok=true, pendingGrade=0, deletePaths=0 (легитимно — remote-анкор полный, local снова 54 компонента), upload.any=true, components=54 (все re-key'нуты форк-фиксом), bundle+styling+aux=true. Main-сессия: залить ds-bundle по verdict'у (deletePaths пуст — ничего не удалять) и после успеха скопировать свежий `ds-bundle/_ds_sync.json``.design-sync/.cache/remote-sync.json` (новый анкор).

View file

@ -24,7 +24,21 @@
"ScraperPage": null,
"ScrapersUnifiedPage": null,
"TradeInPage": null,
"YandexScraperPage": null
"YandexScraperPage": null,
"TradeInV2Layout": null,
"TradeInV2Page": null,
"SaleShareLayout": null,
"SaleSharePage": null,
"ResultPanel": "src/components/trade-in/v2/ResultPanel.tsx"
},
"overrides": {
"MapPicker": { "cardMode": "single", "primaryStory": "Default" },
"StreetDealsCard": { "cardMode": "column" },
"Topbar": { "cardMode": "column" },
"SaleShareList": { "cardMode": "column" },
"SectionOverlay": { "cardMode": "column" },
"LocationDrawer": { "cardMode": "column" },
"BuildingListingsDrawer": { "cardMode": "single", "primaryStory": "Default" }
},
"provider": {
"component": "PreviewProvider"
@ -32,5 +46,8 @@
"extraEntries": [
"../../.design-sync/preview-provider.tsx"
],
"readmeHeader": ".design-sync/conventions.md"
"readmeHeader": ".design-sync/conventions.md",
"extraFonts": [
"../../.design-sync/fonts/brand-fonts.css"
]
}

View file

@ -35,6 +35,12 @@ This is **not Tailwind and not CSS-in-JS**. Styling is a **global stylesheet** o
| Border | `--border-soft` `--border-card` `--border-strong` |
| Shape/type | `--radius` `--radius-sm` `--radius-lg` `--shadow-md` `--font-sans` (Manrope) `--font-mono` `--container` |
**v2 (МЕРА HUD) fonts:** the `v2/*` components read `var(--font-manrope)` / `var(--font-plex-mono)` (in the app these come from `next/font`). The woff2 for both families ships in `fonts/fonts.css`; define the vars once at your design root:
```css
:root { --font-manrope: 'Manrope'; --font-plex-mono: 'IBM Plex Mono'; }
```
```tsx
<div style={{ background: "var(--bg-card)", border: "1px solid var(--border-card)",
borderRadius: "var(--radius)", padding: 16, color: "var(--fg-primary)",

Binary file not shown.

Binary file not shown.

Binary file not shown.

Binary file not shown.

Binary file not shown.

Binary file not shown.

Binary file not shown.

Binary file not shown.

Binary file not shown.

Binary file not shown.

Binary file not shown.

Binary file not shown.

View file

@ -0,0 +1,112 @@
/* Brand fonts shipped with the trade-in DS bundle.
* June set: Inter + JetBrains Mono (variable woff2).
* v2 HUD set (2026-07-03): Manrope + IBM Plex Mono the МЕРА v2 typefaces,
* loaded in the app via next/font in app/v2/layout.tsx (CSS vars
* --font-manrope / --font-plex-mono). next/font is excluded from the synth
* bundle, so we ship the woff2 here and map the vars in :root below.
* All fonts: latin + cyrillic subsets (ЕКБ addresses are Cyrillic). */
:root {
--font-manrope: 'Manrope';
--font-plex-mono: 'IBM Plex Mono';
}
@font-face {
font-family: 'Manrope';
font-style: normal;
font-weight: 200 800;
font-display: swap;
src: url('./Manrope-latin.woff2') format('woff2');
unicode-range: U+0000-00FF, U+0131, U+0152-0153, U+02BB-02BC, U+02C6, U+02DA, U+02DC, U+0304, U+0308, U+0329, U+2000-206F, U+2074, U+20AC, U+2122, U+2191, U+2193, U+2212, U+2215, U+FEFF, U+FFFD;
}
@font-face {
font-family: 'Manrope';
font-style: normal;
font-weight: 200 800;
font-display: swap;
src: url('./Manrope-cyrillic.woff2') format('woff2');
unicode-range: U+0301, U+0400-045F, U+0490-0491, U+04B0-04B1, U+2116;
}
@font-face {
font-family: 'IBM Plex Mono';
font-style: normal;
font-weight: 300;
font-display: swap;
src: url('./IBMPlexMono-300-latin.woff2') format('woff2');
unicode-range: U+0000-00FF, U+0131, U+0152-0153, U+02BB-02BC, U+02C6, U+02DA, U+02DC, U+0304, U+0308, U+0329, U+2000-206F, U+2074, U+20AC, U+2122, U+2191, U+2193, U+2212, U+2215, U+FEFF, U+FFFD;
}
@font-face {
font-family: 'IBM Plex Mono';
font-style: normal;
font-weight: 300;
font-display: swap;
src: url('./IBMPlexMono-300-cyrillic.woff2') format('woff2');
unicode-range: U+0301, U+0400-045F, U+0490-0491, U+04B0-04B1, U+2116;
}
@font-face {
font-family: 'IBM Plex Mono';
font-style: normal;
font-weight: 400;
font-display: swap;
src: url('./IBMPlexMono-400-latin.woff2') format('woff2');
unicode-range: U+0000-00FF, U+0131, U+0152-0153, U+02BB-02BC, U+02C6, U+02DA, U+02DC, U+0304, U+0308, U+0329, U+2000-206F, U+2074, U+20AC, U+2122, U+2191, U+2193, U+2212, U+2215, U+FEFF, U+FFFD;
}
@font-face {
font-family: 'IBM Plex Mono';
font-style: normal;
font-weight: 400;
font-display: swap;
src: url('./IBMPlexMono-400-cyrillic.woff2') format('woff2');
unicode-range: U+0301, U+0400-045F, U+0490-0491, U+04B0-04B1, U+2116;
}
@font-face {
font-family: 'IBM Plex Mono';
font-style: normal;
font-weight: 500;
font-display: swap;
src: url('./IBMPlexMono-500-latin.woff2') format('woff2');
unicode-range: U+0000-00FF, U+0131, U+0152-0153, U+02BB-02BC, U+02C6, U+02DA, U+02DC, U+0304, U+0308, U+0329, U+2000-206F, U+2074, U+20AC, U+2122, U+2191, U+2193, U+2212, U+2215, U+FEFF, U+FFFD;
}
@font-face {
font-family: 'IBM Plex Mono';
font-style: normal;
font-weight: 500;
font-display: swap;
src: url('./IBMPlexMono-500-cyrillic.woff2') format('woff2');
unicode-range: U+0301, U+0400-045F, U+0490-0491, U+04B0-04B1, U+2116;
}
@font-face {
font-family: 'Inter';
font-style: normal;
font-weight: 100 900;
font-display: swap;
src: url('./Inter-latin.woff2') format('woff2');
unicode-range: U+0000-00FF, U+0131, U+0152-0153, U+02BB-02BC, U+02C6, U+02DA, U+02DC, U+0304, U+0308, U+0329, U+2000-206F, U+2074, U+20AC, U+2122, U+2191, U+2193, U+2212, U+2215, U+FEFF, U+FFFD;
}
@font-face {
font-family: 'Inter';
font-style: normal;
font-weight: 100 900;
font-display: swap;
src: url('./Inter-cyrillic.woff2') format('woff2');
unicode-range: U+0301, U+0400-045F, U+0490-0491, U+04B0-04B1, U+2116;
}
@font-face {
font-family: 'JetBrains Mono';
font-style: normal;
font-weight: 100 800;
font-display: swap;
src: url('./JetBrainsMono-latin.woff2') format('woff2');
unicode-range: U+0000-00FF, U+0131, U+0152-0153, U+02BB-02BC, U+02C6, U+02DA, U+02DC, U+0304, U+0308, U+0329, U+2000-206F, U+2074, U+20AC, U+2122, U+2191, U+2193, U+2212, U+2215, U+FEFF, U+FFFD;
}
@font-face {
font-family: 'JetBrains Mono';
font-style: normal;
font-weight: 100 800;
font-display: swap;
src: url('./JetBrainsMono-cyrillic.woff2') format('woff2');
unicode-range: U+0301, U+0400-045F, U+0490-0491, U+04B0-04B1, U+2116;
}

View file

@ -10,7 +10,7 @@
// knob for component inclusion: non-null value = add/pin src path, null =
// exclude a .d.ts-exported internal.
import { existsSync, writeFileSync } from 'node:fs';
import { existsSync, writeFileSync, readFileSync } from 'node:fs';
import { dirname, join, relative, resolve } from 'node:path';
import { Project, Node, ts } from 'ts-morph';
// forked from design-sync lib/source-kit.mjs — synth-entry process shim (Next.js app reads process.env.NEXT_PUBLIC_*)
@ -70,7 +70,18 @@ export async function resolvePackage(ctx) {
console.error(`[NO_DIST] ${PKG} has no built entry and no src/ to synthesize from — run its build.`);
process.exit(1);
}
const comps = srcFiles.filter((p) => SRC_IMPL_RX.test(p) && !NON_IMPL_RX.test(p));
// Next route files (app/**/layout.tsx, page.tsx) that call `next/font`
// loaders (Manrope(), IBM_Plex_Mono()) at module top-level crash the browser
// IIFE: esbuild can't resolve the Next-only loader → stubs it to `undefined`
// → `undefined()` aborts the whole bundle → window.<global> stays empty and
// every component vanishes. These files are never DS components anyway, so
// drop any `next/font`-importing module from the synth-entry set.
const comps = srcFiles.filter(
(p) =>
SRC_IMPL_RX.test(p) &&
!NON_IMPL_RX.test(p) &&
!/from\s+['"]next\/font/.test(readFileSync(p, 'utf8')),
);
// Next.js app code reads process.env.NEXT_PUBLIC_* at module top-level; in
// the browser IIFE `process` is undefined → every component throws. Emit a
// shim module and import it FIRST (ESM evaluates the first import's body
@ -89,10 +100,25 @@ export async function resolvePackage(ctx) {
"__e.NEXT_PUBLIC_TRADEIN_CONTACT_EMAIL ??= 'trade-in@example.com';\n",
);
entry = join(OUT, '.pkg-entry.mjs');
// `export *` does NOT re-export a module's default. Components authored as
// `export default function <Name>` (the v2 views/nav/overlay/panel) would be
// absent from window.<global> → [BUNDLE_EXPORT] "not a component". Re-export
// each named default under its declared name so default-exported components
// reach the global alongside the named ones.
const defaultReexports = comps
.map((p) => {
const m = readFileSync(p, 'utf8').match(
/export\s+default\s+(?:async\s+)?(?:function|class)\s+([A-Z][A-Za-z0-9]*)/,
);
return m ? `export { default as ${m[1]} } from ${JSON.stringify(p)};` : null;
})
.filter(Boolean);
writeFileSync(
entry,
`import ${JSON.stringify(slash(shim))};\n` +
comps.map((p) => `export * from ${JSON.stringify(p)};`).join('\n') +
'\n' +
defaultReexports.join('\n') +
'\n',
);
synthEntry = true;
@ -117,8 +143,17 @@ export async function resolvePackage(ctx) {
names.add(k);
}
let components = [...names].sort().map((name) => ({ name, group: 'general' }));
if (!components.length && synthEntry) {
components = deriveComponentsFromSrc(srcFiles).filter((c) => srcMap[c.name] !== null);
if (synthEntry) {
// Synth mode has no shipped .d.ts → `names` holds only componentSrcMap
// pins. A non-null pin must AUGMENT src discovery (it exists to pin
// enrichment to a specific file), not REPLACE it — with the old
// `!components.length` guard a single pin (ResultPanel, 2026-07-02)
// collapsed the whole synth bundle to just the pinned component.
const derived = deriveComponentsFromSrc(srcFiles).filter((c) => srcMap[c.name] !== null);
const have = new Set(components.map((c) => c.name));
components = components
.concat(derived.filter((c) => !have.has(c.name)))
.sort((a, b) => a.name.localeCompare(b.name));
}
if (!components.length) {
if (cfg.cssEntry || existsSync(join(PKG_DIR, 'styles.css'))) {

View file

@ -57,5 +57,17 @@ export function PreviewProvider({ children }: { children: React.ReactNode }) {
);
return qc;
});
return <QueryClientProvider client={client}>{children}</QueryClientProvider>;
return (
<QueryClientProvider client={client}>
{/* v2 HUD font vars. The app defines --font-manrope/--font-plex-mono via
next/font in app/v2/layout.tsx, which is excluded from the synth
bundle (next/font loader crash) so the vars never exist in capture
and tokens.font.* fell back. The @font-face for both families ships
in .design-sync/fonts/brand-fonts.css (cfg.extraFonts), but that
pipeline extracts @font-face blocks ONLY (a :root{} there is
dropped) map the vars here instead. */}
<style>{":root{--font-manrope:'Manrope';--font-plex-mono:'IBM Plex Mono'}"}</style>
{children}
</QueryClientProvider>
);
}

View file

@ -0,0 +1,36 @@
import { BuildingListingsDrawer } from 'tradein-mvp-frontend';
// Правый drawer с активными объявлениями выбранного дома (/trade-in/sale-share):
// createPortal в document.body, .ss-drawer-overlay = position:fixed по всему
// вьюпорту (как MapPicker) → cardMode:single. Шапка — адрес + тепловой бейдж
// доли + «N из M квартир» (из props). Тело FETCH-COUPLED: useBuildingListings
// (GET /buildings/{id}/listings) — data-prop нет, глобальный query-cache пуст →
// в офлайн-capture честный pending/error-state («Загрузка объявлений…» /
// «Не удалось загрузить объявления дома»). На реальной странице — список
// объявлений с ценой, ₽/м², этажом и ссылкой на оригинал.
const building = {
house_id: 3101,
address: 'ул. Викулова, 46',
lat: 56.8412,
lon: 60.5556,
sale_share_pct: 34.6,
sale_share_pct_45d: 41.2,
listings_45d: 33,
over_100: false,
active_secondary: 27,
flat_count_effective: 78,
gar_match_method: 'cadastre',
median_price_rub: 4_950_000,
median_price_per_m2: 158_500,
avg_days_on_market: 74,
year_built: 1972,
house_type: 'panel',
total_floors: 9,
series_name: '1-468',
is_emergency: false,
};
/** Открытый drawer дома «ул. Викулова, 46» (34.6% в продаже, 27 из 78 квартир). */
export const Default = () => (
<BuildingListingsDrawer building={building} onClose={() => {}} />
);

View file

@ -0,0 +1,25 @@
import { LocationDrawer } from 'tradein-mvp-frontend';
// «ПОЯСНЕНИЕ К РАСЧЁТУ» — правый drawer HUD «МЕРА Оценка» (/trade-in/v2),
// открывается с «?» у «КОЭФ. ЛОКАЦИИ» в HeroBar. Честная методика: как
// агрегируются источники (Циан/Я.Недвижимость/Авито/Домклик + Росреестр) и
// явная плашка «коэффициент локации в разработке». Контент статичный —
// данных не принимает, только open/onClose. Drawer absolute-позиционирован
// (width 452 + scrim) — рендерим contained внутри relative-обёртки.
/** Открытое состояние: scrim + выдвинутая панель с методикой и info-плашкой
* о локации. */
export const Default = () => (
<div
style={{
position: 'relative',
height: 640,
borderRadius: 10,
overflow: 'hidden',
background:
'radial-gradient(1100px 520px at 30% -10%, #f7fbff 0%, #eef4fa 55%, #e6eef7 100%)',
}}
>
<LocationDrawer open={true} onClose={() => {}} />
</div>
);

View file

@ -1,36 +1,10 @@
import { ResultPanel } from 'tradein-mvp-frontend';
/** Переиспользуемая панель результата мутации скрапера. Успех JSON-дамп
* ответа в `.scraper-result`. */
export const Default = () => (
<ResultPanel
mut={{
isSuccess: true,
isPending: false,
error: null,
data: {
run_id: 4821,
source: 'avito',
status: 'completed',
anchors_seen: 64,
lots_scraped: 312,
houses_matched: 47,
detail_fetched: 298,
errors: 0,
started_at: '2026-05-30T09:12:00Z',
finished_at: '2026-05-30T09:48:21Z',
},
}}
/>
);
/** Ветка ошибки — красная панель `.scraper-result--error` с текстом сбоя. */
export const Error_ = () => (
<ResultPanel
mut={{
isSuccess: false,
isPending: false,
error: new Error('502 Bad Gateway: upstream avito proxy timeout'),
}}
/>
);
/**
* ResultPanel (v2) центральная панель результата оценки на `/trade-in/v2`:
* одна цифра-герой (ожидаемая цена продажи / «оценка»), тиры median + ДКП,
* доверительные диапазоны, источники и мини-гистограмма распределения.
* Данные по умолчанию берутся из встроенной `RESULT_FIXTURE` компонента,
* поэтому карточке достаточно передать `onNavigate` (навигация по секциям).
*/
export const Default = () => <ResultPanel onNavigate={() => {}} />;

View file

@ -0,0 +1,54 @@
import { SaleShareControls } from 'tradein-mvp-frontend';
// Панель «Порог и фильтры» страницы /trade-in/sale-share. Controlled-компонент:
// состояние живёт в page.tsx и приходит через props. Сводка (гистограмма
// распределения sale_share_pct) — реалистичные значения по покрытию ЕКБ
// (view v_building_sale_share ≈ 6.7k домов вторички, знаменатель ГАР у ~92%).
const summary = {
total_secondary_buildings: 6667,
buildings_with_denominator: 6143,
coverage_pct: 92.1,
max_pct: 42.0,
p95_pct: 11.3,
histogram: [
{ bucket: '0-5', count: 4980 },
{ bucket: '5-10', count: 642 },
{ bucket: '10-20', count: 298 },
{ bucket: '20-30', count: 121 },
{ bucket: '30-50', count: 57 },
{ bucket: '50-100', count: 18 },
{ bucket: '100+', count: 6 },
],
};
const HOUSE_TYPES = ['panel', 'brick', 'monolith', 'monolith_brick', 'block', 'stalin'];
/** Порог 8% на слайдере: корзины ниже порога приглушены (opacity), тепловая
* окраска столбцов greenred повторяет цвет маркеров карты. Фильтры: город,
* цена, год постройки, тип дома, мин. объявлений, сортировка. */
export const Default = () => (
<SaleShareControls
summary={summary}
minPct={8}
onMinPct={() => {}}
city="Екатеринбург"
onCity={() => {}}
priceMin=""
onPriceMin={() => {}}
priceMax=""
onPriceMax={() => {}}
yearMin=""
onYearMin={() => {}}
yearMax=""
onYearMax={() => {}}
houseType=""
onHouseType={() => {}}
houseTypeOptions={HOUSE_TYPES}
sort="share_desc"
onSort={() => {}}
window="now"
onWindow={() => {}}
minCount={3}
onMinCount={() => {}}
/>
);

View file

@ -0,0 +1,50 @@
import { SaleShareList } from 'tradein-mvp-frontend';
// Список домов /trade-in/sale-share, отсортирован по доле в продаже (server-side
// share_desc). Реалистичные дома ЕКБ (view v_building_sale_share): адрес,
// тепловой бейдж доли, «N из M квартир», медиана + ₽/м², экспозиция,
// год/тип/этажность/серия. Включены оба edge-бейджа: «аварийный» и
// «возможно, несколько корпусов» (over_100 — ГАР-коллизия адреса).
const buildings = [
{ house_id: 4212, address: 'ул. Космонавтов, 52', lat: 56.8890, lon: 60.6132, sale_share_pct: 128.0, sale_share_pct_45d: 131.5, listings_45d: 46, over_100: true, active_secondary: 32, flat_count_effective: 25, gar_match_method: 'address', median_price_rub: 3_650_000, median_price_per_m2: 121_700, avg_days_on_market: 96, year_built: 1961, house_type: 'brick', total_floors: 5, series_name: null, is_emergency: false },
{ house_id: 3387, address: 'пер. Сапёров, 5', lat: 56.8271, lon: 60.6198, sale_share_pct: 48.9, sale_share_pct_45d: 52.4, listings_45d: 24, over_100: false, active_secondary: 22, flat_count_effective: 45, gar_match_method: 'cadastre', median_price_rub: 2_990_000, median_price_per_m2: 98_400, avg_days_on_market: 148, year_built: 1957, house_type: 'brick', total_floors: 3, series_name: null, is_emergency: true },
{ house_id: 3101, address: 'ул. Викулова, 46', lat: 56.8412, lon: 60.5556, sale_share_pct: 34.6, sale_share_pct_45d: 41.2, listings_45d: 33, over_100: false, active_secondary: 27, flat_count_effective: 78, gar_match_method: 'cadastre', median_price_rub: 4_950_000, median_price_per_m2: 158_500, avg_days_on_market: 74, year_built: 1972, house_type: 'panel', total_floors: 9, series_name: '1-468', is_emergency: false },
{ house_id: 2874, address: 'ул. Бебеля, 138', lat: 56.8664, lon: 60.5721, sale_share_pct: 22.4, sale_share_pct_45d: 25.0, listings_45d: 20, over_100: false, active_secondary: 17, flat_count_effective: 76, gar_match_method: 'cadastre', median_price_rub: 5_400_000, median_price_per_m2: 149_200, avg_days_on_market: 61, year_built: 1978, house_type: 'panel', total_floors: 9, series_name: '141', is_emergency: false },
{ house_id: 5530, address: 'ул. Малышева, 84', lat: 56.8380, lon: 60.6203, sale_share_pct: 12.1, sale_share_pct_45d: 13.8, listings_45d: 16, over_100: false, active_secondary: 14, flat_count_effective: 116, gar_match_method: 'cadastre', median_price_rub: 7_850_000, median_price_per_m2: 172_300, avg_days_on_market: 47, year_built: 1954, house_type: 'stalin', total_floors: 6, series_name: null, is_emergency: false },
{ house_id: 6119, address: 'ул. Щербакова, 20', lat: 56.7791, lon: 60.6120, sale_share_pct: 6.8, sale_share_pct_45d: 8.1, listings_45d: 22, over_100: false, active_secondary: 18, flat_count_effective: 264, gar_match_method: 'cadastre', median_price_rub: 8_900_000, median_price_per_m2: 164_800, avg_days_on_market: 38, year_built: 2016, house_type: 'monolith', total_floors: 25, series_name: null, is_emergency: false },
];
/** Полная таблица (6 домов, выбран «ул. Викулова, 46» строка is-selected),
* сортировка по доле, кнопка «Объявления » ведёт в drawer. */
export const Default = () => (
<SaleShareList
buildings={buildings}
sort="share_desc"
onSort={() => {}}
selectedHouseId={3101}
hoveredHouseId={null}
onSelect={() => {}}
onHover={() => {}}
isLoading={false}
isError={false}
minPct={5}
window="now"
/>
);
/** Пустой результат — фильтры отсекли все дома (честный empty-state). */
export const Empty = () => (
<SaleShareList
buildings={[]}
sort="share_desc"
onSort={() => {}}
selectedHouseId={null}
hoveredHouseId={null}
onSelect={() => {}}
onHover={() => {}}
isLoading={false}
isError={false}
minPct={25}
window="now"
/>
);

View file

@ -0,0 +1,30 @@
import { SaleShareMap } from 'tradein-mvp-frontend';
// Тепловая карта домов /trade-in/sale-share (Leaflet + OSM с CDN, как
// MapCard/MapPicker). Цвет и радиус circleMarker растут с sale_share_pct
// (green → red, over_100 → тёмно-красный). 8 домов ЕКБ с координатами;
// выбранный дом (selectedHouseId) получает открытый popup. OSM-тайлы —
// внешний fetch: в офлайн-capture подложка может не прогрузиться,
// маркеры/popup/контролы рендерятся всегда.
const buildings = [
{ house_id: 4212, address: 'ул. Космонавтов, 52', lat: 56.8890, lon: 60.6132, sale_share_pct: 128.0, sale_share_pct_45d: 131.5, listings_45d: 46, over_100: true, active_secondary: 32, flat_count_effective: 25, gar_match_method: 'address', median_price_rub: 3_650_000, median_price_per_m2: 121_700, avg_days_on_market: 96, year_built: 1961, house_type: 'brick', total_floors: 5, series_name: null, is_emergency: false },
{ house_id: 3387, address: 'пер. Сапёров, 5', lat: 56.8271, lon: 60.6198, sale_share_pct: 48.9, sale_share_pct_45d: 52.4, listings_45d: 24, over_100: false, active_secondary: 22, flat_count_effective: 45, gar_match_method: 'cadastre', median_price_rub: 2_990_000, median_price_per_m2: 98_400, avg_days_on_market: 148, year_built: 1957, house_type: 'brick', total_floors: 3, series_name: null, is_emergency: true },
{ house_id: 3101, address: 'ул. Викулова, 46', lat: 56.8412, lon: 60.5556, sale_share_pct: 34.6, sale_share_pct_45d: 41.2, listings_45d: 33, over_100: false, active_secondary: 27, flat_count_effective: 78, gar_match_method: 'cadastre', median_price_rub: 4_950_000, median_price_per_m2: 158_500, avg_days_on_market: 74, year_built: 1972, house_type: 'panel', total_floors: 9, series_name: '1-468', is_emergency: false },
{ house_id: 2874, address: 'ул. Бебеля, 138', lat: 56.8664, lon: 60.5721, sale_share_pct: 22.4, sale_share_pct_45d: 25.0, listings_45d: 20, over_100: false, active_secondary: 17, flat_count_effective: 76, gar_match_method: 'cadastre', median_price_rub: 5_400_000, median_price_per_m2: 149_200, avg_days_on_market: 61, year_built: 1978, house_type: 'panel', total_floors: 9, series_name: '141', is_emergency: false },
{ house_id: 5530, address: 'ул. Малышева, 84', lat: 56.8380, lon: 60.6203, sale_share_pct: 12.1, sale_share_pct_45d: 13.8, listings_45d: 16, over_100: false, active_secondary: 14, flat_count_effective: 116, gar_match_method: 'cadastre', median_price_rub: 7_850_000, median_price_per_m2: 172_300, avg_days_on_market: 47, year_built: 1954, house_type: 'stalin', total_floors: 6, series_name: null, is_emergency: false },
{ house_id: 6119, address: 'ул. Щербакова, 20', lat: 56.7791, lon: 60.6120, sale_share_pct: 6.8, sale_share_pct_45d: 8.1, listings_45d: 22, over_100: false, active_secondary: 18, flat_count_effective: 264, gar_match_method: 'cadastre', median_price_rub: 8_900_000, median_price_per_m2: 164_800, avg_days_on_market: 38, year_built: 2016, house_type: 'monolith', total_floors: 25, series_name: null, is_emergency: false },
{ house_id: 5871, address: 'ул. Крауля, 44', lat: 56.8443, lon: 60.5610, sale_share_pct: 17.9, sale_share_pct_45d: 19.6, listings_45d: 14, over_100: false, active_secondary: 12, flat_count_effective: 67, gar_match_method: 'cadastre', median_price_rub: 5_150_000, median_price_per_m2: 152_900, avg_days_on_market: 58, year_built: 1980, house_type: 'panel', total_floors: 9, series_name: '141', is_emergency: false },
{ house_id: 6402, address: 'ул. 8 Марта, 190', lat: 56.8043, lon: 60.6094, sale_share_pct: 3.4, sale_share_pct_45d: 4.0, listings_45d: 11, over_100: false, active_secondary: 9, flat_count_effective: 262, gar_match_method: 'cadastre', median_price_rub: 9_300_000, median_price_per_m2: 176_400, avg_days_on_market: 33, year_built: 2019, house_type: 'monolith', total_floors: 26, series_name: null, is_emergency: false },
];
/** Карта ЕКБ: 8 тепловых маркеров, выбран «ул. Викулова, 46» открыт popup
* (адрес, % в продаже, N из M квартир). */
export const Default = () => (
<SaleShareMap
buildings={buildings}
selectedHouseId={3101}
hoveredHouseId={null}
onSelect={() => {}}
onHover={() => {}}
/>
);

View file

@ -0,0 +1,39 @@
import { SectionOverlay } from 'tradein-mvp-frontend';
// Glass/blur оверлей секций HUD «МЕРА Оценка» (/trade-in/v2): скобки-уголки,
// шапка «номер секции + заголовок + ← К ОЦЕНКЕ», скроллируемое тело со
// сменной view. Позиционируется absolute относительно артборда — в preview
// рендерим contained внутри relative-обёртки с фоном v2-страницы.
// Данные не передаём: каждая view падает на свой встроенный fixture-набор
// (HISTORY_FIXTURE / ANALYTICS_FIXTURE …) — как storybook/unwired usage.
const Artboard = ({ children }: { children?: unknown }) => (
<div
style={{
position: 'relative',
height: 680,
borderRadius: 10,
overflow: 'hidden',
background:
'radial-gradient(1100px 520px at 30% -10%, #f7fbff 0%, #eef4fa 55%, #e6eef7 100%)',
}}
>
{children as React.ReactNode}
</div>
);
/** Секция 04 «ПРОДАЖИ В ДОМЕ» (active=1 HistoryView на fixture-данных):
* таблица ДКП-продаж дома с ценами и датами. */
export const HistorySection = () => (
<Artboard>
<SectionOverlay active={1} onClose={() => {}} onNavigate={() => {}} />
</Artboard>
);
/** Секция 06 «АНАЛИТИКА ДОМА» (active=3 AnalyticsView): KPI дома,
* динамика цены, недавние продажи. */
export const AnalyticsSection = () => (
<Artboard>
<SectionOverlay active={3} onClose={() => {}} onNavigate={() => {}} />
</Artboard>
);

View file

@ -0,0 +1,135 @@
name: CI Trade-In
# Forgejo Actions pre-merge gate for the SUBPROJECT tradein-mvp/.
# WHY THIS FILE EXISTS (#2208): основной .forgejo/workflows/ci.yml гейтит
# ТОЛЬКО backend/** + frontend/** главного стека — tradein-PR проходили на
# пусто-зелёных чеках (paths-filter no-op), а pytest tradein жил лишь в
# post-merge deploy-tradein.yml. Итог: сломанный tradein-код мержился в main
# и обнаруживался только на деплое. Этот workflow добавляет РЕАЛЬНЫЙ pre-merge
# gate: tradein-backend pytest + tradein-frontend type-check/lint ДО мержа.
on:
# ТОЛЬКО pull_request — НЕТ push-триггера на feature-ветки (CI-шторм #1709,
# см. подробное обоснование в ci.yml). Кратко: раньше push+pull_request на один
# SHA давали разный github.ref → разные concurrency-группы → 2× прогон на
# дефицитных раннерах. В bot-пайплайне каждый коммит идёт через PR, так что
# pull_request гейтит его полностью; push-прогон был чистым дублем.
pull_request:
branches: [main]
concurrency:
# github.ref стабилен на весь PR (refs/pull/<N>/merge) → новый push в ветку PR
# отменяет предыдущий незавершённый прогон ЭТОГО PR вместо накопления.
group: ci-tradein-${{ github.ref }}
cancel-in-progress: true
jobs:
# Paths-filter: гейт бежит ТОЛЬКО когда поменялся tradein-код.
# PR не трогающий tradein-mvp/ → оба job'а no-op'ятся → дёшево.
changes:
runs-on: ubuntu-latest
outputs:
backend: ${{ steps.filter.outputs.backend }}
frontend: ${{ steps.filter.outputs.frontend }}
steps:
- uses: actions/checkout@v4
- uses: dorny/paths-filter@v3
id: filter
with:
filters: |
backend:
- 'tradein-mvp/backend/**'
- 'tradein-mvp/packages/**'
# workspace root: lock-only bump (uv lock --upgrade) или правка
# [tool.uv.workspace] меняют реальные зависимости → гейт обязан бежать.
- 'tradein-mvp/uv.lock'
- 'tradein-mvp/pyproject.toml'
- '.forgejo/workflows/ci-tradein.yml'
frontend:
- 'tradein-mvp/frontend/**'
- '.forgejo/workflows/ci-tradein.yml'
backend-tests:
runs-on: ubuntu-latest
needs: changes
if: needs.changes.outputs.backend == 'true'
defaults:
run:
working-directory: ./tradein-mvp/backend
env:
# psycopg v3 требует parseable URL на импорте; реального коннекта нет —
# DB-тесты мокаются (mirror deploy-tradein.yml test-job).
DATABASE_URL: postgresql+psycopg://test:test@localhost:5432/test
steps:
- uses: actions/checkout@v4
- name: Install uv
# Официальный standalone-инсталлер. НЕ astral-sh/setup-uv — он ломается
# на Forgejo-runner с PEP 668 externally-managed-environment (#666 CI).
run: |
curl -LsSf https://astral.sh/uv/install.sh | sh
echo "$HOME/.local/bin" >> "$GITHUB_PATH"
- name: Cache uv packages
# Кросс-прогонный кэш скачанных/собранных wheel'ов (~/.cache/uv).
# continue-on-error чтобы сбой cache-бэкенда раннера НИКОГДА не ронял gate.
# Ключ по workspace-локу tradein-mvp/uv.lock (tracked с воркспейса #2137).
uses: actions/cache@v4
continue-on-error: true
with:
path: ~/.cache/uv
key: uv-tradein-${{ runner.os }}-${{ hashFiles('tradein-mvp/uv.lock') }}
restore-keys: |
uv-tradein-${{ runner.os }}-
- name: Sync deps (incl. dev group — pytest)
# Workspace-лок tradein-mvp/uv.lock TRACKED (с воркспейса #2137; gitignored
# только старый backend/uv.lock) → --frozen детерминирован и зеркалит
# Dockerfile (uv sync --frozen --no-dev там). uv находит workspace root
# вверх от cwd.
run: uv sync --frozen
- name: Run pytest (tradein-mvp/backend)
# DESELECT (актуализировано 2026-07-02, #2208): test_search_cache_hit падает
# ТОЛЬКО в whole-suite ordering (401 vs 200; в изоляции проходит) — global-state
# leak из другого test-модуля, pre-existing. Второй исторический deselect
# (test_cian_valuation::test_cache_hit_returns_cached) убран — проходит в
# полном прогоне (проверено локально: 2947 passed / 1 failed). Список обязан
# совпадать с test-job в deploy-tradein.yml.
run: |
uv run pytest -q \
--deselect "tests/test_search_api.py::test_search_cache_hit"
frontend-checks:
runs-on: ubuntu-latest
needs: changes
if: needs.changes.outputs.frontend == 'true'
defaults:
run:
working-directory: ./tradein-mvp/frontend
steps:
- uses: actions/checkout@v4
- name: Set up Node
# Node 20 — major из tradein-mvp/frontend/Dockerfile (node:20-alpine).
# npm-кэш setup-node НЕ настраиваем: в tradein-mvp/frontend нет
# package-lock.json (Dockerfile ставит через npm install), а cache=npm
# без lockfile падает. Кэш wheels/node тут не критичен для type-check/lint.
uses: actions/setup-node@v4
with:
node-version: "20"
- name: Install deps (npm install, no lockfile)
# ТОЧНЫЕ флаги из tradein-mvp/frontend/Dockerfile (deps stage):
# --legacy-peer-deps — Tailwind/React 19 peer-dep mismatches;
# --no-audit --no-fund — тише и быстрее в CI. `install` (не `ci`):
# в tradein-mvp/frontend НЕТ package-lock.json (есть pnpm-lock.yaml, но
# Dockerfile ставит именно npm install) → `npm ci` упал бы.
run: npm install --legacy-peer-deps --no-audit --no-fund
- name: Type-check (tsc --noEmit)
# Blocking: любая TS-ошибка → job RED.
run: npm run type-check
- name: Lint (next lint)
# Blocking: любая ESLint-ошибка → job RED.
run: npm run lint

View file

@ -9,6 +9,14 @@ name: Deploy Obsidian
#
# Не пересобирает никаких Docker-образов (CouchDB официальный с DockerHub).
# Не трогает main-стек (backend / frontend / postgres / worker / beat / caddy).
#
# ИСТОРИЯ (2026-07-05): жил в .github/workflows/ с момента миграции с GitHub
# (16.05.2026), помечен в README как «остался на GitHub» — но живого зеркала
# на github.com с настроенными секретами не оказалось: 0 запусков за всю
# историю Forgejo Actions (12000+ прогонов остальных workflow), контейнер
# не пересоздавался с 17.05 до ручного SSH-фикса 04.07. Перенесён сюда —
# единственная директория, которую реально исполняет этот инстанс.
# См. issue #2416.
on:
push:
@ -17,7 +25,7 @@ on:
- "docker-compose.obsidian.yml"
- "scripts/setup-couchdb.sh"
- "docs/obsidian-livesync.md"
- ".github/workflows/deploy-obsidian.yml"
- ".forgejo/workflows/deploy-obsidian.yml"
workflow_dispatch:
concurrency:

View file

@ -103,6 +103,10 @@ jobs:
filters: |
backend:
- 'tradein-mvp/backend/**'
# scraper-kit вкомпилирован в backend-образ (build context tradein-mvp/,
# scheduler_main импортирует пакет) — kit-only изменение обязано
# пересобрать образ, иначе деплой рестартует контейнеры на старом.
- 'tradein-mvp/packages/scraper-kit/**'
frontend:
- 'tradein-mvp/frontend/**'
browser:
@ -117,11 +121,19 @@ jobs:
- 'tradein-mvp/backend/app/services/scheduler.py'
- 'tradein-mvp/backend/app/scheduler_main.py'
- 'tradein-mvp/backend/app/tasks/**'
# #2188: scheduler исполняет matching/dedup при каждом scrape-тике —
# без этих путей scraper-контейнер оставался на старом коде
# (2026-07-02: fias-dedup доехал до tradein-backend, но не до
# tradein-scraper). После USE_KIT_SCHEDULER=true kit-код и есть
# scheduler — его правки тоже обязаны пересоздавать контейнер.
- 'tradein-mvp/backend/app/services/matching/**'
- 'tradein-mvp/backend/app/services/house_dedup_merge.py'
- 'tradein-mvp/packages/scraper-kit/**'
# Quality gate: pytest MUST pass before any image is built/deployed (#666).
# Runs the tradein-mvp/backend suite; a red test blocks build + deploy.
# Tests use mocks + a stub DATABASE_URL — no real Postgres/Redis needed.
# 2 pre-existing order-dependent tests are deselected (see DESELECT note below).
# 1 pre-existing order-dependent test is deselected (see DESELECT note below).
test:
runs-on: ubuntu-latest
needs: changes
@ -146,20 +158,21 @@ jobs:
echo "$HOME/.local/bin" >> "$GITHUB_PATH"
- name: Sync deps (incl. dev group — pytest)
# NB: tradein-mvp/backend/uv.lock is gitignored (see tradein-mvp/.gitignore),
# so --frozen can't work in CI (no lockfile in a fresh checkout). Resolve
# from pyproject.toml like the Dockerfile does (uv sync --no-dev there).
run: uv sync
# Workspace-лок tradein-mvp/uv.lock TRACKED (с воркспейса #2137; gitignored
# только старый backend/uv.lock) → --frozen детерминирован и зеркалит
# Dockerfile (uv sync --frozen --no-dev). Актуализировано в #2208.
run: uv sync --frozen
- name: Run pytest (tradein-mvp/backend)
# DESELECT (2026-05): two pre-existing tests fail only in whole-suite
# ordering (pass in isolation) due to global state leak from other test
# modules — not introduced here. Excluded so the gate is reliably green;
# tracked separately. Everything else (1087 tests) must pass.
# DESELECT (актуализировано 2026-07-02, #2208): test_search_cache_hit падает
# ТОЛЬКО в whole-suite ordering (401 vs 200; в изоляции проходит) — global-state
# leak из другого test-модуля, pre-existing. Второй исторический deselect
# (test_cian_valuation::test_cache_hit_returns_cached) убран — проходит в полном
# прогоне (проверено 2026-07-02: 2947 passed / 1 failed). Список обязан
# совпадать с backend-tests в ci-tradein.yml (pre-merge гейт).
run: |
uv run pytest -q \
--deselect "tests/test_search_api.py::test_search_cache_hit" \
--deselect "tests/test_cian_valuation.py::test_cache_hit_returns_cached"
--deselect "tests/test_search_api.py::test_search_cache_hit"
build-backend:
runs-on: ubuntu-latest
@ -183,7 +196,10 @@ jobs:
- name: Build & push tradein-backend
uses: docker/build-push-action@v6
with:
context: ./tradein-mvp/backend
# Context = tradein-mvp/ (uv workspace root): образу нужен packages/scraper-kit
# для editable install (#2137). Dockerfile — в backend/.
context: ./tradein-mvp
file: ./tradein-mvp/backend/Dockerfile
push: true
cache-from: type=registry,ref=${{ env.IMAGE_BACKEND }}:buildcache
cache-to: type=registry,ref=${{ env.IMAGE_BACKEND }}:buildcache,mode=max
@ -216,10 +232,13 @@ jobs:
context: ./tradein-mvp/frontend
push: true
# basePath=/trade-in baked-in во время build (Next.js)
# NB (#2205): НЕ передаём NEXT_PUBLIC_ENABLE_PREVIEW — preview-роут
# (/ui-preview/estimate, статичная demo-фикстура) собирается ТОЛЬКО в
# dev/CI (a11y/lighthouse). В прод-образе флаг не задан → страница
# уходит в notFound (404), не индексируется и не краулится.
build-args: |
NEXT_PUBLIC_BASE_PATH=/trade-in
NEXT_PUBLIC_API_BASE_URL=/trade-in
NEXT_PUBLIC_ENABLE_PREVIEW=1
cache-from: type=registry,ref=${{ env.IMAGE_FRONTEND }}:buildcache
cache-to: type=registry,ref=${{ env.IMAGE_FRONTEND }}:buildcache,mode=max
tags: |
@ -276,7 +295,13 @@ jobs:
env:
IMAGE_TAG: latest
GHCR_PAT: ${{ secrets.GHCR_PAT }}
SCRAPER_CHANGED: ${{ needs.changes.outputs.scraper == 'true' || needs.changes.outputs.infra == 'true' || github.event_name == 'workflow_dispatch' }}
# Phase 0: generic infra edits (compose / workflow / deploy/**) must NOT
# recreate the scraper and SIGKILL a running multi-hour job. Only genuine
# scraper-code paths (the `scraper` paths-filter already covers
# app/services/scrapers/**, scrape_pipeline.py, scheduler.py,
# scheduler_main.py, app/tasks/**) — or a manual workflow_dispatch —
# should trigger a scraper recreate. (infra term intentionally dropped.)
SCRAPER_CHANGED: ${{ needs.changes.outputs.scraper == 'true' || github.event_name == 'workflow_dispatch' }}
GITHUB_SHA: ${{ github.sha }}
with:
host: ${{ secrets.DEPLOY_HOST }}
@ -314,24 +339,44 @@ jobs:
export IMAGE_TAG="$IMAGE_TAG"
docker compose -p gendesign-tradein -f docker-compose.prod.yml pull
# Селективный up: scraper НЕ пересоздаём на каждый backend-деплой (#1182) —
# один image :latest на backend+scraper, блочный up -d рестартовал бы scraper
# и убивал бегущий sweep. Scraper поднимается только при scraper/infra изменениях.
docker compose -p gendesign-tradein -f docker-compose.prod.yml up -d --no-deps postgres browser backend frontend
if [ "${SCRAPER_CHANGED:-true}" = "true" ]; then
echo "→ scraper paths changed — recreating tradein-scraper"
docker compose -p gendesign-tradein -f docker-compose.prod.yml up -d --no-deps scraper
else
echo "→ scraper unchanged — tradein-scraper left running (подхватит новый image при следующем своём рестарте)"
fi
# ── Порядок деплоя (issue #2216): МИГРАЦИИ ДО НОВОГО app-кода ──────────
# Раньше backend/frontend/scraper поднимались ПЕРЕД миграциями: при сбое
# миграции новый код уже крутился на СТАРОЙ схеме (рассинхрон код↔схема).
# Теперь строго: (1) только postgres → (2) ждём готовности БД →
# (3) ВЕСЬ блок миграций → (4) app-контейнеры → (5) Caddy + health.
# ИНВАРИАНТ ПРИ СБОЕ МИГРАЦИИ: строгий gate делает exit 1 ДО подъёма
# нового кода → старые контейнеры продолжают работать на СТАРОМ коде +
# СТАРОЙ схеме (консистентная пара). Это и есть цель: никогда «новый
# код на старой схеме». Откат = просто ничего не поднимали.
# Применяем SQL миграции (если есть backend/data/sql/*.sql)
# (1) Только БД — чтобы прогнать миграции до нового app-кода.
docker compose -p gendesign-tradein -f docker-compose.prod.yml up -d --no-deps postgres
# (2) Ждём готовности postgres (pg_isready в цикле, НЕ тупой sleep).
echo "→ Ожидание готовности postgres..."
pg_ready=""
for i in $(seq 1 30); do
if docker compose -p gendesign-tradein -f docker-compose.prod.yml exec -T postgres \
pg_isready -U "${TRADEIN_POSTGRES_USER:-tradein}" -d tradein >/dev/null 2>&1; then
pg_ready="yes"; break
fi
sleep 2
done
if [ -z "$pg_ready" ]; then
echo "ERROR: postgres не стал ready за отведённое время — прерываю деплой."
echo " Новый app-код НЕ поднят; старые контейнеры не тронуты."
exit 1
fi
echo "→ postgres ready."
# (3) Применяем SQL миграции (если есть backend/data/sql/*.sql) — ДО app.
# Postgres init load *.sql из /docker-entrypoint-initdb.d ТОЛЬКО при первом
# старте volume. Здесь — для повторных миграций после первого запуска.
# Tracking через _schema_migrations (порт паттерна из deploy.yml):
# каждый .sql применяется РОВНО один раз, failed migration → exit 1
# (никаких swallowed errors). cwd = /opt/gendesign/tradein-mvp.
sleep 5
# NB: цикл берёт только *.sql — data/sql/_manifest_applied.txt (инвариант
# #2216) glob'ом не подхватывается.
# Pre-existence detection ДО CREATE TABLE: если таблицы ещё нет, это
# первый deploy после внедрения tracking на уже-наполненной prod-БД
@ -405,16 +450,113 @@ jobs:
echo "WARNING: TRADEIN_READER_PASSWORD not set in .env.runtime — gendesign_reader без пароля, ETL #976 не сможет подключиться"
fi
# Retry backend lifespan hook AFTER migrations applied.
# tradein-backend startup runs ensure_fdw_user_mapping which needs
# FOREIGN SERVER gendesign_remote (created by 060_postgres_fdw_extension.sql).
# Without restart, the first compose-up's startup hook failed with
# "server gendesign_remote does not exist" because migrations hadn't run yet.
# See PR #493 deploy/1156 for the incident details.
echo "→ Restarting tradein-backend so lifespan hook retries USER MAPPING setup"
docker restart tradein-backend
# Give backend time to come up before Caddy reload + health check below
sleep 5
# (4) Теперь — новый app-код: схема уже актуальна.
#
# #1951: раньше scraper пересоздавался ВТОРОЙ отдельной командой `up -d`,
# уже ПОСЛЕ browser/backend/frontend. Если это происходило посреди
# in-flight sweep'а (avito/cian/rosreestr full-load), процесс убивался на
# лету, а осиротевшая scrape_runs-строка сидела status='running' с
# замёрзшим heartbeat до периодического 6h zombie-reaper'а — false "hang"
# investigation вместо честного deploy-артефакта (см. cian_full_load #404).
# Три меры ниже — все devops-only (shell в deploy-скрипте), БЕЗ правок
# Python в scraper-стартап-пути (scheduler_main.py / scraper-kit /
# app/services/scrapers — намеренно не тронуты, см. PR-описание):
#
# 1) Атомарный recreate — browser/backend/frontend[+scraper] поднимаются
# ОДНИМ `docker compose up -d` инвокейшном (список сервисов собирается
# заранее в $SERVICES), а не двумя последовательными командами.
# 2) Graceful drain — если scraper будет пересоздан, ждём (до 5 мин, poll
# каждые 10s) пока scrape_runs.status='running' не станет 0, ПРЕЖДЕ чем
# инициировать recreate. Таймаут не блокирует деплой навсегда —
# SIGTERM-drain (#1182 Phase 2/3a) + stop_grace_period 120s остаются
# финальной страховкой для того, что не успело дойти до checkpoint'а.
# 3) Startup-reap — сразу после recreate помечаем 'cancelled' любые
# 'running'-строки, чей heartbeat не обновлялся с МОМЕНТА (по часам
# самой БД — SELECT NOW(), без risk clock-skew раннера), взятого
# непосредственно перед stop. Такие строки заведомо осиротели ЭТИМ
# recreate'ом (старый контейнер физически не может писать heartbeat
# после своей остановки) — не ждём 6h периодического reap_zombies().
# 'cancelled' (не 'zombie') — честно отличает «убит деплоем» от
# «непонятно завис» (последнее по-прежнему ловит только 6h-reaper).
# Порог — по метке времени конкретного recreate, а не по фиксированному
# интервалу: не зависит от heartbeat-каденса разных источников и не
# рискует ложно отменить НЕ относящийся к этому recreate run (напр.
# admin-triggered scrape внутри backend, если backend в этом деплое
# не пересоздавался — его heartbeat продолжит расти после checkpoint'а).
SERVICES="browser backend frontend"
SCRAPER_STOP_TS=""
if [ "${SCRAPER_CHANGED:-true}" = "true" ]; then
echo "→ scraper paths changed — waiting for in-flight scrape_runs to drain (up to 5 min)"
drained=""
for i in $(seq 1 30); do
# NB: не сливать "psql не ответил" с "0 running" — иначе неудачный
# прогон психgl молча читается как «слито», и graceful drain
# становится no-op именно в момент проблем с БД во время деплоя.
running_count=""
psql_out="$(docker compose -p gendesign-tradein -f docker-compose.prod.yml exec -T postgres \
psql -U "${TRADEIN_POSTGRES_USER:-tradein}" -d tradein -tAc \
"SELECT COUNT(*) FROM scrape_runs WHERE status='running';" 2>/dev/null)" \
&& running_count="$(printf '%s' "$psql_out" | sed -e 's/^[[:space:]]*//' -e 's/[[:space:]]*$//')"
if [ -n "$running_count" ] && [ "$running_count" = "0" ]; then
drained="yes"; break
fi
if [ -z "$running_count" ]; then
echo " ...не удалось прочитать running_count (psql failed) — считаем как «ещё активен», жду 10s (попытка $i/30)"
else
echo " ...${running_count} активных run(ов) ещё бегут, жду 10s (попытка $i/30)"
fi
sleep 10
done
if [ -n "$drained" ]; then
echo "→ Активных run'ов нет — recreate scraper безопасен."
else
echo "WARNING: активные scrape_runs остались после 5 мин ожидания — recreate продолжится."
echo " SIGTERM-drain (#1182) + stop_grace_period 120s постараются сохранить checkpoint;"
echo " startup-reap ниже подчистит то, что не успеет."
fi
# Checkpoint по часам БД (не раннера) прямо перед recreate.
# NB: tr -d '[:space:]' сломан для timestamptz-литерала — убирает и
# внутренний пробел между датой и временем ("2026-07-04 06:43" →
# "2026-07-0406:43"), CAST(...AS timestamptz) на такое падает молча
# (см. WARNING-фолбэк ниже). sed убирает только leading/trailing.
SCRAPER_STOP_TS="$(docker compose -p gendesign-tradein -f docker-compose.prod.yml exec -T postgres \
psql -U "${TRADEIN_POSTGRES_USER:-tradein}" -d tradein -tAc "SELECT NOW();" 2>/dev/null | sed -e 's/^[[:space:]]*//' -e 's/[[:space:]]*$//')" || SCRAPER_STOP_TS=""
echo "→ scraper checkpoint ts (DB clock): ${SCRAPER_STOP_TS:-unknown}"
SERVICES="$SERVICES scraper"
else
echo "→ scraper unchanged — tradein-scraper left running (подхватит новый image при следующем своём рестарте)"
fi
docker compose -p gendesign-tradein -f docker-compose.prod.yml up -d --no-deps $SERVICES
if [ "${SCRAPER_CHANGED:-true}" = "true" ] && [ -n "${SCRAPER_STOP_TS:-}" ]; then
echo "→ Startup-reap (#1951): помечаем orphaned running-строки, замороженные recreate'ом"
# NB: psql `-c` НЕ поддерживает `:'var'`-подстановку (переменная доходит до
# сервера как литерал → syntax error, см. комментарий выше про TRADEIN_READER_PASSWORD)
# — поэтому подставляем bash-значением напрямую. SCRAPER_STOP_TS сгенерирован
# самим Postgres (SELECT NOW()), не внешний ввод → безопасно.
docker compose -p gendesign-tradein -f docker-compose.prod.yml exec -T postgres \
psql -U "${TRADEIN_POSTGRES_USER:-tradein}" -d tradein -v ON_ERROR_STOP=on -c "
UPDATE scrape_runs
SET status = 'cancelled',
finished_at = NOW(),
error = 'deploy #1951: tradein-scraper recreated mid-run (startup-reap, checkpoint ${SCRAPER_STOP_TS})'
WHERE status = 'running'
AND heartbeat_at < CAST('${SCRAPER_STOP_TS}' AS timestamptz);
" || echo "WARNING: startup-reap query failed — orphaned runs (if any) fall back to the 6h zombie reaper"
fi
# (5) `docker restart tradein-backend` БОЛЬШЕ НЕ НУЖЕН (issue #2216).
# История (PR #493 / deploy 1156): backend раньше поднимался ПЕРЕД
# миграциями, его lifespan-hook (ensure_fdw_user_mapping) падал с
# "server gendesign_remote does not exist" — FOREIGN SERVER создаёт
# 060_postgres_fdw_extension.sql, ещё не прогнанная на тот момент.
# Требовался рестарт для повторной попытки хука. Теперь backend
# стартует на шаге (4), т.е. ПОСЛЕ применения миграций (шаг 3) →
# lifespan-hook гарантированно видит применённые миграции (FOREIGN
# SERVER gendesign_remote существует) уже с первого старта. Рестарт удалён.
# Caddy reload — основной Caddyfile содержит inline tradein routes
# (см. Caddyfile в корне репы). Reload, чтобы Caddy перечитал DNS
@ -423,12 +565,28 @@ jobs:
docker compose -p gendesign -f docker-compose.prod.yml exec -T caddy \
caddy reload --config /etc/caddy/Caddyfile || true
# Health check
# Health check — деплой ВАЛИТСЯ, если backend не поднялся (#2214).
# Раньше цикл после 30 неуспешных попыток молча продолжал скрипт и
# доходил до записи success-маркера → мёртвый backend помечался
# «задеплоено». Теперь: флаг healthy выставляется ТОЛЬКО при HTTP 200
# на /health; после цикла — hard exit 1, если флаг пуст. exit 1
# происходит ДО записи .tradein-deployed-sha (маркер пишется последним,
# ниже) → следующий прогон changes-job возьмёт корректную базу.
# NB set -e: curl стоит в условии `if` (exempt из errexit) — неуспешная
# попытка НЕ фатальна, а лишь провоцирует следующую итерацию цикла.
healthy=""
for i in $(seq 1 30); do
docker compose -p gendesign-tradein -f /opt/gendesign/tradein-mvp/docker-compose.prod.yml \
exec -T backend curl -fsS http://localhost:8000/health && break
if docker compose -p gendesign-tradein -f /opt/gendesign/tradein-mvp/docker-compose.prod.yml \
exec -T backend curl -fsS http://localhost:8000/health >/dev/null 2>&1; then
healthy="yes"; break
fi
sleep 1
done
if [ -z "$healthy" ]; then
echo "ERROR: backend не ответил на /health за 30s — деплой FAILED"
exit 1
fi
echo "→ backend healthy на /health."
# Cleanup старых образов
for repo in ghcr.io/lekss361/gendesign-tradein-backend \

View file

@ -199,6 +199,14 @@ jobs:
# but other callers may invoke the raw path).
chmod +x ops/*.sh 2>/dev/null || true
# Full PDF report bind-source (#2259 PR-D). docker создаёт отсутствующий
# bind-source как root:root — worker пишет PDF под uid 1000 → PermissionError
# → вечный «building». Создаём каталог заранее + chown под контейнерный uid.
# chown под non-root deploy-юзером требует sudo → fallback (|| true — если и
# sudo нет, каталог уже наш и chown не нужен).
mkdir -p reports
chown 1000:1000 reports 2>/dev/null || sudo chown 1000:1000 reports 2>/dev/null || true
# Sentry release tracking
mkdir -p backend
touch backend/.env.runtime

7
.gitignore vendored
View file

@ -1,4 +1,4 @@
# Memory graph (hardlink to C:\mcp\.aim\memory-gendesign.jsonl)
# Memory graph (hardlink to C:\mcp\.aim\memory-gendesign.jsonl)
*.jsonl
!memory/memory-gendesign.jsonl
@ -44,7 +44,6 @@ tradein-mvp/scripts/.*cookies*.json
# Agents and rules ARE tracked: they are team-shared config.
# Explicit per-path list — `.claude/` + `!.claude/agents/` whitelist pattern
# does not work because git cannot re-include files under an ignored parent.
.claude/settings.json
.claude/settings.local.json
.claude/cache/
.claude/sessions/
@ -88,6 +87,10 @@ sf_anton_snapshot.db-*
data/osrm/*
!data/osrm/.gitkeep
# Full PDF report bind-source (#2259 PR-D) — worker пишет PDF в ./reports:/app/reports.
# Артефакты рантайма, не коммитим (untracked и так выживает при deploy git reset --hard).
/reports/
# Log cruft at repo root
debug.log
.ds-sync/

View file

@ -21,7 +21,7 @@ Live: `https://gendsgn.ru/` — Свердловская обл. (ЕКБ, ПЗЗ
2. **Branch + PR mandatory.** Никаких direct push в main. `feat/...` / `fix/...` / `refactor/...` / `docs/...` / `chore/...` → ветка от `forgejo/main``mcp__forgejo__create_pull_request` → PR URL пользователю. После PR create — сразу `Skill loop` polling. `gh` CLI bypassed (2026-05-16). **Полные правила: `.claude/rules/git-pr.md`**.
3. **Agent-first workflow.** Main session orchestrates ONLY — не пишет код inline. Любая задача >1 файл / >50 строк → subagent. Pre-push: spawn `code-reviewer` subagent на staged changes. Post-push review — внешнее окно Claude (НЕ дублировать).
3. **Agent-first workflow.** Main session orchestrates ONLY — не пишет код inline. Любая задача >1 файл / >50 строк → subagent. Pre-push: spawn `code-reviewer` subagent на staged changes. Post-push review — внешнее окно Claude (НЕ дублировать). Лимиты размера subagent-задачи: `.claude/rules/delegation.md`.
4. **psycopg v3 only.** `import psycopg2` → ModuleNotFoundError. `CAST(:x AS type)` в SQL — никогда `:x::type`. **Полные правила: `.claude/rules/backend.md` + `sql.md`**.
@ -41,14 +41,16 @@ Live: `https://gendsgn.ru/` — Свердловская обл. (ЕКБ, ПЗЗ
| `database-expert` | `data/sql/**.sql`, Alembic, EXPLAIN ANALYZE |
| `devops-engineer` | `docker-compose*.yml`, `Caddyfile`, `.github/workflows/**`, `.forgejo/workflows/**` |
| `code-reviewer` | Pre-push lint (security, correctness, conventions) |
| `deep-code-reviewer` | Тщательный review критичных PR (миграции / auth / scrapers) + merge authority при ✅ APPROVE |
| `deep-code-reviewer` | Тщательный review критичных PR (миграции / auth / scrapers) + merge authority при ✅ APPROVE (не эксклюзивно: self-merge разрешён любой сессии с 2026-06-27) |
| `qa-tester` | Post-deploy smoke (playwright / curl / SQL) сразу после merge+deploy — rule #7 |
`auto-*` в `.claude/agents/` — standalone bot-персоны (`/work-as-*`), НЕ для Task-spawn; общий контракт — `_autonomous_pickup.md`.
**Routing:** тривиально (typo, 1-line) → main session. Single-domain clear → worker. Cross-domain / нечётко → `tech-analyst` first. Worker → `code-reviewer` → main commits → push → PR.
## Where to look
- **Path-scoped rules:** `.claude/rules/` (backend, frontend, sql, git-pr, deploy, ui-tokens, ui-conventions, ui-microcopy) — auto-loaded при работе с соответствующими файлами через `paths:` frontmatter
- **Path-scoped rules:** `.claude/rules/` (backend, frontend, sql, git-pr, deploy, tradein, ui-tokens, ui-conventions, ui-microcopy) — auto-loaded при работе с соответствующими файлами через `paths:` frontmatter; `delegation.md` — намеренно БЕЗ `paths:` (грузится в каждой сессии, не добавлять frontmatter)
- **Personal preferences:** `~/.claude/CLAUDE.md` (cross-project: no auto-commit, no Co-Authored-By, no @claude)
- **Workflow memory:** `~/.claude/projects/<repo>/memory/MEMORY.md` index
- **Vault MOCs:** `meta/`, `domains/<area>/<area>-MOC.md`, `decisions/`, `fixes/`, `limitations/`, `code/patterns/`
@ -56,7 +58,7 @@ Live: `https://gendsgn.ru/` — Свердловская обл. (ЕКБ, ПЗЗ
## Don't (cross-domain — specifics → `.claude/rules/`)
- ❌ Direct push в main / merge без approval
- ❌ Direct push в main; merge вне policy (красный CI / tripwires — git-pr.md § Auto-merge policy)
- ❌ `--no-verify` / `--amend` / `git push --force`
- ❌ Knowledge-файлы вне vault (`docs/research/`, `notes/`, `wiki/`)
- ❌ Hardcode credentials в коде / коммитах

View file

@ -16,15 +16,13 @@
# (basic_auth runs before handle), making /health and /preview/* exclusions
# ineffective. With route { }, handlers are matched top-to-bottom as written.
{
servers {
# ВКЛЮЧАЕТ raw Authorization/Cookie headers в access log
# (по default Caddy 2.x редактирует их как "REDACTED").
# Plain password будет в gendsgn.ru.log + auth_audit.log — оба root-only на VPS.
# См. forwarder.py _extract_attempted_username для использования.
log_credentials
}
}
# #2213: global `log_credentials` УБРАН. Он заставлял Caddy писать raw
# Authorization header (Base64 `user:password`, тривиально обратимый) в ОБА лог-файла
# (gendsgn.ru.log + auth_audit.log). Даже при root-only доступе и коротком retention
# это plaintext-пароли пилотов на диске — недопустимый риск против скромной выгоды.
# Trade-off: glitchtip-auth-forwarder больше не извлекает `attempted_username` из 401
# (тег деградирует в "(none)" — forwarder это уже обрабатывает gracefully, не падает).
# Событие basic_auth 401 (remote_ip / uri / method) по-прежнему уходит в GlitchTip.
gendsgn.ru {
encode zstd gzip
@ -88,9 +86,23 @@ gendsgn.ru {
uri strip_prefix /trade-in
reverse_proxy tradein-backend:8000 {
header_up X-Authenticated-User {http.auth.user.id}
# #2213 defense-in-depth: общий секрет Caddy↔tradein-backend. header_up
# с value ПЕРЕЗАПИСЫВАЕТ (стирает) любой клиентский X-Internal-Auth-Secret —
# тот же механизм, что защищает X-Authenticated-User выше. Пусто пока
# TRADEIN_INTERNAL_AUTH_SECRET не задан в .env (fail-open, backend не проверяет).
header_up X-Internal-Auth-Secret {env.TRADEIN_INTERNAL_AUTH_SECRET}
}
}
# gendsgn.ru/sale-share — короткий адрес standalone-продукта «Поиск домов».
# Next basePath=/trade-in → редиректим на канонический /trade-in/sale-share
# (тот же tradein-frontend контейнер; query-string сохраняется). True vanity-URL
# в адресной строке требует отдельного Next-app с basePath=/sale-share.
@saleshare path /sale-share /sale-share/
handle @saleshare {
redir /trade-in/sale-share permanent
}
# Matcher `path /trade-in /trade-in/*` ловит И /trade-in (без слеша),
# И /trade-in/ + /trade-in/anything. Без обоих случаев `handle /trade-in/*`
# пропускал /trade-in без слеша → попадал в общий frontend → пустой ответ.
@ -99,6 +111,9 @@ gendsgn.ru {
# Next.js basePath=/trade-in — фронт сам ждёт префикса в URL
reverse_proxy tradein-frontend:3000 {
header_up X-Authenticated-User {http.auth.user.id}
# #2213: симметрично с /trade-in/api/* — перезаписываем секрет из env
# (стирает клиентский), на случай SSR-forwardʼa фронтом в backend.
header_up X-Internal-Auth-Secret {env.TRADEIN_INTERNAL_AUTH_SECRET}
}
}

View file

@ -161,7 +161,7 @@ docker-compose.uptime.yml Uptime Kuma мониторинг (status.gendsgn.ru
- [`.forgejo/workflows/ci.yml`](.forgejo/workflows/ci.yml) — на PR: ruff lint + mypy (selective strict) + pytest. Блокирует merge при провале.
- [`.forgejo/workflows/deploy.yml`](.forgejo/workflows/deploy.yml) — main: триггер на `backend/**`, `frontend/**`, `Caddyfile`, `docker-compose.prod.yml`, `data/sql/**`. Build backend lean + worker-with-chromium + frontend → push в приватный GHCR → SSH `git reset --hard`, **auto-apply pending `data/sql/NN_*.sql` через `_schema_migrations`** (idempotent, см. ниже про миграции), sed `SENTRY_RELEASE=$IMAGE_TAG` в `backend/.env.runtime`, `compose pull && up -d`, `caddy reload`, `curl /health`.
- [`.forgejo/workflows/deploy-tradein.yml`](.forgejo/workflows/deploy-tradein.yml) — tradein-mvp стек (отдельный пайплайн).
- [`.github/workflows/deploy-obsidian.yml`](.github/workflows/deploy-obsidian.yml) — obsidian (**остался на GitHub**): триггер на `docker-compose.obsidian.yml`, `scripts/setup-couchdb.sh`, `docs/obsidian-livesync.md`. Без сборки образов (couchdb:3 с DockerHub), SSH `compose up -d` + idempotent bootstrap (CORS, DB, лимиты).
- [`.forgejo/workflows/deploy-obsidian.yml`](.forgejo/workflows/deploy-obsidian.yml) — obsidian: триггер на `docker-compose.obsidian.yml`, `scripts/setup-couchdb.sh`, `docs/obsidian-livesync.md`. Без сборки образов (couchdb:3 с DockerHub), SSH `compose up -d` + idempotent bootstrap (CORS, DB, лимиты). *(до 2026-07-05 ошибочно лежал в `.github/workflows/` — там ни разу не исполнился, см. issue #2416; контейнер держался вручную.)*
**Forgejo Secrets / Variables:** `DEPLOY_HOST`, `DEPLOY_USER`, `DEPLOY_SSH_KEY`, `DEPLOY_PORT`. Сервер авторизуется в GHCR однократно через PAT с `read:packages`. `COUCHDB_USER`/`COUCHDB_PASSWORD` — в `backend/.env.runtime` на VPS (не в репе).

View file

@ -70,7 +70,7 @@ users:
admin: admin
kopylov: pilot
user1: pilot
user2: pilot
user2: pilot # «Брусника» — доступ восстановлен 2026-07-13 (снят trial-expire от 2026-07-09)
user3: pilot
user4: pilot
user5: pilot

View file

@ -9,6 +9,13 @@ POST /api/v1/admin/etl/objective-backfill
POST /api/v1/admin/etl/nspd-denorm-backfill
Запустить backfill nspd_parcels/nspd_buildings из всех nspd_quarter_dumps.
GET /api/v1/admin/etl/mapping-review
Список авто-маппингов objective_complex_mapping + обогащение для ревью.
POST /api/v1/admin/etl/mapping-review/{id}/approve
Подтвердить маппинг (is_reviewed=true + note append).
POST /api/v1/admin/etl/mapping-review/{id}/reject
Отклонить маппинг DELETE строки (иначе травит mv_layout_velocity §4.2).
Auth: gendsgn.ru-wide Caddy basic_auth gate (PR #426). App-level X-Admin-Token
header removed 2026-05-23 двойная auth избыточна для pilot.
"""
@ -16,17 +23,24 @@ header removed 2026-05-23 — двойная auth избыточна для pilo
from __future__ import annotations
import logging
from typing import Annotated
from typing import Annotated, Any
from fastapi import APIRouter, Depends, Query
from fastapi import APIRouter, Depends, HTTPException, Query
from sqlalchemy.orm import Session
from app.core.db import get_db
from app.services.etl.mapping_review import (
approve_mapping,
list_mapping_review,
reject_mapping,
)
from app.services.etl.objective_backfill import (
AUTO_ACCEPT_THRESHOLD,
AUTO_ACCEPT_THRESHOLD_V2,
REVIEW_THRESHOLD,
apply_core_matches,
auto_apply_matches,
find_core_matches,
find_match_candidates,
trigger_mv_refresh,
)
@ -104,6 +118,37 @@ def run_objective_backfill(
return result
@router.post("/objective-core-backfill")
def run_objective_core_backfill(
db: Annotated[Session, Depends(get_db)],
dry_run: Annotated[bool, Query(description="Preview без insertions (default True)")] = True,
refresh_mv: Annotated[bool, Query(description="REFRESH mv_layout_velocity после apply")] = True,
) -> dict[str, object]:
"""Core-name + dev-confirm проход (#2177 шаг 2, match_method='auto_core_dev_v5').
Нормализует ЯДРО имени обеих сторон (normalize_complex_name, #2198) и мапит
несопоставленные domrf ЕКБ-объекты на objective-проекты. Пишутся ТОЛЬКО tier_a
(ровно 1 кандидат + подтверждение застройщика). tier_b/ambiguous в counts.
Дефолт dry_run=True (безопасно). Реальный insert ?dry_run=false.
Returns dict:
inserted / conflict_skipped / error_skipped / tier_a_total из apply;
tier_b / ambiguous / skipped_taken счётчики отчёта;
mv_rows_after_refresh строк в MV после REFRESH (0 если dry-run / нет вставок).
"""
report = find_core_matches(db)
result: dict[str, object] = dict(apply_core_matches(db, report, dry_run=dry_run))
result.update(report.counts())
mv_rows = 0
if refresh_mv and not dry_run and result.get("inserted", 0):
mv_rows = trigger_mv_refresh(db)
logger.info("mv_layout_velocity refreshed after core-backfill: %d rows", mv_rows)
result["mv_rows_after_refresh"] = mv_rows
return result
@router.post("/nspd-denorm-backfill")
def run_nspd_denorm_backfill(
limit: Annotated[
@ -124,3 +169,68 @@ def run_nspd_denorm_backfill(
task = backfill_all_dumps.apply_async(kwargs={"limit": limit})
logger.info("nspd-denorm-backfill enqueued: task_id=%s limit=%s", task.id, limit)
return {"task_id": task.id, "status": "enqueued", "limit": limit}
# ════════════════════════════════════════════════════════════════════════════
# Mapping review — глазами подтвердить/отклонить авто-маппинги (feat/…-review)
# ════════════════════════════════════════════════════════════════════════════
@router.get("/mapping-review")
def get_mapping_review(
db: Annotated[Session, Depends(get_db)],
only_unreviewed: Annotated[
bool, Query(description="Только is_reviewed=false (авто-матчи без ревью)")
] = True,
limit: Annotated[int, Query(ge=1, le=500, description="Строк на странице")] = 100,
offset: Annotated[int, Query(ge=0, description="Смещение страницы")] = 0,
) -> dict[str, Any]:
"""Список строк objective_complex_mapping + обогащение для ревьюера.
Обогащение каждой строки:
- domrf_comm_name / domrf_dev_name latest snapshot по domrf_obj_id;
- objective_developers застройщики objective-проекта (агрегат
objective_lots.developer, scoped по project_name).
Сорт: is_reviewed asc, match_score asc NULLS FIRST (сомнительные сверху).
Returns dict:
rows: list обогащённых строк (id, objective_complex_name,
objective_project_id, objective_group, domrf_obj_id, match_method,
match_score, is_reviewed, note, created_at, domrf_comm_name,
domrf_dev_name, objective_developers);
total: всего строк (с учётом only_unreviewed, БЕЗ limit/offset);
limit / offset / only_unreviewed: эхо параметров.
"""
return list_mapping_review(db, only_unreviewed=only_unreviewed, limit=limit, offset=offset)
@router.post("/mapping-review/{mapping_id}/approve")
def approve_mapping_review(
mapping_id: int,
db: Annotated[Session, Depends(get_db)],
) -> dict[str, Any]:
"""Подтвердить маппинг: is_reviewed=true + append «approved <date>» в note.
404 если строки с таким id нет.
"""
result = approve_mapping(db, mapping_id)
if result is None:
raise HTTPException(status_code=404, detail=f"mapping {mapping_id} не найден")
return result
@router.post("/mapping-review/{mapping_id}/reject")
def reject_mapping_review(
mapping_id: int,
db: Annotated[Session, Depends(get_db)],
) -> dict[str, Any]:
"""Отклонить маппинг — DELETE строки (иначе травит mv_layout_velocity §4.2).
Удаляемая строка целиком логируется (logger.info) перед commit в note
писать некуда, строки не будет. 404 если id не найден.
"""
deleted = reject_mapping(db, mapping_id)
if deleted is None:
raise HTTPException(status_code=404, detail=f"mapping {mapping_id} не найден")
return {"status": "deleted", "deleted": deleted}

View file

@ -497,6 +497,21 @@ def trigger_poi_sync() -> dict[str, Any]:
return {"task_id": result.id, "queued_at": "now"}
@router.post("/gisogd-permits-sync")
def trigger_gisogd_permits_sync() -> dict[str, Any]:
"""Manual trigger инкрементальной загрузки РНС/РВЭ ГИСОГД-СО → gisogd_permits (#2367).
Обычно запускается еженедельно через beat (вторник 06:30 МСК). Этот endpoint
для ad-hoc запуска (например после деплоя миграции 187_gisogd_permits.sql или для
внеочередного обновления). Инкрементально: карточки тянутся только для новых/
изменившихся документов, повторный запуск дёшев.
"""
from app.workers.tasks.gisogd_permits_sync import sync_gisogd_permits
result = sync_gisogd_permits.apply_async()
return {"task_id": result.id, "queued_at": "now"}
class TriggerObjectiveEtlRequest(BaseModel):
sqlite_path: str | None = Field(
default=None,
@ -684,12 +699,18 @@ def objective_coverage(
"""
from app.services.objective_etl import get_sqlite_info
# #1964: `lots` = СЫРОЙ COUNT(*) — намеренно. Этот счётчик сравнивается с
# SQLite-источником Антона (ETL-fidelity: «свежее ли SQLite, чем БД»), а SQLite
# хранит сырые строки → дедуп-view сломал бы сравнение. Рядом отдаём
# `lots_physflat` (v_objective_lots_latest — по физлоту) для прозрачности
# инфляции (~2.91×: сырых ~1.75M vs физлотов ~603k).
pg_row = (
db.execute(
text(
"""
SELECT
(SELECT COUNT(*) FROM objective_lots) AS lots,
(SELECT COUNT(*) FROM v_objective_lots_latest) AS lots_physflat,
(SELECT COUNT(*) FROM objective_corpus_room_month) AS crm,
(SELECT COUNT(*) FROM objective_complex_mapping) AS mappings,
(SELECT MAX(snapshot_date) FROM objective_lots) AS last_lot_snapshot,
@ -706,6 +727,7 @@ def objective_coverage(
return {
"pg": {
"lots": int(pg_row["lots"] or 0),
"lots_physflat": int(pg_row["lots_physflat"] or 0),
"corp_room_month": int(pg_row["crm"] or 0),
"mappings": int(pg_row["mappings"] or 0),
"last_lot_snapshot": pg_row["last_lot_snapshot"].isoformat()
@ -1130,7 +1152,7 @@ def trigger_newbuilding_crossload() -> dict[str, Any]:
)
from app.workers.tasks.etl_newbuilding_crossload import etl_newbuilding_crossload
result = etl_newbuilding_crossload.apply_async()
result = etl_newbuilding_crossload.apply_async(kwargs={"triggered_by": "manual"})
return {"task_id": result.id, "queued_at": "now"}
@ -1397,8 +1419,12 @@ _FRESHNESS_SOURCES: list[FreshnessSource] = [
table="nspd_geo_jobs",
work_col="targets_done",
attempt_fallback_col="created_at",
fresh_days=7.0,
stale_days=30.0,
# On-demand источник БЕЗ cron (admin UI / CLI / lazy из analyze) — штучные
# фетчи по активности пользователя. fresh_days=7 флагал каждую неделю
# тишины ложным stale-алертом (расследование 2026-07-04); пороги — под
# реальную каденцию, critical=False и так не трогает overall.
fresh_days=30.0,
stale_days=90.0,
),
FreshnessSource(
source="cadastre",
@ -1409,6 +1435,20 @@ _FRESHNESS_SOURCES: list[FreshnessSource] = [
fresh_days=14.0,
stale_days=45.0,
),
FreshnessSource(
# #2367: реестр РНС/РВЭ ГИСОГД-СО. Data-table режим (плоская таблица без run-
# ledger): свежесть = MAX(fetched_at), upd_24h/_7d = COUNT(*) по окну. Источник
# обновляется ежедневно, тянем еженедельно (beat вторник) → fresh<14d, stale<45d
# (широкий запас на пропуск одного-двух вторников, как cadastre). critical=False.
source="gisogd_permits",
label="ГИСОГД-СО РНС/РВЭ (реестр разрешений)",
table="gisogd_permits",
timestamp_col="fetched_at",
# В timestamp-режиме work_col не используется — валидное имя колонки.
work_col="id",
fresh_days=14.0,
stale_days=45.0,
),
]
@ -1749,6 +1789,20 @@ def trigger_ekburg_permits(
return {"task_id": result.id, "scope": scope, "queued_at": "now"}
# WAF cooldown guard message (#2443 — DOM.РФ hard-banned this VPS's IP 2026-05-24
# после серии failed catalog SSR extras-сессий). Beat schedule для catalog-object
# и catalog-flat scrape'ов ОТКЛЮЧЕН по этой же причине (см. beat_schedule.py) —
# оба ad-hoc admin-эндпоинта ниже бьют по ТОМУ ЖЕ /сервисы/* BrowserSession
# path family, поэтому без явного оператор-override могут углубить бан (#2445 D1).
_WAF_COOLDOWN_GUARD_MSG = (
"Ad-hoc catalog-scrape заблокирован guard'ом: DOM.РФ WAF hard-ban этого VPS IP "
"2026-05-24 (issue #2443), beat schedule для этого таска отключён по той же "
"причине. Повторный ad-hoc запуск может углубить бан. Если ты осознанно "
"принимаешь этот риск (WAF cooldown прошёл, targeted smoke-test и т.п.) — "
"передай i_understand_waf_risk=true в теле запроса."
)
class TriggerKnCatalogObjectsRequest(BaseModel):
region_code: int = Field(default=66, ge=1, le=99)
max_objects: int | None = Field(default=None, ge=1, le=2000)
@ -1760,6 +1814,14 @@ class TriggerKnCatalogObjectsRequest(BaseModel):
"что уже скраплено сегодня."
),
)
i_understand_waf_risk: bool = Field(
default=False,
description=(
"Обязателен (True) для запуска. Guard против случайного re-trigger'а "
"после DOM.РФ WAF hard-ban 2026-05-24 (#2443) — этот scraper бьёт по "
"тому же /сервисы/* BrowserSession path family, что вызвал бан."
),
)
@router.post("/kn-catalog-objects")
@ -1776,7 +1838,14 @@ def trigger_kn_catalog_objects(
- max_objects=None дефолтный лимит таска (300).
- max_objects=3 smoke-тест.
- force=True "Загрузить все": игнорирует skip-today, грузит всё подряд.
WAF cooldown guard (#2443, #2445 D1): требует i_understand_waf_risk=true —
beat schedule для этого таска отключён из-за WAF hard-ban 2026-05-24, ad-hoc
re-trigger без явного подтверждения оператора запрещён.
"""
if not payload.i_understand_waf_risk:
raise HTTPException(status_code=400, detail=_WAF_COOLDOWN_GUARD_MSG)
from app.workers.tasks.scrape_kn_catalog_objects import scrape_kn_catalog_objects
kwargs: dict[str, Any] = {
@ -1794,3 +1863,65 @@ def trigger_kn_catalog_objects(
"force": payload.force,
"queued_at": "now",
}
class TriggerKnCatalogFlatsRequest(BaseModel):
region_code: int = Field(default=66, ge=1, le=99)
max_flats: int | None = Field(default=None, ge=1, le=5000)
force: bool = Field(
default=False,
description=(
"True — игнорировать фильтр свежести ('catalog_updated_at свежий') и "
"грузить ВСЕ квартиры последнего snapshot с непустым catalog_url_hash "
"('Загрузить все'). По умолчанию пропускает то, что скраплено < 30 дней назад."
),
)
i_understand_waf_risk: bool = Field(
default=False,
description=(
"Обязателен (True) для запуска. Guard против случайного re-trigger'а "
"после DOM.РФ WAF hard-ban 2026-05-24 (#2443) — этот scraper ездит по "
"тому же /сервисы/* BrowserSession path family, что и catalog-objects."
),
)
@router.post("/kn-catalog-flats")
def trigger_kn_catalog_flats(
payload: TriggerKnCatalogFlatsRequest,
) -> dict[str, Any]:
"""Manual trigger для catalog-FLAT scraper (#2442): цена/статус/отделка/потолки/
дата обновления + plan-изображения квартир из SSR-страницы каталога.
Селектит domrf_kn_flats WHERE catalog_url_hash IS NOT NULL. До тех пор пока
#2442 Task 1 (elemId → catalog_url_hash) не задеплоен и свежий kn-sweep не
наполнил hash вернёт 0 обработанных строк (ожидаемо, не баг).
- max_flats=None дефолтный лимит таска (300).
- max_flats=3 smoke-тест.
- force=True 'Загрузить все': игнорирует фильтр свежести, грузит всё с hash.
WAF cooldown guard (#2443, #2445 D1): требует i_understand_waf_risk=true —
same /сервисы/* BrowserSession path family как catalog-objects, риск re-trigger
того же WAF-бана.
"""
if not payload.i_understand_waf_risk:
raise HTTPException(status_code=400, detail=_WAF_COOLDOWN_GUARD_MSG)
from app.workers.tasks.scrape_kn_catalog_flats import scrape_kn_catalog_flats
kwargs: dict[str, Any] = {
"region_code": payload.region_code,
"force": payload.force,
}
if payload.max_flats is not None:
kwargs["max_flats"] = payload.max_flats
result = scrape_kn_catalog_flats.apply_async(kwargs=kwargs)
return {
"task_id": result.id,
"region_code": payload.region_code,
"max_flats": payload.max_flats,
"force": payload.force,
"queued_at": "now",
}

View file

@ -35,7 +35,11 @@ from app.core.db import get_db
from app.schemas.chat import ChatAskRequest, ChatAskResponse, ChatIntent, GroundedIn
from app.services.chat.intents import render_answer, route_intent
from app.services.chat.orchestrator import orchestrate_chat
from app.services.chat.retrieval import _FORECAST_SCHEMA_VERSION, get_report_for_chat
from app.services.chat.retrieval import (
_FORECAST_SCHEMA_VERSION,
get_parcel_context_for_chat,
get_report_for_chat,
)
logger = logging.getLogger(__name__)
@ -92,11 +96,46 @@ async def ask(
report_status="pending",
)
# Курируемый паспорт участка + градрегламент (§1 analyze-рана) — отдельный read-only
# seam. §22-отчёт (форсайт) НЕ несёт тер.зону/ЗОУИТ/ЕГРН, поэтому дотягиваем их из
# analyze-1.0 и вливаем в КОПИЮ report_dict под ключ "parcel_context" (tool
# get_parcel_info режет именно его). Analyze-рана нет/сбой чтения → работаем как
# раньше (только форсайт); НЕ меняем pending-поведение (оно завязано на §22-ран выше).
report = await _with_parcel_context(db, payload.cad_num, report)
if settings.llm_enabled:
return await _answer_via_llm(db, payload, report, run_id)
return _answer_deterministic(payload, report, run_id)
async def _with_parcel_context(
db: Session,
cad_num: str,
report: dict[str, Any],
) -> dict[str, Any]:
"""Дотянуть курируемый паспорт участка и влить его в КОПИЮ report_dict.
Read-only: sync-чтение analyze-рана мостим через run_in_threadpool (как §22-отчёт).
None (рана нет) возвращаем report без изменений. Сбой БД глотаем в pending-стиле
эндпоинта: паспорт участка обогащение, его отсутствие не должно ронять чат.
"""
try:
parcel_context = await run_in_threadpool(get_parcel_context_for_chat, db, cad_num)
except Exception:
logger.warning(
"chat: parcel context read failed for cad=%s — continuing without it",
cad_num,
exc_info=True,
)
return report
if not parcel_context:
return report
# Копия: не мутируем report_dict, пришедший из get_report_for_chat.
merged = dict(report)
merged["parcel_context"] = parcel_context
return merged
def _answer_deterministic(
payload: ChatAskRequest,
report: dict[str, Any],

View file

@ -8,9 +8,19 @@ from sqlalchemy import text
from sqlalchemy.orm import Session
from app.core.db import get_db
from app.schemas.concept import ConceptInput, ConceptOutput
from app.schemas.concept import (
ConceptInput,
ConceptOutput,
HouseTypeCatalog,
HouseTypeCatalogItem,
MassingProgram,
MassingRecomputeOutput,
)
from app.services.generative import geometry
from app.services.generative.catalog import HOUSE_TYPES, available_section_types
from app.services.generative.financial import compute_financial
from app.services.generative.geometry import ParcelGeometryError, _parse_polygon
from app.services.generative.teap import synthesize_teap_from_program
logger = logging.getLogger(__name__)
@ -33,6 +43,11 @@ _DISTRICT_SEARCH_RADIUS_M: int = 5000
_PRICE_MIN_RUB: int = 30_000
_PRICE_MAX_RUB: int = 600_000
# Stage 2a (#1965): дефолтный лейбл source для ПРЕДРЕЗОЛВЛЕННОЙ цены в /recompute, когда
# фронт НЕ форвардит подлинный price_source. Консервативный fallback: цена пришла извне
# (фронт её уже резолвил), но без явного происхождения берём нейтральный district-median.
_DEFAULT_PRERESOLVED_SOURCE: str = "objective_district_median"
# Район ЕКБ (ekb_districts) ближайший к точке центроида + его справочная медиана цены.
# Binds через CAST(:name AS type) — psycopg v3 (постфикс-каст к bind-имени запрещён).
# ::geography приклеено к ) / колонке (разрешённое исключение). Read-only, без SAVEPOINT.
@ -57,13 +72,33 @@ _DISTRICT_FOR_POINT_SQL = text(
# Медиана цены продажи жилья из объявлений Objective по району + размер выборки.
# price_per_m2_rub — РУБЛИ. Санитарный диапазон отсекает мусор. CAST psycopg v3.
#
# #1964 physflat-дедуп: objective_lots раздут ~2.91× (мульти objective_lot_id на
# один физлот через пере-листинги) → sample_size и гейт n≥:_MIN_OBJECTIVE_SAMPLE
# были по пере-листингам, вес медианы смещался. Дедупим INLINE через DISTINCT ON
# (physflat-ключ, последний снапшот snapshot_date DESC, id DESC) — НЕ через общий
# physflat-VIEW. Причина (deep-review #1964, прод-EXPLAIN): view не несёт
# district-фильтр внутри, qual не проталкивается ниже DISTINCT ON → view сортирует
# ВСЮ таблицу 1.76M (~5.8 s, request-path внутри analyze_parcel). Inline с district
# В CTE → bitmap по району ~240k строк + sort (~1.7 s), индекс по district работает.
# Сегмент-фильтры (price NOT NULL / диапазон) объективны по физлоту → применяем
# ПОСЛЕ дедупа. Зеркало паттерна market_metrics._STOCK_SQL.
_OBJECTIVE_MEDIAN_SQL = text(
"""
WITH latest AS (
SELECT DISTINCT ON (
project_name, corpus_name, section, floor, lot_number
)
price_per_m2_rub
FROM objective_lots
WHERE district = CAST(:dn AS text)
ORDER BY project_name, corpus_name, section, floor, lot_number,
snapshot_date DESC, id DESC
)
SELECT PERCENTILE_CONT(0.5) WITHIN GROUP (ORDER BY price_per_m2_rub) AS median_ppm2,
COUNT(*) AS sample_size
FROM objective_lots
WHERE district = CAST(:dn AS text)
AND price_per_m2_rub IS NOT NULL
FROM latest
WHERE price_per_m2_rub IS NOT NULL
AND price_per_m2_rub BETWEEN CAST(:lo AS numeric) AND CAST(:hi AS numeric)
"""
)
@ -163,6 +198,32 @@ def _lookup_market_price(db: Session, wkt_point: str) -> tuple[float | None, str
return None, "class_norm"
@router.get("/house-types", response_model=HouseTypeCatalog)
async def list_house_types() -> HouseTypeCatalog:
"""Stage 3b (#1965, эпик #1953) — каталог типовых домов для фронт-пикера.
Read-only проекция ``catalog.HOUSE_TYPES`` (single source of truth по типам
секций): фронт Stage 3b показывает эти типы в режиме «Выбрать дома» и кладёт
выбранные ``section_type`` в ``ConceptInput.building_program``. Без БД, без
side-effects справочник захардкожен в коде (см. модуль ``catalog``). Так
фронт НЕ хардкодит габариты/этажности они приходят отсюда.
"""
return HouseTypeCatalog(
house_types=[
HouseTypeCatalogItem(
section_type=ht.section_type,
label_ru=ht.label_ru,
footprint_w_m=ht.footprint_w_m,
footprint_d_m=ht.footprint_d_m,
footprint_sqm=ht.footprint_sqm,
default_floors=ht.default_floors,
housing_class=ht.housing_class,
)
for ht in HOUSE_TYPES
]
)
@router.post("", response_model=ConceptOutput)
async def create_concept(
payload: ConceptInput,
@ -179,7 +240,26 @@ async def create_concept(
A degenerate parcel (setback consumes everything, malformed geometry) yields a
422 rather than empty variants that is a bad request, not a valid empty result.
Stage 3a (#1965): если задана ``building_program`` (типовые дома из каталога), кладём
РОВНО эту программу вместо жадной max-FAR раскладки (один вариант). Неизвестный
``section_type`` 422 (валидируется тут, до размещения). Если участок не вмещает всю
программу НЕ 422: кладём сколько влезло и отдаём честный ``placed_count`` < requested.
"""
# Stage 3a: валидируем ключи программы по каталогу ДО размещения — неизвестный
# section_type это bad request (422), а не 500 из KeyError в глубине placement.
if payload.building_program:
known = available_section_types()
unknown = sorted({item.section_type for item in payload.building_program} - known)
if unknown:
raise HTTPException(
status_code=422,
detail=(
f"unknown house type(s): {', '.join(unknown)}; "
f"available: {', '.join(sorted(known))}"
),
)
# Рыночную цену продажи жилья считаем ОДИН раз на участок (она едина для всех
# стратегий). DB-lookup — синхронный SQLAlchemy → run_in_threadpool, чтобы не
# блокировать event loop. Падение lookup'а не должно ронять генерацию.
@ -187,9 +267,7 @@ async def create_concept(
price_source: str = "class_norm"
try:
wkt_point = await run_in_threadpool(_parcel_centroid_wkt, payload)
market_price, price_source = await run_in_threadpool(
_lookup_market_price, db, wkt_point
)
market_price, price_source = await run_in_threadpool(_lookup_market_price, db, wkt_point)
except ParcelGeometryError:
# Невалидная геометрия — пусть geometry.generate поднимет её ниже (один 422).
pass
@ -210,3 +288,70 @@ async def create_concept(
logger.warning("concept generation rejected parcel: %s", exc)
raise HTTPException(status_code=422, detail=str(exc)) from exc
return ConceptOutput(variants=variants)
@router.post("/recompute", response_model=MassingRecomputeOutput)
async def recompute_massing(
payload: MassingProgram,
db: Annotated[Session, Depends(get_db)],
) -> MassingRecomputeOutput:
"""Stage 2a (#1965, эпик #1953) — LIVE-пересчёт экономики из 3D-массинга.
Hot, debounced-эндпоинт для интерактивных слайдеров Stage 2b: фронтовый
``computeModel`` отдаёт агрегированную программу (суммарное пятно × этажность),
мы синтезируем :class:`TEAP` (``synthesize_teap_from_program`` те же нормативные
константы, что и обычная генерация) и прогоняем готовый чистый ``compute_financial``
пересчитанный ТЭП + финмодель.
Цена продажи жилья:
* ``market_price_per_sqm`` в теле используется как есть (FAST path, БЕЗ БД)
фронт резолвит цену один раз и шлёт её в каждом keystroke-запросе. Источник
берём из ``payload.price_source`` (ПОДЛИННЫЙ source, который фронт получил из
financial_estimate objective_geo_radius / objective_district_median /
district_reference / class_norm); если фронт его не прислал дефолтный лейбл.
НЕ хардкодим source иначе honest-флаг в UI/PDF врёт о происхождении цены.
* иначе, если передан ``parcel_centroid_wkt`` ``_lookup_market_price`` по центроиду
через ``run_in_threadpool`` (тот же приём, что и в ``create_concept``; sync SQLAlchemy
не должен блокировать event loop). Падение lookup'а → норматив класса, не 500.
* иначе (нет ни цены, ни центроида) норматив класса (``price_source="class_norm"``).
"""
# ── Цена продажи: предрезолвленная (fast) → DB-fallback → норматив класса ───────
market_price: float | None = payload.market_price_per_sqm
price_source: str = "class_norm"
if market_price is not None:
# FAST path: фронт уже резолвил рыночную цену — НЕ ходим в БД на каждый keystroke.
# Используем ПОДЛИННЫЙ source, который фронт форвардит из financial_estimate; если
# его нет — дефолтный лейбл. (При market_price=None compute_financial всё равно
# форсит class_norm, но сюда мы попадаем только при заданной цене.)
price_source = payload.price_source or _DEFAULT_PRERESOLVED_SOURCE
elif payload.parcel_centroid_wkt:
# Fallback: резолвим по центроиду. Sync-lookup → threadpool (как в create_concept).
# Любая ошибка → норматив класса (graceful, без 500) — деградация уже внутри
# _lookup_market_price, защитный пояс на непредвиденное оставляем тут.
try:
market_price, price_source = await run_in_threadpool(
_lookup_market_price, db, payload.parcel_centroid_wkt
)
except Exception as exc:
logger.warning("recompute market-price: unexpected lookup error, class_norm: %s", exc)
market_price, price_source = None, "class_norm"
teap = synthesize_teap_from_program(
total_footprint_sqm=payload.total_footprint_sqm,
floors=payload.floors,
site_area_sqm=payload.site_area_sqm,
housing_class=payload.housing_class,
sections=payload.sections,
)
# compute_financial — чистая, CPU-bound, лёгкая (без БД) → можно вызывать прямо в
# event loop (hot path; threadpool-overhead тут не нужен). price_source игнорируется
# внутри, когда market_price_per_sqm is None (форсится class_norm) — корректно.
financial = compute_financial(
teap=teap,
housing_class=payload.housing_class,
land_cost_rub=payload.land_cost_rub,
market_price_per_sqm=market_price,
price_source=price_source,
development_type=payload.development_type,
)
return MassingRecomputeOutput(teap=teap, financial=financial)

View file

@ -3,6 +3,8 @@
from __future__ import annotations
import logging
import threading
import time
from datetime import date
from typing import Annotated, Any
@ -18,6 +20,32 @@ logger = logging.getLogger(__name__)
router = APIRouter()
# ---------------------------------------------------------------------------
# In-process TTL-кэш результата _query_stats (перф-батч #2, #1953-followup)
# ---------------------------------------------------------------------------
# /landing/stats бьётся на КАЖДЫЙ load хоумпейджа и НЕ кэширован. KPI 3 (доля квартир
# с ценой) делает полный агрегат по physflat-дедупу objective_lots: price_per_m2_rub
# нет в covering-INCLUDE и фильтра нет → ~6.7 s heap-скан 603k физлотов на каждый
# запрос. Данные дневной гранулярности (snapshot_date) → 1 ч staleness приемлем.
#
# Кэшируем ВЕСЬ dict (все 5 KPI), не только KPI 3 — там ещё pg_class deals + 2
# MAX(snapshot_date). Только УСПЕШНЫЙ результат (None/fallback НЕ кэшируем — пустая
# dev-БД / сбой запроса не должны залипать на час). Потокобезопасно (threading.Lock,
# удерживается только на check/store); analyze идёт в threadpool под Uvicorn. Стиль —
# зеркало app.services.weather_cache. time.monotonic устойчив к скачкам системных часов.
_STATS_TTL_S: float = 3600.0 # 1 час — дневная гранулярность данных
# (result_dict, expires_at_monotonic_seconds). Только успешный результат.
_STATS_CACHE: dict[str, Any] | None = None
_STATS_EXPIRES_AT: float = 0.0
_STATS_LOCK = threading.Lock()
def _now() -> float:
"""Монотонный таймер (тесты подменяют этот helper, а не сам `time`)."""
return time.monotonic()
# ---------------------------------------------------------------------------
# Response schema
# ---------------------------------------------------------------------------
@ -69,12 +97,14 @@ def _query_stats(db: Session) -> dict[str, Any] | None:
deals_total = int(row_deals or 0)
# KPI 3: % objective_lots с ценой
# #1964: physflat-дедуп view — доля квартир с ценой по ФИЗИЧЕСКИМ лотам, а не
# по пере-листингам (сырой objective_lots раздут ~2.91× мульти lot_id/физлот).
row_price = db.execute(
text(
"SELECT"
" COUNT(*) FILTER (WHERE price_per_m2_rub IS NOT NULL) * 100.0"
" / NULLIF(COUNT(*), 0)"
" FROM objective_lots"
" FROM v_objective_lots_latest"
)
).scalar()
price_coverage_pct = round(float(row_price or 0.0), 1)
@ -127,6 +157,30 @@ def _query_stats(db: Session) -> dict[str, Any] | None:
return None
def _query_stats_cached(db: Session) -> dict[str, Any] | None:
"""`_query_stats` с in-process TTL-кэшем (TTL=1 ч). См. блок-комментарий вверху.
Кэширует ТОЛЬКО успешный dict; None (сбой/пустая БД) не кэшируется следующий
запрос попробует снова. Lock удерживается лишь на check/store; сам DB-запрос идёт
ВНЕ lock'а (не сериализуем одновременные cold-start, last-write wins — запрос
идемпотентен, TTL длинный).
"""
global _STATS_CACHE, _STATS_EXPIRES_AT
now = _now()
with _STATS_LOCK:
if _STATS_CACHE is not None and _STATS_EXPIRES_AT > now:
return _STATS_CACHE
# MISS / истёк → DB-запрос ВНЕ lock'а.
result = _query_stats(db)
if result is None:
# Не кэшируем fallback — пусть следующий запрос повторит попытку.
return None
with _STATS_LOCK:
_STATS_CACHE = result
_STATS_EXPIRES_AT = _now() + _STATS_TTL_S
return result
@router.get("/landing/stats", response_model=LandingStatsOut)
def landing_stats(
db: Annotated[Session, Depends(get_db)],
@ -142,7 +196,7 @@ def landing_stats(
- paradox: строка-парадокс портфеля для hero CTA
- stale: true если данные недоступны (DB ошибка или пустая БД)
"""
result = _query_stats(db)
result = _query_stats_cached(db)
if result is None:
return LandingStatsOut(**_FALLBACK_DATA)
return LandingStatsOut(**result)

File diff suppressed because it is too large Load diff

View file

@ -114,6 +114,51 @@ class Settings(BaseSettings):
# Empty string = endpoint disabled.
scrape_admin_token: str = ""
# ── #1945 KN-loader anti-ban (throttle + optional proxy) ──────────────────
# DOM.РФ WAF банит IP по volume/rate (HTTP 403 «Доступ заблокирован», БЕЗ
# captcha — подтверждено): per-region sweep гонит ~1548 объектов × 11
# endpoint'ов ≈ 17k запросов через один браузер. На старой concurrency=8
# WAF банил VPS-IP mid-sweep. Лечим двумя рычагами.
#
# Рычаг 1 — throttle. Ограничивает число одновременных in-page fetch().
# Изначально вводился ТОЛЬКО для KN-sweep BrowserSession; с #2445 D2
# (2026-07) domrf_catalog.py / domrf_catalog_object.py (catalog-flat /
# catalog-object scrapers) тоже переиспользуют эти же значения — они бьют
# по тому же /сервисы/* path family, что и вызвало WAF hard-ban 2026-05-24
# (#2443). nspd и прочие скраперы вне этого path family продолжают
# использовать модульный дефолт _BROWSER_CONCURRENCY=8 без изменений.
# 2 — эмпирически безопасный потолок против volume-бана.
# ENV: SCRAPE_KN_BROWSER_CONCURRENCY.
scrape_kn_browser_concurrency: int = 2
# Окно случайной паузы (мс) между запросами KN-sweep (и, с #2445 D2, catalog-
# flat/catalog-object scrape'ов — см. комментарий выше). Шире дефолта
# (6001500), чтобы размазать запросы во времени и не триггерить rate-ban.
# min < max обязателен (иначе random.uniform отдаст границу). При throttle
# ширим до 12003000. ENV: SCRAPE_KN_REQUEST_JITTER_MIN_MS / _MAX_MS.
scrape_kn_request_jitter_min_ms: int = 1200
scrape_kn_request_jitter_max_ms: int = 3000
# Рычаг 2 — прокси (ОПЦИОНАЛЬНО, default None → прямое подключение, поведение
# без изменений). Когда задан — KN-sweep BrowserSession запускает Chromium
# через этот прокси (формат http://user:pass@host:port; парсится в
# Playwright proxy={server,username,password}). Сильнейший рычаг против
# IP-бана: переиспользует тот же mobile-proxy паттерн, что tradein-стек
# (SCRAPER_PROXY_URL). Заводится ТОЛЬКО через окружение прод-контейнера.
# ENV: SCRAPE_KN_PROXY_URL.
scrape_kn_proxy_url: str | None = None
# Рычаг 3 (ГЛАВНЫЙ unblock #1945) — изоляция flats от extras.
# Эмпирически (prod, 2026-06-27): flats endpoint /portal-kn/api/sales/portal/table
# на concurrency=2 = 0 WAF-бан на 100+ объектах; extras /сервисы/api/object/{id}/*
# отдают 403 СРАЗУ (volume-independent, мертвы с 2026-06-03) И ТРАВЯТ cookies
# сессии → последующие flats на той же сессии тоже 403. flats_count (метрика
# #1945) рухнул 3670→9 именно из-за этого.
#
# True (дефолт, NEW): flats тянутся в ЧИСТОЙ flats-only сессии (extras на ней
# НЕ дёргаются НИКОГДА) → flats_count восстанавливается; extras идут отдельным
# best-effort проходом с recycle сессии на каждый WAF-403 (яд не накапливается
# и НЕ касается flats). False: старое поведение (flats+extras в одной сессии,
# poison-prone). ENV: SCRAPE_KN_EXTRAS_ISOLATED.
scrape_kn_extras_isolated: bool = True
# NSPD-scraper (Playwright) УДАЛЁН 2026-05-11. Сменён на bulk geo-fetcher
# через rosreestr2coord — запускается вручную через /admin/scrape/geo.
# Settings оставлены deprecated на случай отката (можно удалить позже).
@ -164,6 +209,13 @@ class Settings(BaseSettings):
# Default rate-limit для nspd_lite fetcher (мс между запросами).
nspd_lite_rate_ms: int = 600
# Каталог хранения собранных PDF-отчётов ПТИЦА (эпик #2259 PR-D). Worker пишет
# PDF сюда, backend читает файл для /report/download. На проде — общий writable
# bind-mount `./reports:/app/reports` у сервисов backend+worker (docker-compose.prod.yml),
# иначе backend не увидит файл, записанный воркером. Для локальной разработки —
# относительный путь ОК (тест мокает render_full_report_pdf, файл не пишется).
reports_dir: str = "/app/reports"
# ETL Антоновского /sf/api/* SQLite → нашу PG (objective_* таблицы).
# На проде монтируется bind-mount-ом docker-compose:
# /opt/gendesign/site-finder/analysis.db -> /data/anton-sqlite/analysis.db
@ -306,5 +358,18 @@ class Settings(BaseSettings):
# Сколько ретраев на 429/5xx до деградации в fallback (циркуит-брейкер-lite).
llm_max_retries: int = 2
# ── DaData /clean/address геокод (objective_backfill geo-pass, #2177) ──────
# Токен + секрет для DaData /clean/address (обогащение адреса → geo_lat/geo_lon).
# На gendesign-проде уже заведены в окружении контейнера (проверено 2026-07-03).
# Оба нужны для clean_address (в отличие от suggest, которому хватает токена).
# None/пусто → dadata_client.clean_address graceful-возвращает None (geo-pass
# тогда reject'ит всё «нет геокода», не падает). ENV: DADATA_API_TOKEN /
# DADATA_API_SECRET. Литерала ключа в коде/тестах нет.
dadata_api_token: str | None = None
dadata_api_secret: str | None = None
# Таймаут одного HTTP-вызова к DaData (сек). Воркер geo-pass не должен висеть
# на недоступном сервисе. ENV: DADATA_TIMEOUT_S.
dadata_timeout_s: float = 8.0
settings = Settings()

View file

@ -0,0 +1,155 @@
# Russian Trusted CA bundle (Минцифры РФ) — публичные сертификаты с gu-st.ru (Госуслуги).
# Нужны для https-доменов с российским УЦ И/ИЛИ неполной серверной цепочкой (#2119).
# Скачаны 2026-07-02: russian_trusted_root_ca_pem.crt + russian_trusted_sub_ca_pem.crt.
-----BEGIN CERTIFICATE-----
MIIFwjCCA6qgAwIBAgICEAAwDQYJKoZIhvcNAQELBQAwcDELMAkGA1UEBhMCUlUx
PzA9BgNVBAoMNlRoZSBNaW5pc3RyeSBvZiBEaWdpdGFsIERldmVsb3BtZW50IGFu
ZCBDb21tdW5pY2F0aW9uczEgMB4GA1UEAwwXUnVzc2lhbiBUcnVzdGVkIFJvb3Qg
Q0EwHhcNMjIwMzAxMjEwNDE1WhcNMzIwMjI3MjEwNDE1WjBwMQswCQYDVQQGEwJS
VTE/MD0GA1UECgw2VGhlIE1pbmlzdHJ5IG9mIERpZ2l0YWwgRGV2ZWxvcG1lbnQg
YW5kIENvbW11bmljYXRpb25zMSAwHgYDVQQDDBdSdXNzaWFuIFRydXN0ZWQgUm9v
dCBDQTCCAiIwDQYJKoZIhvcNAQEBBQADggIPADCCAgoCggIBAMfFOZ8pUAL3+r2n
qqE0Zp52selXsKGFYoG0GM5bwz1bSFtCt+AZQMhkWQheI3poZAToYJu69pHLKS6Q
XBiwBC1cvzYmUYKMYZC7jE5YhEU2bSL0mX7NaMxMDmH2/NwuOVRj8OImVa5s1F4U
zn4Kv3PFlDBjjSjXKVY9kmjUBsXQrIHeaqmUIsPIlNWUnimXS0I0abExqkbdrXbX
YwCOXhOO2pDUx3ckmJlCMUGacUTnylyQW2VsJIyIGA8V0xzdaeUXg0VZ6ZmNUr5Y
Ber/EAOLPb8NYpsAhJe2mXjMB/J9HNsoFMBFJ0lLOT/+dQvjbdRZoOT8eqJpWnVD
U+QL/qEZnz57N88OWM3rabJkRNdU/Z7x5SFIM9FrqtN8xewsiBWBI0K6XFuOBOTD
4V08o4TzJ8+Ccq5XlCUW2L48pZNCYuBDfBh7FxkB7qDgGDiaftEkZZfApRg2E+M9
G8wkNKTPLDc4wH0FDTijhgxR3Y4PiS1HL2Zhw7bD3CbslmEGgfnnZojNkJtcLeBH
BLa52/dSwNU4WWLubaYSiAmA9IUMX1/RpfpxOxd4Ykmhz97oFbUaDJFipIggx5sX
ePAlkTdWnv+RWBxlJwMQ25oEHmRguNYf4Zr/Rxr9cS93Y+mdXIZaBEE0KS2iLRqa
OiWBki9IMQU4phqPOBAaG7A+eP8PAgMBAAGjZjBkMB0GA1UdDgQWBBTh0YHlzlpf
BKrS6badZrHF+qwshzAfBgNVHSMEGDAWgBTh0YHlzlpfBKrS6badZrHF+qwshzAS
BgNVHRMBAf8ECDAGAQH/AgEEMA4GA1UdDwEB/wQEAwIBhjANBgkqhkiG9w0BAQsF
AAOCAgEAALIY1wkilt/urfEVM5vKzr6utOeDWCUczmWX/RX4ljpRdgF+5fAIS4vH
tmXkqpSCOVeWUrJV9QvZn6L227ZwuE15cWi8DCDal3Ue90WgAJJZMfTshN4OI8cq
W9E4EG9wglbEtMnObHlms8F3CHmrw3k6KmUkWGoa+/ENmcVl68u/cMRl1JbW2bM+
/3A+SAg2c6iPDlehczKx2oa95QW0SkPPWGuNA/CE8CpyANIhu9XFrj3RQ3EqeRcS
AQQod1RNuHpfETLU/A2gMmvn/w/sx7TB3W5BPs6rprOA37tutPq9u6FTZOcG1Oqj
C/B7yTqgI7rbyvox7DEXoX7rIiEqyNNUguTk/u3SZ4VXE2kmxdmSh3TQvybfbnXV
4JbCZVaqiZraqc7oZMnRoWrXRG3ztbnbes/9qhRGI7PqXqeKJBztxRTEVj8ONs1d
WN5szTwaPIvhkhO3CO5ErU2rVdUr89wKpNXbBODFKRtgxUT70YpmJ46VVaqdAhOZ
D9EUUn4YaeLaS8AjSF/h7UkjOibNc4qVDiPP+rkehFWM66PVnP1Msh93tc+taIfC
EYVMxjh8zNbFuoc7fzvvrFILLe7ifvEIUqSVIC/AzplM/Jxw7buXFeGP1qVCBEHq
391d/9RAfaZ12zkwFsl+IKwE/OZxW8AHa9i1p4GO0YSNuczzEm4=
-----END CERTIFICATE-----
-----BEGIN CERTIFICATE-----
MIIHQjCCBSqgAwIBAgICEAIwDQYJKoZIhvcNAQELBQAwcDELMAkGA1UEBhMCUlUx
PzA9BgNVBAoMNlRoZSBNaW5pc3RyeSBvZiBEaWdpdGFsIERldmVsb3BtZW50IGFu
ZCBDb21tdW5pY2F0aW9uczEgMB4GA1UEAwwXUnVzc2lhbiBUcnVzdGVkIFJvb3Qg
Q0EwHhcNMjIwMzAyMTEyNTE5WhcNMjcwMzA2MTEyNTE5WjBvMQswCQYDVQQGEwJS
VTE/MD0GA1UECgw2VGhlIE1pbmlzdHJ5IG9mIERpZ2l0YWwgRGV2ZWxvcG1lbnQg
YW5kIENvbW11bmljYXRpb25zMR8wHQYDVQQDDBZSdXNzaWFuIFRydXN0ZWQgU3Vi
IENBMIICIjANBgkqhkiG9w0BAQEFAAOCAg8AMIICCgKCAgEA9YPqBKOk19NFymrE
wehzrhBEgT2atLezpduB24mQ7CiOa/HVpFCDRZzdxqlh8drku408/tTmWzlNH/br
HuQhZ/miWKOf35lpKzjyBd6TPM23uAfJvEOQ2/dnKGGJbsUo1/udKSvxQwVHpVv3
S80OlluKfhWPDEXQpgyFqIzPoxIQTLZ0deirZwMVHarZ5u8HqHetRuAtmO2ZDGQn
vVOJYAjls+Hiueq7Lj7Oce7CQsTwVZeP+XQx28PAaEZ3y6sQEt6rL06ddpSdoTMp
BnCqTbxW+eWMyjkIn6t9GBtUV45yB1EkHNnj2Ex4GwCiN9T84QQjKSr+8f0psGrZ
vPbCbQAwNFJjisLixnjlGPLKa5vOmNwIh/LAyUW5DjpkCx004LPDuqPpFsKXNKpa
L2Dm6uc0x4Jo5m+gUTVORB6hOSzWnWDj2GWfomLzzyjG81DRGFBpco/O93zecsIN
3SL2Ysjpq1zdoS01CMYxie//9zWvYwzI25/OZigtnpCIrcd2j1Y6dMUFQAzAtHE+
qsXflSL8HIS+IJEFIQobLlYhHkoE3avgNx5jlu+OLYe0dF0Ykx1PGNjbwqvTX37R
Cn32NMjlotW2QcGEZhDKj+3urZizp5xdTPZitA+aEjZM/Ni71VOdiOP0igbw6asZ
2fxdozZ1TnSSYNYvNATwthNmZysCAwEAAaOCAeUwggHhMBIGA1UdEwEB/wQIMAYB
Af8CAQAwDgYDVR0PAQH/BAQDAgGGMB0GA1UdDgQWBBTR4XENCy2BTm6KSo9MI7NM
XqtpCzAfBgNVHSMEGDAWgBTh0YHlzlpfBKrS6badZrHF+qwshzCBxwYIKwYBBQUH
AQEEgbowgbcwOwYIKwYBBQUHMAKGL2h0dHA6Ly9yb3N0ZWxlY29tLnJ1L2NkcC9y
b290Y2Ffc3NsX3JzYTIwMjIuY3J0MDsGCCsGAQUFBzAChi9odHRwOi8vY29tcGFu
eS5ydC5ydS9jZHAvcm9vdGNhX3NzbF9yc2EyMDIyLmNydDA7BggrBgEFBQcwAoYv
aHR0cDovL3JlZXN0ci1wa2kucnUvY2RwL3Jvb3RjYV9zc2xfcnNhMjAyMi5jcnQw
gbAGA1UdHwSBqDCBpTA1oDOgMYYvaHR0cDovL3Jvc3RlbGVjb20ucnUvY2RwL3Jv
b3RjYV9zc2xfcnNhMjAyMi5jcmwwNaAzoDGGL2h0dHA6Ly9jb21wYW55LnJ0LnJ1
L2NkcC9yb290Y2Ffc3NsX3JzYTIwMjIuY3JsMDWgM6Axhi9odHRwOi8vcmVlc3Ry
LXBraS5ydS9jZHAvcm9vdGNhX3NzbF9yc2EyMDIyLmNybDANBgkqhkiG9w0BAQsF
AAOCAgEARBVzZls79AdiSCpar15dA5Hr/rrT4WbrOfzlpI+xrLeRPrUG6eUWIW4v
Sui1yx3iqGLCjPcKb+HOTwoRMbI6ytP/ndp3TlYua2advYBEhSvjs+4vDZNwXr/D
anbwIWdurZmViQRBDFebpkvnIvru/RpWud/5r624Wp8voZMRtj/cm6aI9LtvBfT9
cfzhOaexI/99c14dyiuk1+6QhdwKaCRTc1mdfNQmnfWNRbfWhWBlK3h4GGE9JK33
Gk8ZS8DMrkdAh0xby4xAQ/mSWAfWrBmfzlOqGyoB1U47WTOeqNbWkkoAP2ys94+s
Jg4NTkiDVtXRF6nr6fYi0bSOvOFg0IQrMXO2Y8gyg9ARdPJwKtvWX8VPADCYMiWH
h4n8bZokIrImVKLDQKHY4jCsND2HHdJfnrdL2YJw1qFskNO4cSNmZydw0Wkgjv9k
F+KxqrDKlB8MZu2Hclph6v/CZ0fQ9YuE8/lsHZ0Qc2HyiSMnvjgK5fDc3TD4fa8F
E8gMNurM+kV8PT8LNIM+4Zs+LKEV8nqRWBaxkIVJGekkVKO8xDBOG/aN62AZKHOe
GcyIdu7yNMMRihGVZCYr8rYiJoKiOzDqOkPkLOPdhtVlgnhowzHDxMHND/E2WA5p
ZHuNM/m0TXt2wTTPL7JH2YC0gPz/BvvSzjksgzU5rLbRyUKQkgU=
-----END CERTIFICATE-----
# HARICA DV TLS Sub R1 (промежуточный, crt.harica.gr) — ekaterinburg-tr.gazprom.ru
# отдаёт ТОЛЬКО leaf без intermediate (misconfig сервера); Python AIA-chasing не делает.
# Добавлен 2026-07-02.
-----BEGIN CERTIFICATE-----
MIIGwTCCBKmgAwIBAgIQFVuBuF22xppLHnbzSLBwXDANBgkqhkiG9w0BAQsFADBs
MQswCQYDVQQGEwJHUjE3MDUGA1UECgwuSGVsbGVuaWMgQWNhZGVtaWMgYW5kIFJl
c2VhcmNoIEluc3RpdHV0aW9ucyBDQTEkMCIGA1UEAwwbSEFSSUNBIFRMUyBSU0Eg
Um9vdCBDQSAyMDIxMB4XDTIxMDMxOTA5MjQwNFoXDTM2MDMxNTA5MjQwM1owYjEL
MAkGA1UEBhMCR1IxNzA1BgNVBAoMLkhlbGxlbmljIEFjYWRlbWljIGFuZCBSZXNl
YXJjaCBJbnN0aXR1dGlvbnMgQ0ExGjAYBgNVBAMMEUhBUklDQSBEViBUTFMgUlNB
MIICIjANBgkqhkiG9w0BAQEFAAOCAg8AMIICCgKCAgEAvmwtJxeWVmV1TqMSobWh
KsaKFvcBVNNfr2swkNI6l9PbH8e7uXVbpMCfG7VV3N22v65pJZR7Faxlfafs+hxt
qJNS6SunkXsc2368zlVD8ckRJl+ktVz2Ps+q/VsCvG+hcgUxShlM6BpqJ6aJqH3a
zLyP7b98/r+wnrmktQLLATJ9keWFC/tngso76KNctRR92br+cv0niYoacw2+cuWr
Y43J60DcE2fN07hrt4NA2V6I4d2GNWVJHteNSwMsC5GY/opmoN4bhObbAIEne3oZ
SKP/4pzC9aA9pIaD/JYA5jiD2R85neoYVXeR3PzWqHe+3I62JFbDdLgyjgzVu0lE
RkzmIg3UioLBgT7aRxkEQTiiIgdXYHi9CQ00Y9idgBKE0E5GmuJw0JegWk/kozdF
acjUPmTdq7AA1wbrTWOAKJr541B1ObOUe+FG9G1dDwcpSx6OIqNjq04ifoKlSPcN
RoFakaSW2lEeOg/1SCieLtDniftIOnzz0+kGIVU4TTTHrWwPGZs3a96u/JlZb87D
xZGbyxfRgIonWw/WVjBwas4GIGY8oECS43iJh4JaZaqdt4LdE+cwvpZWaKi8N+o1
qovjMLpcnAUZ/cWaH2t266z+ATCGMdEtorhf0fQxE/dUYgJEBmw2r9olJJRkXVDU
t/1shKYedGuayZucrhiCL6sCAwEAAaOCAWcwggFjMBIGA1UdEwEB/wQIMAYBAf8C
AQAwHwYDVR0jBBgwFoAUCkgjpmCkkgoz6pNbxVfqJU29Eu4wVAYIKwYBBQUHAQEE
SDBGMEQGCCsGAQUFBzAChjhodHRwOi8vcmVwby5oYXJpY2EuZ3IvY2VydHMvSEFS
SUNBLVRMUy1Sb290LTIwMjEtUlNBLmNlcjBEBgNVHSAEPTA7MDkGBFUdIAAwMTAv
BggrBgEFBQcCARYjaHR0cDovL3JlcG8uaGFyaWNhLmdyL2RvY3VtZW50cy9DUFMw
HQYDVR0lBBYwFAYIKwYBBQUHAwIGCCsGAQUFBwMBMEIGA1UdHwQ7MDkwN6A1oDOG
MWh0dHA6Ly9jcmwuaGFyaWNhLmdyL0hBUklDQS1UTFMtUm9vdC0yMDIxLVJTQS5j
cmwwHQYDVR0OBBYEFAqIq7yL8PVK7GFEUEBEwYdm3t5RMA4GA1UdDwEB/wQEAwIB
hjANBgkqhkiG9w0BAQsFAAOCAgEAiFIdKz8OS9mWAcAsENHqY0S1qTG+KD8IX6yU
0Ycn8thjErylOClJtq/DAmPGUylA0b2ZctLrIPRWBDqXI0f6VMPDBk7kCBwX8F/T
l6JSBQ+sGpdAx5XQxIurJE3cKbduhetf5mpIGTsIDh2V094bJlxiufi0mmB1M/QE
sohaEXfmQn6nQQtiVZOmTtNr2dZ1lyTRsKHjE9McFIeczZNP/sEgYI+SWijNjp8K
7DAJOmw+8NfUjf7dq4HqLl2MbJSPEaLfNOENjWT+wygPzAKyKl2lcBgHl3FYR14u
qkqMUg25Euq2IbkDPpaiq8i1+3A2w+SPRH5sU+g13dlFyCal47yQIIFbeA+JXSPg
i1PK/ORDJuBAszOD3q7PNf64FPAjN1OKIZAGoCBjHaS0aGED5Jviqao81hSlo0A0
y5zb9JaJPoSV/wyIc3bgS+niWDhBuBCsbmfH1TfRBTfI8O9mXrCSxSxcJ10fhZVy
8SyJc+PeOJcU+VM4emO9mAhV6sv5ioKXdrhUlgLIhbsRJW30EyV5H7Z2MGAGGMSP
Rp1k43BrIngThpXwdQmGOsj6KS7PuqkKhMvIZVQ9lDcFnNI0ySk3BPsuTVO7yo92
03GT/P4nsjwqpCc6XoCpPb42EqyFRHH5GJ9ca1ePkKxs3lApHnW9E4rmIf5tSOnt
jISnqoU=
-----END CERTIFICATE-----
# GlobalSign GCC R6 AlphaSSL CA 2025 (промежуточный, AIA secure.globalsign.com) —
# ri.eias.ru / ri-loader.eias.ru не шлют intermediate. Добавлен 2026-07-02.
-----BEGIN CERTIFICATE-----
MIIFjTCCA3WgAwIBAgIRAIN9TriekS/nLK07x2kt3CAwDQYJKoZIhvcNAQELBQAw
TDEgMB4GA1UECxMXR2xvYmFsU2lnbiBSb290IENBIC0gUjYxEzARBgNVBAoTCkds
b2JhbFNpZ24xEzARBgNVBAMTCkdsb2JhbFNpZ24wHhcNMjUwNTIxMDIzNjUyWhcN
MjcwNTIxMDAwMDAwWjBVMQswCQYDVQQGEwJCRTEZMBcGA1UEChMQR2xvYmFsU2ln
biBudi1zYTErMCkGA1UEAxMiR2xvYmFsU2lnbiBHQ0MgUjYgQWxwaGFTU0wgQ0Eg
MjAyNTCCASIwDQYJKoZIhvcNAQEBBQADggEPADCCAQoCggEBAJ/oiu0Bviq52UUE
ADbFWmgu3rC7KDSMoorLN1Wd03McG3Z1aP71DlPCE33838r72Dfuj5M9LXfiQLJp
Au6MwNExmKOzothw4x0zGf5oBYyrCMGm3fBpLPafwYQ3MchBOWMTbf83rKUPLH48
KCJ0MnU8GUl8oA/J81wIvbbKPuNrFf6hvJDccjzc4NyxLz3A89zjV2g5whCg5O0u
9YX4Zxk9JHuc/LvllOJO4waAYLjbWBJkz3rV3ts1SmSYnJqmyRTIjXwQgRvhEYqt
DbRskt0W7M6cPwCze3GTBN2UHNpHkMs3YmVxku68I0aOQn5+uz//fDROP3z1Z/7I
APteRtECAwEAAaOCAV8wggFbMA4GA1UdDwEB/wQEAwIBhjAdBgNVHSUEFjAUBggr
BgEFBQcDAQYIKwYBBQUHAwIwEgYDVR0TAQH/BAgwBgEB/wIBADAdBgNVHQ4EFgQU
xbSTj28r3B5Iv7cQMIXO0bK7SC0wHwYDVR0jBBgwFoAUrmwFo5MT4qLn4tcc1sfw
f8hnU6AwewYIKwYBBQUHAQEEbzBtMC4GCCsGAQUFBzABhiJodHRwOi8vb2NzcDIu
Z2xvYmFsc2lnbi5jb20vcm9vdHI2MDsGCCsGAQUFBzAChi9odHRwOi8vc2VjdXJl
Lmdsb2JhbHNpZ24uY29tL2NhY2VydC9yb290LXI2LmNydDA2BgNVHR8ELzAtMCug
KaAnhiVodHRwOi8vY3JsLmdsb2JhbHNpZ24uY29tL3Jvb3QtcjYuY3JsMCEGA1Ud
IAQaMBgwCAYGZ4EMAQIBMAwGCisGAQQBoDIKAQMwDQYJKoZIhvcNAQELBQADggIB
AB/uvBuZf4CiuSahwiXn4geF52roAH+6jxsEPTXTfb7bbeMDXsYgRRsOTNA70ruZ
Tnz5DfFMuBhNoFhIFb0qR1izdy6VkdKOqFPNF2dOFI1EcnY9l2ory9mrzHqVbrL4
vzUd17FLUVyjTVU7PAv4nxyhnO1GTeT83YlrdRF31NyR6bvZVTEERHmpbWSgeveJ
LRtaMzlGWiLZ8IwkH7o6GH3jp/KPtDW4Npu8w64HrRZdN2pqQhi7+YKwfHM7H+2U
dM1BGN0sjOWMVbMSB9MtCsleS2Mb7TRZEbOHxECJLLIluQypZr7Pol3+hAqrhyKI
k+6y+Da0NeDuWxW59Ku4NvClqW1UFX1SpfNGhzVfp/CH+vPM1tySomx2jE0EnYZu
GwVucXPBsp5nUWqUV9+143glVuS7GTg9hFPjNBInn17HbCoIIQIOzj5Vd9bK3A9U
GxXNpwenDHEalCsD/4eQYDHPhFE7sNe0D/OXu+FAM02VZkARx37Jp4bDdujvgL9P
vZPR3wThvDN1CTU8Bc3xea3yKFAraKcPZLkhReQUAm2VpR+HSJRPlUpYizlF9WkL
h3KcAVCBJWvnOkVwxyU5QJMcnwW95JlOtx+9100GL99jHE5rs3gXp7F4bg8H01QT
9jVOhBBmQ7nQoXuwI0tqal2QUqZz3eeu62CU7xBwtfYR
-----END CERTIFICATE-----

View file

@ -1,8 +1,72 @@
from typing import Any, Literal
from typing import Annotated, Any, Literal
from pydantic import BaseModel, Field
class BuildingProgramItem(BaseModel):
"""Stage 3a (#1965, эпик #1953) — один пункт программы застройки (типовой дом × N).
Пользователь набирает программу из ТИПОВЫХ домов каталога
(``app.services.generative.catalog.HOUSE_TYPES``) вместо max-FAR жадной раскладки:
«поставь ``count`` секций типа ``section_type`` этажностью ``floors``». Габариты
пятна секции берутся из каталога по ключу ``section_type`` (контракт несёт только
ключ, не геометрию единый справочник на бэке).
"""
# Ключ типа дома из каталога (``HouseType.section_type``). Валидируется на слое
# размещения по ``catalog.available_section_types`` — неизвестный ключ → 422.
section_type: str = Field(..., description="Catalog house-type key (HOUSE_TYPES)")
# Этажность этой группы секций; диапазон шире, чем у target_floors (до 40 — башни).
floors: int = Field(..., ge=1, le=40, description="Floors for this house-type group")
# Сколько секций этого типа разместить.
count: int = Field(..., ge=1, le=50, description="Number of sections to place")
# ADDITIVE (Stage 3c): опциональное ручное пятно секции (ширина × глубина, м).
# None → габариты берутся из каталога по ``section_type`` (byte-for-byte
# backward-compat). Заданы → place_program строит footprint из них вместо
# каталожных (пользователь «вписал пятно»). Оба должны быть заданы вместе;
# частичное задание (только w / только d) трактуется как «не задано» на слое
# размещения. Границы [4, 120] м — вменяемый габарит секции/корпуса.
footprint_w_m: float | None = Field(
None, ge=4, le=120, description="Optional manual section footprint width, m"
)
footprint_d_m: float | None = Field(
None, ge=4, le=120, description="Optional manual section footprint depth, m"
)
class HouseTypeCatalogItem(BaseModel):
"""Stage 3b (#1965, эпик #1953) — один тип дома каталога, ОТДАВАЕМЫЙ фронту.
Read-only проекция ``app.services.generative.catalog.HouseType`` для пикера
Stage 3b: фронт показывает ``label_ru`` + габариты пятна + дефолтную этажность
+ класс, а в ``BuildingProgramItem.section_type`` кладёт ``section_type``. Это
делает каталог ЕДИНЫМ источником истины фронт ничего не хардкодит. Поля
зеркалят dataclass на бэке (см. ``GET /api/v1/concepts/house-types``).
"""
# Стабильный машинный ключ типа — значение для BuildingProgramItem.section_type.
section_type: str = Field(..., description="Catalog house-type key (HOUSE_TYPES)")
# Человекочитаемый русский лейбл для UI.
label_ru: str = Field(..., description="Human-readable Russian label")
# Габариты пятна секции, метры (ширина × глубина).
footprint_w_m: float = Field(..., description="Section footprint width, m")
footprint_d_m: float = Field(..., description="Section footprint depth, m")
# Площадь пятна секции, кв.м (helper для UI: ширина × глубина).
footprint_sqm: float = Field(..., description="Section footprint area, m² (w × d)")
# Дефолтная этажность типа — UI подставляет, пользователь меняет в [1, 40].
default_floors: int = Field(..., description="Default floors for this type")
# Подходящий класс жилья (econom / comfort / business).
housing_class: Literal["econom", "comfort", "business"] = Field(
..., description="Suitable housing class"
)
class HouseTypeCatalog(BaseModel):
"""Stage 3b (#1965) — каталог типовых домов, отдаваемый ``GET /concepts/house-types``."""
house_types: list[HouseTypeCatalogItem]
class ConceptInput(BaseModel):
"""Stage 1a — input contract. Frozen interface for frontend codegen."""
@ -15,6 +79,15 @@ class ConceptInput(BaseModel):
land_cost_rub: float | None = Field(
None, ge=0, description="Optional land cost for financial model"
)
# Stage 3a (#1965): ОПЦИОНАЛЬНАЯ программа застройки из типовых домов каталога.
# ADDITIVE поле (default None). None → существующая жадная раскладка (Stage 1b,
# byte-for-byte backward-compat). Задано → раскладка кладёт РОВНО эту программу
# (place_program): для каждого пункта ставит count секций каталожного пятна с его
# floors, вместо coverage-cap sweep. См. app.services.generative.placement.
building_program: list[BuildingProgramItem] | None = Field(
None,
description="Optional typed house program; None → greedy max-FAR placement",
)
class TEAP(BaseModel):
@ -147,7 +220,71 @@ class ConceptVariant(BaseModel):
buildings_geojson: dict[str, Any]
teap: TEAP
financial: FinancialModel
# Stage 3a (#1965) — честный сигнал частичного размещения для program-режима.
# Когда задан ``ConceptInput.building_program``: ``requested_count`` — сколько секций
# просили (Σ count по программе), ``placed_count`` — сколько реально влезло в участок.
# placed < requested → участок мал, разместилось N из M (фронт Stage 3b показывает
# «разместилось N из M», без hard-422). Оба None в greedy-режиме (программа не задана)
# → ADDITIVE, backward-compat: старый ответ концепции не меняется. Annotated+`= None`:
# рантайм-дефолт виден mypy (без pydantic-плагина), Field несёт только OpenAPI-описание.
placed_count: Annotated[
int | None, Field(description="Stage 3a: sections actually placed (program mode only)")
] = None
requested_count: Annotated[
int | None, Field(description="Stage 3a: sections requested by program (program mode only)")
] = None
class ConceptOutput(BaseModel):
variants: list[ConceptVariant]
class MassingProgram(BaseModel):
"""Stage 2a (#1965) — агрегированная массинг-программа для LIVE-пересчёта экономики.
Контракт POST ``/api/v1/concepts/recompute``: вход уже СВЁРНУТАЯ программа из
фронтового ``computeModel`` интерактивного 3D-массинга (суммарное пятно × этажность),
без покомпонентной геометрии секций. Эндпоинт синтезирует из неё :class:`TEAP` и
прогоняет готовый ``compute_financial`` пересчитанная экономика для дебаунс-слайдеров
Stage 2b.
Класс жилья / тип застройки те же Literal-множества, что у :class:`ConceptInput`
и ``compute_financial`` (single source of truth по допустимым значениям).
"""
total_footprint_sqm: float = Field(
..., ge=0, description="Суммарное пятно застройки всех секций, кв.м (скаляр)"
)
floors: int = Field(..., ge=1, le=40, description="Этажность программы")
sections: int = Field(1, ge=1, le=6, description="Число секций (метаданные программы)")
site_area_sqm: float = Field(..., ge=0, description="Площадь участка для плотности (FAR)")
housing_class: Literal["econom", "comfort", "business"] = "comfort"
development_type: Literal["spot", "mid_rise", "high_rise"] = "mid_rise"
land_cost_rub: float | None = Field(
None, ge=0, description="Стоимость участка для финмодели (опционально)"
)
# Предрезолвленная рыночная цена жилья, руб/кв.м — позволяет hot-эндпоинту ПРОПУСТИТЬ
# per-keystroke DB-lookup (фронт резолвит цену один раз и шлёт её в каждом запросе).
# None → эндпоинт сам сделает _lookup_market_price по центроиду (медленный путь).
market_price_per_sqm: float | None = Field(
None, ge=0, description="Предрезолвленная рыночная цена жилья, руб/кв.м (skip DB lookup)"
)
# ПОДЛИННЫЙ источник предрезолвленной цены, который фронт получил из financial_estimate
# (objective_geo_radius / objective_district_median / district_reference / class_norm).
# Honest-флаг для UI/PDF: НЕ подменяем его захардкоженным лейблом. Применяется ТОЛЬКО
# вместе с market_price_per_sqm; None → дефолтный лейбл (см. эндпоинт recompute_massing).
price_source: str | None = Field(
None, description="Подлинный источник предрезолвленной цены (forward из financial_estimate)"
)
# Центроид участка (WKT-точка WGS84) для DB-fallback цены, когда market_price_per_sqm
# не передан. None → пропускаем lookup и берём норматив класса (нет геопривязки).
parcel_centroid_wkt: str | None = Field(
None, description="WKT-точка центроида участка (WGS84) для DB-резолва цены при fallback"
)
class MassingRecomputeOutput(BaseModel):
"""Stage 2a (#1965) — результат LIVE-пересчёта: синтезированный ТЭП + финмодель."""
teap: TEAP
financial: FinancialModel

View file

@ -36,9 +36,16 @@ class EngineeringStructure(BaseModel):
name: str | None
type: str | None
cad_num: str | None
# Назначение сооружения из НСПД-дампа (params_purpose / object_type_value),
# напр. «1.1. Сооружения электроэнергетики». Заполняется ТОЛЬКО когда отличается
# от `type` (иначе дубль-шум). ADDITIVE (default None).
purpose: str | None = None
distance_to_boundary_m: float
geometry_geojson: dict[str, Any]
readable_address: str | None
# Доп. характеристики из НСПД-дампа (год постройки / длина / собственность),
# собранные в одну читаемую строку. None — данных нет. ADDITIVE (default None).
characteristics: str | None = None
raw_props: dict[str, Any]
source: str
@ -58,6 +65,11 @@ class ConnectionPointsSummary(BaseModel):
in_protection_zone: bool
protection_zones_intersecting: int
total_structures_in_radius: int
# Epic #2445 A1: честные флаги усечения — True, когда истинный count (COUNT(*) без
# LIMIT, по той же выборке) больше длины отдаваемого списка (LIMIT 100/50 в SQL).
# ADDITIVE (default False) — старые клиенты не ломаются.
protection_zones_truncated: bool = False
structures_truncated: bool = False
class ConnectionPointsResponse(BaseModel):
@ -82,6 +94,10 @@ class UtilityInfrastructureFeature(BaseModel):
source_tag: str | None
distance_m: float
geometry_geojson: dict[str, Any]
# Гуманизированные характеристики сети из OSM tags (напряжение / оператор /
# диаметр / материал), собранные в одну строку. None — данных нет.
# ADDITIVE (default None) → не ломает существующих потребителей.
characteristics: str | None = None
class UtilityInfrastructureSummary(BaseModel):
@ -89,6 +105,10 @@ class UtilityInfrastructureSummary(BaseModel):
nearest_distance_m: float | None
# Карта вид сети → расстояние до ближайшего объекта данного вида (м), либо null.
nearest_by_kind: dict[str, float | None]
# Epic #2445 A2: True, когда истинный count (COUNT(*) без LIMIT, по той же
# ST_DWithin-выборке) больше длины отдаваемого `features` (LIMIT :lim, default 200).
# ADDITIVE (default False) — старые клиенты не ломаются.
features_truncated: bool = False
class UtilityInfrastructureResponse(BaseModel):
@ -96,6 +116,190 @@ class UtilityInfrastructureResponse(BaseModel):
summary: UtilityInfrastructureSummary
# ── Connection-capacity schemas (Forgejo #2119 Phase A) ──────────────────────
# «Точки подключения СО СВОБОДНОЙ МОЩНОСТЬЮ» для ПТИЦА §3: центры питания
# (power_supply_centers) с резервом + city-level резервы воды (water_supply_reserves).
class PowerConnectionPoint(BaseModel):
"""Центр питания (ЦП, ПС 35/110 кВ) вблизи участка со свободной мощностью."""
name: str
dzo_name: str | None
voltage_class: str | None # напр. '110/35/10'
load_index: str | None # 'open'|'limited'|'closed'|None
installed_capacity_mva: float | None
current_load_mva: float | None
reserve_mva: float | None # свободная мощность, МВА
reserve_asof: str | None # ISO-дата актуальности резерва
distance_m: float
lat: float | None
lon: float | None
# Пометка для ПС ЕЭСК без опубликованного резерва: свободная мощность у АО «ЕЭСК»
# (городская сеть ЕКБ) не раскрывается WFS/xlsx — берётся по запросу. #2119 B1.
reserve_note: str | None = None
# Единица reserve_mva/installed_capacity_mva: 'МВА' (Россети) или 'МВт'
# (ЕЭСК ПС 35-220). district — адм. район ЕКБ (из xlsx ЕЭСК; NULL у Россети).
# ADDITIVE (default None). #2119 Phase B2.
reserve_unit: str | None = None
district: str | None = None
class PowerCapacitySummary(BaseModel):
"""Сводка по электрике: ближайший ЦП с резервом + count по индексу загрузки."""
total_power_points: int
# Карта индекс загрузки ('open'|'limited'|'closed'|'unknown') → count.
by_load_index: dict[str, int]
# Ближайший ЦП с положительным резервом (или null, если резерв нигде не задан).
nearest_with_reserve: PowerConnectionPoint | None
class WaterCapacityRow(BaseModel):
"""Строка резерва ЦСВ/ЦСК Водоканала (city-level) за последний период."""
system_kind: str # 'water'|'sewerage'
system_name: str # ЦСВ/ЦСК, напр. «ЗФС, ГСВ»
reserve_thousand_m3_day: float | None # может быть отрицательным (дефицит)
note: str | None
period: str | None
class GasGrsRow(BaseModel):
"""Свободная мощность одной городской ГРС (агрегация по выходам). #2119 B1."""
grs_name: str
design_capacity_th_m3_h: float | None # проектная (сумма по выходам), тыс. м³/ч
free_capacity_th_m3_h: float | None # свободная (сумма по выходам), тыс. м³/ч
free_capacity_pct: float | None # средняя свободная, %
upgrade_due: str | None # срок расшивки (макс. по выходам, свободный текст)
outputs_count: int # число строк-выходов ГРС, вошедших в агрегацию
class GasCapacityBlock(BaseModel):
"""Газовый блок: городские ГРС ЕКБ со свободной мощностью. #2119 Фаза B1.
+ счётчики точек выхода газосети (gas_grs_outlet_points, форма 6 ФАС 960 АО
«Екатеринбурггаз») за последний срез. #2119 Фаза B2 PR-4.
"""
# Городские ГРС (Свердловск/Екатеринбург) — агрегация по grs_name. Пусто, если
# таблица gas_grs_capacity ещё не наполнена (weekly-sync, RUN-ON-PROD).
city_grs: list[GasGrsRow]
# Суммарная свободная мощность городских ГРС, тыс. м³/ч (None если данных нет).
total_free_th_m3_h: float | None
# Счётчики точек выхода последнего period_month (по ВСЕМ строкам среза, city-level —
# см. _query_gas_outlet_counts). ADDITIVE (default 0): 0, если таблица
# gas_grs_outlet_points ещё не наполнена (миграция 184 / monthly-sync). #2119 B2 PR-4.
# ВНИМАНИЕ: свободная мощность точек выхода — млн. м³/МЕС (НЕ тыс. м³/ч как у ГРС!).
outlets_total: int = 0 # всего точек выхода в последнем срезе
outlets_deficit: int = 0 # из них с дефицитом (free_capacity_mln_m3 < 0)
outlets_needs_calc: int = 0 # из них «0.000*» — нужен гидравлический расчёт
class GasOutletPoint(BaseModel):
"""Гео-точка выхода газосети в радиусе от участка (для слоя карты §3). #2119 B2 PR-4."""
outlet_name: str # «Точка выхода» — юрлицо/объект/промплощадка (полный текст)
consumer_type: str | None # котельная / ТЭЦ / площадка …
# Свободная мощность точки выхода, млн. м³/МЕС (НЕ тыс. м³/ч как у ГРС!). Может быть
# <0 (дефицит со знаком). NULL, когда в источнике «0.000*» (needs_calc=True).
free_capacity_mln_m3: float | None
needs_calc: bool # «0.000*» — свободная мощность не определена, нужен гидрорасчёт
distance_m: float # расстояние от центроида участка, м
lat: float | None
lon: float | None
class HeatSystemRow(BaseModel):
"""Резерв тепловой мощности одной системы теплоснабжения. #2119 Phase B2."""
org: str # организация-держатель (ЕТК / Екатеринбургэнерго)
system_name: str # имя системы теплоснабжения (напр. «СТ №54»)
reserve_gcal_h: float | None # свободная мощность, Гкал/ч (может быть <0 — дефицит)
period: str | None # напр. '2026-Q2'
class HeatCapacityBlock(BaseModel):
"""Тепловой блок: резервы систем теплоснабжения за последний период. #2119 B2."""
# Строки heat_system_reserves за MAX(period) ПО КАЖДОЙ организации. Пусто, если
# таблица heat_system_reserves ещё не наполнена (weekly-sync, RUN-ON-PROD).
systems: list[HeatSystemRow]
# Суммарная свободная тепловая мощность, Гкал/ч (None если данных нет).
total_reserve_gcal_h: float | None
class NearbyNetworkZone(BaseModel):
"""Сетевая охранная зона cad_zouit рядом с участком — позитив-сигнал §3. #2119 C2/C3.
Охранная зона сети = коридор, где физически проходит сеть «теплотрасса в 24 м,
врезка дешёвая». Зона, ПЕРЕСЕКАЮЩАЯ участок, отдаётся с distance_m=0.
"""
# Вид сети: thermal/electricity/gas/water/communication/pipeline/general.
# general дочищен по name_by_doc (инж.коммуникации на проде часто реально тепло/газ).
net_kind: str
distance_m: int # расстояние от участка до охранной зоны, м (0 = пересекает участок)
name_by_doc: str | None # сырое детальное имя из cad_zouit (для тултипа/дебага)
# Гуманизированная подпись (C3): «Тепловая сеть (по охранной зоне), кад. № …» /
# «Ввод 10 кВ от ТП 3164 (по охранной зоне)» / «Инженерные коммуникации (вид не уточнён)».
human_label: str
class ConnectionCapacityResponse(BaseModel):
power_points: list[PowerConnectionPoint]
power_summary: PowerCapacitySummary
water: list[WaterCapacityRow]
# Газовый блок аддитивен (Phase B1): None если ключ 'gas' не собран.
gas: GasCapacityBlock | None = None
# Гео-точки выхода газосети в радиусе от участка (слой карты §3). ADDITIVE (Phase B2
# PR-4): [] если таблица gas_grs_outlet_points пуста / нет geom в радиусе.
gas_outlet_points: list[GasOutletPoint] = []
# Тепловой блок аддитивен (Phase B2): None если ключ 'heat' не собран.
heat: HeatCapacityBlock | None = None
# Позитив-разрез (Phase C2/C3): сетевые охранные зоны cad_zouit в радиусе 300 м —
# до 2 ближайших на net_kind, гуманизированы. ADDITIVE (default []): [] если сеть
# рядом не найдена / нет geom участка.
nearby_network_zones: list[NearbyNetworkZone] = []
# ── Full PDF report (эпик #2259 PR-D) ────────────────────────────────────────
class ReportBuildResponse(BaseModel):
"""Ответ POST /{cad}/report — enqueue сборки полного PDF-отчёта (#2259 PR-D).
status="building" (202) таска поставлена в очередь, PDF собирается в фоне;
status="ready" (200) готовый кэш (тот же analyze+forecast ран) уже на диске,
качать сразу через /report/download.
"""
status: Literal["building", "ready"]
# Даты базовых ранов (ISO) — контекст, по каким данным собирается/собран отчёт.
analyze_run_at: str | None = None
forecast_run_at: str | None = None
# Дата генерации готового PDF (ISO) — только при status="ready" (кэш-хит); на
# building-ветке None (файла ещё нет). Даёт fast-path кнопке дату без GET /status.
report_generated_at: str | None = None
class ReportStatusResponse(BaseModel):
"""Ответ GET /{cad}/report/status — состояние полного PDF-отчёта (#2259 PR-D).
status: "ready" (метадата-ран report-pdf-1.0 есть И файл на диске),
"building" (есть analyze-ран, но готового отчёта нет вероятно собирается / ещё не
запускался), "none" (нет даже analyze-рана отчётировать нечего).
"""
status: Literal["ready", "building", "none"]
# Даты базовых ранов (ISO) — по каким данным собран/будет собран отчёт.
analyze_run_at: str | None = None
forecast_run_at: str | None = None
# Дата генерации готового PDF (ISO) — только при status="ready".
report_generated_at: str | None = None
# ── NSPD Risk Zones schemas (issue #94 TIER 3) ───────────────────────────────
@ -400,6 +604,25 @@ class TopLayoutRow(BaseModel):
is_oversold: bool # True когда raw sum_deals/supply > 100% (несопоставимые окна)
class SupplyOnlyLayoutRow(BaseModel):
"""Строка блока «по предложению» (#2177 шаг 3).
Структура текущего предложения (комнатность × площадь × цена) для комплексов,
которые сматчены на objective project_name по ЯДРУ имени, но у которых НЕТ
сделок (velocity). Источник objective_lots (снапшот предложения). Это НЕ
velocity-ранжир: темпа продаж здесь нет, только структура остатков в продаже.
"""
project_name: str
room_bucket: str # {studio,1,2,3,4+} — вокабуляр velocity-стороны
area_bin: str
lots_count: int
median_price_m2: float | None # NULL если цены не прошли price-sanity
median_area_m2: float | None
# Маркер источника: всегда 'objective_lots' — честная семантика «без темпа продаж».
source: str = "objective_lots"
class LayoutTzMixRow(BaseModel):
"""Строка рекомендации unit-mix для ТЗ."""
@ -425,10 +648,28 @@ class LayoutTzRecommendation(BaseModel):
class LayoutDataQuality(BaseModel):
"""Метаданные качества данных (coverage)."""
"""Метаданные качества данных (coverage).
objects_with_velocity_data: int
objects_total_in_radius: int
Issue #2177: coverage считается в ГРУППАХ-КОМПЛЕКСАХ, а не в сырых obj_id.
domrf_kn_objects дробит один ЖК на несколько obj_id (разные очереди /
дубли snapshot-строк / гео-кучки безымянных корпусов), из-за чего сырой
знаменатель завышал число «конкурентов» (прод-кейс: 12 obj_id = 5 реальных
комплексов «2 из 12» вместо честных «2 из 5»). Поля objects_* теперь
считают комплексы; raw_objects_total старый счёт obj_id для прозрачности.
"""
objects_with_velocity_data: int # число комплексов с velocity-данными
objects_total_in_radius: int # число комплексов (сгруппированных obj_id)
raw_objects_total: int # сырое число obj_id в радиусе (до группировки)
# Issue #2177 шаг 2b (core-attribution): сколько комплексов покрыто ИМЕННО по
# нормализованному ядру имени (project_name-матч), а НЕ через mapping-obj_id —
# прозрачность источника атрибуции velocity. Кейс «7 Ключей»: 2-я очередь в
# радиусе резолвит сделки проекта по ядру, минуя single-obj_id mapping.
groups_matched_by_core: int = 0
# Issue #2177 шаг 3: сколько комплексов попало в блок «по предложению» —
# сматчены по ядру на objective project_name, БЕЗ velocity, но с лотами в
# продаже (objective_lots). Аддитивно, не пересекается с velocity-покрытием.
groups_with_supply_only: int = 0
velocity_coverage_pct: float
confidence: Literal["high", "medium", "low"]
@ -439,6 +680,10 @@ class BestLayoutsResponse(BaseModel):
top_layouts: list[TopLayoutRow]
recommendation_for_tz: LayoutTzRecommendation
data_quality: LayoutDataQuality
# Issue #2177 шаг 3: fallback-блок «по предложению без темпа продаж» —
# структура текущего предложения для сматченных по ядру комплексов БЕЗ velocity.
# Аддитивно; НЕ смешивается с top_layouts (там velocity-ранжир). Пусто по умолчанию.
supply_only_layouts: list[SupplyOnlyLayoutRow] = Field(default_factory=list)
# ── Analyze endpoint market price (#33) ──────────────────────────────────────
@ -623,6 +868,11 @@ class AnalyzeResponse(BaseModel):
score_label: str | None = None
score_max_reference: float | None = None
score_explanation: str | None = None
# #1926: машиночитаемый advisory-флаг — score это POI-density-эвристика
# (0..score_max_reference), а не абсолютный инвест-рейтинг. Optional для
# обратной совместимости (старые моки/202-стаб → null).
score_is_advisory: bool | None = None
score_advisory: dict[str, Any] | None = None
score_breakdown: dict[str, Any] | None = None
score_breakdown_detailed: list[dict[str, Any]] | None = None
score_top_3_positives: list[dict[str, Any]] | None = None
@ -638,6 +888,9 @@ class AnalyzeResponse(BaseModel):
market_data_coverage_pct: float | None = None
pipeline_24mo: dict[str, Any] | None = None
velocity: dict[str, Any] | None = None
# market_trend: {status: ok|offer_only|source_stale|no_deals, ...}. #2178 добавил
# вложенный offer_trend (динамика цен предложения из objective_lots_history) и
# status='offer_only'. Схема loose (dict) — новые ключи проходят без правки модели.
market_trend: dict[str, Any] | None = None
market_price: dict[str, Any] | None = None
@ -660,6 +913,7 @@ class AnalyzeResponse(BaseModel):
parcel_meta: dict[str, Any] | None = None
recent_permits_in_quarter: list[dict[str, Any]] | None = None
permits_summary: dict[str, Any] | None = None
permits_nearby: dict[str, Any] | None = None
zoning: dict[str, Any] | None = None
success_recommendation: dict[str, Any] | None = None
isochrones_available: bool | None = None

View file

@ -504,6 +504,15 @@ def developer_history(
def developer_portfolio(db: Session, developer_id: str) -> list[dict[str, Any]]:
# #2464 cluster E: без дедупа каждый ЖК возвращался ×N раз (одна строка на
# snapshot_date, ретенции нет) — портфель девелопера раздувался в разы (прод-замер
# ~7.6-9.0× per-developer). DISTINCT ON (obj_id) + snapshot_date DESC NULLS LAST —
# latest-snapshot-per-obj_id (тот же паттерн, что cmp_rows/latest_obj CTE в
# recommend_mix ниже в этом файле). dev_id — стабильный идентификатор
# (не меняется между снапшотами одного ЖК) → фильтруется ДО DISTINCT ON, как
# region_cd/district_name в сиблингах. Итоговый ORDER BY ready_dt вынесен во внешний
# SELECT: DISTINCT ON требует, чтобы его собственный ORDER BY начинался с ключа
# дедупа (obj_id, snapshot_date), не с ready_dt.
rows = (
db.execute(
text(
@ -511,8 +520,15 @@ def developer_portfolio(db: Session, developer_id: str) -> list[dict[str, Any]]:
SELECT obj_id, comm_name, addr, region_cd, flat_count,
square_living, ready_dt, obj_class, escrow,
problem_flag, latitude, longitude, is_ekb
FROM domrf_kn_objects
WHERE dev_id = :dev
FROM (
SELECT DISTINCT ON (obj_id)
obj_id, comm_name, addr, region_cd, flat_count,
square_living, ready_dt, obj_class, escrow,
problem_flag, latitude, longitude, is_ekb
FROM domrf_kn_objects
WHERE dev_id = :dev
ORDER BY obj_id, snapshot_date DESC NULLS LAST
) latest
ORDER BY ready_dt DESC NULLS LAST
"""
),
@ -1797,12 +1813,38 @@ def _active_competitors_count(
Возвращает (count, scope_used). Min 1 чтобы не делить на 0."""
def _q(where_extras: str, params: dict[str, Any]) -> int:
# #2464 cluster E: domrf_kn_objects хранит МНОЖЕСТВО snapshot_date на obj_id
# (UNIQUE(obj_id, snapshot_date), ретенции нет) — прод-замер 4090 строк / 482
# distinct obj_id для site_status='Строящиеся' (8.49× инфляция). Наивный
# COUNT(*) считал каждый исторический снапшот отдельно. Fix: DISTINCT ON
# (obj_id) + snapshot_date DESC NULLS LAST даёт latest-snapshot-per-obj_id
# (сиблинг-паттерн: _L3_FUTURE_SQL #1212 в site_finder/supply_layers.py).
#
# ВСЕ волатильные предикаты применяются ПОСЛЕ DISTINCT ON (во внешнем WHERE),
# а не внутри CTE — иначе DISTINCT ON вернул бы «последний снапшот, ПРОШЕДШИЙ
# фильтр», а не истинно последний снапшот объекта. Прод-замер per-obj_id
# variance across snapshots (authoritative, coordinator 2026-07-08):
# dev_id=0, region_cd=0 (СТАБИЛЬНЫ) · district_name=1, obj_class=29,
# site_status=11 (ВОЛАТИЛЬНЫ).
# → В CTE остаётся ТОЛЬКО region_cd (стабилен + ограничивает стоимость дедупа
# как partition-scope). {where_extras} (district_name/obj_class) и
# site_status='Строящиеся' — во внешнем WHERE, на истинно-последней строке.
# Пример бага, который это чинит: ЖК сменил класс Комфорт→Бизнес; при
# фильтре obj_class внутри CTE взяли бы старый Комфорт-снапшот и посчитали
# его активным Комфортом, хотя его актуальный класс уже Бизнес.
n = db.execute(
text(
f"""
SELECT COUNT(*) FROM domrf_kn_objects
WHERE region_cd = :rc
AND site_status = 'Строящиеся'
WITH latest AS (
SELECT DISTINCT ON (obj_id)
obj_id, site_status, district_name,
obj_class, obj_class_fallback
FROM domrf_kn_objects
WHERE region_cd = :rc
ORDER BY obj_id, snapshot_date DESC NULLS LAST
)
SELECT COUNT(*) FROM latest
WHERE site_status = 'Строящиеся'
{where_extras}
"""
),
@ -2233,14 +2275,28 @@ def _competitors_two_dim(
db.execute(
text(
f"""
WITH active AS (
SELECT DISTINCT ON (obj_id) obj_id, latitude, longitude, district_name
WITH latest AS (
-- #2464 cluster E: сначала истинно-последний снапшот на obj_id,
-- ТОЛЬКО по стабильному region_cd (variance=0). Волатильные
-- предикаты (site_status=11, district_name=1, obj_class=29 per
-- прод-замер) НЕ внутри DISTINCT ON, иначе взяли бы «последний
-- снапшот, прошедший фильтр», а не истинно последний (ЖК,
-- сменивший класс/район/статус, засчитался бы по устаревшему
-- снапшоту). Зеркалит _q() в _active_competitors_count выше.
SELECT DISTINCT ON (obj_id)
obj_id, latitude, longitude, district_name,
site_status, obj_class, obj_class_fallback
FROM domrf_kn_objects
WHERE region_cd = :rc
AND site_status = 'Строящиеся'
ORDER BY obj_id, snapshot_date DESC NULLS LAST
),
active AS (
-- Волатильные фильтры на истинно-последней строке.
SELECT obj_id, latitude, longitude, district_name
FROM latest
WHERE site_status = 'Строящиеся'
AND district_name = :dn
{class_filter}
ORDER BY obj_id, snapshot_date DESC NULLS LAST
),
centroid AS (
SELECT ST_SetSRID(ST_GeomFromText(:centroid), 4326)::geography AS pt

View file

@ -19,7 +19,7 @@ from typing import Any
from sqlalchemy.orm import Session
from app.services.analysis_runs.repository import latest_run_for
from app.services.analysis_runs.repository import ANALYZE_SCHEMA_VERSION, latest_run_for
logger = logging.getLogger(__name__)
@ -28,6 +28,10 @@ logger = logging.getLogger(__name__)
# схеме явный и устойчивый.
_FORECAST_SCHEMA_VERSION = "1.0"
# Максимум названий ЗОУИТ-зон в курируемом срезе (§19: ограничиваем объём, чтобы наружу
# не ушёл длинный «хвост» — сводки хватает списка из первых нескольких зон).
_MAX_ZOUIT_NAMES = 15
def get_report_for_chat(
db: Session,
@ -57,3 +61,116 @@ def get_report_for_chat(
return None, None
return result, run.id
def _as_dict(value: Any) -> dict[str, Any]:
"""Секция payload как dict или {} (graceful — как _as_dict в full_report_html)."""
return value if isinstance(value, dict) else {}
def _put(target: dict[str, Any], key: str, value: Any) -> None:
"""Положить ТОЛЬКО скаляр в курируемый срез (§19 data-residency).
Гейт двойной: и по КЛЮЧУ (вызывающий передаёт фиксированный whitelist), и по ТИПУ
значения. Пропускаем лишь str|int|float|bool dict/list/любая вложенная структура
ОТБРАСЫВАЕТСЯ (иначе сырой под-словарь ЕГРН/зонирования с PII мог бы утечь наружу;
redaction-слой ловит regex-PII в строках, но НЕ структуры). Пустую строку опускаем.
UP038: union-форма isinstance (не кортеж) pre-commit пинит старый ruff.
"""
if not isinstance(value, str | int | float | bool):
return
if value == "":
return
target[key] = value
def _zouit_names(overlaps: list[Any]) -> list[str]:
"""Собрать dedup-список названий ЗОУИТ-зон (type_zone|name), обрезанный до лимита.
§19: ТОЛЬКО строки-названия никакой геометрии/reg_numb/coverage сырых пересечений.
Не-строковые значения (dict/list/число) ОТБРАСЫВАЕМ не приводим str()'ом, иначе
вложенная структура утекла бы как её repr. dict.fromkeys держит порядок и dedup
(зеркало _build_zouit в §1-рендере).
"""
# Фильтруем до str ДО dict.fromkeys: не-str (dict/list) не только небезопасны для
# §19, но и unhashable → уронили бы dict.fromkeys TypeError'ом.
raw = (ov.get("type_zone") or ov.get("name") for ov in overlaps if isinstance(ov, dict))
names = list(dict.fromkeys(n for n in raw if isinstance(n, str) and n))
return names[:_MAX_ZOUIT_NAMES]
def get_parcel_context_for_chat(db: Session, cad_num: str) -> dict[str, Any] | None:
"""Курируемый паспорт участка + градрегламент для чата (§19 data-residency).
READ-ONLY: последний analyze-ран (schema_version='analyze-1.0', НЕ §22-форсайт)
ПЛОСКИЙ whitelist-дикт из ФИКСИРОВАННОГО набора публичных градо-полей. Сырой
analyze-blob во внешний LLM ЗАПРЕЩЁН (см. safe_payload.py) поэтому собираем только
скаляры и список строк по явным ключам, НИКОГДА не протаскивая под-словари/геометрию.
Ключи payload те же, что читает §1-рендер отчёта (full_report_html:_build_parcel_facts
/ _build_zoning / _build_zouit): egrn, nspd_zoning, encumbrance, nspd_zouit_overlaps.
None если analyze-рана/результата нет (graceful, как get_report_for_chat) вызывающий
работает как раньше (только §22-отчёт, без паспорта участка).
"""
run = latest_run_for(db, cad_num, schema_version=ANALYZE_SCHEMA_VERSION)
if run is None:
return None
result = run.result
if not isinstance(result, dict):
logger.warning(
"chat: analyze run %s for cad=%s has non-dict result (%s) — no parcel context",
run.id,
cad_num,
type(result).__name__,
)
return None
egrn = _as_dict(result.get("egrn"))
nspd_zoning = _as_dict(result.get("nspd_zoning"))
encumbrance = _as_dict(result.get("encumbrance"))
overlaps = [ov for ov in (result.get("nspd_zouit_overlaps") or []) if isinstance(ov, dict)]
context: dict[str, Any] = {}
# ── Кадастровые факты (ЕГРН) — скаляры, без геометрии ────────────────────────
_put(context, "address", egrn.get("address"))
_put(context, "area_m2", egrn.get("area_m2"))
_put(context, "land_category", egrn.get("land_category"))
_put(context, "permitted_use_text", egrn.get("permitted_use_text"))
_put(context, "cadastral_value_rub", egrn.get("cadastral_value_rub"))
_put(context, "parcel_status", egrn.get("parcel_status"))
_put(context, "ownership_type", egrn.get("ownership_type"))
# ── Территориальная зона ПЗЗ + лимиты регламента (те же ключи, что §1-рендер) ─
_put(
context,
"zone_code",
nspd_zoning.get("zone_code") or nspd_zoning.get("regulation_zone_index"),
)
_put(context, "zone_name", nspd_zoning.get("zone_name"))
_put(context, "max_far", nspd_zoning.get("max_far"))
_put(context, "max_floors", nspd_zoning.get("max_floors"))
_put(context, "max_height_m", nspd_zoning.get("max_height_m"))
_put(context, "max_building_pct", nspd_zoning.get("max_building_pct"))
_put(context, "min_parcel_area_m2", nspd_zoning.get("min_parcel_area_m2"))
_put(context, "regulation_source", nspd_zoning.get("regulation_source"))
# ── ЗОУИТ-обременения: сводка + список названий (dedup, capped) ───────────────
has_zouit = encumbrance.get("has_zouit")
zouit_count = encumbrance.get("zouit_count")
zouit_names = _zouit_names(overlaps)
# Противоречие «encumbrance=нет, но пересечения ЕСТЬ» — доверяем фактам НСПД (тот же
# приём, что _build_zouit): свежий геослой перекрывает устаревшую сводку.
if not has_zouit and zouit_names:
has_zouit = True
if zouit_count in (None, 0):
zouit_count = len(overlaps)
_put(context, "has_zouit", has_zouit)
_put(context, "zouit_count", zouit_count)
if zouit_names:
context["zouit_zone_names"] = zouit_names
return context or None

View file

@ -14,9 +14,14 @@ tool'ами (см. tools.py). ЭТО ЕДИНСТВЕННОЕ место, чер
РАЗРЕШЕНО в ``section_data`` / ``fields``:
срезы секций отчёта из tools.py (exec_summary / product_tz / future_market /
scoring / confidence / scenarios) это посчитанные advisory-агрегаты.
``parcel_context`` КУРИРУЕМЫЙ срез analyze-рана (retrieval.get_parcel_context_for_chat):
публичные градоданные участка (адрес, площадь, категория, ВРИ, тер.зона ПЗЗ + лимиты
застройки, ЗОУИТ-сводка + названия зон). Строится по фиксированному whitelist'у из
скаляров и списка строк БЕЗ геометрии/координат, без сырых под-словарей, без PII.
ЗАПРЕЩЕНО (НИКОГДА не должно сюда дойти):
сырой ``analyze``-blob / сырые строки БД (rosreestr_deals, parcels, );
сырой ``analyze``-blob целиком / сырые строки БД (rosreestr_deals, parcels, );
геометрия/координаты участка или ЗОУИТ-пересечений;
свободный insight-текст / лиды / любой контент с PII;
любой источник, помеченный confidential.

View file

@ -3,10 +3,11 @@
LLM в tool-loop'е (см. orchestrator.py) просит секции отчёта через function-calling.
Здесь две вещи и НИЧЕГО больше:
1. OpenAI tool-спеки (JSON-schema) пяти read-only секционных tool'ов
1. OpenAI tool-спеки (JSON-schema) шести read-only секционных tool'ов
(`get_exec_summary` / `get_product_recommendation` / `get_forecast` /
`get_risks` / `get_scenarios`). Без параметров каждый отдаёт фиксированную
секцию(и) отчёта (модель не управляет вычислениями, только запрашивает данные).
`get_risks` / `get_scenarios` / `get_parcel_info`). Без параметров каждый отдаёт
фиксированную секцию(и) отчёта (модель не управляет вычислениями, только запрашивает
данные).
2. ЧИСТЫЕ executors: режут УЖЕ-ЗАГРУЖЕННЫЙ in-memory `report_dict`
(`SiteFinderReport.as_dict()`, 8 секций). НИКАКОЙ БД, НИКАКОГО пере-расчёта,
НИКАКОЙ движковой математики только срез готового dict'а.
@ -73,6 +74,16 @@ def get_scenarios(report: dict[str, Any]) -> dict[str, Any]:
return _section(report, "scenarios")
def get_parcel_info(report: dict[str, Any]) -> dict[str, Any]:
"""§1 parcel_context — паспорт участка + градрегламент ПЗЗ + ЗОУИТ.
Курируемый срез analyze-рана (тер.зона ПЗЗ, ВРИ, лимиты застройки, ЗОУИТ), влитый
эндпоинтом под ключ "parcel_context" (retrieval.get_parcel_context_for_chat). Секции
в отчёте нет (analyze-рана не было) маркер «недоступно», как у остальных tool'ов.
"""
return _section(report, "parcel_context")
# ── Реестр имя→executor + имя→секции отчёта (для provenance grounded_in.sections) ─
# ЕДИНЫЙ источник истины: и спеки, и orchestrator берут имена/маппинг отсюда.
@ -84,6 +95,7 @@ _TOOLS: dict[str, tuple[Callable[[dict[str, Any]], dict[str, Any]], tuple[str, .
"get_forecast": (get_forecast, ("future_market",)),
"get_risks": (get_risks, ("scoring", "confidence")),
"get_scenarios": (get_scenarios, ("scenarios",)),
"get_parcel_info": (get_parcel_info, ("parcel_context",)),
}
# RU-описания tool'ов для модели (что внутри секции — чтобы LLM выбирал верный tool).
@ -103,8 +115,11 @@ _TOOL_DESCRIPTIONS: dict[str, str] = {
"Риски участка: специальные индексы §25 (включая каннибализацию портфеля) "
"и уровень/факторы уверенности отчёта."
),
"get_scenarios": (
"Сценарии развития: разброс по консервативному / базовому / агрессивному."
"get_scenarios": ("Сценарии развития: разброс по консервативному / базовому / агрессивному."),
"get_parcel_info": (
"Паспорт участка и градостроительный регламент: адрес, площадь, категория земель, "
"ВРИ, территориальная зона ПЗЗ (код и название), лимиты застройки, "
"ЗОУИТ-обременения."
),
}
@ -128,7 +143,7 @@ def _spec(name: str) -> dict[str, Any]:
def tool_specs() -> list[dict[str, Any]]:
"""Все 5 секционных tool-спек (JSON-schema) для передачи в ``complete(tools=...)``."""
"""Все 6 секционных tool-спек (JSON-schema) для передачи в ``complete(tools=...)``."""
return [_spec(name) for name in _TOOLS]
@ -161,6 +176,7 @@ __all__ = [
"execute_tool",
"get_exec_summary",
"get_forecast",
"get_parcel_info",
"get_product_recommendation",
"get_risks",
"get_scenarios",

View file

@ -0,0 +1,195 @@
"""Минимальный синхронный DaData-клиент — /clean/address → (lat, lon).
Нужен ТОЛЬКО для geo-pass в objective_backfill (#2177): адрес objective-проекта
координаты для haversine-сверки с domrf-точкой. Не тащим tradein-модуль
(`tradein-mvp/.../dadata.py`) импортом это другое приложение с другим settings.
Клиент СИНХРОННЫЙ (httpx.Client): geo-pass гоняется в CLI / sync-Session, без
event-loop. Авторизация /clean/address требует ОБА: token + secret (в отличие от
/suggest, которому хватает токена). Оба берутся из settings (env DADATA_API_TOKEN /
DADATA_API_SECRET).
Возвращает None (не raise) на любой сбой нет credentials, пустой/короткий адрес,
сетевая/HTTP-ошибка, 429 quota, невалидный JSON, адрес не распознан. Вызывающая
сторона (geo-pass) трактует None как «нет геокода» reject-кандидат, проход не
падает.
Прод-инцидент 2026-07-03: у токена может быть ОТКЛЮЧЕНА фича CLEAN («Feature
'CLEAN' disabled for token» стандартизация платная, баланс исчерпан), при этом
БЕСПЛАТНЫЙ suggestions API (10k/день, только Token без X-Secret) работает и тоже
отдаёт geo_lat/geo_lon. Поэтому: cleaner при 401/403 однократный фолбэк на
suggest/address (первая саджеста). Точность саджесты чуть ниже клинера для
наших целей (точка на карте + bbox-гейт) достаточно.
Docs: https://dadata.ru/api/clean/address/ · https://dadata.ru/api/suggest/address/
"""
from __future__ import annotations
import logging
from typing import Any
import httpx
from app.core.config import settings
logger = logging.getLogger(__name__)
DADATA_CLEAN_URL = "https://cleaner.dadata.ru/api/v1/clean/address"
DADATA_SUGGEST_URL = "https://suggestions.dadata.ru/suggestions/api/4_1/rs/suggest/address"
def _credentials() -> tuple[str, str] | None:
"""(token, secret) если ОБА заданы непусто, иначе None (graceful disable)."""
token = (settings.dadata_api_token or "").strip()
secret = (settings.dadata_api_secret or "").strip()
if not token or not secret:
return None
return token, secret
def _coerce_float(value: Any) -> float | None:
"""DaData отдаёт geo_lat/geo_lon строкой — coerce в float безопасно."""
if value is None:
return None
try:
return float(value)
except (TypeError, ValueError):
return None
def clean_address(address: str) -> tuple[float, float] | None:
"""Геокодировать один адрес через DaData /clean/address → (lat, lon).
Args:
address: свободный текст адреса (objective_lots.address).
Returns:
(geo_lat, geo_lon) как float-пара, либо None если:
- credentials не заданы в ENV (graceful disable);
- адрес пустой / короче 3 символов;
- сетевая / HTTP-ошибка (timeout, 5xx, 429 quota, 401/403 auth, 4xx);
- невалидный JSON / пустой массив / не dict;
- DaData не вернул geo_lat/geo_lon (адрес не геокодирован).
"""
if not address or len(address.strip()) < 3:
return None
creds = _credentials()
if creds is None:
logger.debug("dadata_client: credentials не заданы — skip геокод")
return None
token, secret = creds
headers = {
"Authorization": f"Token {token}",
"X-Secret": secret,
"Content-Type": "application/json",
"Accept": "application/json",
}
body = [address.strip()]
try:
with httpx.Client(timeout=settings.dadata_timeout_s) as client:
response = client.post(DADATA_CLEAN_URL, headers=headers, json=body)
except (httpx.TimeoutException, httpx.NetworkError) as exc:
logger.warning("dadata_client: network error для %r: %s", address[:60], exc)
return None
except Exception as exc:
# defensive: любой прочий httpx/клиент-сбой → None, geo-pass reject'ит,
# НЕ роняем весь проход. Логируем на warning (не глотаем молча).
logger.warning("dadata_client: unexpected client error для %r: %s", address[:60], exc)
return None
status = response.status_code
if status == 429:
logger.warning("dadata_client: HTTP 429 — quota exceeded")
return None
if status in (401, 403):
# Фича CLEAN у токена может быть отключена (баланс стандартизации) —
# фолбэк на бесплатный suggestions (тот же token, X-Secret не нужен).
logger.warning("dadata_client: cleaner HTTP %d — фолбэк на suggest/address", status)
return _suggest_geocode(address.strip(), token)
if status >= 400:
logger.warning("dadata_client: HTTP %d для %r", status, address[:60])
return None
try:
payload = response.json()
except ValueError as exc:
logger.warning("dadata_client: невалидный JSON: %s", exc)
return None
if not isinstance(payload, list) or not payload:
logger.info("dadata_client: пустой ответ для %r", address[:60])
return None
item = payload[0]
if not isinstance(item, dict):
logger.warning("dadata_client: первый элемент не dict (%s)", type(item).__name__)
return None
lat = _coerce_float(item.get("geo_lat"))
lon = _coerce_float(item.get("geo_lon"))
if lat is None or lon is None:
logger.info(
"dadata_client: нет координат (qc_geo=%s) для %r",
item.get("qc_geo"),
address[:60],
)
return None
return (lat, lon)
def _suggest_geocode(address: str, token: str) -> tuple[float, float] | None:
"""Фолбэк-геокод через БЕСПЛАТНЫЙ suggest/address (первая саджеста).
Возвращает (lat, lon) из suggestions[0].data.geo_lat/geo_lon либо None.
Та же graceful-семантика, что у clean_address.
"""
headers = {
"Authorization": f"Token {token}",
"Content-Type": "application/json",
"Accept": "application/json",
}
try:
with httpx.Client(timeout=settings.dadata_timeout_s) as client:
response = client.post(
DADATA_SUGGEST_URL,
headers=headers,
json={"query": address, "count": 1},
)
except (httpx.TimeoutException, httpx.NetworkError) as exc:
logger.warning("dadata_client: suggest network error для %r: %s", address[:60], exc)
return None
except Exception as exc:
logger.warning("dadata_client: suggest unexpected error для %r: %s", address[:60], exc)
return None
if response.status_code != 200:
logger.warning("dadata_client: suggest HTTP %d для %r", response.status_code, address[:60])
return None
try:
payload = response.json()
except ValueError as exc:
logger.warning("dadata_client: suggest невалидный JSON: %s", exc)
return None
suggestions = payload.get("suggestions") if isinstance(payload, dict) else None
if not suggestions or not isinstance(suggestions[0], dict):
logger.info("dadata_client: suggest пусто для %r", address[:60])
return None
data = suggestions[0].get("data") or {}
lat = _coerce_float(data.get("geo_lat"))
lon = _coerce_float(data.get("geo_lon"))
if lat is None or lon is None:
logger.info(
"dadata_client: suggest без координат (qc_geo=%s) для %r",
data.get("qc_geo"),
address[:60],
)
return None
return (lat, lon)

View file

@ -0,0 +1,220 @@
"""Review-инструмент авто-маппингов objective_complex_mapping (feat/mapping-review-page).
Авто-проходы (auto_core_dev_v5 / auto_core_geo_v6 / fuzzy_* / ) пишут строки
с is_reviewed=false. Здесь чтение этих строк С ОБОГАЩЕНИЕМ для глаз ревьюера,
подтверждение (approve is_reviewed=true) и отклонение (reject DELETE строки,
чтобы отклонённый маппинг не травил mv_layout_velocity §4.2).
Обогащение:
domrf comm_name / dev_name LATEST snapshot по domrf_obj_id
(snapshot_date DESC как в objective_backfill._DOMRF_UNMAPPED_SQL).
objective developers агрегат objective_lots.developer, SCOPED по
(project_name = objective_complex_name). Урок 3.8с: агрегат по project_name,
НЕ глобальный иначе список застройщиков одного проекта протекает в чужой.
Чтения parametrized `text(...)` + CAST(:x AS type) (backend.md psycopg3).
DELETE (reject) только по PK с RETURNING для полного лога удаляемой строки.
"""
from __future__ import annotations
import logging
from datetime import date
from typing import Any
from sqlalchemy import text
from sqlalchemy.orm import Session
logger = logging.getLogger(__name__)
# ── SQL: строки mapping + обогащение domrf latest snapshot + objective devs ────
# LEFT JOIN LATERAL к domrf_kn_objects: DISTINCT-ON-эквивалент через ORDER BY
# snapshot_date DESC LIMIT 1 (latest snapshot по obj_id). objective devs —
# коррелированный агрегат по project_name (scoped, урок 3.8с).
# Сорт: is_reviewed asc (unreviewed сверху), match_score asc NULLS FIRST
# (самые сомнительные / без скора сверху), id для стабильности.
_LIST_SQL = text("""
SELECT
m.id,
m.objective_complex_name,
m.objective_project_id,
m.objective_group,
m.domrf_obj_id,
m.match_method,
m.match_score,
m.is_reviewed,
m.note,
m.created_at,
d.comm_name AS domrf_comm_name,
d.dev_name AS domrf_dev_name,
odev.developers AS objective_developers
FROM objective_complex_mapping m
LEFT JOIN LATERAL (
SELECT o.comm_name, o.dev_name
FROM domrf_kn_objects o
WHERE o.obj_id = m.domrf_obj_id
ORDER BY o.snapshot_date DESC NULLS LAST
LIMIT 1
) d ON m.domrf_obj_id IS NOT NULL
LEFT JOIN LATERAL (
SELECT ARRAY_REMOVE(ARRAY_AGG(DISTINCT ol.developer), NULL) AS developers
FROM objective_lots ol
WHERE ol.project_name = m.objective_complex_name
AND ol.developer IS NOT NULL
) odev ON TRUE
WHERE (CAST(:only_unreviewed AS boolean) = FALSE OR m.is_reviewed = FALSE)
ORDER BY m.is_reviewed ASC, m.match_score ASC NULLS FIRST, m.id ASC
LIMIT CAST(:limit AS int) OFFSET CAST(:offset AS int)
""")
_COUNT_SQL = text("""
SELECT COUNT(*) AS total
FROM objective_complex_mapping m
WHERE (CAST(:only_unreviewed AS boolean) = FALSE OR m.is_reviewed = FALSE)
""")
# ── SQL: approve — is_reviewed=true + append note. RETURNING для 404-детекта ───
# NULLIF/concat: пустой/NULL note → просто «approved <date>»; иначе append через
# перевод строки. Идемпотентно относительно is_reviewed (повторный approve OK).
_APPROVE_SQL = text("""
UPDATE objective_complex_mapping
SET is_reviewed = TRUE,
note = CASE
WHEN note IS NULL OR note = '' THEN CAST(:approve_note AS text)
ELSE note || E'\\n' || CAST(:approve_note AS text)
END,
updated_at = NOW()
WHERE id = CAST(:id AS bigint)
RETURNING id, objective_complex_name, domrf_obj_id,
match_method, match_score, is_reviewed, note
""")
# ── SQL: reject — DELETE по PK, RETURNING всей строки для лога (травит §4.2) ───
_DELETE_SQL = text("""
DELETE FROM objective_complex_mapping
WHERE id = CAST(:id AS bigint)
RETURNING id, objective_complex_name, objective_project_id, objective_group,
domrf_obj_id, match_method, match_score, is_reviewed, note,
created_at
""")
def _serialize_row(row: Any) -> dict[str, Any]:
"""Сериализовать строку list-запроса в loose dict для API (паттерн admin_etl)."""
score = row.match_score
created = row.created_at
return {
"id": row.id,
"objective_complex_name": row.objective_complex_name,
"objective_project_id": row.objective_project_id,
"objective_group": row.objective_group,
"domrf_obj_id": row.domrf_obj_id,
"match_method": row.match_method,
"match_score": float(score) if score is not None else None,
"is_reviewed": bool(row.is_reviewed),
"note": row.note,
"created_at": created.isoformat() if created else None,
# Обогащение для глаз ревьюера:
"domrf_comm_name": row.domrf_comm_name,
"domrf_dev_name": row.domrf_dev_name,
"objective_developers": list(row.objective_developers or []),
}
def list_mapping_review(
db: Session,
*,
only_unreviewed: bool = True,
limit: int = 100,
offset: int = 0,
) -> dict[str, Any]:
"""Список строк objective_complex_mapping + обогащение + total count.
Сорт: is_reviewed asc, match_score asc NULLS FIRST (сомнительные сверху), id.
Args:
db: SQLAlchemy sync Session.
only_unreviewed: True (default) только is_reviewed=false.
limit: страница (строк).
offset: смещение страницы.
Returns:
dict: rows (list обогащённых строк), total (int, до limit/offset),
limit, offset, only_unreviewed.
"""
rows = db.execute(
_LIST_SQL,
{"only_unreviewed": only_unreviewed, "limit": limit, "offset": offset},
).all()
total = db.execute(_COUNT_SQL, {"only_unreviewed": only_unreviewed}).scalar_one()
return {
"rows": [_serialize_row(r) for r in rows],
"total": int(total),
"limit": limit,
"offset": offset,
"only_unreviewed": only_unreviewed,
}
def approve_mapping(db: Session, mapping_id: int) -> dict[str, Any] | None:
"""Подтвердить маппинг: is_reviewed=true + append «approved <date>» в note.
Returns:
dict обновлённой строки, либо None если id не найден ( 404 в роутере).
"""
approve_note = f"approved {date.today().isoformat()}"
row = db.execute(_APPROVE_SQL, {"id": mapping_id, "approve_note": approve_note}).first()
if row is None:
db.rollback()
return None
db.commit()
logger.info(
"mapping-review approve id=%s name=%r method=%s → is_reviewed=true",
row.id,
row.objective_complex_name,
row.match_method,
)
score = row.match_score
return {
"id": row.id,
"objective_complex_name": row.objective_complex_name,
"domrf_obj_id": row.domrf_obj_id,
"match_method": row.match_method,
"match_score": float(score) if score is not None else None,
"is_reviewed": bool(row.is_reviewed),
"note": row.note,
}
def reject_mapping(db: Session, mapping_id: int) -> dict[str, Any] | None:
"""Отклонить маппинг: DELETE строки (травит mv_layout_velocity §4.2 иначе).
Перед удалением полная строка логируется через logger.info (в note писать
некуда строки не будет). DELETE только по PK с RETURNING.
Returns:
dict удалённой строки (для эха в ответе), либо None если id не найден.
"""
row = db.execute(_DELETE_SQL, {"id": mapping_id}).first()
if row is None:
db.rollback()
return None
created = row.created_at
score = row.match_score
deleted: dict[str, Any] = {
"id": row.id,
"objective_complex_name": row.objective_complex_name,
"objective_project_id": row.objective_project_id,
"objective_group": row.objective_group,
"domrf_obj_id": row.domrf_obj_id,
"match_method": row.match_method,
"match_score": float(score) if score is not None else None,
"is_reviewed": bool(row.is_reviewed),
"note": row.note,
"created_at": created.isoformat() if created else None,
}
logger.info("mapping-review reject DELETE id=%s row=%s", row.id, deleted)
db.commit()
return deleted

View file

@ -1,9 +1,12 @@
"""Cross-load ETL: tradein.houses → gendesign.newbuilding_listings (#976 950-E5).
Читает houses + house_sources из tradein-БД (gendesign_reader role),
Читает market.v_houses + market.v_house_sources из tradein-БД (gendesign_reader role),
UPSERT-ит в gendesign newbuilding_listings по (source, ext_house_id).
Соединение к tradein через psycopg v3 (прямой коннект, НЕ SQLAlchemy).
Источник contract-view market.v_* (#2138), а не сырые public.*-таблицы: явный
column-контракт, устойчивый к scraper-driven ALTER TABLE на стороне tradein.
Запускается:
- ночью Celery beat (03:30 МСК, tasks/etl_newbuilding_crossload.py)
- вручную через POST /api/v1/admin/scrape/newbuilding-crossload
@ -54,8 +57,10 @@ SELECT
h.last_scraped_at,
-- yandex_jk_id из house_sources (LEFT JOIN может отсутствовать)
hs_y.ext_id AS yandex_jk_id
FROM houses h
LEFT JOIN house_sources hs_y
-- Читаем через contract-view market.v_* (#2138 step 2/3), НЕ сырые public.* —
-- стабильный явный column-контракт, не ломается на scraper-driven ALTER TABLE.
FROM market.v_houses h
LEFT JOIN market.v_house_sources hs_y
ON hs_y.house_id = h.id
AND hs_y.ext_source = 'yandex'
WHERE h.lat IS NOT NULL

View file

@ -12,20 +12,40 @@ Schema facts (confirmed via pg MCP):
колонки match_method + match_score + is_reviewed поддерживают audit trail.
match_method history:
'fuzzy' legacy Anton SQLite import (avg score 0.98, 127 rows)
'fuzzy_trgm' pg_trgm backfill, auto-accept threshold=0.85 (первый запуск)
'fuzzy_v2' pg_trgm backfill, pruned threshold=0.80 (второй запуск, #44)
'manual' ручная корректура
'fuzzy' legacy Anton SQLite import (avg score 0.98, 127 rows)
'fuzzy_trgm' pg_trgm backfill, auto-accept threshold=0.85 (первый запуск)
'fuzzy_v2' pg_trgm backfill, pruned threshold=0.80 (второй запуск, #44)
'auto_core_dev_v5' core-name normalize (#2198 normalize_complex_name) обеих сторон +
dev-confirm. trgm по СЫРЫМ именам давал ~300 маппингов, 1253 obj_id
остались (низкая similarity: «Жилой комплекс "Малевич" (ЖК "Малевич")»
vs «Малевич»). Ядро-нормализация схлопывает обёртки exact-core match;
dev-подтверждение (пересечение токенов / подстрока) отсеивает
одноимённые чужие проекты (Объектив = только ЕКБ). #2177 шаг 2.
'auto_core_geo_v6' гео-разрешение остатка core-pass (tier_b / ambiguous) через
DaData-геокод. Адрес objective-проекта (mode objective_lots.
address per project) clean_address (lat, lon); haversine
до domrf-точки (domrf_kn_objects.latitude/longitude, latest
snapshot). tier_b: дистанция порога confirm (гео вместо
dev). ambiguous: геокодим ВСЕХ кандидатов ядра, confirm только
если РОВНО ОДИН в радиусе. score 0.80, is_reviewed=false. #2177.
'manual' ручная корректура
"""
from __future__ import annotations
import argparse
import logging
from dataclasses import dataclass
import math
import re
import sys
from dataclasses import dataclass, field
from sqlalchemy import text
from sqlalchemy.orm import Session
from app.services.dadata_client import clean_address
from app.services.site_finder.best_layouts import normalize_complex_name
logger = logging.getLogger(__name__)
# Порог для auto-insert (высокая уверенность)
@ -234,3 +254,919 @@ def trigger_mv_refresh(db: Session) -> int:
from app.services.site_finder.layout_velocity_refresh import refresh_layout_velocity
return refresh_layout_velocity(db, concurrently=True)
# ════════════════════════════════════════════════════════════════════════════
# Core-name pass (#2177 шаг 2) — auto_core_dev_v5
# ════════════════════════════════════════════════════════════════════════════
# objective_group у всех существующих строк mapping = 'Екатеринбург'
# (Объектив покрывает только ЕКБ). Новые строки пишем с тем же значением.
OBJECTIVE_GROUP = "Екатеринбург"
CORE_MATCH_METHOD = "auto_core_dev_v5"
CORE_MATCH_SCORE = 0.90
CORE_MATCH_NOTE = "#2177 step2: exact-core + dev-confirm"
# Префиксы/обёртки орг-формы застройщика, срезаемые перед сравнением.
# Порядок применения не важен (все режутся из целого норм-имени).
_DEV_JUNK_RE = re.compile(
r"""
\b(?:
ооо | оао | пао | зао | ао | ип
| сз # СЗ = специализированный застройщик (аббрев.)
| специализированн\w* # «специализированный застройщик»
| застройщик\w*
| девелопмент\w*
| групп\w* # групп / группа
| компани\w*
| инвест\w* # инвест / инвестиции (частый шум)
| строительн\w*
)\b
""",
re.VERBOSE,
)
def _norm_dev(raw: str | None) -> str:
"""Нормализовать имя застройщика до сравниваемого ядра.
lower срез кавычек/скобок срез орг-формы (ООО/АО/ПАО/ЗАО/«СЗ»/
«специализированный застройщик»/«девелопмент»/«групп(а)») схлопывание
пробелов. Пустой / None ''.
Namespaced на _NAME_-уровне: возвращаем очищенную строку, дальнейшая
классификация (пересечение токенов / подстрока) в find_core_matches.
"""
if not raw:
return ""
name = raw.strip().lower()
# убрать кавычки-ёлочки/лапки/скобки — оставить только содержательные символы
name = re.sub(r"[«»\"'`()\[\]]", " ", name)
name = _DEV_JUNK_RE.sub(" ", name)
# мусорные разделители → пробел (точки в «Астон.» и т.п. не трогаем на dev-стороне:
# dev-имена обычно без них, но на всякий нормализуем распространённые)
name = re.sub(r"[.,;/\\]", " ", name)
name = re.sub(r"\s+", " ", name).strip()
return name
def _dev_confirms(domrf_dev: str, objective_devs: set[str]) -> bool:
"""True если domrf-застройщик подтверждает связь с objective-проектом.
У одного objective project_name бывает НЕСКОЛЬКО developers (SPV / очереди)
подтверждение по ЛЮБОМУ из них. Критерий (любой):
пересечение токенов норм-имён (общий содержательный токен), ИЛИ
одно норм-имя подстрока другого (в любую сторону), НО только когда
короткая сторона 4 символов: «дом»/«юг»/«ак» как подстрока чужого
длинного имени («Домострой», «Югстройинвест») ложное подтверждение,
а именно dev-confirm последняя защита от generic-core коллизий.
Пустые имена не подтверждают (нет сигнала).
"""
d = _norm_dev(domrf_dev)
if not d:
return False
d_tokens = {t for t in d.split() if len(t) >= 3}
for raw_odev in objective_devs:
o = _norm_dev(raw_odev)
if not o:
continue
if min(len(d), len(o)) >= 4 and (d in o or o in d):
return True
o_tokens = {t for t in o.split() if len(t) >= 3}
if d_tokens & o_tokens:
return True
return False
@dataclass
class CoreMatch:
"""Один core-match domrf-объекта к objective-проекту."""
domrf_obj_id: int
domrf_comm_name: str
domrf_dev_name: str | None
core: str
objective_project_name: str
objective_project_id: int | None
objective_developers: list[str]
dev_confirmed: bool
@dataclass
class CoreMatchReport:
"""Итог классификации core-pass.
tier_a ровно 1 objective-кандидат И dev-подтверждён пишем в БД.
tier_b ровно 1 кандидат, dev НЕ подтверждён кандидат на DaData-гео
проход, в БД НЕ пишем (только отчёт).
ambiguous >1 objective-кандидатов по core в отчёт, разрешение вручную/гео.
skipped_taken objective_complex_name уже занят в mapping (UNIQUE-констрейнт;
его domrf-группа уже покрыта дубли не нужны).
"""
tier_a: list[CoreMatch] = field(default_factory=list)
tier_b: list[CoreMatch] = field(default_factory=list)
ambiguous: list[CoreMatch] = field(default_factory=list)
skipped_taken: list[CoreMatch] = field(default_factory=list)
def counts(self) -> dict[str, int]:
return {
"tier_a": len(self.tier_a),
"tier_b": len(self.tier_b),
"ambiguous": len(self.ambiguous),
"skipped_taken": len(self.skipped_taken),
}
# ── SQL: несопоставленные domrf ЕКБ-объекты, latest snapshot per obj_id ───────
# ЛОВУШКА is_ekb (прод-проверка 2026-07-03): флаг проставляется per-snapshot и на
# свежих снапшотах ПУСТ (все latest-строки is_ekb=false — см. также комментарий
# в 155_objective_mapping_trgm_backfill.sql). `WHERE is_ekb = true` внутри
# DISTINCT ON отдал бы устаревшие имена со старых снапшотов (или 0 строк после
# DISTINCT). Поэтому: ЕКБ-принадлежность = «obj_id имел is_ekb=true хоть на одном
# снапшоте» (1285 живых объектов), а строку берём с LATEST снапшота без флага.
_DOMRF_UNMAPPED_SQL = text("""
SELECT DISTINCT ON (o.obj_id)
o.obj_id, o.comm_name, o.dev_name
FROM domrf_kn_objects o
WHERE o.comm_name IS NOT NULL
AND o.obj_id IN (
SELECT DISTINCT e.obj_id FROM domrf_kn_objects e WHERE e.is_ekb = true
)
AND NOT EXISTS (
SELECT 1 FROM objective_complex_mapping cm
WHERE cm.domrf_obj_id = o.obj_id
)
ORDER BY o.obj_id, o.snapshot_date DESC NULLS LAST
""")
# ── SQL: objective-проекты — distinct project_name + агрегаты developer/id ────
# objective_project_id: в objective_lots есть колонка objective_project_id
# (68_schema_objective.sql). Берём MAX (стабилен per project_name; NULL если нет).
_OBJECTIVE_PROJECTS_SQL = text("""
SELECT
ol.project_name,
ARRAY_REMOVE(ARRAY_AGG(DISTINCT ol.developer), NULL) AS developers,
MAX(ol.objective_project_id) AS objective_project_id
FROM objective_lots ol
WHERE ol.project_name IS NOT NULL
AND ol.project_name <> ''
GROUP BY ol.project_name
""")
# ── SQL: уже занятые objective_complex_name в текущей группе ──────────────────
_TAKEN_NAMES_SQL = text("""
SELECT objective_complex_name
FROM objective_complex_mapping
WHERE objective_group = CAST(:group AS text)
""")
def find_core_matches(db: Session) -> CoreMatchReport:
"""Сопоставить несопоставленные domrf ЕКБ-объекты с objective-проектами по ядру.
Обе стороны нормализуются `normalize_complex_name` (#2198): обёртки вроде
«Жилой комплекс "Малевич" (ЖК "Малевич")» схлопываются до ядра «малевич»,
что даёт exact-core match там, где trgm по сырым именам не дотягивал.
Классификация (см. CoreMatchReport):
tier_a 1 objective-кандидат + dev-подтверждён (пишем);
tier_b 1 кандидат, dev НЕ подтверждён (DaData-гео проход, не пишем);
ambiguous >1 кандидата (ручное/гео разрешение);
skipped objective_complex_name уже в mapping (UNIQUE-констрейнт).
Реального обращения к БД в тестах нет db.execute мокается по паттерну файла.
"""
report = CoreMatchReport()
# objective-сторона: core → [(project_name, {devs}, project_id), ...]
objective_by_core: dict[str, list[tuple[str, set[str], int | None]]] = {}
for row in db.execute(_OBJECTIVE_PROJECTS_SQL).all():
project_name = str(row[0])
developers = {str(d) for d in (row[1] or []) if d}
project_id = int(row[2]) if row[2] is not None else None
core = normalize_complex_name(project_name)
if not core:
continue
objective_by_core.setdefault(core, []).append((project_name, developers, project_id))
# уже занятые objective_complex_name (skip — их группа уже покрыта)
taken_names: set[str] = {
str(r[0]) for r in db.execute(_TAKEN_NAMES_SQL, {"group": OBJECTIVE_GROUP}).all()
}
# domrf-сторона: несопоставленные ЕКБ, latest snapshot per obj_id
for row in db.execute(_DOMRF_UNMAPPED_SQL).all():
obj_id = int(row[0])
comm_name = str(row[1])
dev_name = str(row[2]) if row[2] is not None else None
core = normalize_complex_name(comm_name)
if not core:
continue
candidates = objective_by_core.get(core)
if not candidates:
continue # нет objective-проекта с таким ядром — не наш случай
if len(candidates) > 1:
# >1 objective project_name делят одно ядро → ambiguous (не выбираем вслепую)
proj_name, devs, proj_id = candidates[0]
report.ambiguous.append(
_build_core_match(obj_id, comm_name, dev_name, core, proj_name, proj_id, devs)
)
continue
proj_name, devs, proj_id = candidates[0]
if proj_name in taken_names:
report.skipped_taken.append(
_build_core_match(obj_id, comm_name, dev_name, core, proj_name, proj_id, devs)
)
continue
match = _build_core_match(obj_id, comm_name, dev_name, core, proj_name, proj_id, devs)
if match.dev_confirmed:
report.tier_a.append(match)
else:
report.tier_b.append(match)
logger.info("find_core_matches: %s", report.counts())
return report
def _build_core_match(
obj_id: int,
comm_name: str,
dev_name: str | None,
core: str,
proj_name: str,
proj_id: int | None,
devs: set[str],
) -> CoreMatch:
"""Сконструировать CoreMatch + посчитать dev_confirmed."""
return CoreMatch(
domrf_obj_id=obj_id,
domrf_comm_name=comm_name,
domrf_dev_name=dev_name,
core=core,
objective_project_name=proj_name,
objective_project_id=proj_id,
objective_developers=sorted(devs),
dev_confirmed=_dev_confirms(dev_name or "", devs),
)
def apply_core_matches(
db: Session,
report: CoreMatchReport,
*,
dry_run: bool = True,
) -> dict[str, int]:
"""Вставить ТОЛЬКО tier_a в objective_complex_mapping (auto_core_dev_v5).
tier_b / ambiguous / skipped НЕ пишутся (см. CoreMatchReport). Каждая вставка
в per-row SAVEPOINT (backend.md) с ON CONFLICT (objective_complex_name,
objective_group) DO NOTHING гонка/дубль не роняет весь проход.
Args:
db: SQLAlchemy sync Session.
report: результат find_core_matches().
dry_run: True (default) только логирует counts + первые 30 пар, БЕЗ insert.
Returns:
dict: inserted, conflict_skipped, error_skipped, tier_a_total.
"""
counts = report.counts()
logger.info(
"apply_core_matches (dry_run=%s): tier_a=%d tier_b=%d ambiguous=%d skipped_taken=%d",
dry_run,
counts["tier_a"],
counts["tier_b"],
counts["ambiguous"],
counts["skipped_taken"],
)
for m in report.tier_a[:30]:
logger.info(
" tier_a: obj_id=%s core=%r domrf=%r → objective=%r (dev %r ~ %r)",
m.domrf_obj_id,
m.core,
m.domrf_comm_name,
m.objective_project_name,
m.domrf_dev_name,
m.objective_developers,
)
if dry_run:
return {
"inserted": 0,
"conflict_skipped": 0,
"error_skipped": 0,
"tier_a_total": counts["tier_a"],
}
inserted = 0
conflict_skipped = 0
error_skipped = 0
for m in report.tier_a:
try:
with db.begin_nested():
result = db.execute(
text(
"""
INSERT INTO objective_complex_mapping
(objective_complex_name, objective_project_id, domrf_obj_id,
objective_group, match_method, match_score, is_reviewed, note)
VALUES (
CAST(:name AS text),
CAST(:project_id AS bigint),
CAST(:obj_id AS bigint),
CAST(:group AS text),
CAST(:method AS text),
CAST(:score AS numeric),
CAST(:reviewed AS boolean),
CAST(:note AS text)
)
ON CONFLICT (objective_complex_name, objective_group) DO NOTHING
"""
),
{
"name": m.objective_project_name,
"project_id": m.objective_project_id,
"obj_id": m.domrf_obj_id,
"group": OBJECTIVE_GROUP,
"method": CORE_MATCH_METHOD,
"score": CORE_MATCH_SCORE,
"reviewed": False,
"note": CORE_MATCH_NOTE,
},
)
if result.rowcount > 0:
inserted += 1
else:
conflict_skipped += 1
except Exception as e:
logger.warning(
"core insert failed для %s%s: %s",
m.domrf_comm_name,
m.objective_project_name,
e,
)
error_skipped += 1
db.commit()
logger.info(
"apply_core_matches done: inserted=%d conflict_skipped=%d error_skipped=%d",
inserted,
conflict_skipped,
error_skipped,
)
return {
"inserted": inserted,
"conflict_skipped": conflict_skipped,
"error_skipped": error_skipped,
"tier_a_total": counts["tier_a"],
}
def run_core_pass(*, apply: bool = False) -> dict[str, int]:
"""CLI-обёртка core-pass: открыть Session, найти матчи, (опц.) применить.
Дефолт dry-run. Реальный insert только при apply=True.
"""
from app.core.db import SessionLocal
db = SessionLocal()
try:
report = find_core_matches(db)
return apply_core_matches(db, report, dry_run=not apply)
finally:
db.close()
# ════════════════════════════════════════════════════════════════════════════
# Geo-pass (#2177) — auto_core_geo_v6
# ════════════════════════════════════════════════════════════════════════════
#
# Разрешает ОСТАТОК core-pass (tier_b + ambiguous) географией: адрес objective-
# проекта → DaData-геокод → haversine до domrf-точки. tier_b подтверждается
# близостью (гео вместо dev-confirm); ambiguous — только если РОВНО ОДИН из
# кандидатов ядра в радиусе (иначе остаётся ambiguous).
GEO_MATCH_METHOD = "auto_core_geo_v6"
GEO_MATCH_SCORE = 0.80
GEO_MAX_DISTANCE_M = 500.0
# Бережный потолок вызовов DaData за один прогон (МАКС-тариф, но не жжём квоту
# на прогон-цикле). Превышение → стоп с warning, недобранные кандидаты reject'ятся
# как «без геокода».
GEO_DADATA_CALL_LIMIT = 200
_EARTH_RADIUS_M = 6_371_000.0
def _haversine_m(lat1: float, lon1: float, lat2: float, lon2: float) -> float:
"""Расстояние (метры) между двумя точками по формуле гаверсинуса."""
p1 = math.radians(lat1)
p2 = math.radians(lat2)
dphi = math.radians(lat2 - lat1)
dlambda = math.radians(lon2 - lon1)
a = math.sin(dphi / 2) ** 2 + math.cos(p1) * math.cos(p2) * math.sin(dlambda / 2) ** 2
return 2 * _EARTH_RADIUS_M * math.asin(min(1.0, math.sqrt(a)))
@dataclass
class GeoMatch:
"""Один подтверждённый гео-матч domrf-объекта к objective-проекту.
origin 'tier_b' | 'ambiguous' (из какого класса core-pass пришёл).
distance_m гаверсинус адрес-проекта domrf-точки (метры).
"""
domrf_obj_id: int
domrf_comm_name: str
core: str
objective_project_name: str
objective_project_id: int | None
address: str
distance_m: float
origin: str
@dataclass
class GeoReject:
"""Отклонённый гео-кандидат (для отчёта). reason: 'no_address' |
'no_geocode' | 'too_far' | 'ambiguous_multi' | 'call_limit'.
distance_m None когда дистанцию посчитать не удалось (нет адреса/геокода/
координат domrf).
"""
domrf_obj_id: int
domrf_comm_name: str
objective_project_name: str
origin: str
reason: str
distance_m: float | None = None
@dataclass
class GeoMatchReport:
"""Итог geo-pass.
confirmed подтверждённые гео-матчи (пишутся в apply_geo_matches).
rejected кандидаты без адреса/геокода/слишком далёкие/оставшиеся ambiguous.
dadata_calls сколько раз реально дёрнули DaData (для контроля квоты).
call_limit_hit True если упёрлись в GEO_DADATA_CALL_LIMIT (часть кандидатов
не обработана см. rejected c reason='call_limit').
"""
confirmed: list[GeoMatch] = field(default_factory=list)
rejected: list[GeoReject] = field(default_factory=list)
dadata_calls: int = 0
call_limit_hit: bool = False
def counts(self) -> dict[str, int]:
return {
"confirmed": len(self.confirmed),
"rejected": len(self.rejected),
"dadata_calls": self.dadata_calls,
}
# ── SQL: mode (самый частый непустой) адрес objective_lots per project_name ────
# Адресов на проект много (по корпусам) — берём mode: GROUP BY (project_name,
# address) → COUNT DESC, DISTINCT ON (project_name) отбирает верхнюю строку.
# NULLIF(trim,'') отсекает пустые адреса до агрегации.
_OBJECTIVE_ADDRESS_SQL = text("""
SELECT DISTINCT ON (project_name)
project_name, address
FROM (
SELECT
ol.project_name AS project_name,
NULLIF(trim(ol.address), '') AS address,
COUNT(*) AS cnt
FROM objective_lots ol
WHERE ol.project_name IS NOT NULL
AND ol.project_name <> ''
AND NULLIF(trim(ol.address), '') IS NOT NULL
GROUP BY ol.project_name, NULLIF(trim(ol.address), '')
) agg
ORDER BY project_name, cnt DESC, address
""")
# ── SQL: latest-snapshot координаты domrf per obj_id (для набора obj_id) ───────
# DISTINCT ON (obj_id) + snapshot_date DESC — берём свежую точку. Фильтр по
# конкретным obj_id (кандидаты geo-pass), NULL-координаты отбрасываются.
_DOMRF_COORDS_SQL = text("""
SELECT DISTINCT ON (o.obj_id)
o.obj_id, o.latitude, o.longitude
FROM domrf_kn_objects o
WHERE o.obj_id = ANY(:obj_ids)
AND o.latitude IS NOT NULL
AND o.longitude IS NOT NULL
ORDER BY o.obj_id, o.snapshot_date DESC NULLS LAST
""")
def _load_project_addresses(db: Session) -> dict[str, str]:
"""project_name → mode-адрес (самый частый непустой) из objective_lots."""
out: dict[str, str] = {}
for row in db.execute(_OBJECTIVE_ADDRESS_SQL).all():
name = str(row[0])
addr = row[1]
if addr:
out[name] = str(addr)
return out
def _load_domrf_coords(db: Session, obj_ids: list[int]) -> dict[int, tuple[float, float]]:
"""obj_id → (lat, lon) latest snapshot для заданного набора obj_id."""
if not obj_ids:
return {}
out: dict[int, tuple[float, float]] = {}
for row in db.execute(_DOMRF_COORDS_SQL, {"obj_ids": obj_ids}).all():
lat = float(row[1])
lon = float(row[2])
out[int(row[0])] = (lat, lon)
return out
def find_geo_matches(db: Session, *, max_distance_m: float = GEO_MAX_DISTANCE_M) -> GeoMatchReport:
"""Разрешить остаток core-pass (tier_b + ambiguous) через DaData-геокод.
Алгоритм:
1. find_core_matches берём report.tier_b и report.ambiguous.
2. Для каждого кандидата: адрес проекта (mode objective_lots.address)
clean_address (lat, lon); haversine до domrf-точки.
tier_b: дистанция max_distance_m confirm (гео вместо dev);
ambiguous: геокодим ВСЕХ кандидатов ядра; confirm только если РОВНО
ОДИН в радиусе (иначе остаётся ambiguous);
нет адреса / DaData null / дистанция больше reject c дистанцией.
3. Кэш геокода в памяти прогона (адреса повторяются). Лимит вызовов DaData
GEO_DADATA_CALL_LIMIT при превышении стоп с warning, остаток reject.
Args:
db: SQLAlchemy sync Session.
max_distance_m: радиус подтверждения (метры), default 500.
Returns:
GeoMatchReport (confirmed / rejected / dadata_calls / call_limit_hit).
Реального обращения к БД / DaData в тестах нет оба мокаются по паттерну файла.
"""
core_report = find_core_matches(db)
report = GeoMatchReport()
tier_b = core_report.tier_b
ambiguous = core_report.ambiguous
if not tier_b and not ambiguous:
logger.info("find_geo_matches: нет tier_b/ambiguous кандидатов — nothing to do")
return report
addresses = _load_project_addresses(db)
# ── ambiguous: нужны ВСЕ objective-кандидаты ядра, не только первый ────────
# core-report.ambiguous хранит по одному CoreMatch на domrf-obj_id (первый
# кандидат). Восстанавливаем полный список project_name'ов ядра из objective-
# стороны, чтобы геокодить всех и выбрать единственного в радиусе.
objective_by_core = _load_objective_by_core(db)
# obj_id'ы, для которых нужны координаты domrf (tier_b + ambiguous)
needed_obj_ids = [m.domrf_obj_id for m in tier_b] + [m.domrf_obj_id for m in ambiguous]
domrf_coords = _load_domrf_coords(db, needed_obj_ids)
# Кэш геокода в памяти прогона: address → (lat, lon) | None (None = «пробовали,
# DaData не дал»; кэшируем и его, чтобы не жечь квоту на повтор одного адреса).
geo_cache: dict[str, tuple[float, float] | None] = {}
def _geocode(addr: str) -> tuple[float, float] | None:
"""Геокод с кэшем + лимитом вызовов. None при отсутствии/лимите.
Адреса objective_lots идут БЕЗ города («ул. Гидрострой, д. 2а»)
DaData на таком отдаёт qc_geo=5 (нет координат): прод-прогон 2026-07-03
дал confirmed=0/45 именно из-за этого. Объектив покрывает только ЕКБ
безопасно префиксуем городом, если его нет в строке.
"""
if addr in geo_cache:
return geo_cache[addr]
if report.dadata_calls >= GEO_DADATA_CALL_LIMIT:
report.call_limit_hit = True
return None
query = addr if "екатеринбург" in addr.lower() else f"г Екатеринбург, {addr}"
report.dadata_calls += 1
coords = clean_address(query)
geo_cache[addr] = coords
return coords
# ── tier_b: 1 кандидат, подтверждаем близостью ────────────────────────────
for m in tier_b:
domrf_pt = domrf_coords.get(m.domrf_obj_id)
addr = addresses.get(m.objective_project_name)
if not addr:
report.rejected.append(_geo_reject(m, "tier_b", "no_address"))
continue
if domrf_pt is None:
# у domrf нет координат — дистанцию не посчитать; не жжём квоту на геокод
report.rejected.append(_geo_reject(m, "tier_b", "no_geocode"))
continue
if report.dadata_calls >= GEO_DADATA_CALL_LIMIT and addr not in geo_cache:
report.call_limit_hit = True
report.rejected.append(_geo_reject(m, "tier_b", "call_limit"))
continue
coords = _geocode(addr)
if coords is None:
report.rejected.append(_geo_reject(m, "tier_b", "no_geocode"))
continue
dist = _haversine_m(coords[0], coords[1], domrf_pt[0], domrf_pt[1])
if dist <= max_distance_m:
report.confirmed.append(_geo_confirm(m, addr, dist, "tier_b"))
else:
report.rejected.append(_geo_reject(m, "tier_b", "too_far", dist))
# ── ambiguous: геокодим всех кандидатов ядра, confirm если РОВНО ОДИН близко ─
for m in ambiguous:
domrf_pt = domrf_coords.get(m.domrf_obj_id)
if domrf_pt is None:
report.rejected.append(_geo_reject(m, "ambiguous", "no_geocode"))
continue
candidates = objective_by_core.get(m.core, [])
in_radius: list[tuple[str, int | None, str, float]] = []
any_geocoded = False
geocoded_count = 0
for proj_name, _devs, proj_id in candidates:
addr = addresses.get(proj_name)
if not addr:
continue
if report.dadata_calls >= GEO_DADATA_CALL_LIMIT and addr not in geo_cache:
report.call_limit_hit = True
continue
coords = _geocode(addr)
if coords is None:
continue
any_geocoded = True
geocoded_count += 1
dist = _haversine_m(coords[0], coords[1], domrf_pt[0], domrf_pt[1])
if dist <= max_distance_m:
in_radius.append((proj_name, proj_id, addr, dist))
# Ревью 2026-07-03: «ровно один в радиусе» доверяем ТОЛЬКО когда
# геокодились ВСЕ кандидаты ядра. Иначе «один» — артефакт отсутствия
# данных (сосед без адреса/геокода мог быть ближе) → слепой confirm
# запрещён, остаётся честный reject partial_geocode.
if len(in_radius) == 1 and geocoded_count < len(candidates):
report.rejected.append(_geo_reject(m, "ambiguous", "partial_geocode", in_radius[0][3]))
elif len(in_radius) == 1:
proj_name, proj_id, addr, dist = in_radius[0]
report.confirmed.append(
GeoMatch(
domrf_obj_id=m.domrf_obj_id,
domrf_comm_name=m.domrf_comm_name,
core=m.core,
objective_project_name=proj_name,
objective_project_id=proj_id,
address=addr,
distance_m=dist,
origin="ambiguous",
)
)
elif not any_geocoded:
reason = "call_limit" if report.call_limit_hit else "no_geocode"
report.rejected.append(_geo_reject(m, "ambiguous", reason))
else:
# 0 в радиусе, или >1 в радиусе → остаётся ambiguous
nearest = min((d for *_, d in in_radius), default=None)
report.rejected.append(_geo_reject(m, "ambiguous", "ambiguous_multi", nearest))
logger.info(
"find_geo_matches: %s call_limit_hit=%s",
report.counts(),
report.call_limit_hit,
)
return report
def _load_objective_by_core(db: Session) -> dict[str, list[tuple[str, set[str], int | None]]]:
"""core → [(project_name, {devs}, project_id), ...] (как в find_core_matches)."""
out: dict[str, list[tuple[str, set[str], int | None]]] = {}
for row in db.execute(_OBJECTIVE_PROJECTS_SQL).all():
project_name = str(row[0])
developers = {str(d) for d in (row[1] or []) if d}
project_id = int(row[2]) if row[2] is not None else None
core = normalize_complex_name(project_name)
if not core:
continue
out.setdefault(core, []).append((project_name, developers, project_id))
return out
def _geo_confirm(m: CoreMatch, address: str, distance_m: float, origin: str) -> GeoMatch:
return GeoMatch(
domrf_obj_id=m.domrf_obj_id,
domrf_comm_name=m.domrf_comm_name,
core=m.core,
objective_project_name=m.objective_project_name,
objective_project_id=m.objective_project_id,
address=address,
distance_m=distance_m,
origin=origin,
)
def _geo_reject(
m: CoreMatch, origin: str, reason: str, distance_m: float | None = None
) -> GeoReject:
return GeoReject(
domrf_obj_id=m.domrf_obj_id,
domrf_comm_name=m.domrf_comm_name,
objective_project_name=m.objective_project_name,
origin=origin,
reason=reason,
distance_m=distance_m,
)
def apply_geo_matches(
db: Session,
report: GeoMatchReport,
*,
dry_run: bool = True,
) -> dict[str, int]:
"""Вставить report.confirmed в objective_complex_mapping (auto_core_geo_v6).
Та же per-row SAVEPOINT + ON CONFLICT (objective_complex_name, objective_group)
DO NOTHING механика, что в apply_core_matches. Каждая строка:
match_method='auto_core_geo_v6', match_score=0.80, is_reviewed=false,
note с дистанцией в метрах и origin (tier_b/ambiguous).
Args:
db: SQLAlchemy sync Session.
report: результат find_geo_matches().
dry_run: True (default) только counts + первые 30 пар с дистанциями,
БЕЗ insert.
Returns:
dict: inserted, conflict_skipped, error_skipped, confirmed_total.
"""
counts = report.counts()
logger.info(
"apply_geo_matches (dry_run=%s): confirmed=%d rejected=%d dadata_calls=%d",
dry_run,
counts["confirmed"],
counts["rejected"],
counts["dadata_calls"],
)
for gm in report.confirmed[:30]:
logger.info(
" confirm[%s]: obj_id=%s core=%r domrf=%r → objective=%r (%.0f м, addr=%r)",
gm.origin,
gm.domrf_obj_id,
gm.core,
gm.domrf_comm_name,
gm.objective_project_name,
gm.distance_m,
gm.address[:60],
)
if dry_run:
return {
"inserted": 0,
"conflict_skipped": 0,
"error_skipped": 0,
"confirmed_total": counts["confirmed"],
}
inserted = 0
conflict_skipped = 0
error_skipped = 0
for gm in report.confirmed:
note = f"#2177 geo-pass ({gm.origin}): {gm.distance_m:.0f} м to domrf point"
try:
with db.begin_nested():
result = db.execute(
text(
"""
INSERT INTO objective_complex_mapping
(objective_complex_name, objective_project_id, domrf_obj_id,
objective_group, match_method, match_score, is_reviewed, note)
VALUES (
CAST(:name AS text),
CAST(:project_id AS bigint),
CAST(:obj_id AS bigint),
CAST(:group AS text),
CAST(:method AS text),
CAST(:score AS numeric),
CAST(:reviewed AS boolean),
CAST(:note AS text)
)
ON CONFLICT (objective_complex_name, objective_group) DO NOTHING
"""
),
{
"name": gm.objective_project_name,
"project_id": gm.objective_project_id,
"obj_id": gm.domrf_obj_id,
"group": OBJECTIVE_GROUP,
"method": GEO_MATCH_METHOD,
"score": GEO_MATCH_SCORE,
"reviewed": False,
"note": note,
},
)
if result.rowcount > 0:
inserted += 1
else:
conflict_skipped += 1
except Exception as e:
logger.warning(
"geo insert failed для %s%s: %s",
gm.domrf_comm_name,
gm.objective_project_name,
e,
)
error_skipped += 1
db.commit()
logger.info(
"apply_geo_matches done: inserted=%d conflict_skipped=%d error_skipped=%d",
inserted,
conflict_skipped,
error_skipped,
)
return {
"inserted": inserted,
"conflict_skipped": conflict_skipped,
"error_skipped": error_skipped,
"confirmed_total": counts["confirmed"],
}
def run_geo_pass(
*, apply: bool = False, max_distance_m: float = GEO_MAX_DISTANCE_M
) -> dict[str, int]:
"""CLI-обёртка geo-pass: открыть Session, найти гео-матчи, (опц.) применить.
Дефолт dry-run. Реальный insert только при apply=True.
"""
from app.core.db import SessionLocal
db = SessionLocal()
try:
report = find_geo_matches(db, max_distance_m=max_distance_m)
return apply_geo_matches(db, report, dry_run=not apply)
finally:
db.close()
def _main(argv: list[str] | None = None) -> int:
parser = argparse.ArgumentParser(
description="objective_backfill passes (#2177): core-pass (auto_core_dev_v5), "
"geo-pass (auto_core_geo_v6)."
)
parser.add_argument(
"--core-pass",
action="store_true",
help="Запустить core-name + dev-confirm проход (auto_core_dev_v5).",
)
parser.add_argument(
"--geo-pass",
action="store_true",
help="Запустить DaData-гео проход по tier_b/ambiguous остатку core-pass "
"(auto_core_geo_v6).",
)
parser.add_argument(
"--max-distance-m",
type=float,
default=GEO_MAX_DISTANCE_M,
help="Радиус подтверждения гео-матча в метрах (geo-pass; "
f"default {GEO_MAX_DISTANCE_M:.0f}).",
)
parser.add_argument(
"--apply",
action="store_true",
help="Реально писать матчи в БД (по умолчанию — dry-run).",
)
args = parser.parse_args(argv)
logging.basicConfig(
level=logging.INFO,
format="%(asctime)s %(levelname)s %(name)s: %(message)s",
)
if not args.core_pass and not args.geo_pass:
parser.print_help()
return 0
if args.core_pass:
result = run_core_pass(apply=args.apply)
logger.info("core-pass result: %s", result)
if args.geo_pass:
geo_result = run_geo_pass(apply=args.apply, max_distance_m=args.max_distance_m)
logger.info("geo-pass result: %s", geo_result)
return 0
if __name__ == "__main__":
sys.exit(_main())

File diff suppressed because it is too large Load diff

File diff suppressed because it is too large Load diff

View file

@ -0,0 +1,494 @@
"""Оркестратор полного PDF-отчёта ПТИЦА (эпик #2259 PR-D).
Собирает единый PDF `/site-finder/analysis/{cad}` из уже-персистнутых ранов + дотянутых
на лету блоков (connection-capacity, концепция), рендерит карты (PR-C) и пишет файл на
volume `/app/reports/` с метадата-строкой в `analysis_runs` (schema `report-pdf-1.0`).
ПОТОК (:func:`build_full_report`):
1. analyze-ран (`latest_run_for(..., schema_version=ANALYZE_SCHEMA_VERSION)`) нет
ValueError (отчёт без базового анализа бессмысленен).
2. forecast-ран (`latest_run_for(..., schema_version="1.0")`) нет best-effort
СИНХРОННО считаем его тут же (`_ensure_forecast_run`, зеркало Celery-таски форсайта,
~2030с) и перечитываем; всё ещё нет (сбой/тонкие данные) Part B (§4§6) деградирует
«нет данных», отчёт всё равно валиден (передаём {} в part_b).
3. КЭШ-ключ = (analyze_run_id, forecast_run_id). Если метадата-ран `report-pdf-1.0` с
теми же id уже есть И файл на месте cache-hit, PDF не пере-рендерим.
4. connection-capacity (`get_connection_capacity`) best-effort, для §3-резервов.
5. концепция server-side генерация (как POST /concepts): `ConceptInput` из geom
участка (дефолт count=3 comfort) `geometry.generate`. Best-effort: упала/нет geom
отчёт без §7-концепции (§5 деградирует в рыночный контекст).
6. HTML (PR-A/B) + карты (PR-C: `render_parcel_map_png` / `render_concept_footprint_png`
`embed_map_png`, PNG max_px=1400) PDF (:func:`render_full_report_pdf`).
6b. DOCX-вариант (PR-F, `build_full_report_docx`) из ТЕХ ЖЕ исходных словарей + ТЕХ ЖЕ
карт-PNG (НЕ рендерим карты дважды) рядом `.docx`-файл.
7. Запись файлов (PDF + DOCX, атомарно tmp+os.replace) + метадата-ран `report-pdf-1.0`
(result = pdf_path/docx_path/analyze_run_id/forecast_run_id/generated_at/size_bytes/
docx_size_bytes). Старые раны без docx_path download?format=docx отдаёт 404.
WeasyPrint импортируется ЛОКАЛЬНО внутри :func:`render_full_report_pdf` (тяжёлый native
ломает pytest-сбор на хостах без GTK/Pango; образец `layout_tz_pdf.render_layout_tz_pdf`).
"""
from __future__ import annotations
import datetime as _dt
import logging
import os
import re
from pathlib import Path
from typing import Any
from sqlalchemy.orm import Session
from app.core.config import settings
from app.services.analysis_runs.repository import (
ANALYZE_SCHEMA_VERSION,
latest_run_for,
persist_analysis_run,
)
from app.services.exporters.full_report_docx import build_full_report_docx
from app.services.exporters.full_report_html import (
MAP_CONCEPT_PLACEHOLDER,
MAP_PARCEL_PLACEHOLDER,
build_full_report_html,
build_full_report_html_part_a,
build_full_report_html_part_b,
)
from app.services.exporters.report_maps import (
embed_map_png,
render_concept_footprint_png,
render_parcel_map_png,
)
logger = logging.getLogger(__name__)
# schema_version метадата-строки отчёта в analysis_runs. Собственная схема (НЕ analyze /
# НЕ §22-форсайт "1.0") — по ней ищем существующий отчёт для cache-hit.
REPORT_SCHEMA_VERSION = "report-pdf-1.0"
# schema_version §22-форсайт-рана (зеркало `_FORECAST_SCHEMA_VERSION` в parcels.py — НЕ
# "forecast-1.0", а именно "1.0", это SiteFinderReport._SCHEMA_VERSION).
_FORECAST_SCHEMA_VERSION = "1.0"
# Горизонты best-effort синхронного форсайта (мес). Зеркало Celery-таски
# `forecast_site_finder_report(horizon=12)`: `_horizons_for(12)` = sorted({6,12,18,24}|{12})
# = [6,12,18,24] (forecast.py) = orchestrator._DEFAULT_HORIZONS. Держим тот же набор,
# чтобы «холодный» участок получил §4§6 идентичные ленивому GET /forecast-пути.
_FORECAST_HORIZONS: tuple[int, ...] = (6, 12, 18, 24)
# Верхняя граница длинной стороны карт-PNG (px) — печатный A4, 1400 достаточно для
# ~150 dpi на ширину колонки, но не раздувает PDF гигабайтными растрами.
_MAP_MAX_PX = 1400
# Радиус поиска ЦП для connection-capacity (м) — тот же дефолт, что у эндпоинта
# GET /{cad}/connection-capacity (parcels.py) и функции get_connection_capacity.
_CONNECTION_CAPACITY_RADIUS_M = 3000
def render_full_report_pdf(html: str) -> bytes:
"""Отрендерить готовый HTML полного отчёта в PDF-байты (WeasyPrint).
Тонкая обёртка: WeasyPrint импортируется ЛОКАЛЬНО (тяжёлый native не нужен при
импорте модуля, иначе ломает pytest-сбор на macOS без GTK/Pango). Образец
`layout_tz_pdf.render_layout_tz_pdf`.
Args:
html: полный HTML-документ (`build_full_report_html`).
Returns:
PDF-байты (готовы к записи файла / FileResponse).
"""
from weasyprint import HTML
pdf_bytes: bytes = HTML(string=html).write_pdf()
logger.info("render_full_report_pdf: %d bytes", len(pdf_bytes))
return pdf_bytes
def _largest_polygon_geojson(geom: dict[str, Any]) -> dict[str, Any]:
"""MultiPolygon → GeoJSON крупнейшего полигона-контура; Polygon и прочее — как есть.
Многоконтурный участок приходит как MultiPolygon, а concept-стек (`parse_parcel` +
`_parcel_centroid_wkt`) принимает только Polygon `ParcelGeometryError`. Берём
контур с максимальной площадью (тот же приём, что generative-геометрия применяет к
buildable-мультиполигону после буфера, geometry.py:263-265). Любой сбой парсинга /
не-MultiPolygon возвращаем geom без изменений (best-effort, не роняем концепцию).
Args:
geom: GeoJSON-геометрия участка (Polygon / MultiPolygon / Feature-обёртка).
Returns:
GeoJSON Polygon крупнейшего контура (если вход был MultiPolygon), иначе `geom`.
"""
geom_dict = geom.get("geometry") if geom.get("type") == "Feature" else geom
if not isinstance(geom_dict, dict) or geom_dict.get("type") != "MultiPolygon":
return geom
try:
from shapely.geometry import mapping, shape
multi = shape(geom_dict)
if multi.is_empty or not hasattr(multi, "geoms"):
return geom
largest = max(multi.geoms, key=lambda g: g.area)
logger.info(
"build_full_report: multi-contour участок — взят крупнейший контур из %d",
len(list(multi.geoms)),
)
return dict(mapping(largest))
except Exception:
# Вырожденная/битая геометрия — отдаём как есть, concept-стек сам решит (best-effort).
logger.exception("build_full_report: не удалось выделить крупнейший контур MultiPolygon")
return geom
def _generate_concept_result(db: Session, analyze: dict[str, Any]) -> dict[str, Any] | None:
"""Сгенерировать концепцию server-side как это делает POST /concepts (best-effort).
Концепция НЕ персистится генерируем на лету из геометрии участка с ДЕФОЛТНОЙ
программой (count=3 comfort три стратегии жадной раскладки). Повторяем поток
`create_concept` (concepts.py): рыночная цена через `_lookup_market_price` по
центроиду `geometry.generate` сериализуем `ConceptOutput`. Любая ошибка (нет
geom / вырожденный участок / сбой БД) None: отчёт валиден без §7-концепции,
§5 деградирует в рыночный контекст.
Args:
db: SQLAlchemy session (для lookup рыночной цены).
analyze: persist-payload analyze-рана (несёт `geom_geojson` участка).
Returns:
Сериализованный `ConceptOutput` (dict `{"variants": [...]}`) или None.
"""
geom = analyze.get("geom_geojson")
if not isinstance(geom, dict):
logger.info("build_full_report: analyze-payload без geom_geojson → §7-концепция пропущена")
return None
# Многоконтурный участок → geom = MultiPolygon, а concept-стек (parse_parcel +
# _parcel_centroid_wkt через _parse_polygon) принимает ТОЛЬКО Polygon и роняет
# ParcelGeometryError("expected Polygon, got MultiPolygon"). Берём крупнейший контур —
# ровно как generative-геометрия после буфера (geometry.py:263-265). Иначе §7 и
# market-price молча деградируют на любом мультиконтуре.
geom = _largest_polygon_geojson(geom)
try:
# Lazy import — тяжёлый generative-стек не нужен на module-load; concepts-хелперы
# цены живут в API-слое (он знает БД), переиспользуем ИМЕННО их (single source).
from app.api.v1.concepts import _lookup_market_price, _parcel_centroid_wkt
from app.schemas.concept import ConceptInput, ConceptOutput
from app.services.generative import geometry
payload = ConceptInput(parcel_geojson=geom) # дефолт: housing_class="comfort", 3 стратегии
market_price: float | None = None
price_source = "class_norm"
try:
wkt_point = _parcel_centroid_wkt(payload)
market_price, price_source = _lookup_market_price(db, wkt_point)
except Exception as exc:
# Рыночная цена best-effort — деградируем в норматив класса, не роняем концепцию.
logger.warning("build_full_report: market-price lookup упал, class_norm: %s", exc)
variants = geometry.generate(
payload,
market_price_per_sqm=market_price,
price_source=price_source,
)
result: dict[str, Any] = ConceptOutput(variants=variants).model_dump()
logger.info("build_full_report: концепция сгенерирована (%d вариантов)", len(variants))
return result
except Exception:
# Вырожденный участок / сбой generative-стека — отчёт валиден БЕЗ §7-концепции.
logger.exception("build_full_report: генерация концепции упала → §7 без концепции")
return None
def _get_connection_capacity(db: Session, cad: str) -> dict[str, Any] | None:
"""Дотянуть connection-capacity для §3-резервов (best-effort).
connection-capacity НЕ в persist-payload (ленивый эндпоинт) тянем `get_connection_capacity`
как это делает GET /{cad}/connection-capacity. Ошибка (участок без geom / сбой БД) None:
§3 деградирует к тому, что есть в analyze-payload (OSM-сети + НСПД-сооружения).
"""
try:
from app.services.site_finder.connection_capacity_lookup import get_connection_capacity
return get_connection_capacity(db, cad, _CONNECTION_CAPACITY_RADIUS_M)
except Exception:
logger.exception("build_full_report: connection-capacity lookup упал → §3 без резервов")
return None
def _ensure_forecast_run(
db: Session,
cad: str,
analyze_row: Any,
analyze: dict[str, Any],
) -> None:
"""Best-effort синхронно посчитать §22-форсайт для «холодного» участка (#2259 gap).
§22-форсайт-ран ("1.0") существует ТОЛЬКО если пользователь открывал страницу участка
(§22 строится лениво GET /forecast-поллингом Celery-таской `forecast_site_finder_report`).
На холодном участке полный отчёт выходил БЕЗ §4§6. Здесь тот же compute+persist, что
делает Celery-таска, но СИНХРОННО и ПРЯМЫМ вызовом: мы УЖЕ внутри worker'а
(build_full_report_task), поэтому НЕ .delay считаем inline (~2030с) и персистим тем же
контрактом, чтобы последующий `latest_run_for("1.0")` в build_full_report поймал свежий ран
(его новый id корректно войдёт в кэш-ключ отчёта cache-miss §4§6 попадут в PDF).
Зеркало `forecast_site_finder_report(horizon=12)` (workers/tasks/forecast.py):
horizons = `_FORECAST_HORIZONS` (= `_horizons_for(12)` = [6,12,18,24]);
district = денорм-колонка рана fallback analyze["district"]["district_name"];
`build_site_finder_report(...)` `report.as_dict()` `persist_analysis_run(...,
schema_version=d["schema_version"], status="done", ...)`.
Best-effort: ЛЮБОЙ сбой/долгий compute logger.warning + return (НЕ exception форсайт
может честно не собраться на тонких данных, GlitchTip-шум не нужен). Тогда отчёт, как и
раньше, выйдет без §4§6 (part_b деградирует «нет данных»).
Args:
db: SQLAlchemy session (та же, что у build_full_report свою НЕ открываем).
cad: кадастровый номер участка.
analyze_row: Row analyze-рана (несёт денорм `district`).
analyze: persist-payload analyze-рана (district-fallback + competitors для сегмента).
"""
try:
# Lazy import — тяжёлый forecasting-стек не нужен на module-load (как concept-стек).
from app.services.forecasting.orchestrator import build_site_finder_report
district = analyze_row.district or (analyze.get("district") or {}).get("district_name")
logger.info(
"build_full_report: холодный участок cad=%s — best-effort синхронный §22-форсайт "
"(district=%s horizons=%s)",
cad,
district,
_FORECAST_HORIZONS,
)
report = build_site_finder_report(
db,
analyze=analyze,
cad_num=cad,
district=district,
horizons=_FORECAST_HORIZONS,
)
d = report.as_dict()
new_id = persist_analysis_run(
db,
cad_num=cad,
result=d,
params={"horizon": 12, "source": "full-report-inline-forecast"},
district=district,
confidence=(d.get("confidence") or {}).get("level"),
status="done",
schema_version=d["schema_version"], # "1.0" (SiteFinderReport._SCHEMA_VERSION)
created_by=None,
segment=(d.get("meta") or {}).get("segment"),
)
logger.info("build_full_report: §22-форсайт посчитан inline cad=%s run_id=%s", cad, new_id)
except Exception:
# Форсайт может честно не собраться (тонкие данные / сбой §9.x-шва) или занять
# слишком долго — деградируем в отчёт без §4§6 (warning, НЕ exception: держим
# GlitchTip-шум в узде, ведёт себя как ленивая Celery-таска, которая тоже best-effort).
logger.warning(
"build_full_report: inline §22-форсайт не собрался cad=%s → отчёт без §4§6",
cad,
exc_info=True,
)
def _find_cached_report(
db: Session,
cad: str,
analyze_run_id: int | None,
forecast_run_id: int | None,
) -> dict[str, Any] | None:
"""Найти готовый метадата-ран `report-pdf-1.0` с тем же кэш-ключом + существующим файлом.
Кэш-ключ = (analyze_run_id, forecast_run_id). Совпали ОБА id И файл `pdf_path` на
месте возвращаем result метадата-рана (cache-hit, не пере-рендерим). Иначе None.
"""
row = latest_run_for(db, cad, schema_version=REPORT_SCHEMA_VERSION)
if row is None:
return None
result = row.result if isinstance(row.result, dict) else {}
if (
result.get("analyze_run_id") == analyze_run_id
and result.get("forecast_run_id") == forecast_run_id
):
pdf_path = result.get("pdf_path")
if isinstance(pdf_path, str) and os.path.exists(pdf_path):
return result
return None
def _cad_safe(cad: str) -> str:
"""Кад.номер → безопасный компонент имени файла (whitelist).
Кад.номер приходит из URL-path жёстко ограничиваем алфавит `[0-9:]` (цифры +
разделители кад.номера РФ), ВСЁ прочее вырезаем, затем `:` `_`. Так в имя файла
физически не попадут `/`, `..`, пробелы или спецсимволы (path-traversal / инъекция
в путь), даже если path-param окажется мусорным.
"""
return re.sub(r"[^0-9:]", "", cad).replace(":", "_")
def _atomic_write(path: Path, data: bytes) -> None:
"""Атомарно записать байты в `path`: `.<pid>.tmp` рядом → `os.replace`.
Два конкурентных POST в один день целятся в ОДИН путь (имя несёт только дату)
прямой `write_bytes` мог бы interleave-писать байты обоих рендеров в один файл
(битый вывод). `os.replace` атомарен в пределах одной FS download всегда видит
целый файл (свой или чужой). Общий для PDF и DOCX (тот же приём).
"""
tmp_path = path.with_suffix(f".{os.getpid()}.tmp")
tmp_path.write_bytes(data)
os.replace(tmp_path, path)
def build_full_report(db: Session, cad: str) -> dict[str, Any]:
"""Собрать (или вернуть из кэша) полный PDF-отчёт участка + метадата-ран. #2259 PR-D.
Оркестрация: analyze-ран (нет ValueError) forecast-ран (нет Part B «нет данных»)
cache-check по (analyze_run_id, forecast_run_id) connection-capacity (§3-резервы,
best-effort) концепция server-side (best-effort) HTML (PR-A/B) + карты (PR-C)
PDF (WeasyPrint) запись файла на volume + метадата-ран `report-pdf-1.0`.
Args:
db: SQLAlchemy session.
cad: кадастровый номер участка.
Returns:
Метадата отчёта: {pdf_path, analyze_run_id, forecast_run_id, generated_at,
size_bytes} (+ `cache_hit: True` если вернули существующий файл).
Raises:
ValueError: нет analyze-рана для участка (нечего отчётировать).
"""
analyze_row = latest_run_for(db, cad, schema_version=ANALYZE_SCHEMA_VERSION)
if analyze_row is None:
raise ValueError(f"нет analyze-рана для {cad!r} — сначала запустите /analyze")
analyze: dict[str, Any] = analyze_row.result or {}
analyze_run_id = int(analyze_row.id)
forecast_row = latest_run_for(db, cad, schema_version=_FORECAST_SCHEMA_VERSION)
if forecast_row is None:
# Холодный участок: §22-форсайт-ран ("1.0") строится лениво GET /forecast-поллингом
# и на не-открытом участке отсутствует → отчёт выходил без §4§6. Best-effort
# считаем его СИНХРОННО прямо тут (мы в worker'е) и перечитываем — самодостаточность
# отчёта важнее +2030с (как §7-концепция генерится в оркестраторе). Сбой/долго →
# forecast_row остаётся None, part_b деградирует «нет данных» (как раньше).
_ensure_forecast_run(db, cad, analyze_row, analyze)
forecast_row = latest_run_for(db, cad, schema_version=_FORECAST_SCHEMA_VERSION)
forecast: dict[str, Any] = (forecast_row.result or {}) if forecast_row is not None else {}
forecast_run_id = int(forecast_row.id) if forecast_row is not None else None
# Cache-hit: тот же (analyze, forecast) ран-ключ + файл на месте → не пере-рендерим.
cached = _find_cached_report(db, cad, analyze_run_id, forecast_run_id)
if cached is not None:
logger.info(
"build_full_report: cache-hit cad=%s analyze=%s forecast=%s path=%s",
cad,
analyze_run_id,
forecast_run_id,
cached.get("pdf_path"),
)
return {**cached, "cache_hit": True}
# connection-capacity (§3-резервы) + концепция (§7) — оба best-effort.
connection_capacity = _get_connection_capacity(db, cad)
concept = _generate_concept_result(db, analyze)
# Адрес участка для титула (egrn.address; None → строка адреса не рисуется).
egrn = analyze.get("egrn") if isinstance(analyze.get("egrn"), dict) else {}
address = egrn.get("address") if isinstance(egrn, dict) else None
generated_at = _dt.datetime.now(_dt.UTC)
generated_at_ru = generated_at.strftime("%d.%m.%Y")
# HTML (PR-A/B): §1§3 (+ connection-capacity в §3) и §4§7 (пустой forecast → «нет данных»).
part_a = build_full_report_html_part_a(
analyze, cad=cad, connection_capacity=connection_capacity
)
part_b = build_full_report_html_part_b(forecast, concept, cad=cad, analyze_result=analyze)
html = build_full_report_html(
part_a,
part_b,
cad=cad,
address=address if isinstance(address, str) else None,
generated_at=generated_at_ru,
)
# Карты (PR-C): участок (§1/§3) + концепция (§7, лучший вариант). Оба graceful:
# None → embed_map_png ставит плашку «карта недоступна».
parcel_png = render_parcel_map_png(analyze, max_px=_MAP_MAX_PX)
html = embed_map_png(html, MAP_PARCEL_PLACEHOLDER, parcel_png)
concept_png: bytes | None = None
if isinstance(concept, dict):
variants = concept.get("variants")
if isinstance(variants, list) and variants and isinstance(variants[0], dict):
concept_png = render_concept_footprint_png(
variants[0], analyze.get("geom_geojson"), max_px=_MAP_MAX_PX
)
html = embed_map_png(html, MAP_CONCEPT_PLACEHOLDER, concept_png)
pdf_bytes = render_full_report_pdf(html)
# DOCX-вариант (PR-F): из ТЕХ ЖЕ исходных словарей + ТЕХ ЖЕ карт-PNG (parcel_png /
# concept_png уже отрендерены выше — НЕ рендерим карты дважды). Зеркалит §1§7 PDF.
docx_bytes = build_full_report_docx(
analyze,
forecast,
concept,
connection_capacity,
cad=cad,
address=address if isinstance(address, str) else None,
generated_at=generated_at_ru,
parcel_map_png=parcel_png,
concept_map_png=concept_png,
)
# Запись файлов на volume + метадата-ран. Каталог создаём (parents, exist_ok).
reports_dir = Path(settings.reports_dir)
reports_dir.mkdir(parents=True, exist_ok=True)
base_name = f"gendesign_report_{_cad_safe(cad)}_{generated_at_ru}"
pdf_path = reports_dir / f"{base_name}.pdf"
docx_path = reports_dir / f"{base_name}.docx"
# Атомарная запись обоих файлов (tmp+os.replace — см. `_atomic_write`).
_atomic_write(pdf_path, pdf_bytes)
_atomic_write(docx_path, docx_bytes)
size_bytes = len(pdf_bytes)
docx_size_bytes = len(docx_bytes)
result: dict[str, Any] = {
"pdf_path": str(pdf_path),
"docx_path": str(docx_path),
"analyze_run_id": analyze_run_id,
"forecast_run_id": forecast_run_id,
"generated_at": generated_at.isoformat(),
"size_bytes": size_bytes,
"docx_size_bytes": docx_size_bytes,
}
# Метадата-ран `report-pdf-1.0` (best-effort persist; провал не роняет отчёт — файлы
# уже записаны, просто следующий вызов не поймает cache-hit и пере-рендерит).
persist_analysis_run(
db,
cad_num=cad,
result=result,
params={"source": "report-pdf-task"},
district=analyze_row.district,
confidence=None,
status="done",
schema_version=REPORT_SCHEMA_VERSION,
created_by=None,
)
logger.info(
"build_full_report: cad=%s pdf=%s (%d B) docx=%s (%d B) analyze=%s forecast=%s",
cad,
pdf_path,
size_bytes,
docx_path,
docx_size_bytes,
analyze_run_id,
forecast_run_id,
)
return result

View file

@ -14,15 +14,20 @@ from app.schemas.parcel import BestLayoutsResponse
logger = logging.getLogger(__name__)
def render_layout_tz_pdf(
def build_layout_tz_html(
response: BestLayoutsResponse,
*,
cad_num: str,
parcel_address: str | None = None,
radius_km: float,
time_window: str,
) -> bytes:
"""Render ТЗ PDF от best-layouts response.
) -> str:
"""Собрать HTML ТЗ из best-layouts response (PURE — без WeasyPrint/БД/сети).
Вынесено из :func:`render_layout_tz_pdf`, чтобы HTML-разметку (наличие блоков,
RU-микрокопию, форматирование /м²) можно было проверять юнит-тестами БЕЗ
native GTK/Pango (WeasyPrint на macOS/Windows отсутствует). PDF-обёртка ниже
просто рендерит этот HTML.
Args:
response: BestLayoutsResponse от /best-layouts endpoint
@ -32,7 +37,7 @@ def render_layout_tz_pdf(
time_window: окно анализа (last_month/quarter/year)
Returns:
PDF bytes (готово для StreamingResponse)
HTML-строка документа (готова для WeasyPrint).
"""
today = dt.date.today().strftime("%d.%m.%Y")
safe_cad = _html.escape(cad_num)
@ -45,6 +50,18 @@ def render_layout_tz_pdf(
return "<td>—</td>"
return f"<td>{val:,.0f}".replace(",", " ") + " ₽</td>"
def _price_m2_cell(val: float | None) -> str:
"""Ячейка цены ₽/м² (тыс-разделитель — пробел). None → «—» (graceful)."""
if val is None:
return "<td>—</td>"
return f"<td>{val:,.0f}".replace(",", " ") + " ₽/м²</td>"
def _area_cell(val: float | None) -> str:
"""Ячейка площади, м² (1 знак). None → «—» (graceful)."""
if val is None:
return "<td>—</td>"
return f"<td>{val:.1f} м²</td>"
# Top layouts table rows
top_rows = "".join(
"<tr>"
@ -70,6 +87,42 @@ def render_layout_tz_pdf(
for m in response.recommendation_for_tz.mix
)
# §4.2 «По предложению (без темпа продаж)» (#2177 шаг 3). Аддитивный fallback-блок:
# структура текущего предложения (комплекс/тип/площадь/лотов/медиана ₽/м²) по
# комплексам, сматченным по ядру имени, но БЕЗ velocity. Graceful — на старых
# снапшотах поле отсутствует / пустой список (default []), тогда секцию не рисуем.
supply_only = getattr(response, "supply_only_layouts", None) or []
supply_only_rows = "".join(
"<tr>"
f"<td>{_html.escape(s.project_name)}</td>"
f"<td>{_html.escape(s.room_bucket)}</td>"
f"<td>{_html.escape(s.area_bin)}</td>"
f"{_area_cell(s.median_area_m2)}"
f"<td>{s.lots_count}</td>"
f"{_price_m2_cell(s.median_price_m2)}"
"</tr>"
for s in supply_only
)
supply_only_section = (
f"""
<h2>По предложению (без темпа продаж)</h2>
<p class="caveat">
Комплексы сматчены по ядру имени на объявления Объектива, но по ним НЕТ сделок
(темпа продаж) ниже только структура остатков в продаже (objective_lots),
БЕЗ ранжирования по скорости. Дополняет топ продаж выше, не заменяет его.
</p>
<table>
<thead><tr>
<th>Комплекс</th><th>Комнаты</th><th>Площадь</th>
<th>Ср. площадь</th><th>Лотов в продаже</th><th>Медиана, /м²</th>
</tr></thead>
<tbody>{supply_only_rows}</tbody>
</table>
"""
if supply_only
else ""
)
rec = response.recommendation_for_tz
safe_rationale = _html.escape(rec.rationale_text)
weighted_price = (
@ -95,6 +148,8 @@ def render_layout_tz_pdf(
th {{ background: #f5f5f5; font-weight: bold; }}
.rationale {{ background: #f8f8f8; padding: 10px; border-left: 3px solid #4a90e2;
margin: 1em 0; }}
.caveat {{ background: #fffbeb; padding: 8px 10px; border-left: 3px solid #c9a132;
margin: 0.6em 0; color: #7a5c12; font-size: 9.5pt; }}
.footer {{ margin-top: 2em; padding-top: 1em; border-top: 1px solid #ddd;
color: #888; font-size: 9pt; }}
.confidence-high {{ color: #2a8c2a; }}
@ -133,11 +188,11 @@ def render_layout_tz_pdf(
</tr></thead>
<tbody>{top_rows}</tbody>
</table>
{supply_only_section}
<h2>Качество данных</h2>
<p>
Покрытие: {dq.objects_with_velocity_data} из
{dq.objects_total_in_radius} ЖК с данными velocity
{dq.objects_total_in_radius} комплексов с данными velocity
({dq.velocity_coverage_pct:.1f}%)
</p>
<p>
@ -154,6 +209,40 @@ def render_layout_tz_pdf(
</body>
</html>"""
return html
def render_layout_tz_pdf(
response: BestLayoutsResponse,
*,
cad_num: str,
parcel_address: str | None = None,
radius_km: float,
time_window: str,
) -> bytes:
"""Render ТЗ PDF от best-layouts response.
Тонкая обёртка над :func:`build_layout_tz_html` собирает HTML (PURE) и
рендерит его WeasyPrint'ом в PDF-байты.
Args:
response: BestLayoutsResponse от /best-layouts endpoint
cad_num: кадастровый номер участка
parcel_address: optional human address (если known через geocoder)
radius_km: радиус анализа конкурентов
time_window: окно анализа (last_month/quarter/year)
Returns:
PDF bytes (готово для StreamingResponse)
"""
html = build_layout_tz_html(
response,
cad_num=cad_num,
parcel_address=parcel_address,
radius_km=radius_km,
time_window=time_window,
)
# WeasyPrint импортируем локально — тяжёлый; не нужен при импорте модуля
# (иначе ломает сбор pytest на хостах без native-libs, напр. macOS).
from weasyprint import HTML

View file

@ -0,0 +1,386 @@
"""Серверный статик-рендер карт для PDF-отчёта ПТИЦА (эпик #2259 PR-C).
Рендерит два вида PNG-карт БЕЗ браузера (headless matplotlib + geopandas + contextily):
:func:`render_parcel_map_png` карта участка (§1/§3): полигон участка (заливка +
контур), поверх риск-зоны НСПД / красные линии / ЗОУИТ-пересечения (если геометрии
присутствуют в persist-payload) + конкуренты точками. Подложка OSM-тайлы через
contextily; auto-bbox = участок + буфер.
:func:`render_concept_footprint_png` карта концепции (§7): контур участка + пятна
застройки (footprint'ы корпусов из ``ConceptVariant.buildings_geojson``) с подписью
этажности (``properties.floors``).
Плюс :func:`embed_map_png` подстановка ``<img src="data:image/png;base64,…">`` вместо
строки-плейсхолдера в готовом HTML (``png=None`` плашка «карта недоступна»).
CRS-контракт: все входные геометрии EPSG:4326 (GeoJSON ``geom_geojson`` /
``geometry_geojson``, WKT ``geom_wkt`` в 4326, конкуренты lat/lng). Рисуем в Web Mercator
(EPSG:3857), т.к. OSM-тайлы contextily идут в 3857 иначе подложка не совпадёт со слоями.
GRACEFUL: нет ``geom_geojson`` ``None`` (нечего рисовать). Тайлы OSM недоступны/таймаут
белый фон + слои, PNG всё равно возвращается (никогда не роняем экспорт из-за сети).
Битые отдельные геометрии слоёв пропускаются с ``logger.warning``, не валят всю карту.
HEADLESS: ``matplotlib.use("Agg")`` вызывается на уровне модуля ДО импорта pyplot импорт
``report_maps`` не требует X11/display (запускается в Celery-воркере/Docker без GUI).
"""
from __future__ import annotations
import base64
import io
import logging
from collections.abc import Iterator
from concurrent.futures import ThreadPoolExecutor
from concurrent.futures import TimeoutError as FuturesTimeoutError
from contextlib import contextmanager
from typing import Any
import matplotlib
# Headless-бэкенд ДО импорта pyplot — модуль импортируется в Celery-воркере без display.
matplotlib.use("Agg")
import matplotlib.pyplot as plt
from matplotlib.figure import Figure
from shapely.geometry import Point, shape
from shapely.geometry.base import BaseGeometry
logger = logging.getLogger(__name__)
# ── Константы рендера ──────────────────────────────────────────────────────────
_WGS84 = "EPSG:4326"
_WEB_MERCATOR = "EPSG:3857"
# Буфер вокруг участка при auto-bbox (доля от большей стороны bbox). Даёт «воздух»
# по краям, чтобы контур участка не липнул к рамке и был виден контекст подложки.
_BBOX_PAD_FRAC: float = 0.25
# Таймаут ожидания OSM-тайлов (сек). Тайлы тянутся синхронно внутри contextily —
# оборачиваем в поток с bounded-таймаутом, чтобы медленный/недоступный tile-server
# не подвешивал воркер. Превышение → graceful fallback на белый фон.
_BASEMAP_TIMEOUT_S: float = 10.0
# Базовый физический размер фигуры (дюймы). Итоговый пиксель-размер = fig_inches * dpi,
# ограничивается сверху ``max_px`` пересчётом dpi (см. ``_figure_to_png``).
_FIG_INCHES: float = 6.4
# Палитра слоёв (edgecolor, facecolor, alpha) — консистентно с семантикой отчёта.
_PARCEL_STYLE = {"edgecolor": "#1d4ed8", "facecolor": "#3b82f6", "alpha": 0.28, "linewidth": 1.8}
_RISK_STYLE = {"edgecolor": "#b91c1c", "facecolor": "#ef4444", "alpha": 0.30, "linewidth": 1.0}
_REDLINE_STYLE = {"color": "#dc2626", "linewidth": 1.6, "linestyle": "--"}
_ZOUIT_STYLE = {"edgecolor": "#b45309", "facecolor": "#f59e0b", "alpha": 0.22, "linewidth": 1.0}
_OPPORTUNITY_STYLE = {
"edgecolor": "#15803d",
"facecolor": "#22c55e",
"alpha": 0.20,
"linewidth": 0.9,
}
_COMPETITOR_STYLE = {"color": "#7c3aed", "marker": "o", "markersize": 5.0, "linestyle": "none"}
_CONCEPT_FOOTPRINT_STYLE = {
"edgecolor": "#0f766e",
"facecolor": "#14b8a6",
"alpha": 0.55,
"linewidth": 1.2,
}
_MAP_UNAVAILABLE_HTML = (
'<div class="map-placeholder">Карта недоступна — геоданные участка отсутствуют '
"в отчёте</div>"
)
# ── Парсинг входных геометрий (EPSG:4326) ──────────────────────────────────────
def _geom_from_geojson(geojson: dict[str, Any] | None) -> BaseGeometry | None:
"""GeoJSON-geometry dict → shapely-geометрия (EPSG:4326). Битый вход → None+warning."""
if not isinstance(geojson, dict):
return None
try:
geom = shape(geojson)
except Exception as exc: # любой битый GeoJSON: пропустить слой, не валить всю карту
logger.warning("report_maps: не разобрал GeoJSON-геометрию: %s", exc)
return None
if geom.is_empty:
return None
return geom
def _geom_from_wkt(wkt: str | None) -> BaseGeometry | None:
"""WKT-строка (EPSG:4326) → shapely-геометрия. Битый вход → None+warning."""
if not wkt or not isinstance(wkt, str):
return None
try:
from shapely import wkt as shapely_wkt
geom = shapely_wkt.loads(wkt)
except Exception as exc: # битый WKT слоя: пропустить геометрию, не валить всю карту
logger.warning("report_maps: не разобрал WKT-геометрию: %s", exc)
return None
if geom.is_empty:
return None
return geom
def _to_mercator(geoms: list[BaseGeometry]) -> Any:
"""Список shapely-геометрий (EPSG:4326) → GeoSeries в Web Mercator (EPSG:3857).
Импорт geopandas локальный (тяжёлый), вызывается только когда есть что рисовать.
"""
import geopandas as gpd
series = gpd.GeoSeries(geoms, crs=_WGS84)
return series.to_crs(_WEB_MERCATOR)
# ── Basemap (OSM-тайлы) с bounded-таймаутом и graceful fallback ────────────────
def _add_basemap(ax: Any) -> bool:
"""Подложить OSM-тайлы под слои с таймаутом. True — легли, False — fallback (белый фон).
contextily тянет тайлы синхронно (сеть) оборачиваем в поток с таймаутом, чтобы
недоступный tile-сервер не подвешивал воркер. Любая ошибка/таймаут белый фон,
слои остаются видны, PNG всё равно рендерится.
"""
import contextily as cx
def _fetch() -> None:
cx.add_basemap(ax, crs=_WEB_MERCATOR, source=cx.providers.OpenStreetMap.Mapnik)
try:
with ThreadPoolExecutor(max_workers=1) as pool:
pool.submit(_fetch).result(timeout=_BASEMAP_TIMEOUT_S)
return True
except FuturesTimeoutError:
logger.warning(
"report_maps: OSM basemap timeout (%.0fs) — fallback белый фон", _BASEMAP_TIMEOUT_S
)
return False
except Exception as exc: # тайлы недоступны: graceful fallback на белый фон, не валим экспорт
logger.warning("report_maps: OSM basemap недоступен (%s) — fallback белый фон", exc)
return False
# ── Общие хелперы фигуры ───────────────────────────────────────────────────────
@contextmanager
def _managed_figure() -> Iterator[tuple[Figure, Any]]:
"""Контекст-менеджер фигуры: создать axes, ГАРАНТИРОВАННО закрыть в finally.
pyplot держит созданные фигуры в глобальном реестре (``plt.get_fignums()``) до
явного ``plt.close``. В долгоживущем Celery-воркере любое исключение между
созданием фигуры и ``savefig`` без close оставляло бы residual-окно растущий
RSS (утечка на 1 фигуру за каждый упавший рендер). ``finally`` закрывает фигуру
на ЛЮБОМ пути выхода успех, per-geometry-ошибка, неожиданный raise.
"""
fig = plt.figure(figsize=(_FIG_INCHES, _FIG_INCHES))
try:
ax = fig.add_subplot(1, 1, 1)
ax.set_axis_off()
ax.set_aspect("equal")
yield fig, ax
finally:
plt.close(fig)
def _apply_bbox(ax: Any, parcel_merc: Any) -> None:
"""Выставить границы axes по bbox участка + буфер (``_BBOX_PAD_FRAC``)."""
minx, miny, maxx, maxy = parcel_merc.total_bounds
span = max(maxx - minx, maxy - miny) or 1.0
pad = span * _BBOX_PAD_FRAC
ax.set_xlim(minx - pad, maxx + pad)
ax.set_ylim(miny - pad, maxy + pad)
def _figure_to_png(fig: Figure, max_px: int) -> bytes:
"""Фигуру → PNG-bytes с dpi, подобранным так, чтобы длинная сторона ≤ ``max_px``.
Закрытие фигуры НЕ здесь: им владеет ``_managed_figure`` (finally покрывает и
путь, где ``savefig`` сам кинул исключение).
"""
dpi = max(72.0, min(300.0, float(max_px) / _FIG_INCHES))
buf = io.BytesIO()
fig.savefig(buf, format="png", dpi=dpi, bbox_inches="tight", pad_inches=0.05)
return buf.getvalue()
# ── Публичный API ──────────────────────────────────────────────────────────────
def render_parcel_map_png(analyze_result: dict[str, Any], *, max_px: int = 1600) -> bytes | None:
"""Карта участка + слои НСПД + конкуренты → PNG-bytes. Нет ``geom_geojson`` → None.
Args:
analyze_result: persist-payload ``analysis_runs.result`` (schema analyze-1.0).
Читаемые ключи с геометрией: ``geom_geojson`` (обязателен), ``nspd_risk_zones``,
``nspd_red_lines``, ``nspd_opportunity_parcels`` (WKT ``geom_wkt``),
``nspd_zouit_overlaps`` (``geometry_geojson``), ``competitors`` (``lat``/``lng``).
max_px: верхняя граница длинной стороны PNG в пикселях.
Returns:
PNG-bytes, либо None если геометрии участка нет (нечего рисовать).
"""
parcel = _geom_from_geojson(analyze_result.get("geom_geojson"))
if parcel is None:
return None
with _managed_figure() as (fig, ax):
# Участок — базовый слой, задаёт bbox.
parcel_merc = _to_mercator([parcel])
parcel_merc.plot(ax=ax, **_PARCEL_STYLE)
_apply_bbox(ax, parcel_merc)
# Риск-зоны (WKT).
_plot_wkt_layer(ax, analyze_result.get("nspd_risk_zones"), _RISK_STYLE)
# ЗОУИТ-пересечения (GeoJSON geometry_geojson).
_plot_geojson_layer(ax, analyze_result.get("nspd_zouit_overlaps"), _ZOUIT_STYLE)
# Opportunity-участки (WKT).
_plot_wkt_layer(ax, analyze_result.get("nspd_opportunity_parcels"), _OPPORTUNITY_STYLE)
# Красные линии (WKT, линии — без заливки: _REDLINE_STYLE несёт color, не facecolor).
_plot_wkt_layer(ax, analyze_result.get("nspd_red_lines"), _REDLINE_STYLE)
# Конкуренты — точки lat/lng.
_plot_competitors(ax, analyze_result.get("competitors"))
_add_basemap(ax) # graceful: False → просто белый фон, PNG всё равно вернётся
return _figure_to_png(fig, max_px)
def render_concept_footprint_png(
concept_variant: dict[str, Any],
parcel_geojson: dict[str, Any] | None,
*,
max_px: int = 1600,
) -> bytes | None:
"""Карта концепции: контур участка + пятна корпусов с подписью этажности → PNG-bytes.
Args:
concept_variant: dict-представление ``ConceptVariant`` с ключом ``buildings_geojson``
(FeatureCollection, ``properties.floors`` / ``footprint_sqm`` / ``section_id``).
parcel_geojson: GeoJSON-геометрия участка (контур-подложка). None рисуем только
корпуса (bbox по ним).
max_px: верхняя граница длинной стороны PNG в пикселях.
Returns:
PNG-bytes, либо None если нет ни участка, ни корпусов (нечего рисовать).
"""
fc = concept_variant.get("buildings_geojson") if isinstance(concept_variant, dict) else None
features = fc.get("features", []) if isinstance(fc, dict) else []
footprints: list[BaseGeometry] = []
floors_labels: list[int | None] = []
for feat in features:
if not isinstance(feat, dict):
continue
geom = _geom_from_geojson(feat.get("geometry"))
if geom is None:
continue
footprints.append(geom)
props = feat.get("properties") or {}
raw_floors = props.get("floors") if isinstance(props, dict) else None
floors_labels.append(int(raw_floors) if isinstance(raw_floors, int | float) else None)
parcel = _geom_from_geojson(parcel_geojson)
if parcel is None and not footprints:
return None
with _managed_figure() as (fig, ax):
# bbox: по участку если есть, иначе по совокупности footprint'ов.
bbox_source = _to_mercator([parcel]) if parcel is not None else _to_mercator(footprints)
if parcel is not None:
# Участок — только контур (без заливки), корпуса рисуются поверх.
_to_mercator([parcel]).plot(ax=ax, facecolor="none", edgecolor="#1d4ed8", linewidth=1.8)
if footprints:
fp_merc = _to_mercator(footprints)
fp_merc.plot(ax=ax, **_CONCEPT_FOOTPRINT_STYLE)
# Подпись этажности в центроиде каждого корпуса.
for geom_merc, floors in zip(fp_merc.geometry, floors_labels, strict=True):
if floors is None:
continue
c = geom_merc.centroid
ax.annotate(
f"{floors} эт.",
(c.x, c.y),
ha="center",
va="center",
fontsize=7.0,
color="#0f172a",
)
_apply_bbox(ax, bbox_source)
_add_basemap(ax)
return _figure_to_png(fig, max_px)
def embed_map_png(html: str, placeholder: str, png: bytes | None) -> str:
"""Заменить строку-``placeholder`` в HTML на ``<img>`` с data-URI PNG.
``png=None`` плашка «карта недоступна» (тот же класс ``.map-placeholder``, что и
в full_report_html стиль уже есть в шаблоне).
"""
if png is None:
return html.replace(placeholder, _MAP_UNAVAILABLE_HTML)
b64 = base64.b64encode(png).decode("ascii")
img = (
f'<img class="report-map" src="data:image/png;base64,{b64}" '
'style="width:100%;max-width:100%;height:auto;display:block;margin:8pt 0;'
'border:1px solid #e5e7eb;border-radius:6pt;" alt="Карта участка" />'
)
return html.replace(placeholder, img)
# ── Приватные слой-плоттеры ────────────────────────────────────────────────────
def _plot_wkt_layer(ax: Any, items: Any, style: dict[str, Any]) -> None:
"""Отрисовать слой из списка dict'ов с ключом ``geom_wkt`` (EPSG:4326).
Заливка/линия задаётся ``style`` (наличие ``facecolor`` полигон, ``color`` без
``facecolor`` линия) geopandas ``plot`` разбирает это сам по типу геометрии.
"""
geoms = _collect_wkt_geoms(items)
if not geoms:
return
_to_mercator(geoms).plot(ax=ax, **style)
def _plot_geojson_layer(ax: Any, items: Any, style: dict[str, Any]) -> None:
"""Отрисовать слой из списка dict'ов с ключом ``geometry_geojson`` (EPSG:4326)."""
if not isinstance(items, list):
return
geoms = [
g
for it in items
if isinstance(it, dict)
and (g := _geom_from_geojson(it.get("geometry_geojson"))) is not None
]
if not geoms:
return
_to_mercator(geoms).plot(ax=ax, **style)
def _collect_wkt_geoms(items: Any) -> list[BaseGeometry]:
"""Собрать shapely-геометрии из списка dict'ов по ключу ``geom_wkt``."""
if not isinstance(items, list):
return []
out: list[BaseGeometry] = []
for it in items:
if not isinstance(it, dict):
continue
geom = _geom_from_wkt(it.get("geom_wkt"))
if geom is not None:
out.append(geom)
return out
def _plot_competitors(ax: Any, competitors: Any) -> None:
"""Отрисовать конкурентов точками из ``lat``/``lng`` (EPSG:4326).
Внимание: у ``Competitor`` долгота в ключе ``lng`` (не ``lon``) см. schemas/parcel.py.
"""
if not isinstance(competitors, list):
return
pts: list[BaseGeometry] = []
for c in competitors:
if not isinstance(c, dict):
continue
lat, lng = c.get("lat"), c.get("lng")
if isinstance(lat, int | float) and isinstance(lng, int | float):
pts.append(Point(float(lng), float(lat)))
if not pts:
return
_to_mercator(pts).plot(ax=ax, **_COMPETITOR_STYLE)

View file

@ -32,7 +32,9 @@ from typing import Any
logger = logging.getLogger(__name__)
# ── Named-константы: заголовки секций (по одной на содержательную секцию §13) ──
# Тот же набор из шести содержательных секций, что рисует excel.py (#991).
# Шесть секций зеркалят excel.py (#991); §13.7 «Уверенность» — доп. секция, портирована
# из report_docx/report_md для parity между экспортёрами (audit epic #2445 item C2;
# excel.py её пока не несёт — отдельный gap, вне scope этой правки).
_TITLE_DOC: str = "Site Finder v2 — советующий отчёт §13"
_TITLE_SUMMARY: str = "Сводка"
@ -41,6 +43,7 @@ _TITLE_FUTURE_MARKET: str = "Будущий рынок"
_TITLE_PRODUCT_TZ: str = "Продукт ТЗ"
_TITLE_SCENARIOS: str = "Сценарии"
_TITLE_SCORING: str = "Скоринг"
_TITLE_CONFIDENCE: str = "Уверенность"
# ── Named-константы: микрокопия / плейсхолдеры (зеркало excel.py) ──────────────
@ -308,15 +311,21 @@ def _future_supply_pairs(future_supply: Any) -> dict[str, Any]:
def _build_summary(report: dict[str, Any]) -> str:
"""Блок «Сводка»: cover + ADVISORY-маркер + вердикт + ключевые числа + контекст."""
"""Блок «Сводка»: cover + ADVISORY-маркер + вердикт + ключевые числа + контекст.
Уровень уверенности здесь только сводный badge (`overall_confidence`), зеркало
report_docx/report_md._build_summary. Полный разбор (rationale + факторы-драйверы)
живёт в отдельной секции §13.7 «Уверенность» (`_build_confidence`) раньше
(до parity-фикса #2445 C2) он дублировался здесь тонкой 2-строчной таблицей, что
расходилось с docx/md и не переживало dict-значный фактор; убрано, чтобы не было
двух версий одних и тех же данных в одном документе.
"""
exec_summary = _as_dict(report.get("exec_summary"))
meta = _as_dict(report.get("meta"))
confidence = _as_dict(report.get("confidence"))
headline = exec_summary.get("headline")
verdict = exec_summary.get("verdict")
key_numbers = _as_dict(exec_summary.get("key_numbers"))
factors = _as_dict(confidence.get("factors"))
cad = _esc(meta.get("cad_num"))
district = _esc(meta.get("district"))
@ -329,10 +338,6 @@ def _build_summary(report: dict[str, Any]) -> str:
("Сформировано", meta.get("generated_at")),
("Версия схемы", meta.get("schema_version")),
]
confidence_pairs: list[tuple[str, Any]] = [
("Уровень", _level_ru(confidence.get("level"))),
("Обоснование", confidence.get("rationale")),
]
overall_conf = _esc(_level_ru(exec_summary.get("overall_confidence")))
return f"""
@ -351,12 +356,6 @@ def _build_summary(report: dict[str, Any]) -> str:
<h3>Ключевые числа</h3>
{_dict_kv_table(key_numbers)}
<h3>Уверенность отчёта</h3>
{_kv_table(confidence_pairs)}
<h3>Факторы уверенности</h3>
{_dict_kv_table(factors)}
<h3>Контекст</h3>
{_kv_table(context_pairs)}
</div>
@ -624,6 +623,43 @@ def _build_scoring(report: dict[str, Any]) -> str:
<h3>Специальные индексы</h3>
{_data_table(["Индекс", "Значение", "Метка"], index_rows)}
</div>
"""
def _build_confidence(report: dict[str, Any]) -> str:
"""§13.7 «Уверенность»: уровень + обоснование + факторы-драйверы (таблица).
Parity fix (audit epic #2445 item C2): report_docx/report_md уже несли эту секцию
(§22.7/§13.7) PDF был единственным экспортёром без нее. Портировано 1-в-1 (та же
4-колоночная таблица «Фактор/Значение/Уровень/Комментарий»), но через HTML-примитивы
report_pdf (`_data_table`), а не python-docx/Markdown API.
"""
confidence = _as_dict(report.get("confidence"))
level = _level_ru(confidence.get("level"))
rationale = confidence.get("rationale")
factors = _as_dict(confidence.get("factors"))
# Факторы #990: {name: {value, level, note}} ИЛИ плоское {name: value}. Defensive:
# если значение — dict, раскладываем на value/level/note; иначе кладём как есть
# (зеркало report_docx._build_confidence / report_md._build_confidence).
factor_rows: list[list[Any]] = []
for name, payload in factors.items():
if isinstance(payload, dict):
factor_rows.append(
[name, payload.get("value"), _level_ru(payload.get("level")), payload.get("note")]
)
else:
factor_rows.append([name, payload, _DASH, _DASH])
return f"""
<div class="section" id="confidence">
<h2>{html.escape(_TITLE_CONFIDENCE)}</h2>
<span class="badge">Уровень: {_esc(level)}</span>
<p class="verdict">{_esc(rationale)}</p>
<h3>Факторы уверенности</h3>
{_data_table(["Фактор", "Значение", "Уровень", "Комментарий"], factor_rows)}
<div class="footer">{html.escape(_FOOTER_NOTE)}</div>
</div>
@ -631,7 +667,8 @@ def _build_scoring(report: dict[str, Any]) -> str:
# Реестр построителей секций. Порядок = порядок блоков в документе (зеркало
# `_SHEET_BUILDERS` у excel.py — тот же набор из шести содержательных секций §13).
# `_SHEET_BUILDERS` у excel.py + report_docx/report_md — Сводка → Рынок сейчас →
# Будущий рынок → Продукт ТЗ → Сценарии → Скоринг → Уверенность §13.7, #2445 C2).
_SECTION_BUILDERS: tuple[Any, ...] = (
_build_summary,
_build_market_now,
@ -639,11 +676,12 @@ _SECTION_BUILDERS: tuple[Any, ...] = (
_build_product_tz,
_build_scenarios,
_build_scoring,
_build_confidence,
)
def _build_html(report: dict[str, Any]) -> str:
"""Склеить HTML-документ из шести секций §13. PURE (только строки). Graceful."""
"""Склеить HTML-документ из семи секций §13 (+ §13.7 Уверенность). PURE. Graceful."""
sections = "".join(builder(report) for builder in _SECTION_BUILDERS)
return f"""<!DOCTYPE html>
<html lang="ru">
@ -666,10 +704,10 @@ def export_report_pdf(report: Any) -> bytes:
"""§13 Отрендерить `SiteFinderReport` (#987) в PDF-документ и вернуть БАЙТЫ.
По одному блоку на содержательную секцию §13 (Сводка / Рынок сейчас / Будущий
рынок / Продукт ТЗ / Сценарии / Скоринг тот же набор, что и `export_report_xlsx`).
Шапки таблиц с заливкой, RU-метки, числа округлены, None "". На блоке «Сводка»
заметный ADVISORY-маркер (отчёт советующий). ВСЕ динамические строки экранируются
`html.escape`.
рынок / Продукт ТЗ / Сценарии / Скоринг / Уверенность §13.7 parity с
report_docx/report_md, #2445 C2). Шапки таблиц с заливкой, RU-метки, числа
округлены, None "". На блоке «Сводка» заметный ADVISORY-маркер (отчёт
советующий). ВСЕ динамические строки экранируются `html.escape`.
ДЕТЕРМИНИРОВАННО, БЕЗ LLM/БД/сети. Принимает КАК `SiteFinderReport`-инстанс, ТАК и
его `as_dict()`-словарь (нормализуется через `_normalize`). GRACEFUL: частичный/

View file

@ -371,12 +371,20 @@ def _month_grid(start: date, end: date) -> list[date]:
def _query_key_rate_monthly(db: Session, *, since: date) -> dict[date, float]:
"""Ресэмпл дневного key_rate (region 'rf') → {month1st: value}. Graceful → {}."""
"""Ресэмпл дневного key_rate (region 'rf') → {month1st: value}. Graceful → {}.
SAVEPOINT (#2464 cluster A finding #2): `db` — общая §22-сессия отчёта; при сбое
этого запроса БЕЗ SAVEPOINT транзакция Postgres остаётся aborted и следующие
запросы get_monthly_macro (inflation, mortgage) + все ПОСЛЕДУЮЩИЕ §9.x-слои на
той же сессии тоже падают. `with db.begin_nested():` откатывает ТОЛЬКО этот
SAVEPOINT (ROLLBACK TO SAVEPOINT), внешняя транзакция остаётся рабочей.
"""
try:
rows = db.execute(
_KEY_RATE_MONTHLY_SQL,
{"itype": "key_rate", "region": "rf", "since": since},
).all()
with db.begin_nested():
rows = db.execute(
_KEY_RATE_MONTHLY_SQL,
{"itype": "key_rate", "region": "rf", "since": since},
).all()
except Exception:
logger.exception("get_monthly_macro: key_rate query failed")
return {}
@ -389,9 +397,14 @@ def _query_inflation_monthly(db: Session, *, since: date) -> dict[date, float]:
Ряд УЖЕ месячный (obs_date = 1-е число, залит cbr_macro_sync) берём как есть
через reuse get_macro_series (свой SQL не пишем). _month_start страховка.
Сбой/пустой ряд {} (НЕ crash), inflation_yoy тогда None по всей сетке.
SAVEPOINT (#2464 cluster A finding #2): см. `_query_key_rate_monthly` — та же
общая §22-сессия, тот же риск отравления транзакции для последующих запросов
(mortgage-поля + §9.x-слои). `with db.begin_nested():` изолирует сбой в SAVEPOINT.
"""
try:
series = get_macro_series(db, "inflation_yoy", region="rf", since=since)
with db.begin_nested():
series = get_macro_series(db, "inflation_yoy", region="rf", since=since)
except Exception:
logger.exception("get_monthly_macro: inflation_yoy query failed")
return {}
@ -404,11 +417,20 @@ def _query_mortgage_monthly(db: Session, *, since: date) -> dict[str, dict[date,
Возвращает {field: {month1st: value}}. obs_date уже нормализован к 1-му числу
в backfill, но _month_start применяем повторно (страховка). Сбой одного ряда
не валит остальные (graceful: пустой подсловарь).
SAVEPOINT (#2464 cluster A finding #2): все 5 полей читаются на ОДНОЙ `db`-Session
(get_monthly_macro вызывается внутри общей §22-сессии отчёта). Без SAVEPOINT сбой
ОДНОГО поля оставляет транзакцию Postgres aborted каждое СЛЕДУЮЩЕЕ поле в этом
же цикле тоже падает (хотя его данные были бы доступны), а `except` здесь молча
отдаёт [] по каждому, маскируя каскад под «нормальную» построчную деградацию.
`with db.begin_nested():` SAVEPOINT на КАЖДОЕ поле: сбой откатывает только его
SAVEPOINT (ROLLBACK TO SAVEPOINT), сессия остаётся рабочей для следующего поля.
"""
out: dict[str, dict[date, float]] = {}
for indicator_type, field in _MORTGAGE_FIELDS:
try:
series = get_macro_series(db, indicator_type, region="sverdl", since=since)
with db.begin_nested():
series = get_macro_series(db, indicator_type, region="sverdl", since=since)
except Exception:
logger.exception("get_monthly_macro: mortgage series %s failed", indicator_type)
series = []

View file

@ -124,7 +124,7 @@ def _primary_horizon(horizons: Sequence[int]) -> int:
return horizons[0] if horizons else _PREFERRED_PRIMARY_HORIZON
def _safe_call(label: str, fn: Any) -> Any:
def _safe_call(label: str, db: Session, fn: Any) -> Any:
"""Вызвать §9.x-сервис graceful: сбой → None + logger.exception (не crash отчёта).
Зеркало product_scoring._safe_call: любой §9.x-слой может бросить (тонкие данные / нет
@ -133,15 +133,29 @@ def _safe_call(label: str, fn: Any) -> Any:
широкий Exception (изоляция одного слоя от отчёта) с ОБЯЗАТЕЛЬНЫМ logger.exception
НЕ молчаливое глотание. §9.x уже graceful внутри; это belt-and-suspenders на шве.
SAVEPOINT (#2464 cluster A finding #1): все §9.x-слои шарят ОДИН `db`-Session на
отчёт (module docstring `forecast_request_cache.py`). Без обёртки сбойный
`db.execute` внутри слоя оставляет транзакцию Postgres в состоянии aborted
(«current transaction is aborted, commands ignored until end of transaction
block») КАЖДЫЙ последующий слой на той же сессии тоже падает, хотя его данные
были бы доступны. `with db.begin_nested():` заводит SAVEPOINT НА ВЕСЬ вызов слоя
(слой может делать несколько `db.execute` внутри себя например §9.6 внутри
§9.8/§11); при исключении SAVEPOINT откатывается автоматически (ROLLBACK TO
SAVEPOINT), внешняя транзакция остаётся рабочей для следующего слоя. НЕ
`db.rollback()` тот откатил бы ВСЮ внешнюю транзакцию (см. `backend.md` §
SAVEPOINT pattern, established anti-pattern).
Args:
label: имя слоя для лога (диагностика какой §9.x-вызов деградировал).
db: общая §22-сессия (для SAVEPOINT вокруг вызова слоя).
fn: нулевой-аргумент thunk вокруг §9.x-вызова.
Returns:
Результат `fn()` или None при исключении.
"""
try:
return fn()
with db.begin_nested():
return fn()
except Exception:
logger.exception("orchestrator: §9.x layer %s failed → section degraded", label)
return None
@ -334,21 +348,25 @@ def _build_site_finder_report_impl(
# ── 2. §9.x-слои, каждый graceful через _safe_call (ГЕТЕРОГЕННЫЕ сигнатуры) ──
market_metrics = _safe_call(
"market_metrics",
db,
lambda: compute_market_metrics(db, district=district, premise_kind=_PREMISE_KIND),
)
supply_rows = _safe_call(
"supply_layers",
db,
lambda: compute_all_layers(db, district=district, premise_kind=_PREMISE_KIND),
)
supply_layers = _summarize_supply_layers(supply_rows)
future_supply = _safe_call(
"future_supply",
db,
lambda: compute_future_supply_pressure(
db, district=district, horizon_months=primary, premise_kind=_PREMISE_KIND
),
)
forecasts = _safe_call(
"demand_supply_forecast",
db,
lambda: compute_demand_supply_forecast(
db, spec=spec, district=district, cad_num=cad_num, horizons=horizon_list
),
@ -359,6 +377,7 @@ def _build_site_finder_report_impl(
# _safe_call оборачивает любой сбой → None → штатно деградируем (collapse=False).
scenarios_result = _safe_call(
"scenarios",
db,
lambda: compute_scenarios(
db, spec=spec, district=district, cad_num=cad_num, horizons=horizon_list
),
@ -371,18 +390,21 @@ def _build_site_finder_report_impl(
scenarios, scenarios_collapsed, scenarios_collapse_reason = scenarios_result
product_scores = _safe_call(
"score_card",
db,
lambda: compute_score_card(
db, spec=spec, district=district, cad_num=cad_num, horizon_months=primary
),
)
special_indices = _safe_call(
"special_indices",
db,
lambda: compute_special_indices(
db, spec=spec, district=district, cad_num=cad_num, horizons=horizon_list
),
)
recommendation_overlay = _safe_call(
"forecast_overlay",
db,
lambda: build_forecast_overlay(
db,
district=district,
@ -393,7 +415,7 @@ def _build_site_finder_report_impl(
)
# ── Макро-свежесть (audit MEDIUM): только лог; проводка в отчёт — 3b ─────────
macro = _safe_call("monthly_macro", lambda: get_monthly_macro(db))
macro = _safe_call("monthly_macro", db, lambda: get_monthly_macro(db))
macro_as_of = _macro_as_of(macro)
if macro_as_of is not None:
logger.info(

View file

@ -835,13 +835,15 @@ def _poi_weight_sum(db: Session, *, cad_num: str) -> float | None:
compute_poi_weighted_top7. Нет геометрии / нет POI / сбой None (infra_fit unavailable).
"""
try:
coords = (
db.execute(
_PARCEL_CENTROID_SQL, {"cad_num": cad_num, "quarter": _quarter_from_cad(cad_num)}
with db.begin_nested():
coords = (
db.execute(
_PARCEL_CENTROID_SQL,
{"cad_num": cad_num, "quarter": _quarter_from_cad(cad_num)},
)
.mappings()
.first()
)
.mappings()
.first()
)
except Exception:
logger.exception(
"product_scoring: centroid lookup failed (cad_num=%s) → infra n/a", cad_num
@ -850,9 +852,10 @@ def _poi_weight_sum(db: Session, *, cad_num: str) -> float | None:
if not coords or coords.get("lat") is None or coords.get("lon") is None:
return None
try:
response = compute_poi_weighted_top7(
db, cad_num, float(coords["lat"]), float(coords["lon"]), radius_m=_POI_RADIUS_M
)
with db.begin_nested():
response = compute_poi_weighted_top7(
db, cad_num, float(coords["lat"]), float(coords["lon"]), radius_m=_POI_RADIUS_M
)
except Exception:
logger.exception("product_scoring: poi_score failed (cad_num=%s) → infra n/a", cad_num)
return None

View file

@ -152,17 +152,31 @@ def _as_dict_list(items: Sequence[Any] | None) -> list[dict[str, Any]]:
def _deal_count(analyze: dict[str, Any], market_metrics: dict[str, Any] | None) -> int | None:
"""Число сделок (продаж) за окно — для deal_count #990. PURE.
"""Число сделок (продаж) ЗА ОКНО — для deal_count #990. PURE.
Предпочитаем `market_metrics.n_sold` (§9.2 прямой счётчик проданных лотов выборки);
fallback на `analyze.market_pulse.competitors_total` НЕ берём (это число ЖК, не
сделок пошло бы в analog_count). Нет сигнала None (#990 → тянет в low).
#1964: НЕ берём `market_metrics.n_sold` — это КУМУЛЯТИВНЫЙ all-time проданный сток
(десятки-сотни тысяч), а #990/confidence_engine помечает фактор «за {window} мес»
(6) и гейтит порогами 6-мес окна (high50). n_sold 50 ВСЕГДА гейт бессмыслен и
подпись врёт. Берём оконное число = unit_velocity × window_months, где unit_velocity
(§9.2) = units_sold_window / window_months из _SALES_WINDOW_SQL (он
contract_date-windowed) произведение восстанавливает units_sold за окно. Так
подпись «за 6 мес» честная и порог 50/15 осмысленный (прод EKB: ~24 876 за 6 мес
vs ~380 921 кумулятивно).
unit_velocity отсутствует (None / нет market_metrics) None (#990 трактует как
«нет данных», тянет в low). window_months 0 / нет None (нечем масштабировать).
Fallback на `analyze.market_pulse.competitors_total` НЕ берём (это число ЖК, не
сделок пошло бы в analog_count).
"""
if market_metrics is not None:
n_sold = market_metrics.get("n_sold")
if isinstance(n_sold, int):
return n_sold
return None
if market_metrics is None:
return None
unit_velocity = market_metrics.get("unit_velocity")
window_months = market_metrics.get("window_months")
if not isinstance(unit_velocity, int | float):
return None
if not isinstance(window_months, int) or window_months <= 0:
return None
return round(float(unit_velocity) * window_months)
def _analog_count(analyze: dict[str, Any], market_metrics: dict[str, Any] | None) -> int | None:
@ -239,7 +253,8 @@ def _deal_count_months(market_metrics: dict[str, Any] | None) -> int | None:
"""Окно наблюдения для deal_count (мес) — для deal_count_months #990. PURE.
Читает тот же `market_metrics.window_months` (§9.2), что и `_history_months`
именно за это окно считается n_sold. Нет None (#990 пропускает суффикс «за N мес»).
#1964: именно за это окно `_deal_count` считает сделки (unit_velocity ×
window_months). Нет None (#990 пропускает суффикс «за N мес»).
"""
if market_metrics is not None:
window = market_metrics.get("window_months")

View file

@ -549,6 +549,12 @@ def _query_source_a(
Graceful {} при сбое/пустых данных. price_bucket в spec для Source A
игнорируется (агрегат не несёт per-lot цены) фиксируется логом.
SAVEPOINT (#2464 cluster A finding #4): `db` — общая §22-сессия отчёта (может
переиспользоваться другими §9.x-слоями после этого вызова). Без SAVEPOINT сбойный
`db.execute` оставляет транзакцию Postgres aborted все ПОСЛЕДУЮЩИЕ запросы на
той же сессии тоже падают. `with db.begin_nested():` откатывает ТОЛЬКО этот
SAVEPOINT при сбое, оставляя внешнюю транзакцию рабочей.
"""
if spec.price_bucket is not None:
logger.info(
@ -567,7 +573,8 @@ def _query_source_a(
"room_bucket": spec.room_bucket,
}
try:
rows = db.execute(_SOURCE_A_SQL, params).mappings().all()
with db.begin_nested():
rows = db.execute(_SOURCE_A_SQL, params).mappings().all()
except Exception:
logger.exception("build_sales_series: source A query failed")
return {}
@ -582,6 +589,9 @@ def _query_source_b(
Graceful {} при сбое/пустых данных. Передаёт bucket-пороги/-метки в SQL
(зеркало pure-helpers), чтобы room×area / price сегментация считалась тем же
правилом и в БД, и в Python.
SAVEPOINT (#2464 cluster A finding #4): см. `_query_source_a` — та же общая
§22-сессия, тот же риск отравления транзакции для последующих слоёв/запросов.
"""
# district (админ-имя ЕКБ) → набор informal микро (objective_lots хранит микро).
# None → EKB-wide (без district-фильтра).
@ -613,7 +623,8 @@ def _query_source_b(
"p_unknown": PRICE_BUCKET_UNKNOWN,
}
try:
rows = db.execute(_SOURCE_B_SQL, params).mappings().all()
with db.begin_nested():
rows = db.execute(_SOURCE_B_SQL, params).mappings().all()
except Exception:
logger.exception("build_sales_series: source B query failed")
return {}

View file

@ -828,22 +828,62 @@ def _unavailable(key: str, *, reason: str) -> SpecialIndex:
# `market_metrics._SALES_WINDOW_SQL`: bind `:has_district` (boolean) переключает
# ветку, `:districts` (text[]) — список микро. «Продан» = is_sold IS TRUE ИЛИ
# contract_date заполнена. psycopg v3: CAST, НИКОГДА :x::type.
# #1964: считаем по ФИЗИЧЕСКИМ лотам (дедуп) — сырой objective_lots раздут ~2.91×
# (мульти lot_id на физлот); доля ипотеки — отношение, к дублям устойчива, но
# n_sold/n_mortgage теперь честные (по физлотам).
#
# Перф-батч #2 (#1953-followup): дедупим INLINE через DISTINCT ON (physflat-ключ,
# последний снапшот), НЕ через v_objective_lots_latest. Причина: view (DISTINCT ON ol.*)
# — оптимизационный барьер: WHERE снаружи НЕ проталкивается ниже DISTINCT ON →
# планировщик материализует ВСЮ таблицу (1.76M, Parallel Seq Scan + external merge sort
# 66-81 МБ/воркер) ПЕРЕД фильтром (~5.4-6.4 s district-scoped, прод-EXPLAIN). Inline:
# bitmap по objective_lots_district_class_idx + узкий sort (~3-5 МБ/воркер) ~0.37 s
# (≈15×).
#
# ⚠️ КОРРЕКТНОСТЬ дедупа (слои фильтров РАЗНЫЕ — не путать):
# • premise_kind — ПОЛНОСТЬЮ стабилен (0 mixed-групп на проде) → в WHERE ДО DISTINCT ON.
# • district — почти стабилен (4 transient-группы на 605k физлотов). Протолкнут ДО
# DISTINCT ON ОСОЗНАННО: это index-драйвер objective_lots_district_class_idx; увод
# фильтра ПОСЛЕ дедупа = full seq-scan, убивает перф-фикс. Принятый компромисс
# паттерна parcels.py/#1964 (≤4 возможных дрейфа при district-scoped запросе).
# • class — МЕНЯЮЩИЙСЯ (248 transient-групп на проде): класс физлота дрейфует между
# снапшотами. Фильтр lower(class)=… ДО DISTINCT ON ловил бы СТАРЫЙ снапшот под
# старым классом (прод-замер: бизнес 16637→16657 +20, стандарт 128721→128914 +193
# vs view). View фильтрует class на ПОСЛЕДНЕМ снапшоте → class-фильтр применяем
# ПОСЛЕ дедупа (в sold-CTE). Тогда эквивалентность view бит-в-бит по ВСЕМ классам.
# • sold-фильтр (is_sold / contract_date) — тоже МЕНЯЮЩИЙСЯ: ПОСЛЕ дедупа (на
# ПОСЛЕДНЕМ снапшоте), как делал view. Протолкнуть ДО DISTINCT ON СЛОМАЛО БЫ
# семантику (включил бы когда-либо-проданные + дедуп среди sold-снапшотов).
_ARTIFICIAL_DEMAND_SQL = text(
"""
WITH sold AS (
SELECT
ol.objective_lot_id,
(ol.encumbrance_type IS NOT NULL OR ol.bank_name IS NOT NULL) AS has_mortgage
FROM objective_lots ol
WHERE ol.premise_kind = CAST(:premise_kind AS text)
AND (ol.is_sold IS TRUE OR ol.contract_date IS NOT NULL)
WITH latest AS (
SELECT DISTINCT ON (
project_name, corpus_name, section, floor, lot_number
)
objective_lot_id,
is_sold,
contract_date,
class,
encumbrance_type,
bank_name
FROM objective_lots
WHERE premise_kind = CAST(:premise_kind AS text)
AND (
CAST(:has_district AS boolean) IS FALSE
OR ol.district = ANY(CAST(:districts AS text[]))
OR district = ANY(CAST(:districts AS text[]))
)
ORDER BY project_name, corpus_name, section, floor,
lot_number, snapshot_date DESC, id DESC
),
sold AS (
SELECT
objective_lot_id,
(encumbrance_type IS NOT NULL OR bank_name IS NOT NULL) AS has_mortgage
FROM latest
WHERE (is_sold IS TRUE OR contract_date IS NOT NULL)
AND (
CAST(:obj_class AS text) IS NULL
OR lower(ol.class) = lower(CAST(:obj_class AS text))
OR lower(class) = lower(CAST(:obj_class AS text))
)
)
SELECT
@ -954,16 +994,29 @@ def _query_parcel_centroid(db: Session, *, cad_num: str) -> tuple[float, float]
Нет геометрии / сбой None (гео-веса упадут на floor; overlap считается по остальным
осям, индекс НЕ деградирует целиком). Параметризовано (psycopg v3). Детерминированно.
SAVEPOINT НА ВНУТРЕННЕМ swallow-сайте (#2464 cluster A, RELEASE-trap): этот helper
зовётся из `_build_cannibalization` builder, обёрнутый внешним SAVEPOINT в
`compute_special_indices._run`. БЕЗ собственного `db.begin_nested():` сбойный
`db.execute` тут проглатывается локально ( None), оставляя транзакцию Postgres
aborted; тогда ВНЕШНИЙ `_run`-savepoint выходит «нормально» и делает RELEASE
SAVEPOINT, а RELEASE в aborted-tx САМ падает (Postgres в aborted допускает только
ROLLBACK / ROLLBACK TO SAVEPOINT) внешний except ловит уже RELEASE-ошибку, но
транзакция так и не откачена отравление каскадит в следующий §25-индекс. Savepoint
ИМЕННО ЗДЕСЬ (в точке перехвата) откатывает сбой (ROLLBACK TO SAVEPOINT), поэтому
внешний RELEASE проходит. Тот же принцип, что saturation.py: savepoint у db.execute
в точке, где ошибка ловится.
"""
try:
row = (
db.execute(
_PARCEL_CENTROID_SQL,
{"cad_num": cad_num, "quarter": _quarter_from_cad(cad_num)},
with db.begin_nested():
row = (
db.execute(
_PARCEL_CENTROID_SQL,
{"cad_num": cad_num, "quarter": _quarter_from_cad(cad_num)},
)
.mappings()
.first()
)
.mappings()
.first()
)
except Exception:
logger.exception("cannibalization: centroid query failed for cad_num=%s", cad_num)
return None
@ -1617,9 +1670,20 @@ def compute_special_indices(
segment = spec.as_dict()
def _run(key: str, builder: Any) -> SpecialIndex:
"""Выполнить builder одного индекса в собственном try/except (graceful)."""
"""Выполнить builder одного индекса в собственном try/except (graceful).
SAVEPOINT (#2464 cluster A finding #3): все шесть builder'ов делят ОДНУ `db`-
Session (общая сессия §22-отчёта). Без SAVEPOINT сбойный `db.execute` внутри
builder'а оставляет транзакцию Postgres aborted — КАЖДЫЙ следующий индекс на
той же сессии тоже падает (хотя его данные были бы доступны), что маскируется
под шесть независимых деградаций. `with db.begin_nested():` заводит SAVEPOINT
на ВЕСЬ builder() (некоторые builder'ы делают несколько db.execute внутри
себя) сбой откатывает только его SAVEPOINT, сессия остаётся рабочей для
следующего индекса.
"""
try:
return builder() # type: ignore[no-any-return]
with db.begin_nested():
return builder() # type: ignore[no-any-return]
except Exception:
# Сбой одного индекса НЕ валит карточку: деградация-None, остальные считаются.
logger.exception(

View file

@ -0,0 +1,141 @@
"""Generative Design — Stage 3a (#1965): каталог типовых домов (house-type catalog).
Контракт Stage 3a: вместо max-FAR жадной раскладки (Stage 1b) пользователь выбирает
ТИПОВЫЕ дома и говорит, сколько секций каждого типа поставить. Этот модуль справочник
таких типов: для каждого ``section_type`` он несёт габариты пятна секции (ширина × глубина,
метры), дефолтную этажность и подходящий класс жилья.
ИСТОЧНИК: это РАЗУМНЫЙ ДЕФОЛТНЫЙ каталог (sane-default), а не выгрузка из БД. Габариты
типовые размеры секций массового жилья РФ (панель/монолит/башня/малоэтажка), округлённые
до реалистичных значений. Каталог намеренно захардкожен в коде на Stage 3a: миграции БД
сейчас нет (см. эпик #1953). Когда понадобится редактируемый застройщиком каталог — его
ПРОДВИГАЮТ в БД-таблицу с тем же контрактом (``section_type`` footprint/floors/class),
а этот модуль станет seed'ом/фолбэком. До тех пор — single source of truth по типам.
Детерминированно, без LLM / внешних API / БД.
"""
from __future__ import annotations
import logging
from dataclasses import dataclass
from typing import Literal
logger = logging.getLogger(__name__)
# Класс жилья — то же Literal-множество, что у ConceptInput / compute_teap (single
# source of truth по допустимым значениям; рассинхрон тут = ошибка типов в mypy-strict).
HousingClass = Literal["econom", "comfort", "business"]
@dataclass(frozen=True)
class HouseType:
"""Один типовой дом каталога (тип секции МКД / малоэтажки).
section_type стабильный машинный КЛЮЧ типа (латиница, snake_case); это и есть
значение, которое фронт кладёт в ``BuildingProgramItem.section_type``.
label_ru человекочитаемый русский лейбл для UI (Stage 3b его показывает).
footprint_w_m ширина пятна секции, метры.
footprint_d_m глубина пятна секции, метры.
default_floors дефолтная этажность типа (UI подставляет, пользователь может менять
в пределах контракта BuildingProgramItem [1, 40]).
housing_class подходящий класс жилья (драйвит ТЭП/финмодель ниже по конвейеру).
"""
section_type: str
label_ru: str
footprint_w_m: float
footprint_d_m: float
default_floors: int
housing_class: HousingClass
@property
def footprint_sqm(self) -> float:
"""Площадь пятна секции, кв.м (ширина × глубина)."""
return self.footprint_w_m * self.footprint_d_m
# ── Каталог типовых домов (sane-default, см. модульный docstring про источник) ──────
# Покрываем основные форматы массового жилья РФ:
# * панель-эконом — длинная неглубокая секция, средняя этажность, эконом-класс;
# * монолит-комфорт— чуть шире/глубже, комфорт-класс, типовая «свечка» 14 этажей;
# * башня-бизнес — компактное квадратное пятно, высотная (точечная) застройка;
# * малоэтажка-комфорт — широкая невысокая секция (3 этажа), низкоплотная застройка;
# * таунхаус — узкое неглубокое пятно блокированной застройки, 3 этажа.
# Габариты — реалистичные типовые размеры; этажность — характерная для формата.
HOUSE_TYPES: tuple[HouseType, ...] = (
HouseType(
section_type="panel_econom",
label_ru="Панельная секция (эконом)",
footprint_w_m=24.0,
footprint_d_m=15.0,
default_floors=9,
housing_class="econom",
),
HouseType(
section_type="monolith_comfort",
label_ru="Монолитная секция (комфорт)",
footprint_w_m=21.0,
footprint_d_m=18.0,
default_floors=14,
housing_class="comfort",
),
HouseType(
section_type="tower_business",
label_ru="Башня (бизнес)",
footprint_w_m=18.0,
footprint_d_m=18.0,
default_floors=25,
housing_class="business",
),
HouseType(
section_type="lowrise_comfort",
label_ru="Малоэтажная секция (комфорт)",
footprint_w_m=30.0,
footprint_d_m=14.0,
default_floors=3,
housing_class="comfort",
),
HouseType(
section_type="townhouse",
label_ru="Таунхаус",
footprint_w_m=12.0,
footprint_d_m=10.0,
default_floors=3,
housing_class="comfort",
),
)
# Индекс по ключу для O(1)-лукапа. Построен один раз при импорте; ключи уникальны
# (assert ниже ловит дубликат типа на старте, а не молча затирает запись).
_BY_KEY: dict[str, HouseType] = {ht.section_type: ht for ht in HOUSE_TYPES}
assert len(_BY_KEY) == len(HOUSE_TYPES), "duplicate section_type key in HOUSE_TYPES"
def get_house_type(section_type: str) -> HouseType:
"""Найти тип дома по ключу ``section_type``. Бросает :class:`KeyError`, если нет.
Вызывающий слой (placement) обязан валидировать ключи программы заранее (см.
:func:`available_section_types`) неизвестный ключ здесь это программная ошибка,
а не пользовательский ввод, поэтому KeyError, а не тихий None.
"""
try:
return _BY_KEY[section_type]
except KeyError as exc:
raise KeyError(
f"unknown house type {section_type!r}; " f"available: {', '.join(sorted(_BY_KEY))}"
) from exc
def available_section_types() -> frozenset[str]:
"""Множество допустимых ключей ``section_type`` каталога (для валидации программы)."""
return frozenset(_BY_KEY)
__all__ = [
"HOUSE_TYPES",
"HouseType",
"HousingClass",
"available_section_types",
"get_house_type",
]

View file

@ -24,6 +24,11 @@ import logging
# импорт из модуля удовлетворяет strict no-implicit-reexport.
from ezdxf.enums import TextEntityAlignment
from ezdxf.filemanagement import new as ezdxf_new
# Modelspace определён в ezdxf.layouts.layout и не в __all__ пакета ezdxf.layouts;
# импорт из модуля-определителя удовлетворяет strict no-implicit-reexport (как ezdxf_new).
from ezdxf.layouts.layout import Modelspace
from shapely.coords import CoordinateSequence
from shapely.geometry import Polygon
from app.schemas.concept import ConceptVariant
@ -44,7 +49,7 @@ _LAYER_BUILDINGS = "BUILDINGS"
_LABEL_HEIGHT_M = 2.0
def _ring_points(coords: object) -> list[tuple[float, float]]:
def _ring_points(coords: CoordinateSequence) -> list[tuple[float, float]]:
"""Кольцо (exterior/interior) как список (x, y) для LWPolyline (без замыкающей точки)."""
pts = list(coords)
# Shapely дублирует первую точку в конце; close=True у ezdxf замкнёт сам.
@ -53,7 +58,7 @@ def _ring_points(coords: object) -> list[tuple[float, float]]:
return [(float(x), float(y)) for x, y in pts]
def _add_polygon(msp: object, poly: Polygon, layer: str) -> None:
def _add_polygon(msp: Modelspace, poly: Polygon, layer: str) -> None:
"""Нарисовать полигон на слое: внешнее кольцо + каждое внутреннее (отверстие).
LWPolyline не умеет дырки, поэтому каждое interior-кольцо эмитируется отдельной
@ -151,15 +156,15 @@ def _feature_to_metric_polygon(parcel: Parcel, feature: object) -> Polygon | Non
return None
coords = geometry.get("coordinates")
# Shapely mapping() emits nested tuples; accept both tuple and list.
if not isinstance(coords, (list, tuple)) or not coords:
if not isinstance(coords, list | tuple) or not coords:
return None
ring = coords[0]
if not isinstance(ring, (list, tuple)) or len(ring) < 4:
if not isinstance(ring, list | tuple) or len(ring) < 4:
return None
metric_pts: list[tuple[float, float]] = []
for pt in ring:
if not isinstance(pt, (list, tuple)) or len(pt) < 2:
if not isinstance(pt, list | tuple) or len(pt) < 2:
return None
lon, lat = float(pt[0]), float(pt[1])
x, y = parcel.wgs84_to_metric(lon, lat)

View file

@ -66,9 +66,7 @@ def _strategy_label(strategy: str) -> str:
def _teap_table(variants: Sequence[ConceptVariant]) -> str:
"""HTML-таблица ТЭП по всем вариантам (строки — показатели, колонки — стратегии)."""
headers = "".join(
f"<th>{html.escape(_strategy_label(v.strategy))}</th>" for v in variants
)
headers = "".join(f"<th>{html.escape(_strategy_label(v.strategy))}</th>" for v in variants)
rows: list[tuple[str, list[str]]] = [
("Пятно застройки, кв.м", [_fmt_int(v.teap.built_area_sqm) for v in variants]),
("Общая площадь (GFA), кв.м", [_fmt_int(v.teap.total_floor_area_sqm) for v in variants]),
@ -107,9 +105,7 @@ def _fmt_irr(financial: FinancialModel) -> str:
def _financial_table(variants: Sequence[ConceptVariant]) -> str:
"""HTML-таблица финмодели (деньги в млн руб; полный каскад + БДР + DCF)."""
headers = "".join(
f"<th>{html.escape(_strategy_label(v.strategy))}</th>" for v in variants
)
headers = "".join(f"<th>{html.escape(_strategy_label(v.strategy))}</th>" for v in variants)
rows: list[tuple[str, list[str]]] = [
(
"Выручка — жильё, млн руб",
@ -188,9 +184,13 @@ def _build_html(variants: Sequence[ConceptVariant]) -> str:
"точность метрик зависит от реального графика. Где IRR помечен «оценочный» — поток "
"вырожденный (нет смены знака), показан аннуализированный ROI вместо DCF-IRR. "
"НДС — реализация жилья и услуги застройщика по ДДУ освобождены (ст. 149 НК РФ), "
"входной НДС по СМР встроен в себестоимость; НДС начисляется только на нежилую часть "
"(машиноместа). Налог на прибыль — 25% (с 2025). Цены и себестоимость "
"— рыночные ориентиры. Коммерческие/офисные площади не учитываются (нет в ТЭП).</p>"
"входной НДС по СМР встроен в себестоимость; НДС начисляется на нежилые части — "
"машиноместа и коммерцию/офисы 1-го этажа (встроенный НДС в добавленной стоимости "
"каждой части). Налог на прибыль — 25% (с 2025). Цены и себестоимость — рыночные "
"ориентиры. Коммерция/офисы 1-го этажа учтены по нормативной доле от общей площади "
"и продаются с умеренной наценкой к цене жилья того же класса; себестоимость СМР "
"нежилого — по той же ставке, что и жильё (отдельной строки затрат нет, повторного "
"учёта в затратах нет).</p>"
"</body></html>"
)

View file

@ -30,7 +30,8 @@ from dataclasses import dataclass
from typing import Any
from pyproj import CRS, Transformer
from shapely.geometry import Polygon, mapping, shape
from pyproj.exceptions import CRSError
from shapely.geometry import MultiPolygon, Polygon, mapping, shape
from shapely.geometry.base import BaseGeometry
from shapely.ops import transform as shapely_transform
@ -53,6 +54,12 @@ MIN_BUILDABLE_AREA_SQM: float = 50.0
# удержать время в бюджете (<=10 c/вариант). MVP-упрощение.
MAX_GRID_CELLS: int = 20_000
# Валидные диапазоны WGS84 lon/lat (градусы). Используются, чтобы отловить участок,
# присланный в проекции (метры, напр. МСК-66/UTM), ДО того как _metric_transformers
# соберёт AEQD с lat_0/lon_0 далеко за пределами градусов -> pyproj.CRSError.
_LON_RANGE: tuple[float, float] = (-180.0, 180.0)
_LAT_RANGE: tuple[float, float] = (-90.0, 90.0)
# WGS84 (вход контракта).
_WGS84 = CRS.from_epsg(4326)
@ -142,6 +149,19 @@ def _parse_polygon(parcel_geojson: dict[str, Any]) -> Polygon:
except (KeyError, TypeError, ValueError, AttributeError) as exc:
raise ParcelGeometryError(f"cannot parse GeoJSON geometry: {exc}") from exc
# Многоконтурный участок (MultiPolygon): берём крупнейший контур. У некоторых
# КН-участков в Росреестре несколько разрозненных полигонов — концепцию строим
# по основному (наибольшему по площади) контуру; остальные обычно вкрапления.
if isinstance(geom, MultiPolygon):
if geom.is_empty or not geom.geoms:
raise ParcelGeometryError("parcel polygon is empty")
n_contours = len(geom.geoms)
geom = max(geom.geoms, key=lambda g: g.area)
logger.info(
"parse_parcel: MultiPolygon → крупнейший контур из %d",
n_contours,
)
if geom.geom_type != "Polygon":
raise ParcelGeometryError(f"expected Polygon, got {geom.geom_type}")
if geom.is_empty:
@ -157,6 +177,27 @@ def _parse_polygon(parcel_geojson: dict[str, Any]) -> Polygon:
return polygon
def _assert_wgs84_bounds(polygon: Polygon) -> None:
"""Проверить, что bbox полигона лежит в валидных диапазонах WGS84 lon/lat.
Root-cause фикс: фронтовый баг / плохой геокодер / demo-payload иногда шлёт
``parcel_geojson`` в проекции (метры МСК-66/UTM, напр. [500000, 6200000]) вместо
WGS84 lon/lat. Такой полигон синтаксически валиден (это просто Polygon), но
дальше он бы дошёл до :func:`_metric_transformers`, где centroid.y (~6_200_000)
подставится в ``+lat_0=`` -> pyproj.CRS.from_proj4 упадёт CRSError (opaque 500).
Ловим здесь, ДО построения проекции, с понятным сообщением (422).
"""
minx, miny, maxx, maxy = polygon.bounds
lon_ok = _LON_RANGE[0] <= minx and maxx <= _LON_RANGE[1]
lat_ok = _LAT_RANGE[0] <= miny and maxy <= _LAT_RANGE[1]
if not (lon_ok and lat_ok):
raise ParcelGeometryError(
"parcel_geojson coordinates out of WGS84 lon/lat range "
f"(lon∈{_LON_RANGE}, lat∈{_LAT_RANGE}, got bounds={polygon.bounds}) — "
"coordinates appear to be projected (e.g. МСК-66/UTM metres), not WGS84 lon/lat"
)
def _metric_transformers(polygon_wgs84: Polygon) -> tuple[Transformer, Transformer]:
"""Построить пару трансформеров WGS84<->метрический AEQD вокруг центроида участка.
@ -164,10 +205,16 @@ def _metric_transformers(polygon_wgs84: Polygon) -> tuple[Transformer, Transform
координат и точен на масштабе квартала не нужен выбор UTM-зоны.
"""
centroid = polygon_wgs84.centroid
metric_crs = CRS.from_proj4(
f"+proj=aeqd +lat_0={centroid.y} +lon_0={centroid.x} "
"+x_0=0 +y_0=0 +ellps=WGS84 +datum=WGS84 +units=m +no_defs"
)
try:
metric_crs = CRS.from_proj4(
f"+proj=aeqd +lat_0={centroid.y} +lon_0={centroid.x} "
"+x_0=0 +y_0=0 +ellps=WGS84 +datum=WGS84 +units=m +no_defs"
)
except CRSError as exc:
# Belt-and-suspenders: _assert_wgs84_bounds должен был отловить это раньше,
# но на случай иной координатной патологии не даём CRSError утечь наружу
# непойманным 500 — это невалидная геометрия участка, т.е. 422.
raise ParcelGeometryError(f"cannot build metric CRS for parcel centroid: {exc}") from exc
to_metric = Transformer.from_crs(_WGS84, metric_crs, always_xy=True)
to_wgs84 = Transformer.from_crs(metric_crs, _WGS84, always_xy=True)
return to_metric, to_wgs84
@ -240,9 +287,11 @@ def parse_parcel(
"""Stage 1a: ConceptInput -> :class:`Parcel` (метрика + buildable + grid).
Raises:
ParcelGeometryError: полигон невалиден или пятно застройки вырождается.
ParcelGeometryError: полигон невалиден, координаты не в WGS84 lon/lat, или
пятно застройки вырождается.
"""
polygon_wgs84 = _parse_polygon(payload.parcel_geojson)
_assert_wgs84_bounds(polygon_wgs84)
to_metric, to_wgs84 = _metric_transformers(polygon_wgs84)
def _fwd(xs: Any, ys: Any) -> tuple[Any, Any]:

View file

@ -23,13 +23,19 @@ Deterministic, no LLM / no external API / no DB.
from __future__ import annotations
import logging
import math
from dataclasses import dataclass
from shapely.geometry import Polygon, box
from shapely.strtree import STRtree
from app.schemas.concept import ConceptInput, ConceptVariant
from app.services.generative import financial, teap
from app.schemas.concept import (
TEAP,
BuildingProgramItem,
ConceptInput,
ConceptVariant,
)
from app.services.generative import catalog, financial, teap
from app.services.generative.geometry import Parcel, ParcelGeometryError
logger = logging.getLogger(__name__)
@ -92,6 +98,53 @@ def _resolve_floors(target_floors: int, factor: float) -> int:
return max(_FLOORS_MIN, min(_FLOORS_MAX, floors))
class _Placer:
"""Аккумулятор размещённых секций + STRtree-индекс для проверки разрывов.
Извлечён из жадной раскладки, чтобы и :func:`_greedy_place` (coverage-cap sweep), и
:func:`place_program` (фиксированная программа типовых домов, Stage 3a) пользовались
ОДНОЙ И ТОЙ ЖЕ collision/setback-машиной, а не дублировали её. Состояние: принятые
footprints, их буферизованные на ``half_gap`` версии и перестраиваемый по ним STRtree.
"""
def __init__(self) -> None:
self.placed: list[Polygon] = []
self.built_area: float = 0.0
# Буферизованные footprints для проверки разрыва; индекс STRtree по ним.
self._buffered: list[Polygon] = []
self._tree: STRtree | None = None
def try_place(self, footprint: Polygon, buildable: Polygon, half_gap: float) -> bool:
"""Попытаться принять ``footprint``. True — принят, False — не лёг.
Принимается, если целиком внутри ``buildable`` (covers допускает касание границы)
И не нарушает разрыв ``half_gap*2`` с уже принятыми (буферим кандидата на
``half_gap`` и проверяем пересечение с буферизованными соседями через STRtree
две секции с зазором >= gap не пересекутся).
"""
if not buildable.covers(footprint):
return False
candidate_buf = footprint.buffer(half_gap, join_style="mitre")
if self._tree is not None:
for idx in self._tree.query(candidate_buf):
if candidate_buf.intersects(self._buffered[idx]):
return False
self.placed.append(footprint)
self.built_area += footprint.area
self._buffered.append(candidate_buf)
self._tree = STRtree(self._buffered)
return True
def _centered_footprint(cx: float, cy: float, width: float, depth: float) -> Polygon:
"""Прямоугольное пятно секции ``width × depth``, центрированное на (cx, cy), метры."""
half_w = width / 2.0
half_d = depth / 2.0
return box(cx - half_w, cy - half_d, cx + half_w, cy + half_d)
def _greedy_place(
parcel: Parcel,
spec: StrategySpec,
@ -103,63 +156,32 @@ def _greedy_place(
* кандидат-якоря центры ячеек сетки в фиксированном порядке;
* footprint строится центрированно на якоре;
* принимается, если целиком внутри buildable area И не нарушает разрыв ``gap_m``
с уже принятыми (проверка через STRtree по buffered-footprints);
с уже принятыми (проверка через STRtree, см. :class:`_Placer`);
* раскладка останавливается, когда пятно достигает ``coverage_cap`` от buildable
area (регулятор плотности по типу застройки) это также ограничивает число
размещений и держит O(n^2)-перестройку STRtree в бюджете.
"""
buildable = parcel.buildable_m
max_built = buildable.area * coverage_cap
placed: list[Polygon] = []
built_area = 0.0
# Буферизованные footprints для проверки разрыва; индекс STRtree по ним.
buffered: list[Polygon] = []
tree: STRtree | None = None
half_w = spec.section_w / 2.0
half_d = spec.section_d / 2.0
placer = _Placer()
half_gap = spec.gap_m / 2.0
for cell in parcel.grid:
if built_area >= max_built:
if placer.built_area >= max_built:
break
footprint = box(
cell.cx - half_w,
cell.cy - half_d,
cell.cx + half_w,
cell.cy + half_d,
)
# Целиком внутри пятна застройки (covers допускает касание границы).
if not buildable.covers(footprint):
continue
# Разрыв между секциями: буферим кандидата на half_gap и проверяем пересечение
# с буферизованными соседями — две секции с зазором >= gap_m не пересекутся.
candidate_buf = footprint.buffer(half_gap, join_style="mitre")
if tree is not None:
collision = False
for idx in tree.query(candidate_buf):
if candidate_buf.intersects(buffered[idx]):
collision = True
break
if collision:
continue
placed.append(footprint)
built_area += footprint.area
buffered.append(candidate_buf)
tree = STRtree(buffered)
footprint = _centered_footprint(cell.cx, cell.cy, spec.section_w, spec.section_d)
placer.try_place(footprint, buildable, half_gap)
logger.info(
"strategy=%s placed %d sections (%.0fx%.0f m, gap=%.0f m, coverage<=%.0f%%)",
spec.name,
len(placed),
len(placer.placed),
spec.section_w,
spec.section_d,
spec.gap_m,
coverage_cap * 100,
)
return placed
return placer.placed
def _footprints_to_geojson(
@ -187,6 +209,268 @@ def _footprints_to_geojson(
return {"type": "FeatureCollection", "features": features}
# ── Stage 3a (#1965): program-driven placement (типовые дома вместо max-FAR sweep) ──
# Разрыв между секциями в program-режиме (м). Пункт программы не несёт gap (контракт —
# только тип/этажность/количество), поэтому берём один нормативный противопожарный/
# инсоляционный зазор для всех секций программы — середина диапазона стратегий 1b
# (max_area gap=6 … max_insolation gap=15). Достаточно консервативно для реалистичной
# раскладки, не патологически разрежено.
_PROGRAM_GAP_M: float = 10.0
@dataclass(frozen=True)
class _PlacedSection:
"""Одна размещённая секция программы: пятно + этажность + тип каталога (для GeoJSON)."""
footprint: Polygon
floors: int
section_type: str
@dataclass(frozen=True)
class PlacedProgram:
"""Результат program-раскладки: что легло + честный счётчик «N из M».
``sections`` размещённые секции (пятна + этажность + тип) в порядке программы.
``requested_count`` сколько секций просили (Σ count по программе).
``placed_count`` сколько реально влезло (== len(sections)). placed < requested
участок мал, разместилось N из M (без hard-422 честный сигнал для Stage 3b).
"""
sections: tuple[_PlacedSection, ...]
requested_count: int
placed_count: int
def place_program(
parcel: Parcel,
program: list[BuildingProgramItem],
*,
gap_m: float = _PROGRAM_GAP_M,
) -> PlacedProgram:
"""Stage 3a: разложить РОВНО заданную программу типовых домов на участок.
Для каждого пункта программы (``section_type`` из каталога × ``count`` секций) кладём
до ``count`` секций каталожного пятна на сетку участка, переиспользуя ТУ ЖЕ
collision/STRtree/setback-машину, что и жадная раскладка (:class:`_Placer`) никакого
coverage-cap, стоп-критерий = достигнут ``count`` для пункта или кончились свободные
якоря. Пункты обрабатываются по порядку; накопленные секции участвуют в проверке
разрыва для последующих (общий :class:`_Placer`).
Если участок не вмещает все запрошенные секции, НЕ роняем 422 кладём сколько влезло
и возвращаем честный ``placed_count``/``requested_count`` (разместилось N из M).
Raises:
KeyError: ``section_type`` пункта нет в каталоге (валидируется на API-слое до
размещения; здесь это программная ошибка контракта, не пользовательский ввод).
"""
buildable = parcel.buildable_m
half_gap = gap_m / 2.0
placer = _Placer()
placed_sections: list[_PlacedSection] = []
requested = 0
for item in program:
house = catalog.get_house_type(item.section_type)
requested += item.count
# Ручное пятно (Stage 3c): если пользователь задал ОБА габарита — кладём
# его вместо каталожного («вписать пятно»). Частичное задание игнорируем
# (нужны и ширина, и глубина), падая обратно на каталог.
if item.footprint_w_m is not None and item.footprint_d_m is not None:
fp_w, fp_d = item.footprint_w_m, item.footprint_d_m
else:
fp_w, fp_d = house.footprint_w_m, house.footprint_d_m
placed_for_item = 0
for cell in parcel.grid:
if placed_for_item >= item.count:
break
footprint = _centered_footprint(cell.cx, cell.cy, fp_w, fp_d)
if placer.try_place(footprint, buildable, half_gap):
placed_sections.append(
_PlacedSection(
footprint=footprint,
floors=item.floors,
section_type=item.section_type,
)
)
placed_for_item += 1
if placed_for_item < item.count:
logger.warning(
"program: type=%s placed %d of %d sections (%.0fx%.0f m) — участок мал",
item.section_type,
placed_for_item,
item.count,
house.footprint_w_m,
house.footprint_d_m,
)
result = PlacedProgram(
sections=tuple(placed_sections),
requested_count=requested,
placed_count=len(placed_sections),
)
logger.info(
"program placed %d of %d sections across %d type(s)",
result.placed_count,
result.requested_count,
len(program),
)
return result
def _placed_program_to_geojson(
parcel: Parcel,
sections: tuple[_PlacedSection, ...],
) -> dict[str, object]:
"""Размещённые секции программы -> WGS84 FeatureCollection (контракт buildings_geojson).
Зеркалит :func:`_footprints_to_geojson`, но каждая секция несёт СВОИ floors и тип
каталога (program-режим смешивает типы/этажности), а ``strategy`` помечается
``"program"`` маркер, что вариант построен из программы, а не из 1b-стратегии.
"""
features: list[dict[str, object]] = []
for i, sec in enumerate(sections):
geom_wgs = parcel.metric_geom_to_wgs84(sec.footprint)
features.append(
{
"type": "Feature",
"geometry": geom_wgs,
"properties": {
"section_id": i + 1,
"floors": sec.floors,
"footprint_sqm": round(float(sec.footprint.area), 1),
"section_type": sec.section_type,
"strategy": "program",
},
}
)
return {"type": "FeatureCollection", "features": features}
def _aggregate_program_teap(
sections: tuple[_PlacedSection, ...],
*,
site_area_sqm: float,
housing_class: teap.HousingClass,
) -> TEAP:
"""Свести размещённую программу (СМЕШАННАЯ этажность) в один :class:`TEAP` — ТОЧНО.
``compute_teap`` берёт ОДНУ этажность на список пятен, поэтому при смешанной по типам
этажности нельзя просто скормить ему все пятна с одним числом (округлённая «средняя»
этажность даёт дрейф GFA ~1%). Вместо этого группируем секции по этажности, считаем
``compute_teap`` для каждой однородной группы и СУММИРУЕМ результаты:
* built / GFA / office / residential аддитивны сумма точна (GFA = Σ площадь_i×floors_i);
* apartments ``Σ floor(жилая_g / avg)`` по группам: физически корректнее, чем
``floor(Σжилая / avg)`` (нельзя «склеивать» дробные квартиры между корпусами);
* parking пересчитываем от ИТОГОВОГО числа квартир по той же норме класса (ceil от
суммы, а не сумма ceil иначе пер-группное округление вверх задвоит места);
* density (FAR) от суммарной GFA и площади участка (защита от деления на ноль).
Единый источник всех нормативных коэффициентов остаётся ``teap``-модуль (классовые
словари), новых магических чисел нет.
"""
if not sections:
return teap.compute_teap(
footprints=[], floors=0, site_area_sqm=site_area_sqm, housing_class=housing_class
)
# Группируем по этажности; внутри группы compute_teap корректен (одна этажность).
groups: dict[int, list[Polygon]] = {}
for sec in sections:
groups.setdefault(sec.floors, []).append(sec.footprint)
built = 0.0
gfa = 0.0
office = 0.0
residential = 0.0
apartments = 0
for floors, fps in groups.items():
# site_area_sqm здесь не важна для аддитивных полей — FAR пересчитаем в конце.
group = teap.compute_teap(
footprints=fps,
floors=floors,
site_area_sqm=site_area_sqm,
housing_class=housing_class,
)
built += group.built_area_sqm
gfa += group.total_floor_area_sqm
office += group.office_area_sqm
residential += group.residential_area_sqm
apartments += group.apartments_count
density = gfa / site_area_sqm if site_area_sqm > 0 else 0.0
parking_norm = teap._PARKING_PER_APARTMENT[housing_class]
parking_spaces = math.ceil(apartments * parking_norm)
return TEAP(
built_area_sqm=round(built, 1),
total_floor_area_sqm=round(gfa, 1),
office_area_sqm=round(office, 1),
residential_area_sqm=round(residential, 1),
apartments_count=apartments,
density=round(density, 3),
parking_spaces=parking_spaces,
)
def place_program_variant(
parcel: Parcel,
payload: ConceptInput,
*,
market_price_per_sqm: float | None = None,
price_source: str = "class_norm",
) -> ConceptVariant | None:
"""Stage 3a: построить ОДИН вариант из ``payload.building_program`` (типовые дома).
Раскладывает программу (:func:`place_program`), сводит размещённые пятна в ТЭП
(:func:`_aggregate_program_teap` точная GFA по группам этажности) и финмодель, и
наклеивает честный сигнал частичного размещения (``placed_count``/``requested_count``).
Возвращает ``None``, если ни одна секция не легла (участок не вмещает даже одну секцию
программы) вызывающий отбракует, как и в жадном пути.
``payload.building_program`` ДОЛЖЕН быть задан (вызывается только из program-ветки).
"""
program = payload.building_program
if not program: # защитный инвариант: эту ветку зовут только при заданной программе
raise ValueError("place_program_variant called without building_program")
placed = place_program(parcel, program)
if placed.placed_count == 0:
logger.warning(
"program placed 0 of %d sections — участок не вмещает программу, отбраковка",
placed.requested_count,
)
return None
teap_result = _aggregate_program_teap(
placed.sections,
site_area_sqm=parcel.site_area_sqm,
housing_class=payload.housing_class,
)
financial_result = financial.compute_financial(
teap=teap_result,
housing_class=payload.housing_class,
land_cost_rub=payload.land_cost_rub,
market_price_per_sqm=market_price_per_sqm,
price_source=price_source,
development_type=payload.development_type,
)
buildings_geojson = _placed_program_to_geojson(parcel, placed.sections)
# Program-вариант репортуется под "balanced" (контракт strategy — фиксированный
# Literal трёх стратегий 1b; program-режим не вводит новую стратегию, маркер режима
# лежит в properties.strategy="program" каждой фичи GeoJSON). Один вариант на программу.
return ConceptVariant(
strategy="balanced",
buildings_geojson=buildings_geojson,
teap=teap_result,
financial=financial_result,
placed_count=placed.placed_count,
requested_count=placed.requested_count,
)
def place_strategy(
parcel: Parcel,
payload: ConceptInput,
@ -260,7 +544,31 @@ def place_all_strategies(
``market_price_per_sqm`` / ``price_source`` (рыночная калибровка цены жилья, PR-2)
прокидываются неизменными в каждую стратегию цена едина для участка.
Stage 3a (#1965): если задана ``payload.building_program`` — раскладываем РОВНО эту
программу типовых домов (:func:`place_program_variant`, один вариант), а НЕ три жадные
стратегии. ``building_program is None`` существующий жадный путь без изменений.
"""
if payload.building_program:
program_variant = place_program_variant(
parcel,
payload,
market_price_per_sqm=market_price_per_sqm,
price_source=price_source,
)
if program_variant is None:
raise ParcelGeometryError(
"программа застройки не вместила ни одной секции — "
"участок слишком узкий/мелкий для выбранных типов домов"
)
logger.info(
"placed program variant: %d of %d sections, %dкв",
program_variant.placed_count,
program_variant.requested_count,
program_variant.teap.apartments_count,
)
return [program_variant]
variants = [
variant
for spec in _STRATEGIES
@ -288,7 +596,10 @@ def place_all_strategies(
__all__ = [
"FLOOR_HEIGHT_M",
"PlacedProgram",
"StrategySpec",
"place_all_strategies",
"place_program",
"place_program_variant",
"place_strategy",
]

View file

@ -136,4 +136,76 @@ def compute_teap(
return teap
__all__ = ["HousingClass", "compute_teap"]
def synthesize_teap_from_program(
*,
total_footprint_sqm: float,
floors: int,
site_area_sqm: float,
housing_class: HousingClass,
sections: int = 1,
) -> TEAP:
"""Свести АГРЕГИРОВАННУЮ массинг-программу (скалярное пятно × этажность) в :class:`TEAP`.
Stage 2a (#1965): LIVE-пересчёт экономики из интерактивного 3D-массинга. Фронтовый
``computeModel`` отдаёт уже СВЁРНУТУЮ программу суммарное пятно застройки (кв.м) и
этажность, без покомпонентной геометрии секций. Здесь мы строим из этого тот же
:class:`TEAP`, что и :func:`compute_teap`/``synthesize_teap_from_buildability``, чтобы
прогнать его через готовый :func:`compute_financial`.
Зеркалит :func:`synthesize_teap_from_buildability` ОДИН-В-ОДИН, только источник
площадей СКАЛЯРНОЕ пятно, а не ``max_far × area``:
* ``built`` (пятно) = ``total_footprint_sqm`` (как есть, сумма по секциям).
* ``GFA`` (total_floor) = ``total_footprint_sqm × floors``.
* ``office`` (нежилое) = ``GFA × _OFFICE_SHARE_OF_GFA[class]`` вырезается из GFA
до расчёта жилой (жилая ужимается ровно на эту площадь, total GFA не меняется).
* ``residential`` = ``(GFA office) × _EFFICIENCY_BY_CLASS[class]``.
* ``apartments`` = ``floor(residential / _AVG_APARTMENT_SQM[class])``.
* ``parking`` = ``ceil(apartments × _PARKING_PER_APARTMENT[class])``.
* ``density`` = ``GFA / site_area_sqm`` (FAR), защита от деления на ноль.
Чистая функция без БД / LLM / внешних API. Те же нормативные константы, что и
остальные синтезаторы (single source of truth, без новых магических чисел).
Args:
total_footprint_sqm: суммарное пятно застройки всех секций, кв.м (скаляр).
floors: этажность (общая для программы); вне диапазона клампится к >= 0.
site_area_sqm: площадь участка для плотности (FAR).
housing_class: класс жилья задаёт эффективность/средний лот/парковку/нежилое.
sections: число секций метаданные программы; на ТЭП НЕ влияет (площади уже
свёрнуты во ``total_footprint_sqm``). Принимается для симметрии с контрактом
фронта и будущей пер-секционной логики; здесь намеренно не используется.
"""
del sections # метаданные программы; площади уже агрегированы — на ТЭП не влияют
built_area = max(0.0, total_footprint_sqm)
total_floor_area = built_area * max(0, floors)
# Нежилое (коммерция/офисы 1-го этажа) вырезаем из GFA ДО расчёта жилой — точно как
# compute_teap / synthesize_teap_from_buildability (single source of truth по долям).
office_share = _OFFICE_SHARE_OF_GFA[housing_class]
office_area = total_floor_area * office_share
residential_gfa = total_floor_area - office_area
efficiency = _EFFICIENCY_BY_CLASS[housing_class]
residential_area = residential_gfa * efficiency
avg_apartment = _AVG_APARTMENT_SQM[housing_class]
apartments_count = math.floor(residential_area / avg_apartment) if avg_apartment else 0
density = total_floor_area / site_area_sqm if site_area_sqm > 0 else 0.0
parking_norm = _PARKING_PER_APARTMENT[housing_class]
parking_spaces = math.ceil(apartments_count * parking_norm)
return TEAP(
built_area_sqm=round(built_area, 1),
total_floor_area_sqm=round(total_floor_area, 1),
office_area_sqm=round(office_area, 1),
residential_area_sqm=round(residential_area, 1),
apartments_count=apartments_count,
density=round(density, 3),
parking_spaces=parking_spaces,
)
__all__ = ["HousingClass", "compute_teap", "synthesize_teap_from_program"]

View file

@ -48,22 +48,24 @@ _SYSTEM_BASE = PromptTemplate(
# (см. services/chat/{tools,safe_payload}.py), НЕ зашиваются в литерал промпта.
_CHAT_SYSTEM = PromptTemplate(
name="chat_system",
version=2,
version=3,
template=(
"Ты — ассистент по инвестиционному форсайт-отчёту земельного участка (РФ). "
"Отвечай на русском языке, по-деловому, нейтрально, без маркетинга и без emoji.\n\n"
"ЖЁСТКИЕ ПРАВИЛА:\n"
"1. Отвечай ТОЛЬКО на основе данных, полученных через инструменты (секции отчёта). "
"Чтобы получить нужные числа, вызови подходящий инструмент.\n"
"Чтобы получить нужные числа, вызови подходящий инструмент. Для вопросов про сам "
"участок — адрес, площадь, категорию земель, ВРИ, территориальную зону ПЗЗ и её "
"код/название, лимиты застройки, ЗОУИТ-обременения — вызови get_parcel_info.\n"
"2. НИКОГДА не выдумывай числа, классы, доли или выводы. Если в полученной секции "
"данных нет (или помечено available=false) — честно скажи, что этих данных в "
"отчёте нет. Не подставляй правдоподобные значения.\n"
"3. Все числа в ответе бери ВЕРБАТИМ из секций инструментов, ничего не пересчитывай.\n"
"4. Тон советующий: отчёт помогает принять решение, но НЕ является основанием для "
"инвестиционного решения. Не давай гарантий доходности.\n"
"5. Вопросы вне отчёта по участку (градостроительная документация / ПЗЗ-разрешения, "
"сравнение с другими участками, юридические заключения) — вежливо скажи, что это вне "
"области отчёта, и предложи вопросы по самому форсайту.\n"
"5. Вопросы, выходящие за рамки данных отчёта и паспорта участка (сравнение с другими "
"участками, юридические заключения, получение разрешений/согласований) — вежливо "
"скажи, что это вне области отчёта, и предложи вопросы по участку и его форсайту.\n"
"6. При перечислении квартирографии / сегментов указывай ТИП понятно "
"(студия, 1-к, 2-к, 3-к, евро-форматы, 80+ м²) и долю в % если она есть. "
"НЕ нумеруй порядковыми номерами и НЕ склеивай номер с типом через дефис "

View file

@ -33,15 +33,19 @@ Wiring (отдельный PR):
from __future__ import annotations
import asyncio
import json
import logging
import re
from datetime import date
from html.parser import HTMLParser
from typing import Any
from urllib.parse import urljoin
from sqlalchemy import text
from sqlalchemy.orm import Session
from app.core.config import settings
from app.services.scrapers.flat_plans import download_plan_image, upsert_flat_plans
from app.services.scrapers.stealth import BASE_URL, BrowserSession, jitter_sleep
logger = logging.getLogger(__name__)
@ -87,6 +91,51 @@ _STATUS_KW_RE = re.compile(
# не имеющим отношения к статусу продажи (#1609 follow-up, #1686).
_STATUS_BADGE_CLS_RE = re.compile(r"(?<![a-z])status(?![a-z])", re.IGNORECASE)
# ── plan-image extraction ─────────────────────────────────────────────────────
#
# Страница квартиры каталога DOM.РФ — Next.js SSR (тот же стек что и страница
# объекта, см. domrf_catalog_object.py). План квартиры рендерится через next/image,
# поэтому реальный URL картинки живёт в JSON-блоке <script id="__NEXT_DATA__">
# внутри pageProps, а НЕ в готовом <img src>. Стратегия extraction (в порядке
# приоритета, best-effort — DOM.РФ может переименовать ключи без уведомления):
#
# 1. __NEXT_DATA__ pageProps: ищем значение под одним из кандидат-ключей
# (planImage / planImageUrl / layoutImage / floorPlan / scheme / image / photo).
# 2. Fallback: bare <img>, чей src/alt/class намекают на планировку
# (plan / layout / scheme / планировк), либо <img> рядом с текстом «Планировка».
#
# URL нормализуется в абсолютный через urljoin(BASE_URL, ...): DOM.РФ отдаёт файлы
# и абсолютными (https://xn--80az8a.../api/ext/file/... — см. documents.py), и
# относительными (/_next/image?url=..., /api/ext/file/...). Мусорные картинки
# (data:-inline base64, captcha, лого застройщика, миниатюры) отсекаются.
# Ключи в pageProps, под которыми может лежать URL планировки. Проверяются по
# порядку; первое непустое строковое значение с image-like URL выигрывает.
_NEXT_PLAN_KEY_RE = re.compile(
r"(?:plan|layout|scheme|планировк|floor)",
re.IGNORECASE,
)
# Признак «это картинка» в значении: расширение файла ИЛИ file-API DOM.РФ.
_IMAGE_URL_RE = re.compile(
r"(?:\.(?:png|jpe?g|webp|svg)(?:\?|$)|/api/ext/file/|/_next/image)",
re.IGNORECASE,
)
# Токены, по которым отсекаем НЕ-планировочные картинки (лого, аватары, иконки,
# captcha, миниатюры, банеры). Проверяются по src/alt/class.
_IMAGE_NOISE_RE = re.compile(
r"(?:logo|avatar|icon|captcha|robot|sprite|banner|placeholder|thumb|"
r"favicon|badge|marker|pin)",
re.IGNORECASE,
)
# Признак «это планировка» в атрибутах <img> (src / alt / class).
_IMAGE_PLAN_HINT_RE = re.compile(
r"(?:plan|layout|scheme|планировк)",
re.IGNORECASE,
)
def _classify_status_kw(matched_text: str) -> str | None:
"""Классифицировать совпавшее ключевое слово в STATUS_*.
@ -217,8 +266,19 @@ class _TextCollector(HTMLParser):
# Список записей: (class_hint, full_text)
self.blocks: list[tuple[str, str]] = []
self._buf: list[str] = []
# Отдельный, аддитивный сбор <img>-атрибутов (issue #2440): нужен для
# extraction plan_image_url как fallback к __NEXT_DATA__. НЕ влияет на
# text-extraction — img остаётся void-тегом, фрейм в стек не пушится.
# Формат: (attrs_dict, index_в_blocks) — index привязывает картинку к
# позиции в потоке текстовых блоков для proximity-поиска «Планировка».
self.images: list[tuple[dict[str, str], int]] = []
def handle_starttag(self, tag: str, attrs: list[tuple[str, str | None]]) -> None:
if tag == "img":
# Собираем атрибуты img ДО early-return (void-тег): index = текущая
# длина blocks, т.е. картинка «привязана» к ближайшему предыдущему
# закрытому текстовому блоку. Text-extraction ниже не меняется.
self.images.append(({k: (v or "") for k, v in attrs}, len(self.blocks)))
if tag in self._VOID_TAGS:
return # void-теги не имеют endtag — не открываем фрейм (issue #1608)
attr_dict = {k: (v or "") for k, v in attrs}
@ -266,6 +326,134 @@ def _find_text_near(
return None
def _normalize_plan_url(url: str | None) -> str | None:
"""Нормализовать URL картинки планировки в абсолютный, отсеять мусор.
- data:-inline (base64) отбрасываем (это не удалённый файл, часто captcha/robot).
- Относительный путь ('/api/ext/file/...', '/_next/image?url=...') absolute
через urljoin(BASE_URL, ...).
- Пустое/None None.
"""
if not url:
return None
u = url.strip()
if not u or u.lower().startswith("data:"):
return None
if u.startswith("//"):
# protocol-relative → https
return "https:" + u
if u.startswith("http://") or u.startswith("https://"):
return u
return urljoin(BASE_URL + "/", u.lstrip("/"))
def _extract_plan_from_next_data(html: str) -> str | None:
"""Извлечь URL планировки из <script id="__NEXT_DATA__"> pageProps.
Страница квартиры каталога Next.js SSR: план рендерится через next/image,
URL картинки живёт в JSON-блоке. Рекурсивно обходим pageProps и ищем строковое
значение под ключом, матчащим _NEXT_PLAN_KEY_RE, чьё значение выглядит как
image-URL (_IMAGE_URL_RE) и НЕ является шумом (_IMAGE_NOISE_RE).
Возвращает первый подходящий URL (нормализованный) или None.
"""
match = re.search(
r'<script\s+id=["\']__NEXT_DATA__["\'][^>]*>(.+?)</script>',
html,
re.DOTALL,
)
if not match:
return None
try:
blob = json.loads(match.group(1).strip())
except (json.JSONDecodeError, ValueError):
return None
page_props = blob.get("props", {}).get("pageProps")
root: Any = page_props if isinstance(page_props, dict) else blob
# BFS по вложенному dict/list; ключ+значение проверяем на plan-hint.
stack: list[Any] = [root]
seen = 0
while stack and seen < 20_000: # cap: защита от патологически глубокого JSON
node = stack.pop()
seen += 1
if isinstance(node, dict):
for key, value in node.items():
if (
isinstance(value, str)
and _NEXT_PLAN_KEY_RE.search(str(key))
and _IMAGE_URL_RE.search(value)
and not _IMAGE_NOISE_RE.search(value)
):
normalized = _normalize_plan_url(value)
if normalized:
return normalized
if isinstance(value, dict | list):
stack.append(value)
elif isinstance(node, list):
stack.extend(node)
return None
def _extract_plan_from_imgs(
images: list[tuple[dict[str, str], int]], blocks: list[tuple[str, str]]
) -> str | None:
"""Fallback: выбрать <img> планировки из собранных _TextCollector.images.
Приоритет:
1. <img>, чей src/alt/class содержит plan/layout/scheme/планировк прямое
совпадение, самый надёжный сигнал.
2. <img> рядом (в потоке DOM) с текстовым блоком «Планировка».
Шумовые картинки (лого/иконки/captcha/миниатюры) отсекаются по _IMAGE_NOISE_RE,
inline data:-картинки и пустые src через _normalize_plan_url.
"""
# Уровень 1: явный plan-hint в атрибутах.
for attrs, _idx in images:
src = attrs.get("src", "")
alt = attrs.get("alt", "")
cls = attrs.get("class", "")
haystack = f"{src} {alt} {cls}"
if _IMAGE_NOISE_RE.search(haystack):
continue
if _IMAGE_PLAN_HINT_RE.search(haystack):
normalized = _normalize_plan_url(src)
if normalized:
return normalized
# Уровень 2: <img>, чей index стоит сразу после блока «Планировка».
plan_block_indices = [
i for i, (_cls, text) in enumerate(blocks) if re.search(r"планировк", text, re.IGNORECASE)
]
if plan_block_indices:
for attrs, idx in images:
src = attrs.get("src", "")
if _IMAGE_NOISE_RE.search(f"{src} {attrs.get('alt', '')} {attrs.get('class', '')}"):
continue
# картинка «принадлежит» блоку «Планировка», если её index в пределах
# ±2 текстовых блоков от лейбла.
if any(abs(idx - pbi) <= 2 for pbi in plan_block_indices):
normalized = _normalize_plan_url(src)
if normalized:
return normalized
return None
def extract_plan_image_url(html: str, collector: _TextCollector) -> str | None:
"""Извлечь URL картинки планировки из SSR-HTML страницы квартиры.
Two-strategy (issue #2440):
1. __NEXT_DATA__ pageProps (primary DOM.РФ Next.js SSR).
2. bare <img> fallback (по plan-hint атрибутам либо proximity к «Планировка»).
Возвращает абсолютный URL или None. Best-effort структура DOM.РФ хрупкая.
"""
url = _extract_plan_from_next_data(html)
if url:
return url
return _extract_plan_from_imgs(collector.images, collector.blocks)
def parse_catalog_flat(html: str) -> dict[str, Any]:
"""Извлечь поля из SSR-HTML страницы квартиры DOM.РФ.
@ -277,6 +465,7 @@ def parse_catalog_flat(html: str) -> dict[str, Any]:
- ceiling_height_m (float) Высота потолков в метрах
- section_no (int) Номер подъезда / секции
- catalog_updated_at (date) Дата обновления информации на странице
- plan_image_url (str) URL картинки планировки (issue #2440)
Парсинг хрупкий по природе (SSR HTML DOM.РФ меняется без уведомлений).
Все extraction best-effort KeyError/AttributeError обёрнуты внутри.
@ -468,6 +657,14 @@ def parse_catalog_flat(html: str) -> dict[str, Any]:
except ValueError:
pass
# ── Шаг 4: plan_image_url (issue #2440) ──────────────────────────────────
# __NEXT_DATA__ pageProps (primary) → bare <img> fallback. collector.images
# собран на Шаге 1 тем же feed(). None если план не найден (напр. проданная
# квартира без планировки на странице).
plan_url = extract_plan_image_url(html, collector)
if plan_url:
result["plan_image_url"] = plan_url
logger.debug(
"parse_catalog_flat: extracted fields=%s",
list(result.keys()),
@ -544,16 +741,63 @@ def upsert_catalog_data(
# ── Per-flat scrape orchestration ─────────────────────────────────────────────
async def _handle_plan_image(
db: Session,
session: BrowserSession,
ods_id: str,
obj_id: int,
plan_url: str,
snapshot_date: date,
outcome: dict[str, Any],
) -> None:
"""Записать URL планировки в flat_plans, скачать бинарник, дописать метаданные.
Порядок (issue #2440):
1. upsert_flat_plans с одним только plan_image_url URL зафиксирован даже
если скачивание упадёт (частичный прогресс не теряется).
2. download_plan_image reuse BrowserSession (cookies/TLS/throttle).
3. повторный upsert_flat_plans с local_path/размерами COALESCE внутри
upsert защищает уже скачанные файлы от перезаписи.
Мутирует outcome['plan_downloaded']. Ошибки логируются внутри download/upsert,
наружу не пробрасываются планировка не должна ронять catalog-scrape квартиры.
"""
plan_row: dict[str, Any] = {
"ods_id": ods_id,
"obj_id": obj_id,
"plan_image_url": plan_url,
}
upsert_flat_plans(db, obj_id, [plan_row], snapshot_date)
meta = await download_plan_image(session, plan_url, ods_id, obj_id)
if meta:
outcome["plan_downloaded"] = True
upsert_flat_plans(db, obj_id, [{**plan_row, **meta}], snapshot_date)
async def scrape_one_flat(
session: BrowserSession,
db: Session,
ods_id: str,
catalog_url_hash: str,
obj_id: int | None = None,
snapshot_date: date | None = None,
) -> dict[str, Any]:
"""Scrape одной квартиры: fetch HTML → parse → upsert.
"""Scrape одной квартиры: fetch HTML → parse → upsert (+ планировка).
Возвращает dict с результатом: {ods_id, success, fields_extracted, updated}.
Ошибки fetch/parse логируются, не бросаются caller обрабатывает результат.
Помимо catalog-полей (price/status/finishing/...) извлекает plan_image_url
(issue #2440): пишет URL в domrf_kn_flat_plans через upsert_flat_plans, затем
скачивает бинарник (download_plan_image) и повторным upsert'ом дописывает
local_path/размеры (COALESCE-защита уже скачанных файлов внутри upsert).
Скачивание inline в рамках того же BrowserSession: reuse cookies/TLS,
download_binary несёт свой лёгкий throttle (jitter 200500 мс), отдельная
beat-задача не заводится (требование #2440). obj_id/snapshot_date нужны для
записи в flat_plans; если не переданы plan-upsert пропускается с warning.
Возвращает dict с результатом: {ods_id, success, fields_extracted, updated,
plan_found, plan_downloaded}. Ошибки fetch/parse/download логируются, не
бросаются caller обрабатывает результат.
"""
outcome: dict[str, Any] = {
"ods_id": ods_id,
@ -561,6 +805,8 @@ async def scrape_one_flat(
"success": False,
"fields_extracted": 0,
"updated": False,
"plan_found": False,
"plan_downloaded": False,
"error": None,
}
@ -590,6 +836,19 @@ async def scrape_one_flat(
outcome["fields_extracted"] = len([v for v in data.values() if v is not None])
outcome["updated"] = upsert_catalog_data(db, ods_id, catalog_url_hash, data)
# ── plan_image_url → domrf_kn_flat_plans (issue #2440) ───────────────────
plan_url = data.get("plan_image_url")
if plan_url:
outcome["plan_found"] = True
if obj_id is not None and snapshot_date is not None:
await _handle_plan_image(db, session, ods_id, obj_id, plan_url, snapshot_date, outcome)
else:
logger.warning(
"catalog scrape ods_id=%s: plan_image_url found but obj_id/snapshot_date "
"missing — flat_plans upsert skipped",
ods_id,
)
# success отражает прохождение пайплайна fetch+parse без исключения; реально
# ли затронута строка в БД — см. outcome['updated']. Батч-статистика считает
# отдельный stats['updated'], чтобы не рапортовать ложно высокий success при
@ -617,11 +876,12 @@ async def scrape_catalog_batch(
region_code: int = 66,
headed: bool = False,
load_state: str | None = None,
snapshot_date: date | None = None,
) -> dict[str, Any]:
"""Scrape пачки квартир каталога DOM.РФ.
`flats` список dict'ов с ключами {ods_id, catalog_url_hash}.
Типовой источник: SELECT ods_id, catalog_url_hash FROM domrf_kn_flats
`flats` список dict'ов с ключами {ods_id, catalog_url_hash [, obj_id]}.
Типовой источник: SELECT ods_id, obj_id, catalog_url_hash FROM domrf_kn_flats
WHERE catalog_url_hash IS NOT NULL
AND (catalog_updated_at IS NULL OR catalog_updated_at < NOW() - INTERVAL '30 days')
LIMIT :batch_size.
@ -629,18 +889,25 @@ async def scrape_catalog_batch(
Использует один BrowserSession на весь пакет (bootstrapped 1 раз).
jitter_sleep между запросами встроен в fetch_catalog_html (через BrowserSession._sem).
obj_id/snapshot_date нужны для записи планировки в domrf_kn_flat_plans (issue
#2440). snapshot_date по умолчанию = сегодня (дата первичного обнаружения URL).
Returns:
{total, success, updated, failed, fields_total}
{total, success, updated, failed, fields_total, plans_found, plans_downloaded}
- success: прошли fetch+parse без исключения
- updated: реально затронули строку в БД (issue #1610) — отражает
фактическое число записанных квартир, в отличие от success
- plans_found / plans_downloaded: планировки найдены / скачаны (#2440)
"""
snap = snapshot_date or date.today()
stats: dict[str, Any] = {
"total": len(flats),
"success": 0,
"updated": 0,
"failed": 0,
"fields_total": 0,
"plans_found": 0,
"plans_downloaded": 0,
}
if not flats:
@ -648,9 +915,10 @@ async def scrape_catalog_batch(
return stats
logger.info(
"scrape_catalog_batch: starting %d flats region=%d",
"scrape_catalog_batch: starting %d flats region=%d snapshot_date=%s",
len(flats),
region_code,
snap,
)
async with BrowserSession(
@ -659,21 +927,42 @@ async def scrape_catalog_batch(
load_state=load_state,
# auth=None — страницы каталога публичные, Basic auth не нужен
auth=None,
# #2445 D2 anti-ban: тот же /сервисы/* path family, что вызвал DOM.РФ WAF
# hard-ban 2026-05-24 (#2443). Без явного override BrowserSession тихо
# наследует модульный дефолт stealth.py (concurrency=8, jitter 600-1500ms).
# Переиспользуем throttled настройки, принятые scrape_kn.py после #1945
# (settings.scrape_kn_* — общий anti-ban рычаг, НЕ специфичный для KN-sweep,
# несмотря на имя).
concurrency=settings.scrape_kn_browser_concurrency,
jitter_min_ms=settings.scrape_kn_request_jitter_min_ms,
jitter_max_ms=settings.scrape_kn_request_jitter_max_ms,
) as session:
# Warm-up: WAF DOM.РФ (2026-05-24) требует session cookies для SSR-страниц.
# Зеркалит scrape_catalog_objects. Idempotent на весь batch.
await session.warm_up()
for flat in flats:
ods_id = flat.get("ods_id", "")
catalog_url_hash = flat.get("catalog_url_hash", "")
obj_id_raw = flat.get("obj_id")
obj_id = int(obj_id_raw) if obj_id_raw is not None else None
if not ods_id or not catalog_url_hash:
logger.warning("scrape_catalog_batch: skip flat with missing ods_id/hash: %r", flat)
stats["failed"] += 1
continue
outcome = await scrape_one_flat(session, db, ods_id, catalog_url_hash)
outcome = await scrape_one_flat(
session, db, ods_id, catalog_url_hash, obj_id=obj_id, snapshot_date=snap
)
if outcome["success"]:
stats["success"] += 1
stats["fields_total"] += outcome["fields_extracted"]
if outcome["updated"]:
stats["updated"] += 1
if outcome.get("plan_found"):
stats["plans_found"] += 1
if outcome.get("plan_downloaded"):
stats["plans_downloaded"] += 1
else:
stats["failed"] += 1
@ -689,11 +978,14 @@ async def scrape_catalog_batch(
raise
logger.info(
"scrape_catalog_batch done: total=%d success=%d updated=%d failed=%d fields_total=%d",
"scrape_catalog_batch done: total=%d success=%d updated=%d failed=%d "
"fields_total=%d plans_found=%d plans_downloaded=%d",
stats["total"],
stats["success"],
stats["updated"],
stats["failed"],
stats["fields_total"],
stats["plans_found"],
stats["plans_downloaded"],
)
return stats

View file

@ -24,6 +24,7 @@ from typing import Any
from sqlalchemy import text
from sqlalchemy.orm import Session
from app.core.config import settings
from app.services.scrapers.stealth import BASE_URL, BrowserSession, WafBlockedError, jitter_sleep
logger = logging.getLogger(__name__)
@ -440,6 +441,16 @@ async def scrape_catalog_objects(
region_code=region_code,
# Страницы каталога публичные — Basic auth не нужен
auth=None,
# #2445 D2 anti-ban: этот scraper ходит по тому же /сервисы/* path family,
# что вызвал DOM.РФ WAF hard-ban 2026-05-24 (#2443). Без явного override
# BrowserSession тихо наследует модульный дефолт stealth.py (concurrency=8,
# jitter 600-1500ms) — тот самый throttle-less профиль, что и раньше
# триггерил volume-ban. Переиспользуем throttled настройки, принятые
# scrape_kn.py после #1945 (settings.scrape_kn_* — общий anti-ban рычаг,
# НЕ специфичный для KN-sweep, несмотря на имя).
concurrency=settings.scrape_kn_browser_concurrency,
jitter_min_ms=settings.scrape_kn_request_jitter_min_ms,
jitter_max_ms=settings.scrape_kn_request_jitter_max_ms,
) as session:
# Warm-up: visit /сервисы/каталог-новостроек/ to obtain WAF cookies.
# DOM.РФ WAF (2026-05-24) блокирует SSR-страницы объектов без этих cookies.

View file

@ -33,7 +33,7 @@ from app.services.scrapers.documents import extract_documents, upsert_documents
# obj_checks import temporarily disabled — endpoint /checks returns 404 (run #19).
# Re-enable with _fetch_obj_checks_safe when endpoint is found (see TODO in Phase B/C).
# from app.services.scrapers.obj_checks import extract_obj_checks, upsert_obj_checks
from app.services.scrapers.stealth import BASE_URL, BrowserSession
from app.services.scrapers.stealth import BASE_URL, BrowserSession, WafBlockedError
logger = logging.getLogger(__name__)
@ -432,6 +432,14 @@ def _norm_flat(row: dict[str, Any], region_cd: int | None) -> dict[str, Any]:
digest = hashlib.sha256(str(elem).encode("utf-8")).digest()[:8]
flat_id = int.from_bytes(digest, "big") % (2**63 - 1)
# catalog_url_hash — тот же elemId (hash-часть URL квартиры в каталоге DOM.РФ):
# https://наш.дом.рф/сервисы/каталог-квартир/квартира/{catalog_url_hash}.
# Читаем ЗДЕСЬ (а не только в fallback-ветке flat_id выше) — при наличии flatId
# ветка fallback не выполняется и elem остаётся непрочитанным. Пустую строку
# нормализуем в None: `or None` зеркалит трактовку `if elem:` в fallback-ветке
# и общий null-handling соседних optional-полей (_g отдаёт None при отсутствии).
catalog_url_hash = _g(row, "elemId") or None
price_rub = _g(row, "price")
price_per_m2 = _g(row, "pricePerSquareMeter")
total_area = _g(row, "totalArea")
@ -476,7 +484,10 @@ def _norm_flat(row: dict[str, Any], region_cd: int | None) -> dict[str, Any]:
"ceiling_height_m": None, # заполнит catalog scraper 22d
"key_handover_dt": None, # заполнит catalog scraper 22d
"catalog_updated_at": None, # заполнит catalog scraper 22d
"catalog_url_hash": None, # заполнит catalog scraper 22d
# catalog_url_hash приходит из kn-API (elemId) уже здесь — нужен catalog
# scraper'у (22d) для построения URL страницы квартиры. Ранее хардкодился
# в None, из-за чего flat-catalog scraper не мог выбрать ни одной строки.
"catalog_url_hash": catalog_url_hash,
}
@ -524,14 +535,29 @@ UPSERT_OBJECT_SQL = text(
:snapshot_date
)
ON CONFLICT (obj_id, snapshot_date) DO UPDATE SET
-- #2464 cluster E: hobj_id/short_addr/dev_inn/region_cd/floor_min/floor_max/
-- problem_flag/green_house были в INSERT (populated every call by
-- _norm_object) но пропущены в DO UPDATE SET re-run скрапера для
-- уже существующего (obj_id, snapshot_date) оставлял эти 8 колонок stale.
-- Все 8 direct EXCLUDED (не COALESCE): freshly computed каждый вызов,
-- в т.ч. problem_flag должен ОБНУЛЯТЬСЯ, если проблема на объекте снята
-- (COALESCE зафиксировал бы устаревший "проблемный" статус навсегда).
hobj_id = EXCLUDED.hobj_id,
comm_name = EXCLUDED.comm_name,
addr = EXCLUDED.addr,
short_addr = EXCLUDED.short_addr,
region_cd = EXCLUDED.region_cd,
dev_id = EXCLUDED.dev_id,
dev_name = EXCLUDED.dev_name,
dev_inn = EXCLUDED.dev_inn,
floor_min = EXCLUDED.floor_min,
floor_max = EXCLUDED.floor_max,
flat_count = EXCLUDED.flat_count,
square_living = EXCLUDED.square_living,
ready_dt = EXCLUDED.ready_dt,
problem_flag = EXCLUDED.problem_flag,
site_status = EXCLUDED.site_status,
green_house = EXCLUDED.green_house,
escrow = EXCLUDED.escrow,
obj_class = COALESCE(EXCLUDED.obj_class, domrf_kn_objects.obj_class),
wall_type = COALESCE(EXCLUDED.wall_type, domrf_kn_objects.wall_type),
@ -910,6 +936,9 @@ UPSERT_PHOTO_SQL = text(
:local_path, :thumb_path, :downloaded_at
)
ON CONFLICT (obj_id, obj_file_id) DO UPDATE SET
-- #2464 cluster F: build_type в INSERT column list (populated every call)
-- но пропущен здесь re-run скрапера оставлял build_type stale.
build_type = EXCLUDED.build_type,
ord_num = EXCLUDED.ord_num,
photo_url = EXCLUDED.photo_url,
photo_dttm = EXCLUDED.photo_dttm,
@ -1142,7 +1171,7 @@ async def fetch_obj_checks(sess: BrowserSession, obj_id: int) -> tuple[Any, str]
# ── _fetch_*_safe wrappers for asyncio.gather in Phase B/C ───────────────────
# Каждый wrapper возвращает (kind, full_url, result_or_exception).
# Exceptions НЕ raise — помещаются в возвращаемый tuple.
# BrowserSession._sem (Semaphore(_BROWSER_CONCURRENCY)=8) bounds concurrency per-request.
# BrowserSession._sem (size = per-instance concurrency, #1945 KN=2) bounds concurrency.
async def _fetch_flats_safe(
@ -1450,6 +1479,247 @@ async def download_photos(
return locals_out, thumbs_out
# ── result processing (shared by combined + isolated #1945 passes) ────────────
# #1945 isolated extras pass: после стольких подряд полностью-WAF объектов
# обрываем пас (extras заблокированы целиком — нет смысла жечь ~1500 recycle).
_EXTRAS_ABORT_AFTER_CONSEC_WAF = 5
# 5 doc-endpoint kinds, accumulated into a single per-object documents upsert.
_DOC_KINDS = frozenset(
(
"doc_rpd",
"doc_developer_report",
"doc_project_documentation",
"doc_documentation_other",
"doc_permits",
)
)
def _process_flats_result(
db: Session,
run_id: int,
obj_id: int,
full_url: str,
result: list[dict[str, Any]] | Exception,
snapshot_date: date,
region_code: int,
) -> int:
"""Upsert flats from a _fetch_flats_safe result. Returns flats inserted.
Errors kn_scrape_failures (via _classify_and_log), 0 returned. Вынесено из
Phase-B/C loop чтобы isolated-flats-pass (#1945) и combined-loop делили один код.
"""
if isinstance(result, Exception):
_classify_and_log(db, run_id, obj_id, "flats", full_url, result)
return 0
if result:
return upsert_flats(db, result, snapshot_date, region_code)
return 0
async def _process_extras_result(
sess: BrowserSession,
db: Session,
run_id: int,
obj_id: int,
kind_tag: str,
full_url: str,
result: Any,
snapshot_date: date,
extras_counts: dict[str, int],
all_docs: list[dict[str, Any]],
*,
download_photos_binary: bool,
pdir: Path,
) -> None:
"""Handle one extras endpoint result (sale_graph/sales_agg/infra/photos/docs).
Mutates extras_counts / all_docs in place. Errors kn_scrape_failures.
Doc-kinds накапливаются в all_docs (единый upsert делает вызывающий код).
Вынесено из loop чтобы combined- и isolated-extras-pass (#1945) делили код.
"""
if isinstance(result, Exception):
_classify_and_log(db, run_id, obj_id, kind_tag, full_url, result)
return
if kind_tag in ("sale_graph_apartments", "sale_graph_parking"):
sg_type = kind_tag.replace("sale_graph_", "")
rows_sg, _ = result
extras_counts["sale_graph_rows"] += upsert_sale_graph(
db, obj_id, sg_type, rows_sg, snapshot_date
)
elif kind_tag == "sales_agg":
agg_data, _ = result
extras_counts["sales_agg_rows"] += upsert_sales_agg(db, obj_id, agg_data, snapshot_date)
elif kind_tag == "infrastructure":
pois_data, _ = result
extras_counts["infra_rows"] += upsert_infrastructure(db, obj_id, pois_data, snapshot_date)
elif kind_tag == "photos":
photos_data, _ = result
local_paths: dict[str, str] = {}
thumb_paths: dict[str, str] = {}
if download_photos_binary and photos_data:
local_paths, thumb_paths = await download_photos(sess, obj_id, photos_data, pdir)
extras_counts["photos_downloaded"] += len(local_paths)
extras_counts["photos_rows"] += upsert_photos(
db, obj_id, photos_data, local_paths, thumb_paths
)
elif kind_tag in _DOC_KINDS:
doc_items, _ = result
all_docs.extend(extract_documents(doc_items or []))
def _extras_coros(sess: BrowserSession, obj_id: int) -> list[Any]:
"""The 10 extras-endpoint coroutines for one object (flats NOT included)."""
return [
_fetch_sale_graph_safe(sess, obj_id, "apartments"),
_fetch_sale_graph_safe(sess, obj_id, "parking"),
_fetch_sales_agg_safe(sess, obj_id),
_fetch_infrastructure_safe(sess, obj_id),
_fetch_photos_safe(sess, obj_id),
_fetch_doc_rpd_safe(sess, obj_id),
_fetch_doc_developer_report_safe(sess, obj_id),
_fetch_doc_project_documentation_safe(sess, obj_id),
_fetch_doc_documentation_other_safe(sess, obj_id),
_fetch_doc_permits_safe(sess, obj_id),
]
def _is_waf_poisoned(results: list[tuple[str, str, Any]]) -> bool:
"""Любой WafBlockedError среди результатов = сессия отравлена (cookies флагнуты).
После этого ВСЕ запросы на этой сессии (incl. следующего объекта) пойдут в 403,
поэтому вызывающий код должен пересоздать BrowserSession перед продолжением.
"""
return any(isinstance(r, WafBlockedError) for _kind, _url, r in results)
async def _run_extras_pass(
*,
db: Session,
run_id: int,
region_code: int,
all_objects: list[dict[str, Any]],
start_index: int,
snapshot_date: date,
extras_counts: dict[str, int],
pdir: Path,
download_photos_binary: bool,
load_state: str | None,
headed: bool,
browser_concurrency: int | None,
request_jitter_min_ms: int | None,
request_jitter_max_ms: int | None,
proxy_url: str | None,
) -> int:
"""#1945 isolated extras pass: extras в ОТДЕЛЬНОЙ BrowserSession от flats.
Ключ изоляции: на ЛЮБОЙ WafBlockedError (extras-403 травит cookies сессии)
закрываем сессию и открываем НОВУЮ (свежие cookies/контекст) перед следующим
объектом яд не накапливается и НИКОГДА не касается flats (flats-сессия уже
закрыта). Best-effort: каждый сбой kn_scrape_failures, run не падает.
Возвращает суммарный request_count всех extras-сессий (для метрики run'а).
"""
total = len(all_objects)
total_requests = 0
log_progress(
db, run_id, f"extras-пас (isolated): старт {total - start_index} объектов", stage="extras"
)
def _new_session() -> BrowserSession:
return BrowserSession(
region_code=region_code,
headed=headed,
load_state=load_state,
concurrency=browser_concurrency,
jitter_min_ms=request_jitter_min_ms,
jitter_max_ms=request_jitter_max_ms,
proxy_url=proxy_url,
)
# extras сейчас мертвы (403 с 2026-06-03). Recycle на КАЖДЫЙ объект (полный
# перезапуск браузера ~1500×) бессмыслен, если extras заблокированы целиком.
# Поэтому считаем подряд идущие полностью-WAF объекты и обрываем пас после
# порога — extras остаются best-effort opt-in, но не жгут час впустую.
consecutive_all_waf = 0
aborted = False
sess = _new_session()
await sess.__aenter__()
try:
await sess.warm_up()
for i in range(start_index, total):
obj = all_objects[i]
obj_id = obj.get("objId") or obj.get("obj_id")
if not obj_id:
continue
results = await asyncio.gather(*_extras_coros(sess, obj_id), return_exceptions=False)
all_docs: list[dict[str, Any]] = []
for kind_tag, full_url, result in results:
await _process_extras_result(
sess,
db,
run_id,
obj_id,
kind_tag,
full_url,
result,
snapshot_date,
extras_counts,
all_docs,
download_photos_binary=download_photos_binary,
pdir=pdir,
)
if all_docs:
ins, _skip = upsert_documents(db, obj_id, all_docs)
extras_counts["documents_rows"] += ins
poisoned = _is_waf_poisoned(results)
# Полностью-WAF объект = ни один extras не отдал JSON (все 403).
all_waf = all(isinstance(r, WafBlockedError) for _k, _u, r in results)
consecutive_all_waf = consecutive_all_waf + 1 if all_waf else 0
if consecutive_all_waf >= _EXTRAS_ABORT_AFTER_CONSEC_WAF:
aborted = True
log_progress(
db,
run_id,
f"extras-пас оборван: {consecutive_all_waf} объектов подряд"
" полностью WAF-403 (extras заблокированы целиком)",
level="warn",
stage="extras",
)
break
# RECYCLE: если сессия отравлена WAF-403 — пересоздаём перед next obj.
if poisoned:
total_requests += sess.request_count
await sess.__aexit__(None, None, None)
sess = _new_session()
await sess.__aenter__()
await sess.warm_up()
if (i + 1) % 10 == 0:
_checkpoint(db, run_id, i + 1)
finally:
total_requests += sess.request_count
await sess.__aexit__(None, None, None)
if aborted:
logger.info("extras isolated pass aborted early — extras WAF-blocked wholesale")
log_progress(
db,
run_id,
f"extras-пас done: sale_graph={extras_counts['sale_graph_rows']}"
f" agg={extras_counts['sales_agg_rows']} infra={extras_counts['infra_rows']}"
f" photos={extras_counts['photos_rows']} docs={extras_counts['documents_rows']}",
stage="extras",
)
return total_requests
# ── orchestrator ─────────────────────────────────────────────────────────────
@ -1660,9 +1930,25 @@ async def run_region_sweep(
photos_dir: Path | str | None = None,
place_override: str | None = None,
resume_from_run_id: int | None = None,
browser_concurrency: int | None = None,
request_jitter_min_ms: int | None = None,
request_jitter_max_ms: int | None = None,
proxy_url: str | None = None,
extras_isolated: bool = True,
) -> dict[str, Any]:
"""Full sweep for one region: bootstrap → fetch objects → per-object processing.
#1945 anti-ban: ``browser_concurrency`` / ``request_jitter_*_ms`` / ``proxy_url``
прокидываются в BrowserSession sweep'а, чтобы троттлить/проксировать ТОЛЬКО KN-
прогон (другие скраперы используют модульные дефолты stealth.py без изменений).
None у каждого = дефолт stealth.py (concurrency=8, jitter 6001500, без прокси).
``extras_isolated`` (#1945 ГЛАВНЫЙ unblock): True (дефолт) → flats тянутся в
ЧИСТОЙ flats-only сессии (extras на ней НЕ дёргаются НИКОГДА 403-яд extras не
травит cookies flats-сессии flats_count восстанавливается), а extras идут
ОТДЕЛЬНЫМ best-effort проходом с recycle сессии на WAF-403. False старое
поведение (flats+extras в одной сессии, poison-prone).
Resume-aware: если `resume_from_run_id` задан, читаем objects_snapshot и
progress_obj_index из исходного run и продолжаем с этого индекса. Новый
run_id создаётся (с resumed_from_run_id ссылкой), исходный остаётся
@ -1802,6 +2088,10 @@ async def run_region_sweep(
headed=headed,
load_state=load_state,
save_state=save_state,
concurrency=browser_concurrency,
jitter_min_ms=request_jitter_min_ms,
jitter_max_ms=request_jitter_max_ms,
proxy_url=proxy_url,
) as sess:
# Warm-up: visit /сервисы/каталог-новостроек/ to obtain WAF cookies
# (___dmpkit___, domain_sid). Required since 2026-05-24 — without these
@ -1881,9 +2171,16 @@ async def run_region_sweep(
# ── Phase B/C — per-object processing (resumable, parallel per-object) ─
# Все endpoint'ы одного obj_id запускаются параллельно через asyncio.gather.
# BrowserSession._sem (Semaphore(_BROWSER_CONCURRENCY)=8) ограничивает запросы.
# BrowserSession._sem (size = browser_concurrency, #1945 троттлит KN до 2)
# ограничивает запросы.
# DB upserts выполняются последовательно после gather — один db Session
# не thread-safe для параллельной записи.
#
# #1945 ИЗОЛЯЦИЯ: при extras_isolated=True эта (warm) сессия делает
# ТОЛЬКО flats — extras на ней НЕ дёргаются (их 403-яд не травит cookies
# flats-сессии → flats_count восстанавливается). extras уходят отдельным
# best-effort проходом ниже (после закрытия flats-сессии) с recycle на
# WAF-403. При extras_isolated=False — старое combined-поведение.
pdir = Path(photos_dir) if photos_dir else PHOTOS_DIR_DEFAULT
total = len(all_objects)
for i in range(start_index, total):
@ -1898,17 +2195,12 @@ async def run_region_sweep(
if fetch_flats:
coros.append(_fetch_flats_safe(sess, obj_id))
if extras:
coros.append(_fetch_sale_graph_safe(sess, obj_id, "apartments"))
coros.append(_fetch_sale_graph_safe(sess, obj_id, "parking"))
coros.append(_fetch_sales_agg_safe(sess, obj_id))
coros.append(_fetch_infrastructure_safe(sess, obj_id))
coros.append(_fetch_photos_safe(sess, obj_id))
coros.append(_fetch_doc_rpd_safe(sess, obj_id))
coros.append(_fetch_doc_developer_report_safe(sess, obj_id))
coros.append(_fetch_doc_project_documentation_safe(sess, obj_id))
coros.append(_fetch_doc_documentation_other_safe(sess, obj_id))
coros.append(_fetch_doc_permits_safe(sess, obj_id))
# extras в combined-сессии ТОЛЬКО когда изоляция выключена —
# иначе они отравят flats (#1945). В isolated-режиме extras
# обрабатываются отдельным проходом ниже.
combined_extras = extras and not extras_isolated
if combined_extras:
coros.extend(_extras_coros(sess, obj_id))
# TODO: obj_checks endpoint not found at /api/object/{id}/checks (404).
# 6 чек-боксов "Проверено на наш.дом.рф" вероятно inline в kn/object payload.
# Re-enable после investigation структуры объекта (separate PR).
@ -1922,64 +2214,28 @@ async def run_region_sweep(
# Sequential upsert — DB session не thread-safe
all_docs: list[dict[str, Any]] = []
_doc_kinds = frozenset(
(
"doc_rpd",
"doc_developer_report",
"doc_project_documentation",
"doc_documentation_other",
"doc_permits",
)
)
for kind_tag, full_url, result in results:
if isinstance(result, Exception):
_classify_and_log(db, run_id, obj_id, kind_tag, full_url, result)
continue
if kind_tag == "flats":
flats_list: list[dict[str, Any]] = result # type: ignore[assignment]
if flats_list:
total_flats += upsert_flats(db, flats_list, snapshot_date, region_code)
elif kind_tag in ("sale_graph_apartments", "sale_graph_parking"):
sg_type = kind_tag.replace("sale_graph_", "")
rows_sg, _ = result # type: ignore[misc]
extras_counts["sale_graph_rows"] += upsert_sale_graph(
db, obj_id, sg_type, rows_sg, snapshot_date
total_flats += _process_flats_result(
db, run_id, obj_id, full_url, result, snapshot_date, region_code
)
else:
await _process_extras_result(
sess,
db,
run_id,
obj_id,
kind_tag,
full_url,
result,
snapshot_date,
extras_counts,
all_docs,
download_photos_binary=download_photos_binary,
pdir=pdir,
)
elif kind_tag == "sales_agg":
agg_data, _ = result # type: ignore[misc]
extras_counts["sales_agg_rows"] += upsert_sales_agg(
db, obj_id, agg_data, snapshot_date
)
elif kind_tag == "infrastructure":
pois_data, _ = result # type: ignore[misc]
extras_counts["infra_rows"] += upsert_infrastructure(
db, obj_id, pois_data, snapshot_date
)
elif kind_tag == "photos":
photos_data, _ = result # type: ignore[misc]
local_paths: dict[str, str] = {}
thumb_paths: dict[str, str] = {}
if download_photos_binary and photos_data:
local_paths, thumb_paths = await download_photos(
sess, obj_id, photos_data, pdir
)
extras_counts["photos_downloaded"] += len(local_paths)
extras_counts["photos_rows"] += upsert_photos(
db, obj_id, photos_data, local_paths, thumb_paths
)
elif kind_tag in _doc_kinds:
# Каждый из 5 doc-endpoint'ов отдаёт свой список документов.
# Накапливаем в all_docs — единый upsert после цикла.
doc_items, _ = result # type: ignore[misc]
all_docs.extend(extract_documents(doc_items or []))
# Единый upsert всех документов объекта после обработки 5 endpoint'ов.
# Единый upsert всех документов объекта после обработки doc-endpoint'ов.
if all_docs:
ins, _skip = upsert_documents(db, obj_id, all_docs)
extras_counts["documents_rows"] += ins
@ -1998,12 +2254,49 @@ async def run_region_sweep(
f" downloaded={extras_counts['photos_downloaded']}"
f" docs={extras_counts['documents_rows']}"
f" checks={extras_counts['checks_rows']}",
stage="extras" if extras else "fetch_flats",
stage="extras" if combined_extras else "fetch_flats",
)
_checkpoint(db, run_id, total)
request_count = sess.request_count
# ── Phase C (isolated) — extras best-effort пас в ОТДЕЛЬНОЙ сессии ──────
# flats-сессия закрыта. Теперь (только если extras=True И extras_isolated)
# тянем extras в disposable-сессиях: на каждый WAF-403 recycle всю
# BrowserSession (свежие cookies/контекст) перед следующим объектом, чтобы
# яд НЕ накапливался. extras сейчас мертвы (403 с 2026-06-03) — пас целиком
# best-effort: любые сбои уходят в kn_scrape_failures и НЕ роняют run.
if extras and extras_isolated:
try:
ex_req = await _run_extras_pass(
db=db,
run_id=run_id,
region_code=region_code,
all_objects=all_objects,
start_index=start_index,
snapshot_date=snapshot_date,
extras_counts=extras_counts,
pdir=pdir,
download_photos_binary=download_photos_binary,
load_state=load_state,
headed=headed,
browser_concurrency=browser_concurrency,
request_jitter_min_ms=request_jitter_min_ms,
request_jitter_max_ms=request_jitter_max_ms,
proxy_url=proxy_url,
)
request_count += ex_req
except Exception as e:
# best-effort: extras-пас НИКОГДА не должен ронять уже собранные flats.
logger.warning("extras isolated pass failed (best-effort): %s", e)
log_progress(
db,
run_id,
f"extras-пас (best-effort) упал: {type(e).__name__}: {str(e)[:200]}",
level="warn",
stage="extras",
)
# ── Phase D — finalize ─────────────────────────────────────────────
db.execute(
text(

View file

@ -33,15 +33,36 @@ Issue #297 sub-task 22c.
from __future__ import annotations
import io
import logging
from datetime import date
from typing import Any
import re
from datetime import UTC, datetime
from pathlib import Path
from typing import TYPE_CHECKING, Any
from urllib.parse import urlsplit
from sqlalchemy import text
from sqlalchemy.orm import Session
if TYPE_CHECKING:
from datetime import date
from app.services.scrapers.stealth import BrowserSession
logger = logging.getLogger(__name__)
# Базовая директория для скачанных бинарников планировок. Зеркалит конвенцию
# domrf_kn.py:PHOTOS_DIR_DEFAULT (data/raw/domrf_photos) — тот же raw-корень,
# отдельная папка. Файлы: {base}/{obj_id}/{safe_ods_id}.{ext}.
FLAT_PLANS_DIR_DEFAULT = Path("data/raw/domrf_flat_plans")
# Разрешённые расширения картинок планировок. png — дефолт (DOM.РФ отдаёт планы
# растром), svg на всякий случай (векторные схемы встречаются).
_ALLOWED_PLAN_EXTS = (".png", ".jpg", ".jpeg", ".webp", ".svg")
# Замена небезопасных для имени файла символов в ods_id (напр. "65136/1/1.4.3").
_ODS_UNSAFE_RE = re.compile(r"[^A-Za-z0-9._-]+")
# ── payload parsing ───────────────────────────────────────────────────────────
@ -212,30 +233,131 @@ def upsert_flat_plans(
return inserted
# ── download stub ─────────────────────────────────────────────────────────────
# ── image download ────────────────────────────────────────────────────────────
def download_plan_image_stub(
def _safe_ods_filename(ods_id: str) -> str:
"""Превратить ods_id (напр. '65136/1/1.4.3') в безопасное имя файла.
Слэши и прочие небезопасные символы '_'. Результат используется как
basename в {base_dir}/{obj_id}/{safe}.{ext}.
"""
return _ODS_UNSAFE_RE.sub("_", ods_id).strip("_") or "plan"
def _ext_from_url(url: str) -> str:
"""Определить расширение файла по URL планировки. Default .png.
DOM.РФ отдаёт планы преимущественно PNG; расширение берём из path, если
оно в белом списке, иначе .png (file-API URL /api/ext/file/... часто без
расширения контент-тип уточнится по факту, но png безопасный дефолт).
"""
path = urlsplit(url).path.lower()
for ext in _ALLOWED_PLAN_EXTS:
if path.endswith(ext):
return ".jpg" if ext == ".jpeg" else ext
return ".png"
def _image_dims(data: bytes) -> tuple[int | None, int | None]:
"""Получить (width_px, height_px) картинки через Pillow. None при ошибке.
Pillow (PIL) уже зависимость репо (pyproject: pillow>=10.4.0, используется в
app.services.photos.thumbs). SVG Pillow не парсит вернёт (None, None),
это ОК (размеры для векторных планов необязательны).
"""
try:
from PIL import Image
with Image.open(io.BytesIO(data)) as im:
return int(im.width), int(im.height)
except Exception as exc:
logger.debug("flat plan image dims parse failed: %s", exc)
return None, None
async def download_plan_image(
session: BrowserSession,
plan_image_url: str,
ods_id: str,
dest_dir: str | None = None,
) -> str:
"""Заглушка для скачивания бинарника планировки.
obj_id: int,
base_dir: Path | None = None,
) -> dict[str, Any] | None:
"""Скачать бинарник планировки квартиры и вернуть метаданные для upsert.
Реальная реализация отдельный Celery task (22d-track, issue #299).
До реализации бросает NotImplementedError, чтобы случайный вызов
не прошёл незаметно.
Реиспользует BrowserSession.download_binary тот же паттерн что download_photos
в domrf_kn.py: тянет файл через browser context (те же cookies / TLS-fingerprint,
что прошли WAF), с лёгким throttle (jitter 200500 мс встроен в download_binary).
Файл сохраняется в {base_dir}/{obj_id}/{safe_ods_id}.{ext}. Если файл уже есть
на диске повторно НЕ качаем (idempotent, экономит WAF-бюджет), но всё равно
возвращаем метаданные (размеры/размер читаются с диска).
Args:
plan_image_url: URL картинки планировки.
session: Активный BrowserSession (bootstrapped + warm_up).
plan_image_url: Абсолютный URL картинки планировки.
ods_id: Идентификатор квартиры (для имени файла).
dest_dir: Директория для сохранения (None = MEDIA_ROOT/flat_plans/).
obj_id: ID объекта DOM.РФ (подпапка).
base_dir: Корень для сохранения (None = FLAT_PLANS_DIR_DEFAULT).
Raises:
NotImplementedError: всегда до реализации.
Returns:
dict с ключами local_path, width_px, height_px, size_bytes, downloaded_at
для передачи в upsert_flat_plans. None при ошибке скачивания.
"""
raise NotImplementedError(
"download_plan_image_stub: скачивание планировок не реализовано. "
"Реальный downloader — отдельный Celery task (issue #299, 22d-track). "
f"URL={plan_image_url!r} ods_id={ods_id!r} dest_dir={dest_dir!r}"
root = base_dir or FLAT_PLANS_DIR_DEFAULT
obj_dir = root / str(obj_id)
ext = _ext_from_url(plan_image_url)
local = obj_dir / f"{_safe_ods_filename(ods_id)}{ext}"
# Idempotent: уже скачан — читаем метаданные с диска, не жжём WAF-запрос.
if local.exists() and local.stat().st_size > 0:
data = local.read_bytes()
width_px, height_px = _image_dims(data)
logger.debug("flat plan already on disk ods_id=%s path=%s", ods_id, local)
return {
"local_path": str(local),
"width_px": width_px,
"height_px": height_px,
"size_bytes": local.stat().st_size,
"downloaded_at": datetime.now(UTC),
}
try:
data = await session.download_binary(plan_image_url)
except Exception as exc:
logger.warning(
"download_plan_image failed ods_id=%s obj_id=%s url=%s: %s",
ods_id,
obj_id,
plan_image_url,
exc,
)
return None
if not data:
logger.warning("download_plan_image empty body ods_id=%s url=%s", ods_id, plan_image_url)
return None
obj_dir.mkdir(parents=True, exist_ok=True)
try:
local.write_bytes(data)
except OSError as exc:
logger.warning("download_plan_image write failed ods_id=%s path=%s: %s", ods_id, local, exc)
return None
width_px, height_px = _image_dims(data)
logger.info(
"download_plan_image ok ods_id=%s obj_id=%s bytes=%d dims=%sx%s",
ods_id,
obj_id,
len(data),
width_px,
height_px,
)
return {
"local_path": str(local),
"width_px": width_px,
"height_px": height_px,
"size_bytes": len(data),
"downloaded_at": datetime.now(UTC),
}

View file

@ -0,0 +1,501 @@
"""Загрузчик реестра РНС/РВЭ Свердловской области из открытого API ГИСОГД-СО (#2367).
Источник: ``gisogd66.midural.ru`` региональная ГИС органов гос. власти Свердловской
области. Открытый JSON-API раздела 13 («Сведения о документах в деле о земельном
участке»), БЕЗ авторизации (bare curl 200, разведка 2026-07-04).
Три эндпоинта (все GET JSON, base ``https://gisogd66.midural.ru/api/v1/{schema}``):
* ``/gisogddocgroups/razdel13`` группы документов раздела (DocRS/DocRV/);
* ``/gisogddocgroupdocs/razdel13/{group}`` весь список группы одним ответом (без
пагинации): key, name, docNum, dateDoc, numReg, dateReg;
* ``/gisogddocs/razdel13/{key}`` карточка документа: spatialUnit (кад.,
может быть несколько через запятую / отсутствовать), infosetGeoJson (GeoJSON Polygon
WGS84 lon/lat, может отсутствовать hasGeometry=false), approvedOrganization, docNum,
dateDoc, dateReg.
Две схемы (тянем обе, sverdregion первым): ``agate_sverdregion`` (регион-агрегат,
стабильные карточки, покрывает и ЕКБ) и ``agate_ekbgo`` (ЕКБ, свежее на ~неделю, но
часть старых ключей битая на портале). Один документ может встретиться в обеих
бизнес-ключ ``(doc_group, doc_num)``; при конфликте предпочитаем запись с более
поздним ``date_reg``.
Инкрементальность: список группы сверяется с БД по (source_schema, source_key, date_reg);
карточки (второй запрос дорогой) тянутся ТОЛЬКО для новых / изменившихся документов.
Politeness: единый httpx-клиент с limits(max_connections=5), timeout=20с, 2 ретрая на
сетевых ошибках. Карточки (5xx-битые записи портала) НЕ ретраятся deterministically
broken, не транзиент. Circuit breaker: 50 карточек подряд с ошибкой прерываем группу
(битый сегмент). UPSERT per-row SAVEPOINT (битая строка не валит батч). Периодический
db.commit() (каждые 200 записей + после каждой schema/group), чтобы партиал пережил
soft_time_limit-kill.
"""
from __future__ import annotations
import json
import logging
import re
import time
from datetime import date, datetime
from typing import Any
import httpx
from sqlalchemy import text
from sqlalchemy.orm import Session
from app.core.db import SessionLocal
logger = logging.getLogger(__name__)
# ── Константы источника ───────────────────────────────────────────────────────
GISOGD_BASE = "https://gisogd66.midural.ru/api/v1"
_SECTION = "razdel13"
# Схемы источника — обе тянем. Порядок ВАЖЕН: sverdregion первым.
# sverdregion (регион-агрегат) отдаёт карточки стабильно и покрывает в т.ч. ЕКБ-доки
# через агрегацию, поэтому он — базовый источник. ekbgo вторым как freshness-энхансер
# (свежее на ~неделю), НО у него большой сегмент старых ключей (префикс ~100001007xxx)
# детерминированно отдаёт 500 на карточках (сломано на стороне портала, 2026-07-04) —
# гоняем его после того, как регион уже покрыл основную массу (см. circuit breaker ниже).
SCHEMAS: tuple[str, ...] = ("agate_sverdregion", "agate_ekbgo")
# Группа источника (doc group key) → наш doc_group-код в БД (CHECK IN ('RS','RV')).
GROUP_CODE: dict[str, str] = {"DocRS": "RS", "DocRV": "RV"}
_HTTP_TIMEOUT = 20.0
_MAX_CONNECTIONS = 5
_MAX_RETRIES = 2 # ретраи (helper делает суммарно до 1 + _MAX_RETRIES попыток)
_RETRY_BACKOFF_S = 1.5
_CARD_SLEEP_S = 0.05 # вежливая пауза между карточками (второй, «дорогой» запрос)
_COMMIT_EVERY = 200 # периодический commit внутри группы (партиал переживает time-limit)
# Circuit breaker: столько карточек ПОДРЯД с ошибкой fetch → прерываем группу (портал
# отдаёт битый сегмент старых ключей детерминированным 5xx; остаток догоним будущими ранами).
_CARD_FAIL_BREAKER = 50
USER_AGENT = "GenDesign/1.0 (+https://gendsgn.ru) Site Finder gisogd permits loader"
# Валидация кадастрового номера: NN:NN:NNNNNN(N):N (6-7 знаков в третьем блоке).
_CAD_RE = re.compile(r"^\d{2}:\d{2}:\d{6,7}:\d+$")
# ── HTTP helpers (module-level — мокаются в тестах) ───────────────────────────
def _make_client() -> httpx.Client:
"""httpx.Client с пулом (max_connections=5), таймаутом и UA. Закрывать вызывающим."""
return httpx.Client(
base_url=GISOGD_BASE,
timeout=_HTTP_TIMEOUT,
follow_redirects=True,
headers={"User-Agent": USER_AGENT},
limits=httpx.Limits(max_connections=_MAX_CONNECTIONS),
)
def _get_json(client: httpx.Client, path: str, *, retries_5xx: int = _MAX_RETRIES) -> Any:
"""GET path → JSON с ретраями на 5xx / сетевых ошибках.
Сетевые Timeout/TransportError ретраятся всегда (до 1 + _MAX_RETRIES попыток)
это транзиент. 5xx ретраятся ``retries_5xx`` раз: списки группы с дефолтом
(транзиентная нагрузка портала), карточки с ``retries_5xx=0`` (5xx карточки
детерминированно битые записи сегмента портала, не транзиент). 4xx сразу raise.
"""
last_exc: Exception | None = None
for attempt in range(_MAX_RETRIES + 1):
try:
resp = client.get(path)
if resp.status_code >= 500:
last_exc = httpx.HTTPStatusError(
f"{resp.status_code} on {path}", request=resp.request, response=resp
)
# 5xx: ретраим только пока attempt в пределах retries_5xx (0 → сразу raise).
if attempt >= retries_5xx:
raise last_exc
logger.warning(
"gisogd66: %s (попытка %d) на %s", resp.status_code, attempt + 1, path
)
time.sleep(_RETRY_BACKOFF_S * (attempt + 1))
continue
resp.raise_for_status()
return resp.json()
except (httpx.TimeoutException, httpx.TransportError) as exc:
last_exc = exc
logger.warning(
"gisogd66: сетевая ошибка (попытка %d) на %s: %s", attempt + 1, path, exc
)
time.sleep(_RETRY_BACKOFF_S * (attempt + 1))
assert last_exc is not None
raise last_exc
def fetch_group_list(client: httpx.Client, schema: str, group: str) -> list[dict[str, Any]]:
"""Весь список документов группы (без пагинации) → list of dict.
Поля элемента: key, name, docNum, dateDoc, numReg, dateReg. Пустой список при
неожиданном (не-list) ответе.
"""
data = _get_json(client, f"/{schema}/gisogddocgroupdocs/{_SECTION}/{group}")
if not isinstance(data, list):
logger.warning("gisogd66: %s/%s список не-list (%s)", schema, group, type(data).__name__)
return []
logger.info("gisogd66: %s/%s список документов=%d", schema, group, len(data))
return data
def fetch_doc_card(client: httpx.Client, schema: str, key: str | int) -> dict[str, Any] | None:
"""Карточка документа → dict (или None при не-dict / ошибке карточки).
Поля: spatialUnit, infosetGeoJson, hasGeometry, approvedOrganization, docNum,
dateDoc, dateReg, name.
5xx карточки НЕ ретраятся (retries_5xx=0): битый сегмент старых ключей портала
отдаёт детерминированный 500 3 попытки с backoff = ~12с потерянного времени/карточку.
"""
data = _get_json(client, f"/{schema}/gisogddocs/{_SECTION}/{key}", retries_5xx=0)
if not isinstance(data, dict):
logger.warning("gisogd66: карточка %s/%s не-dict (%s)", schema, key, type(data).__name__)
return None
return data
# ── Парсинг полей ─────────────────────────────────────────────────────────────
def _parse_date(value: Any) -> date | None:
"""ISO-datetime / date-строка источника → date. None если пусто / не парсится.
Источник отдаёт «2026-06-02T00:00:00» (dateDoc) и «2026-06-19T15:55:11» (dateReg).
"""
if not value:
return None
if isinstance(value, datetime):
return value.date()
if isinstance(value, date):
return value
s = str(value).strip()
if not s:
return None
# Обрезаем таймзону-суффикс 'Z' если вдруг придёт, парсим ISO.
try:
return datetime.fromisoformat(s.replace("Z", "+00:00")).date()
except ValueError:
pass
for fmt in ("%Y-%m-%d", "%d.%m.%Y"):
try:
return datetime.strptime(s[:10], fmt).date()
except ValueError:
continue
logger.debug("gisogd66: не распарсил дату %r", value)
return None
def parse_cad_nums(spatial_unit: Any) -> list[str]:
"""spatialUnit → список валидных кадастровых номеров (может быть несколько через запятую).
Split по запятой, strip, валидация по _CAD_RE. Невалидные (мусор) отбрасываются с
warning-логом. Отсутствие / пусто []. Дубликаты схлопываются (сохраняя порядок).
"""
if not spatial_unit:
return []
out: list[str] = []
seen: set[str] = set()
for part in str(spatial_unit).split(","):
cad = part.strip()
if not cad:
continue
if not _CAD_RE.match(cad):
logger.warning(
"gisogd66: невалидный кадастровый номер %r в spatialUnit — отброшен", cad
)
continue
if cad not in seen:
seen.add(cad)
out.append(cad)
return out
def extract_geojson(card: dict[str, Any]) -> str | None:
"""infosetGeoJson карточки → GeoJSON-строка (для ST_GeomFromGeoJSON) или None.
Источник кладёт GeoJSON КАК СТРОКУ в поле infosetGeoJson. Пусто / hasGeometry=false
/ невалидный JSON None (лоадер запишет geom=NULL). Приведение PolygonMultiPolygon
делается в SQL через ST_Multi здесь возвращаем сырой GeoJSON как есть.
"""
raw = card.get("infosetGeoJson")
if not raw:
return None
if isinstance(raw, dict):
# На случай если источник когда-нибудь отдаст объект, а не строку.
return json.dumps(raw)
s = str(raw).strip()
if not s:
return None
try:
json.loads(s) # sanity-парс: битую строку не отправляем в PostGIS
except (ValueError, TypeError) as exc:
logger.warning("gisogd66: битый infosetGeoJson (%s) — geom=NULL", exc)
return None
return s
# ── Инкрементальность ─────────────────────────────────────────────────────────
def _existing_reg_dates(db: Session, schema: str) -> dict[str, date | None]:
"""{source_key: date_reg} для уже загруженных из этой схемы записей.
Используется для инкрементальной сверки: карточку тянем только если key новый ИЛИ
date_reg в списке отличается от сохранённого.
"""
rows = db.execute(
text("SELECT source_key, date_reg FROM gisogd_permits WHERE source_schema = :schema"),
{"schema": schema},
).all()
return {str(r[0]): r[1] for r in rows}
# ── UPSERT ────────────────────────────────────────────────────────────────────
def _upsert_permit(db: Session, rec: dict[str, Any]) -> str:
"""UPSERT одной записи в gisogd_permits по (doc_group, doc_num). Per-row SAVEPOINT.
Конфликт-резолв: при коллизии бизнес-ключа обновляем ТОЛЬКО если у новой записи
date_reg НЕ старше сохранённой (EXCLUDED.date_reg >= existing) предпочитаем
более позднюю регистрацию (или запись без даты не затирает датированную).
Returns: 'inserted' | 'updated' | 'skipped_unchanged'.
"""
result = db.execute(
text("""
INSERT INTO gisogd_permits
(doc_group, doc_num, doc_name, date_doc, date_reg,
approved_organization, source_schema, source_key,
cad_nums, geom, fetched_at, updated_at)
VALUES (
:doc_group, :doc_num, :doc_name, :date_doc, :date_reg,
:approved_org, :source_schema, :source_key,
CAST(:cad_nums AS text[]),
-- CAST обязателен: без него ни одно вхождение :geojson не даёт Postgres
-- тип параметра (ST_GeomFromGeoJSON перегружена text/json/jsonb)
-- psycopg.errors.AmbiguousParameter на КАЖДОЙ строке (прод-прогон #2367).
CASE WHEN CAST(:geojson AS text) IS NULL THEN NULL
ELSE ST_Multi(ST_SetSRID(
ST_GeomFromGeoJSON(CAST(:geojson AS text)), 4326)) END,
NOW(), NOW()
)
ON CONFLICT (doc_group, doc_num) DO UPDATE
SET doc_name = EXCLUDED.doc_name,
date_doc = EXCLUDED.date_doc,
date_reg = EXCLUDED.date_reg,
approved_organization = EXCLUDED.approved_organization,
source_schema = EXCLUDED.source_schema,
source_key = EXCLUDED.source_key,
cad_nums = EXCLUDED.cad_nums,
geom = EXCLUDED.geom,
updated_at = NOW()
WHERE EXCLUDED.date_reg IS NOT DISTINCT FROM gisogd_permits.date_reg
OR EXCLUDED.date_reg > gisogd_permits.date_reg
OR gisogd_permits.date_reg IS NULL
RETURNING (xmax = 0) AS is_insert
"""),
{
"doc_group": rec["doc_group"],
"doc_num": rec["doc_num"],
"doc_name": rec["doc_name"],
"date_doc": rec["date_doc"],
"date_reg": rec["date_reg"],
"approved_org": rec["approved_organization"],
"source_schema": rec["source_schema"],
"source_key": rec["source_key"],
"cad_nums": rec["cad_nums"],
"geojson": rec["geojson"],
},
).scalar()
if result is None:
# WHERE-гейт не пропустил апдейт (у нас более старый date_reg) → без изменений.
return "skipped_unchanged"
return "inserted" if result else "updated"
# ── Оркестрация ───────────────────────────────────────────────────────────────
def load_gisogd_permits(db: Session | None = None) -> dict[str, int]:
"""Инкрементально загрузить РНС/РВЭ ГИСОГД-СО (обе схемы, обе группы) в gisogd_permits.
Для каждой (schema, group): список сверка с БД по (source_key, date_reg)
карточки только для новых/изменённых UPSERT по (doc_group, doc_num) с
предпочтением позднего date_reg. ``db`` для совместимости сигнатуры; None своя
SessionLocal.
Returns: счётчики {listed, fetched_details, inserted, updated, skipped_unchanged,
no_cad, no_geom, card_failed, group_aborted}.
"""
owns_session = db is None
if db is None:
db = SessionLocal()
counts = {
"listed": 0,
"fetched_details": 0,
"inserted": 0,
"updated": 0,
"skipped_unchanged": 0,
"no_cad": 0,
"no_geom": 0,
"card_failed": 0, # каждая упавшая карточка (5xx / сеть / не-dict)
"group_aborted": 0, # групп прервано circuit breaker'ом (битый сегмент портала)
}
client = _make_client()
try:
for schema in SCHEMAS:
existing = _existing_reg_dates(db, schema)
for group_key, doc_group in GROUP_CODE.items():
try:
items = fetch_group_list(client, schema, group_key)
except Exception as exc:
logger.exception("gisogd66: список %s/%s упал: %s", schema, group_key, exc)
continue
counts["listed"] += len(items)
_sync_group(db, client, schema, doc_group, items, existing, counts)
# Commit после КАЖДОЙ (schema, group), а не один раз в конце: партиал
# должен пережить soft_time_limit-kill (иначе теряем весь прогон).
db.commit()
except Exception as exc:
db.rollback()
logger.exception("gisogd66: outer tx rolled back: %s", exc)
raise
finally:
client.close()
if owns_session:
db.close()
logger.info("load_gisogd_permits done: %s", counts)
return counts
def _sync_group(
db: Session,
client: httpx.Client,
schema: str,
doc_group: str,
items: list[dict[str, Any]],
existing: dict[str, date | None],
counts: dict[str, int],
) -> None:
"""Обработать одну (schema, group): инкрементальная сверка + карточки + UPSERT.
Периодический commit каждые ``_COMMIT_EVERY`` обработанных записей (партиал переживёт
time-limit kill). Circuit breaker: ``_CARD_FAIL_BREAKER`` карточек ПОДРЯД с ошибкой
fetch прерываем группу (битый сегмент портала остаток догоним будущими ранами).
Любая успешная карточка сбрасывает счётчик подряд-фейлов.
"""
processed = 0 # успешно обработанных (upsert/skip) с последнего commit
consecutive_fail = 0 # карточек подряд с ошибкой fetch (для circuit breaker)
for item in items:
key = item.get("key")
if key is None:
continue
source_key = str(key)
list_reg = _parse_date(item.get("dateReg"))
# Инкрементальность: пропускаем карточку, если key уже есть И date_reg совпал.
if source_key in existing and existing[source_key] == list_reg:
counts["skipped_unchanged"] += 1
consecutive_fail = 0 # skip — не фейл карточки, сбрасываем breaker
processed += 1
if processed % _COMMIT_EVERY == 0:
db.commit()
continue
card: dict[str, Any] | None
try:
card = fetch_doc_card(client, schema, source_key)
except Exception as exc:
logger.warning("gisogd66: карточка %s/%s упала: %s", schema, source_key, exc)
card = None
finally:
time.sleep(_CARD_SLEEP_S)
if card is None:
# Неуспех fetch (5xx / сеть / не-dict): считаем и двигаем circuit breaker.
counts["card_failed"] += 1
consecutive_fail += 1
if consecutive_fail >= _CARD_FAIL_BREAKER:
logger.error(
"gisogd66: %s/%s: %d карточек подряд 5xx — прерываю группу "
"(битый сегмент портала)",
schema,
doc_group,
_CARD_FAIL_BREAKER,
)
counts["group_aborted"] += 1
return
continue
consecutive_fail = 0 # успешная карточка сбрасывает breaker
counts["fetched_details"] += 1
rec = _build_record(item, card, schema, doc_group, source_key, list_reg)
if rec is None:
continue
if not rec["cad_nums"]:
counts["no_cad"] += 1
if rec["geojson"] is None:
counts["no_geom"] += 1
try:
with db.begin_nested(): # SAVEPOINT — откат только этой строки
outcome = _upsert_permit(db, rec)
except Exception as exc:
logger.warning("gisogd66: upsert %s %r упал: %s", doc_group, rec["doc_num"], exc)
continue
counts[outcome] += 1
processed += 1
if processed % _COMMIT_EVERY == 0:
db.commit()
def _build_record(
item: dict[str, Any],
card: dict[str, Any],
schema: str,
doc_group: str,
source_key: str,
list_reg: date | None,
) -> dict[str, Any] | None:
"""Список-элемент + карточка → нормализованная запись для UPSERT. None если нет doc_num.
doc_num берём из карточки, fallback на список; strip обязателен (в списке ~25%
номеров с ведущими/хвостовыми пробелами). Даты предпочитаем из карточки, fallback
на список. cad_nums из spatialUnit карточки, geojson из infosetGeoJson.
"""
raw_num = card.get("docNum") or item.get("docNum")
doc_num = str(raw_num).strip() if raw_num else ""
if not doc_num:
logger.warning("gisogd66: карточка %s/%s без docNum — пропуск", schema, source_key)
return None
return {
"doc_group": doc_group,
"doc_num": doc_num,
"doc_name": card.get("name") or item.get("name"),
"date_doc": _parse_date(card.get("dateDoc")) or _parse_date(item.get("dateDoc")),
"date_reg": _parse_date(card.get("dateReg")) or list_reg,
"approved_organization": card.get("approvedOrganization"),
"source_schema": schema,
"source_key": source_key,
"cad_nums": parse_cad_nums(card.get("spatialUnit")),
"geojson": extract_geojson(card),
}
__all__ = [
"GROUP_CODE",
"SCHEMAS",
"extract_geojson",
"fetch_doc_card",
"fetch_group_list",
"load_gisogd_permits",
"parse_cad_nums",
]

View file

@ -19,12 +19,21 @@ searchMap отдаёт в properties **только {id}** — название/
TLS: mkrf.ru использует сертификат, верификация по умолчанию включена.
При SSLError автоматический fallback на verify=False (аналогично ekburg_permits #242).
WAF-hygiene (#2445 item D3): sibling-скрапер без backoff словил бан IP по WAF
DOM.РФ 24.05.2026 (#2443) — okn-mk.mkrf.ru имеет тот же risk shape (гос. CMS,
десятки последовательных запросов без паузы). Идиома пэйсинга/backoff/ретраев
портирована из ``gisogd66.py`` (``_RETRY_BACKOFF_S`` + сон между попытками) и
``objective.py`` (явная обработка 429/Retry-After): rate-limit между запросами
(``_wait_rate_limit``), экспоненциальный backoff на 429/5xx с капом
(``_do_request``/``_do_detail_request``), уважение заголовка Retry-After.
"""
from __future__ import annotations
import logging
import re
import time
import urllib.parse
from typing import Any
@ -42,6 +51,47 @@ _DETAIL_URL = "https://okn-mk.mkrf.ru/Show/Show"
_REQUEST_TIMEOUT = 30.0
_DETAIL_TIMEOUT = 10.0
# ── WAF-hygiene: пэйсинг + backoff (#2445 item D3) ────────────────────────────
# Вежливая пауза между последовательными запросами (мс) — тот же порядок величины,
# что и у objective.py (_DEFAULT_RATE_MS=500) / nspd_client.py (DEFAULT_RATE_MS=600).
_RATE_LIMIT_MS = 400
# Ретраи на 429/5xx (helper делает суммарно до 1 + _MAX_RETRIES попыток).
_MAX_RETRIES = 3
# Экспоненциальный backoff на 429/5xx: base * 2**attempt, капается _MAX_BACKOFF_S
# (аналогично objective.py: 30 * 2**attempt, cap 300s — здесь эндпоинт легче,
# поэтому база/потолок меньше, но идиома та же).
_RETRY_BACKOFF_BASE_S = 2.0
_MAX_BACKOFF_S = 60.0
# process-локальная метка времени последнего запроса (обеих эндпоинтов) —
# rate-limit общий, чтобы 33 list-запроса и до 1500 detail-запросов не долбили
# WAF без пауз даже при чередовании вызовов.
_last_request_ts: float = 0.0
def _wait_rate_limit() -> None:
"""Выдержать паузу ``_RATE_LIMIT_MS`` с последнего запроса к okn-mk.mkrf.ru.
Общий для searchMap и detail обе цели один и тот же WAF-хост.
"""
global _last_request_ts
elapsed_ms = (time.time() - _last_request_ts) * 1000.0
if elapsed_ms < _RATE_LIMIT_MS:
time.sleep((_RATE_LIMIT_MS - elapsed_ms) / 1000.0)
_last_request_ts = time.time()
def _backoff_seconds(attempt: int, retry_after_hdr: str | None) -> float:
"""Секунды ожидания перед повтором после 429/5xx.
Retry-After (если валиден) уважается первым, но капается _MAX_BACKOFF_S
аналогично objective.py. Иначе экспоненциальный backoff от attempt (0-based).
"""
if retry_after_hdr and retry_after_hdr.isdigit():
return min(float(retry_after_hdr), _MAX_BACKOFF_S)
return min(_RETRY_BACKOFF_BASE_S * (2**attempt), _MAX_BACKOFF_S)
# Browser headers -- required: without User-Agent the server returns 403.
# Content-Type включён явно: при content= (raw bytes/str) httpx не выставляет его
# автоматически — нужно прописать вручную. Конфликта с data= нет, т.к. data= не используется.
@ -99,7 +149,7 @@ def fetch_okn_points(address: str, category_type: int) -> list[dict[str, Any]]:
Для PostGIS-вставки: lon=coordinates[1], lat=coordinates[0].
"""
form_data = _build_form_data(address, category_type)
features = _do_request(form_data, verify=True)
features = _do_request(form_data, verify=True, attempt=0)
return features
@ -123,19 +173,37 @@ def _do_request(
form_data: list[tuple[str, str]],
*,
verify: bool,
attempt: int = 0,
) -> list[dict[str, Any]]:
"""Выполнить HTTP POST к searchMap. При SSL-ошибке с verify=True повторяет с verify=False."""
"""Выполнить HTTP POST к searchMap. При SSL-ошибке с verify=True повторяет с verify=False.
WAF-hygiene: rate-limit перед каждой попыткой (``_wait_rate_limit``), на 429/5xx
экспоненциальный backoff и retry до ``_MAX_RETRIES`` раз (аналогично objective.py).
"""
# urllib.parse.urlencode гарантированно кодирует список кортежей с повторяющимися ключами
# (data[0][name]=...&data[0][value]=...&data[1][name]=...). Использование data= с list[tuple]
# конфликтует с ручным Content-Type → httpx не urlencode-ит, бросает
# «expected bytes, tuple found». content= передаёт тело как raw str.
body = urllib.parse.urlencode(form_data)
_wait_rate_limit()
try:
with httpx.Client(timeout=_REQUEST_TIMEOUT, verify=verify, headers=_HEADERS) as client:
resp = client.post(
_SEARCHMAP_URL,
content=body,
)
if resp.status_code == 429 or resp.status_code >= 500:
if attempt < _MAX_RETRIES:
wait = _backoff_seconds(attempt, resp.headers.get("Retry-After"))
logger.warning(
"okn_egrkn_client: HTTP %d от ЕГРКН (попытка %d/%d), retry через %.1fs",
resp.status_code,
attempt + 1,
_MAX_RETRIES,
wait,
)
time.sleep(wait)
return _do_request(form_data, verify=verify, attempt=attempt + 1)
resp.raise_for_status()
except httpx.HTTPStatusError as exc:
logger.error("okn_egrkn_client: HTTP %d от ЕГРКН: %s", exc.response.status_code, exc)
@ -145,7 +213,7 @@ def _do_request(
logger.warning(
"okn_egrkn_client: SSL-ошибка при verify=True, повторяем с verify=False: %s", exc
)
return _do_request(form_data, verify=False)
return _do_request(form_data, verify=False, attempt=attempt)
logger.error("okn_egrkn_client: ошибка подключения к ЕГРКН: %s", exc)
return []
except Exception as exc:
@ -275,17 +343,36 @@ def fetch_okn_detail(source_id: str) -> dict[str, Any]:
"""
if not source_id:
return {}
return _do_detail_request(source_id, verify=True)
return _do_detail_request(source_id, verify=True, attempt=0)
def _do_detail_request(source_id: str, *, verify: bool) -> dict[str, Any]:
"""HTTP GET карточки + парсинг. SSL-fallback аналогичен searchMap."""
def _do_detail_request(source_id: str, *, verify: bool, attempt: int = 0) -> dict[str, Any]:
"""HTTP GET карточки + парсинг. SSL-fallback аналогичен searchMap.
WAF-hygiene: rate-limit перед каждой попыткой (общий с searchMap один WAF-хост),
на 429/5xx экспоненциальный backoff и retry до ``_MAX_RETRIES`` раз.
"""
params = {"id": source_id}
_wait_rate_limit()
try:
with httpx.Client(
timeout=_DETAIL_TIMEOUT, verify=verify, headers=_DETAIL_HEADERS
) as client:
resp = client.get(_DETAIL_URL, params=params)
if resp.status_code == 429 or resp.status_code >= 500:
if attempt < _MAX_RETRIES:
wait = _backoff_seconds(attempt, resp.headers.get("Retry-After"))
logger.warning(
"okn_egrkn_client: detail HTTP %d source_id=%s "
"(попытка %d/%d), retry через %.1fs",
resp.status_code,
source_id,
attempt + 1,
_MAX_RETRIES,
wait,
)
time.sleep(wait)
return _do_detail_request(source_id, verify=verify, attempt=attempt + 1)
resp.raise_for_status()
except httpx.HTTPStatusError as exc:
logger.warning(
@ -299,7 +386,7 @@ def _do_detail_request(source_id: str, *, verify: bool) -> dict[str, Any]:
logger.warning(
"okn_egrkn_client: SSL-ошибка detail с verify=True, retry verify=False: %s", exc
)
return _do_detail_request(source_id, verify=False)
return _do_detail_request(source_id, verify=False, attempt=attempt)
logger.warning("okn_egrkn_client: detail connect error source_id=%s: %s", source_id, exc)
return {}
except Exception as exc:

View file

@ -5,6 +5,12 @@ HTML server-rendered (нет JS-рендеринга), httpx достаточн
TLS: сайт использует российский УЦ verify=False (аналогично ekburg_permits #242).
Politeness (#2445 D4): пауза ``_RATE_S`` перед каждым запросом (search / карточка
документа / PDF) тот же идиом, что nspd_client/objective/gisogd66 (300-500мс). Этот
VPS уже словил WAF-бан от DOM.РФ 24.05.2026 (#2443) за непейсованные последовательные
запросы; pravo.gov66.ru сайт той же формы (гос-портал region, строгий WAF). Ретрай с
backoff на 429/5xx идиом gisogd66._get_json.
Публичный API:
search_documents(query, law_type, since) list[dict]
get_document_pdf_url(doc_id) str | None
@ -15,6 +21,7 @@ from __future__ import annotations
import logging
import re
import time
from typing import Any
import httpx
@ -28,6 +35,14 @@ _SEARCH_URL = f"{_BASE_URL}/search/"
# Тайм-аут щедрый: сервер иногда медленный.
_TIMEOUT = httpx.Timeout(60.0, connect=15.0)
# Politeness pacing (#2445 D4): пауза ПЕРЕД каждым запросом к pravo.gov66.ru — тот же
# идиом (300-500мс), что nspd_client/objective/gisogd66. Не непейсованный WAF-риск,
# аналогичный DOM.РФ-бану #2443.
_RATE_S = 0.4
# Ретраи на 429/5xx: экспоненциальный backoff, суммарно до 1 + _MAX_RETRIES попыток.
_MAX_RETRIES = 2
_RETRY_BACKOFF_S = 2.0
# Числовые коды типов документов (select#id_law_type на сайте).
# Текстовое значение «Постановление» — неверный параметр, сервер не матчит → 0 хитов.
_LAW_TYPE_POSTANOVLENIE = "1" # Постановление Правительства Свердловской области
@ -41,6 +56,43 @@ _RE_PDF_PATH = re.compile(r"/media/pravo/(?!ecp/)[^\s\"']+\.pdf", re.IGNORECASE)
_RE_PDF_ECP_PATH = re.compile(r"/media/pravo/ecp/[^\s\"']+\.pdf", re.IGNORECASE)
def _get_with_backoff(
client: httpx.Client,
url: str,
*,
params: dict[str, str] | None = None,
) -> httpx.Response:
"""GET с politeness-паузой перед запросом + backoff-ретраем на 429/5xx.
Пауза ``_RATE_S`` ПЕРЕД каждой попыткой (в т.ч. первой): пейсинг
последовательных запросов к pravo.gov66.ru, не только ретраев (#2445 D4).
429/5xx ретраятся до ``_MAX_RETRIES`` раз с экспоненциальным backoff;
прочие статусы (включая 4xx кроме 429) сразу ``raise_for_status()``.
"""
last_exc: Exception | None = None
for attempt in range(_MAX_RETRIES + 1):
time.sleep(_RATE_S)
resp = client.get(url, params=params)
if resp.status_code == 429 or resp.status_code >= 500:
last_exc = httpx.HTTPStatusError(
f"{resp.status_code} on {url}", request=resp.request, response=resp
)
if attempt >= _MAX_RETRIES:
raise last_exc
logger.warning(
"pravo_gov66_client: %s (попытка %d) на %s — backoff",
resp.status_code,
attempt + 1,
url,
)
time.sleep(_RETRY_BACKOFF_S * (attempt + 1))
continue
resp.raise_for_status()
return resp
assert last_exc is not None
raise last_exc
def search_documents(
query: str,
law_type: str = _LAW_TYPE_POSTANOVLENIE,
@ -65,8 +117,7 @@ def search_documents(
try:
with httpx.Client(timeout=_TIMEOUT, verify=False, follow_redirects=True) as client:
resp = client.get(_SEARCH_URL, params=params)
resp.raise_for_status()
resp = _get_with_backoff(client, _SEARCH_URL, params=params)
html = resp.text
except Exception as exc:
logger.error("pravo_gov66_client.search_documents: ошибка сети q=%r: %s", query, exc)
@ -117,8 +168,7 @@ def get_document_pdf_url(doc_id: str) -> str | None:
doc_url = f"{_BASE_URL}/{doc_id}/"
try:
with httpx.Client(timeout=_TIMEOUT, verify=False, follow_redirects=True) as client:
resp = client.get(doc_url)
resp.raise_for_status()
resp = _get_with_backoff(client, doc_url)
html = resp.text
except Exception as exc:
logger.error(
@ -151,6 +201,10 @@ def _extract_pdf_url(html: str) -> str | None:
def fetch_pdf(url: str) -> bytes:
"""Загружает PDF по URL.
Пейсинг + backoff на 429/5xx через ``_get_with_backoff`` (#2445 D4) — caller
(reservation_ingest) НЕ должен добавлять свой sleep перед этим вызовом, пауза
уже внутри клиента.
Args:
url: абсолютный URL PDF-файла.
@ -161,8 +215,7 @@ def fetch_pdf(url: str) -> bytes:
httpx.HTTPError: при сетевой ошибке (caller должен обработать gracefully).
"""
with httpx.Client(timeout=_TIMEOUT, verify=False, follow_redirects=True) as client:
resp = client.get(url)
resp.raise_for_status()
resp = _get_with_backoff(client, url)
return resp.content

Some files were not shown because too many files have changed in this diff Show more