Commit graph

199 commits

Author SHA1 Message Date
lekss361
cc03ffa32d fix(velocity): cross-stack contract + NULL safety per pre-push review
3 pre-push code-reviewer findings fixed:

1. TS union extended ('objective'|'sale_graph'|'rosreestr_fallback')
   + UI conditions handle both objective и sale_graph как valid sources.
2. COALESCE(deals_total_vol_m2, deals_total_count * 45.0) — NULL safety
   for DKP-only rows (vol_m2 nullable, count > 0).
3. room_bucket parking filter — verified false positive (все 5 buckets
   apartments: студия/1/2/3/4+).

Refs: PR #157 pre-push code-reviewer
2026-05-15 09:24:32 +03:00
lekss361
a21f440dcf feat(velocity): migrate D2 sales source from stale sale_graph to objective_corpus_room_month
Closes #156
2026-05-15 09:24:32 +03:00
lekss361
c39b53cf53
fix(site-finder): cascade InFailedSqlTransaction — velocity internal rollback (#155)
* fix(site-finder): db.rollback() в velocity catches + SAVEPOINT zoning/success_rec

velocity.py 3 internal try/except возвращали None БЕЗ db.rollback(). PR #154
SAVEPOINT не помог — exception не propagates из velocity. Postgres tx
aborted внутри, cascade InFailedSqlTransaction на следующих queries.

Fix:
- velocity.py: db.rollback() в 3 catch блоках
- parcels.py: SAVEPOINT обёртки для zoning + success_recommendation

Refs: user reports 2026-05-15 cascade 500

* fix(velocity): SAVEPOINT per query (not db.rollback) — correct SQLAlchemy 2.0 pattern

Per PR #155 bot review — мой db.rollback() в catches был НЕПРАВИЛЬНЫЙ.

Session.rollback() = ENTIRE outer tx (не savepoint). Outer SAVEPOINT context
становится orphaned → InvalidRequestError на __exit__.

Correct fix: wrap каждый db.execute в свой with db.begin_nested():
- Failure → __exit__ propagates exception → SAVEPOINT rolls back
- Outer tx остаётся clean
- velocity returns None gracefully → caller продолжает

Все 3 SQL queries в velocity.py теперь SAVEPOINT-wrapped:
- compute_velocity: competitor_query
- compute_velocity: sales_rows query
- _get_ekb_median: median query

db.rollback() removed из всех 3 catches.

Refs: PR #155 bot review

---------

Co-authored-by: lekss361 <claudestars@proton.me>
2026-05-15 08:52:38 +03:00
lekss361
b13c4be0ed
fix(parcels): SAVEPOINT around velocity compute → не abort outer tx (#154)
Production POST /parcels/{cad}/analyze падает с 500:
  sqlalchemy.exc.InternalError: (psycopg.errors.InFailedSqlTransaction)
  current transaction is aborted, commands ignored until end of transaction block

## Root cause

`compute_velocity()` SQL fails (probably на cad's без conkurrentов / sparse
sale_graph data) → exception caught в try/except → НО db.rollback() отсутствует
→ transaction остаётся в aborted state.

Следующая query (_geotech_risk на line 828) пытается выполниться → крашится
с InFailedSqlTransaction.

## Fix

Wrap velocity в `with db.begin_nested()` — SAVEPOINT pattern (consistent
with PR #124 pzz_loader fix). Failure внутри savepoint:
- Rollbacks ТОЛЬКО savepoint
- Outer transaction остаётся clean
- Subsequent queries (_geotech_risk и пр.) работают

Pattern matches feedback_subagent_delegation audit recommendation для
in-loop / per-section exception handling.

## Impact

POST /parcels/{cad}/analyze больше не 500 при velocity failure. Возвращает
`velocity: null` + остальные fields normal.

Refs: user report 2026-05-15 InFailedSqlTransaction

Co-authored-by: lekss361 <claudestars@proton.me>
2026-05-15 08:29:24 +03:00
lekss361
d0b8eda6ad
fix(deploy): source root .env (где POSTGRES_USER) не backend/.env (#153)
PR #151 deploy упал с 'POSTGRES_USER: unbound variable'.

source backend/.env — backend/.env содержит app vars (SCRAPE_ADMIN_TOKEN,
OPENROUTESERVICE_API_KEY и пр.), НО НЕ POSTGRES_USER/PASSWORD/DB.

Postgres credentials живут в /opt/gendesign/.env (root) — оттуда их читает
compose через env_file для postgres service. Это canonical location для
DB creds на этом стеке.

Fix: source .env (root) — corresponds к compose env_file resolution.

После merge → next deploy auto-apply применит pending #89/#91 (90 уже
applied via MCP).

Refs: #150, incident 2026-05-15 deploy #25901378924

Co-authored-by: lekss361 <claudestars@proton.me>
2026-05-15 08:19:17 +03:00
lekss361
af39e95f83
fix(site-finder): SQLAlchemy syntax error :weights::jsonb → CAST(:weights AS jsonb) (#152)
Production POST /weight-profiles → 500 SyntaxError.

## Root cause

SQLAlchemy text() parser трактует ':weights' followed by '::jsonb' двусмысленно:
- ':weights' — named param binding (должен стать %(weights)s)
- '::jsonb' — PG cast operator

Parser НЕ распознаёт :weights как параметр когда сразу следует :: cast.
Result: psycopg видит literal ':weights' в SQL → syntax error.

```
LINE 5: ($1, $2, :weights::jsonb, $3, $4)
                   ^
parameters: {user_id, profile_name, is_default, description}
            ↑ note: 'weights' missing — SQLAlchemy skipped it
```

## Fix

Replace ambiguous :weights::jsonb с CAST(:weights AS jsonb):
- _INSERT (line 102)
- _UPDATE dynamic 'weights = :weights::jsonb' (line 273)

CAST() syntax не имеет ambiguity, SQLAlchemy корректно bind'ит :weights.

## Affected endpoints

- POST /api/v1/admin/site-finder/weight-profiles (create)
- PUT /api/v1/admin/site-finder/weight-profiles/{id} (update with weights)

Refs: user report 2026-05-15

Co-authored-by: lekss361 <claudestars@proton.me>
2026-05-15 08:12:18 +03:00
lekss361
7c43bc5496
feat(infra): auto-apply data/sql/*.sql migrations in deploy pipeline (#150) (#151)
* fix(frontend): apiFetchWithStatus body stream double-read crash

Per user report 2026-05-14: 'Failed to execute text on Response: body stream
already read' при analyze с non-JSON error response.

apiFetchWithStatus делал .json() (consumes stream), потом .text() в catch
(FAILS — stream already consumed). Fetch API: body — ReadableStream,
consumable один раз.

Fix: read text() once → JSON.parse on string. На fail (HTML / non-JSON)
оборачиваем в {detail: rawText}. Никаких double-reads.

Site-finder cad search больше не крашится на error pages.

* feat(infra): auto-apply data/sql/*.sql migrations in deploy pipeline (#150)

Закрывает root cause production 500 на /api/v1/admin/site-finder/weight-profiles
(а также 2 unapplied migrations: #89/#91).

## Why это нужно

После audit обнаружили что:
- backend/alembic/versions/ пустой (Alembic configured но не используется)
- deploy.yml не имеет migration step
- 47+ raw SQL files накопилось без auto-apply
- 3 файла unapplied: 87 (engineering), 89 (drop brin), 90 (weight profiles)

User report: GET weight-profiles → 500 потому что user_weight_profiles
table не существует на prod.

## Changes

### deploy.yml +39 lines
- Path trigger расширен на data/sql/*.sql
- New step 'Apply DB migrations' между compose pull и compose up -d:
  - Idempotent CREATE TABLE _schema_migrations
  - Loop по data/sql/*.sql sorted → skip applied → psql with ON_ERROR_STOP=on
  - Record applied filename в tracking table
  - Exit 1 на failure → containers НЕ обновляются (no partial state)

### NN collision fix
- 87_engineering_networks_191fz.sql → 91_engineering_networks_191fz.sql
  (collision с 87_on_demand_indexes.sql, мой renumber 85→87 был неудачным)

### CLAUDE.md +32 lines
- Subsection 'Auto-apply' под Migration pattern
- Bootstrap procedure docs
- Idempotency requirement
- Failure recovery

### scripts/bootstrap_schema_migrations.sh +100 lines
- One-time seed _schema_migrations с 48 already-applied files
- 3 файла intentionally NOT seeded: 89/90/91 (auto-apply на first deploy)
- Usage docs + verification query

## Vault

domains/infra/Runbook_Auto_Apply_Migrations.md NEW
infra-MOC.md updated

## Required manual steps ДО merge

1. SSH gendesign (tunnel)
2. POSTGRES creds export
3. bash scripts/bootstrap_schema_migrations.sh — seed 48 already-applied files
4. Verify: SELECT COUNT(*) FROM _schema_migrations → 48
5. После merge → first deploy auto-apply #89/#90/#91 → 500 closes

Closes #150

---------

Co-authored-by: lekss361 <claudestars@proton.me>
2026-05-15 08:06:54 +03:00
lekss361
142ed95d36
fix(frontend): apiFetchWithStatus body stream double-read crash (#149)
Per user report 2026-05-14: 'Failed to execute text on Response: body stream
already read' при analyze с non-JSON error response.

apiFetchWithStatus делал .json() (consumes stream), потом .text() в catch
(FAILS — stream already consumed). Fetch API: body — ReadableStream,
consumable один раз.

Fix: read text() once → JSON.parse on string. На fail (HTML / non-JSON)
оборачиваем в {detail: rawText}. Никаких double-reads.

Site-finder cad search больше не крашится на error pages.

Co-authored-by: lekss361 <claudestars@proton.me>
2026-05-15 07:39:21 +03:00
lekss361
088c1ab2cf
feat(site-finder): VelocityBlock в MarketTab (#34 sub-PR 2/2 FINAL) (#147)
* feat(site-finder): VelocityBlock в MarketTab (#34 sub-PR 2/2 FINAL)

D2 velocity-score frontend: score gauge, period meta, top 5 competitors table.
TS types (Velocity/VelocityCompetitor/VelocityPeriod) добавлены.
Guard "velocity" in data для backward-compat pre-#146.

tsc 0 errors, ESLint 0 warnings.

Closes #34

* fix(site-finder): align VelocityBlock TS types with backend response shape

Per #147 bot review — cross-stack contract mismatch:

Backend velocity.py (PR #146 merged) returns:
  obj_id, name, dev_name, obj_class, district_name, distance_m, total_sqm_period

Frontend types had:
  obj_id, name, class, distance_m, total_sqm

Result: c.class undefined → never showed class chip; c.total_sqm undefined →
Math.round(NaN) = NaN, 'не число' в table column для всех 5 строк.

## Fix

- VelocityCompetitor.class → obj_class
- VelocityCompetitor.total_sqm → total_sqm_period
- Added optional dev_name, district_name fields
- Updated VelocityBlock.tsx — все 3 references (guard, chip render, table cell)

Refs: #34, #147

---------

Co-authored-by: lekss361 <claudestars@proton.me>
2026-05-15 01:44:16 +03:00
lekss361
e93bb6121e
feat(site-finder): D2 velocity-score from domrf_kn_sale_graph (#34 sub-PR 1/2) (#146)
compute_velocity service queries competitor sales в радиусе 3км:
- ST_MakePoint(longitude, latitude) — domrf_kn_objects не имеет geom column
- JOIN domrf_kn_sale_graph за 6 мес (area_sq primary, realised*45 fallback)
- Normalize vs ЕКБ-wide median → velocity_score 0..1
- confidence: high/medium/low (competitors_count + months_observed)
- Top 5 sample competitors для UI

Integration: analyze_parcel.response['velocity'] top-level field.

Schema corrections vs spec:
- obj_name → comm_name
- region_code → region_cd
- contracted (INT) → area_sq (м²)

Tests: 102/102 pass.
Vault: Module_Velocity_Service.md NEW.

Closes #144 (sub-PR 2 frontend закроет #34)

Co-authored-by: lekss361 <claudestars@proton.me>
2026-05-15 01:24:53 +03:00
lekss361
95647a1c82
feat(site-finder): G5 gate verdict banner in OverviewTab (#32 sub-PR 2/2 FINAL) (#143)
GateVerdictBanner с 4 visual states + expand/collapse details.
Types: GateVerdict/GateBlocker/GateWarning/GateVerdictSource/GateVerdictLabel.
Renders at top of OverviewTab driven by backend aggregator (#142).

tsc 0 errors, ESLint 0 warnings.

Vault: code/modules/Module_Gate_Verdict_Banner.md NEW.

Closes #32

Co-authored-by: lekss361 <claudestars@proton.me>
2026-05-15 01:09:34 +03:00
lekss361
76a88b2827
feat(site-finder): gate verdict aggregator — can-build-MKD signal (#32 G5 sub-PR 1/2) (#142)
Pure-function aggregator collapses nspd_zoning / nspd_zouit_overlaps /
nspd_engineering_nearby / nspd_dump в один GateVerdict TypedDict.

Logic:
- ПЗЗ не Ж-* → BLOCKER
- ЗОУИТ sub=17 (инжен. охранная) → BLOCKER
- Другие ЗОУИТ → WARNING
- Нет инжен. сетей в 200м → WARNING
- nspd_dump stale → source: nspd_dump_partial
- Нет dump → can_build_mkd: 'unknown'

Integration: новое поле gate_verdict в analyze_parcel response.

Tests: 15/15 pass (mock-based).

Vault: code/modules/Module_Gate_Verdict.md NEW.

Closes #140 (sub-PR 2 frontend закроет #32)

Co-authored-by: lekss361 <claudestars@proton.me>
2026-05-15 00:59:53 +03:00
lekss361
b42c9dd8f5
feat(site-finder): weight profile panel + analyze options (#114 sub-PR 4/4 FINAL) (#139)
Closing FINAL sub-PR for #114 Custom POI weights — UI complete.

- weightProfiles.ts: TS types + 4 TanStack Query hooks + 11 categories
- WeightProfilePanel.tsx: collapsible panel, sliders, save dialog
- useSiteAnalysis.ts: accepts AnalyzeOptions, query string for backend
- page.tsx: lifts weights state, integrates panel

Approach: 'save first' для ephemeral (no temp profile creation).

tsc 0 errors, ESLint 0 warnings.

Vault: Module_Weight_Profiles_Frontend.md NEW.

Closes #114

Co-authored-by: lekss361 <claudestars@proton.me>
2026-05-15 00:50:02 +03:00
lekss361
7250aa6187
feat(site-finder): weight profiles endpoints + analyze integration (#114 sub-PR 3/4) (#138)
Admin CRUD router + non-breaking analyze_parcel integration. 46 tests pass.

Refs: #114

Co-authored-by: lekss361 <claudestars@proton.me>
2026-05-15 00:40:06 +03:00
lekss361
5aea78c2d8
feat(site-finder): weight profiles service — Pydantic + CRUD (#114 sub-PR 2/4) (#137)
Per #114 sub-PR 2/4. Pydantic v2 models + CRUD service. 14 passing tests.

API: list/get/get_default/create/update/delete_profile + resolve_weights
fallback to system defaults.

Validation: ALLOWED_CATEGORIES guard + weight bounds [-2.0, 3.0].
Default-uniqueness в одной transaction (job_settings.py pattern).

Vault: code/modules/Module_Weight_Profiles_Service.md.

Refs: #114

Co-authored-by: lekss361 <claudestars@proton.me>
2026-05-15 00:29:37 +03:00
lekss361
f3f1e7f6bc
feat(db): schema user_weight_profiles (#114 sub-PR 1/4 — foundation) (#136)
Per #114 Custom POI weights — Макс feedback "садики и Мегамарт должны
иметь разные веса". Sub-PR 1/4 per feedback_split_big_issues:

1. **#114 sub-PR 1 (this)**: schema migration
2. #114 sub-PR 2: backend Pydantic models + CRUD service
3. #114 sub-PR 3: POST /parcels/{cad}/analyze?profile= override + admin endpoint
4. #114 sub-PR 4: frontend 25 sliders UI + live preview

## Schema

`data/sql/90_user_weight_profiles.sql`:

- Table `user_weight_profiles` — SERIAL PK + user_id + profile_name +
  weights JSONB + is_default + timestamps
- Unique constraint (user_id, profile_name)
- B-tree index user_idx
- Partial index `WHERE is_default = TRUE` (fast default lookup)
- Trigger auto-updated_at via `set_updated_at()` function

## Compat with existing POI categories

`weights` JSONB compatible с current `_POI_WEIGHTS` const в
`backend/app/api/v1/parcels.py`:
- school 1.5, kindergarten 1.5, pharmacy 0.8, hospital 0.6
- shop_mall 1.2, shop_supermarket 1.0, shop_small 0.5
- park 1.8, bus_stop 0.3, metro_stop 1.5, tram_stop -0.5

## Idempotency

- BEGIN/COMMIT обёртка
- IF NOT EXISTS на CREATE TABLE/INDEX
- DROP TRIGGER IF EXISTS → CREATE TRIGGER (safe re-run)
- CREATE OR REPLACE FUNCTION (idempotent)

## Apply

```bash
psql postgresql://gendesign:...@localhost:15432/gendesign \
  -f data/sql/90_user_weight_profiles.sql
```

После apply таблица inert (нет backend code yet). Sub-PR 2 добавит CRUD.

## Vault

`code/schemas/Schema_User_Weight_Profiles.md` — created (status:
ready_to_apply, full sub-PR plan).

Refs: #114

Co-authored-by: lekss361 <claudestars@proton.me>
2026-05-15 00:19:33 +03:00
lekss361
6a256bfc1f
refactor(workers): split celery_app.py god-object into 3 modules (#135)
Per audit batch #127 P3 god objects (issue #134).

Split layout:
- celery_app.py: 270 → 35 lines (config only)
- beat_schedule.py: 207 lines NEW (DB-driven + fallback + hardcoded)
- lifecycle.py: 168 lines NEW (worker_process_init + worker_ready handlers)

No business logic changes — move-only structural refactor. Public API
(celery_app.conf.beat_schedule) unchanged.

Verify:
- beat keys: kn-region-66, objective_sync, refresh-ekb-districts-medians,
  nspd-geo-zombie-cleanup, poi-sync-weekly, noise-sync-weekly
- ruff + ruff-format clean
- AST parse OK

Vault: Module_Beat_Schedule + Module_Worker_Lifecycle NEW;
Module_Celery_App updated.

Closes #134
Refs: #127

Co-authored-by: lekss361 <claudestars@proton.me>
2026-05-15 00:09:38 +03:00
lekss361
2416ed0db1
refactor(frontend): extract SectionLabel + EmptyState + adminStyles shared (#132)
* fix(docs): block javascript: URLs in renderMarkdown href (XSS guard)

Per audit batch #127 P2 security minor (issue #130).

## Risk

`renderMarkdown` substituted `[text](url)` → `<a href="...url...">` verbatim
без URL validation. Edge case: `[click](javascript:alert(1))` →
`<a href="javascript:alert(1)">click</a>` execute'нется при click.

Currently low risk (markdown source = build-time fs.readFileSync из
public/docs/, checked into repo), но защита-в-глубину: если когда-либо
markdown source станет user-supplied, vuln materialize.

## Fix

`frontend/src/app/docs/b2b-channels/renderMarkdown.ts`:

1. New `safeUrl(url)` function — allowlist `https://`, `http://`, `/`, `#`,
   `mailto:`. Everything else → `"#"`.
2. Link regex replacement switched from backreference string to callback
   so URL passes через `safeUrl` ДО injection в href.
3. URL unescape → safeUrl → re-escape (HTML escape applied per-line).

## Test cases verified

- `[ok](https://example.com)` → href=`https://example.com` 
- `[ok](/local)` → href=`/local` 
- `[ok](#anchor)` → href=`#anchor` 
- `[ok](mailto:foo@bar.com)` → href=`mailto:foo@bar.com` 
- `[bad](javascript:alert(1))` → href=`#` 
- `[bad](data:text/html,...)` → href=`#` 
- `[bad](vbscript:msgbox(1))` → href=`#` 

## Checks

- tsc: 0 errors
- lint: 0 warnings
- No existing `__tests__/renderMarkdown` to break

## Vault

`fixes/Bug_RenderMarkdown_JavascriptUrl_May14.md` — created.

Closes #130

* refactor(frontend): SectionLabel + EmptyState + adminStyles shared (fixup)

Per audit batch #127 P2 hygiene (issue #129). Previous commit лошил
файлы из-за pre-commit (видимо). Fixup-коммит с actual diff:

## New shared modules (3)

- frontend/src/components/ui/SectionLabel.tsx
- frontend/src/components/ui/EmptyState.tsx
- frontend/src/lib/adminStyles.ts

## Replacements

- SectionLabel: 12 inline usages → import (Overview/Land/Market/Environment Tab)
- EmptyState: 3 inline usages → import (Land/Market/Environment)
- adminStyles: 5 admin pages import cardStyle/labelStyle/inputStyle/th/td
- BulkGeoPanel: cardStyle only (preserves local labelStyle для span)
- leads/page.tsx: cardStyle с marginTop extend; th/td/inputStyle local (divergent)

## Visual regression: ZERO

All px/colors/CSS properties copied verbatim. Files с divergent values left local.

## Checks

- ruff (no files)
- prettier
- tsc + lint clean (per agent run)

Closes #129
Refs: #127

* docs(adminStyles): JSDoc warning про divergent local overrides

Per bot review #132 non-blocking minor — задокументировать почему
leads/page.tsx и BulkGeoPanel.tsx имеют local overrides, чтобы будущий
читатель не «унифицировал» обратно по ошибке.

Refs: #129, #132

---------

Co-authored-by: lekss361 <claudestars@proton.me>
2026-05-14 23:55:38 +03:00
lekss361
389fd31fad
refactor(admin): extract _check_token into shared AdminTokenAuth dep (#133)
Per audit batch #127 P2 hygiene (issue #128).

## Problem

Identical `_check_token` function в 3 admin files:
- backend/app/api/v1/admin_scrape.py:37-44
- backend/app/api/v1/admin_leads.py:26-33
- backend/app/api/v1/admin_jobs.py:24-31

## Fix

New `backend/app/core/deps.py` с `verify_admin_token` + Annotated alias:
```python
AdminTokenAuth = Annotated[None, Depends(verify_admin_token)]
```

Заменено в 27 endpoints (18 admin_scrape + 6 admin_leads + 3 admin_jobs):
- Removed local `_check_token` definitions
- `_: None = Depends(_check_token)` → `_: AdminTokenAuth`
- Removed unused imports (`Header`, `HTTPException`, `settings` где не используется)

## Semantics preserved

Все 3 original implementations были identical (503 if env missing + 401 if
mismatch, same header `X-Admin-Token`, same env `SCRAPE_ADMIN_TOKEN`).

Минор: 503 detail text unified to `"admin disabled — set SCRAPE_ADMIN_TOKEN"`
(в оригинале три разные строки — `"admin scrape disabled..."` etc).
Cosmetic — detail в HTTP 503 не парсится клиентским кодом.

## Tests

`uv run pytest tests/ -x -k admin` → 64/64 passed.

## Vault

`code/patterns/Pattern_Admin_Token_Dependency_May14.md` — created.

Closes #128
Refs: #127

Co-authored-by: lekss361 <claudestars@proton.me>
2026-05-14 23:45:44 +03:00
lekss361
8647114b22
fix(docs): block javascript: URLs in renderMarkdown href (XSS guard) (#131)
Per audit batch #127 P2 security minor (issue #130).

## Risk

`renderMarkdown` substituted `[text](url)` → `<a href="...url...">` verbatim
без URL validation. Edge case: `[click](javascript:alert(1))` →
`<a href="javascript:alert(1)">click</a>` execute'нется при click.

Currently low risk (markdown source = build-time fs.readFileSync из
public/docs/, checked into repo), но защита-в-глубину: если когда-либо
markdown source станет user-supplied, vuln materialize.

## Fix

`frontend/src/app/docs/b2b-channels/renderMarkdown.ts`:

1. New `safeUrl(url)` function — allowlist `https://`, `http://`, `/`, `#`,
   `mailto:`. Everything else → `"#"`.
2. Link regex replacement switched from backreference string to callback
   so URL passes через `safeUrl` ДО injection в href.
3. URL unescape → safeUrl → re-escape (HTML escape applied per-line).

## Test cases verified

- `[ok](https://example.com)` → href=`https://example.com` 
- `[ok](/local)` → href=`/local` 
- `[ok](#anchor)` → href=`#anchor` 
- `[ok](mailto:foo@bar.com)` → href=`mailto:foo@bar.com` 
- `[bad](javascript:alert(1))` → href=`#` 
- `[bad](data:text/html,...)` → href=`#` 
- `[bad](vbscript:msgbox(1))` → href=`#` 

## Checks

- tsc: 0 errors
- lint: 0 warnings
- No existing `__tests__/renderMarkdown` to break

## Vault

`fixes/Bug_RenderMarkdown_JavascriptUrl_May14.md` — created.

Closes #130

Co-authored-by: lekss361 <claudestars@proton.me>
2026-05-14 23:39:47 +03:00
lekss361
8a266502a1
fix(sql): wrap 73_complexes_master.sql in BEGIN/COMMIT (atomic apply) (#125)
Per code review audit (May 14): файл без transaction wrapper — backfill CTE
INSERTs + view recreations executed как individual auto-commit statements.
Mid-file crash → orphan rows в complexes/complex_sources с no rollback path.

## Fix

`BEGIN;` после header comments (line 31, перед section 1).
`COMMIT;` в конце файла после последнего COMMENT ON VIEW.

Wraps все 6 секций atomically:
1-2: CREATE TABLE + CREATE INDEX для complexes + complex_sources
3-5: 3 backfill CTE INSERTs с ON CONFLICT DO NOTHING
6: DROP VIEW IF EXISTS + CREATE VIEW + COMMENT для v_complex_full

## No CONCURRENTLY split needed

Проверил — все 10 indexes используют plain `CREATE INDEX IF NOT EXISTS` без
CONCURRENTLY keyword. Безопасно обернуть в одну tx.

## Idempotency preserved

Все existing guards остаются:
- CREATE TABLE/INDEX IF NOT EXISTS
- ON CONFLICT (source, source_id) DO NOTHING
- DROP VIEW IF EXISTS

Re-running file двукратно — safe.

## Vault

`fixes/Migration_73_Complexes_Atomic_May14.md` — created (status: resolved).

Co-authored-by: lekss361 <claudestars@proton.me>
2026-05-14 23:29:28 +03:00
lekss361
a3e7d21e3a
fix(pzz_loader): replace bare db.rollback() with SAVEPOINT per-row (#124)
Per code review audit (May 14): transaction correctness bug в
`sync_pzz_zones_to_db()` — inner `db.rollback()` в loop сбрасывал outer
transaction на per-row failures → counts inserted/updated НЕ matched DB state.

## Fix

Wrap UPSERT в loop через `db.begin_nested()` (SAVEPOINT):
- Failure одной row → rollback только её savepoint, outer tx alive
- Successfully inserted rows accumulate в outer tx → commit final
- Outer try/except добавлен для unexpected error → rollback + log + raise
- `finally: db.close()` preserved

Pattern consistent с `domrf_kn.py` (где SAVEPOINT уже работает).

## Preserved

- Function signature + return shape {fetched, inserted, updated, skipped} — unchanged
- Logging contracts
- skipped count теперь корректно включает DB-failure rows (consistent with prior behavior)

## Tests

- No test_pzz_loader.py — skipped
- ruff + AST passed

## Vault

`fixes/Bug_Pzz_Loader_Missing_Savepoint_May14.md` — created (status: resolved).

Co-authored-by: lekss361 <claudestars@proton.me>
2026-05-14 23:29:23 +03:00
lekss361
8dfab7613d
fix(nspd-geo): replace f-string bulk INSERT with parametrized batch execute (#123)
Per code review audit (May 14): real SQL injection vector в `enqueue_geo_job`.
Manual `chr(39)*2` quote escape НЕ injection-safe — не защищает от backslash
escape, Unicode quote tricks. Если cad_num когда-либо придёт из user input —
exploitable.

## Change

`backend/app/workers/tasks/nspd_geo.py:298-310` — заменил f-string VALUES
concatenation на SQLAlchemy `text(...)` + list of dicts (psycopg v3 handles
batch executemany под капотом).

Pattern consistent с остальными INSERT'ами в этом же файле (_log,
_start_job, _save_quarter, _save_parcel, _save_building).

## Preserved

- ON CONFLICT (job_id, cad_num, thematic_id) DO NOTHING — verbatim
- Return value (int(job_id))
- Surrounding tx/commit/close logic
- DB schema, function signature, caller interface — unchanged

## Tests

- No test_nspd_geo.py существует — skipped
- ruff check passed

## Vault

`fixes/Bug_Nspd_Geo_Sql_Injection_May14.md` — created (status: resolved).

No backward compat / migration concerns — pure in-place query rewrite.

Co-authored-by: lekss361 <claudestars@proton.me>
2026-05-14 23:29:12 +03:00
lekss361
7b32edc34d
fix(db): DROP dead BRIN index rosreestr_deals_period_brin (#122)
Per code review audit (May 14): BRIN index на partitioned parent table dead
weight. PG16 НЕ propagates такие индексы на child partitions; pg_stat
показывает idx_scan=0 → никогда не использовался.

## Why dead

rosreestr_deals — yearly partitioned по period_start_date. Queries делают
partition pruning через partition key. BRIN index на parent существует, но:
1. На child partitions его нет (PG16 не reflects)
2. Parent сам не имеет данных (partitioned out)
3. Queries никогда не trigger BRIN scan

Maintenance overhead (storage, autoanalyze) без value.

## Migration

`data/sql/89_drop_dead_brin_rosreestr_deals.sql` — атомарный DROP с idempotent
`IF EXISTS`. Apply order: после 88_*.

## Cleanup

`data/sql/01_schema_rosreestr_deals.sql:65-67` — закомментировал оригинальное
CREATE с note: fresh DB bootstrap НЕ создаст index заново.

## Rollback path

Если в будущем нужен range scan optimization — re-create per-partition:
```sql
CREATE INDEX rosreestr_deals_2024q1_period_brin
  ON rosreestr_deals_2024q1 USING BRIN (period_start_date);
-- per-partition × N partitions
```

Но обычно partition pruning достаточно. Измерь EXPLAIN перед re-creation.

## Vault

`fixes/Migration_Drop_Dead_Brin_May14.md` создан (status: ready_to_apply).

Apply на prod: `psql ... -f data/sql/89_drop_dead_brin_rosreestr_deals.sql`
после merge.

Co-authored-by: lekss361 <claudestars@proton.me>
2026-05-14 23:29:01 +03:00
lekss361
f5794173de
fix(celery): disable nspd-harvest-stale-quarters beat (Bug post-mortem #1) (#121)
Per fixes/Bug_NSPD_WMS_NotBulk_2026_May14 action item #1: отключить beat
schedule `nspd-harvest-stale-quarters` до Sprint 2 fix (grid sampling rewrite).

## Why

harvest_quarter в текущей реализации пишет почти пустые dumps в
nspd_quarter_dumps из-за single-pixel WMS GetFeatureInfo bug (детали в
vault post-mortem). Beat запуск Mon 04:00 МСК (next: 2026-05-18 04:00 МСК):

- Потратит rate-limit budget на ~12k EKB кварталов
- Заполнит nspd_quarter_dumps мусорными rows
- analyze_parcel начнёт отдавать пустые nspd_zoning/nspd_zouit_overlaps/
  nspd_engineering_nearby для всех пользователей (через quarter_dump_lookup)

## What changed

`backend/app/workers/celery_app.py` — закомментирован `schedule[
"nspd-harvest-stale-quarters"]` блок (~10 строк) с inline-комментом
explanation + ссылкой на vault post-mortem.

## What НЕ changed

- `backend/app/workers/tasks/nspd_sync.py` — task code (harvest_quarter,
  harvest_stale_quarters) остаётся, можно re-enable после Sprint 2 fix
- Admin endpoint для manual trigger остаётся работающим
- Все остальные beat tasks (poi-sync, noise-sync, zombie-cleanup,
  refresh-ekb-districts-medians, kn-region-*) — не тронуты

## Vault

- fixes/Bug_NSPD_WMS_NotBulk_2026_May14.md — status `identified` → `mitigated`
  (уже обновлено через vault audit pass)

## Re-enable

После Sprint 2 grid-sampling rewrite (см. Bug post-mortem → Sprint 2
fix-strategy) — uncomment блок + verify через staging.

Co-authored-by: lekss361 <claudestars@proton.me>
2026-05-14 23:24:53 +03:00
lekss361
b3f32ae927
chore(infra): correct frontend healthcheck + tighten depends_on (re-apply #117) (#120)
Forward-fix после incident May 14 (см. vault events/Incident_Prod_Down_May14_2026).
PR #117 был полностью reverted PR #119 (Caddy depends_on frontend: service_healthy
+ failing wget --spider /  → Caddy не стартовал → vsё прод down ~25 мин).

Этот PR делает то же что планировал #117, но корректно:

### Frontend healthcheck — TCP probe через node (НЕ HTTP)

```yaml
test: ["CMD", "node", "-e",
  "require('net').createConnection({port:3000,host:'127.0.0.1'})
   .once('connect',function(){process.exit(0)})
   .once('error',function(){process.exit(1)})"]
start_period: 60s
retries: 6
interval: 15s
```

- НЕ зависит от HTTP status (200/404/308 — все OK если порт слушает)
- `node` гарантированно в alpine image (запускает server.js)
- `127.0.0.1` — никакой DNS ambiguity (busybox alpine localhost IPv6 vs IPv4)
- 60s + 6×15s = 150s window — с запасом на Next.js standalone cold start

### Caddy depends_on — backend healthy, frontend started (НЕ healthy)

```yaml
caddy:
  depends_on:
    backend: { condition: service_healthy }    # backend имеет /health endpoint
    frontend: { condition: service_started }   # safety net — НЕ блокирует Caddy
```

Lesson from incident: frontend health flaky (Next.js cold start variable).
Strict service_healthy для frontend → одна misconfig = full outage обоих доменов
(gendsgn.ru + obsidian.gendsgn.ru через тот же Caddy на shared network).

### backend/worker/beat redis: service_started → service_healthy

Redis уже имеет `redis-cli ping` healthcheck. Tightening безопасно — был
artifact pre-healthcheck Redis.

### Pin rosreestr2coord>=5.0.0 (lock уже 5.3.3)

v5 убрала delay параметр; код адаптирован (rate limit на worker уровне).
Защита от silent downgrade при `uv lock --upgrade`.

### Что НЕ повторено из #117

- Caddy depends_on frontend: service_healthy — заменено на service_started
- wget --spider / — заменено на node TCP probe

### Vault обновлён

- `events/Incident_Prod_Down_May14_2026.md` — постмортем + timeline + lessons
- Update Changelog в `domains/infra/infra-MOC.md` (если есть)

Closes incident. Re-applies #117 goals safely.

Co-authored-by: lekss361 <claudestars@proton.me>
2026-05-14 23:24:47 +03:00
lekss361
6ea2dcd46c
Revert "chore(infra): healthcheck frontend + tighten depends_on; pin rosreestr2coord>=5 (#117)" (#119)
This reverts commit bade2f7772.

Co-authored-by: lekss361 <claudestars@proton.me>
2026-05-14 23:02:58 +03:00
lekss361
d4cb56d093
fix(sql): renumber engineering_networks 85→87 + dedup classid + add SRID 4326 (#118)
* fix(sql): renumber engineering_networks 85→87, dedup classid, add SRID

Three critical fixes на untracked 85_engineering_networks_191fz.sql per
code review (May 14) — до первого применения на prod.

1. **NN collision fix**: renumber 85_engineering_networks_191fz.sql → 87_
   85_pzz_zones_ekb.sql уже занимает 85_ prefix. При alphabetic deploy
   order `85_e*` применился бы ДО `85_p*` — но 87_ ставит нас после
   86_v_bucket_success_score.sql без зависимости на 85_pzz.

2. **Duplicate classid '907015504'**:
   - Был дважды: 'Опора металлическая' + 'Опора железобетонная'.
   - Второй ON CONFLICT DO UPDATE молча перетёр бы первый → NSI код
     металлической опоры терялся бы.
   - Fix: 907015504 остаётся за металлической (lower-numbered subtype
     per Минстрой NSI source), железобетонная переезжает на 907015507
     (заполняет gap 906→508). Помечено comment-меткой 'verify with
     Минстрой NSI table' до prod deploy.

3. **Untyped geometry**:
   - Было: `geom GEOMETRY` без SRID constraint → разрешало бы insert
     mix MSK-66 (srid=100000) и WGS84 (srid=4326) → silent ST_DWithin
     corruption.
   - Fix: `geom GEOMETRY(Geometry, 4326)`. Capital G позволяет
     mixed Point/LineString/Polygon, SRID 4326 жёстко фиксирует WGS84.
     ETL loader должен делать ST_Transform(msk66_raw, 4326) перед
     insert (см. backend/app/core/crs.py для CRS utility).

Idempotency: BEGIN...COMMIT, IF NOT EXISTS, ON CONFLICT DO UPDATE
паттерн — уже корректные в файле.

Closes Day-1 critical #2 from code review audit (May 14).
Pre-deploy: verify 907015507 classid assignment against Минстрой
official NSI table; reassign and re-run migration if needed.

* fix(sql): remove provisional 907015507, fix Фонати→Фонари typo

Addresses review blocker on PR #118:

- Provisional classid 907015507 ('Опора железобетонная (столб)') убран
  из seed insert. Commented out с TODO(NSI-verify) — следующий шаг
  верифицировать против официальной NSI-таблицы Минстроя ДО восстановления
  строки. Без верификации ON CONFLICT DO UPDATE name перетёр бы корректное
  значение в eng_ref_classid.
- Опечатка 'Фонати электрические' → 'Фонари электрические' (907015602).
- Sort: 904015504/505/506/508 теперь в численном порядке (после удаления
  провизорной 507).

Live seed теперь покрывает 504 (металл столб), 505 (металл ферма),
506 (металл фермовый столб), 508 (железобет ферма). Строка 507
restored after Минстрой NSI verification (см. follow-up).

---------

Co-authored-by: lekss361 <claudestars@proton.me>
2026-05-14 22:51:11 +03:00
lekss361
bade2f7772
chore(infra): healthcheck frontend + tighten depends_on; pin rosreestr2coord>=5 (#117)
- Add wget healthcheck to frontend service (alpine-compatible) — Caddy
  больше не роутит трафик до того как Next.js готов.
- Caddy depends_on backend/frontend: service_started → service_healthy.
- backend/worker/beat redis dependency: service_started → service_healthy
  (Redis уже имеет redis-cli ping healthcheck).
- Pin rosreestr2coord>=5.0.0 (v5 убрала delay параметр; код уже адаптирован
  per CLAUDE.md); uv resolved 5.3.3.

Closes Day-1 quick wins #3+4 from code review audit (May 14).

Co-authored-by: lekss361 <claudestars@proton.me>
2026-05-14 22:42:09 +03:00
lekss361
1c2f80a4b8
feat(site-finder): integrate nspd_quarter_dumps cache в analyze_parcel (#94 Sprint 1.1 FINAL) (#116)
* feat(site-finder): integrate nspd_quarter_dumps cache в analyze_parcel (#94 Sprint 1.1 #4 FINAL)

Замыкает Sprint 1.1 из #94 part 2 plan. После этого PR пользователь видит
свежие НСПД данные в UI (frontend integration — отдельный PR).

Backend (new app/services/site_finder/quarter_dump_lookup.py):
- `derive_quarter_cad(cad_num)` — 3/4/5-сегмент → quarter (3-segment)
- `get_quarter_dump_data(db, cad_num, parcel_wkt)` — main entrypoint:
  - Reads nspd_quarter_dumps row для derived quarter
  - Freshness threshold: 180 days
  - Missing/stale/harvest_error → trigger harvest_quarter.apply_async() fire-
    and-forget (lazy import против circular), return EMPTY_DUMP_RESULT
  - Fresh + parcel_wkt=None → metadata only (no spatial queries)
  - Fresh + geometry → 3 spatial queries via jsonb_array_elements + ST_Transform
    (3857→4326) + ST_Intersects / ST_DWithin
- 3 private helpers:
  - `_get_zoning` — point-in-polygon parcel centroid vs territorial_zones, LIMIT 1
  - `_get_zouit_overlaps` — все zouit_% layers пересекающиеся с parcel
  - `_get_engineering_nearby` — engineering_structures в 200m, sorted by distance
- `EMPTY_DUMP_RESULT` module-level constant — DRY для no-dump fallback (used
  in get_quarter_dump_data internal + analyze_parcel try/except wrap)

Backend (parcels.py):
- Import EMPTY_DUMP_RESULT + get_quarter_dump_data
- Call wrapped в try/except — если nspd_quarter_dumps недоступна (DB timeout
  / table missing) → EMPTY_DUMP_RESULT fallback вместо 500 (consistent с
  resilience pattern других optional fetches)
- Response gets 4 new fields:
  - nspd_zoning: dict | None (G1 ПЗЗ — zone_code, zone_name, source)
  - nspd_zouit_overlaps: list[dict] (G3 — overlaps в parcel, per ЗОУИТ group)
  - nspd_engineering_nearby: list[dict] (I3 — engineering structures в 200m)
  - nspd_dump: dict (freshness metadata — available, fetched_at_utc, stale,
    harvest_triggered, total_features)

Tests: 13 new в test_quarter_dump_lookup.py (mock-based, no real DB):
- derive_quarter_cad 5 edge cases (3seg, 4seg, 5seg, invalid, whitespace)
- get_quarter_dump no_row → harvest triggered
- stale (>180d) → harvest triggered, stale=True
- harvest_error row → retry harvest triggered
- parcel_wkt=None → metadata only (1 DB call)
- fresh + zoning extraction
- fresh + zouit_overlaps list
- fresh integration: все 4 keys present

47 pre-existing tests still pass.

Code review (code-reviewer pre-push): MINOR, 0 blocking. Applied 2 of 4:
-  #1: try/except wrap around get_quarter_dump_data в analyze_parcel
  (защита от DB unavailability) + DRY через EMPTY_DUMP_RESULT module const
-  #2: removed redundant nspd_zoning.fetched_at_utc (DRY — freshness в
  nspd_dump.fetched_at_utc)
- ⏭ Deferred (acceptable): #3 ad-hoc harvest_quarter retry cooldown для
  harvest_error rows (только при high traffic + persistent NSPD errors);
  #4 raw_props в response — tech debt, убрать вместе с frontend PR

Performance note: 3 spatial queries per analyze adds ~10-50ms on typical
~100-feature quarter. Mitigation if quarters grow dense: materialized
per-layer sub-table (отдельная DB issue).

Closes Sprint 1.1 part of #94. Frontend rendering этих 4 полей — отдельный
PR (next: #112 / #115 / #114).

* fix(site-finder): address PR #116 auto-review M1-M5

M1 (mutation risk): replace EMPTY_DUMP_RESULT direct refs with
_make_empty_result() factory. dict(...) shallow copy left nested
nspd_dump shared by reference across concurrent requests — single
mutation pollutes module sentinel for all subsequent calls. Now
каждый caller gets independent dict.

M2 (O(N) spatial scan): SELECT extended denormalized counts
(territorial_zones_count, zouit_count, engineering_count). Each
spatial helper accepts layer_counts and early-returns when count=0
— skips heavy jsonb_array_elements + ST_Transform/ST_Intersects
scan entirely. Critical для quarters с 2000+ features.

M4 (documentation): _trigger_harvest docstring describes known
burst/no-dedup limitation + TODO Redis SETNX (отдельный PR).

M5 (test fragility): _make_db_mock_with_spatial docstring describes
positional-call contract — db.execute order (0=dump, 1=zoning, 2=zouit,
3=engineering) и зависимость от count-values.

+4 new tests (17 total, all pass):
- test_make_empty_result_returns_independent_copies (mutation safety)
- test_make_empty_result_overrides
- test_early_exit_all_counts_zero_no_spatial_queries
- test_early_exit_partial_counts

Per auto-review on 3068a9c.

* fix(site-finder): rename _make_empty_result → make_empty_result (public) per PR #116 review

M1 residual fix: parcels.py exception path использовал EMPTY_DUMP_RESULT
singleton ref вместо factory. Сейчас readonly access, но нарушает
documented invariant модуля.

Rename `_make_empty_result` → `make_empty_result` (public API), import в
parcels.py, использовать в try/except fallback. Каждый request получает
независимый dict — никаких shared references.

M4 (Redis SETNX dedup) + M5 (test fragility) — deferred per review,
documented в code/issue. Acceptable trade-offs:
- M4: UPSERT idempotency делает данные safe; burst-duplicate task'и тратят
  WAF traffic впустую но не повреждают данные.
- M5: docstring contractually describes positional-call order.

17/17 tests pass. ruff/format clean.

Per auto-review on aef8308.

---------

Co-authored-by: lekss361 <claudestars@proton.me>
2026-05-13 09:14:19 +03:00
lekss361
f3b0ce5687
feat(nspd): harvest_quarter Celery task + beat + admin endpoint (#94 pt.4 part A) (#111)
* feat(nspd): harvest_quarter Celery task + beat + admin endpoint (#94 pt.4/4 part A)

Sprint 1.1 item #3 из #94 part 2. Замыкает harvest cycle: Celery task
оборачивает NSPDClient.search_by_quarter (PR #109) + UPSERT в
nspd_quarter_dumps (PR #110). После — analyze_parcel будет читать dumps
вместо on-demand NSPD HTTP в request-цикле (Sprint 1.1 #4, следующий PR).

Backend:
- New `backend/app/workers/tasks/nspd_sync.py`:
  - `harvest_quarter(quarter_cad, region_code=66, include_zouit=True,
    include_risks=False)` — single-quarter harvest. autoretry_for=
    (NspdLiteWafError,) с retry_backoff=True, max_retries=3,
    soft_time_limit=120s. Generic exception → upsert error row с
    harvest_error=str(exc), counts=0 (НЕ re-raise).
  - `harvest_stale_quarters(region_code, max_age_days=90, batch_size=50)` —
    beat fanout: SELECT cad_quarters_geom EXCEPT fresh dumps → apply_async per cad.
  - UPSERT использует ST_Multi(ST_Transform(ST_SetSRID(ST_GeomFromGeoJSON,3857),4326))
    для quarter_geom (schema MultiPolygon strict per database-expert contract).
- `celery_app.py`:
  - Add `nspd_sync` в `include`
  - Beat entry `nspd-harvest-stale-quarters` Mon 04:00 МСК, batch_size=50,
    max_age_days=90
  - `worker_ready` sanity check `SELECT 1 FROM nspd_quarter_dumps LIMIT 0`
    (non-fatal critical log per Bug_Worker_Ready_EarlyReturn_Fixed pattern)
- `admin_scrape.py`:
  - `HarvestQuarterRequest` Pydantic с pattern `^\d+:\d+:\d+$` (quarter only)
  - `POST /api/v1/admin/scrape/nspd/harvest-quarter` manual trigger,
    requires X-Admin-Token, returns task_id.

Tests (`test_nspd_sync.py`): 14 mock-based (no real NSPD/DB):
- happy path: search_by_quarter mock → verify SQL params/structure
- empty quarter → row with quarter_geom=NULL, total=0, no harvest_error
- WAF error → autoretry kicks in (raise re-propagated)
- Generic error → upsert error row
- Fanout: 50 stale → 50 apply_async calls
- Layer names preservation (all 5 zouit + 11 risks if enabled)
- Geometry SRID handling

Code review (code-reviewer pre-push): APPROVE, 0 blocking, 5 minor deferred:
- inline datetime import in error branch (style)
- layers_fetched array literal via string concat (current layer names safe)
- harvest_stale_quarters error rows always re-tried (intended, undocumented)
- test call_args.kwargs fragile (works for current callsite)
- beat collision with refresh-ekb-districts-medians every ~2 years (negligible)

Part of #94. Next: Sprint 1.1 #4 — analyze_parcel reads dump cache.

* fix(nspd-sync): address PR #111 auto-review M1-M5+M7 findings

M1 (runtime bug): remove WHERE region_code from outer cad_quarters_geom SELECT
in harvest_stale_quarters — column does not exist in prod schema
(cad_number, geom, raw_props, fetched_at, source). Would crash on first beat
tick. Inner nspd_quarter_dumps subquery keeps region_code filter unchanged.

M2 (correctness): _upsert_dump теперь делает db.rollback() в except перед
db.close() — prevents PendingRollbackError на следующем session use, если
PostGIS отбракует geometry или JSON cast failure.

M3 (low): worker_ready breadcrumb INSERT тоже теперь имеет rollback —
consistency с остальными session usages в celery_app.py.

M4 (style): layers_fetched — заменил manual string concat
'{' + ','.join(...) + '}' на native Python list. psycopg v3 сам сериализует
list → text[]. Защита от future layer names с запятыми/фигурными скобками.

M5 (style): import datetime as _dt поднят из else branch в top-level imports.

M7 (test cleanup): удалил dead __enter__/__exit__ mock setup в
test_harvest_stale_quarters_fanout / _empty. harvest_stale_quarters не
использует session как context manager — moc'и no-op.

14/14 tests pass. ruff/format clean.

Per auto-review on 9b2a289.

---------

Co-authored-by: lekss361 <claudestars@proton.me>
2026-05-12 18:45:57 +03:00
lekss361
6fce81e10c
feat(db): add nspd_quarter_dumps table for PKK harvest persistence (#94 pt.3/4) (#110)
Sprint 1.1 item #2 из плана #94 part 2. Persistent storage для QuarterDump
snapshots (PR #109). Foundation для Celery harvest_quarter task (next PR)
и analyze_parcel integration (после).

Schema (data/sql/88_nspd_quarter_dumps.sql):
- Table `nspd_quarter_dumps`: 17 columns matching QuarterDump dataclass.
  - PK quarter_cad (natural key, one row per quarter)
  - quarter_geom (MultiPolygon, 4326) + bbox_3857 (Polygon, 3857) — dual-index
    стратегия: bbox для scheduler queries без transform, geom для joins с
    PZZ/district в 4326
  - 7 per-layer count columns + total_features (computed denorm для быстрых
    статистических queries без распаковки JSON)
  - features_json JSONB — array of ~117 NSPDFeature objects (~120KB per row,
    TOAST handles transparently). Geometry в raw EPSG:3857 — caller transforms.
  - layers_fetched TEXT[] mirrors QuarterDump.layers_fetched tuple
  - harvest_duration_ms (nullable — NULL = "not measured"), harvest_error TEXT
  - region_code SMALLINT NOT NULL (no DEFAULT — explicit per insert)
- 6 indexes: PK, GIST(bbox_3857), GIST(quarter_geom), B-tree(fetched_at_utc DESC),
  B-tree(region_code), partial B-tree(fetched_at_utc WHERE harvest_error IS NULL)
- View v_quarter_dumps_freshness — admin monitoring (AGE + is_failed flag)

Конвенции:
- BEGIN/COMMIT atomic
- IF NOT EXISTS на каждом CREATE — idempotent
- COMMENT ON TABLE + 7 COMMENT ON COLUMN + comments на каждом index
- psycopg v3 compatible (DDL only, no %s)

Применено к prod в рамках database-expert verification (DDL only, idempotent).

Vault entry: code/schemas/Schema_Nspd_Quarter_Dumps.md (upsert template +
EXPLAIN plans + cross-refs).

Code review (code-reviewer pre-push): APPROVE, 2 minor non-blocking:
- MINOR-1 quarter_geom MultiPolygon vs Polygon — resolved в Celery PR via
  ST_Multi(ST_Transform(...)) на ingest contract
- MINOR-2 layers_fetched default '{}' — documented в comment как
  partial-failure path для error rows

Next: Celery harvest_quarter task + beat schedule.

Part of #94 Sprint 1.1.

Co-authored-by: lekss361 <claudestars@proton.me>
2026-05-12 18:22:50 +03:00
lekss361
9ee0b07003
feat(scrapers): search_by_quarter orchestrator + QuarterDump (#94 pt.2/4) (#109)
Sprint 1.1 item #1 из плана #94 part 2. Foundation для PKK harvest pipeline —
1 vacuum (search) + N layer fetches → comprehensive snapshot всех NSPD данных
в пределах квартала. Базис для G1 #28 ПЗЗ, G3 #30 ЗОУИТ, P2 #46 neighbors,
E1 #51 parcels backfill, #96 ЕГРН помещения.

Backend (nspd_client.py):
- New QuarterDump frozen dataclass (slots=True): quarter + per-layer feature
  lists (parcels/buildings/territorial_zones/red_lines/engineering + zouit
  dict + risks dict) + bbox_3857 + layers_fetched (immutable tuple) +
  fetched_at_utc + total_features property.
- New NSPDClient.search_by_quarter(quarter_cad, include_zouit=True,
  include_risks=False): search → bbox → bulk fetch per layer phase.
  Cost 6/11/22 requests.
- New _geojson_bbox_3857() module-level helper — recursive coord walker.
- Class constants QUARTER_CORE_LAYERS / QUARTER_ZOUIT_LAYERS /
  QUARTER_RISK_LAYERS.

Empty-quarter (NSPD не нашёл cad): quarter=None, bbox=None, all lists empty,
zouit/risks dicts populated с пустыми lists (структурно стабильно),
layers_fetched=('search',).

Tests: +12 tests (31 total, no network).

Code review (code-reviewer pre-push): MINOR, fixed 3 of 5:
- datetime import → module-level
- layers_fetched → tuple[str, ...] (immutable in frozen dataclass)
- docstring clarified empty-quarter semantics
Bonus: ruff UP038 isinstance tuple → union syntax.

Part of #94. Sprint 1.1: 4 PRs total. Next: migration → Celery → integration.

Co-authored-by: lekss361 <claudestars@proton.me>
2026-05-12 18:11:30 +03:00
lekss361
bc092b58f1
feat(scrapers): NSPD client foundation (#94) — search + WMS + layers (#98)
* feat(scrapers): NSPD client foundation (#94) — search_by_cad + WMS + layers

Foundation для G1 #28 ПЗЗ, G2 #29 ВРИ, G3 #30 ЗОУИТ, E1 #51 cad_parcels,
I3 #44 engineering, и поддержки on-demand #93.

Backend (`app/services/scrapers/nspd_client.py`):
- `NSPDClient` с 4 core методами:
  - `search_by_cad(cad, thematic_id)` — REST /api/geoportal/v2/search
    возвращает GeoJSON + ВРИ + land_category + cost_value за 1 запрос
  - `get_feature_info(layer_id, lon, lat, buffer_m=100)` — WMS GetFeatureInfo
    на точке: какие feature'ы layer'а покрывают (lon,lat)
  - `get_features_in_bbox(layer_id, bbox_3857)` — bulk через WMS workaround
    (WFS GetCapabilities → 404, поэтому большой bbox + центральная I/J точка)
  - `list_layers(theme_id)` — каталог слоёв в теме (PKK=1, ARN=665)
- Typed responses: NSPDFeature, NSPDSearchResult, NSPDLayer (frozen dataclasses)
- LAYERS catalog: 32 layer-id с семантическими именами (territorial_zones=875838,
  zouit_engineering=37578, risk_flooding=872205, etc) — TIER 1-6 per #94
- Coordinate helpers: lonlat_to_3857(), bbox_around_point_m()
- Reuse: HEADERS + SSL ctx + fetch_geoportal из existing nspd_lite (WAF-compatible
  urllib trick). Rate limit через rate_ms.
- WAF/Rate-limit: raises NspdLiteWafError на 403/429 — caller backoff.

Tests (`backend/tests/test_nspd_client.py`): 14 unit tests, no network.
- LAYERS catalog sanity (territorial_zones=875838 закрывает G1)
- Coordinate transforms (zero, ЕКБ center, bbox)
- NSPDFeature.from_raw parsing (full / missing fields)
- NSPDSearchResult helpers (empty/first)
- _walk_layer_tree (flat / nested / empty)
- search_by_cad через monkeypatch fetch_geoportal

Также включён micro-fix из PR #95 review (cosmetic):
- parcels.py inline comment "до 25s" → "до 15s" (matched _INLINE_FETCH_WAIT_S)

Scope NOT в этом PR (отдельные follow-up issues, sequential rule):
- Schema migrations (cad_parcels_geom +columns, nspd_territorial_zones,
  nspd_zouit, nspd_red_lines, nspd_risk_zones tables) → отдельный PR
- Celery tasks (sync_territorial_zones_bbox, sync_zouit_*, sync_risk_*) →
  отдельный PR использует client
- Admin endpoints (trigger / status) → отдельный PR
- Замена on-demand fetch на nspd_client → PR after schema ready

Closes part of #94 (foundation only — sub-issues for schema/tasks).

* fix(scrapers): address PR #98 auto-review minor feedback

Backend (nspd_client.py):
1. (line 100+) bbox_around_point_m signature split на multi-line.
2. (lazy imports) import json, time → module-level.
3. (duplicate SSL ctx) Reuse _SSL_CTX из nspd_lite через explicit import.
4. (shape defense) list_layers чекает {"data": [...]} wrapper + non-dict/list garbage → warning + empty.
5. (cad_num docstring) search_by_cad документирует формат + ссылку на validate_cad_format.

Tests: +5 mock тестов (19/19 passed):
- get_feature_info URL builder + parsing
- get_features_in_bbox bbox propagation
- list_layers_walks_tree_response
- list_layers_handles_data_wrapper
- list_layers_handles_garbage_response

Backend (parcels.py): docstring '~25с' → '~15с'.

Per auto-review on be14388.

---------

Co-authored-by: lekss361 <claudestars@proton.me>
2026-05-12 09:47:45 +03:00
lekss361
49fac806fc
feat(site-finder): auto-fetch cadastre geometry on-demand (#93) (#95)
* feat(site-finder): auto-fetch cadastre geometry on-demand (#93)

Когда пользователь вводит cad-номер которого нет в БД (cad_quarters_geom /
cad_buildings / cad_parcels_geom), вместо 404 «Загрузи через NSPD geo» (dead-
end для non-admin) — теперь backend автоматически инициирует NSPD fetch,
frontend показывает loading state с polling /fetch-status каждые 2с.

Backend:
- Новый модуль app/services/site_finder/cadastre_fetch.py с helpers:
  find_or_enqueue_fetch (atomic check + enqueue с дедупликацией по cad),
  fetch_status (smart polling endpoint — отличает not_in_nspd от failed),
  detect_thematic_id (3-сегм quarter / 4-сегм parcel / 5-сегм building),
  validate_cad_format.
- Reuse: enqueue_geo_job + process_nspd_geo_job (workers/tasks/nspd_geo).
  source_kind='auto_on_demand' отличает от bulk; rate_ms=200, priority=9.
- POST /parcels/{cad}/analyze graceful fallback: inline await до 25s
  (fast path), затем 202 + job_id + eta_seconds для polling, либо
  400/404/503 в зависимости от статуса (с Retry-After 60s на 503).
- GET /parcels/{cad}/fetch-status новый endpoint для polling.

Frontend:
- useSiteAnalysis расширен: fetchingState + cancel(). POST analyze + при 202
  polling каждые 2с (max 60 итераций = 2 мин cap). status=ready → re-trigger
  analyze; not_in_nspd/failed/invalid_format → typed errors.
- apiFetchWithStatus<T> + HTTPError в lib/api.ts — status-aware variant
  для 200 vs 202.
- FetchingState.tsx: animated spinner, progress bar (linear до etaSeconds,
  asymptote после), elapsed counter, cancel button. Светло-голубая
  scheme отличается от обычного pending skeleton.
- site-finder/page.tsx: FetchingState когда fetchingState активен; обычный
  pending skeleton — только при первичном analyze без 202.

Edge cases (per #93 acceptance):
- cad валидный в НСПД, fetch <25s → inline 200 OK
- cad валидный, fetch >25s → 202 + frontend polls → ready → analyze
- cad валидный, не в НСПД → 404 с понятным сообщением + формат hint
- cad invalid format → 400 + формат hint
- NSPD rate-limited / failed → 503 + Retry-After 60s
- Параллельные запросы на тот же cad → один job, оба клиента poll'ят (дедуп
  через find_active_on_demand_job).

Closes #93.

* fix(site-finder): address PR #95 auto-review minor feedback

Backend (cadastre_fetch.py):
1. (race condition) Advisory lock `pg_try_advisory_xact_lock(hashtext(cad_num))`
   обёрнут вокруг шагов "check active job → enqueue" в find_or_enqueue_fetch.
   Lock transaction-scoped, released at COMMIT. Параллельные запросы на тот
   же cad: первый получает lock и enqueue; второй lock=false → re-check
   active job (уже виден после первого COMMIT) → возвращает тот же job_id.
   Docstring обновлён, упоминание SELECT FOR UPDATE удалено.

Backend (parcels.py):
3. (threadpool exhaustion) _INLINE_FETCH_WAIT_S снижен 25 → 15s с подробным
   комментарием: tradeoff про default Starlette anyio threadpool (40 slots)
   и concurrent burst saturation. 15s баланс: НСПД avg 5-15s для quarter,
   ~70% fast path; остальные 30% получают 202 + polling без блока.

Data (87_on_demand_indexes.sql):
2. (missing index) New migration:
   - `nspd_geo_targets_cad_num_idx ON nspd_geo_targets(cad_num)` — для
     find_active/recent_completed_job (existing UNIQUE composite не покрывает
     WHERE cad_num=:c).
   - `nspd_geo_jobs_source_status_idx ON nspd_geo_jobs(source_kind, status)`
     composite для filter auto_on_demand + queued/running.
   Idempotent (CREATE INDEX IF NOT EXISTS), не блокер при текущем размере,
   критично при росте on-demand traffic.

Frontend (useSiteAnalysis.ts):
4. (UI flicker) setFetchingState(null) перемещён ПОСЛЕ `await second
   apiFetch`. Иначе между clear и resolve есть момент когда mutation
   isPending=true + fetchingState=null → пустой экран ~1 RTT.

NOT addressed (rebuttal):
5. (Tailwind convention) — проверил: в проекте нет ни globals.css, ни
   @tailwind directives. ВСЕ существующие site-finder components используют
   inline styles (ConfidenceBadge, GeotechRiskBlock, ScoreBreakdownPanel etc).
   Tailwind в deps но не wired up. Keep inline styles для consistency.
6. (animate-spin) — требует Tailwind globals (см. #5). `<style jsx>`
   keyframes — built-in Next.js, работает.

Per auto-review on 2252236.

---------

Co-authored-by: lekss361 <claudestars@proton.me>
2026-05-12 09:02:17 +03:00
lekss361
f4a865060f
feat(site-finder): D4 pipeline 24mo — future competition (#36) (#90)
* feat(site-finder): D4 pipeline 24mo — future competition (#36)

Backend (parcels.py):
- Запрос к domrf_kn_objects в радиусе 5км с ready_dt BETWEEN NOW() AND NOW()+24mo.
- _aggregate_pipeline() — сводка: objects_count, flats_total, by_class (эконом/
  комфорт/бизнес/...), by_quarter (хронологически, для UI bar), severity
  (low <500 / medium <3000 / high) per spec, top_objects (десятка по flat_count desc).
- Поле analyze.pipeline_24mo. Backward-compat — optional.

Frontend:
- Pipeline24moBlock.tsx — severity badge + 3 summary numbers (объектов, квартир,
  горизонт/радиус), by-class chips, гистограмма bar по кварталам сдачи
  (нормирована на max), разворачиваемый top-N список с классом + датой сдачи.
- Добавлен в MarketTab выше "Market trend".
- TS типы: Pipeline24mo, PipelineObject, PipelineQuarterSlot.

Closes #36. Relates to #19 (Конкурентный 360 — закрыт ранее в #36 scope).

* fix(site-finder): address PR #90 auto-review feedback

Must-fix (3):
1. distance_m falsy guard: `if obj.get("distance_m") is not None` вместо
   `if obj.get("distance_m")` — centroid-on-building даёт 0.0 (falsy float),
   raw Decimal иначе упал бы в JSON serialization.
2. SQL plan note добавлен про seq scan ~3000 строк OK; при росте — нужен
   GIST/index на (latitude, longitude) — отдельный issue для database-expert
   (будет создан separately).
3. obj_class NULL bug помечен в docstring _aggregate_pipeline с reference на
   fixes/Bug_Kn_API_Obj_Class_Always_Null_OPEN. D6/#38 — fix плановый.

Cleanup (3 из 5):
4. CLASS_LABEL.null dead key убран — JSON null приходит as absent key, не
   "null" string.
6. Magic numbers вынесены: PIPELINE_RADIUS_M=5000, PIPELINE_HORIZON_MONTHS=24,
   PIPELINE_SEVERITY_MEDIUM_THRESHOLD=500, PIPELINE_SEVERITY_HIGH_THRESHOLD=3000,
   PIPELINE_TOP_OBJECTS_LIMIT=10. SQL query теперь через f-string подставляет
   их (защищённое от injection — это int литералы).
8. obj.ready_dt formatting через fmtMonth() с new Date + toLocaleDateString —
   robust к datetime suffix vs date-only, fallback к substring(0,7) при NaN.

Не сделано (defer):
5. Async 3 HTTP calls (pre-existing pattern, нужен ThreadPoolExecutor refactor
   отдельным PR — затрагивает weather/air_quality fetch architecture).
7. ST_GeomFromText дважды — CSE справляется на этом масштабе.

Per auto-review on ade511b.

* fix(site-finder): address PR #90 auto-review minor feedback

1. TS PipelineObject.distance_m — `number | null` для отражения defensive
   Python guard (`if obj.get("distance_m") is not None`). Comment объясняет
   почему.
2. Pipeline SQL: `text(f"...")` → `text("...")` + parameters. radius_m и
   horizon_months через `:param` placeholders + `cast(:horizon_months || ' months'
   AS interval)`. Consistency с остальными SQL в файле, plus защита от
   accidental injection при будущих изменениях.
3. top_objects: explicit field selection вместо `dict(r) for r in rows`.
   Раньше leak'ило все колонки из CTE `SELECT *` (latitude/longitude/
   snapshot_date/region_cd/dev_id) в API response. Теперь только nominated
   fields: obj_id, comm_name, dev_name, obj_class, flat_count, ready_dt,
   distance_m. Schema clean.

Per auto-review on 4e431bf.

---------

Co-authored-by: lekss361 <claudestars@proton.me>
2026-05-12 08:34:02 +03:00
lekss361
1f8fd77825
feat(site-finder): X1 score breakdown + verbal explain (#47) (#92)
Backend (parcels.py):
- POI scoring loop teper строит score_breakdown_detailed: per-factor list с
  verbal explain (через _verbal_for_poi helper) и группировкой
  (_POI_GROUP: Социалка / Торговля / Парки / Транспорт / Шум/трамвай / Локация).
- center_bonus добавлен как synthetic factor группы "Локация" с weight=1.0
  (decay не применяется — bonus IS the value).
- factor key включает enumerate idx — prevents React key collision когда
  два POI одной категории совпадают по округлённому расстоянию.
- Skip факторов с |contribution| < 0.01 (POI > 1км) — UI шуму не нужен.
- abs_total fallback на 1.0 — защита от division-by-zero для empty factors.
- Top-3 positives/negatives: explicit ascending sort для негативов
  ("most-negative first" очевидно из кода).
- score_by_group: stacked-bar данные с count + contribution_pct.
- group_totals type: dict[str, dict[str, float | int]] (count это int).

Frontend:
- Новый ScoreBreakdownPanel.tsx: stacked-bar по группам с tooltip + legend,
  топ-3 плюса (▲ зелёный) / топ-3 минуса (▼ красный) с verbal, отдельная
  строка "Снижают балл — Шум/трамвай: ..." для negative groups, разворачиваемая
  таблица всех факторов (sticky thead, scrollable).
- Интегрирован в OverviewTab под секцией "Балл".
- TS типы: FactorContribution, ScoreGroupTotal.

Closes #47.

NB: branch создан заново из-за rebase mess (см. PR #87 comments). Логически
эквивалентно но history clean.

Co-authored-by: lekss361 <claudestars@proton.me>
2026-05-12 08:17:00 +03:00
lekss361
09611dc690
feat(site-finder): P1 geometry suitability score (#45) (#89)
* feat(site-finder): P1 geometry suitability score (#45)

Backend (parcels.py):
- _polygon_suitability() — Shapely metrics on parcel WGS84 polygon:
  area_ha/m2 (projected via cos(lat) к UTM-like meters), perimeter_m,
  aspect_ratio (long/short side of MABR), convex_hull_ratio (area / hull area),
  min_inscribed_rect_dim_m.
- Composite suitability score 0..1:
  base = area_subscore (0 при <0.2 ha, linear → 1 при ≥0.5 ha)
  −0.3 если aspect_ratio > 5 (вытянутый)
  −0.3 если convex_hull_ratio < 0.65 (изрезанный)
  −0.5 если min_inscribed_rect_dim_m < 30 (узкий)
- Label: микро / подходящий / сложная форма / слабо подходит
- Recommendation: "для МКД 16+ нужно >0.3 ha + минимум 40м"
- В analyze response → geometry_suitability.

Frontend:
- Новый GeometrySuitabilityBlock.tsx с color-coded badge + метрики grid +
  penalties + recommendation
- Добавлен в LandTab (выше "Зонирование (ПЗЗ)")
- TS типы расширены: GeometrySuitability

Closes #45.

* fix(site-finder): address PR #89 auto-review minor feedback

Backend (parcels.py):
1. Фактическая ошибка фикс: recommendation теперь различает "строительный
   минимум 30м" (physical penalty trigger) и "комфорт МКД 12-16эт 40м"
   (recommendation level). Пользователю чётко видны 2 порога.
2. Lazy imports → module-level: `from shapely import wkt as _shp_wkt`,
   `from shapely.geometry import Polygon`.
3. MABR inner exception теперь логирует: `logger.debug("MABR computation
   failed, falling back to sqrt(area): %s", mabr_err)`.
4. Magic numbers вынесены в константы: _GEOM_MIN_AREA_HA, _GEOM_AREA_SCORE_FULL_HA,
   _GEOM_ASPECT_PENALTY_THRESHOLD, _GEOM_CONVEX_PENALTY_THRESHOLD,
   _GEOM_MIN_WIDTH_PHYSICAL_M, _GEOM_MIN_WIDTH_COMFORT_M, _GEOM_LABEL_MICRO_HA,
   _GEOM_LABEL_GOOD, _GEOM_LABEL_MEDIUM, плюс penalty константы.
5. Label "микро" теперь комбинируется с penalties: "микро, узкий" — UI видит
   обе проблемы. Pure "микро" остаётся когда нет penalty.

Frontend (GeometrySuitabilityBlock.tsx + types):
6. Заменил emoji ⚠ (U+26A0) на текстовый <strong>"Проблемы формы:"</strong> —
   стабильнее в WeasyPrint PDF-экспорте и cross-platform.
- TS типы: GeometrySuitabilityBaseLabel экспортирован, label расширен до
  `string` для допуска combo-labels; helper colorForLabel() парсит base часть.

Per auto-review on f4e7491.

* fix(site-finder): P1 #45 — extract 0.3 ha to _GEOM_AREA_COMFORT_HA

Auto-review нашёл: recommendation упоминала "от 0.3 га" но в constants block
не было 0.3 — magic literal в строке. Тот же класс проблемы что 30/40м inconsistency
из прошлого review, только в другом поле.

Fix: новая константа `_GEOM_AREA_COMFORT_HA = 0.3` с комментарием
"рекомендуемая комфортная площадь МКД (recommendation)". Размещена между
_GEOM_MIN_AREA_HA (физический минимум) и _GEOM_AREA_SCORE_FULL_HA (premium) —
третий semantic threshold. Recommendation теперь f-string использует константу.

Per auto-review on ae5e8de.

---------

Co-authored-by: lekss361 <claudestars@proton.me>
2026-05-12 08:00:12 +03:00
lekss361
843314c040
feat(site-finder): P2 cad_buildings соседи + overlap check (#46) (#91)
* feat(site-finder): P2 cad_buildings соседи + overlap check (#46)

Backend (parcels.py):
- _parse_floors() helper для TEXT column (cad_buildings.floors хранится как
  строка, могут быть диапазоны "5-7"). Возвращает верхнюю границу.
- _neighbors_summary(db, geom_wkt, our_cad) — query соседей в 100м (GIST):
  cad_num, building_name, floors, year_built, cost_value, area, address, distance.
  Aggregate: avg_floors_100m, max_floors_100m, median_cost_per_m2_100m,
  count_buildings_100m. Outliers cost/m² фильтруются (1k < x < 500k).
- Overlap check: ST_Intersects + ST_Area(ST_Intersection) > 50 m² (transformed
  to UTM 32641 для метров). Если есть → has_existing_buildings: true +
  overlap_buildings list.
- В response → neighbors_summary.

Frontend:
- Новый NeighborsBlock.tsx: hard red warn block для overlap (с building names +
  overlap_m2 + "Инвестиции невозможны без сноса"); summary metrics (avg/max
  floors, median price); toggle "Показать N ближайших" → таблица.
- Border меняется на красный при has_existing_buildings — visual cue.
- Добавлен в LandTab выше "Зонирование (ПЗЗ)".
- TS типы: NeighborBuilding, OverlapBuilding, NeighborsSummary.

Closes #46. Closes #21 (cad_buildings в Site Finder фильтрах).

* fix(site-finder): address PR #91 auto-review minor feedback

Backend (parcels.py):
1. (medium) Aggregation loop _neighbors_summary теперь обёрнут в try/except
   (ValueError, TypeError) с fallback к data_available:False + log warning.
   Защищает от non-numeric cost_value/area придёт в строке (e.g. "N/A") —
   ранее весь endpoint падал 500.
2. Magic numbers вынесены: _COST_PER_M2_MIN=1000, _COST_PER_M2_MAX=500_000.
3. _parse_floors docstring + inline note про malformed parts ("5а-7" filter,
   multi-range "1-2-3" max acceptable degradation).

Frontend (NeighborsBlock.tsx):
5. Русский plural fix: pluralBuildings(n) helper — 1 здание, 2-4 здания,
   5+/11-14 зданий. Раньше "3 зданий" — теперь "3 здания".

Не сделано (defer):
4. ST_Area для overlap query — практически 0-5 buildings в ЕКБ, GIST scan OK.
6. Discriminated union для NeighborsSummary — refactor а не bug.
7. Vault entry для P2 — добавится batch'ем после merge всех текущих PR.

Per auto-review on 60d53bb.

---------

Co-authored-by: lekss361 <claudestars@proton.me>
2026-05-12 01:30:53 +03:00
lekss361
3777a77a42
feat(site-finder): X2 confidence indicator + caveats (#48) (#88)
Backend (parcels.py):
- _compute_confidence() composite score 0..1 from 7 subscores: poi_freshness,
  geom_source (parcel vs quarter), district, market_trend (rosreestr_deals depth),
  competitors, environment (noise/air/weather availability), zoning (placeholder
  до G1).
- confidence_label: high (>0.75) / medium (0.4-0.75) / low (<0.4)
- confidence_caveats: list of конкретных проблем для UI
- confidence_breakdown: per-subscore 0..1 для прозрачности

Это stub-версия (полная — после G1/G2/D1/D2). Использует только текущие сигналы.

Frontend:
- Новый ConfidenceBadge.tsx — color-coded (green/yellow/red) badge с %
- Caveats для low — показываются сразу; для medium/high — под toggle
- Toggle "Подробнее" → breakdown per-subscore + полный список caveats
- Размещён в начале OverviewTab (выше "Район")
- TS типы расширены: confidence, confidence_label, confidence_breakdown, confidence_caveats

Closes #48.

Co-authored-by: lekss361 <claudestars@proton.me>
2026-05-12 01:04:25 +03:00
lekss361
485c221c3b
docs(claude): formalize Auto-review passed as approval signal (#85)
* docs(claude): formalize "Auto-review passed" as approval signal + polling loop

Per user feedback after PR #84 merge:
- Critical rule 5: "Auto-review passed" / "LGTM" от auto-review bot = approval
  (наряду с явным "merge it" / "approved" от human user)
- Add "Polling loop для PR" section: 60s ScheduleWakeup pattern,
  decisive states (merged / approved / changes / no-news), cap 30 iters

This formalizes what main session already did: after fixup commit, poll
comments, detect auto-review verdict, and auto-merge if passed.

* fix(claude): address PR #85 auto-review feedback

- Fix cyrillic/latin typo (rule 5 second sub-bullet)
- Add SHA marker validation to polling loop (race condition on stale LGTM after fixup-push)
- Add explicit "Auto-merge scope" rule 6 — allowlist (docs/UI/.claude) vs blocklist (migrations/API/infra/secrets)
- Sync "Когда auto-mode" section with new merge approval signals
- Lower cap to 5 iters for scope-blocked PRs (human required anyway)
- Self-modifying CLAUDE.md rules → always require human

Per auto-review on e66e1ea.

---------

Co-authored-by: lekss361 <claudestars@proton.me>
2026-05-12 00:43:25 +03:00
lekss361
a35a3d02ea
feat(site-finder): deep-link tabs via ?tab= URL param (#86)
* feat(site-finder): deep-link tabs via ?tab=env|land|market URL param

Tab state now syncs with URL — можно поделиться ссылкой на конкретный
таб (например /site-finder?tab=market для тренда цен по последнему cad).

- useRouter + useSearchParams (Next 15 app router, client-component)
- router.replace (не push) чтобы tab switches не засоряли history
- useEffect on searchParams → back/forward работают
- overview = default, не пишется в URL (clean)
- Type-guard isTabId(): только valid values из URL переключают state

* fix(site-finder): address PR #86 auto-review minor recommendations

- Use usePathname() for router.replace fallback (avoid dangling "?" in Next 15)
- Remove stale-closure guard in useEffect — React bails Object.is, no eslint-disable
- Wrap page in <Suspense> for useSearchParams (Next 15 App Router req for static)
- Lazy useState initializer — skip wasted CPU on re-renders

Per auto-review on c47e56b.

---------

Co-authored-by: lekss361 <claudestars@proton.me>
2026-05-12 00:33:49 +03:00
lekss361
aadf47709b
docs(claude): mandate branch+PR workflow, ban direct push to main (#84)
* docs(claude): mandate branch+PR workflow, ban direct push to main

* docs(claude): sync TL;DR + Don't-do-these with new PR workflow

Address auto-review feedback:
- TL;DR rule 5 → branch+PR mandate (was: "user commits themselves")
- TL;DR rule 6 → no --no-verify/--force/--amend (split from rule 5)
- Don't-do-these → drop conflicting "no auto-commit" line, add
  "no direct push to main" + "no merge without approval"
- Critical workflow rules: add explicit rule 2 — worker-agents don't
  commit themselves; main session commits on feature branch
- Restore "Squash on merge" wording in PR section (was dropped)
- Auto-mode section: clarify that main session does commits on feature
  branch (worker-agents leave changes staged)

* fix(claude): Code-reviewer diagram had stale 'push origin HEAD:main' — replace with branch+PR flow

---------

Co-authored-by: lekss361 <claudestars@proton.me>
2026-05-12 00:16:06 +03:00
lekss361
053ce29a40 docs(claude): add code-reviewer agent — review workflow before push 2026-05-11 23:44:32 +03:00
lekss361
13119f9cb5 feat(docs): /docs/b2b-channels (rendered) + /docs/b2b-channels.md (raw) for sharing 2026-05-11 23:36:02 +03:00
lekss361
59466de811 fix(deploy): aggressive docker image prune (drop until=72h filter to avoid 50GB buildup) 2026-05-11 23:26:31 +03:00
lekss361
a1201aa94e feat(site-finder): tabbed UX - hero KPI band + 4 tabs (Overview/Env/Land/Market) + sticky map 2026-05-11 23:21:39 +03:00
lekss361
199adc31e6 fix(site-finder): UX - sticky map, collapsible sections, score header sticky 2026-05-11 23:10:56 +03:00
lekss361
99ca42e5d0 fix(site-finder): isochrones map re-renders on mode change (key includes coords hash) 2026-05-11 23:03:25 +03:00
lekss361
2cbcea9e73 fix(zoning): NSPD blocks bot access — fallback to deep-links (PKK6 API closed 2024) 2026-05-11 22:56:50 +03:00