15 commits

Author	SHA1	Message	Date
bot-backend	479f01ae1a	Merge remote-tracking branch 'forgejo/main' into HEAD ... # Conflicts: # backend/app/services/site_finder/gate_verdict.py	2026-06-15 21:07:05 +03:00
Light1YT	86e9ea2937	fix(week-review): автофиксы код-ревью — 169 issue (label «week ревью 1») ... Многоагентный аудит + имплементация: один воркер на файл, точечные правки. Верификация: py_compile (47/47 .py) + tsc --noEmit (0 ошибок). Unit-тесты не прогонялись (окружение не поднято: rollup native dep / нет pytest-venv). Полностью исправлено (169): #1336, #1337, #1339, #1340, #1341, #1342, #1343, #1345, #1346, #1348, #1349, #1350, #1351, #1354, #1356, #1358, #1359, #1360, #1362, #1364, #1365, #1366, #1367, #1368, #1369, #1370, #1371, #1372, #1373, #1374, #1375, #1376, #1377, #1378, #1379, #1380, #1381, #1382, #1384, #1385, #1386, #1387, #1388, #1389, #1390, #1391, #1392, #1394, #1395, #1396, #1397, #1399, #1400, #1401, #1402, #1403, #1404, #1408, #1409, #1410, #1411, #1412, #1413, #1414, #1415, #1416, #1417, #1418, #1420, #1423, #1425, #1426, #1427, #1428, #1429, #1430, #1431, #1432, #1433, #1434, #1435, #1437, #1438, #1439, #1440, #1441, #1442, #1443, #1444, #1445, #1446, #1447, #1448, #1449, #1450, #1451, #1452, #1453, #1454, #1455, #1456, #1457, #1458, #1459, #1460, #1461, #1462, #1463, #1464, #1465, #1466, #1467, #1468, #1469, #1471, #1472, #1473, #1474, #1476, #1478, #1479, #1481, #1482, #1483, #1484, #1485, #1487, #1488, #1489, #1490, #1491, #1492, #1493, #1494, #1495, #1496, #1497, #1499, #1500, #1501, #1502, #1504, #1505, #1506, #1507, #1510, #1514, #1515, #1516, #1517, #1518, #1519, #1521, #1522, #1523, #1524, #1525, #1526, #1527, #1528, #1529, #1531, #1532, #1533, #1534, #1535, #1536, #1537, #1538 Частично (9, in-file часть, остаток cross-file): #1361, #1419, #1422, #1424, #1470, #1475, #1477, #1480, #1498 Требуют cross-file (3, не тронуты): #1338, #1363, #1421 Пропущено (1): #1539 Не входило в партию: 22 needs-Leha issue (нужны решения владельца). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>	2026-06-15 20:21:11 +05:00
bot-backend	e5e0ccb283	feat(site-finder): сетевые обременения из ЗОУИТ (#1070, Refs #103) (#1328)	2026-06-14 14:49:32 +00:00
Light1YT	50555821fb	feat(site_finder): routing POI decay (#41) + OSM utility distances (#44) + ЗОУИТ geom (#255bk) ... #41: per-category routing-radius decay через reusable ORS /matrix client (foot-walk время + per-category пороги + decay-curves), opt-in ?decay=routing, graceful straight-line fallback (analyze не 500 при ORS down) + length-guard durations==destinations. #44: +5 OSM-категорий (transformer/gas/water/sewerage/heat) via Overpass nwr; nearest_{water_main,substation,gas,heat}_m в analyze.utilities. #255bk: ST_AsGeoJSON(CAST(geom AS geometry)) в _get_cad_zouit_overlaps → geom_geojson. Closes #41 Closes #44	2026-06-13 22:31:56 +05:00
Light1YT	2e428405d2	fix(site-finder): расширить except в parcel_ird_overlaps до DataError (#1095) ... parcel_ird_overlaps ловил только (OperationalError, ProgrammingError). При D9b-wiring в analyze malformed WKT в ST_GeomFromText давал PostGIS ERROR → SQLAlchemy DataError, который пробивал try/except → analyze падал вместо graceful-degrade. - ird_overlay_lookup: + DataError в imports и в except tuple. - quarter_dump_lookup: симметричное расширение в _get_engineering_*. - test_ird_overlay_lookup: parametrized test_graceful_when_db_fails покрывает 3 класса (Operational/Programming + DataError для WKT). Closes #1095	2026-06-13 15:02:50 +05:00
lekss361	81cf274e80	fix(#234): address re-review nits — Settings singleton + docstring sync ... 1. _acquire_harvest_lock: Settings() instance → singleton 'settings' import from app.core.config — соответствует convention (scrape_kn.py, objective.py, celery_app.py все используют singleton). 2. _trigger_harvest docstring: убрана incorrect фраза «остальные ... видят harvest_eta_seconds от первого запроса» (код возвращает eta=None для denied-lock callers). Заменена на actual behavior: graceful no-data UI с ПКК fallback на FE. PR #250 re-review followup (issues #3 + docstring sync).	2026-05-17 09:31:43 +03:00
lekss361	b99ac76463	Merge remote-tracking branch 'forgejo/main' into fix/234-nspd-harvest-eta-badge ... # Conflicts: # backend/app/services/site_finder/quarter_dump_lookup.py	2026-05-17 09:19:12 +03:00
lekss361	990c109756	fix(#234): NSPD harvest ETA badge + SETNX dedupe (Sub-PR C) ... Backend (quarter_dump_lookup.py): - _acquire_harvest_lock: Redis SETNX TTL=120s на quarter, защищает от burst N concurrent analyze, ставящих N одинаковых harvest task в очередь - _trigger_harvest: использует lock перед apply_async, возвращает False если lock уже взят (другой запрос триггернул раньше) - make_empty_result/EMPTY_DUMP_RESULT: новое поле harvest_eta_seconds в nspd_dump dict, типичный harvest_quarter = 60с - /analyze: пробрасывает поле через nspd_dump dict (нет typed schema — response_model=None для /analyze endpoint, dict уходит как есть) Frontend (NspdFreshnessBadge, NspdZoningBlock): - Countdown «НСПД: загрузка ~Nс» вместо бесконечного спиннера - После остановки countdown (remaining=0) NspdZoningBlock показывает «загрузка дольше обычного» + ссылку на ПКК вместо infinite skeleton Tests: 5 новых unit + 2 для empty_result schema (всего +7, pass) Closes #234 (UX-side; data-side resolves когда Sub-PR B + D merged).	2026-05-17 09:13:06 +03:00
lekss361	10b79da2e9	fix(#243): cad_zouit fallback срабатывает когда nspd dump полностью отсутствует (#248)	2026-05-17 06:03:19 +00:00
lekss361	f70be684da	feat(#29,#232): wire cad_parcels.permitted_use in analyze + cad_zouit fallback (G2+G3) ... TASK A (#29 G2): add parcel_meta to analyze response - New ParcelMeta Pydantic schema in app/schemas/parcel.py - SELECT from cad_parcels WHERE cad_num=:c in analyze_parcel() (step 9f) - Returns permitted_use_established_by_document, land_record_category_type, land_record_subtype, cost_value; None when row absent - Tests: test_analyze_parcel_meta.py (found + not-found cases) TASK B (#232 G3): cad_zouit fallback in _get_zouit_overlaps - When nspd_quarter_dumps has zouit_count==0, fall back to ST_Intersects query on cad_zouit (3483 rows, GIST indexed) - Overlaps tagged with source='cad_zouit'; format compatible with NSPD path - gate_verdict.py: BLOCKER_TYPE_ZONE_KEYWORDS tuple for keyword-based classification (охранная зона / трубопровод / электр / газ -> blocker; СЗЗ -> warning); NSPD subcategory path preserved backward-compat - Tests: 6 new test cases in test_gate_verdict.py covering cad_zouit path and backward-compat for NSPD subcategory path Updated db.execute call sequence in test_analyze_*.py (index shift +1 at pos 10).	2026-05-17 08:17:22 +03:00
lekss361	fdb54834e5	fix(nspd): rename migration 89→98, fix red lines ST_Area→ST_Length (#220) ... - data/sql/98_*: rename from 89 to avoid collision with existing 89_drop_dead_brin - _get_red_lines: ST_Length(ST_Intersection(planar)::geography) instead of ST_Area(ST_Intersection(::geography, ::geography)) — fixes PostGIS 3.4 tolerance error and returns correct non-zero length for LINESTRING intersections - Rename intersection_area_sqm → intersection_length_m across schema, TS types, frontend component, and tests; add test_get_red_lines_db_exception_returns_empty Addresses review-bot feedback on PR #220.	2026-05-16 19:33:14 +03:00
lekss361	ed3c128528	feat(nspd): TIER 4 opportunity layers + red lines (#94 PR2 of 4) ... - NSPDClient: QUARTER_OPPORTUNITY_LAYERS (auction/scheme/free/future/oopt) + QuarterDump.opportunity field - nspd_sync: harvest_quarter accepts include_opportunity, denorm cols has_auction_parcels + opportunity_count in UPSERT - quarter_dump_lookup: - _get_opportunity_parcels (sort by distance, early-exit on count=0) - _get_red_lines (query existing dump.red_lines core path, layer='red_lines') - SQL 89_*: has_auction_parcels + opportunity_count + partial index - Pydantic: OpportunityParcel + RedLine schemas - /analyze: nspd_opportunity_parcels + nspd_red_lines fields - Frontend: NspdOpportunityBlock + NspdRedLinesBlock + LandTab integration - Tests: 28 total (11 PR1 + 17 PR2), all pass Red lines uses existing core harvest path (layer 879243 already in dump.red_lines from PR1+core) — no duplicate harvest, no red_lines_count_v2 column needed. Part of #94	2026-05-16 19:06:22 +03:00
lekss361	1595157ebf	feat(nspd): enable TIER 3 risk zones harvest + extract in analyze (#94 PR1) ... - harvest_stale_quarters fanout passes include_risks=True to all queued tasks - _get_risk_zones() with ST_Intersects + ST_Area intersection calc - _extract_features_by_layer() generic helper for future PR2+ reuse - RiskZone Pydantic schema; /analyze returns nspd_risk_zones list - NspdRiskZonesBlock component with severity color-coding (red/yellow/orange) - 11 unit tests for extract/intersect/null-area/db-exception paths Part of #94 (TIER 3 risk layers — flooding, landslide, burns, erosion etc)	2026-05-16 16:11:59 +03:00
lekss361	4deb1b0284	feat(parcels): connection-points endpoint (Forgejo #115 Phase 1 backend) (#190)	2026-05-16 05:58:27 +00:00
lekss361	1c2f80a4b8	feat(site-finder): integrate nspd_quarter_dumps cache в analyze_parcel (#94 Sprint 1.1 FINAL) (#116) ... * feat(site-finder): integrate nspd_quarter_dumps cache в analyze_parcel (#94 Sprint 1.1 #4 FINAL) Замыкает Sprint 1.1 из #94 part 2 plan. После этого PR пользователь видит свежие НСПД данные в UI (frontend integration — отдельный PR). Backend (new app/services/site_finder/quarter_dump_lookup.py): - `derive_quarter_cad(cad_num)` — 3/4/5-сегмент → quarter (3-segment) - `get_quarter_dump_data(db, cad_num, parcel_wkt)` — main entrypoint: - Reads nspd_quarter_dumps row для derived quarter - Freshness threshold: 180 days - Missing/stale/harvest_error → trigger harvest_quarter.apply_async() fire- and-forget (lazy import против circular), return EMPTY_DUMP_RESULT - Fresh + parcel_wkt=None → metadata only (no spatial queries) - Fresh + geometry → 3 spatial queries via jsonb_array_elements + ST_Transform (3857→4326) + ST_Intersects / ST_DWithin - 3 private helpers: - `_get_zoning` — point-in-polygon parcel centroid vs territorial_zones, LIMIT 1 - `_get_zouit_overlaps` — все zouit_% layers пересекающиеся с parcel - `_get_engineering_nearby` — engineering_structures в 200m, sorted by distance - `EMPTY_DUMP_RESULT` module-level constant — DRY для no-dump fallback (used in get_quarter_dump_data internal + analyze_parcel try/except wrap) Backend (parcels.py): - Import EMPTY_DUMP_RESULT + get_quarter_dump_data - Call wrapped в try/except — если nspd_quarter_dumps недоступна (DB timeout / table missing) → EMPTY_DUMP_RESULT fallback вместо 500 (consistent с resilience pattern других optional fetches) - Response gets 4 new fields: - nspd_zoning: dict | None (G1 ПЗЗ — zone_code, zone_name, source) - nspd_zouit_overlaps: list[dict] (G3 — overlaps в parcel, per ЗОУИТ group) - nspd_engineering_nearby: list[dict] (I3 — engineering structures в 200m) - nspd_dump: dict (freshness metadata — available, fetched_at_utc, stale, harvest_triggered, total_features) Tests: 13 new в test_quarter_dump_lookup.py (mock-based, no real DB): - derive_quarter_cad 5 edge cases (3seg, 4seg, 5seg, invalid, whitespace) - get_quarter_dump no_row → harvest triggered - stale (>180d) → harvest triggered, stale=True - harvest_error row → retry harvest triggered - parcel_wkt=None → metadata only (1 DB call) - fresh + zoning extraction - fresh + zouit_overlaps list - fresh integration: все 4 keys present 47 pre-existing tests still pass. Code review (code-reviewer pre-push): MINOR, 0 blocking. Applied 2 of 4: - ✅ #1: try/except wrap around get_quarter_dump_data в analyze_parcel (защита от DB unavailability) + DRY через EMPTY_DUMP_RESULT module const - ✅ #2: removed redundant nspd_zoning.fetched_at_utc (DRY — freshness в nspd_dump.fetched_at_utc) - ⏭ Deferred (acceptable): #3 ad-hoc harvest_quarter retry cooldown для harvest_error rows (только при high traffic + persistent NSPD errors); #4 raw_props в response — tech debt, убрать вместе с frontend PR Performance note: 3 spatial queries per analyze adds ~10-50ms on typical ~100-feature quarter. Mitigation if quarters grow dense: materialized per-layer sub-table (отдельная DB issue). Closes Sprint 1.1 part of #94. Frontend rendering этих 4 полей — отдельный PR (next: #112 / #115 / #114). * fix(site-finder): address PR #116 auto-review M1-M5 M1 (mutation risk): replace EMPTY_DUMP_RESULT direct refs with _make_empty_result() factory. dict(...) shallow copy left nested nspd_dump shared by reference across concurrent requests — single mutation pollutes module sentinel for all subsequent calls. Now каждый caller gets independent dict. M2 (O(N) spatial scan): SELECT extended denormalized counts (territorial_zones_count, zouit_count, engineering_count). Each spatial helper accepts layer_counts and early-returns when count=0 — skips heavy jsonb_array_elements + ST_Transform/ST_Intersects scan entirely. Critical для quarters с 2000+ features. M4 (documentation): _trigger_harvest docstring describes known burst/no-dedup limitation + TODO Redis SETNX (отдельный PR). M5 (test fragility): _make_db_mock_with_spatial docstring describes positional-call contract — db.execute order (0=dump, 1=zoning, 2=zouit, 3=engineering) и зависимость от count-values. +4 new tests (17 total, all pass): - test_make_empty_result_returns_independent_copies (mutation safety) - test_make_empty_result_overrides - test_early_exit_all_counts_zero_no_spatial_queries - test_early_exit_partial_counts Per auto-review on 3068a9c. * fix(site-finder): rename _make_empty_result → make_empty_result (public) per PR #116 review M1 residual fix: parcels.py exception path использовал EMPTY_DUMP_RESULT singleton ref вместо factory. Сейчас readonly access, но нарушает documented invariant модуля. Rename `_make_empty_result` → `make_empty_result` (public API), import в parcels.py, использовать в try/except fallback. Каждый request получает независимый dict — никаких shared references. M4 (Redis SETNX dedup) + M5 (test fragility) — deferred per review, documented в code/issue. Acceptable trade-offs: - M4: UPSERT idempotency делает данные safe; burst-duplicate task'и тратят WAF traffic впустую но не повреждают данные. - M5: docstring contractually describes positional-call order. 17/17 tests pass. ruff/format clean. Per auto-review on aef8308. --------- Co-authored-by: lekss361 <claudestars@proton.me>	2026-05-13 09:14:19 +03:00