lekss361
|
fdb54834e5
|
fix(nspd): rename migration 89→98, fix red lines ST_Area→ST_Length (#220)
- data/sql/98_*: rename from 89 to avoid collision with existing 89_drop_dead_brin
- _get_red_lines: ST_Length(ST_Intersection(planar)::geography) instead of
ST_Area(ST_Intersection(::geography, ::geography)) — fixes PostGIS 3.4
tolerance error and returns correct non-zero length for LINESTRING intersections
- Rename intersection_area_sqm → intersection_length_m across schema, TS types,
frontend component, and tests; add test_get_red_lines_db_exception_returns_empty
Addresses review-bot feedback on PR #220.
|
2026-05-16 19:33:14 +03:00 |
|
lekss361
|
ed3c128528
|
feat(nspd): TIER 4 opportunity layers + red lines (#94 PR2 of 4)
- NSPDClient: QUARTER_OPPORTUNITY_LAYERS (auction/scheme/free/future/oopt)
+ QuarterDump.opportunity field
- nspd_sync: harvest_quarter accepts include_opportunity, denorm cols
has_auction_parcels + opportunity_count in UPSERT
- quarter_dump_lookup:
- _get_opportunity_parcels (sort by distance, early-exit on count=0)
- _get_red_lines (query existing dump.red_lines core path, layer='red_lines')
- SQL 89_*: has_auction_parcels + opportunity_count + partial index
- Pydantic: OpportunityParcel + RedLine schemas
- /analyze: nspd_opportunity_parcels + nspd_red_lines fields
- Frontend: NspdOpportunityBlock + NspdRedLinesBlock + LandTab integration
- Tests: 28 total (11 PR1 + 17 PR2), all pass
Red lines uses existing core harvest path (layer 879243 already in dump.red_lines
from PR1+core) — no duplicate harvest, no red_lines_count_v2 column needed.
Part of #94
|
2026-05-16 19:06:22 +03:00 |
|
lekss361
|
1595157ebf
|
feat(nspd): enable TIER 3 risk zones harvest + extract in analyze (#94 PR1)
- harvest_stale_quarters fanout passes include_risks=True to all queued tasks
- _get_risk_zones() with ST_Intersects + ST_Area intersection calc
- _extract_features_by_layer() generic helper for future PR2+ reuse
- RiskZone Pydantic schema; /analyze returns nspd_risk_zones list
- NspdRiskZonesBlock component with severity color-coding (red/yellow/orange)
- 11 unit tests for extract/intersect/null-area/db-exception paths
Part of #94 (TIER 3 risk layers — flooding, landslide, burns, erosion etc)
|
2026-05-16 16:11:59 +03:00 |
|
lekss361
|
1c2f80a4b8
|
feat(site-finder): integrate nspd_quarter_dumps cache в analyze_parcel (#94 Sprint 1.1 FINAL) (#116)
* feat(site-finder): integrate nspd_quarter_dumps cache в analyze_parcel (#94 Sprint 1.1 #4 FINAL)
Замыкает Sprint 1.1 из #94 part 2 plan. После этого PR пользователь видит
свежие НСПД данные в UI (frontend integration — отдельный PR).
Backend (new app/services/site_finder/quarter_dump_lookup.py):
- `derive_quarter_cad(cad_num)` — 3/4/5-сегмент → quarter (3-segment)
- `get_quarter_dump_data(db, cad_num, parcel_wkt)` — main entrypoint:
- Reads nspd_quarter_dumps row для derived quarter
- Freshness threshold: 180 days
- Missing/stale/harvest_error → trigger harvest_quarter.apply_async() fire-
and-forget (lazy import против circular), return EMPTY_DUMP_RESULT
- Fresh + parcel_wkt=None → metadata only (no spatial queries)
- Fresh + geometry → 3 spatial queries via jsonb_array_elements + ST_Transform
(3857→4326) + ST_Intersects / ST_DWithin
- 3 private helpers:
- `_get_zoning` — point-in-polygon parcel centroid vs territorial_zones, LIMIT 1
- `_get_zouit_overlaps` — все zouit_% layers пересекающиеся с parcel
- `_get_engineering_nearby` — engineering_structures в 200m, sorted by distance
- `EMPTY_DUMP_RESULT` module-level constant — DRY для no-dump fallback (used
in get_quarter_dump_data internal + analyze_parcel try/except wrap)
Backend (parcels.py):
- Import EMPTY_DUMP_RESULT + get_quarter_dump_data
- Call wrapped в try/except — если nspd_quarter_dumps недоступна (DB timeout
/ table missing) → EMPTY_DUMP_RESULT fallback вместо 500 (consistent с
resilience pattern других optional fetches)
- Response gets 4 new fields:
- nspd_zoning: dict | None (G1 ПЗЗ — zone_code, zone_name, source)
- nspd_zouit_overlaps: list[dict] (G3 — overlaps в parcel, per ЗОУИТ group)
- nspd_engineering_nearby: list[dict] (I3 — engineering structures в 200m)
- nspd_dump: dict (freshness metadata — available, fetched_at_utc, stale,
harvest_triggered, total_features)
Tests: 13 new в test_quarter_dump_lookup.py (mock-based, no real DB):
- derive_quarter_cad 5 edge cases (3seg, 4seg, 5seg, invalid, whitespace)
- get_quarter_dump no_row → harvest triggered
- stale (>180d) → harvest triggered, stale=True
- harvest_error row → retry harvest triggered
- parcel_wkt=None → metadata only (1 DB call)
- fresh + zoning extraction
- fresh + zouit_overlaps list
- fresh integration: все 4 keys present
47 pre-existing tests still pass.
Code review (code-reviewer pre-push): MINOR, 0 blocking. Applied 2 of 4:
- ✅ #1: try/except wrap around get_quarter_dump_data в analyze_parcel
(защита от DB unavailability) + DRY через EMPTY_DUMP_RESULT module const
- ✅ #2: removed redundant nspd_zoning.fetched_at_utc (DRY — freshness в
nspd_dump.fetched_at_utc)
- ⏭ Deferred (acceptable): #3 ad-hoc harvest_quarter retry cooldown для
harvest_error rows (только при high traffic + persistent NSPD errors);
#4 raw_props в response — tech debt, убрать вместе с frontend PR
Performance note: 3 spatial queries per analyze adds ~10-50ms on typical
~100-feature quarter. Mitigation if quarters grow dense: materialized
per-layer sub-table (отдельная DB issue).
Closes Sprint 1.1 part of #94. Frontend rendering этих 4 полей — отдельный
PR (next: #112 / #115 / #114).
* fix(site-finder): address PR #116 auto-review M1-M5
M1 (mutation risk): replace EMPTY_DUMP_RESULT direct refs with
_make_empty_result() factory. dict(...) shallow copy left nested
nspd_dump shared by reference across concurrent requests — single
mutation pollutes module sentinel for all subsequent calls. Now
каждый caller gets independent dict.
M2 (O(N) spatial scan): SELECT extended denormalized counts
(territorial_zones_count, zouit_count, engineering_count). Each
spatial helper accepts layer_counts and early-returns when count=0
— skips heavy jsonb_array_elements + ST_Transform/ST_Intersects
scan entirely. Critical для quarters с 2000+ features.
M4 (documentation): _trigger_harvest docstring describes known
burst/no-dedup limitation + TODO Redis SETNX (отдельный PR).
M5 (test fragility): _make_db_mock_with_spatial docstring describes
positional-call contract — db.execute order (0=dump, 1=zoning, 2=zouit,
3=engineering) и зависимость от count-values.
+4 new tests (17 total, all pass):
- test_make_empty_result_returns_independent_copies (mutation safety)
- test_make_empty_result_overrides
- test_early_exit_all_counts_zero_no_spatial_queries
- test_early_exit_partial_counts
Per auto-review on 3068a9c.
* fix(site-finder): rename _make_empty_result → make_empty_result (public) per PR #116 review
M1 residual fix: parcels.py exception path использовал EMPTY_DUMP_RESULT
singleton ref вместо factory. Сейчас readonly access, но нарушает
documented invariant модуля.
Rename `_make_empty_result` → `make_empty_result` (public API), import в
parcels.py, использовать в try/except fallback. Каждый request получает
независимый dict — никаких shared references.
M4 (Redis SETNX dedup) + M5 (test fragility) — deferred per review,
documented в code/issue. Acceptable trade-offs:
- M4: UPSERT idempotency делает данные safe; burst-duplicate task'и тратят
WAF traffic впустую но не повреждают данные.
- M5: docstring contractually describes positional-call order.
17/17 tests pass. ruff/format clean.
Per auto-review on aef8308.
---------
Co-authored-by: lekss361 <claudestars@proton.me>
|
2026-05-13 09:14:19 +03:00 |
|
