fix(#231): switch search_by_quarter area layers to grid-walk (Sub-PR B) #260

Merged

lekss361 merged 3 commits from fix/231-nspd-search-by-quarter-grid into main

2026-05-17 07:26:09 +00:00

Author SHA1 Message Date

Author	SHA1	Message	Date
lekss361	570c987e78	fix(#231 ): repair test_rate_limit_semaphore regression + docstring sync Re-review feedback (PR #260 head `8191a85`): 1. tests/scrapers/test_nspd_bulk_client.py:360 — test referenced bulk_mod._SEMAPHORE which was renamed to _SEMAPHORE_LIMIT in previous commit. AttributeError на полном pytest run. Заменено: assert bulk_mod._SEMAPHORE_LIMIT == 3 (smoke). max_concurrent <= 3 уже валидирует throttling — capacity check был лишним. 2. app/services/scrapers/nspd_client.py docstrings (lines 483, 506) — обновлены ссылки NSPDBulkClient._SEMAPHORE(3) → NSPDBulkClient._sem (per-instance, capacity=3) чтобы соответствовать новой архитектуре.	2026-05-17 10:22:16 +03:00
lekss361	8191a85102	fix(#231 ): address re-review blockers — per-instance semaphore + time_limit + test mocks PR #260 second-opinion review 🔴 BLOCK → fixes: 1. nspd_bulk_client.py: _SEMAPHORE module-level → per-instance self._sem создаётся в __aenter__ под текущий event loop. Sequential asyncio.run() в search_by_quarter (11 layers × asyncio.run() per layer) bind'ил module-level semaphore к first loop'у → RuntimeError на втором вызове → все cells return_exceptions=True проглатывали → grid возвращал 0 features. 2. nspd_sync.py: harvest_quarter добавлен time_limit=900 (hard) рядом с soft_time_limit=600. Safety net если task игнорит SoftTimeLimitExceeded. 3. tests/test_nspd_client.py: добавлен _make_fake_grid_walk helper + monkeypatch NSPDClient.get_features_in_bbox_grid в 3 тестах (search_by_quarter_core_only / _with_zouit / _layers_fetched_with_risks). Без этого area layers били бы по живому NSPD API → CI flaky. Tests: 37 passed, 1 skipped (3 area-dispatch + 34 grid-walk).	2026-05-17 10:13:34 +03:00
lekss361	fdced94626	fix(#231 ): switch search_by_quarter area layers to grid-walk dispatch (Sub-PR B) - Add _GRID_WALK_LAYERS frozenset (territorial_zones, red_lines, engineering_structures, zouit_, risk_) - _fetch_layer dispatch: area layers → get_features_in_bbox_grid(grid_n=7, step_m=50) [foundation из PR #247], EGRN layers (parcels, buildings) → legacy get_features_in_bbox - Enrich engineering_structures features с classified_kind через classify_engineering_kind (PR #247 classifier) - Per-layer logger.info heartbeat (method + count) для прогресса - Bump harvest_quarter soft_time_limit 120 → 600s (grid-walk: 11 layers × 49 cells × ~70ms ≈ 40s + retries margin) - TestFetchLayerDispatch (3 tests): 34 passed, 1 skipped Pre-push deep-code-reviewer: APPROVE (1 medium nit — opportunity_oopt не в grid-walk, защищено через include_opportunity=False default; follow-up issue рекомендован).	2026-05-17 09:46:42 +03:00

lekss361

570c987e78

fix(#231 ): repair test_rate_limit_semaphore regression + docstring sync

Re-review feedback (PR #260 head 8191a85):

1. tests/scrapers/test_nspd_bulk_client.py:360 — test referenced bulk_mod._SEMAPHORE
   which was renamed to _SEMAPHORE_LIMIT in previous commit. AttributeError на
   полном pytest run. Заменено: assert bulk_mod._SEMAPHORE_LIMIT == 3 (smoke).
   max_concurrent <= 3 уже валидирует throttling — capacity check был лишним.

2. app/services/scrapers/nspd_client.py docstrings (lines 483, 506) — обновлены
   ссылки NSPDBulkClient._SEMAPHORE(3) → NSPDBulkClient._sem (per-instance,
   capacity=3) чтобы соответствовать новой архитектуре.

2026-05-17 10:22:16 +03:00

lekss361

8191a85102

fix(#231 ): address re-review blockers — per-instance semaphore + time_limit + test mocks

PR #260 second-opinion review 🔴 BLOCK → fixes:

1. nspd_bulk_client.py: _SEMAPHORE module-level → per-instance self._sem
   создаётся в __aenter__ под текущий event loop. Sequential asyncio.run()
   в search_by_quarter (11 layers × asyncio.run() per layer) bind'ил
   module-level semaphore к first loop'у → RuntimeError на втором вызове →
   все cells return_exceptions=True проглатывали → grid возвращал 0 features.

2. nspd_sync.py: harvest_quarter добавлен time_limit=900 (hard) рядом с
   soft_time_limit=600. Safety net если task игнорит SoftTimeLimitExceeded.

3. tests/test_nspd_client.py: добавлен _make_fake_grid_walk helper +
   monkeypatch NSPDClient.get_features_in_bbox_grid в 3 тестах
   (search_by_quarter_core_only / _with_zouit / _layers_fetched_with_risks).
   Без этого area layers били бы по живому NSPD API → CI flaky.

Tests: 37 passed, 1 skipped (3 area-dispatch + 34 grid-walk).

2026-05-17 10:13:34 +03:00

lekss361

fdced94626

fix(#231 ): switch search_by_quarter area layers to grid-walk dispatch (Sub-PR B)

- Add _GRID_WALK_LAYERS frozenset (territorial_zones, red_lines,
  engineering_structures, zouit_*, risk_*)
- _fetch_layer dispatch: area layers → get_features_in_bbox_grid(grid_n=7,
  step_m=50) [foundation из PR #247], EGRN layers (parcels, buildings) →
  legacy get_features_in_bbox
- Enrich engineering_structures features с classified_kind через
  classify_engineering_kind (PR #247 classifier)
- Per-layer logger.info heartbeat (method + count) для прогресса
- Bump harvest_quarter soft_time_limit 120 → 600s
  (grid-walk: 11 layers × 49 cells × ~70ms ≈ 40s + retries margin)
- TestFetchLayerDispatch (3 tests): 34 passed, 1 skipped

Pre-push deep-code-reviewer: APPROVE (1 medium nit — opportunity_oopt не в
grid-walk, защищено через include_opportunity=False default; follow-up issue
рекомендован).

2026-05-17 09:46:42 +03:00