Commit graph

5 commits

Author SHA1 Message Date
2c22c3f7ea feat(site-finder): per-building помещения/машино-места + parking_ratio (on-demand MVP, #96)
All checks were successful
CI / frontend-tests (push) Has been skipped
CI / changes (pull_request) Successful in 6s
CI / changes (push) Successful in 8s
CI / frontend-tests (pull_request) Has been skipped
CI / openapi-codegen-check (push) Successful in 1m59s
CI / openapi-codegen-check (pull_request) Successful in 1m58s
CI / backend-tests (push) Successful in 8m48s
CI / backend-tests (pull_request) Successful in 8m48s
Deploy / changes (push) Successful in 6s
Deploy / build-frontend (push) Has been skipped
Deploy / build-backend (push) Successful in 4m58s
Deploy / build-worker (push) Successful in 5m46s
Deploy / deploy (push) Successful in 1m30s
Оживляет мёртвый foundation list_objects_in_building (#168 Q3-deferred). Fix реального
блокера: NSPDOptions не парсил objdocId (camelCase из NSPD search) → objdoc_id всегда None,
метод нельзя было вызвать. AliasChoices(objdoc_id, objdocId) + registers.

premises_lookup.get_building_premises(cad_num) — резолв objdoc → list_objects_in_building,
graceful (WAF/сеть/not-found → None). parking_ratio = машино-места/помещения (None при 0
помещений, 0.0 при реальном отсутствии паркинга). Verified live 66:41:0106036:183.

MVP on-demand (без bulk/схемы/prod-записей). parking_ratio готов, но НЕ wired в analyze:
конкуренты приходят из ДОМ.РФ без cad_num → нужен domrf↔cad_buildings geom-match (отдельная
задача). Сервис примет cad_num как только matching появится. 11 тестов, ruff clean.

Refs #96
2026-06-14 18:48:16 +05:00
1883271908 fix(cadastre): NSPD 500-retry + per-layer skip (#252) + disable dead pzz-beat (#259) + geom backfill (#200)
Some checks failed
CI / changes (push) Successful in 7s
CI / frontend-tests (push) Has been skipped
CI / changes (pull_request) Successful in 6s
CI / frontend-tests (pull_request) Has been skipped
CI / backend-tests (push) Successful in 6m43s
CI / backend-tests (pull_request) Successful in 6m41s
Deploy / build-backend (push) Blocked by required conditions
Deploy / deploy (push) Blocked by required conditions
Deploy / build-worker (push) Blocked by required conditions
Deploy / build-frontend (push) Blocked by required conditions
Deploy / changes (push) Has been cancelled
#252: NspdBulkServerError на HTTP 5xx + WMS <ServiceException>-body → quarter-level
autoretry; _grid_walk_category флагует layer_failed только если ВСЕ cells 5xx (none ok),
network-errors исключены — пустой/флапающий слой не ложно-скипается. harvest_meta.layer_X_failed.

#259: pzz-sync-monthly beat отключён (PKK6 deprecated, pzz_zones_ekb=0, заменён
zone_regulation_cache+NSPD dumps). logger.exception→warning (стоп GlitchTip BACKEND-1B),
raise сохранён для admin-endpoint.

#200: geom_unavailable флаг (миграция 151) + backfill_parcel_geom task — grid-walk
layer 36368 per quarter лечит 930/964 NULL-geom участков (NSPD search вернул центроиды),
остаток marks unavailable (не ретраить вечно). SAVEPOINT per-cell, idempotent.

Closes #252
Closes #259
Closes #200
2026-06-13 20:45:05 +05:00
lekss361
570c987e78 fix(#231): repair test_rate_limit_semaphore regression + docstring sync
Re-review feedback (PR #260 head 8191a85):

1. tests/scrapers/test_nspd_bulk_client.py:360 — test referenced bulk_mod._SEMAPHORE
   which was renamed to _SEMAPHORE_LIMIT in previous commit. AttributeError на
   полном pytest run. Заменено: assert bulk_mod._SEMAPHORE_LIMIT == 3 (smoke).
   max_concurrent <= 3 уже валидирует throttling — capacity check был лишним.

2. app/services/scrapers/nspd_client.py docstrings (lines 483, 506) — обновлены
   ссылки NSPDBulkClient._SEMAPHORE(3) → NSPDBulkClient._sem (per-instance,
   capacity=3) чтобы соответствовать новой архитектуре.
2026-05-17 10:22:16 +03:00
lekss361
99e2210919
fix(cadastre): NSPD response parse — properties.category + top-level meta (#168) (#177)
* fix(cadastre): NSPD response parse — properties.category + top-level meta (#168)

* fix(cadastre): test fixtures use real NSPD shape — properties.category + top-level meta (#168)

Bot review of #177 flagged that fixtures still used legacy properties.categoryId
+ data.meta — meaning CI exercised only fallback branches, not primary paths
where the actual bug lived. Pattern of 3 consecutive "live-only" parse fixes
(#175 verify, #176 headers, #177 parse) confirmed need for test coverage.

Changes:
- test_nspd_bulk_client.py: sample_quarter_response → properties.category +
  meta moved to top-level (real NSPD shape verified live)
- test_cadastre_bulk.py: 7 fixtures categoryId → category (regex replace)
- test_nspd_bulk_feature_parse_basic: primary path now exercised

Plus schema hardening per bot review:
- NSPDBulkFeature.category_id: `is not None` check (not truthy `or`)
  to avoid edge case category=0; int() wrapped in try/except so
  non-numeric ID (e.g. "ЗУ") doesn't crash upsert_features loop.

35/35 tests pass locally.

---------

Co-authored-by: lekss361 <claudestars@proton.me>
2026-05-15 15:40:01 +03:00
lekss361
b7889d113b
feat(cadastre): NSPDBulkClient + schemas for bulk ingest (#168 PR2/5) (#170)
Add async NSPD client (nspd_bulk_client.py) with 3 new methods:
- search_by_quarter: REST /api/geoportal/v2/search/geoportal — snapshot 20 per cat + meta totals
- wms_feature_info: WMS GetFeatureInfo для grid-walk discovery
- list_objects_in_building: tab-group-data (Q3 deferred — код ready)

Rate limit: asyncio.Semaphore(3) + 0.05s jitter.
429 → exponential backoff 2→4→8s, max 3 retries.
403 (WAF) → raise immediately, no retry.

Pydantic schemas (nspd_bulk.py): NSPDBulkFeature, QuarterSnapshot, ObjectsListing
with extra='allow' to preserve unknown fields → raw_props JSONB downstream.

15 unit tests + 4 e2e (real NSPD, skipped in CI by default).

Co-authored-by: lekss361 <claudestars@proton.me>
2026-05-15 12:50:41 +03:00