5da66bbc93
* feat(cadastre): bulk_harvest_quarter Celery task + grid-walker + saga state (#168 PR3/5) Add bulk cadastre harvest pipeline: - services/cadastre/bulk_harvest.py: async harvest_quarter() orchestrator (4 phases) + 7 upsert helpers (parcels/buildings/constructions/oncs/enks/zouit/quarter_stats) using CAST(:x AS jsonb) pattern, begin_nested() SAVEPOINT per grid-walk upsert - services/cadastre/grid_geometry.py: quarter_bbox_3857 (PostGIS ST_Extent) + generate_grid_click_points (15x15 = 225 sub-bbox grid) - workers/tasks/scrape_cadastre.py: bulk_harvest_quarter_task (acks_late=True, dont_autoretry_for NspdBulkWafError), enqueue_cadastre_harvest, cleanup_zombies - api/v1/admin_cadastre.py: 5 endpoints behind AdminTokenAuth — create/list/get/cancel/resume - Tests: 13 service unit + 8 API tests * fixup(cadastre): drop importorskip (PR2 merged) — imports now top-level (#168 PR3) * fixup(cadastre): tile_size unified param + register slow marker (#168 PR3) Blocker #1: rename tile_width/tile_height → tile_size in generate_grid_click_points. Callers in bulk_harvest.py and test_cadastre_bulk.py used tile_size; def had tile_width+tile_height → TypeError at runtime. Blocker #2 (10 failures in test_admin_cadastre.py) was side-effect of #1: TypeError at import chain (app.main → admin_cadastre → bulk_harvest) broke FastAPI app load → dependency_overrides AttributeError. Now resolved. Also register `slow` pytest marker in pyproject.toml to suppress PytestUnknownMarkWarning. * fixup(cadastre): use importlib.util.find_spec instead of import (#168 PR3) Root cause Blocker #2: `import app.workers.tasks.scrape_cadastre` at module top-level REBOUND `app` from FastAPI instance (line 14) to the Python package. All subsequent `app.dependency_overrides[get_db] = ...` failed with AttributeError because `app` was now the namespace module. Fix: probe module availability with importlib.util.find_spec — does not import or rebind names. FastAPI `app` instance stays intact. All 28 cadastre tests pass locally. --------- Co-authored-by: lekss361 <claudestars@proton.me> |
||
|---|---|---|
| .. | ||
| cadastre | ||
| exporters | ||
| generative | ||
| photos | ||
| scrapers | ||
| site_finder | ||
| __init__.py | ||
| analytics_queries.py | ||
| analytics_refresh.py | ||
| job_settings.py | ||
| objective_etl.py | ||
| objective_sync_config.py | ||