lekss361
ae04dcb4b9
perf(cadastre): freshness skip + grid-walk heartbeat + reduce WMS log noise
Three operational improvements after ekb_full v3 completed at 99.1%:
Fix H — per-quarter freshness skip (scrape_cadastre.py):
- enqueue_cadastre_harvest queries cad_quarter_stats.fetched_at per quarter
- Skip quarters where fetched_at >= NOW() - skip_fresh_hours (default 1h)
- extra_config.skip_fresh_hours=0 disables skip (full re-fetch)
- Saves ~30k NSPD calls on repeated runs of same scope
Fix I — heartbeat in grid-walk loop (bulk_harvest.py:_grid_walk_category):
- 225 cells x 70-300ms = 15-75s heartbeat silence
- Pass update_progress callback to _grid_walk_category
- Call every heartbeat_every (50) cells with phase progress info
- Prevents cleanup_zombies false-positive cancellation
Fix J — NSPD WMS HTTP 500 noise to debug (bulk_harvest.py:_grid_walk_category):
- NSPD WMS endpoint frequently returns HTTP 500 (server-side issue)
- Was: logger.warning each, 200+ warnings per quarter
- Now: logger.debug (visible via --log-level=DEBUG)
40 tests passing.