e695bca1e4
cadastre_jobs.phase_state is shared between ALL parallel Celery workers of the same job. The early-exit triggered as soon as the FIRST worker wrote phase=done via progress_cb — all subsequent workers (reading same shared row) bailed without work. Effect: pilot v7 made 5 NSPD requests for 50 quarters (=0.1/quarter). Full ekb_full job #8: 4950 req / 2408 quarters = 2/quarter — 95% workers early-exited without doing snapshot phase. Root cause of 1.6% parcels coverage. Fix A/B/C only partially helped because most workers never reached Phase 1/1.5. Solution: delete the early-exit. Idempotency is guaranteed by ON CONFLICT DO UPDATE in every upsert_* — re-enqueued tasks write same rows without duplicates. Closes part of #168 (root-cause fix for race condition revealed by #182 metrics). Co-authored-by: lekss361 <claudestars@proton.me> |
||
|---|---|---|
| .. | ||
| api | ||
| core | ||
| models | ||
| schemas | ||
| scrapers | ||
| services | ||
| workers | ||
| __init__.py | ||
| main.py | ||