Light1YT
c70154ed61
Public overpass-api.de throttled later heavy query-types (429/504) → only ~1099/3000 utility elements loaded. Harden the loader: - per-query retry-with-backoff on 429/5xx (esp. 504): up to 3 attempts per endpoint, exponential backoff (10s/30s/60s) + jitter, honoring the Retry-After header when present. - mirror fallback: OVERPASS_ENDPOINTS list (overpass-api.de → kumi.systems → private.coffee); exhausting one endpoint rotates to the next mirror. - inter-query delay bumped 1s → 3s (gentler on the public endpoint). - a query that fails all retries+mirrors is logged and skipped (partial-load, not abort) — existing per-element SAVEPOINT/UPSERT kept. Tests: 429-then-200 retry, persistent-429 primary → mirror fallback, 504 → mirror, all-fail skip-without-abort, one-failing-query-does-not- abort-others, plus Retry-After/backoff helper units. All mocked, no real Overpass. |
||
|---|---|---|
| .. | ||
| api | ||
| core | ||
| models | ||
| observability | ||
| schemas | ||
| scrapers | ||
| services | ||
| templates | ||
| workers | ||
| __init__.py | ||
| main.py | ||