Light1YT
f87b6ada4c
Root: city-sweep counters (yandex/avito/cian/domclick/newbuilding) wrote lots_fetched/lots_inserted only into scrape_runs.counters jsonb; the dedicated columns total_seen/new_count (schema 015) were never SET, so admin/observability reported total_seen=0 despite real rows saved (audit #1871/#1926). Fix (Option A): single-place extraction in scrape_runs.py via _column_counts() maps counters lots_fetched->total_seen and lots_inserted->new_count (with explicit total_seen/new_count keys taking priority). update_heartbeat, mark_done, mark_failed and mark_banned now SET both columns with COALESCE(CAST(:x AS int), col) so a missing key preserves the existing value. Covers all sweep sources at once since every pipeline routes counters through these finalizers. |
||
|---|---|---|
| .. | ||
| api | ||
| core | ||
| observability | ||
| schemas | ||
| services | ||
| tasks | ||
| __init__.py | ||
| main.py | ||
| scheduler_main.py | ||