fix(tradein): persist scrape_runs.total_seen/new_count from sweep counters (#1926) #1928

Merged
bot-backend merged 1 commit from fix/1926-scrape-total-seen into main 2026-06-26 18:09:18 +00:00

1 commit

Author SHA1 Message Date
f87b6ada4c fix(tradein): persist scrape_runs.total_seen/new_count columns from sweep counters (#1926)
All checks were successful
CI / changes (pull_request) Successful in 6s
CI / frontend-tests (pull_request) Has been skipped
CI / backend-tests (pull_request) Has been skipped
CI / openapi-codegen-check (pull_request) Has been skipped
Root: city-sweep counters (yandex/avito/cian/domclick/newbuilding) wrote
lots_fetched/lots_inserted only into scrape_runs.counters jsonb; the dedicated
columns total_seen/new_count (schema 015) were never SET, so admin/observability
reported total_seen=0 despite real rows saved (audit #1871/#1926).

Fix (Option A): single-place extraction in scrape_runs.py via _column_counts()
maps counters lots_fetched->total_seen and lots_inserted->new_count (with
explicit total_seen/new_count keys taking priority). update_heartbeat, mark_done,
mark_failed and mark_banned now SET both columns with COALESCE(CAST(:x AS int),
col) so a missing key preserves the existing value. Covers all sweep sources at
once since every pipeline routes counters through these finalizers.
2026-06-26 22:59:27 +05:00