feat(scrapers): ночной avito detail-enrichment backfill для legacy listings (Closes #1551) #1552

Merged
bot-backend merged 1 commit from feat/avito-detail-backfill into main 2026-06-16 10:32:58 +00:00

1 commit

Author SHA1 Message Date
227e82dc01 feat(tradein): avito detail-enrichment nightly backfill (#1551)
All checks were successful
CI / changes (push) Successful in 7s
CI / backend-tests (push) Has been skipped
CI / frontend-tests (push) Has been skipped
CI / openapi-codegen-check (push) Has been skipped
CI / changes (pull_request) Successful in 6s
CI / backend-tests (pull_request) Has been skipped
CI / frontend-tests (pull_request) Has been skipped
CI / openapi-codegen-check (pull_request) Has been skipped
- New task app/tasks/avito_detail_backfill.py with run_avito_detail_backfill()
  * Single snapshot SELECT at start (guarantees termination)
  * Same proxy/AsyncSession path as scrape_pipeline.py step 5
  * Budget guard (budget_sec), consecutive block abort (mark_done not mark_failed)
  * rotate_ip() on every AvitoBlockedError; rollback on generic Exception (#1368)
  * start = time.monotonic() initialized before try so except can reference it
- Scheduler wiring: trigger_avito_detail_backfill_run() + elif in scheduler_loop()
- Migration 112: scrape_schedules INSERT window 09-12 UTC, batch_size=800,
  budget_sec=3600, request_delay_sec=6, max_consecutive_blocks=5
- 7 unit tests (no pytest-mock, unittest.mock only): all 7 passing,
  full CI suite 1809 passed
2026-06-16 13:25:29 +03:00