feat(scrapers): live pacing-регулятор + data-quality coverage API #1829

Merged
lekss361 merged 1 commit from feat/scrapers-config-apis into main 2026-06-20 12:31:48 +00:00

1 commit

Author SHA1 Message Date
b6634ea40c feat(scrapers): live pacing-регулятор (GET/PUT /scraper/pacing) + data-quality coverage endpoint
All checks were successful
CI / changes (pull_request) Successful in 7s
CI / backend-tests (pull_request) Has been skipped
CI / frontend-tests (pull_request) Has been skipped
CI / openapi-codegen-check (pull_request) Has been skipped
- browser/server.py: GET /pacing и PUT /pacing — read/write _MIN_PAGE_INTERVAL_BY_PROVIDER
  in-memory; валидация source in PROVIDERS, interval_s >= 0; логирует изменения; сбрасывается
  к env-дефолту на рестарте by design
- backend/admin.py: GET /scraper/pacing — прокси к browser /pacing, PacingResponse pydantic;
  PUT /scraper/pacing/{source} — прокси PUT с валидацией ge=0 le=120; 502/503 при недоступности
- backend/admin.py: GET /scraper/data-quality — single-pass FILTER-агрегаты по listings WHERE
  is_active GROUP BY source (description/photo_urls/address/lat/lon/kitchen_area_m2/living_area_m2/
  ceiling_height/ceiling_height_m/metro_stations); houses (total/avito_validated_at%/rating_score%/
  house_type%); house_reviews count
- browser/test_server_pacing.py: 10 новых тестов GET+PUT /pacing (16 total, все зелёные)
- backend/tests/test_scraper_admin_apis.py: тесты pacing-прокси + data-quality shape/pct-range
  (21 total, все зелёные)
2026-06-20 12:47:03 +03:00