fix(scrapers): avito ban-triggered IP rotation (#1790) разблокирует houses-enrichment (#1789) + rating_score #1848

Merged

lekss361 merged 1 commit from fix/avito-ban-rotation-houses into main

2026-06-20 18:28:30 +00:00

Author	SHA1	Message	Date
bot-backend	7f036a96af	fix(scrapers): avito ban-triggered IP rotation (#1790 ) — changeip on block, bounded All checks were successful CI / changes (pull_request) Successful in 6s Details CI / backend-tests (pull_request) Has been skipped Details CI / frontend-tests (pull_request) Has been skipped Details CI / openapi-codegen-check (pull_request) Has been skipped Details - Добавлен `_rotate_proxy_ip()` в scrape_pipeline.py: вызывает changeip-URL (settings.avito_proxy_rotate_url) + settle-sleep (settings.avito_proxy_rotate_settle_s, default 9s), логирует old→new IP. - В `run_avito_pipeline` (Steps 4/5/5b) и `run_avito_city_sweep` (Phase 4/5/5b): при достижении consecutive-block порога — сначала ротация IP, сброс счётчика, продолжение; только при исчерпании бюджета (enrich_rotations_done >= avito_proxy_max_rotations) — abort (bounded). #1820 graceful сохранён: один блок ниже порога ротацию не вызывает. - Добавлено `avito_proxy_rotate_settle_s: float = 9.0` в config.py (ENV: AVITO_PROXY_ROTATE_SETTLE_S). avito_proxy_max_rotations был 4 (с #1731). - Тесты: rotation_called / bounded_abort / single_block_no_rotation.	2026-06-20 21:24:12 +03:00

Author

SHA1

Message

Date

bot-backend

7f036a96af

fix(scrapers): avito ban-triggered IP rotation (#1790 ) — changeip on block, bounded

CI / changes (pull_request) Successful in 6s

Details

CI / backend-tests (pull_request) Has been skipped

Details

CI / frontend-tests (pull_request) Has been skipped

Details

CI / openapi-codegen-check (pull_request) Has been skipped

Details

- Добавлен `_rotate_proxy_ip()` в scrape_pipeline.py: вызывает changeip-URL
  (settings.avito_proxy_rotate_url) + settle-sleep
  (settings.avito_proxy_rotate_settle_s, default 9s), логирует old→new IP.
- В `run_avito_pipeline` (Steps 4/5/5b) и `run_avito_city_sweep` (Phase 4/5/5b):
  при достижении consecutive-block порога — сначала ротация IP, сброс счётчика,
  продолжение; только при исчерпании бюджета (enrich_rotations_done >=
  avito_proxy_max_rotations) — abort (bounded). #1820 graceful сохранён: один
  блок ниже порога ротацию не вызывает.
- Добавлено `avito_proxy_rotate_settle_s: float = 9.0` в config.py
  (ENV: AVITO_PROXY_ROTATE_SETTLE_S). avito_proxy_max_rotations был 4 (с #1731).
- Тесты: rotation_called / bounded_abort / single_block_no_rotation.

2026-06-20 21:24:12 +03:00