|
|
dc7a84cf2e
|
feat(tradein): per-anchor watchdog timeout in city sweeps (#880)
run_{avito,cian,yandex}_city_sweep had no per-anchor watchdog: a stalled HTTP
request (proxy/connection hang) blocked the asyncio task forever — heartbeat
froze mid-anchor, run stayed 'running' until the 6h zombie-detect. Live-repro:
cian sweep #24 hung on anchor 3, #29 on anchor 1.
Wrap each anchor's processing in asyncio.wait_for(coro, ANCHOR_TIMEOUT_SEC=240).
TimeoutError -> logger.warning + errors_count++ + continue (sweep finishes,
reaches mark_done). Timeout branch sits before the existing except Exception.
yandex/cian also bump consecutive_failures -> existing MAX_CONSECUTIVE_FAILURES
abort fires on 3 consecutive hung anchors. CianScraper SERP session already
has timeout=30. 5 unit tests (all 3 sweeps), network-free.
Refs #880, #581, #559
|
2026-05-31 13:33:39 +03:00 |
|
