fix(tradein/scraper-kit): scale cian anchor watchdog for proxy-pool browser fetches (#2160) #2272

Merged
lekss361 merged 1 commit from fix/tradein-cian-anchor-timeout-pool into main 2026-07-03 07:09:51 +00:00

1 commit

Author SHA1 Message Date
bot-backend
8a10d943d7 fix(tradein/scraper-kit): scale cian anchor watchdog for proxy-pool browser fetches (#2160)
All checks were successful
CI Trade-In / changes (pull_request) Successful in 11s
CI / changes (pull_request) Successful in 11s
CI Trade-In / frontend-checks (pull_request) Has been skipped
CI / backend-tests (pull_request) Has been skipped
CI / frontend-tests (pull_request) Has been skipped
CI / openapi-codegen-check (pull_request) Has been skipped
CI Trade-In / backend-tests (pull_request) Successful in 1m51s
Fixed ANCHOR_TIMEOUT_SEC=240 guillotined every cian anchor when running via
USE_PROXY_POOL_BROWSER=true: each SERP fetch goes through camoufox with relaunch
on proxy swap (13-45s/page), and an anchor = 4 room-buckets x pages_per_anchor
SERP fetches + detail_top_n details + houses-enrich >> 240s. save_listings never
ran, lots_fetched=0, and after 3 consecutive skips the run was mark_banned.

Introduces _cian_anchor_timeout_s() mirroring the avito/yandex/domclick scaling
pattern: max(ANCHOR_TIMEOUT_SEC, buckets*pages*(delay+per_serp) + details*per_detail
+ houses_budget). Defaults -> 1580s worst-case watchdog (hang guard, not expected
duration). Applied to both the scraper-kit copy (strangler target) and the active
app.services.scrape_pipeline copy that run 581 actually hit.
2026-07-03 10:05:06 +03:00