feat(yandex): route gate-API fetch through camoufox (sustainable, no proxy burn) #1699

Merged
lekss361 merged 1 commit from feat/yandex-camoufox-transport into main 2026-06-17 18:28:51 +00:00

1 commit

Author SHA1 Message Date
a8d7be68ad feat(yandex): route gate-API fetch through camoufox BrowserFetcher
Some checks failed
CI / changes (push) Has been cancelled
CI / backend-tests (push) Has been cancelled
CI / frontend-tests (push) Has been cancelled
CI / openapi-codegen-check (push) Has been cancelled
CI / changes (pull_request) Has been cancelled
CI / backend-tests (pull_request) Has been cancelled
CI / frontend-tests (pull_request) Has been cancelled
CI / openapi-codegen-check (pull_request) Has been cancelled
Replace the system-curl SOCKS5 subprocess transport with the shared
BrowserFetcher (camoufox). camoufox's real-browser fingerprint is not
tarpitted by yandex and does not burn the mobile-proxy IP; raw system-curl
got the proxy IP flagged.

One BrowserFetcher is opened per scraper session in __aenter__ and reused
across all page fetches. _http_get now fetches via the browser, extracts the
gate-API JSON from camoufox's HTML <pre> wrapper, and returns it as the body
of a _CurlResponse (status 200 on success, 0 on fetch/extraction failure) so
all existing callers and the tarpit retry/rotate logic keep working unchanged.

URL building, _parse_gate_json, pagination, combos and field mapping are
untouched. Removed the dead curl subprocess constants/method.
2026-06-17 21:16:48 +03:00