Commit graph

4 commits

Author SHA1 Message Date
e5f4e37415 feat(scrapers): route cian sessions through mobile proxy (Refs #806)
Снимает egress-часть #639: cian-скрейперы ходили с datacenter-IP (без proxies)
→ даже валидные куки не фетчили с прод-бокса. Прокидываем тот же mobile-proxy,
что у avito (#623/#805).

- config: generic scraper_proxy_url (env SCRAPER_PROXY_URL → fallback
  AVITO_PROXY_URL → None/direct). Прод .env.runtime с AVITO_PROXY_URL работает
  без изменений env.
- avito.py + scrape_pipeline._avito_proxies(): на settings.scraper_proxy_url
  (поведение идентично).
- cian.py / cian_detail.fetch_detail / cian_session.verify_session: proxies=
  {http,https} из settings.scraper_proxy_url; пусто → direct (dev no-op).

Cookie-refresh DMIR_AUTH (#639) — НЕ тронут (needs-human). 7 тестов
(precedence + dict-shape + fallback), ruff clean.
2026-05-30 22:01:21 +03:00
8f2ff10a1c fix(cian): verify_session на curl_cffi impersonate — ban != cookies expired (Refs #768) (#785)
All checks were successful
Deploy Trade-In / test (push) Successful in 24s
Deploy Trade-In / build-backend (push) Successful in 38s
Deploy Trade-In / changes (push) Successful in 5s
Deploy Trade-In / build-frontend (push) Has been skipped
Deploy Trade-In / deploy (push) Successful in 34s
Co-authored-by: bot-backend <bot-backend@gendsgn.local>
Co-committed-by: bot-backend <bot-backend@gendsgn.local>
2026-05-30 16:40:54 +00:00
99dd403c45 fix(tradein): CIAN_REQUIRED_COOKIES — add real auth cookies (DMIR_AUTH + Cian session) (#480)
All checks were successful
Deploy Trade-In / build-backend (push) Successful in 39s
Deploy Trade-In / deploy (push) Successful in 31s
Deploy Trade-In / changes (push) Successful in 4s
Deploy Trade-In / build-frontend (push) Has been skipped
Updated CIAN_REQUIRED_COOKIES filter в cian_session.py по реальному DevTools-дампу logged-in cian.ru сессии (2026-05-23). Добавлены critical auth cookies: DMIR_AUTH (httpOnly), _CIAN_GK, _yasc, _ym_visorc, uxfb_card_satisfaction. Legacy entries сохранены как backward-compat fallback. Regression test test_required_cookies_includes_real_auth гарантирует что все реальные cookies проходят filter.

Fixes Stage 4 PR #457 (filter guessed без real probe — DMIR_AUTH был dropped → verify_session falsey).
2026-05-23 14:52:24 +00:00
2dd76ea5e8 feat(tradein): cian_session cookies management + admin endpoints (Wave 4 Worker A) (#457)
Some checks failed
Deploy Trade-In / changes (push) Successful in 4s
Deploy Trade-In / build-frontend (push) Has been skipped
Deploy Trade-In / build-backend (push) Successful in 39s
Deploy Trade-In / deploy (push) Has been cancelled
2026-05-23 13:22:00 +00:00