Commit graph

10 commits

Author SHA1 Message Date
a07fe55db9 fix(tradein): route yandex/valuation/imv/newbuilding scrapers through scraper_proxy_url
Yandex SERP was being banned (datacenter IP) and on-demand scrapers (cian_valuation,
cian_newbuilding, avito_imv, avito_detail) silently returned None due to same cause.
Mobile proxy was already wired for cian.py/cian_detail.py/avito.py but missing elsewhere.

- yandex_realty: proxies= added to _CurlCffiSession in __aenter__
- yandex_valuation: proxies= added to _CurlCffiSession in __aenter__
- cian_valuation: proxies= added to AsyncSession (async with ... fetch block)
- cian_newbuilding: proxies= added to both own-session paths (fetch_newbuilding +
  resolve_cian_zhk_url); shared-session callers unaffected
- avito_imv: proxies= added to own-session path only (shared CffiAsyncSession from
  AvitoScraper already carries proxy)
- avito_detail: proxies= added to own-session path only (scrape_pipeline passes shared
  session with proxy; admin endpoint creates own-session -> now proxied)

All use pattern: _proxy = settings.scraper_proxy_url; proxies={"http":_proxy,"https":_proxy}
if _proxy else None -- matching cian.py (#806 reference). proxy=None = direct connection (dev).

Tests: 9 new proxy-wiring unit tests added to test_scraper_proxy.py (20 total, all pass).
2026-05-31 09:57:17 +03:00
6bbb026cb4 fix(yandex-valuation): off-by-one cell index in _parse_row_cells (#552)
Some checks failed
Deploy Trade-In / build-backend (push) Successful in 40s
Deploy Trade-In / deploy (push) Has been cancelled
Deploy Trade-In / changes (push) Successful in 5s
Deploy Trade-In / build-frontend (push) Has been skipped
2026-05-24 19:57:17 +00:00
f7a5c5d4f7 fix(yandex-valuation): curl_cffi chrome120 — Yandex gates SSR on Chrome TLS (#545)
All checks were successful
Deploy Trade-In / changes (push) Successful in 5s
Deploy Trade-In / build-frontend (push) Has been skipped
Deploy Trade-In / build-backend (push) Successful in 39s
Deploy Trade-In / deploy (push) Successful in 45s
2026-05-24 16:42:07 +00:00
8a88ea6508 refactor(yandex-valuation): DOM-based parser via .OffersArchiveSearchOffers__row (#544)
Some checks failed
Deploy Trade-In / changes (push) Successful in 5s
Deploy Trade-In / build-frontend (push) Has been skipped
Deploy Trade-In / build-backend (push) Successful in 40s
Deploy Trade-In / deploy (push) Failing after 10m3s
2026-05-24 16:25:56 +00:00
3ed58e3e7a fix(yandex-valuation): relax area regex lookbehind (2-комн regression) (#542)
All checks were successful
Deploy Trade-In / changes (push) Successful in 5s
Deploy Trade-In / build-frontend (push) Has been skipped
Deploy Trade-In / build-backend (push) Successful in 52s
Deploy Trade-In / deploy (push) Successful in 35s
2026-05-24 16:02:06 +00:00
490bc8ca5d fix(yandex-valuation): tighten area regex + chronological date ordering (#541)
All checks were successful
Deploy Trade-In / build-backend (push) Successful in 43s
Deploy Trade-In / changes (push) Successful in 5s
Deploy Trade-In / build-frontend (push) Has been skipped
Deploy Trade-In / deploy (push) Successful in 35s
2026-05-24 15:37:48 +00:00
9505471129 fix(yandex-valuation): cap area_m2 sanity (drop >10000 or ≤0) (#538)
All checks were successful
Deploy Trade-In / changes (push) Successful in 5s
Deploy Trade-In / build-frontend (push) Has been skipped
Deploy Trade-In / build-backend (push) Successful in 42s
Deploy Trade-In / deploy (push) Successful in 37s
2026-05-24 15:14:13 +00:00
7ef4e91e50 fix(yandex-valuation): parse removed_date, fix total_floors regex, normalize NBSP (#526)
All checks were successful
Deploy Trade-In / deploy (push) Successful in 34s
Deploy Trade-In / changes (push) Successful in 6s
Deploy Trade-In / build-frontend (push) Has been skipped
Deploy Trade-In / build-backend (push) Successful in 42s
2026-05-24 14:12:16 +00:00
b21d7c7c85 feat(tradein): scraper_settings live-config + Yandex admin trigger endpoints (#484)
All checks were successful
Deploy Trade-In / changes (push) Successful in 4s
Deploy Trade-In / build-frontend (push) Has been skipped
Deploy Trade-In / build-backend (push) Successful in 39s
Deploy Trade-In / deploy (push) Successful in 32s
2026-05-23 15:28:34 +00:00
7056ecb23b feat(tradein): yandex_valuation.py — anonymous house-history scraper (#468)
Some checks failed
Deploy Trade-In / deploy (push) Blocked by required conditions
Deploy Trade-In / changes (push) Successful in 4s
Deploy Trade-In / build-frontend (push) Has been skipped
Deploy Trade-In / build-backend (push) Has been cancelled
2026-05-23 13:46:20 +00:00