gendesign/tradein-mvp/backend/app/services
lekss361 ead2ac16c5 feat(tradein): yandex_realty.py — DOM SERP refactor (vtorichka, 23 cards/page)
Replace dead JSON-state extraction (Yandex moved to chunked/ephemeral state
post-hydration) with selectolax DOM parsing using helpers from PR #456.

Changes:
- New URL: /{city}/kupit/kvartira/vtorichniy-rynok/?page=N (path-based, no geo)
- Selector: [data-test="OffersSerpItem"] x ~23 per page
- Per-card extraction: offer_id (URL regex), title (rooms/area/floor regex),
  price/ppm2 (regex + parse_rub), bargain flag, publish_date (parse_ru_date),
  street address, photos (avatars.mds.yandex domain filter + size upgrade),
  newbuilding linkage (house_source='yandex_realty_nb' + house_ext_id)
- ScrapedLot mapping: lat/lon=None (no coords on SERP — geocode-missing
  fills later), listing_segment="vtorichka"
- Pagination: fetch_around_multi_room -> up to MAX_PAGES=3 pages, dedup by
  source_id. **kwargs swallow keeps old rooms/sort/pages callers compat.
- Preserved: BaseScraper interface, name="yandex" (matches existing rows +
  avoids breaking dedup_hash; Wave 5 conflict resolution can alias if needed),
  request_delay_sec=5.0, sleep_between_requests, tenacity retry.

Tests: 10 unit tests against hand-crafted fixture HTML (single card / studio /
no-price / empty page / multi-card / URL builder / city override) — all pass.
Ruff clean.
2026-05-23 16:21:47 +03:00
..
exporters fix(tradein): security hotfix — C-3 config default + C-4 empty estimate persist + C-6 PDF URL allowlist (#435) 2026-05-23 09:38:23 +00:00
scrapers feat(tradein): yandex_realty.py — DOM SERP refactor (vtorichka, 23 cards/page) 2026-05-23 16:21:47 +03:00
__init__.py feat: add tradein-mvp subproject (Trade-In Estimator под /trade-in) 2026-05-21 00:25:39 +03:00
brand.py feat: add tradein-mvp subproject (Trade-In Estimator под /trade-in) 2026-05-21 00:25:39 +03:00
estimator.py feat(tradein): integrate Avito IMV as 5th evaluation source (on-demand cached) (#452) 2026-05-23 13:08:22 +00:00
geocoder.py fix(tradein): map-picker подставляет уличный адрес, а не название POI 2026-05-22 13:32:39 +05:00
house_metadata.py feat(tradein): кадастр зданий как источник house_metadata (#393) 2026-05-22 10:54:42 +05:00
scrape_pipeline.py feat(tradein): scrape_pipeline.py — Avito full orchestrator (search → houses → detail) (#451) 2026-05-23 12:55:13 +00:00