gendesign/tradein-mvp/backend/app/services/scrapers
Light1YT cad3b77171 fix(tradein): Avito scraper now extracts listing_date для всех форматов
Real production state: **0/9302 (100%)** Avito listings имели NULL
listing_date. Audit показал regex matched только relative numeric
("5 дней назад") — Avito SERP сейчас использует:
- Keyword shortcuts «сегодня в 12:30», «вчера в 18:45», «позавчера», «только что»
- Absolute «18 мая», «5 июня в 12:30» (year inferred — current, или
  rollback к прошлому году если month в будущем больше чем на 30d)

Changes:
- Expand _parse_relative_date():
  * keyword check first (без \d+ regex match)
  * relative numeric (existing) — без изменений
  * absolute date regex с _RU_MONTHS dict + roll-back на prev year heuristic

Tests: 10/10 pass (4 existing + 6 new). Ruff clean.

Impact: 9302 Avito listings + new daily scrapes теперь будут иметь
listing_date. Это **прямо повысит linkage rate в /sales-vs-listings**
(PR K) с current ~5% до 30%+ (теоретически до 80% при listing_date
плотности).

Existing 9302 NULL — будут пересобраны при следующем full sweep после
deploy. Backfill detail page для each — отдельный operation скрипт
(out of scope, можно через docker exec ad-hoc).

Refs #564 (linkage rate improvement).
2026-05-27 17:55:01 +05:00
..
__init__.py feat: add tradein-mvp subproject (Trade-In Estimator под /trade-in) 2026-05-21 00:25:39 +03:00
avito.py fix(tradein): Avito scraper now extracts listing_date для всех форматов 2026-05-27 17:55:01 +05:00
avito_detail.py fix(tradein-avito): strip Emotion CSS from listings.address (#502) 2026-05-24 10:52:50 +00:00
avito_exceptions.py fix(tradein/avito): anti-bot hardening — sleep + abort + shared session (#487) 2026-05-23 20:08:40 +00:00
avito_houses.py fix(tradein/avito): anti-bot hardening — sleep + abort + shared session (#487) 2026-05-23 20:08:40 +00:00
avito_imv.py fix(tradein-phase-c): house_type 'monolith'→'monolithic' + log 400 body (#537) 2026-05-24 15:05:49 +00:00
base.py feat(tradein): hook matching в save_listings — fill listing_sources (PR I) (#595) 2026-05-27 10:02:37 +00:00
cian.py feat(tradein): global scraper delay setting (applies across all scrapers) (#485) 2026-05-23 15:43:28 +00:00
cian_detail.py feat(tradein): cian_session cookies management + admin endpoints (Wave 4 Worker A) (#457) 2026-05-23 13:22:00 +00:00
cian_newbuilding.py feat(tradein): cian_newbuilding.py — ЖК catalog scraper (chart + reliability + offers, Wave 4 Worker C) (#459) 2026-05-23 13:23:11 +00:00
cian_state_parser.py feat(tradein): cian_state_parser shared utility + ScrapedLot Cian fields (#447) 2026-05-23 12:35:01 +00:00
cian_valuation.py fix(tradein/cian): sanitize valuation cache writes (047 UNIQUE + 026/044 cols) (#523) 2026-05-24 13:44:39 +00:00
n1.py feat(tradein): global scraper delay setting (applies across all scrapers) (#485) 2026-05-23 15:43:28 +00:00
yandex_detail.py feat(tradein): scraper_settings live-config + Yandex admin trigger endpoints (#484) 2026-05-23 15:28:34 +00:00
yandex_helpers.py fix(tradein): yandex_realty captures house number + parse_rub strips NBSP (#593) 2026-05-27 07:23:36 +00:00
yandex_newbuilding.py feat(tradein): scraper_settings live-config + Yandex admin trigger endpoints (#484) 2026-05-23 15:28:34 +00:00
yandex_realty.py fix(tradein): yandex_realty captures house number + parse_rub strips NBSP (#593) 2026-05-27 07:23:36 +00:00
yandex_valuation.py fix(yandex-valuation): off-by-one cell index in _parse_row_cells (#552) 2026-05-24 19:57:17 +00:00