Commit graph

5 commits

Author SHA1 Message Date
bot-backend
03d7884a80 fix(tradein/houses): персистить house_class из avito «Класс жилья» и yandex parse (#2003)
All checks were successful
CI Trade-In / changes (pull_request) Successful in 8s
CI / changes (pull_request) Successful in 8s
CI Trade-In / frontend-checks (pull_request) Has been skipped
CI / backend-tests (pull_request) Has been skipped
CI / frontend-tests (pull_request) Has been skipped
CI / openapi-codegen-check (pull_request) Has been skipped
CI Trade-In / backend-tests (pull_request) Successful in 1m37s
houses.house_class (comfort/business/premium/economy) заполнен у 0% из 6958
домов. Два независимых бага:

Bug 2 (quick win): avito EXPAND_PARAMS_MAP не извлекал «Класс жилья»
(developmentPage/новостройки) в колонку — добавлена нормализация RU→канон
+ COALESCE-write в _persist_house (avito = топ-приоритетный источник,
перезаписывает при наличии значения, None не затирает существующее).
Зеркальная правка в scraper-kit копии (parity-тест этого требует).

Bug 1 (частично): resolve_house_field/HOUSE_FIELD_PRIORITY
(conflict_resolution.py) подтверждён dead code (0 прод-вызовов) — не
трогал. Вместо этого починен yandex_newbuilding_sweep.py: парсил
house_class, но нигде не персистил (терялось безвозвратно) — добавлен
fill-only UPDATE (COALESCE(houses.house_class, :new) — yandex_realty_nb
низший приоритет, не перетирает более приоритетный источник).
2026-07-04 03:46:47 +03:00
cd91d5e279 fix(scrapers): avito pipeline — новостройки house graceful (без housePage), AvitoBlockedError per-item не крашит прогон (#1820)
All checks were successful
CI / changes (pull_request) Successful in 7s
CI / backend-tests (pull_request) Has been skipped
CI / frontend-tests (pull_request) Has been skipped
CI / openapi-codegen-check (pull_request) Has been skipped
2026-06-20 09:53:46 +03:00
464e9faf7b fix(scrapers): avito_houses читает MFE props-вложенность виджетов — отзывы/история/тип ЖК теперь долетают (#1789)
All checks were successful
CI / changes (pull_request) Successful in 5s
CI / backend-tests (pull_request) Has been skipped
CI / frontend-tests (pull_request) Has been skipped
CI / openapi-codegen-check (pull_request) Has been skipped
2026-06-19 19:21:49 +03:00
a3f45a6722 fix(tradein): decode Avito houses URL-encoded __preloadedState__ (#566)
Avito switched the houses-catalog state blob from JSON.parse("<js-escaped>")
to a URL-encoded string literal, wrapped the 10 widgets in
placeholders[0].widgets, и re-nested developmentData под props.housePage.
JS-unescape path silently failed → houses rows перестали наполняться
(starve'ит #6 same-building + #569 deal centroids).

Extract decode в pure _extract_preloaded_state (urllib.parse.unquote,
JSON.parse fallback для старой формы), add _flatten_widgets, и сделать
_parse_house_page tolerant к обеим developmentData/ratingPreview формам.
Real-HTML fixture (Yumasheva-6 EKB) + 4 regression-теста.

code-reviewer:  APPROVE no critical (regex anchoring verified, decode
safe — no eval/unicode_escape, backward-compat реальный, delisted-house
envelope degrade'ит gracefully, fixture без PII/secrets). Tests: 22 passed.

Closes #566
2026-05-28 15:58:19 +05:00
5258a8f023 feat(tradein): avito_houses.py — Houses Catalog parser (state + reviews + history + recs) (#449)
All checks were successful
Deploy Trade-In / changes (push) Successful in 4s
Deploy Trade-In / build-frontend (push) Has been skipped
Deploy Trade-In / build-backend (push) Successful in 44s
Deploy Trade-In / deploy (push) Successful in 19s
2026-05-23 12:37:24 +00:00