gendesign/tradein-mvp/backend/app/services/scrapers
lekss361 65b7b73691 fix(tradein): cian state-extraction для нового _cianConfig формата + auth из header-frontend (#639)
Cian сменил формат встраивания state (2026-05): `.push({key:..,value:..})` →
`window._cianConfig['mfe'] = (... || []).concat([{"key":..,"value":..},..])` с
JSON-кавычками. `_RE_CIAN_PUSH` матчил 0 → extract_state=None → verify_session
заворачивал ВАЛИДНЫЕ куки → scheduler.py скипал cian-бэкфилл («cookies expired»).
Ломало весь cian (SERP/valuation/detail/newbuilding — общий extract_state).

Плюс auth-сигнал (user.isAuthenticated/userId) уехал из valuation-MFE (там user
теперь anonymous) в site-header MFE `header-frontend`.

- cian_state_parser.py: `_extract_from_concat` (regex .concat([..]) + string-aware
  balance-scan `[...]` → json.loads массива → item по key). extract_state пробует
  concat первым, .push остаётся fallback для legacy-фикстур.
- cian_session.py: verify читает auth из `_MFE_AUTH='header-frontend'`, не valuation.
- tests: concat-формат, ']'/'[' внутри строк, legacy push fallback.

Диагностировано вживую на проде (валидная сессия userId 75544897): _RE_CIAN_PUSH=0
матчей при 12 живых _cianConfig; concat-парсер извлекает initialState; isAuth=true
в header-frontend. 34 passed, ruff clean.

Refs #639, #806.
2026-05-30 22:04:26 +03:00
..
__init__.py feat: add tradein-mvp subproject (Trade-In Estimator под /trade-in) 2026-05-21 00:25:39 +03:00
avito.py feat(tradein): Avito mobile proxy egress + firewall detect + MSK date (#623, #726) (#788) 2026-05-30 16:52:35 +00:00
avito_detail.py feat(tradein): infer repair_state from listing description text (#622) (#632) 2026-05-28 15:37:34 +00:00
avito_exceptions.py fix(tradein): avito content-block (HTTP-200) не маскируется под end-of-pagination (Refs #754) (#779) 2026-05-30 16:28:23 +00:00
avito_houses.py fix(tradein): decode Avito houses URL-encoded __preloadedState__ (#566) 2026-05-28 15:58:19 +05:00
avito_imv.py fix(tradein-imv): harden Avito IMV anti-bot path + offline contract tests (#565) 2026-05-28 16:29:05 +05:00
base.py feat(estimator): exclude city-centroid listings from radius analogs (Refs #769 Part E) 2026-05-30 21:22:21 +03:00
cian.py feat(tradein): infer repair_state from listing description text (#622) (#632) 2026-05-28 15:37:34 +00:00
cian_detail.py feat(tradein): infer repair_state from listing description text (#622) (#632) 2026-05-28 15:37:34 +00:00
cian_newbuilding.py feat(tradein): houses.cian_zhk_url column + backfill (#568) (#627) 2026-05-28 13:57:43 +00:00
cian_state_parser.py fix(tradein): cian state-extraction для нового _cianConfig формата + auth из header-frontend (#639) 2026-05-30 22:04:26 +03:00
cian_valuation.py fix(tradein/cian): sanitize valuation cache writes (047 UNIQUE + 026/044 cols) (#523) 2026-05-24 13:44:39 +00:00
n1.py feat(scrapers/n1): извлекать floor/total_floors из ведущего N/M-токена (Refs #795) (#802) 2026-05-30 17:31:13 +00:00
repair_state_normalizer.py feat(tradein): infer repair_state from listing description text (#622) (#632) 2026-05-28 15:37:34 +00:00
snapshot_writer.py fix(tradein): implement listings_snapshots + offer_price_history writers (#567) (#624) 2026-05-28 13:49:52 +00:00
yandex_detail.py feat(tradein): infer repair_state from listing description text (#622) (#632) 2026-05-28 15:37:34 +00:00
yandex_helpers.py fix(tradein): Yandex listing_date — keyword + absolute-no-year (#602) 2026-05-27 13:11:22 +00:00
yandex_newbuilding.py feat(tradein): scraper_settings live-config + Yandex admin trigger endpoints (#484) 2026-05-23 15:28:34 +00:00
yandex_realty.py fix(scrapers): yandex area из embedded-state JSON (Refs #775) (#783) 2026-05-30 16:38:12 +00:00
yandex_valuation.py fix(yandex-valuation): off-by-one cell index in _parse_row_cells (#552) 2026-05-24 19:57:17 +00:00