Commit graph

6 commits

Author SHA1 Message Date
bc504d9519 fix(tradein): M1 review — short_address для match_or_create_house (alias keyspace)
All checks were successful
CI / changes (push) Successful in 8s
CI / changes (pull_request) Successful in 6s
CI / backend-tests (push) Has been skipped
CI / frontend-tests (push) Has been skipped
CI / openapi-codegen-check (push) Has been skipped
CI / backend-tests (pull_request) Has been skipped
CI / frontend-tests (pull_request) Has been skipped
CI / openapi-codegen-check (pull_request) Has been skipped
2026-06-16 08:35:38 +03:00
b72af1cc49 fix(tradein): avito houses dedup через match_or_create_house + merge-миграция дублей (108)
All checks were successful
CI / changes (push) Successful in 7s
CI / backend-tests (push) Has been skipped
CI / frontend-tests (push) Has been skipped
CI / openapi-codegen-check (push) Has been skipped
Корень проблемы: avito_houses.py обходил канонический matching-путь через
прямой INSERT ON CONFLICT(source, ext_house_id), создавая дубли для физически
одного дома (~33% таблицы houses = ~4479 лишних строк).

Forward fix: upsert_house/upsert_house_with_url → _persist_house(db, h, url),
который вызывает match_or_create_house (3-tier: source_exact→fingerprint→geo→new,
с pg_advisory_xact_lock) и затем UPDATE с enrichment-полями. avito-авторитетные
поля (rating, developer, технические хар-ки) перезаписываются напрямую;
address/lat/lon/year_built — через COALESCE чтобы не затирать canonical-данные.

Merge-миграция 108: cluster_key по cadastral/address/geo → canonical (max
не-NULL полей, geom присутствует, min(id)) → репойнт всех 11 зависимых таблиц
с обработкой UNIQUE-коллизий → DELETE дублей → backfill house_sources/aliases.
2026-06-15 22:40:46 +03:00
92f44ac968 feat(tradein): route avito pipeline SERP+detail+houses through BrowserFetcher (#915) (#916)
All checks were successful
Deploy Trade-In / changes (push) Successful in 5s
Deploy Trade-In / build-frontend (push) Has been skipped
Deploy Trade-In / test (push) Successful in 30s
Deploy Trade-In / build-backend (push) Successful in 46s
Deploy Trade-In / deploy (push) Successful in 37s
Deploy Trade-In / build-browser (push) Has been skipped
Co-authored-by: bot-backend <bot-backend@gendsgn.local>
Co-committed-by: bot-backend <bot-backend@gendsgn.local>
2026-05-31 18:10:38 +00:00
a3f45a6722 fix(tradein): decode Avito houses URL-encoded __preloadedState__ (#566)
Avito switched the houses-catalog state blob from JSON.parse("<js-escaped>")
to a URL-encoded string literal, wrapped the 10 widgets in
placeholders[0].widgets, и re-nested developmentData под props.housePage.
JS-unescape path silently failed → houses rows перестали наполняться
(starve'ит #6 same-building + #569 deal centroids).

Extract decode в pure _extract_preloaded_state (urllib.parse.unquote,
JSON.parse fallback для старой формы), add _flatten_widgets, и сделать
_parse_house_page tolerant к обеим developmentData/ratingPreview формам.
Real-HTML fixture (Yumasheva-6 EKB) + 4 regression-теста.

code-reviewer:  APPROVE no critical (regex anchoring verified, decode
safe — no eval/unicode_escape, backward-compat реальный, delisted-house
envelope degrade'ит gracefully, fixture без PII/secrets). Tests: 22 passed.

Closes #566
2026-05-28 15:58:19 +05:00
222389cba9 fix(tradein/avito): anti-bot hardening — sleep + abort + shared session (#487)
All checks were successful
Deploy Trade-In / changes (push) Successful in 6s
Deploy Trade-In / build-frontend (push) Has been skipped
Deploy Trade-In / build-backend (push) Successful in 40s
Deploy Trade-In / deploy (push) Successful in 36s
2026-05-23 20:08:40 +00:00
5258a8f023 feat(tradein): avito_houses.py — Houses Catalog parser (state + reviews + history + recs) (#449)
All checks were successful
Deploy Trade-In / changes (push) Successful in 4s
Deploy Trade-In / build-frontend (push) Has been skipped
Deploy Trade-In / build-backend (push) Successful in 44s
Deploy Trade-In / deploy (push) Successful in 19s
2026-05-23 12:37:24 +00:00