fix(tradein): avito houses dedup через match_or_create_house + merge-миграция дублей (108) #1546

Merged
lekss361 merged 2 commits from fix/avito-houses-dedup-merge into main 2026-06-16 05:38:24 +00:00

2 commits

Author SHA1 Message Date
bc504d9519 fix(tradein): M1 review — short_address для match_or_create_house (alias keyspace)
All checks were successful
CI / changes (push) Successful in 8s
CI / changes (pull_request) Successful in 6s
CI / backend-tests (push) Has been skipped
CI / frontend-tests (push) Has been skipped
CI / openapi-codegen-check (push) Has been skipped
CI / backend-tests (pull_request) Has been skipped
CI / frontend-tests (pull_request) Has been skipped
CI / openapi-codegen-check (pull_request) Has been skipped
2026-06-16 08:35:38 +03:00
b72af1cc49 fix(tradein): avito houses dedup через match_or_create_house + merge-миграция дублей (108)
All checks were successful
CI / changes (push) Successful in 7s
CI / backend-tests (push) Has been skipped
CI / frontend-tests (push) Has been skipped
CI / openapi-codegen-check (push) Has been skipped
Корень проблемы: avito_houses.py обходил канонический matching-путь через
прямой INSERT ON CONFLICT(source, ext_house_id), создавая дубли для физически
одного дома (~33% таблицы houses = ~4479 лишних строк).

Forward fix: upsert_house/upsert_house_with_url → _persist_house(db, h, url),
который вызывает match_or_create_house (3-tier: source_exact→fingerprint→geo→new,
с pg_advisory_xact_lock) и затем UPDATE с enrichment-полями. avito-авторитетные
поля (rating, developer, технические хар-ки) перезаписываются напрямую;
address/lat/lon/year_built — через COALESCE чтобы не затирать canonical-данные.

Merge-миграция 108: cluster_key по cadastral/address/geo → canonical (max
не-NULL полей, geom присутствует, min(id)) → репойнт всех 11 зависимых таблиц
с обработкой UNIQUE-коллизий → DELETE дублей → backfill house_sources/aliases.
2026-06-15 22:40:46 +03:00