feat(tradein): houses.cian_zhk_url column + backfill (#568) #627

Merged

Light1YT merged 2 commits from fix/568-houses-cian-zhk-url into main

2026-05-28 13:57:44 +00:00

Light1YT commented

2026-05-28 13:54:52 +00:00

Owner

Summary

Migration tradein-mvp/backend/data/sql/071_houses_cian_zhk_url.sql (ADD COLUMN + partial index).
cian_newbuilding.py: persist cian_zhk_url + resolve_cian_zhk_url() resolver.
cian_history_backfill.py: activate the houses enrichment block (SAVEPOINT-isolated per house).
scripts/backfill_cian_zhk_url.py one-time URL backfill.

Post-deploy

migration 071 auto-applies; then run scripts/backfill_cian_zhk_url.py before history backfill.

Test plan

migration applies
≥80% houses with cian_internal_house_id get cian_zhk_url

Closes #568

## Summary - Migration tradein-mvp/backend/data/sql/071_houses_cian_zhk_url.sql (ADD COLUMN + partial index). - cian_newbuilding.py: persist cian_zhk_url + resolve_cian_zhk_url() resolver. - cian_history_backfill.py: activate the houses enrichment block (SAVEPOINT-isolated per house). - scripts/backfill_cian_zhk_url.py one-time URL backfill. ## Post-deploy - migration 071 auto-applies; then run scripts/backfill_cian_zhk_url.py before history backfill. ## Test plan - [ ] migration applies - [ ] ≥80% houses with cian_internal_house_id get cian_zhk_url Closes #568

Light1YT added 2 commits 2026-05-28 13:54:52 +00:00

feat(tradein): houses.cian_zhk_url column + backfill (#568 ) c47a42c0a5

Add TEXT column cian_zhk_url to houses (migration 071) with partial index.
This unblocks fetch_newbuilding() which requires a ZHK slug URL — the
existing cian_internal_house_id (bigint) is not sufficient for the API call.

Activate the do_houses block in backfill_cian_history(): query houses with
cian_zhk_url set + no price_dynamics rows, then fetch + save per house.

Add resolve_cian_zhk_url() in cian_newbuilding.py: follows Cian redirect
from https://cian.ru/zhk/<id>/ to canonical slug URL, for use by the new
backfill script (scripts/backfill_cian_zhk_url.py).

Store cian_zhk_url into NewbuildingEnrichment and persist it via
save_newbuilding_enrichment UPDATE (COALESCE — never overwrites non-NULL).

Remove stale TODO comments referencing #525 from cian_history_backfill.py
and admin.py docstring. Enable DO_HOUSES=true by default in cian_backfill_all.sh.

Post-deploy required: apply 071 migration, then run
scripts/backfill_cian_zhk_url.py to populate existing rows.

fix(tradein): SAVEPOINT-isolate houses backfill + ruff/shell fixes (#568 ) cfa24c3bcf