Light1YT
c47a42c0a5
Add TEXT column cian_zhk_url to houses (migration 071) with partial index. This unblocks fetch_newbuilding() which requires a ZHK slug URL — the existing cian_internal_house_id (bigint) is not sufficient for the API call. Activate the do_houses block in backfill_cian_history(): query houses with cian_zhk_url set + no price_dynamics rows, then fetch + save per house. Add resolve_cian_zhk_url() in cian_newbuilding.py: follows Cian redirect from https://cian.ru/zhk/<id>/ to canonical slug URL, for use by the new backfill script (scripts/backfill_cian_zhk_url.py). Store cian_zhk_url into NewbuildingEnrichment and persist it via save_newbuilding_enrichment UPDATE (COALESCE — never overwrites non-NULL). Remove stale TODO comments referencing #525 from cian_history_backfill.py and admin.py docstring. Enable DO_HOUSES=true by default in cian_backfill_all.sh. Post-deploy required: apply 071 migration, then run scripts/backfill_cian_zhk_url.py to populate existing rows. |
||
|---|---|---|
| .. | ||
| local-cian | ||
| avito-playwright-sweep.py | ||
| backfill-house-imv.py | ||
| backfill-yandex-addresses-ekb.py | ||
| backfill_001_address_fingerprints.py | ||
| backfill_002_house_sources.sql | ||
| backfill_003_listing_sources.sql | ||
| backfill_cian_zhk_url.py | ||
| cian_backfill_all.sh | ||
| inspect-houses-html.py | ||
| inspect-search-html.py | ||
| local-sweep-ekb-cian.py | ||
| local-sweep-ekb-yandex.py | ||
| local-sweep-ekb.py | ||
| probe-avito-1anchor.py | ||
| probe-confidence-distribution.py | ||
| probe-db-counts.py | ||
| probe-db-leak-full.py | ||
| probe-db-region-leak.py | ||
| probe-db-url-leak.py | ||
| probe-imv-by-address.py | ||
| probe-migration-028.py | ||
| probe-migration-status.py | ||
| probe-parser-region-filter.py | ||
| README.md | ||
| sweep-yandex-valuation-ekb.py | ||
| test-playwright.py | ||
| upload-cian-cookies.sh | ||
tradein-mvp scripts -- backfill
One-shot manual operator scripts. NOT auto-applied by deploy.yml (which only runs tradein-mvp/backend/data/sql/NN_*.sql).
Order
Run in numbered sequence after PR #470 (matching module) merged:
cd tradein-mvp/backend
export DATABASE_URL="postgresql+psycopg://user:pass@host:5432/tradein"
# 001 -- backfill houses.address_fingerprint (Python, idempotent)
uv run python ../scripts/backfill_001_address_fingerprints.py
# 002 -- backfill house_sources link rows (SQL, idempotent)
psql "${DATABASE_URL/postgresql+psycopg/postgresql}" -f ../scripts/backfill_002_house_sources.sql
# 003 -- backfill listing_sources link rows (SQL, idempotent)
psql "${DATABASE_URL/postgresql+psycopg/postgresql}" -f ../scripts/backfill_003_listing_sources.sql
All scripts idempotent -- re-running is safe (ON CONFLICT DO NOTHING on the SQL; row-level UPDATE WHERE NULL on the Python).
Other scripts
upload-cian-cookies.sh-- one-shot Cian session cookie upload (see inline docs)