e868b01278
SQL migration 064: adds houses.imv_status + last_imv_attempt_at + imv_error_reason
columns, creates house_imv_evaluations (1:1 with houses, latest IMV result)
and house_suggestions (8 Avito similar lots per house, UNIQUE on (house_id, ext_item_id)).
Script scripts/backfill-house-imv.py: resumable iterator that picks houses with
imv_status='pending' AND lat/lon NOT NULL, runs evaluate_via_imv() per house with
median lot-params from linked listings, saves house_imv_evaluations +
house_placement_history (from result.placement_history items) + house_suggestions
(from result.suggestions items). Configurable --batch, --delay, --limit-total.
Survives restart via imv_status state machine ('pending' -> 'ok'/'not_found'/
'transient_error'/'error'/'no_params'/'no_address'). Long backoff on
IMVAuthError (anti-captcha).
|
||
|---|---|---|
| .. | ||
| backfill-house-imv.py | ||
| backfill_001_address_fingerprints.py | ||
| backfill_002_house_sources.sql | ||
| backfill_003_listing_sources.sql | ||
| probe-confidence-distribution.py | ||
| probe-imv-by-address.py | ||
| README.md | ||
| upload-cian-cookies.sh | ||
tradein-mvp scripts -- backfill
One-shot manual operator scripts. NOT auto-applied by deploy.yml (which only runs tradein-mvp/backend/data/sql/NN_*.sql).
Order
Run in numbered sequence after PR #470 (matching module) merged:
cd tradein-mvp/backend
export DATABASE_URL="postgresql+psycopg://user:pass@host:5432/tradein"
# 001 -- backfill houses.address_fingerprint (Python, idempotent)
uv run python ../scripts/backfill_001_address_fingerprints.py
# 002 -- backfill house_sources link rows (SQL, idempotent)
psql "${DATABASE_URL/postgresql+psycopg/postgresql}" -f ../scripts/backfill_002_house_sources.sql
# 003 -- backfill listing_sources link rows (SQL, idempotent)
psql "${DATABASE_URL/postgresql+psycopg/postgresql}" -f ../scripts/backfill_003_listing_sources.sql
All scripts idempotent -- re-running is safe (ON CONFLICT DO NOTHING on the SQL; row-level UPDATE WHERE NULL on the Python).
Other scripts
upload-cian-cookies.sh-- one-shot Cian session cookie upload (see inline docs)