b32628e262
Multi-Source Integration Phase 1.3. Manual operator scripts for legacy data
migration. NOT auto-applied -- run explicitly via uv/psql.
scripts/:
- backfill_001_address_fingerprints.py -- Python, populates houses.address_fingerprint
for rows where NULL (uses matching/normalize.address_fingerprint from PR #470)
- backfill_002_house_sources.sql -- INSERT INTO house_sources FROM houses,
ON CONFLICT (ext_source, ext_id) DO NOTHING (idempotent)
- backfill_003_listing_sources.sql -- INSERT INTO listing_sources FROM listings,
same pattern
- README.md -- order + usage
All idempotent -- re-running safe. confidence=1.0, matched_method='backfill'.
27 lines
1 KiB
Markdown
27 lines
1 KiB
Markdown
# tradein-mvp scripts -- backfill
|
|
|
|
One-shot manual operator scripts. **NOT auto-applied by deploy.yml** (which only runs `tradein-mvp/backend/data/sql/NN_*.sql`).
|
|
|
|
## Order
|
|
|
|
Run in numbered sequence after PR #470 (matching module) merged:
|
|
|
|
```bash
|
|
cd tradein-mvp/backend
|
|
export DATABASE_URL="postgresql+psycopg://user:pass@host:5432/tradein"
|
|
|
|
# 001 -- backfill houses.address_fingerprint (Python, idempotent)
|
|
uv run python ../scripts/backfill_001_address_fingerprints.py
|
|
|
|
# 002 -- backfill house_sources link rows (SQL, idempotent)
|
|
psql "${DATABASE_URL/postgresql+psycopg/postgresql}" -f ../scripts/backfill_002_house_sources.sql
|
|
|
|
# 003 -- backfill listing_sources link rows (SQL, idempotent)
|
|
psql "${DATABASE_URL/postgresql+psycopg/postgresql}" -f ../scripts/backfill_003_listing_sources.sql
|
|
```
|
|
|
|
All scripts idempotent -- re-running is safe (ON CONFLICT DO NOTHING on the SQL; row-level UPDATE WHERE NULL on the Python).
|
|
|
|
## Other scripts
|
|
|
|
- `upload-cian-cookies.sh` -- one-shot Cian session cookie upload (see inline docs)
|