Previous confidence used raw n_analogs which inflated to 'high' when MIN_ANALOGS_PER_SOURCE quota + same-building lots concentrated count on few unique addresses. New logic: - high: unique_addr >= 7 AND iqr_pct < 15% - medium: unique_addr >= 4 OR (unique_addr >= 2 AND iqr_pct < 25%) - downgrade one step if avg_lots_per_addr > 2.5 (anti-concentration bias) probe-confidence-distribution.py added for tuning p75 threshold on real data. |
||
|---|---|---|
| .. | ||
| backfill_001_address_fingerprints.py | ||
| backfill_002_house_sources.sql | ||
| backfill_003_listing_sources.sql | ||
| probe-confidence-distribution.py | ||
| probe-imv-by-address.py | ||
| README.md | ||
| upload-cian-cookies.sh | ||
tradein-mvp scripts -- backfill
One-shot manual operator scripts. NOT auto-applied by deploy.yml (which only runs tradein-mvp/backend/data/sql/NN_*.sql).
Order
Run in numbered sequence after PR #470 (matching module) merged:
cd tradein-mvp/backend
export DATABASE_URL="postgresql+psycopg://user:pass@host:5432/tradein"
# 001 -- backfill houses.address_fingerprint (Python, idempotent)
uv run python ../scripts/backfill_001_address_fingerprints.py
# 002 -- backfill house_sources link rows (SQL, idempotent)
psql "${DATABASE_URL/postgresql+psycopg/postgresql}" -f ../scripts/backfill_002_house_sources.sql
# 003 -- backfill listing_sources link rows (SQL, idempotent)
psql "${DATABASE_URL/postgresql+psycopg/postgresql}" -f ../scripts/backfill_003_listing_sources.sql
All scripts idempotent -- re-running is safe (ON CONFLICT DO NOTHING on the SQL; row-level UPDATE WHERE NULL on the Python).
Other scripts
upload-cian-cookies.sh-- one-shot Cian session cookie upload (see inline docs)