fix(backfill): drop nonexistent kadastr_num column from listings SELECT #1732

Merged

lekss361 merged 1 commit from fix/backfill-kadastr-column into main

2026-06-17 20:53:27 +00:00

lekss361 commented

2026-06-17 20:48:56 +00:00

Owner

Summary

python -m scripts.backfill_listing_sources (the house_id_fk backlog drainer) crashed on every run: UndefinedColumn: column "kadastr_num" does not exist. The column was dropped in data/sql/094_cadastral_unify.sql (0% data); the script's listings SELECT still referenced it.

Fix

scripts/backfill_listing_sources.py: removed kadastr_num from the SELECT, the ListingRow dataclass, and the row mapping. The downstream matcher call used cadastral_number or row.kadastr_num (the fallback was already dead — 0% data) → now just cadastral_number. building_cadastral_number still passed; match_or_create_house uses building_cadastral_number or cadastral_number, so no behavior loss.

Why it matters (overnight fill)

This script drains the static house_id_fk backlog (~16k avito + ~14k cian active rows unlinked). With #1727 now linking new/re-scraped rows in realtime, this clears the existing pool. Broken since the 094 migration.

Test plan

pytest -k "backfill or listing_source" → 196 passed; ruff clean
regression guard: SELECT no longer references kadastr_num, still has both cadastral cols
Post-merge: python -m scripts.backfill_listing_sources runs end-to-end → active avito/cian house_id_fk climbs

Refs #1188

## Summary `python -m scripts.backfill_listing_sources` (the house_id_fk backlog drainer) crashed on every run: `UndefinedColumn: column "kadastr_num" does not exist`. The column was dropped in `data/sql/094_cadastral_unify.sql` (0% data); the script's listings SELECT still referenced it. ## Fix `scripts/backfill_listing_sources.py`: removed `kadastr_num` from the SELECT, the `ListingRow` dataclass, and the row mapping. The downstream matcher call used `cadastral_number or row.kadastr_num` (the fallback was already dead — 0% data) → now just `cadastral_number`. `building_cadastral_number` still passed; `match_or_create_house` uses `building_cadastral_number or cadastral_number`, so no behavior loss. ## Why it matters (overnight fill) This script drains the static house_id_fk backlog (~16k avito + ~14k cian active rows unlinked). With #1727 now linking new/re-scraped rows in realtime, this clears the existing pool. Broken since the 094 migration. ## Test plan - [x] `pytest -k "backfill or listing_source"` → 196 passed; ruff clean - [x] regression guard: SELECT no longer references kadastr_num, still has both cadastral cols - [ ] Post-merge: `python -m scripts.backfill_listing_sources` runs end-to-end → active avito/cian house_id_fk climbs Refs #1188

lekss361 added 1 commit 2026-06-17 20:48:57 +00:00