gendesign/tradein-mvp/backend/app/tasks
bot-backend edd187b47f feat(cadastral): geo-nearest building matcher via local cad mirror
Backfill listings.building_cadastral_number (was 0%) from the nearest
cadastral building. gendesign_cad_buildings is a postgres_fdw foreign table
with no geom — a per-listing FDW nearest query is ~1.16s/row (~13h for 43k
listings). Instead materialize the FDW once into a LOCAL cad_buildings_local
table (Point geom + GIST), then run a fast local KNN nearest-neighbour join.

Perf: the distance gate uses geometry-space ST_DWithin(geom, point, deg) (GIST
index, no geography cast) + geom <-> KNN order + ST_DistanceSphere metric
recheck on the single nearest row. A geography-cast ST_DWithin in the WHERE
defeated the index (58s+, full update never finished); the geometry-gate design
does the whole ~41.9k-listing UPDATE in ~5.4s (refresh+match ~7s end-to-end),
matching 16027 listings at 50m across 2229 distinct buildings.

The match is GEO-NEAREST (approximate): a street-level-geocoded listing matches
the nearest building within threshold_m (default 50m), not necessarily its exact
cadastral building. Exact cadastral + parcel-containment deferred (cad_parcels
FDW not exposed). Threshold is logged.

- 124: cad_buildings_local table (empty) + GIST. Migration does not read the FDW
  (deploy-independent); populated by the refresh job.
- 125: scrape_schedules seed source=cadastral_geo_match, enabled, next_run_at
  tomorrow 09:00 UTC (after geocode_missing).
- tasks/cadastral_geo_match.py: refresh_cad_buildings_local (bulk FDW scan),
  match_listings_to_buildings (chunked LATERAL KNN UPDATE), run_cadastral_geo_match
  run-lifecycle wrapper.
- scheduler: trigger_cadastral_geo_match_run + dispatch (sync DB-only, executor).
2026-06-18 10:18:54 +03:00
..
__init__.py feat(tradein): search matview + indexes (Phase 3.1) (#469) 2026-05-23 14:01:18 +00:00
asking_to_sold_ratio.py fix(tradein): guard listing_segment во всех читателях listings — novostroyki вне comps (#1186) 2026-06-12 11:52:09 +03:00
avito_detail_backfill.py feat(browser): per-source proxy pool behind FEATURE_BROWSER_POOL_ENABLED (Phase 1) 2026-06-18 09:01:50 +03:00
cadastral_geo_match.py feat(cadastral): geo-nearest building matcher via local cad mirror 2026-06-18 10:18:54 +03:00
cian_history_backfill.py feat(browser): per-source proxy pool behind FEATURE_BROWSER_POOL_ENABLED (Phase 1) 2026-06-18 09:01:50 +03:00
deactivate_stale_avito.py feat(scrapers): segment-aware generic deactivate_stale for yandex/cian (#1558) 2026-06-16 16:09:19 +00:00
geocode_missing.py fix(geocode): set geocode_tried_at on successful geocode UPDATE 2026-06-16 10:07:51 +03:00
listing_source_snapshot.py feat(tradein): per-source listing price history — schema + daily snapshot (#570) (#646) 2026-05-29 12:18:04 +00:00
newbuilding_enrich_backfill.py feat(tradein-scheduler): nightly newbuilding-enrichment schedule (#973) 2026-06-13 16:24:03 +05:00
refresh_search_matview.py feat(tradein): search matview + indexes (Phase 3.1) (#469) 2026-05-23 14:01:18 +00:00
rosreestr_quarter_poll.py feat(tradein): scheduled monthly Rosreestr new-quarter poll → ingest alert (#888) (#894) 2026-05-31 13:13:55 +00:00
sber_index_pull.py feat(tradein): СберИндекс monthly city-level price index pull (#887, data-layer) (#890) 2026-05-31 12:44:56 +00:00
yandex_address_backfill.py feat(tradein): schedule yandex address backfill task (#855) (#866) 2026-05-31 07:31:10 +00:00
yandex_detail_backfill.py fix(tradein): review minors — scheduler comment + save-error log label 2026-06-16 14:14:07 +03:00
yandex_newbuilding_sweep.py fix(tradein): yandex sweep — guard ext_id=NULL перед fetch_jk + cleanup httpx (#974 review M1/M3) 2026-06-15 18:41:08 +03:00