lekss361/gendesign
1
0
Fork 0
Code Issues 57 Pull requests Projects Releases Packages Wiki Activity Actions
gendesign/tradein-mvp/backend/data/sql
History
Light1YT 6ce60a0d8c
All checks were successful
CI / changes (pull_request) Successful in 7s
Details
CI / backend-tests (pull_request) Has been skipped
Details
CI / frontend-tests (pull_request) Has been skipped
Details
CI / openapi-codegen-check (pull_request) Has been skipped
Details
fix(tradein): backfill house_id_fk via source-identity path (#1781) 
Migration 130 only filled house_id_fk for listings carrying
house_source/house_ext_id. Prod (2026-06-27): 1884 active NULL-FK rows
remain, 0 with house_source/ext_id (130 re-run = no-op). They were
matched via the per-listing-identity path the realtime mirror uses when
no house catalog id exists: match_or_create_house(ext_source=source,
ext_id=source_id) -> Tier-1 house_sources lookup (confidence 1.0).

136 replays that exact lookup offline: UPDATE listings SET house_id_fk
= house_sources.house_id WHERE (ext_source=source, ext_id=source_id) and
house_id_fk IS NULL. Faithful to base.py:704-750; idempotent (NULL
guard); deterministic (no fuzzy/geo); collision-safe (catalog ids live
under distinct ext_source like 'cian_newbuilding'). Dry-run BEGIN/ROLLBACK
fills 4608 NULL-FK rows (744 active), restoring estimator Tier-S
"same-building" grouping. The remaining ~1140 active need re-scrape/
re-match (scraper coverage — #1781 Лёха).

Refs #1781
2026-06-27 05:01:02 +05:00
..
001_trade_in_estimates.sql feat: add tradein-mvp subproject (Trade-In Estimator под /trade-in) 2026-05-21 00:25:39 +03:00
002_core_tables.sql feat: add tradein-mvp subproject (Trade-In Estimator под /trade-in) 2026-05-21 00:25:39 +03:00
003_seed_deals.sql fix(tradein): убрать TRUNCATE из seed — деплой больше не wipe-ит данные 2026-05-21 13:19:24 +03:00
004_extend_trade_in_estimates.sql feat: add tradein-mvp subproject (Trade-In Estimator под /trade-in) 2026-05-21 00:25:39 +03:00
005_geocode_tracking.sql feat(tradein): geocode-missing для deals (#4 завершение) 2026-05-21 20:04:19 +03:00
007_estimate_photos.sql feat(tradein): backend загрузки фото квартиры (#394) 2026-05-22 11:05:35 +05:00
008_crm_fields.sql feat(tradein): CRM-поля в форме оценки (#395) 2026-05-22 11:15:15 +05:00
009_houses.sql feat(tradein): SQL migrations 009-018 — houses + listings extension + IMV cache (#440) 2026-05-23 11:41:44 +00:00
010_houses_alter.sql feat(tradein): SQL migrations 009-018 — houses + listings extension + IMV cache (#440) 2026-05-23 11:41:44 +00:00
011_listings_alter.sql feat(tradein): SQL migrations 009-018 — houses + listings extension + IMV cache (#440) 2026-05-23 11:41:44 +00:00
012_sellers.sql feat(tradein): SQL migrations 009-018 — houses + listings extension + IMV cache (#440) 2026-05-23 11:41:44 +00:00
013_listings_alter_seller.sql feat(tradein): SQL migrations 009-018 — houses + listings extension + IMV cache (#440) 2026-05-23 11:41:44 +00:00
014_house_reviews.sql feat(tradein): SQL migrations 009-018 — houses + listings extension + IMV cache (#440) 2026-05-23 11:41:44 +00:00
015_scrape_runs.sql feat(tradein): SQL migrations 009-018 — houses + listings extension + IMV cache (#440) 2026-05-23 11:41:44 +00:00
016_listings_snapshots.sql feat(tradein): SQL migrations 009-018 — houses + listings extension + IMV cache (#440) 2026-05-23 11:41:44 +00:00
017_house_placement_history.sql feat(tradein): SQL migrations 009-018 — houses + listings extension + IMV cache (#440) 2026-05-23 11:41:44 +00:00
018_avito_imv_evaluations.sql feat(tradein): SQL migrations 009-018 — houses + listings extension + IMV cache (#440) 2026-05-23 11:41:44 +00:00
019_listings_alter_cian.sql feat(tradein): SQL migrations 019-028 — Cian schema + matching + cookies 2026-05-23 15:14:49 +03:00
020_houses_alter_cian.sql feat(tradein): SQL migrations 019-028 — Cian schema + matching + cookies 2026-05-23 15:14:49 +03:00
021_management_companies.sql feat(tradein): SQL migrations 019-028 — Cian schema + matching + cookies 2026-05-23 15:14:49 +03:00
022_agents_table.sql feat(tradein): SQL migrations 019-028 — Cian schema + matching + cookies 2026-05-23 15:14:49 +03:00
023_offer_price_history.sql feat(tradein): SQL migrations 019-028 — Cian schema + matching + cookies 2026-05-23 15:14:49 +03:00
024_houses_price_dynamics.sql feat(tradein): SQL migrations 019-028 — Cian schema + matching + cookies 2026-05-23 15:14:49 +03:00
025_house_reliability_checks.sql feat(tradein): SQL migrations 019-028 — Cian schema + matching + cookies 2026-05-23 15:14:49 +03:00
026_external_valuations.sql feat(tradein): SQL migrations 019-028 — Cian schema + matching + cookies 2026-05-23 15:14:49 +03:00
027_cian_session_cookies.sql feat(tradein): SQL migrations 019-028 — Cian schema + matching + cookies 2026-05-23 15:14:49 +03:00
028_matching_tables.sql feat(tradein): SQL migrations 019-028 — Cian schema + matching + cookies 2026-05-23 15:14:49 +03:00
029_extend_matching_valuation_dynamics.sql feat(tradein): SQL migration 029 — extend matching/valuation/dynamics tables (PR #445 follow-up) (#448) 2026-05-23 12:35:03 +00:00
030_avito_imv_cache_key_unique.sql fix(tradein): 030 — UNIQUE constraint on avito_imv_evaluations.cache_key (unblocks PR #452) (#454) 2026-05-23 13:04:21 +00:00
031_houses_alter_yandex.sql fix(tradein-sql): rename dup 030, add trade_in_estimates geom, drop dup indexes (#500) 2026-05-24 10:04:40 +00:00
032_yandex_history.sql feat(tradein): SQL migrations 030-032 — Yandex schema (listings/houses/history) (#453) 2026-05-23 13:00:54 +00:00
033_listings_alter_yandex.sql fix(tradein-sql): rename dup 030, add trade_in_estimates geom, drop dup indexes (#500) 2026-05-24 10:04:40 +00:00
034_trade_in_estimates_geom.sql fix(tradein-sql): rename dup 030, add trade_in_estimates geom, drop dup indexes (#500) 2026-05-24 10:04:40 +00:00
035_drop_duplicate_indexes.sql fix(tradein-sql): rename dup 030, add trade_in_estimates geom, drop dup indexes (#500) 2026-05-24 10:04:40 +00:00
040_houses_extend.sql feat(tradein): SQL 040-046 — matching schema delta (Phase 1.1) (#464) 2026-05-23 13:30:22 +00:00
041_house_sources_noop.sql feat(tradein): SQL 040-046 — matching schema delta (Phase 1.1) (#464) 2026-05-23 13:30:22 +00:00
042_listing_sources_price_divergence_idx.sql feat(tradein): SQL 040-046 — matching schema delta (Phase 1.1) (#464) 2026-05-23 13:30:22 +00:00
043_house_reviews_extend.sql feat(tradein): SQL 040-046 — matching schema delta (Phase 1.1) (#464) 2026-05-23 13:30:22 +00:00
044_external_valuations_link.sql feat(tradein): SQL 040-046 — matching schema delta (Phase 1.1) (#464) 2026-05-23 13:30:22 +00:00
045_house_placement_history_extend.sql feat(tradein): SQL 040-046 — matching schema delta (Phase 1.1) (#464) 2026-05-23 13:30:22 +00:00
046_views.sql feat(tradein): SQL 040-046 — matching schema delta (Phase 1.1) (#464) 2026-05-23 13:30:22 +00:00
047_cian_history_sanitize.sql fix(tradein/cian): sanitize valuation cache writes (047 UNIQUE + 026/044 cols) (#523) 2026-05-24 13:44:39 +00:00
050_search_optimization.sql fix(tradein-sql): remove duplicate CREATE INDEX from 050 (regression from PR #500) (#515) 2026-05-24 12:16:27 +00:00
051_scrape_runs_extend.sql feat(tradein): city sweep — auto ЕКБ pipeline (#477) 2026-05-23 14:38:07 +00:00
052_scrape_schedules.sql feat(tradein): in-app scheduler — UI-managed schedule (replaces SSH crontab) (#482) 2026-05-23 14:54:25 +00:00
053_scraper_settings.sql feat(tradein): scraper_settings live-config + Yandex admin trigger endpoints (#484) 2026-05-23 15:28:34 +00:00
054_scraper_settings_global.sql feat(tradein): global scraper delay setting (applies across all scrapers) (#485) 2026-05-23 15:43:28 +00:00
060_postgres_fdw_extension.sql feat(tradein): postgres_fdw live read of gendesign.cad_buildings (replaces snapshot) (#493) 2026-05-24 08:57:30 +00:00
061_drop_legacy_cad_buildings.sql feat(tradein): postgres_fdw live read of gendesign.cad_buildings (replaces snapshot) (#493) 2026-05-24 08:57:30 +00:00
062_clean_avito_addresses.sql fix(week-review): автофиксы код-ревью — 169 issue (label «week ревью 1») 2026-06-15 20:21:11 +05:00
063_backfill_houses_and_link_listings.sql feat(tradein-sql): bootstrap houses + link 98.8% of listings (Phase A+B) (#527) 2026-05-24 14:17:32 +00:00
064_house_imv_phase_c.sql feat(tradein): Phase C — Avito IMV per-house backfill (SQL + script) (#534) 2026-05-24 14:50:31 +00:00
065_trade_in_estimates_floor_optional.sql feat(tradein): этаж/этажность optional + best test presets по deal count (#558) 2026-05-24 21:44:44 +00:00
066_address_mismatch_audit.sql feat(tradein): Phase 1 of #582 — address mismatch audit infrastructure 2026-05-25 12:50:30 +05:00
067_v_street_sales_vs_listings.sql feat(tradein): sales-vs-listings endpoint (PR K, foundation #564) (#597) 2026-05-27 11:55:44 +00:00
068_drop_v_street_sales_vs_listings.sql chore(tradein): drop spurious v_street_sales_vs_listings (#598) 2026-05-27 12:23:18 +00:00
069_trade_in_estimates_dadata_fields.sql feat(tradein): DaData on-demand enrichment в estimate flow (PR Q1) 2026-05-27 18:50:12 +05:00
070_houses_dadata_enrichment.sql feat(tradein): DaData /clean/address backfill для houses (PR Q3) 2026-05-27 19:43:07 +05:00
071_houses_cian_zhk_url.sql feat(tradein): houses.cian_zhk_url column + backfill (#568) (#627) 2026-05-28 13:57:43 +00:00
072_scrape_schedules_seed_cian_rosreestr.sql feat(tradein): автоматизация cian-backfill + rosreestr-import через scheduler (#560 #563) (#630) 2026-05-28 15:31:24 +00:00
073_normalize_repair_state.sql refactor(tradein): normalize repair_state raw→enum at ingest (#621) (#629) 2026-05-28 15:18:11 +00:00
075_backfill_repair_state_from_description.sql feat(tradein): infer repair_state from listing description text (#622) (#632) 2026-05-28 15:37:34 +00:00
076_account_estimate_quota.sql feat(tradein): аккаунт praktika + лимит 15 оценок/мес, чистка PRINZIP в UI (#635) 2026-05-28 22:04:58 +00:00
077_dedup_hash_plain_key_backfill.sql fix(tradein): plain natural-key dedup_hash for rosreestr deals (#576) (#638) 2026-05-29 06:56:35 +00:00
078_scrape_schedules_seed_yandex_sweep.sql feat(scheduler): dormant Yandex Realty city-sweep (#561) (#642) 2026-05-29 08:50:10 +00:00
079_listing_source_history.sql feat(tradein): per-source listing price history — schema + daily snapshot (#570) (#646) 2026-05-29 12:18:04 +00:00
080_asking_to_sold_ratios.sql fix(week-review): автофиксы код-ревью — 169 issue (label «week ревью 1») 2026-06-15 20:21:11 +05:00
081_trade_in_estimates_expected_sold.sql feat(tradein): estimator additive expected_sold price (#648 S3) (#661) 2026-05-29 13:37:33 +00:00
082_scrape_schedules_seed_ratio_refresh.sql feat(tradein): daily asking→sold ratio refresh (#648 S4) (#663) 2026-05-29 13:49:36 +00:00
083_trade_in_estimates_created_by.sql fix(tradein): scope GET /history to caller, close cross-pilot data leak (#656) (#670) 2026-05-29 15:59:03 +00:00
084_brand_praktika_fill.sql feat(tradein): white-label branded page for client «Практика» (#657) (#672) 2026-05-29 16:09:23 +00:00
084_scrape_schedules_seed_n1_sweep.sql feat(scrapers): wire N1.ru city sweep — pipeline + scheduler + admin endpoint (#575) (#745) 2026-05-30 15:43:38 +00:00
085_quarter_price_index_fdw.sql feat(db): quarter_price_index FDW foreign table + monthly refresh (Refs #762) (#797) 2026-05-30 17:16:28 +00:00
086_deals_address_trgm_index.sql fix(tradein): data-layer tail A2+B+C — trgm index, FDW options, matview refresh (Refs #769) (#804) 2026-05-30 17:40:54 +00:00
087_fdw_server_options.sql fix(tradein): data-layer tail A2+B+C — trgm index, FDW options, matview refresh (Refs #769) (#804) 2026-05-30 17:40:54 +00:00
088_scrape_schedules_seed_search_matview_refresh.sql fix(tradein): data-layer tail A2+B+C — trgm index, FDW options, matview refresh (Refs #769) (#804) 2026-05-30 17:40:54 +00:00
089_listings_geo_precision.sql feat(estimator): exclude city-centroid listings from radius analogs (Refs #769 Part E) 2026-05-30 21:22:21 +03:00
090_scrape_schedules_seed_deactivate_stale_avito.sql feat(tradein): nightly task to deactivate stale avito listings (#759) (#862) 2026-05-31 07:19:37 +00:00
091_scrape_schedules_seed_yandex_address_backfill.sql feat(tradein): schedule yandex address backfill task (#855) (#866) 2026-05-31 07:31:10 +00:00
092_sber_price_index.sql feat(tradein): СберИндекс monthly city-level price index pull (#887, data-layer) (#890) 2026-05-31 12:44:56 +00:00
093_scrape_schedules_seed_sber_index_pull.sql feat(tradein): СберИндекс monthly city-level price index pull (#887, data-layer) (#890) 2026-05-31 12:44:56 +00:00
094_cadastral_unify.sql refactor(tradein): unify kadastr_num -> cadastral_number (#732) 2026-05-31 15:41:52 +03:00
095_dead_schema.sql fix(tradein): 095 drop v_data_quality before dropping sale_type_text (#731) 2026-05-31 16:57:18 +03:00
096_scrape_schedules_seed_rosreestr_quarter_poll.sql feat(tradein): scheduled monthly Rosreestr new-quarter poll → ingest alert (#888) (#894) 2026-05-31 13:13:55 +00:00
097_index_hygiene.sql perf(tradein): index hygiene — drop dup/redundant + de-partial is_active (#730) (#897) 2026-05-31 13:29:11 +00:00
098_asking_to_sold_ratios_tiered.sql feat(tradein): ppm²-tier segmentation of asking→sold ratio (#928, flag OFF) (#934) 2026-05-31 21:22:39 +00:00
099_brand_praktika_logo_wordmark.sql fix(tradein): praktika wordmark logo + form button cutoff on short viewports 2026-06-01 10:15:49 +03:00
100_enable_deactivate_stale_avito.sql fix(tradein): enable deactivate_stale_avito schedule on prod (#1096) 2026-06-06 19:08:22 +00:00
101_gendesign_reader_role.sql feat(sf): cross-load ETL tradein→gendesign newbuilding_listings — вариант A (#976) 2026-06-12 11:18:47 +03:00
102_grant_listings_gendesign_reader.sql feat(market): GRANT listings → gendesign_reader — unit-level доступ SF (#1191) 2026-06-12 12:04:41 +03:00
103_scrape_schedules_seed_newbuilding_enrich.sql feat(tradein-scheduler): nightly newbuilding-enrichment schedule (#973) 2026-06-13 16:24:03 +05:00
104_index_hygiene_geom_dedup.sql chore(tradein-db): drop duplicate listings_geom_active_idx (#730) 2026-06-13 11:53:13 +00:00
105_market_schema_yandex_enrichment.sql feat(tradein): yandex_newbuilding sweep-entrypoint + market-схема enrichment, seed dormant (#974) 2026-06-15 18:28:58 +03:00
106_scrape_schedules_seed_yandex_newbuilding_sweep.sql feat(tradein): yandex_newbuilding sweep-entrypoint + market-схема enrichment, seed dormant (#974) 2026-06-15 18:28:58 +03:00
107_scrape_schedules_seed_cian_city_sweep.sql fix(tradein): переименовать seed-миграцию 101→103 (коллизия с gendesign_reader #976) 2026-06-15 19:10:20 +03:00
108_clean_avito_addresses_v2.sql fix(migrations): живые миграции для прод-фиксов #1361/#1419/#1364 2026-06-15 21:31:28 +03:00
108_merge_duplicate_houses.sql fix(tradein): миграция 108 — typo cl.cluster_key + collision-safe репойнт (dup-vs-dup) 2026-06-16 08:46:44 +03:00
109_asking_to_sold_ratio_segment_filter.sql fix(migrations): живые миграции для прод-фиксов #1361/#1419/#1364 2026-06-15 21:31:28 +03:00
110_scrape_schedules_seed_geocode_missing_listings.sql fix(tradein): wire nightly geocode backfill для listings в in-app scheduler 2026-06-16 10:01:49 +03:00
111_listings_avito_detail_fields.sql fix(tradein): review minors — ceiling_height_m numeric(5,2) + strengthen finishing test 2026-06-16 10:24:20 +03:00
112_scrape_schedules_seed_avito_detail_backfill.sql feat(tradein): avito detail-enrichment nightly backfill (#1551) 2026-06-16 13:25:29 +03:00
113_deactivate_ghost_duplicate_listings.sql fix(db): deactivate pre-#753 ghost duplicate listings (167 rows) 2026-06-16 15:40:30 +03:00
113_yandex_detail_backfill.sql feat(tradein): yandex detail backfill — save_detail_enrichment + nightly task (#1553) 2026-06-16 13:58:20 +03:00
114_disable_n1_sweep.sql chore(tradein): деактивировать N1.ru scraper — источник переехал в Cian 2026-06-16 14:50:51 +03:00
115_scrape_schedules_seed_deactivate_stale_yandex_cian.sql feat(scrapers): segment-aware generic deactivate_stale for yandex/cian (#1558) 2026-06-16 16:09:19 +00:00
116_offer_price_history_change_trigger.sql feat(tradein/db): derived price-history trigger (avito/yandex) + last_seen_at deactivate index (#1559) 2026-06-16 16:48:58 +00:00
117_listings_last_seen_deactivate_index.sql feat(tradein/db): derived price-history trigger (avito/yandex) + last_seen_at deactivate index (#1559) 2026-06-16 16:48:58 +00:00
118_enable_cian_city_sweep.sql feat(scrapers): enable cian_city_sweep (proxy alive, sweep is cookieless) (#1568) 2026-06-16 17:08:47 +00:00
119_yandex_city_sweep_center_combos.sql feat(tradein): yandex city sweep — EKB-center rooms×price combos (replaces 5-anchor shallow) (#1658) 2026-06-16 19:57:37 +00:00
120_restore_partial_active_indexes.sql fix(sql): restore partial predicate on 5 listings indexes (#1398) 2026-06-17 20:50:51 +03:00
121_remove_brand_praktika.sql fix(tradein): remove «Практика» white-label brand (DB row + account mapping + assets) 2026-06-19 12:53:37 +03:00
121_yandex_rich_fields.sql feat(yandex): map rich entity fields (predictedPrice, seller, listing_date, trend, metro) 2026-06-17 22:58:16 +03:00
122_enable_domclick_city_sweep.sql feat(domclick): wire DomClick citywide sweep into pipeline+scheduler (seeded disabled) 2026-06-17 23:21:49 +03:00
123_avito_newbuilding_sweep_schedule.sql feat(avito): dedicated newbuilding (novostroyka) citywide sweep 2026-06-18 09:45:54 +03:00
124_cad_buildings_local.sql feat(cadastral): geo-nearest building matcher via local cad mirror 2026-06-18 10:18:54 +03:00
124_deglue_avito_addresses.sql fix(avito): de-glue house number from district marker in _clean_address (#1773) 2026-06-19 13:33:43 +05:00
125_scrape_schedules_seed_cadastral_geo_match.sql feat(cadastral): geo-nearest building matcher via local cad mirror 2026-06-18 10:18:54 +03:00
126_scrape_schedules_seed_cian_full_load.sql feat(tradein): schedule cian_full_load (exhaustive secondary) + anchor sweep NB-only 2026-06-18 13:25:23 +03:00
127_scrape_schedules_seed_avito_full_load.sql feat(tradein): avito exhaustive full-load (room×price bisection) — mirror cian 2026-06-18 13:55:18 +03:00
128_listings_card_hash.sql feat(listings): card_hash column + snapshot-dedup on unchanged cards 2026-06-18 22:01:54 +03:00
129_avito_full_load_incremental_split.sql feat(scheduler): per-source interval_days cadence (weekly exhaustive avito) 2026-06-18 23:10:18 +03:00
130_backfill_listings_house_id_fk.sql fix(db): #1781 backfill listings.house_id_fk from house_sources 2026-06-19 16:20:32 +05:00
130_ekb_geoportal_buildings.sql feat(tradein): EKB geoportal building ingest + geocoder tier (complete city buildings) 2026-06-19 17:51:07 +03:00
131_fix_diff_percent_overflow.sql fix(tradein/db): widen offer_price_history.diff_percent numeric(5,2)→(8,2) + clamp в триггере (краш full_load) 2026-06-20 11:43:23 +03:00
132_scrape_schedules_seed_house_imv.sql feat(scheduler): wire house_imv_backfill as scheduled source (#854) (#1910) 2026-06-26 07:45:08 +00:00
133_listings_uq_source_source_id.sql fix(migration-133): dedup via cascade-delete, not child re-point (prod hotfix) 2026-06-26 08:16:27 +00:00
134_listings_geom_geography_gist.sql perf(tradein): GIST index on (geom::geography) for estimator radius filter (#1189) (#1924) 2026-06-26 17:12:49 +00:00
135_scrape_schedules_seed_house_dedup_merge.sql feat(tradein): recurring house-dedup merge (schedule dormant) (#1772) (#1933) 2026-06-26 21:39:30 +00:00
136_backfill_listings_house_id_fk_source_identity.sql fix(tradein): backfill house_id_fk via source-identity path (#1781) 2026-06-27 05:01:02 +05:00