lekss361/gendesign
1
0
Fork 0
Code Issues 57 Pull requests Projects Releases Packages Wiki Activity Actions
gendesign/data/sql
History
lekss361 f80a2c1d05 fix(scrape): SQLAlchemy text() парсил ':00:' regex как named-param 
В nspd_kn.py:get_pending_cads и admin_scrape.py:nspd_coverage regex-фильтры
quarter_cad_number !~ '^00:00:' и !~ ':0000000$' падали с psycopg.errors.
DataError — двоеточие в text() трактуется как ':param'. Заменил на NOT LIKE
с bindparams (быстрее regex и без issue с парсингом).

Эффект на проде:
- /api/v1/admin/scrape/nspd/coverage возвращал 500 → теперь 200
- Кнопка «Запустить sweep» больше не дизейбленa (pending=964 теперь видно)
- Сам scraper тоже бы валился на этом же баге при beat-запуске
2026-04-30 23:16:11 +03:00
..
01_schema_rosreestr_deals.sql add data 2026-04-26 22:30:52 +03:00
02_load_all_quarters.sh fix 2026-04-30 22:16:44 +03:00
02b_load_via_tunnel.py fix 2026-04-30 22:16:44 +03:00
03_materialized_views.sql add data 2026-04-26 22:30:52 +03:00
10_schema_domrf.sql add data 2026-04-26 22:30:52 +03:00
11_load_domrf.py add data 2026-04-26 22:30:52 +03:00
20_schema_domrf_extras.sql add data 2026-04-26 22:30:52 +03:00
21_load_domrf_extras.py add data 2026-04-26 22:30:52 +03:00
30_scrape_domrf.py add anal 2 2026-04-27 13:05:36 +03:00
31_schema_domrf_full.sql add data 2026-04-26 22:30:52 +03:00
31_schema_domrf_raw.sql add data 2026-04-26 22:30:52 +03:00
32_load_domrf_full.py add data 2026-04-26 22:30:52 +03:00
32_load_domrf_raw.py add data 2026-04-26 22:30:52 +03:00
33_schema_domrf_lazy.sql add anal 2 2026-04-27 13:05:36 +03:00
34_load_domrf_lazy.py add anal 2 2026-04-27 13:05:36 +03:00
35_schema_domrf_stats.sql add anal 2 2026-04-27 13:05:36 +03:00
36_parse_xlsx_stats.py add anal 2 2026-04-27 13:05:36 +03:00
37_cleanup_redundant.sql add anal 2 2026-04-27 13:05:36 +03:00
38_relations_fix.sql add anal 2 2026-04-27 13:05:36 +03:00
39_territory_aliases.sql add anal 2 2026-04-27 13:05:36 +03:00
40_relations.sql add data 2026-04-26 22:30:52 +03:00
40_scrape_history.py add anal 2 2026-04-27 13:05:36 +03:00
41_load_history.py add anal 2 2026-04-27 13:05:36 +03:00
42_schema_breakdown_widen.sql add anal 2 2026-04-27 13:05:36 +03:00
43_anton_import.sql add anal 2 2026-04-27 13:05:36 +03:00
44_import_anton_db.py add anal 2 2026-04-27 13:05:36 +03:00
45_relations_v2.sql add anal 2 2026-04-27 13:05:36 +03:00
50_schema_kn_extensions.sql add interactive parser of наш.дом.рф ЖК+flats with stealth Playwright 2026-04-27 17:58:40 +03:00
50_scrape_kn_flats.py add domrf extras parser (sale_graph + sales_agg + infra + photos) + 2026-04-27 18:35:46 +03:00
51_schema_kn_extras.sql add domrf extras parser (sale_graph + sales_agg + infra + photos) + 2026-04-27 18:35:46 +03:00
52_import_prinzip_crm.py ship #13-16: multi-target Dockerfile, per-object drill-in, CRM pipeline, 2026-04-27 19:32:37 +03:00
52_schema_prinzip_crm.sql ship #13-16: multi-target Dockerfile, per-object drill-in, CRM pipeline, 2026-04-27 19:32:37 +03:00
53_schema_kn_thumbs.sql fix ui lag 2026-04-27 20:26:55 +03:00
54_schema_scrape_log.sql add logs 2026-04-27 20:51:44 +03:00
55_schema_scrape_resume.sql add log scrapper 2026-04-27 21:50:44 +03:00
56_fetch_ekb_districts.py feat(geo): ЖК → district mapping через PostGIS-полигоны OSM (Limit_District_Geometry closed) 2026-04-29 08:03:38 +03:00
56_schema_ekb_districts_geom.sql feat(geo): ЖК → district mapping через PostGIS-полигоны OSM (Limit_District_Geometry closed) 2026-04-29 08:03:38 +03:00
57_kn_objects_district.sql feat(geo): ЖК → district mapping через PostGIS-полигоны OSM (Limit_District_Geometry closed) 2026-04-29 08:03:38 +03:00
58_fetch_cad_quarters_nspd.py sprint1: nspd scraper industrialization, per-bucket elasticity, cadastre cross-check, sentry releases 2026-04-30 21:51:19 +03:00
58_schema_cad_quarters.sql sprint1: nspd scraper industrialization, per-bucket elasticity, cadastre cross-check, sentry releases 2026-04-30 21:51:19 +03:00
59_backfill_obj_cad_quarter.sql sprint1: nspd scraper industrialization, per-bucket elasticity, cadastre cross-check, sentry releases 2026-04-30 21:51:19 +03:00
60_v_zk_rosreestr_velocity.sql sprint1: nspd scraper industrialization, per-bucket elasticity, cadastre cross-check, sentry releases 2026-04-30 21:51:19 +03:00
61_import_nspd_batch.py sprint1: nspd scraper industrialization, per-bucket elasticity, cadastre cross-check, sentry releases 2026-04-30 21:51:19 +03:00
62_scrape_nspd_full.py sprint1: nspd scraper industrialization, per-bucket elasticity, cadastre cross-check, sentry releases 2026-04-30 21:51:19 +03:00
63_schema_nspd_runs.sql fix 2026-04-30 22:16:44 +03:00
64_v_zk_rosreestr_velocity.sql fix 2026-04-30 22:16:44 +03:00
65_partitions_rosreestr_2024_h1.sql fix 2026-04-30 22:16:44 +03:00
66_indexes_recommend.sql audit-fix: 27mo окно, MODE bucket, индексы recommend_mix 2026-04-30 22:41:43 +03:00
67_refresh_ekb_districts_median.sql fix(scrape): SQLAlchemy text() парсил ':00:' regex как named-param 2026-04-30 23:16:11 +03:00
decode_to_file.py add data 2026-04-26 22:30:52 +03:00
upload_server.py add data 2026-04-26 22:30:52 +03:00