11 commits

Author	SHA1	Message	Date
bot-backend	dfc1ee8be0	feat: ревью-инструмент авто-маппингов Объектив↔ДОМ.РФ (/admin/mapping-review) (#2278)	2026-07-03 07:59:13 +00:00
bot-backend	830d9f429a	fix(etl): гео-проход — префикс «г Екатеринбург» адресам без города (#2177 остаток) (#2245)	2026-07-03 01:41:19 +00:00
bot-backend	031bafde86	feat(etl): гео-проход auto_core_geo_v6 для tier_b/ambiguous mapping (#2177 остаток) (#2244)	2026-07-03 01:19:59 +00:00
bot-backend	e9c8d05e1d	feat(etl): auto_core_dev_v5 — core-name+dev backfill mapping'а (#2177 шаг 2) (#2237)	2026-07-02 21:59:16 +00:00
lekss361	1c184959a8	feat(etl): read tradein data via market.v_* contract views (#2138 step 2/3)	2026-07-02 15:00:38 +00:00
bot-backend	9c52e0b29f	feat(etl): housing-class normalization fallback via yandex_realty trigram match (#38) (#1911) ... housing-class normalization fallback via yandex_realty trigram (#38): migration 169 + ETL + COALESCE consumers. Refs #38	2026-06-26 07:44:25 +00:00
bot-backend	14f3ef2019	fix(week-review): backend-аудит v2 — 82 фиксов (#1660) ... Co-authored-by: bot-backend <bot-backend@gendsgn.local> Co-committed-by: bot-backend <bot-backend@gendsgn.local>	2026-06-17 17:13:38 +00:00
bot-backend	04da30b57a	fix(sf): psql stdin-bootstrap пароля gendesign_reader + COALESCE-protect внешних id (review fixup, #976) ... - Создан tradein-mvp/ops/db-bootstrap/set_gendesign_reader_password.sql: set_config GUC + format('%L') + \o /dev/null — зеркальный паттерн ops/db-bootstrap/set_tradein_fdw_password.sql. Пароль не попадает в stdout/logs (leak-protected). - deploy-tradein.yml: шаг bootstrap переписан на stdin-redirect (< ops/db-bootstrap/set_gendesign_reader_password.sql) вместо сломанного -c "ALTER ROLE ... PASSWORD :'TRADEIN_READER_PASSWORD'", которое давало syntax error at ':' (psql не интерполирует :'var' внутри -c строки). - newbuilding_crossload.py ON CONFLICT DO UPDATE: yandex_jk_id и cian_internal_house_id → COALESCE(EXCLUDED.col, table.col), чтобы cian-запись не затирала yandex_jk_id NULL'ом и наоборот. - Тест test_upsert_sql_coalesce_external_ids проверяет наличие COALESCE в DO UPDATE секции для обоих id.	2026-06-12 11:26:40 +03:00
bot-backend	1a4778d006	feat(sf): cross-load ETL tradein→gendesign newbuilding_listings — вариант A (#976) ... - docker-compose.prod.yml: backend + worker добавлены в сеть gendesign_shared (нужен прямой TCP к tradein-postgres для psycopg ETL) - tradein-mvp/data/sql/101_gendesign_reader_role.sql: роль gendesign_reader (LOGIN, SELECT на houses/house_sources/houses_price_dynamics/ house_reliability_checks/house_reviews; пароль через TRADEIN_READER_PASSWORD bootstrap в deploy-tradein.yml) - .forgejo/workflows/deploy-tradein.yml: bootstrap ALTER ROLE gendesign_reader PASSWORD из env после миграций; warn (не exit 1) если переменная не задана - backend/app/core/config.py: settings.tradein_database_url (env TRADEIN_DATABASE_URL) - backend/app/services/etl/newbuilding_crossload.py: run_crossload() — psycopg server-side cursor, batch 500, UPSERT ON CONFLICT (source, ext_house_id), SAVEPOINT per-row; disabled-режим если url пуст - backend/app/workers/tasks/etl_newbuilding_crossload.py: Celery task - backend/app/workers/beat_schedule.py: nightly 03:30 МСК (00:30 UTC) - backend/app/api/v1/admin_scrape.py: POST /admin/scrape/newbuilding-crossload - backend/tests/services/test_newbuilding_crossload.py: 10 тестов без реальной БД needs-human: TRADEIN_READER_PASSWORD → tradein .env.runtime; TRADEIN_DATABASE_URL → gendesign backend/.env.runtime	2026-06-12 11:18:47 +03:00
lekss361	2694e3180b	feat(etl): fuzzy matcher v2 — pruned threshold 0.85→0.80 for objective mapping coverage ... - Add AUTO_ACCEPT_THRESHOLD_V2 = 0.80 constant to objective_backfill.py - Add min_threshold param to find_match_candidates() (default REVIEW_THRESHOLD=0.6) - Add match_method param to auto_apply_matches() (default 'fuzzy_trgm') - Add ?v2=true query param to POST /api/v1/admin/etl/objective-backfill - v2 mode: threshold=0.80, method='fuzzy_v2', search from 0.80 (not 0.60) - Return type widened to dict[str,object] to include threshold_used + match_method_used DB audit: 1068 unmapped EKB objs; v1 adds ~133 rows, v2 adds ~47 more. Expected coverage: 8.5% → ~20% after sequential v1+v2 runs on prod. Run: POST /objective-backfill (v1) then POST /objective-backfill?v2=true (v2). Part of task #44 Part A, epic #271.	2026-05-17 16:28:20 +03:00
lekss361	dbf31cc911	feat(etl): objective_complex_mapping backfill via pg_trgm (#203) (#214)	2026-05-16 12:47:09 +00:00