59 commits

Author	SHA1	Message	Date
bot-backend	46269b527a	feat(tradein): exhaustive Cian EKB secondary load — room×price partitioning ... Обходит Cian SERP-cap (~54 стр): адаптивно бьёт цену на бакеты внутри комнатности (totalOffers-driven split, minprice/maxprice), пагинирует каждый полностью, дедуп по source_id. Region-wide один проход (anchor'ы для Cian избыточны) + IP-rotation (cian_proxy_rotate_url). Endpoint POST /admin/scrape/cian-full-load.	2026-05-31 22:43:05 +03:00
bot-backend	62d989b8a3	feat(tradein): Cian browser auto-login — POST /admin/scrape/cian/auto-login (#639) (#917) ... Co-authored-by: bot-backend <bot-backend@gendsgn.local> Co-committed-by: bot-backend <bot-backend@gendsgn.local>	2026-05-31 18:24:51 +00:00
lekss361	997eda489c	fix(tradein): single-run guard на avito-city-sweep admin-endpoint (#874) ... Co-authored-by: lekss361 <lekss361@gendsgn.local> Co-committed-by: lekss361 <lekss361@gendsgn.local>	2026-05-31 08:55:33 +00:00
bot-backend	60353beaaf	feat(tradein): yandex city-sweep — один прогон SERP+address-enrich (#867) ... Co-authored-by: bot-backend <bot-backend@gendsgn.local> Co-committed-by: bot-backend <bot-backend@gendsgn.local>	2026-05-31 07:43:13 +00:00
bot-backend	f2a123e2aa	feat(tradein): cian city-sweep — один прогон SERP+detail(+price-history)+houses (#864) ... Co-authored-by: bot-backend <bot-backend@gendsgn.local> Co-committed-by: bot-backend <bot-backend@gendsgn.local>	2026-05-31 07:24:34 +00:00
bot-backend	d78ae9f649	feat(tradein): avito city-sweep — IMV-фаза (один прогон = SERP+detail+houses+IMV) (#861) ... Co-authored-by: bot-backend <bot-backend@gendsgn.local> Co-committed-by: bot-backend <bot-backend@gendsgn.local>	2026-05-31 07:09:42 +00:00
bot-backend	5373f5db63	feat(tradein): admin backfill endpoints — cian price-history (T8a) + yandex address (T10) + house IMV (T7) (#856) ... Co-authored-by: bot-backend <bot-backend@gendsgn.local> Co-committed-by: bot-backend <bot-backend@gendsgn.local>	2026-05-31 06:37:10 +00:00
bot-backend	4c275e04ea	fix(tradein): word-boundary street match + cian unescape robustness (Refs #769 A1+D) (#798) ... Co-authored-by: bot-backend <bot-backend@gendsgn.local> Co-committed-by: bot-backend <bot-backend@gendsgn.local>	2026-05-30 17:18:33 +00:00
bot-backend	38d44f46b7	fix(pdf): brand-spoof + median=0 empty-state + SSRF + дисклеймер + sources (Refs #772) (#789) ... Co-authored-by: bot-backend <bot-backend@gendsgn.local> Co-committed-by: bot-backend <bot-backend@gendsgn.local>	2026-05-30 16:55:56 +00:00
bot-backend	bcd035034a	fix(tradein): SSRF host-allowlist на admin scrape + удалён dead ADMIN_TOKEN (Refs #756) (#778) ... Co-authored-by: bot-backend <bot-backend@gendsgn.local> Co-committed-by: bot-backend <bot-backend@gendsgn.local>	2026-05-30 16:27:19 +00:00
bot-backend	9954f4962f	fix(tradein): /sales-vs-listings street_only не house_linked (Refs #757) (#777) ... Co-authored-by: bot-backend <bot-backend@gendsgn.local> Co-committed-by: bot-backend <bot-backend@gendsgn.local>	2026-05-30 16:26:08 +00:00
bot-backend	a74591d7cc	fix(tradein): atomic conditional quota increment — close TOCTOU (#747) (#749) ... Co-authored-by: bot-backend <bot-backend@gendsgn.local> Co-committed-by: bot-backend <bot-backend@gendsgn.local>	2026-05-30 15:52:34 +00:00
bot-backend	6762a532eb	feat(scrapers): wire N1.ru city sweep — pipeline + scheduler + admin endpoint (#575) (#745) ... Co-authored-by: bot-backend <bot-backend@gendsgn.local> Co-committed-by: bot-backend <bot-backend@gendsgn.local>	2026-05-30 15:43:38 +00:00
bot-backend	575d9abbf7	feat(tradein): add data_quality flag to /sales-vs-listings (#721) (#723) ... Co-authored-by: bot-backend <bot-backend@gendsgn.local> Co-committed-by: bot-backend <bot-backend@gendsgn.local>	2026-05-30 13:21:08 +00:00
bot-backend	466774dbd8	fix(tradein): rehydrate POST-only fields on GET /estimate/{id} (#696) (#710) ... Co-authored-by: bot-backend <bot-backend@gendsgn.local> Co-committed-by: bot-backend <bot-backend@gendsgn.local>	2026-05-30 11:05:52 +00:00
bot-backend	737599e690	fix(tradein): scope GET /estimate/{id} + /pdf to owner, close IDOR (#690) (#702) ... Co-authored-by: bot-backend <bot-backend@gendsgn.local> Co-committed-by: bot-backend <bot-backend@gendsgn.local>	2026-05-30 08:56:51 +00:00
bot-backend	29679a9545	fix(tradein): time-budget estimate enrichment + observable errors to stop opaque 502 (#654) (#673) ... Co-authored-by: bot-backend <bot-backend@gendsgn.local> Co-committed-by: bot-backend <bot-backend@gendsgn.local>	2026-05-29 17:22:02 +00:00
bot-backend	47fdd6e4df	fix(tradein): scope GET /history to caller, close cross-pilot data leak (#656) (#670) ... Co-authored-by: bot-backend <bot-backend@gendsgn.local> Co-committed-by: bot-backend <bot-backend@gendsgn.local>	2026-05-29 15:59:03 +00:00
Light1YT	35bd0238ef	feat(tradein): estimator additive expected_sold price (#648 S3) (#661)	2026-05-29 13:37:33 +00:00
lekss361	bd3e5c8216	feat(tradein): аккаунт praktika + лимит 15 оценок/мес, чистка PRINZIP в UI (#635)	2026-05-28 22:04:58 +00:00
Light1YT	1731704ddc	feat(tradein): houses.cian_zhk_url column + backfill (#568) (#627)	2026-05-28 13:57:43 +00:00
Light1YT	492dd0aba6	feat(tradein): бейдж точности адреса (qc_geo) на карточке оценки (accuracy #1) ... Честность к пользователю: показываем насколько точно определён адрес — «до дома» / «до улицы» / «приблизительно» — рядом с confidence. - schemas/trade_in.py: address_precision: Literal["house","street","approximate"]|None - estimator.py: _qc_geo_to_precision (qc_geo 0→house, 1→street, ≥2→approximate, None→None), set в основном return + _empty_estimate - api/v1/trade_in.py: get_estimate + estimate_pdf теперь rehydrate'ят DaData-поля (canonical_address, house_cadnum, house_fias_id, address_precision, metro_nearest) из row — раньше на ?id= reopen / в PDF вся Q1-обогащёнка пропадала (поймал code-reviewer). Колонки уже persisted, фикс read-path. - frontend: AddressPrecision type + precision-badge в HeroSummary (.card-meta), токены --success/--accent-2/--bg-card-alt, без новых токенов. Канон: tech-analyst (план) → backend-engineer + frontend-engineer → code-reviewer (✅ APPROVE, поймал reopen-path gap) → fold-in fix. Tests: 7 passed, tsc clean, ruff clean. match_house_readonly wiring отложен (fast-follow).	2026-05-28 15:37:33 +05:00
Light1YT	39da0f0644	feat(tradein): sales-vs-listings endpoint (PR K, foundation #564) (#597)	2026-05-27 11:55:44 +00:00
Light1YT	42ab74e2e8	feat(tradein): MapPicker snap marker — Phase 5 of #582 (#594)	2026-05-27 09:30:17 +00:00
Light1YT	88cdfd6adb	feat(rbac): role-based access control via X-Authenticated-User middleware (#585) ... Backend RBAC + Caddy header_up. Closes part of #fixes-rbac-pra.	2026-05-26 06:18:40 +00:00
lekss361	a63e459e25	feat(tradein): GET /street-deals — ДКП-сделки Росреестра по улице target адреса (#555)	2026-05-24 20:05:48 +00:00
lekss361	0ac54d1de0	feat(tradein): sell-time sensitivity block (4 цены × медиана срока продажи) ... Backend: GET /api/v1/trade-in/estimate/{id}/sell-time-sensitivity - 4 фиксированных бакета по premium к медиане года: −5% / медиана (±3%) / +5% / +10% - Median exposure_days + p25/p75 per bucket - Filter outliers: last_price > start_price * 0.7 (отбрасываем подозрительно дешёвые) - Benchmark: median ₽/м² за последние 2 года Frontend: новый SellTimeSensitivity component (4-card grid) - color-coded buckets (green→blue→yellow→red) - median highlighted синей рамкой - показывает "~31 дн" + "обычно 31-61 дн" + count лотов - встроен в HouseAnalyticsSection между KpiRow и PriceHistoryChart Использует существующий house_placement_history (15k+ rows).	2026-05-24 22:43:32 +03:00
lekss361	967e7185ca	feat(tradein): cian price-change badges + split chart Avito/Yandex (#547)	2026-05-24 18:24:00 +00:00
lekss361	80850a591e	feat(tradein): house analytics section (chart + KPI + sold list) (#546)	2026-05-24 18:09:45 +00:00
lekss361	ba9cc3aef1	feat(tradein/cian): valuation backfill + runner script для prod (#539)	2026-05-24 15:16:45 +00:00
lekss361	bc594e3f79	feat(tradein/cian): batch backfill task для offer_price_history + houses_price_dynamics (#535)	2026-05-24 14:55:47 +00:00
lekss361	0d10eb400f	fix(tradein-api): PlacementHistoryEntry schema + None checks (PR #528 follow-up) (#532)	2026-05-24 14:33:53 +00:00
lekss361	7dc11e4339	feat(tradein-api): surface backfilled houses + placement history (#528)	2026-05-24 14:22:50 +00:00
lekss361	591039493c	feat(tradein/cian): admin endpoints + UI for detail+newbuilding history scrape (#525)	2026-05-24 14:05:55 +00:00
lekss361	d9c6b1bd01	chore(tradein): remove dead mock-fallback code from trade_in.py (#521)	2026-05-24 13:11:13 +00:00
lekss361	0767fb7401	fix(tradein-photos): sanitize uploaded images via Pillow re-encode (#511)	2026-05-24 11:32:04 +00:00
lekss361	94cf199276	feat(tradein): postgres_fdw live read of gendesign.cad_buildings (replaces snapshot) (#493) ... Replaces tradein.cad_buildings snapshot with live postgres_fdw foreign table reading gendesign.v_tradein_cad_buildings. Fixes /trade-in/api/v1/geocode/reverse 500 (Nominatim ban) and address_not_geocoded for cadastre addresses (e.g. Хохрякова 81). Security (deep-review fixes): - 100_tradein_fdw_role.sql: passwordless CREATE ROLE; password set by deploy.yml ALTER ROLE bootstrap reading GENDESIGN_FDW_PASSWORD from backend/.env.runtime (via psql :'pw' var → format %L — injection-safe). - core/fdw.py: regex whitelist [A-Za-z0-9_-]{32,256} on password, ValueError without echoing value, try/rollback on commit. - 060_postgres_fdw_extension.sql: connect_timeout='3' on FOREIGN SERVER + ALTER ADD/SET fallback. - geocoder.py: _cadastral_forward_sync / _cadastral_reverse_sync wrapped in asyncio.to_thread. - 100_*.sql: REVOKE ALL ON ALL TABLES/SEQUENCES/FUNCTIONS IN SCHEMA public; only GRANT SELECT on v_tradein_cad_buildings. - pg_user_mappings query handles PUBLIC mapping (usename IS NULL). Tests: 3 SQL-injection guards on ensure_fdw_user_mapping + rewritten cadastral suite.	2026-05-24 08:57:30 +00:00
lekss361	222389cba9	fix(tradein/avito): anti-bot hardening — sleep + abort + shared session (#487)	2026-05-23 20:08:40 +00:00
lekss361	52716a46c6	fix(tradein): geocode backfill + remove Avito exclusion from estimator (#490)	2026-05-23 20:03:48 +00:00
lekss361	e79c9cec88	feat(tradein): global scraper delay setting (applies across all scrapers) (#485) ... Add global request delay floor + per-source delay management. - 054: seed 'global'=0 + per-source rows (avito=7, cian=5, n1=5, domrf=5, rosreestr=5) using ON CONFLICT DO NOTHING; reuses existing 053 table from #484 - scraper_settings.py: get_scraper_delay() = max(per_source, global); _GLOBAL_KEY, refactored _get_setting_cached(); preserved yandex umbrella alias logic - admin.py: GET /scraper-settings + PUT /scraper-settings/{source} (ge=0.0, le=60.0); cache invalidation on update; CAST(:d AS numeric) per psycopg v3 - avito/cian/n1 scrapers wired (yandex was wired by #484) - 18 unit tests pass (10 original updated for max() + 8 new global delay tests) Closes follow-up gap from #486 (Cian admin slider min=0 now accepted by API).	2026-05-23 15:43:28 +00:00
lekss361	b21d7c7c85	feat(tradein): scraper_settings live-config + Yandex admin trigger endpoints (#484)	2026-05-23 15:28:34 +00:00
lekss361	0202720096	feat(tradein): in-app scheduler — UI-managed schedule (replaces SSH crontab) (#482)	2026-05-23 14:54:25 +00:00
lekss361	9643979d2c	feat(tradein): Phase 3.2 — /api/v1/search endpoint + Redis cache (#479) ... POST /api/v1/search reading listings_search_mv with ~20 cross-source filters, parameterized SQL (CAST(:x AS type), psycopg v3), whitelisted ORDER BY (Pydantic Literal), Redis 5min hot cache with graceful degradation (singleton pool via lru_cache). Verified vs data/sql/050_search_optimization.sql: matview column refs (total_area, lng, house_rating, sources[], has_avito/cian/yandex), SQL injection safety, cache failure swallow, router prefix. Deep-code-reviewer: APPROVE.	2026-05-23 14:44:22 +00:00
lekss361	2914ff2ef2	feat(tradein): city sweep — auto ЕКБ pipeline (#477) ... Stage 4d: full city sweep ЕКБ (5 anchors × N pages + enrichment + cooperative cancel). - avito.py fetch_around(pages=N) — pagination backward compat default=1 - scrape_pipeline.run_avito_city_sweep — anchors loop + heartbeat + cancel check - scrape_runs.py utility (create/heartbeat/done/failed/cancelled/list_recent) - admin.py 3 endpoints: start (BackgroundTasks) / runs / cancel - migration 051 ADD COLUMN IF NOT EXISTS + DROP/ADD CHECK constraint (text+CHECK, не enum — safe в transaction) - deploy/avito-city-sweep.sh с random delay 0-3h (anti-pattern) - 16 offline tests Deep review approved: migration idempotent, cancel priority correct (mark_done guarded by status='running'), no SQL injection, psycopg v3 compliant.	2026-05-23 14:38:07 +00:00
lekss361	dbad7a626b	fix(tradein): убрать require_admin полностью — Caddy basic_auth достаточен (#474)	2026-05-23 14:21:01 +00:00
lekss361	b5d206bfa8	feat(tradein): Stage 4a — API endpoints for Avito enrichment + IMV benchmark UI (#460)	2026-05-23 13:34:38 +00:00
lekss361	2dd76ea5e8	feat(tradein): cian_session cookies management + admin endpoints (Wave 4 Worker A) (#457)	2026-05-23 13:22:00 +00:00
Light1YT	9b2a54788b	fix(tradein): восстановленная оценка показывает этаж/площадь, не 0/0 ... AggregatedEstimate не нёс параметры целевой квартиры (площадь, этаж, комнаты, год, тип дома). HeroSummary брал их из формы-инпута — у свежей оценки работало, но при открытии оценки по ссылке (?id=, из Истории) формы нет → «Сводка» показывала «ЭТАЖ 0/0, Площадь 0 м²». Параметры теперь проходят сквозь весь стек: - AggregatedEstimate + TS-тип получили area_m2/rooms/floor/ total_floors/year_built/house_type/repair_state/has_balcony; - estimate_quality заполняет их из payload, get_estimate — из строки trade_in_estimates (колонки там уже есть); - HeroSummary читает из estimate с откатом на input. Заодно в TS-тип добавлен пропущенный est_days_on_market.	2026-05-22 15:50:19 +05:00
Light1YT	0b14538f7d	fix(tradein): бюджет по времени в geocode-missing — cron не таймаутит ... cron geocode-missing-шаг дёргает эндпоинт через `curl -m 320`, а геокодинг 100 адресов упирается в Nominatim (1 req/sec + typo-тиры на провалах) и выходит за 320с. При таймауте curl cron-loop обрывался — объявления Cian/N1 оставались без координат и были невидимы для оценки (ST_DWithin по geom). Сейчас ~200 таких объявлений в БД. geocode_missing теперь выходит из цикла по бюджету 240с (заведомо меньше cron-таймаута) и возвращает remaining > 0 — cron штатно продолжает следующим чанком. Заодно убран мёртвый import asyncio.	2026-05-22 14:37:15 +05:00
TradeIn Deploy	18c2e64182	fix(tradein): GET /estimate/{id} возвращает полный набор полей ... При открытии оценки по ссылке (?id=) или перезагрузке страницы get_estimate отдавал огрызок: без sources_used, confidence_explanation, target_address, координат, data_freshness_minutes. Из-за этого карточка «Сбор данных по квартире» пустела до «0 / 7 источников · ожидает запрос» — хотя данные в БД есть. Теперь SELECT и ответ включают полный набор полей AggregatedEstimate. period_months 24 → 12 (совпадает с estimator / DEALS_PERIOD_MONTHS).	2026-05-22 13:20:53 +05:00