479f01ae1a
Merge remote-tracking branch 'forgejo/main' into HEAD
...
CI / changes (push) Successful in 7s
CI / changes (pull_request) Successful in 8s
CI / frontend-tests (push) Successful in 1m7s
CI / openapi-codegen-check (push) Failing after 2m9s
CI / frontend-tests (pull_request) Successful in 1m4s
CI / openapi-codegen-check (pull_request) Failing after 1m37s
CI / backend-tests (pull_request) Failing after 9m7s
CI / backend-tests (push) Failing after 9m11s
# Conflicts:
# backend/app/services/site_finder/gate_verdict.py
2026-06-15 21:07:05 +03:00
483fdb0caa
Merge pull request 'feat(tradein): scheduler-источник cian_city_sweep (newbuilding Phase 4), seed dormant ( #973 )' ( #1296 ) from feat/cian-city-sweep-schedule into main
...
Deploy Trade-In / changes (push) Successful in 8s
Deploy Trade-In / build-frontend (push) Has been skipped
Deploy Trade-In / build-browser (push) Has been skipped
Deploy Trade-In / test (push) Successful in 32s
Deploy Trade-In / build-backend (push) Successful in 56s
Deploy Trade-In / deploy (push) Successful in 42s
Reviewed-on: #1296
2026-06-15 17:48:33 +00:00
1241d7fe4d
fix(tradein): переименовать seed-миграцию 101→103 (коллизия с gendesign_reader #976 )
CI / changes (pull_request) Successful in 7s
CI / changes (push) Successful in 7s
CI / backend-tests (push) Has been skipped
CI / frontend-tests (push) Has been skipped
CI / openapi-codegen-check (push) Has been skipped
CI / backend-tests (pull_request) Has been skipped
CI / frontend-tests (pull_request) Has been skipped
CI / openapi-codegen-check (pull_request) Has been skipped
2026-06-15 19:10:20 +03:00
b0a4df48e1
feat(tradein): scheduler-источник cian_city_sweep (newbuilding Phase 4), seed dormant ( #973 )
2026-06-15 19:09:26 +03:00
35bc792cf2
fix(tradein): обновить test_scraper_proxy под BrowserFetcher-транспорт fetch_newbuilding — зелёный CI-гейт ( #972 )
CI / changes (push) Successful in 6s
CI / backend-tests (push) Has been skipped
CI / frontend-tests (push) Has been skipped
CI / openapi-codegen-check (push) Has been skipped
CI / changes (pull_request) Successful in 5s
CI / backend-tests (pull_request) Has been skipped
CI / frontend-tests (pull_request) Has been skipped
CI / openapi-codegen-check (pull_request) Has been skipped
2026-06-15 19:02:13 +03:00
b3e7a9f9fa
Merge pull request 'fix(tradein): cian fetch_newbuilding через BrowserFetcher (camoufox) — initialState под анти-ботом ( #972 )' ( #1540 ) from fix/cian-newbuilding-browser-fetch into main
...
Deploy Trade-In / changes (push) Successful in 6s
Deploy Trade-In / build-frontend (push) Has been skipped
Deploy Trade-In / test (push) Failing after 40s
Deploy Trade-In / deploy (push) Has been skipped
Deploy Trade-In / build-browser (push) Has been skipped
Deploy Trade-In / build-backend (push) Has been skipped
Reviewed-on: #1540
2026-06-15 15:44:42 +00:00
c9daf8e268
fix(tradein): yandex sweep — guard ext_id=NULL перед fetch_jk + cleanup httpx ( #974 review M1/M3)
...
CI / backend-tests (push) Has been skipped
CI / frontend-tests (push) Has been skipped
CI / openapi-codegen-check (push) Has been skipped
CI / backend-tests (pull_request) Has been skipped
CI / changes (push) Successful in 7s
CI / changes (pull_request) Successful in 8s
CI / frontend-tests (pull_request) Has been skipped
CI / openapi-codegen-check (pull_request) Has been skipped
M1: добавлен guard перед вызовом fetch_jk — если ext_id пустой/None при наличии
jk_slug, логируем warning, incrementим failed_fetch и пропускаем (continue).
Пустой jk_id формирует битый URL /{slug}-/ и коллизию на UNIQUE ext_id=''.
M3: в admin.py /scrape/yandex-newbuilding заменён `async with YandexNewbuildingScraper()`
на прямое `scraper = YandexNewbuildingScraper()` — fetch_jk использует внутренний
BrowserFetcher, httpx-клиент BaseScraper открывался вхолостую.
Тесты: добавлен test_sweep_slug_present_but_ext_id_null_skips_fetch.
2026-06-15 18:41:08 +03:00
8279b0ee1f
fix(week-review): cross-file добивка код-ревью ( #1338 , #1363 , #1419 , #1421 , #1424 , #1480 )
...
- #1338 concepts.py: geometry.generate вынесен в run_in_threadpool (не блокирует event loop)
- #1363 house_imv_backfill.py + scrape_pipeline.py: периодический heartbeat в долгих IMV-фазах (reap_zombies не помечает живой sweep zombie)
- #1419 site-finder-api.ts: placeholderData keepPreviousData для bbox-запроса (KPI не мигает «0/0» при пане/зуме)
- #1421 site-finder-api.ts: area_ha nullable + адаптер не подставляет 0; null-гейт потребителей EntryMap.tsx, ParcelDrawer.tsx
- #1424/#1480 EstimateResult.tsx: блок «Параметры объекта» скрыт при восстановлении по shared-ссылке (нет мусора «0 м²/Студия/Этаж 0 из 0»)
Верификация: tsc --noEmit 0 ошибок; py_compile OK.
2026-06-15 20:29:00 +05:00
feab585ad5
feat(tradein): yandex_newbuilding sweep-entrypoint + market-схема enrichment, seed dormant ( #974 )
...
CI / backend-tests (push) Has been skipped
CI / frontend-tests (push) Has been skipped
CI / openapi-codegen-check (push) Has been skipped
CI / changes (pull_request) Successful in 5s
CI / backend-tests (pull_request) Has been skipped
CI / frontend-tests (pull_request) Has been skipped
CI / openapi-codegen-check (pull_request) Has been skipped
CI / changes (push) Successful in 7s
- SQL 105: CREATE SCHEMA market + market.yandex_jk_enrichment (ext_id UNIQUE, upsert-ready),
houses.yandex_jk_slug, GRANT gendesign_reader на market-слой
- SQL 106: seed scrape_schedules yandex_newbuilding_sweep (enabled=false dormant, окно 02-05 UTC)
- yandex_newbuilding.py: fetch_jk → BrowserFetcher (camoufox), add resolve_yandex_jk_slug
(SERP siteId query через BrowserFetcher, slug из _JK_SLUG_RE href)
- tasks/yandex_newbuilding_sweep.py: enrich_yandex_newbuilding_sweep (resolve slug →
fetch_jk → UPSERT market.yandex_jk_enrichment, SAVEPOINT per house, dry_run, resumable)
- admin.py: POST /scrape/yandex-newbuilding-sweep + GET .../runs
- scheduler.py: trigger_yandex_newbuilding_sweep_run + dispatch elif branch
- tests: 8 task tests + 3 scheduler dispatch tests (20 pass)
2026-06-15 18:28:58 +03:00
86e9ea2937
fix(week-review): автофиксы код-ревью — 169 issue (label «week ревью 1»)
...
Многоагентный аудит + имплементация: один воркер на файл, точечные правки.
Верификация: py_compile (47/47 .py) + tsc --noEmit (0 ошибок). Unit-тесты
не прогонялись (окружение не поднято: rollup native dep / нет pytest-venv).
Полностью исправлено (169): #1336 , #1337 , #1339 , #1340 , #1341 , #1342 , #1343 , #1345 , #1346 , #1348 , #1349 , #1350 , #1351 , #1354 , #1356 , #1358 , #1359 , #1360 , #1362 , #1364 , #1365 , #1366 , #1367 , #1368 , #1369 , #1370 , #1371 , #1372 , #1373 , #1374 , #1375 , #1376 , #1377 , #1378 , #1379 , #1380 , #1381 , #1382 , #1384 , #1385 , #1386 , #1387 , #1388 , #1389 , #1390 , #1391 , #1392 , #1394 , #1395 , #1396 , #1397 , #1399 , #1400 , #1401 , #1402 , #1403 , #1404 , #1408 , #1409 , #1410 , #1411 , #1412 , #1413 , #1414 , #1415 , #1416 , #1417 , #1418 , #1420 , #1423 , #1425 , #1426 , #1427 , #1428 , #1429 , #1430 , #1431 , #1432 , #1433 , #1434 , #1435 , #1437 , #1438 , #1439 , #1440 , #1441 , #1442 , #1443 , #1444 , #1445 , #1446 , #1447 , #1448 , #1449 , #1450 , #1451 , #1452 , #1453 , #1454 , #1455 , #1456 , #1457 , #1458 , #1459 , #1460 , #1461 , #1462 , #1463 , #1464 , #1465 , #1466 , #1467 , #1468 , #1469 , #1471 , #1472 , #1473 , #1474 , #1476 , #1478 , #1479 , #1481 , #1482 , #1483 , #1484 , #1485 , #1487 , #1488 , #1489 , #1490 , #1491 , #1492 , #1493 , #1494 , #1495 , #1496 , #1497 , #1499 , #1500 , #1501 , #1502 , #1504 , #1505 , #1506 , #1507 , #1510 , #1514 , #1515 , #1516 , #1517 , #1518 , #1519 , #1521 , #1522 , #1523 , #1524 , #1525 , #1526 , #1527 , #1528 , #1529 , #1531 , #1532 , #1533 , #1534 , #1535 , #1536 , #1537 , #1538
Частично (9, in-file часть, остаток cross-file): #1361 , #1419 , #1422 , #1424 , #1470 , #1475 , #1477 , #1480 , #1498
Требуют cross-file (3, не тронуты): #1338 , #1363 , #1421
Пропущено (1): #1539
Не входило в партию: 22 needs-Leha issue (нужны решения владельца).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-15 20:21:11 +05:00
480fbba3ca
fix(tradein): cian fetch_newbuilding через BrowserFetcher (camoufox) — initialState под анти-ботом ( #972 )
...
CI / changes (push) Successful in 7s
CI / backend-tests (push) Has been skipped
CI / frontend-tests (push) Has been skipped
CI / openapi-codegen-check (push) Has been skipped
CI / changes (pull_request) Successful in 6s
CI / frontend-tests (pull_request) Has been skipped
CI / openapi-codegen-check (pull_request) Has been skipped
CI / backend-tests (pull_request) Has been skipped
curl_cffi получает zhk-*.cian.ru без initialState; tradein-browser (camoufox) тянет
страницу с 1.17MB initialState. Заменяем транспорт только для page-fetch;
session= param сохранён для backward-compat вызывающих. resolve_cian_zhk_url_via_search
остаётся на curl_cffi SERP. Тесты: 21 passed.
2026-06-15 18:19:32 +03:00
f376d8ce05
fix(tradein-browser): non-fatal launch + background proxy retry (crash-loop 3141x) ( #1330 )
Deploy Trade-In / deploy (push) Successful in 1m18s
Deploy Trade-In / changes (push) Successful in 7s
Deploy Trade-In / test (push) Has been skipped
Deploy Trade-In / build-frontend (push) Has been skipped
Deploy Trade-In / build-backend (push) Has been skipped
Deploy Trade-In / build-browser (push) Successful in 2m2s
2026-06-14 15:18:58 +00:00
4454244ac2
chore(tradein-db): drop duplicate listings_geom_active_idx ( #730 )
...
Deploy Trade-In / changes (push) Successful in 6s
Deploy Trade-In / build-frontend (push) Has been skipped
Deploy Trade-In / build-browser (push) Has been skipped
Deploy Trade-In / test (push) Successful in 33s
Deploy Trade-In / build-backend (push) Successful in 24s
Deploy Trade-In / deploy (push) Successful in 39s
После того как 097 снял предикат WHERE is_active с listings_geom_active_idx,
тот стал точным дублем оригинального full listings_geom_idx (002) — оба
GIST(geom) без предиката. listings_geom_idx покрывает нагрузку (active-запросы
добавляют WHERE is_active поверх того же GIST-скана). Освобождает ~3.2 МБ.
Остальное из аудита #730 (5 dup/redundant дропов + 5 is_active де-партиалов)
уже применено миграцией 097 на prod (verify read-only подтвердил: все ABSENT
/ без предиката). Это follow-up на единственный дубль — побочный эффект 097.
Closes #730
2026-06-13 11:53:13 +00:00
6444dc0ebd
fix(tradein-frontend): остановить retry-storm на 401 — refetchOnMount:false + backoff ( #800 )
...
CI / backend-tests (push) Has been skipped
CI / frontend-tests (push) Has been skipped
CI / changes (pull_request) Successful in 6s
CI / backend-tests (pull_request) Has been skipped
CI / frontend-tests (pull_request) Has been skipped
CI / changes (push) Successful in 7s
Deploy Trade-In / build-backend (push) Blocked by required conditions
Deploy Trade-In / build-frontend (push) Blocked by required conditions
Deploy Trade-In / build-browser (push) Blocked by required conditions
Deploy Trade-In / test (push) Blocked by required conditions
Deploy Trade-In / deploy (push) Blocked by required conditions
Deploy Trade-In / changes (push) Has been cancelled
На стойком 401/протухшей сессии фронт повторял /me + /trade-in/quota в
бесконечном цикле (184+ запросов): errored query + дефолтный
refetchOnMount=true → каждый новый observer (useMe дёргают RouteGuard,
Topbar, UserMenu, useBrand) ре-фетчил при mount = re-subscribe storm.
Глобально выключаем refetchOnMount/onReconnect в QueryClient + экспоненциальный
retryDelay (cap 30s); те же гарды на useQuota. retry-предикат 4xx→false уже был.
Graceful session-screen (RouteGuard/NoAccessScreen) уже на месте.
Closes #800
2026-06-13 16:47:58 +05:00
e1442d3b1d
feat(tradein-scheduler): nightly newbuilding-enrichment schedule ( #973 )
...
CI / changes (pull_request) Successful in 6s
CI / changes (push) Successful in 7s
CI / backend-tests (pull_request) Has been skipped
CI / frontend-tests (pull_request) Has been skipped
CI / backend-tests (push) Has been skipped
CI / frontend-tests (push) Has been skipped
Deploy Trade-In / test (push) Successful in 33s
Deploy Trade-In / build-backend (push) Successful in 1m3s
Deploy Trade-In / changes (push) Successful in 5s
Deploy Trade-In / build-frontend (push) Has been skipped
Deploy Trade-In / build-browser (push) Has been skipped
Deploy Trade-In / deploy (push) Successful in 40s
Регистрирует отдельный in-app scheduler-источник для cian_newbuilding
enrichment-backfill (houses_price_dynamics / house_reliability_checks /
house_reviews), который раньше бежал только инлайн в Cian full-sweep.
- run_newbuilding_enrich() wrapper (heartbeat → backfill → mark_done/failed),
делегирует backfill_newbuilding_enrichment (#972 , уже в main).
- trigger_newbuilding_enrich_run() + dispatch-ветка в scheduler_loop
(зеркалит trigger_sber_index_pull_run): _claim_run guard, async task,
zombie-reap наследуется; idempotency наследуется (skip уже обогащённых,
per-house SAVEPOINT, bounded per-fire limit дренирует backlog 306 домов).
- seed-миграция 103: source='newbuilding_enrich', окно 00:00-01:00 UTC
(03:00-04:00 МСК, до 01:00+ UTC sweep-блока), limit=25, next_run_at на завтра.
Засеяно enabled=FALSE (dormant): на prod весь external-HTTP scraping намеренно
на паузе. Расписание+триггер готовы, но не стартуют до намеренного возобновления
(UPDATE ... SET enabled=true). Не активирую одиночный источник при общей паузе.
Closes #973
2026-06-13 16:24:03 +05:00
d64a57cfcd
Merge pull request 'fix(tradein): guard listing_segment во всех оценочных читателях listings — novostroyki вне comps ( #1186 )' ( #1192 ) from fix/tradein-segment-guards into main
...
Deploy Trade-In / changes (push) Successful in 6s
Deploy Trade-In / build-frontend (push) Has been skipped
Deploy Trade-In / build-browser (push) Has been skipped
Deploy Trade-In / test (push) Successful in 32s
Deploy Trade-In / build-backend (push) Successful in 1m2s
Deploy Trade-In / deploy (push) Successful in 40s
Reviewed-on: #1192
2026-06-12 09:06:28 +00:00
9ffadb9ea6
feat(market): GRANT listings → gendesign_reader — unit-level доступ SF ( #1191 )
...
CI / changes (push) Successful in 6s
CI / backend-tests (push) Has been skipped
CI / frontend-tests (push) Has been skipped
CI / backend-tests (pull_request) Has been skipped
CI / frontend-tests (pull_request) Has been skipped
CI / changes (pull_request) Successful in 6s
- listings + offer_price_history SELECT для gendesign_reader
- fail-fast DO-блок: RAISE EXCEPTION если роль не существует (м.101 не применён)
- НЕ грантятся listing_sources/snapshots/events (внутренний ETL, не нужен SF)
- Контракт: consumer фильтрует vtorichka канон-предикатом сам
Closes #1191
2026-06-12 12:04:41 +03:00
a9a489bba5
refactor(tradein): убрать dead listing_segment param из _fetch_anchor_comps + негативные guard-asserts (review fixup, #1186 )
...
CI / frontend-tests (push) Has been skipped
CI / changes (pull_request) Successful in 6s
CI / backend-tests (pull_request) Has been skipped
CI / frontend-tests (pull_request) Has been skipped
CI / changes (push) Successful in 6s
CI / backend-tests (push) Has been skipped
- Удалён параметр listing_segment: str | None = None из сигнатуры _fetch_anchor_comps
(guard хардкожен в SQL-блоках Tier A/C, параметр не использовался)
- Убраны seg_counts / target_segment в caller-блоке SAME-BUILDING ANCHOR (~L2174)
- Docstring Tier C обновлён: «вторичка-канон guard (#1186 ): NULL = legacy вторичка»
- test_segment_guard_1186.py: добавлены 4 негативных assert'а:
· no bare = 'vtorichka' без IS NULL OR
· no deprecated <> / != 'novostroyki' form
· _fetch_anchor_comps не содержит listing_segment в сигнатуре
· Tier C не использует CAST(:segment AS text) (параметрическую форму)
2026-06-12 12:01:00 +03:00
b70ae46885
fix(tradein): guard listing_segment во всех читателях listings — novostroyki вне comps ( #1186 )
...
Канон-предикат: (listing_segment IS NULL OR listing_segment = 'vtorichka')
NULL = legacy вторичка до миграции 011 (sweep'ы были secondary-only) — отбрасывать нельзя.
Затронуты все SELECT-пути, влияющие на оценку и корректирующие коэффициенты:
- estimator.py: _COMMON_WHERE (Tier S/H), Tier W inline, Tier A anchor (добавлен впервые),
Tier C anchor (заменён с параметрического CAST(:segment) на канон — прежний вариант
отбрасывал NULL-строки при segment='vtorichka').
- asking_to_sold_ratio.py: ask_side, ask_global (_REDERIVE_SQL); bounds (_REDERIVE_BOUNDS_SQL);
ask_tiered, ask_side_all, ask_global (_REDERIVE_TIERED_SQL) — 6 мест.
- sber_index.py: reconciliation-запрос (диагностический asking-median).
Не затронуты: скрейперы/scrape_pipeline, search matview (catalog display, не оценка),
admin stats, geocoding, cian price history, matching — обосновано в инвентаризации.
Тесты: новый test_segment_guard_1186.py (статика SQL + поведение mock); обновлён
test_asking_to_sold_ratio.py (subset-тест нормализует guard, добавленный в refresh но
отсутствующий в 080-seed).
2026-06-12 11:52:09 +03:00
04da30b57a
fix(sf): psql stdin-bootstrap пароля gendesign_reader + COALESCE-protect внешних id (review fixup, #976 )
...
CI / changes (pull_request) Successful in 6s
CI / changes (push) Successful in 7s
CI / frontend-tests (push) Has been skipped
CI / frontend-tests (pull_request) Has been skipped
CI / backend-tests (push) Successful in 6m23s
CI / backend-tests (pull_request) Successful in 6m21s
- Создан tradein-mvp/ops/db-bootstrap/set_gendesign_reader_password.sql:
set_config GUC + format('%L') + \o /dev/null — зеркальный паттерн
ops/db-bootstrap/set_tradein_fdw_password.sql. Пароль не попадает
в stdout/logs (leak-protected).
- deploy-tradein.yml: шаг bootstrap переписан на stdin-redirect
(< ops/db-bootstrap/set_gendesign_reader_password.sql) вместо сломанного
-c "ALTER ROLE ... PASSWORD :'TRADEIN_READER_PASSWORD'", которое давало
syntax error at ':' (psql не интерполирует :'var' внутри -c строки).
- newbuilding_crossload.py ON CONFLICT DO UPDATE: yandex_jk_id и
cian_internal_house_id → COALESCE(EXCLUDED.col, table.col), чтобы
cian-запись не затирала yandex_jk_id NULL'ом и наоборот.
- Тест test_upsert_sql_coalesce_external_ids проверяет наличие COALESCE
в DO UPDATE секции для обоих id.
2026-06-12 11:26:40 +03:00
1a4778d006
feat(sf): cross-load ETL tradein→gendesign newbuilding_listings — вариант A ( #976 )
...
- docker-compose.prod.yml: backend + worker добавлены в сеть gendesign_shared
(нужен прямой TCP к tradein-postgres для psycopg ETL)
- tradein-mvp/data/sql/101_gendesign_reader_role.sql: роль gendesign_reader
(LOGIN, SELECT на houses/house_sources/houses_price_dynamics/
house_reliability_checks/house_reviews; пароль через TRADEIN_READER_PASSWORD
bootstrap в deploy-tradein.yml)
- .forgejo/workflows/deploy-tradein.yml: bootstrap ALTER ROLE gendesign_reader
PASSWORD из env после миграций; warn (не exit 1) если переменная не задана
- backend/app/core/config.py: settings.tradein_database_url (env TRADEIN_DATABASE_URL)
- backend/app/services/etl/newbuilding_crossload.py: run_crossload() — psycopg
server-side cursor, batch 500, UPSERT ON CONFLICT (source, ext_house_id),
SAVEPOINT per-row; disabled-режим если url пуст
- backend/app/workers/tasks/etl_newbuilding_crossload.py: Celery task
- backend/app/workers/beat_schedule.py: nightly 03:30 МСК (00:30 UTC)
- backend/app/api/v1/admin_scrape.py: POST /admin/scrape/newbuilding-crossload
- backend/tests/services/test_newbuilding_crossload.py: 10 тестов без реальной БД
needs-human: TRADEIN_READER_PASSWORD → tradein .env.runtime;
TRADEIN_DATABASE_URL → gendesign backend/.env.runtime
2026-06-12 11:18:47 +03:00
07b06617fd
test(tradein): убрать мёртвый monkeypatch в test_scheduler_main (review fixup)
CI / changes (push) Successful in 6s
CI / changes (pull_request) Successful in 5s
CI / backend-tests (push) Has been skipped
CI / frontend-tests (push) Has been skipped
CI / backend-tests (pull_request) Has been skipped
CI / frontend-tests (pull_request) Has been skipped
2026-06-12 10:56:27 +03:00
72a4c122c4
feat(tradein): отдельный scraper-контейнер — scheduler вне API-процесса ( #1182 )
...
Причина: docker restart tradein-backend (при каждом API-деплое) убивал бегущие
sweep'ы avito/cian/rosreestr — они могут идти часами, и их прерывание середине
батча создавало zombie-runs и пропуски данных.
Что сделано:
- scheduler_main.py — standalone entrypoint для tradein-scraper; SIGTERM/SIGINT
отменяют scheduler_loop чисто; GlitchTip init без Starlette/FastAPI интеграций.
- docker-compose.prod.yml — новый сервис scraper (тот же image, команда python -m
app.scheduler_main, только tradein-net); SCHEDULER_ENABLE=false в backend.
- deploy-tradein.yml — селективный up -d --no-deps: scraper пересоздаётся только
при изменениях scraper/infra путей (SCRAPER_CHANGED), не при каждом backend-деплое.
- tests/test_scheduler_main.py — импорт, kill-switch _should_run(), чистая отмена _run().
2026-06-12 10:50:50 +03:00
86828d0388
feat(rbac): add analyst role + §19 audit-log middleware ( #962 )
...
CI / changes (push) Successful in 6s
CI / frontend-tests (push) Has been skipped
CI / changes (pull_request) Successful in 6s
Deploy Trade-In / changes (push) Successful in 7s
Deploy / changes (push) Successful in 6s
CI / frontend-tests (pull_request) Has been skipped
Deploy Trade-In / build-frontend (push) Has been skipped
Deploy Trade-In / build-browser (push) Has been skipped
Deploy Trade-In / test (push) Successful in 36s
Deploy / build-backend (push) Successful in 1m39s
Deploy / build-frontend (push) Has been skipped
Deploy Trade-In / build-backend (push) Successful in 48s
Deploy / build-worker (push) Successful in 2m41s
Deploy / deploy (push) Failing after 4s
Deploy Trade-In / deploy (push) Successful in 47s
CI / backend-tests (push) Successful in 6m38s
CI / backend-tests (pull_request) Successful in 6m30s
EPIC18/§19. analyst sees everything (deals, insights, exports, site-finder,
analytics, concept) EXCEPT admin/data-management. Enforcement is backend-hard
(the existing rbac_guard already 403s any non-admin role on /api/v1/admin/*, so
adding the role auto-blocks it) + frontend (deny_paths via /me) + audit.
§19 audit: new best-effort HTTP middleware logs the sensitive actions
(analyze / forecast / forecast-export) to a new audit_log table after the
response. Audit failures never break or delay-fail the request (2-layer
try/except + finally close). Registered INNER to rbac_guard so only authorized
requests are audited (a 403 short-circuits before audit). classify_path matches
export before forecast (anchored).
- auth/roles.yaml: analyst role (paths /**, deny admin-mgmt) + analysttest QA user
- core/auth.py (+ tradein mirror): Role Literal += analyst
- core/audit_middleware.py (new) + main.py registration
- data/sql/144_audit_log.sql (idempotent; auto-applies)
- tests: analyst rbac (403 admin / 200 parcels) + 11 audit cases
No data-level ACL (analyst sees data per policy) -> no #948 dependency. No new
deps. parcels.py untouched. Real analyst logins still need adding to
caddy/users.caddy.snippet (devops).
Refs #962 .
2026-06-08 12:16:19 +05:00
b0fe292a63
fix(tradein): working cian ЖК-url resolver via cat.php SERP ( #972 )
...
CI / changes (push) Successful in 6s
CI / backend-tests (push) Has been skipped
CI / frontend-tests (push) Has been skipped
CI / changes (pull_request) Successful in 5s
Deploy Trade-In / test (push) Successful in 31s
Deploy Trade-In / build-backend (push) Successful in 1m20s
CI / backend-tests (pull_request) Has been skipped
CI / frontend-tests (pull_request) Has been skipped
Deploy Trade-In / changes (push) Successful in 5s
Deploy Trade-In / build-frontend (push) Has been skipped
Deploy Trade-In / build-browser (push) Has been skipped
Deploy Trade-In / deploy (push) Successful in 46s
The legacy resolve_cian_zhk_url hit /zhk/<id>/ which now 404s, leaving the
318 geo-matched cian houses (house_sources.ext_id set, cian_zhk_url NULL)
unfetchable -> the newbuilding-enrichment backfill matched 0 houses.
Add resolve_cian_zhk_url_via_search(nb_id): fetch the cat.php newbuilding SERP
and extract the canonical zhk-<slug>.cian.ru url, ANCHORED on the
<h1 data-name="Title"> header anchor (NOT naive first-match — the SERP carries
promo/recommendation zhk-* links before the title that would otherwise resolve
the wrong ЖК and silently corrupt enrichment). Validated against the real
2.81MB prod SERP + an adversarial poisoned-recommendation test.
Wire into newbuilding_enrich_backfill: broaden selection to "has ext_id OR
cian_zhk_url", resolve+persist the url under a SAVEPOINT before enriching,
rate-limited + resumable + idempotent. Keep the old resolver (deprecated).
Bounded prod proof (5 houses, direct): 5/5 urls resolved+persisted, 3/5 fully
enriched (+18 price_dynamics, +3 reliability); the 2 misses were direct-mode
anti-bot on the 2nd fetch. Full 318-run gated on the cian mobile proxy
(mproxy.site) being restored. code-reviewer APPROVE (SQL/idempotency) +
resolver hardened against wrong-ЖК. 19 tests green.
Refs #972 .
2026-06-08 11:13:06 +05:00
bcb1285341
feat(tradein): backfill task for cian newbuilding enrichment ( #972 )
...
Deploy Trade-In / changes (push) Successful in 7s
Deploy Trade-In / build-frontend (push) Has been skipped
Deploy Trade-In / build-browser (push) Has been skipped
Deploy Trade-In / deploy (push) Successful in 46s
Deploy Trade-In / test (push) Successful in 37s
Deploy Trade-In / build-backend (push) Successful in 45s
950-E1: app/tasks/newbuilding_enrich_backfill.py selects houses linked to
ext_source='cian_newbuilding' with a fetchable cian_zhk_url and runs the existing
fetch_newbuilding + save_newbuilding_enrichment over each, populating
houses_price_dynamics + house_reliability_checks (+ house_reviews when present).
- Idempotent + resumable: skips already-enriched houses; force= re-run dup-safe
(price_dynamics UPSERT via dim_key, reliability dedup guard with id-DESC
tiebreaker, reviews ON CONFLICT (source,ext_review_id)).
- Anti-bot aware: get_scraper_delay('cian') + jitter, SAVEPOINT-per-house
(one captcha/failure logs + continues, never aborts the batch).
- limit= for bounded proof runs; sizing counters (total/fetchable/pending).
Fix cian_newbuilding._extract_transport_rate: cian transportAccessibilityRate
drifted to a nested dict and broke save_newbuilding_enrichment's CAST bind on
LIVE data (cannot adapt type 'dict') — coerce dict/float/bad -> int|None
(bool excluded before int). Fixes the live enrichment crash, not just backfill.
house_reviews stays ~0 for now: cian reviews are in a separate XHR, not the ЖК
initialState — extending fetch_newbuilding for them is a documented follow-up.
Tested: 8 new unit tests; isolated-DB proof-run landed 6 price_dynamics + 1
reliability row, idempotency proven across re-runs. Refs #972 .
2026-06-07 14:09:24 +00:00
ea5c1e29f6
test(tradein): quality-gate anti-regression тесты на 5 P0-фиксов ( #1098 )
...
Deploy Trade-In / changes (push) Successful in 6s
Deploy Trade-In / build-frontend (push) Has been skipped
Deploy Trade-In / build-browser (push) Has been skipped
Deploy Trade-In / test (push) Successful in 31s
Deploy Trade-In / build-backend (push) Successful in 24s
Deploy Trade-In / deploy (push) Successful in 35s
13 anti-regression тестов в test_781_quality_gate.py на 5 merged P0-фиксов (#755 anchor safe-guard, #753 stable dedup_hash, #773 expected_sold anchor-only, #754 avito block-page HTTP-200, #740 insufficient_data). Tests-only, прод-код не тронут. #755-scope verify: radius _filter_outliers n<5 bypass корректно не менялся (MAD-clip только в anchor). 325 passed.
Closes #781 .
Co-authored-by: lekss361 <lekss361@gendsgn.local>
Co-committed-by: lekss361 <lekss361@gendsgn.local>
2026-06-06 19:27:23 +00:00
e792bed325
fix(tradein): white-label без logo не показывает «Мера»-глиф ( #1097 )
...
Deploy Trade-In / test (push) Has been skipped
Deploy Trade-In / build-browser (push) Has been skipped
Deploy Trade-In / build-frontend (push) Successful in 1m42s
Deploy Trade-In / deploy (push) Successful in 38s
Deploy Trade-In / changes (push) Successful in 6s
Deploy Trade-In / build-backend (push) Has been skipped
Topbar white-label-no-logo fallback: MeraMark → первая буква имени клиента (нейтральный плейсхолдер) — бренд не утекает в клиентскую шапку. MeraMark остаётся только в дефолтной ветке. OfferCard: intentional-комментарии у «Trade-In» (механика сделки). tsc 0 + build green.
Closes #1081 .
Co-authored-by: lekss361 <lekss361@gendsgn.local>
Co-committed-by: lekss361 <lekss361@gendsgn.local>
2026-06-06 19:09:13 +00:00
8815239e02
fix(tradein): enable deactivate_stale_avito schedule on prod ( #1096 )
...
Deploy Trade-In / build-frontend (push) Has been skipped
Deploy Trade-In / build-browser (push) Has been skipped
Deploy Trade-In / test (push) Successful in 33s
Deploy Trade-In / deploy (push) Blocked by required conditions
Deploy Trade-In / changes (push) Successful in 6s
Deploy Trade-In / build-backend (push) Has been cancelled
Миграция 100_enable_deactivate_stale_avito.sql — идемпотентный UPDATE scrape_schedules SET enabled=true, next_run_at=NOW() WHERE source='deactivate_stale_avito' AND enabled=false. Чинит QA-находку #759 : 090 ON CONFLICT DO NOTHING оставил pre-seed disabled-строку → task ни разу не отработала. Non-destructive, guard на повторный прогон. +7 тестов.
Closes #759 .
Co-authored-by: lekss361 <lekss361@gendsgn.local>
Co-committed-by: lekss361 <lekss361@gendsgn.local>
2026-06-06 19:08:22 +00:00
813c4782a8
feat(tradein): rebrand default product «Trade-In» → «Мера» + new mark ( #1079 )
...
Deploy Trade-In / changes (push) Successful in 6s
Deploy Trade-In / test (push) Has been skipped
Deploy Trade-In / build-browser (push) Has been skipped
Deploy Trade-In / build-backend (push) Has been skipped
Deploy Trade-In / build-frontend (push) Successful in 1m40s
Deploy Trade-In / deploy (push) Successful in 44s
Ребрендинг дефолтного (не white-label) продукта «Trade-In» → «Мера» + inline-SVG логотип (буква М над размерной линией). Topbar MeraMark заменяет TI-бейдж, wordmark/крошки/футер/meta → «Мера». White-label бренды клиентов, сервисный термин «трейд-ин» в прозе OfferCard/HeroTransparency и code-идентификаторы /trade-in/ не тронуты. tsc 0 + build green.
Co-authored-by: lekss361 <lekss361@gendsgn.local>
Co-committed-by: lekss361 <lekss361@gendsgn.local>
2026-06-06 18:11:35 +00:00
702f322b22
fix(tradein): collapse CRM block + density so form fits laptops
...
Форма ввода (10 полей + CRM-блок 221px) не помещалась в .form-card
(max-height 100vh-104px) на ноутбуках: переполнение 58px @1440×900,
~158px @13" MacBook. CRM «для менеджера» теперь сворачиваемый (по умолчанию
закрыт, открывается по клику) + лёгкое уплотнение отступов (.field 12→9,
form-body/foot padding). Высота формы 854→646px → влезает без скролла на
1366×768 / 1440×900 / 13" MacBook @100%. Sub-720 — fallback на скролл-фикс
из этой же ветки. Проверено вживую (injection) на проде, tsc --noEmit clean.
2026-06-01 10:58:43 +03:00
ed15559f80
fix(tradein): praktika wordmark logo + form button cutoff on short viewports
...
Logo «поехало»: brands.logo_url хотлинкал квадратный знак logo-pink.svg
(viewBox 69×69, preserveAspectRatio=none, без width/height) → CSS height:24/width:auto
рендерил его 300×24 расплющенным. Self-host полного wordmark (символ #i-logo-new,
viewBox 1498×194) в /trade-in/brands/praktika.svg; миграция 099 перенаправляет
brands.logo_url; Topbar рендерит только wordmark (убран дублирующий текст-название);
.brand-logo защищён object-fit:contain + max-width.
Кнопка «Оценить» обрезалась @100% на 13" MacBook: .form-card max-height+overflow:hidden
выдавливал .form-foot когда flex-shrink тела не срабатывал. Теперь скроллом владеет
карточка (overflow-y:auto), body overflow-y:visible, foot flex-shrink:0.
Проверено вживую на проде 1280×720.
2026-06-01 10:15:49 +03:00
a23e650fae
fix(tradein): system-curl subprocess transport for Yandex SERP over SOCKS5 ( #940 )
...
Deploy Trade-In / build-frontend (push) Has been skipped
Deploy Trade-In / changes (push) Successful in 5s
Deploy Trade-In / build-browser (push) Has been skipped
Deploy Trade-In / test (push) Successful in 30s
Deploy Trade-In / build-backend (push) Successful in 44s
Deploy Trade-In / deploy (push) Successful in 36s
Co-authored-by: lekss361 <lekss361@gendsgn.local>
Co-committed-by: lekss361 <lekss361@gendsgn.local>
2026-05-31 22:39:06 +00:00
921c14e054
feat(trade-in): OfferCard — убрать редактируемую цену + ползунок торга, разбивка на дефолтах ( #939 )
...
Deploy Trade-In / changes (push) Successful in 5s
Deploy Trade-In / test (push) Has been skipped
Deploy Trade-In / build-browser (push) Has been skipped
Deploy Trade-In / build-backend (push) Has been skipped
Deploy Trade-In / build-frontend (push) Successful in 1m34s
Deploy Trade-In / deploy (push) Successful in 37s
Co-authored-by: bot-backend <bot-backend@gendsgn.local>
Co-committed-by: bot-backend <bot-backend@gendsgn.local>
2026-05-31 22:31:20 +00:00
567549c3aa
feat(tradein): cian full-load secondary-only — drop novostroyki at parse ( #938 )
...
Deploy Trade-In / build-backend (push) Successful in 40s
Deploy Trade-In / deploy (push) Successful in 35s
Deploy Trade-In / changes (push) Successful in 5s
Deploy Trade-In / build-frontend (push) Has been skipped
Deploy Trade-In / build-browser (push) Has been skipped
Deploy Trade-In / test (push) Successful in 30s
Co-authored-by: bot-backend <bot-backend@gendsgn.local>
Co-committed-by: bot-backend <bot-backend@gendsgn.local>
2026-05-31 21:35:32 +00:00
d93e96fb0c
feat(tradein): checkpoint/resume for yandex full-load — survive restart ( #930 ) ( #937 )
...
Deploy Trade-In / build-browser (push) Has been skipped
Deploy Trade-In / build-backend (push) Has been cancelled
Deploy Trade-In / deploy (push) Blocked by required conditions
Deploy Trade-In / changes (push) Successful in 6s
Deploy Trade-In / build-frontend (push) Has been skipped
Deploy Trade-In / test (push) Successful in 30s
Co-authored-by: lekss361 <lekss361@gendsgn.local>
Co-committed-by: lekss361 <lekss361@gendsgn.local>
2026-05-31 21:34:23 +00:00
3294bc2021
fix(tradein): exclude novostroyki from radius-median comp pool ( #935 )
...
Deploy Trade-In / test (push) Successful in 31s
Deploy Trade-In / deploy (push) Blocked by required conditions
Deploy Trade-In / changes (push) Successful in 5s
Deploy Trade-In / build-frontend (push) Has been skipped
Deploy Trade-In / build-browser (push) Has been skipped
Deploy Trade-In / build-backend (push) Has been cancelled
Co-authored-by: bot-backend <bot-backend@gendsgn.local>
Co-committed-by: bot-backend <bot-backend@gendsgn.local>
2026-05-31 21:33:23 +00:00
571980c56e
feat(tradein): ppm²-tier segmentation of asking→sold ratio ( #928 , flag OFF) ( #934 )
...
Deploy Trade-In / changes (push) Successful in 5s
Deploy Trade-In / build-frontend (push) Has been skipped
Deploy Trade-In / build-browser (push) Has been skipped
Deploy Trade-In / test (push) Successful in 31s
Deploy Trade-In / build-backend (push) Successful in 44s
Deploy Trade-In / deploy (push) Successful in 36s
Co-authored-by: lekss361 <lekss361@gendsgn.local>
Co-committed-by: lekss361 <lekss361@gendsgn.local>
2026-05-31 21:22:39 +00:00
846c5014c6
feat(tradein): checkpoint/resume for cian full-load — survive restart ( #930 ) ( #932 )
...
Deploy Trade-In / test (push) Successful in 31s
Deploy Trade-In / changes (push) Successful in 5s
Deploy Trade-In / build-frontend (push) Has been skipped
Deploy Trade-In / build-browser (push) Has been skipped
Deploy Trade-In / build-backend (push) Successful in 44s
Deploy Trade-In / deploy (push) Successful in 38s
Co-authored-by: bot-backend <bot-backend@gendsgn.local>
Co-committed-by: bot-backend <bot-backend@gendsgn.local>
2026-05-31 21:11:48 +00:00
2bcccdf619
feat(tradein): exhaustive Yandex realty load — adaptive room×price + concurrency ( #931 )
...
Deploy Trade-In / deploy (push) Blocked by required conditions
Deploy Trade-In / changes (push) Successful in 6s
Deploy Trade-In / build-frontend (push) Has been skipped
Deploy Trade-In / build-browser (push) Has been skipped
Deploy Trade-In / test (push) Successful in 31s
Deploy Trade-In / build-backend (push) Has been cancelled
Co-authored-by: lekss361 <lekss361@gendsgn.local>
Co-committed-by: lekss361 <lekss361@gendsgn.local>
2026-05-31 21:10:26 +00:00
5e94d558b5
perf(tradein): cian full-load incremental save + concurrent pagination ( #929 )
...
Deploy Trade-In / changes (push) Successful in 5s
Deploy Trade-In / build-frontend (push) Has been skipped
Deploy Trade-In / build-browser (push) Has been skipped
Deploy Trade-In / test (push) Successful in 31s
Deploy Trade-In / build-backend (push) Successful in 43s
Deploy Trade-In / deploy (push) Successful in 36s
Co-authored-by: bot-backend <bot-backend@gendsgn.local>
Co-committed-by: bot-backend <bot-backend@gendsgn.local>
2026-05-31 20:27:27 +00:00
ecf8add2ce
fix(browser): pageError.location needs valid defaults, not just optional chaining ( #918 follow-up) ( #927 )
...
Deploy Trade-In / changes (push) Successful in 5s
Deploy Trade-In / test (push) Has been skipped
Deploy Trade-In / build-frontend (push) Has been skipped
Deploy Trade-In / build-backend (push) Has been skipped
Deploy Trade-In / deploy (push) Successful in 1m6s
Deploy Trade-In / build-browser (push) Successful in 2m0s
Co-authored-by: bot-backend <bot-backend@gendsgn.local>
Co-committed-by: bot-backend <bot-backend@gendsgn.local>
2026-05-31 20:15:39 +00:00
0883f6d0f1
feat(tradein): exhaustive Cian EKB secondary load — room×price partitioning ( #926 )
...
Deploy Trade-In / changes (push) Successful in 5s
Deploy Trade-In / build-frontend (push) Has been skipped
Deploy Trade-In / build-browser (push) Has been skipped
Deploy Trade-In / test (push) Successful in 30s
Deploy Trade-In / build-backend (push) Successful in 43s
Deploy Trade-In / deploy (push) Successful in 37s
Co-authored-by: bot-backend <bot-backend@gendsgn.local>
Co-committed-by: bot-backend <bot-backend@gendsgn.local>
2026-05-31 19:54:19 +00:00
26c0b94131
Merge pull request 'fix(tradein): add cian_proxy_url to test _mock_settings — unblock deploy ( #923 )' ( #925 ) from fix/cian-proxy-test-mock into main
...
Deploy Trade-In / changes (push) Successful in 5s
Deploy Trade-In / build-frontend (push) Has been skipped
Deploy Trade-In / build-browser (push) Has been skipped
Deploy Trade-In / test (push) Successful in 30s
Deploy Trade-In / build-backend (push) Successful in 45s
Deploy Trade-In / deploy (push) Successful in 37s
Reviewed-on: #925
2026-05-31 19:34:41 +00:00
b9690d4d63
fix(tradein): add cian_proxy_url to test _mock_settings — unblock deploy ( #923 )
...
PR #923 switched Cian scrapers to settings.cian_proxy_url but the shared
_mock_settings helper in test_scraper_proxy.py still only exposed scraper_proxy_url,
so pre-existing cian proxy-wiring tests broke with AttributeError → deploy pytest gate failed.
Mirror scraper_proxy_url into cian_proxy_url in the mock.
2026-05-31 22:34:16 +03:00
ca0943689d
fix(tradein): skip TLS verify for sberindex.ru pull — server omits LE intermediate ( #922 )
2026-05-31 22:33:42 +03:00
02af421799
feat(tradein): dedicated Cian mobile proxy — separate egress from Avito ( #923 )
...
Deploy Trade-In / changes (push) Successful in 6s
Deploy Trade-In / build-frontend (push) Has been skipped
Deploy Trade-In / build-browser (push) Has been skipped
Deploy Trade-In / test (push) Failing after 30s
Deploy Trade-In / build-backend (push) Has been skipped
Deploy Trade-In / deploy (push) Has been skipped
Co-authored-by: bot-backend <bot-backend@gendsgn.local>
Co-committed-by: bot-backend <bot-backend@gendsgn.local>
2026-05-31 19:30:14 +00:00
3bd107f923
test(tradein): fix #794 test regressions on main (broken by #919 early squash-merge) ( #921 )
...
Deploy Trade-In / changes (push) Successful in 5s
Deploy Trade-In / build-frontend (push) Has been skipped
Deploy Trade-In / build-browser (push) Has been skipped
Deploy Trade-In / test (push) Successful in 33s
Deploy Trade-In / build-backend (push) Successful in 45s
Deploy Trade-In / deploy (push) Successful in 45s
Co-authored-by: lekss361 <lekss361@gendsgn.local>
Co-committed-by: lekss361 <lekss361@gendsgn.local>
2026-05-31 19:05:10 +00:00
4999c877ae
feat(tradein): СберИндекс time-adjust for frozen Rosreestr ДКП + per-dashboard filter fix ( #794 ) ( #919 )
...
Deploy Trade-In / build-frontend (push) Has been skipped
Deploy Trade-In / build-backend (push) Has been skipped
Deploy Trade-In / deploy (push) Has been skipped
Deploy Trade-In / changes (push) Successful in 5s
Deploy Trade-In / build-browser (push) Has been skipped
Deploy Trade-In / test (push) Failing after 33s
Co-authored-by: lekss361 <lekss361@gendsgn.local>
Co-committed-by: lekss361 <lekss361@gendsgn.local>
2026-05-31 18:48:22 +00:00
19cf1162a0
fix(browser): null-guard pageError.location in playwright bundle ( #915 ) ( #918 )
...
Deploy Trade-In / changes (push) Successful in 5s
Deploy Trade-In / test (push) Has been skipped
Deploy Trade-In / build-frontend (push) Has been skipped
Deploy Trade-In / build-backend (push) Has been skipped
Deploy Trade-In / build-browser (push) Successful in 2m22s
Deploy Trade-In / deploy (push) Has been cancelled
Co-authored-by: bot-backend <bot-backend@gendsgn.local>
Co-committed-by: bot-backend <bot-backend@gendsgn.local>
2026-05-31 18:45:41 +00:00