Commit graph

14 commits

Author SHA1 Message Date
febc88d37b fix(ci): deploy-tradein paths-filter base = last deployed SHA (накопленный diff, fail-safe build-all)
All checks were successful
CI / changes (pull_request) Successful in 5s
CI / backend-tests (pull_request) Has been skipped
CI / frontend-tests (pull_request) Has been skipped
CI / openapi-codegen-check (pull_request) Has been skipped
При быстрых мержах dorny/paths-filter без explicit base диффает per-merge-commit
(parent of merge), а не накопленно от последнего деплоя. Итог: backend-мерж #1829
не попадает в diff frontend-мержа #1830 → build-backend skipped, образа нет.

Решение (host-file mechanism):
- deploy-джоба пишет `echo "$GITHUB_SHA" > /opt/gendesign/.tradein-deployed-sha`
  в самом конце SSH-скрипта — только после успешного up+миграций.
- changes-джоба читает файл по SSH (те же DEPLOY_* секреты), валидирует SHA как
  40-char hex + git merge-base --is-ancestor HEAD, передаёт в `base:` paths-filter.
- Если файл отсутствует / SHA не предок HEAD / SSH недоступен → FAIL-SAFE:
  step `set-all` эмитит все outputs=true, paths-filter пропускается,
  все образы собираются (better safe than sorry).

Почему host-file, не git-ref: в workflows не используется `gitea.token` /
`GITHUB_TOKEN` с contents:write — добавлять новое разрешение ради ref push
ненужный риск. DEPLOY_* секреты уже есть, SSH уже есть.

Не изменено: concurrency.cancel-in-progress=false, workflow_dispatch форс,
логика build/deploy/migrations.
2026-06-20 16:05:19 +03:00
4811920d28 fix(tradein): yandex combos watchdog timeout (P0) + deploy scraper filter gap (#1659)
All checks were successful
Deploy Trade-In / changes (push) Successful in 7s
Deploy Trade-In / build-browser (push) Successful in 28s
Deploy Trade-In / test (push) Successful in 34s
Deploy Trade-In / build-frontend (push) Successful in 1m46s
Deploy Trade-In / deploy (push) Successful in 46s
Deploy Trade-In / build-backend (push) Successful in 53s
Co-authored-by: lekss361 <lekss361@gendsgn.local>
Co-committed-by: lekss361 <lekss361@gendsgn.local>
2026-06-16 20:22:27 +00:00
04da30b57a fix(sf): psql stdin-bootstrap пароля gendesign_reader + COALESCE-protect внешних id (review fixup, #976)
All checks were successful
CI / changes (pull_request) Successful in 6s
CI / changes (push) Successful in 7s
CI / frontend-tests (push) Has been skipped
CI / frontend-tests (pull_request) Has been skipped
CI / backend-tests (push) Successful in 6m23s
CI / backend-tests (pull_request) Successful in 6m21s
- Создан tradein-mvp/ops/db-bootstrap/set_gendesign_reader_password.sql:
  set_config GUC + format('%L') + \o /dev/null — зеркальный паттерн
  ops/db-bootstrap/set_tradein_fdw_password.sql. Пароль не попадает
  в stdout/logs (leak-protected).
- deploy-tradein.yml: шаг bootstrap переписан на stdin-redirect
  (< ops/db-bootstrap/set_gendesign_reader_password.sql) вместо сломанного
  -c "ALTER ROLE ... PASSWORD :'TRADEIN_READER_PASSWORD'", которое давало
  syntax error at ':' (psql не интерполирует :'var' внутри -c строки).
- newbuilding_crossload.py ON CONFLICT DO UPDATE: yandex_jk_id и
  cian_internal_house_id → COALESCE(EXCLUDED.col, table.col), чтобы
  cian-запись не затирала yandex_jk_id NULL'ом и наоборот.
- Тест test_upsert_sql_coalesce_external_ids проверяет наличие COALESCE
  в DO UPDATE секции для обоих id.
2026-06-12 11:26:40 +03:00
1a4778d006 feat(sf): cross-load ETL tradein→gendesign newbuilding_listings — вариант A (#976)
- docker-compose.prod.yml: backend + worker добавлены в сеть gendesign_shared
  (нужен прямой TCP к tradein-postgres для psycopg ETL)
- tradein-mvp/data/sql/101_gendesign_reader_role.sql: роль gendesign_reader
  (LOGIN, SELECT на houses/house_sources/houses_price_dynamics/
   house_reliability_checks/house_reviews; пароль через TRADEIN_READER_PASSWORD
   bootstrap в deploy-tradein.yml)
- .forgejo/workflows/deploy-tradein.yml: bootstrap ALTER ROLE gendesign_reader
  PASSWORD из env после миграций; warn (не exit 1) если переменная не задана
- backend/app/core/config.py: settings.tradein_database_url (env TRADEIN_DATABASE_URL)
- backend/app/services/etl/newbuilding_crossload.py: run_crossload() — psycopg
  server-side cursor, batch 500, UPSERT ON CONFLICT (source, ext_house_id),
  SAVEPOINT per-row; disabled-режим если url пуст
- backend/app/workers/tasks/etl_newbuilding_crossload.py: Celery task
- backend/app/workers/beat_schedule.py: nightly 03:30 МСК (00:30 UTC)
- backend/app/api/v1/admin_scrape.py: POST /admin/scrape/newbuilding-crossload
- backend/tests/services/test_newbuilding_crossload.py: 10 тестов без реальной БД

needs-human: TRADEIN_READER_PASSWORD → tradein .env.runtime;
             TRADEIN_DATABASE_URL → gendesign backend/.env.runtime
2026-06-12 11:18:47 +03:00
72a4c122c4 feat(tradein): отдельный scraper-контейнер — scheduler вне API-процесса (#1182)
Причина: docker restart tradein-backend (при каждом API-деплое) убивал бегущие
sweep'ы avito/cian/rosreestr — они могут идти часами, и их прерывание середине
батча создавало zombie-runs и пропуски данных.

Что сделано:
- scheduler_main.py — standalone entrypoint для tradein-scraper; SIGTERM/SIGINT
  отменяют scheduler_loop чисто; GlitchTip init без Starlette/FastAPI интеграций.
- docker-compose.prod.yml — новый сервис scraper (тот же image, команда python -m
  app.scheduler_main, только tradein-net); SCHEDULER_ENABLE=false в backend.
- deploy-tradein.yml — селективный up -d --no-deps: scraper пересоздаётся только
  при изменениях scraper/infra путей (SCRAPER_CHANGED), не при каждом backend-деплое.
- tests/test_scheduler_main.py — импорт, kill-switch _should_run(), чистая отмена _run().
2026-06-12 10:50:50 +03:00
0566fed5a2 ci(tradein): build+push tradein-browser image in deploy workflow (#905)
The new browser service references ghcr.io/lekss361/gendesign-tradein-browser;
without a build job the deploy's compose-up would fail pulling a missing image.
Adds build-browser job (mirrors build-backend, context ./tradein-mvp/browser),
a browser path-filter (tradein-mvp/browser/**), IMAGE_BROWSER env, and wires
build-browser into deploy needs + the no-failure gate.

Refs #905
2026-05-31 17:29:13 +03:00
ae1a35140c ci(tradein): pass NEXT_PUBLIC_ENABLE_PREVIEW=1 to frontend build (#801) 2026-05-30 18:40:22 +00:00
c30b3c714e fix(ci): drop --frozen in tradein test job (uv.lock is gitignored) (#678)
All checks were successful
Deploy Trade-In / build-backend (push) Successful in 40s
Deploy Trade-In / changes (push) Successful in 4s
Deploy Trade-In / test (push) Successful in 23s
Deploy Trade-In / build-frontend (push) Successful in 24s
Deploy Trade-In / deploy (push) Successful in 34s
Co-authored-by: bot-backend <bot-backend@gendsgn.local>
Co-committed-by: bot-backend <bot-backend@gendsgn.local>
2026-05-29 21:23:56 +00:00
22f1a9c1fc fix(ci): install uv via official installer in tradein test job (PEP 668) (#676)
Some checks failed
Deploy Trade-In / changes (push) Successful in 4s
Deploy Trade-In / test (push) Failing after 6s
Deploy Trade-In / build-backend (push) Has been skipped
Deploy Trade-In / build-frontend (push) Successful in 23s
Deploy Trade-In / deploy (push) Has been skipped
Co-authored-by: bot-backend <bot-backend@gendsgn.local>
Co-committed-by: bot-backend <bot-backend@gendsgn.local>
2026-05-29 20:45:12 +00:00
be63b249b5 chore(tradein): close pre-commit ruff gap + add CI pytest gate (#666) (#675)
Some checks failed
Deploy Trade-In / changes (push) Successful in 4s
Deploy Trade-In / test (push) Failing after 4s
Deploy Trade-In / build-backend (push) Has been skipped
Deploy Trade-In / build-frontend (push) Successful in 25s
Deploy Trade-In / deploy (push) Has been skipped
Co-authored-by: bot-backend <bot-backend@gendsgn.local>
Co-committed-by: bot-backend <bot-backend@gendsgn.local>
2026-05-29 19:18:48 +00:00
40002cfcd2 fix(deploy-tradein): track SQL migrations with _schema_migrations + baseline (#637)
All checks were successful
Deploy Trade-In / changes (push) Successful in 5s
Deploy Trade-In / build-backend (push) Successful in 26s
Deploy Trade-In / build-frontend (push) Successful in 31s
Deploy Trade-In / deploy (push) Successful in 44s
2026-05-29 06:34:01 +00:00
ed49f6b953 fix(deploy): resolve FDW bootstrap race + extract bootstrap DO block to .sql file (#499)
Some checks failed
Deploy Trade-In / changes (push) Successful in 6s
Deploy / changes (push) Successful in 5s
Deploy Trade-In / build-backend (push) Successful in 24s
Deploy Trade-In / build-frontend (push) Successful in 23s
Deploy / build-backend (push) Successful in 26s
Deploy / build-worker (push) Successful in 25s
Deploy / build-frontend (push) Successful in 25s
Deploy Trade-In / deploy (push) Successful in 33s
Deploy / deploy (push) Failing after 22s
2026-05-24 09:46:00 +00:00
TradeIn Deploy
ee43248b11 fix(tradein): HOSTNAME=0.0.0.0 + NEXT_PUBLIC_API_BASE_URL build-arg
502 fix: Next.js standalone server.js биндился на HOSTNAME=<container-id>
вместо 0.0.0.0 → Caddy не мог достучаться до tradein-frontend:3000.
Теперь HOSTNAME=0.0.0.0 и в Dockerfile (runner ENV), и в compose.

API routing fix: фронт собирался без NEXT_PUBLIC_API_BASE_URL, fetch шёл
на same-origin /api/* (главный gendesign-backend) вместо /trade-in/api/*.
Добавлен build-arg в Dockerfile + workflow.
2026-05-21 10:21:32 +03:00
lekss361
02267d0306 feat: add tradein-mvp subproject (Trade-In Estimator под /trade-in)
- FastAPI backend: PostGIS estimator + 3 scrapers (Avito/Cian/Yandex)
- Next.js 15 frontend: tradein.html mockup design, basePath=/trade-in
- WeasyPrint PDF (Брусника-style 4-page report)
- Address autocomplete с typo-tolerance + 6 EKB presets
- Изолированный docker stack gendesign-tradein (отдельная postgres БД)
- Caddy inline routes: gendsgn.ru/trade-in/* и /trade-in/api/v1/*
- Forgejo Actions: .forgejo/workflows/deploy-tradein.yml (shell-based GHCR login)
- Триггер только по paths: tradein-mvp/** (не пересекается с deploy.yml)
- Образы: ghcr.io/lekss361/gendesign-tradein-{backend,frontend}:latest

Первый запуск на сервере (вручную, один раз):
  - создать /opt/gendesign/tradein-mvp/.env.runtime (postgres pwd, contact email)
  - docker network create gendesign_shared (если нет)
  - docker compose -p gendesign-tradein up -d
  - docker compose -p gendesign exec caddy caddy reload
2026-05-21 00:25:39 +03:00