gendesign/docker-compose.prod.yml
Light1YT 26f9605c06
All checks were successful
CI / changes (pull_request) Successful in 7s
CI / backend-tests (pull_request) Has been skipped
CI / frontend-tests (pull_request) Has been skipped
CI / openapi-codegen-check (pull_request) Has been skipped
feat(devops): self-hosted OSRM routing engine for site-finder (#39)
Stand up an internal-only OSRM container so /analyze can later use real
road/walking distances to POI instead of straight-line ST_Distance. Infra
only — the /analyze integration is a separate follow-up (A2/A3).

- docker-compose.prod.yml: osrm service (osrm/osrm-backend, --algorithm mld,
  mem_limit 1.5g, default network, no public port — backend reaches it at
  http://osrm:5000). TCP /dev/tcp healthcheck (image has no curl). backend does
  NOT depend_on osrm, so a crash-loop before the graph is built won't block deploy.
- scripts/build_osrm.sh: idempotent MLD build/refresh (extract→partition→customize,
  not contract). Defaults to Свердл-clip из Ural-FO via osmium (lighter RAM);
  CLIP=0 zero-clip and EKB_TIGHT=1 / WALK=1 options documented. Monthly-refresh note.
- docs/osrm-routing.md: region/pbf decision, RAM table, VPS run-commands, verify curl.
- .gitignore data/osrm/* (don't commit ~1.5GB build output); .gitkeep holds the dir.

Refs #39
2026-06-27 00:13:41 +05:00

309 lines
14 KiB
YAML
Raw Blame History

This file contains ambiguous Unicode characters

This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.

# Production compose. Pulls pre-built images from GHCR
# (built and pushed by .github/workflows/deploy.yml).
#
# Place at /opt/gendesign/docker-compose.prod.yml on the VM.
# Required env in /opt/gendesign/.env:
# POSTGRES_DB, POSTGRES_USER, POSTGRES_PASSWORD
# IMAGE_TAG (defaults to "latest")
#
# Optional (LLM chat #960/#957 — dormant until BOTH set; written to
# backend/.env.runtime by deploy.yml from Forgejo secret/var only when non-empty):
# OPENAI_API_KEY — Forgejo Actions secret (sensitive); unset → openai_api_key=None
# LLM_ENABLED — Forgejo Actions variable (bool, "true"); unset → llm_enabled=False
#
# In production the browser talks to Caddy on :80/:443.
# Caddy proxies /api/* and /health straight to backend, the rest to frontend.
# So the frontend uses same-origin relative URLs — no NEXT_PUBLIC_API_BASE_URL needed.
#
# Postgres + Redis run alongside the app on the same VM (Discovery mode).
# Volumes are shared with docker-compose.yml so switching between files preserves data.
services:
postgres:
image: postgis/postgis:16-3.4
restart: unless-stopped
environment:
POSTGRES_DB: ${POSTGRES_DB}
POSTGRES_USER: ${POSTGRES_USER}
POSTGRES_PASSWORD: ${POSTGRES_PASSWORD}
# Bind to 127.0.0.1 only — accessible from host (for SSH tunnel) but not from public internet.
# UFW additionally blocks 5432 from outside.
ports:
- "127.0.0.1:5432:5432"
volumes:
- postgres_data:/var/lib/postgresql/data
- ./backend/db/init:/docker-entrypoint-initdb.d:ro
healthcheck:
test: ["CMD-SHELL", "pg_isready -U $${POSTGRES_USER}"]
interval: 5s
timeout: 5s
retries: 10
networks:
default: {}
shared:
aliases:
- gendesign-postgres
redis:
image: redis:7-alpine
restart: unless-stopped
volumes:
- redis_data:/data
healthcheck:
test: ["CMD", "redis-cli", "ping"]
interval: 10s
timeout: 3s
retries: 5
# OSRM routing engine (#39 — site-finder /analyze road/walking distances to POI
# вместо straight-line ST_Distance). INFRA-only здесь: интеграция в /analyze —
# отдельный follow-up (A2/A3). Backend ходит к нему по http://osrm:5000 через
# default-сеть. НЕ публикуется наружу (нет ports:) — internal-only, Caddy его
# не проксирует.
#
# Region: Свердловская обл (oblast-clip из Ural-FO extract). Префикс .osrm-файлов
# задаётся OSRM_REGION (default `sverdlovsk`). Файлы /data/<region>.osrm* кладёт
# scripts/build_osrm.sh в ./data/osrm/ (host) → /data (container). Артефакты
# .gitignored (не коммитим ~1.5GB build output) — на VPS строятся offline.
#
# --algorithm mld → pipeline extract→partition→customize (НЕ contract). max-table-size
# поднят для будущих many-to-many table-запросов (изохроны A2).
#
# ВАЖНО: пока scripts/build_osrm.sh не отработал, /data/<region>.osrm нет →
# osrm-routed падает и контейнер crash-loop'ит (restart: unless-stopped). Это
# БЕЗВРЕДНО для деплоя: backend НЕ depends_on osrm, поэтому `compose up -d` в
# deploy.yml не блокируется. Оркестратор строит граф offline и поднимает osrm
# отдельно (docs/osrm-routing.md).
osrm:
image: osrm/osrm-backend:latest
restart: unless-stopped
command: osrm-routed --algorithm mld --max-table-size 8000 /data/${OSRM_REGION:-sverdlovsk}.osrm
volumes:
- ./data/osrm:/data
mem_limit: 1.5g
# TCP-probe порта 5000 через bash /dev/tcp. osrm/osrm-backend (Debian-slim)
# НЕ содержит curl/wget — HTTP-healthcheck падал бы на missing binary (ср.
# glitchtip #644). bash в образе есть → /dev/tcp надёжно проверяет, что
# osrm-routed слушает (= граф загружен в RAM, движок принимает запросы).
# start_period щедрый: загрузка MLD-графа Свердл в RAM занимает время.
# Полный route-probe (driving) — в scripts/build_osrm.sh и в PR-инструкции
# для оркестратора (curl с хоста VPS, см. docs/osrm-routing.md).
healthcheck:
test: ["CMD-SHELL", "timeout 3 bash -c '</dev/tcp/127.0.0.1/5000' || exit 1"]
interval: 30s
timeout: 5s
retries: 5
start_period: 40s
networks: [default]
backend:
image: ghcr.io/lekss361/gendesign-backend:${IMAGE_TAG:-latest}
restart: unless-stopped
# .env.runtime пишется deploy.yml через SSH (SENTRY_RELEASE=$IMAGE_TAG).
# required: false — compose не падает если файла нет (первый деплой).
env_file:
- path: ./backend/.env
- path: ./backend/.env.runtime
required: false
depends_on:
postgres:
condition: service_healthy
redis:
condition: service_healthy
ports:
- "127.0.0.1:8000:8000"
volumes:
# RBAC roles config — single source of truth (auth/roles.yaml в репо).
# FastAPI middleware и /api/v1/me читают этот файл при старте (см.
# app/core/auth.py). Read-only — мутация только через PR.
- ./auth/roles.yaml:/app/auth/roles.yaml:ro
healthcheck:
test: ["CMD", "curl", "-fsS", "http://localhost:8000/health"]
interval: 30s
timeout: 5s
retries: 5
start_period: 20s
# #976 cross-DB ETL tradein→gendesign: backend подключается к tradein-postgres
# через gendesign_shared network (tradein-postgres уже в этой сети).
# default — обязательно явно, иначе сервис выпадет из дефолтной сети.
networks: [default, shared]
frontend:
image: ghcr.io/lekss361/gendesign-frontend:${IMAGE_TAG:-latest}
restart: unless-stopped
ports:
- "127.0.0.1:3000:3000"
depends_on:
- backend
# TCP-only probe через node (есть в alpine image, запускает сам server.js).
# НЕ использует HTTP semantics — проверяет только что порт 3000 слушает.
# Если Next.js процесс жив и порт открыт → healthy, независимо от HTTP status.
# start_period 60s даёт время Next.js standalone bundle bootstrap.
healthcheck:
test: ["CMD", "node", "-e", "require('net').createConnection({port:3000,host:'127.0.0.1'}).once('connect',function(){process.exit(0)}).once('error',function(){process.exit(1)})"]
interval: 15s
timeout: 5s
retries: 6
start_period: 60s
worker:
# Отдельный chromium-образ (+200 МБ Playwright). См. backend/Dockerfile target=runner-with-chromium.
image: ghcr.io/lekss361/gendesign-worker:${IMAGE_TAG:-latest}
restart: unless-stopped
env_file:
- path: ./backend/.env
- path: ./backend/.env.runtime
required: false
depends_on:
postgres:
condition: service_healthy
redis:
condition: service_healthy
volumes:
- ./data:/app/data # playwright_state.json + photo binaries
# Read-only bind-mount Антоновского /sf/ SQLite — для objective_etl task.
# На VPS файл лежит в /opt/gendesign/site-finder/analysis.db; путь внутри
# контейнера задан settings.objective_anton_sqlite_path (default
# /data/anton-sqlite/analysis.db).
- /opt/gendesign/site-finder:/data/anton-sqlite:ro
command: ["celery", "-A", "app.workers.celery_app", "worker", "--loglevel=info", "--concurrency=8", "--queues=celery,scrape_kn,geo"]
# #976 cross-DB ETL tradein→gendesign: worker запускает etl_newbuilding_crossload task,
# которому нужен прямой TCP-доступ к tradein-postgres через gendesign_shared.
# default — обязательно явно, иначе сервис выпадет из дефолтной сети.
networks: [default, shared]
beat:
# Lean backend-образ (без Chromium) — beat только триггерит таски в Redis.
image: ghcr.io/lekss361/gendesign-backend:${IMAGE_TAG:-latest}
restart: unless-stopped
env_file:
- path: ./backend/.env
- path: ./backend/.env.runtime
required: false
depends_on:
redis:
condition: service_healthy
# --schedule=/tmp/...: default location `/app/celerybeat-schedule` падает
# с Permission denied — WORKDIR `/app` принадлежит root, `app` (uid 1000)
# не может создавать там файлы. `/tmp` всегда writable; schedule-файл
# хранит только last_run_at для periodic tasks — потеря на restart OK,
# beat перестроит из `celery_app.conf.beat_schedule` на старте.
command: ["celery", "-A", "app.workers.celery_app", "beat", "--loglevel=info", "--schedule=/tmp/celerybeat-schedule"]
glitchtip-web:
image: glitchtip/glitchtip:6.1.6
container_name: glitchtip-web
# profiles: ["glitchtip"] keeps this service from starting on plain `compose up -d`.
# Bootstrap script activates the profile after DB + secrets are ready.
# On subsequent deploys, set COMPOSE_PROFILES=glitchtip in /opt/gendesign/.env.
profiles: ["glitchtip"]
depends_on:
postgres:
condition: service_healthy
redis:
condition: service_started
environment:
DATABASE_URL: postgres://glitchtip:${GLITCHTIP_DB_PASS}@postgres:5432/glitchtip
REDIS_URL: redis://redis:6379/2
SECRET_KEY: ${GLITCHTIP_SECRET}
PORT: "8080"
EMAIL_URL: consolemail://
GLITCHTIP_DOMAIN: https://errors.gendsgn.ru
DEFAULT_FROM_EMAIL: errors@gendsgn.ru
ENABLE_USER_REGISTRATION: "true"
ENABLE_ORGANIZATION_CREATION: "false"
restart: always
mem_limit: 512m
# Образ glitchtip:6.1.6 НЕ содержит wget/curl (только python3) → старый
# wget-healthcheck падал на missing-binary → контейнер вечно unhealthy (#644).
# Пробуем канонический /_health/ (GET → 200) через встроенный python3.
healthcheck:
test: ["CMD", "python3", "-c", "import urllib.request,sys; sys.exit(0 if urllib.request.urlopen('http://localhost:8080/_health/', timeout=8).status == 200 else 1)"]
interval: 30s
timeout: 10s
retries: 5
start_period: 60s
networks: [default]
glitchtip-worker:
image: glitchtip/glitchtip:6.1.6
container_name: glitchtip-worker
profiles: ["glitchtip"]
depends_on:
postgres:
condition: service_healthy
redis:
condition: service_started
command: ./bin/run-celery-with-beat.sh
environment:
DATABASE_URL: postgres://glitchtip:${GLITCHTIP_DB_PASS}@postgres:5432/glitchtip
REDIS_URL: redis://redis:6379/2
SECRET_KEY: ${GLITCHTIP_SECRET}
CELERY_WORKER_AUTOSCALE: "1,3"
restart: always
mem_limit: 384m
networks: [default]
caddy:
image: caddy:2
restart: unless-stopped
ports:
- "80:80"
- "443:443"
volumes:
- ./Caddyfile:/etc/caddy/Caddyfile:ro
- ./caddy/users.caddy.snippet:/etc/caddy/caddy/users.caddy.snippet:ro
- ./preview:/srv/preview:ro
- caddy_data:/data
- caddy_config:/config
- caddy_logs:/var/log/caddy
# backend: service_healthy — backend имеет /health endpoint, готов до Caddy.
# frontend: service_started — НЕ service_healthy: даже если frontend healthcheck
# дребезжит, Caddy всё равно стартует (можно отдавать 502 для /, но
# obsidian.gendsgn.ru, api.gendsgn.ru и т.д. остаются доступны).
depends_on:
backend:
condition: service_healthy
frontend:
condition: service_started
networks:
- default
# shared — для маршрута obsidian.gendsgn.ru → couchdb (отдельный stack).
# Если obsidian-стек не задеплоен, Caddy просто получит 502 на этом маршруте,
# main-приложение не страдает.
- shared
glitchtip-auth-forwarder:
# Собирается локально на VPS при деплое (не тянется из GHCR).
# deploy.yml запускает: docker compose build glitchtip-auth-forwarder
build: ./ops/glitchtip-auth-forwarder
container_name: gendesign-auth-forwarder
restart: unless-stopped
environment:
GLITCHTIP_DSN: ${GLITCHTIP_DSN}
APP_ENV: production
APP_RELEASE: auth-forwarder-1
CADDY_LOG_FILE: /var/log/caddy/auth_audit.log
STATE_FILE: /state/offset.json
volumes:
# Read-only доступ к Caddy access log
- caddy_logs:/var/log/caddy:ro
# Persistent offset — выживает при перезапуске контейнера
- auth_forwarder_state:/state
depends_on:
- caddy
volumes:
postgres_data:
redis_data:
caddy_data:
caddy_config:
caddy_logs:
auth_forwarder_state:
networks:
# Внешняя сеть, создаётся вне compose (см. docs/obsidian-livesync.md).
# Связывает main-stack (Caddy) и obsidian-stack (CouchDB).
shared:
external: true
name: gendesign_shared