Commit graph

27 commits

Author SHA1 Message Date
0658399f01 fix(devops): cleanup-merged-worktrees.sh — 3 бага, найденные вживую на 69 забытых worktree (#2418) 2026-07-04 13:00:14 +00:00
b7d35cfee0 chore(claude): security hardening — dangerous-commands hook, settings в git, restricted DB, throttle (#2150) 2026-07-02 15:23:11 +00:00
845f699176 chore(claude-config): secret-read + psycopg2 hooks, tradein.md rule, tradein-mvp globs (#1975) 2026-06-27 10:03:19 +00:00
8a3bae0cda feat(analyze): per-category OSRM routing (foot vs driving) (#39 A3)
All checks were successful
CI / changes (pull_request) Successful in 6s
CI / frontend-tests (pull_request) Has been skipped
CI / openapi-codegen-check (pull_request) Successful in 1m46s
CI / backend-tests (pull_request) Successful in 11m55s
Walk-relevant POIs (school/shop/park/kindergarten/pharmacy/stops) now route
via a FOOT OSRM graph (osrm-walk service), car-relevant POIs (mall/hospital +
unknown) keep the DRIVING graph. Validation showed driving overstated
pedestrian-proximity distance — median 1.6-2.9x straight-line (#39).

- config: osrm_walk_local_url + osrm_walk_categories (frozenset, 9 walk cats)
- osrm_client_local: base_url override on get_road_distances_m (default unchanged)
- _apply_osrm_road_distances: split POIs by category, per-group OSRM call with
  independent graceful fallback (one server down -> its group keeps straight-line),
  in-place write-back by original index; never raises; flag-OFF byte-identical
- docker-compose: osrm-walk service (foot graph, internal, mem_limit 1.5g)
- build_osrm.sh: CAR=0 gate for foot-only refresh (doesn't touch live car graph)
- tests: per-category split, per-group fallback, asymmetric intra-group write-back

Still flag-gated (use_osrm_distances OFF) — enabling is a product decision.
Refs #39
2026-06-27 04:30:25 +05:00
4fe20a9cbd feat(devops): self-hosted OSRM routing engine for site-finder (#39) (#1929)
All checks were successful
Deploy / changes (push) Successful in 7s
Deploy / build-worker (push) Successful in 46s
Deploy / deploy (push) Successful in 1m18s
Deploy / build-frontend (push) Successful in 31s
Deploy / build-backend (push) Successful in 33s
2026-06-26 19:18:21 +00:00
86e9ea2937 fix(week-review): автофиксы код-ревью — 169 issue (label «week ревью 1»)
Многоагентный аудит + имплементация: один воркер на файл, точечные правки.
Верификация: py_compile (47/47 .py) + tsc --noEmit (0 ошибок). Unit-тесты
не прогонялись (окружение не поднято: rollup native dep / нет pytest-venv).

Полностью исправлено (169): #1336, #1337, #1339, #1340, #1341, #1342, #1343, #1345, #1346, #1348, #1349, #1350, #1351, #1354, #1356, #1358, #1359, #1360, #1362, #1364, #1365, #1366, #1367, #1368, #1369, #1370, #1371, #1372, #1373, #1374, #1375, #1376, #1377, #1378, #1379, #1380, #1381, #1382, #1384, #1385, #1386, #1387, #1388, #1389, #1390, #1391, #1392, #1394, #1395, #1396, #1397, #1399, #1400, #1401, #1402, #1403, #1404, #1408, #1409, #1410, #1411, #1412, #1413, #1414, #1415, #1416, #1417, #1418, #1420, #1423, #1425, #1426, #1427, #1428, #1429, #1430, #1431, #1432, #1433, #1434, #1435, #1437, #1438, #1439, #1440, #1441, #1442, #1443, #1444, #1445, #1446, #1447, #1448, #1449, #1450, #1451, #1452, #1453, #1454, #1455, #1456, #1457, #1458, #1459, #1460, #1461, #1462, #1463, #1464, #1465, #1466, #1467, #1468, #1469, #1471, #1472, #1473, #1474, #1476, #1478, #1479, #1481, #1482, #1483, #1484, #1485, #1487, #1488, #1489, #1490, #1491, #1492, #1493, #1494, #1495, #1496, #1497, #1499, #1500, #1501, #1502, #1504, #1505, #1506, #1507, #1510, #1514, #1515, #1516, #1517, #1518, #1519, #1521, #1522, #1523, #1524, #1525, #1526, #1527, #1528, #1529, #1531, #1532, #1533, #1534, #1535, #1536, #1537, #1538

Частично (9, in-file часть, остаток cross-file): #1361, #1419, #1422, #1424, #1470, #1475, #1477, #1480, #1498
Требуют cross-file (3, не тронуты): #1338, #1363, #1421
Пропущено (1): #1539

Не входило в партию: 22 needs-Leha issue (нужны решения владельца).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-15 20:21:11 +05:00
6883d14177 fix(ops): repair broken main-DB backup + harden auth scripts/docs (#71 #427 #429 #428)
Some checks failed
CI / changes (push) Successful in 7s
CI / backend-tests (push) Has been skipped
CI / frontend-tests (push) Has been skipped
CI / changes (pull_request) Successful in 6s
CI / backend-tests (pull_request) Has been skipped
CI / frontend-tests (pull_request) Has been skipped
Deploy / build-backend (push) Blocked by required conditions
Deploy / build-worker (push) Blocked by required conditions
Deploy / build-frontend (push) Blocked by required conditions
Deploy / deploy (push) Blocked by required conditions
Deploy / changes (push) Has been cancelled
#71 (CRITICAL): backup.sh committed 100644 → git reset --hard on deploy
re-asserts non-exec mode → raw-path cron fails Permission denied (last good
dump 2026-05-27, no S3). Commit 100755 + chmod ops/*.sh in deploy.yml +
size sanity-check (never prune good dumps for a truncated one) + keep-N
retention + optional S3 (redacted /etc/default template). Modeled on the
working backup-tradein-db.sh.

#427: widen basic_auth username regex ^[a-z][a-z0-9_.-]{1,62}$ + literal-escape
  dotted names in grep probes.
#429: replace list_users.sh false-positive grep with anchored awk over basic_auth block.
#428: PILOT_ACCESS.md support email → pilot@gendsgn.ru.

Closes #71
Closes #427
Closes #429
Closes #428
2026-06-13 20:13:04 +05:00
6e0039dd47 chore(agents): reviewer back to Opus (revert model-part of aa3d012)
Per user decision: reviewer = Opus (merge-authority needs strong reasoning on verdict);
остальные loop-роли остаются Sonnet. Реверсит только model-часть aa3d012 (reviewer→sonnet).
- start-bot.ps1: $model = reviewer? opus : sonnet + --model $model
- auto-code-reviewer.md: frontmatter model: opus + doc reviewer на Opus
2026-05-31 17:16:57 +03:00
5fa9588935 chore(agents): bot harness tuning — per-role model, forgejo MCP deferral, vault capture, #893 curl hardening
- start-bot.ps1: --model per-role (sonnet; opus только reviewer); CLAUDE_AUTOCOMPACT_PCT_OVERRIDE=60
- .claude/mcp/*.json: forgejo (~90 tools) deferred во всех ролях; postgres-gendesign deferred в backend/reviewer
- auto-analyst.md: knowledge-capture step (status/done → inbox draft → vault-overlord) — закрывает 'волт не обновляется'
- forgejo ops через mcp__forgejo__* вместо сырого curl: auto-code-reviewer/backend/qa workflow + _autonomous_pickup contract
- #893 guard: бан curl→/tmp→python3 (Windows 404+FileNotFoundError); deferred→ToolSearch; stale tool-name cleanup (deep-reviewer phases)
2026-05-31 16:26:17 +03:00
aa3d012fdf chore(agents): bot-окна на Sonnet (start-bot --model) + context-hygiene + reviewer/resolver→sonnet (weekly Opus-пул) 2026-05-31 15:39:42 +03:00
5cfab0b23e fix(bots): подключить postgres-tradein MCP в analyst/backend bot-окна (#857) 2026-05-31 06:44:46 +00:00
lekss361
2df8f78747 chore(bots): per-window MCP isolation via --strict-mcp-config
Каждое bot-окно грузит ТОЛЬКО нужные роли серверы (не все 7+ в каждом окне).
Выигрыш: не спавнить heavy-серверы зря (docker postgres в frontend/qa), right-size
eager-набор, чистый tool-surface на роль. (Контекст tool-search и так бережёт —
deferred ≈ бесплатно; это про процессы + eager + ясность.)

- .claude/mcp/{analyst,backend,frontend,reviewer,qa}.json — self-contained per-role
  конфиги. Секреты НЕ инлайнятся: ${OBSIDIAN_API_KEY}/${GENDESIGN_DB_URI}/
  ${GLITCHTIP_TOKEN} через env-подстановку (.mcp.json env expansion, подтв. в доках).
  Матрица: forgejo+obsidian+context7 eager везде; pg-gendesign eager у backend/reviewer,
  deferred у analyst; UI (playwright/a11y/lighthouse/shadcn) только frontend/qa; fetch+
  glitchtip у backend; glitchtip у qa.
- start-bot.ps1: извлекает MCP-секреты из ~/.claude.json (единый источник) → export →
  запускает `claude --strict-mcp-config --mcp-config .claude/mcp/<role>.json`.

Main/интерактивное окно (без start-bot.ps1) по-прежнему видит полный .mcp.json+user.
2026-05-30 11:05:24 +03:00
lekss361
edc5823622 chore(bots): per-role launcher scripts + move bot pipeline to forgejo MCP
(a) Launcher scripts — больше не экспортировать токен руками:
- scripts/start-bot.ps1 <role> — выставляет per-bot identity + токены (FORGEJO_ACCESS_TOKEN
  для forgejo MCP + FORGEJO_TOKEN для curl-fallback), git-identity, forgejo-bot remote,
  verify, затем запускает claude.
- scripts/start-{analyst,backend,frontend,reviewer,qa}.ps1 — тонкие врапперы.

(b) Bots on forgejo MCP (goern) вместо curl:
- _autonomous_pickup.md: новая секция «Forgejo операции — mcp__forgejo__* (PRIMARY)» с полным
  curl→MCP mapping (list_repo_issues / create_pull_request / merge_pull_request /
  get_pull_request_diff / create_pull_review / add+remove_issue_labels / update_issue /
  issue_state_change / create_issue_comment). curl_forgejo помечен FALLBACK.
- 5 auto-*.md + 5 work-as-*.md: директива «Forgejo API → mcp__forgejo__* tools (mapping в
  _autonomous_pickup); curl только fallback» + указатель на start-bot.ps1.

Gotcha задокументирован: include_org_labels:false на user-репо (lekss361 не org).
Требует: рестарт Claude Code (forgejo MCP eager из .mcp.json) + FORGEJO_ACCESS_TOKEN per-window
(делает start-bot.ps1).
2026-05-30 10:38:10 +03:00
lekss361
5673dd6298 chore(claude): extend self-extending tripwire + pause-bots guard for stale-claims
Two safety follow-ups к autonomous-pipeline (flagged в multiagent-аудите):

- Self-extending guard расширен: reviewer-bot NEVER merge'ит PR, меняющий
  _autonomous_pickup.md (claim/kill-switch/merge-FSM contract) или любой
  work-as-*.md (persona activation) — раньше защищались только git-pr.md
  Auto-merge policy + CLAUDE.md + auto-code-reviewer.md. Закрывает дыру, где
  bot мог изменить claim/kill-switch logic без human. Правки в git-pr.md,
  auto-code-reviewer.md, work-as-reviewer.md.
- cleanup-stale-claims.sh: pause-bots early-exit. Без него cron освобождал
  wip-claim worker'а, приостановленного mid-work (он держит claim до un-pause
  per _autonomous_pickup.md), теряя его работу.
2026-05-29 17:36:20 +03:00
lekss361
8c9d1344cc fix(stale-claims): avoid SIGPIPE 141 — pass file path не pipe
Root cause: `cat $ISSUES_TMP | python3 <<EOF ... sys.exit(0)` — Python exits
0 early, cat получает SIGPIPE, set -o pipefail → script exits 141 = failure.

Fix: передача file path через env var вместо stdin pipe. Python читает файл
напрямую. No pipe → no SIGPIPE.

Verified локально на VPS с bot-qa token: exit 0, '[OK] 0 wip issues —
nothing to do'.

Also remove diagnostic steps from workflow (added в PR #611, нужно для
поиска root cause, теперь убираем).

Refs: PR #611 (diagnostic), runs #1514-1520 failures.
2026-05-28 01:05:56 +03:00
lekss361
e4dc866e20 fixup(claude): address review-bot blockers on PR #610
🔴 Critical (3):

1. cleanup-stale-claims.sh: export CUTOFF + STALE_HOURS
   Без `export` Python heredoc child process получал "0" вместо реального
   cutoff → cron всегда видел все issues как fresh → ничего не освобождал.
   Plus added abort при CUTOFF=0 в Python heredoc для defense-in-depth.

2. cleanup-stale-claims.sh: datetime.UTC → datetime.timezone.utc
   datetime.UTC требует Python 3.11+. Forgejo runner ubuntu-latest имеет 3.10.
   datetime.timezone.utc works в 3.8+.

3. cleanup-stale-claims.sh: invert label order — DELETE wip FIRST, потом
   POST ready. Если step 2 (add ready) fails — issue в neutral state без
   status/*, менее опасно чем оба status/ready+status/wip одновременно.

🟠 + 🟡 + 🟢 (5):

4. Added pagination loop (defensive cap 10 pages × 50 = 500 issues), но
   реализована через temp file + Python parsing (не bash JSON concat).

5. Removed dead code: iso_to_epoch() shell helper не использовался.

6. _autonomous_pickup.md: fix broken `Out-Null | if (-not $?)` pattern.
   GetEnvironmentVariable НЕ throws при missing — возвращает $null. $? у
   Get* всегда $true. Заменено на прямую проверку результата.

7. stale-claims.yml: добавлен concurrency group чтобы overlapping runs
   не stomp'ились (хотя 5-min timeout делает overlap unlikely, defensive).

8. Documented updated_at != heartbeat assumption в header script'а.

Не сделано (low/optional):

- Setup script tokens в memory (UX trade-off — не critical)
- Token suffix в console output (минор info leak в shell history)

Refs: PR #610 review by review-bot (sha=297eb29, verdict=changes)
2026-05-28 00:33:27 +03:00
lekss361
297eb29d65 feat(claude): env-vars refactor + stale-claim cron (Phase 2 finalization)
#11: slash-commands и _autonomous_pickup.md теперь читают persistent User-scope
env vars напрямую (FORGEJO_TOKEN_<ROLE>) вместо file-based из ~/.claude/secrets/.

Один раз: scripts/setup-bot-env.ps1 (запускается user'ом локально, кладёт
токены из vault в Windows User registry). После этого окна grab credentials
через [System.Environment]::GetEnvironmentVariable() — никаких файлов.

Все 5 work-as-*.md обновлены (analyst/backend/frontend/reviewer/qa).

#12: stale-claim cleanup — cron каждые 30 минут.

- scripts/cleanup-stale-claims.sh — bash, parses Forgejo API + Python для JSON
- .forgejo/workflows/stale-claims.yml — cron schedule `*/30 * * * *`

Освобождает issues застрявшие в status/wip >4h:
  - assignees=[]
  - +status/ready -status/wip
  - comment "stale claim released, worker likely crashed"

Использует secret FORGEJO_BOT_QA_TOKEN (минимум прав = write:issue).

Setup: secret нужно завести в Forgejo Actions UI после merge —
git.gendsgn.ru/lekss361/gendesign/-/settings/actions/secrets.

Refs: PRs #608, #609. Closes TODO с PR #608 (stale-claim cron).
2026-05-28 00:24:45 +03:00
d954be1331 fix(auth): scripts/auth/*.sh — правильный SNIPPET path (#589) 2026-05-26 08:39:24 +00:00
03ab1faa7c chore(claude): aggressive worktree cleanup via Forgejo PR API (#498) 2026-05-24 09:34:48 +00:00
998c8a17a8 chore(claude): agents view readiness — cleanup, .worktreeinclude, workflow (#497) 2026-05-24 09:20:26 +00:00
73bdff41a7 chore(claude-config): align with 2026 best practices (#495) 2026-05-24 08:48:42 +00:00
cc2d148967 feat(security): закрыть gendsgn.ru basic_auth gate (multi-user, 11 users) (#426)
All checks were successful
Deploy / changes (push) Successful in 5s
Deploy / build-worker (push) Successful in 30s
Deploy / build-frontend (push) Successful in 28s
Deploy / build-backend (push) Successful in 31s
Deploy / deploy (push) Successful in 53s
Pilot demo 28.05.2026: closed gendsgn.ru/* (включая trade-in/) basic_auth gate. /health и /preview/* остаются public через route { } wrapper (Caddy 2.x directive ordering fix).

11 users (admin + user1..user10), bcrypt cost=14, snippet в git (audit trail). Scripts: scripts/auth/{add,remove,list}_user.sh.

Fixup PR #426 (route{} wrap + log rotation + base64 busybox-compat + runbook audit log fix).
2026-05-23 08:16:10 +00:00
021b0589b2 fix(infra): bootstrap_glitchtip — \gexec instead of DO block (psql var trap) (#219) 2026-05-16 15:54:29 +00:00
d3aa96336b feat(infra): GlitchTip self-hosted error tracking (#204 devops) (#205)
Some checks failed
Deploy / build-frontend (push) Blocked by required conditions
Deploy / deploy (push) Blocked by required conditions
Deploy / changes (push) Successful in 5s
Deploy / build-backend (push) Has been cancelled
Deploy / build-worker (push) Has been cancelled
2026-05-16 15:08:50 +00:00
lekss361
7c43bc5496
feat(infra): auto-apply data/sql/*.sql migrations in deploy pipeline (#150) (#151)
* fix(frontend): apiFetchWithStatus body stream double-read crash

Per user report 2026-05-14: 'Failed to execute text on Response: body stream
already read' при analyze с non-JSON error response.

apiFetchWithStatus делал .json() (consumes stream), потом .text() в catch
(FAILS — stream already consumed). Fetch API: body — ReadableStream,
consumable один раз.

Fix: read text() once → JSON.parse on string. На fail (HTML / non-JSON)
оборачиваем в {detail: rawText}. Никаких double-reads.

Site-finder cad search больше не крашится на error pages.

* feat(infra): auto-apply data/sql/*.sql migrations in deploy pipeline (#150)

Закрывает root cause production 500 на /api/v1/admin/site-finder/weight-profiles
(а также 2 unapplied migrations: #89/#91).

## Why это нужно

После audit обнаружили что:
- backend/alembic/versions/ пустой (Alembic configured но не используется)
- deploy.yml не имеет migration step
- 47+ raw SQL files накопилось без auto-apply
- 3 файла unapplied: 87 (engineering), 89 (drop brin), 90 (weight profiles)

User report: GET weight-profiles → 500 потому что user_weight_profiles
table не существует на prod.

## Changes

### deploy.yml +39 lines
- Path trigger расширен на data/sql/*.sql
- New step 'Apply DB migrations' между compose pull и compose up -d:
  - Idempotent CREATE TABLE _schema_migrations
  - Loop по data/sql/*.sql sorted → skip applied → psql with ON_ERROR_STOP=on
  - Record applied filename в tracking table
  - Exit 1 на failure → containers НЕ обновляются (no partial state)

### NN collision fix
- 87_engineering_networks_191fz.sql → 91_engineering_networks_191fz.sql
  (collision с 87_on_demand_indexes.sql, мой renumber 85→87 был неудачным)

### CLAUDE.md +32 lines
- Subsection 'Auto-apply' под Migration pattern
- Bootstrap procedure docs
- Idempotency requirement
- Failure recovery

### scripts/bootstrap_schema_migrations.sh +100 lines
- One-time seed _schema_migrations с 48 already-applied files
- 3 файла intentionally NOT seeded: 89/90/91 (auto-apply на first deploy)
- Usage docs + verification query

## Vault

domains/infra/Runbook_Auto_Apply_Migrations.md NEW
infra-MOC.md updated

## Required manual steps ДО merge

1. SSH gendesign (tunnel)
2. POSTGRES creds export
3. bash scripts/bootstrap_schema_migrations.sh — seed 48 already-applied files
4. Verify: SELECT COUNT(*) FROM _schema_migrations → 48
5. После merge → first deploy auto-apply #89/#90/#91 → 500 closes

Closes #150

---------

Co-authored-by: lekss361 <claudestars@proton.me>
2026-05-15 08:06:54 +03:00
lekss361
769b16df47 fix(worker): port objective_etl from psycopg2 → psycopg v3
Worker крашился на старте с `ModuleNotFoundError: No module named 'psycopg2'`
при импорте app/workers/tasks/objective_etl, потому что app/services/objective_etl
писал на psycopg2 API, а в pyproject.toml есть только psycopg[binary]>=3.2.0.

Изменения:
- psycopg2 → psycopg (v3)
- psycopg2.extras.execute_values → _bulk_upsert helper, эмулирующий тот же
  behavior через cur.executemany (в psycopg v3 это pipeline-mode, overhead
  на сетевые round-trips минимален)
- psycopg.connect совместим с тем же connection string

Тесты: AST parse ✓, ruff ✓, import ✓
2026-05-11 12:07:57 +03:00
lekss361
b0f5c38894 feat(deploy): split obsidian stack from main + path-filtered GHA
Obsidian Self-hosted LiveSync (CouchDB) выведен в отдельный docker-compose
stack и GHA workflow. Любой стек теперь редеплоится независимо.

* docker-compose.obsidian.yml: только couchdb в shared external network
* docker-compose.prod.yml: убрана couchdb-секция; caddy подключён к
  shared network для маршрута obsidian.gendsgn.ru → couchdb:5984
* .github/workflows/deploy-obsidian.yml: новый workflow с path-filters
  (триггерится только при изменениях obsidian-related файлов)
* .github/workflows/deploy.yml: + path-filters (триггерится только при
  main-related файлах) + создание shared network в bootstrap
* docs/obsidian-livesync.md: обновлены инструкции под split-архитектуру

После push:
1. ssh gendesign 'docker network create gendesign_shared'  (один раз)
2. Прописать COUCHDB_USER/PASSWORD в backend/.env.runtime
3. Добавить DNS A-record obsidian.gendsgn.ru → IP VPS
4. git push → GHA задеплоит обе части независимо

Преимущества:
- Изменения main не разрывают LiveSync клиентов
- Изменения obsidian не рестартуют main
- Падение obsidian-стека не влияет на основное приложение
2026-05-11 09:51:31 +03:00