feat(llm): foundational LLM infra package + §19 redaction (#960) #1168

Merged

bot-backend merged 1 commit from feat/llm-infra-960 into main

2026-06-08 11:01:46 +00:00

bot-backend commented

2026-06-08 10:44:44 +00:00

Collaborator

Summary

Foundational LLM infrastructure (#960, ТЗ §12.4/§7.4) — new backend/app/services/llm/ package. No consumers yet (#956 граддок-extraction / #957 chat will consume it). Provider: external OpenAI (owner decision); architecture keeps the forecasting engine fully deterministic — LLM is an optional layer that always degrades to a deterministic fallback.

Safety-by-default (the whole point of the gating)

llm_enabled=False by default + openai_api_key=None → complete() returns the fallback before constructing any provider / opening any socket. Prod cannot call OpenAI until deliberately enabled. (Proven by a httpx.Client instantiation tripwire test.)
§19 redaction, mandatory on the external path:
- allowlist-first SafePayload contract — callers physically cannot pass raw DB rows, only vetted fields/aggregates/public text;
- is_confidential=True → hard block (RedactionRefusedError → fallback, provider never called);
- RU-PII regex scrub (phone / email / СНИЛС / ИНН / passport / ФИО) as defense-in-depth;
- scrub_safe_payload() runs unconditionally before any external call.
No secret leakage: key only in the Authorization header, never logged/echoed; no key literal anywhere (grep sk- empty).
Fallback-always: disabled / no-key / timeout / 429 / 5xx / network / redaction-refusal / call-cap all degrade to a deterministic LLMResult(fallback_used=True) — never an unhandled exception. Bounded retry+backoff.

Design

provider.py — abstract LLMProvider (with is_external seam for a future RU-hosted GigaChat/YandexGPT provider, no caller changes) + OpenAIProvider (raw httpx, tool/function-calling pass-through).
redaction.py — the §19 layer. client.py — enabled→redact→guardrails→provider→retry→fallback. prompts.py — minimal versioned templates. Config extended in core/config.py.
Sync httpx core (Celery-friendly for #956; #957 async endpoint bridges via threadpool). No openai SDK → pyproject.toml/uv.lock untouched (no deploy-abort risk).

Test plan

uv run pytest tests/services/llm/ -q → 47 passed
ruff check + mypy (strict) → clean
Safety props proven in tests: no-network-when-disabled (tripwire), is_confidential hard-block (provider.calls==0), scrub-before-call ordering, redacted request body, key-not-in-body, bounded retry
code-reviewer: ✅ APPROVE (4 safety properties verified empirically)
Prod enable (devops, separate): set OPENAI_API_KEY in backend+worker runtime env (out of git) and LLM_ENABLED=true after the §19 sign-off + key rotation. Until then: dormant, zero network.

Notes

Owner pasted an OpenAI key in chat → treat as exposed/rotate; the working key must live only in prod secrets, never the repo.
Deferred to consumers (#956/#957): harden free-text PII regex for real граддок text; enforce (not just log) the daily cost cap.

Refs #960

## Summary Foundational LLM infrastructure (#960, ТЗ §12.4/§7.4) — new `backend/app/services/llm/` package. **No consumers yet** (#956 граддок-extraction / #957 chat will consume it). Provider: external **OpenAI** (owner decision); architecture keeps the forecasting engine fully deterministic — LLM is an optional layer that always degrades to a deterministic fallback. ### Safety-by-default (the whole point of the gating) - **`llm_enabled=False` by default** + `openai_api_key=None` → `complete()` returns the fallback **before constructing any provider / opening any socket**. Prod cannot call OpenAI until deliberately enabled. (Proven by a `httpx.Client` instantiation tripwire test.) - **§19 redaction, mandatory on the external path:** - allowlist-first `SafePayload` contract — callers physically cannot pass raw DB rows, only vetted fields/aggregates/public text; - `is_confidential=True` → **hard block** (`RedactionRefusedError` → fallback, provider never called); - RU-PII regex scrub (phone / email / СНИЛС / ИНН / passport / ФИО) as defense-in-depth; - `scrub_safe_payload()` runs unconditionally before any external call. - **No secret leakage:** key only in the `Authorization` header, never logged/echoed; no key literal anywhere (`grep sk- ` empty). - **Fallback-always:** disabled / no-key / timeout / 429 / 5xx / network / redaction-refusal / call-cap all degrade to a deterministic `LLMResult(fallback_used=True)` — never an unhandled exception. Bounded retry+backoff. ### Design - `provider.py` — abstract `LLMProvider` (with `is_external` seam for a future RU-hosted GigaChat/YandexGPT provider, no caller changes) + `OpenAIProvider` (raw httpx, tool/function-calling pass-through). - `redaction.py` — the §19 layer. `client.py` — enabled→redact→guardrails→provider→retry→fallback. `prompts.py` — minimal versioned templates. Config extended in `core/config.py`. - **Sync httpx core** (Celery-friendly for #956; #957 async endpoint bridges via threadpool). **No `openai` SDK** → `pyproject.toml`/`uv.lock` untouched (no deploy-abort risk). ## Test plan - [x] `uv run pytest tests/services/llm/ -q` → **47 passed** - [x] `ruff check` + `mypy` (strict) → clean - [x] Safety props proven in tests: no-network-when-disabled (tripwire), is_confidential hard-block (provider.calls==0), scrub-before-call ordering, redacted request body, key-not-in-body, bounded retry - [x] code-reviewer: ✅ APPROVE (4 safety properties verified empirically) - [ ] Prod enable (devops, separate): set `OPENAI_API_KEY` in backend+worker runtime env (out of git) and `LLM_ENABLED=true` **after** the §19 sign-off + key rotation. Until then: dormant, zero network. ## Notes - Owner pasted an OpenAI key in chat → **treat as exposed/rotate**; the working key must live only in prod secrets, never the repo. - Deferred to consumers (#956/#957): harden free-text PII regex for real граддок text; enforce (not just log) the daily cost cap. Refs #960

bot-backend added 1 commit 2026-06-08 10:44:44 +00:00

feat(llm): foundational LLM infra package with §19 redaction + deterministic fallback (#960 )

CI / changes (pull_request) Successful in 5s

Details

CI / backend-tests (push) Successful in 6m25s

Details

CI / backend-tests (pull_request) Successful in 6m20s

Details

Deploy / changes (push) Successful in 5s

Details

Deploy / build-frontend (push) Has been skipped

Details

Deploy / build-backend (push) Successful in 1m38s

Details

Deploy / build-worker (push) Successful in 2m42s

Details

Deploy / deploy (push) Successful in 1m13s

Details

CI / changes (push) Successful in 5s

Details

CI / frontend-tests (push) Has been skipped

Details

CI / frontend-tests (pull_request) Has been skipped

Details

4af7ba5a40

Optional external-OpenAI layer over the deterministic forecasting engine. Gated by
llm_enabled (default False) so prod makes no network calls until deliberately enabled.
Allowlist-first SafePayload contract + is_confidential hard-block + RU-PII regex scrub
(mandatory on the external path). Abstract LLMProvider seam (is_external) for a future
RU-hosted provider. Sync httpx core (Celery-friendly); tool/function-calling pass-through;
timeout + bounded 429/5xx retry + per-request call cap, all degrading to fallback.
Raw httpx (no openai SDK -> no pyproject/lock drift). 47 tests, ruff + mypy clean.

Refs #960