Commit graph

6 commits

Author SHA1 Message Date
5c5be890ab fix(llm): INN redaction requires context/checksum, stop eating large numbers (#1640)
Some checks failed
CI / changes (push) Successful in 8s
CI / changes (pull_request) Successful in 7s
CI / frontend-tests (push) Has been skipped
CI / openapi-codegen-check (push) Has been skipped
CI / backend-tests (push) Has been skipped
CI / frontend-tests (pull_request) Has been skipped
CI / openapi-codegen-check (pull_request) Successful in 1m42s
CI / backend-tests (pull_request) Failing after 8m59s
_INN_RE now requires an explicit ИНН/inn keyword anchor (case-insensitive) within
~20 chars before a 10/12-digit block. Bare digit sequences without the keyword are
no longer candidates — eliminates false-positives on large monetary amounts such as
1 200 000 000 (which coincidentally passes the ФНС checksum). Checksum validation
is kept as a second gate to avoid redacting e.g. «ИНН 1234567890» with bad digits.

_inn_repl updated to use match.group(1) (digit-only capture group) instead of the
full match that now also includes the keyword prefix.

7 new regression tests in test_redaction.py: bare large numbers not redacted,
keyword-cued real INNs (10/12 digit) still redacted, bad-checksum + keyword left
intact, latin «inn:» accepted.
2026-06-17 20:41:39 +03:00
20efc3bf64 merge: resolve conflicts with main (#1660) into week-review-finish
Some checks failed
CI / backend-tests (pull_request) Failing after 9m11s
CI / backend-tests (push) Failing after 9m22s
CI / openapi-codegen-check (push) Successful in 2m14s
CI / changes (pull_request) Successful in 6s
CI / changes (push) Successful in 8s
CI / frontend-tests (pull_request) Successful in 1m13s
CI / frontend-tests (push) Successful in 1m2s
CI / openapi-codegen-check (pull_request) Successful in 1m47s
5 conflicts resolved:
- redaction.py: merged INN checksum + _PHONE_LOCAL_RE + _Repl alias from main
- recommendation.py: kept commercial_sell_through_pct rename (#1635)
- report_pdf.py: kept _scenario_deficit_cell + int|None return type (#1590)
- parcel.py: kept main's detailed docstring (functionally identical)
- alembic/env.py: kept HEAD (app.models import in __init__ already covers all)
2026-06-17 20:21:06 +03:00
14f3ef2019 fix(week-review): backend-аудит v2 — 82 фиксов (#1660)
All checks were successful
Deploy / build-worker (push) Successful in 2m47s
Deploy / deploy (push) Successful in 1m20s
Deploy / changes (push) Successful in 9s
Deploy / build-frontend (push) Has been skipped
Deploy / build-backend (push) Successful in 1m52s
Co-authored-by: bot-backend <bot-backend@gendsgn.local>
Co-committed-by: bot-backend <bot-backend@gendsgn.local>
2026-06-17 17:13:38 +00:00
249182b678 fix(week-review): дожим ревью — #1569, #1590, #1642 + feat #801 MSW-preview
All checks were successful
CI / frontend-tests (pull_request) Successful in 1m2s
CI / openapi-codegen-check (push) Successful in 2m13s
CI / openapi-codegen-check (pull_request) Successful in 1m45s
CI / backend-tests (push) Successful in 9m29s
CI / backend-tests (pull_request) Successful in 9m29s
CI / changes (push) Successful in 8s
CI / changes (pull_request) Successful in 8s
CI / frontend-tests (push) Successful in 1m7s
Финиш-волна по cross-file/partial остаткам аудита + a11y-харнесс.

Полностью (4): #1569 alembic регистрирует все ORM-модели (+убран фантомный Parcel) ·
#1590 «Индекс дефицита» больше не лжёт на fallback-горизонте (report_pdf+report_md) ·
#1642 thumbs mtime-freshness (нет устаревшей миниатюры) ·
#801 /__preview/estimate mock-render для axe/lighthouse (env-gated bypass).

Частично (остаток cross-file/домен, открыты): #1593 #1635 #1640.
needs-leha (открыт): #1650 (нужна новая миграция вьюхи supply-слоёв).

Verify: tsc --noEmit 0; py_compile OK.

Closes #1569
Closes #1590
Closes #1642
Closes #801
2026-06-17 11:24:34 +05:00
f12c03588d fix(llm): close §19 redaction regex gaps for bare phone / CAPS name / bare SNILS (#1207)
All checks were successful
Deploy / changes (push) Successful in 7s
Deploy / build-frontend (push) Has been skipped
CI / frontend-tests (push) Has been skipped
CI / frontend-tests (pull_request) Has been skipped
Deploy / build-worker (push) Successful in 2m54s
CI / changes (push) Successful in 8s
CI / changes (pull_request) Successful in 6s
CI / backend-tests (push) Successful in 6m22s
CI / backend-tests (pull_request) Successful in 6m25s
Deploy / build-backend (push) Successful in 1m46s
Deploy / deploy (push) Successful in 1m10s
`_PHONE_RE` требовал префикс +7/8 + хотя бы один разделитель → copy-paste форма
«79221234567»/«89221234567» (самая частая) уходила в OpenAI как есть. `_FULLNAME_RE`
требовал строчные после первой буквы → стандарт ЕГРН/паспортных выписок «ИВАНОВ
ИВАН ИВАНОВИЧ» не матчил. СНИЛС без разделителей не покрывался вообще (`_SNILS_RE`
ждёт формат «NNN-NNN-NNN NN»). Для chat-пути scrub — единственный барьер перед
OpenAI (SafePayload.text без allowlist'а), при llm_enabled=True PII покидает РФ
(нарушение §19/152-ФЗ).

Patch:
- _PHONE_BARE_RE: (?<!\d)[78]\d{10}(?!\d) — 11-значные с префиксом 7/8.
- _SNILS_BARE_RE: (?<!\d)\d{11}(?!\d) — любые 11-значные без разделителей.
- _FULLNAME_RE: альтернатива в каждом слове — Titlecase | CAPS-2+.
- Конфликт phone-bare vs SNILS-bare разрешён порядком в _PII_PATTERNS: phone
  раньше SNILS (7/8-префикс семантически точнее).
- INN (ровно 10/12) с SNILS-bare не пересекается по длине.

8 новых юнит-тестов покрывают все три дыры + конфликт-резолюцию. 24/24 redaction
тестов + 55/55 LLM-suite зелёные.

Closes #1207
2026-06-13 09:56:58 +05:00
4af7ba5a40 feat(llm): foundational LLM infra package with §19 redaction + deterministic fallback (#960)
All checks were successful
CI / changes (pull_request) Successful in 5s
CI / backend-tests (push) Successful in 6m25s
CI / backend-tests (pull_request) Successful in 6m20s
Deploy / changes (push) Successful in 5s
Deploy / build-frontend (push) Has been skipped
Deploy / build-backend (push) Successful in 1m38s
Deploy / build-worker (push) Successful in 2m42s
Deploy / deploy (push) Successful in 1m13s
CI / changes (push) Successful in 5s
CI / frontend-tests (push) Has been skipped
CI / frontend-tests (pull_request) Has been skipped
Optional external-OpenAI layer over the deterministic forecasting engine. Gated by
llm_enabled (default False) so prod makes no network calls until deliberately enabled.
Allowlist-first SafePayload contract + is_confidential hard-block + RU-PII regex scrub
(mandatory on the external path). Abstract LLMProvider seam (is_external) for a future
RU-hosted provider. Sync httpx core (Celery-friendly); tool/function-calling pass-through;
timeout + bounded 429/5xx retry + per-request call cap, all degrading to fallback.
Raw httpx (no openai SDK -> no pyproject/lock drift). 47 tests, ruff + mypy clean.

Refs #960
2026-06-08 15:44:16 +05:00