diff --git a/.claude/commands/work-as-analyst.md b/.claude/commands/work-as-analyst.md index 5fbd91ba..877d0a23 100644 --- a/.claude/commands/work-as-analyst.md +++ b/.claude/commands/work-as-analyst.md @@ -44,7 +44,7 @@ Write-Host "✓ PAT belongs to $($me.login) — analyst persona ready" Следую правилам из `.claude/agents/auto-analyst.md` + `.claude/agents/_autonomous_pickup.md`. -**Что делаю каждый /loop tick (30m):** +**Что делаю каждый /loop tick (15m):** 1. Kill-switch check (label `pause-bots` на repo) 2. Read новые commits + vault inbox + closed-since-last-tick Forgejo issues diff --git a/.claude/commands/work-as-reviewer.md b/.claude/commands/work-as-reviewer.md index 7429e3d4..99a53585 100644 --- a/.claude/commands/work-as-reviewer.md +++ b/.claude/commands/work-as-reviewer.md @@ -40,7 +40,7 @@ curl -sH "Authorization: token $FORGEJO_TOKEN" "$FORGEJO_URL/api/v1/user/tokens" Следую правилам из `.claude/agents/auto-code-reviewer.md` + `_autonomous_pickup.md` + `.claude/agents/code-reviewer.md` + `.claude/rules/git-pr.md`. -**Per-tick workflow (5m):** +**Per-tick workflow (2m):** 1. Kill-switch check 2. GET pulls `status/review` без approve, oldest first, limit=1 diff --git a/.claude/rules/backend.md b/.claude/rules/backend.md index 39aee438..d4978802 100644 --- a/.claude/rules/backend.md +++ b/.claude/rules/backend.md @@ -1,5 +1,7 @@ --- -paths: backend/**/*.py +paths: + - backend/**/*.py + - tradein-mvp/backend/**/*.py --- # Backend conventions — Python 3.12 / FastAPI diff --git a/.claude/rules/deploy.md b/.claude/rules/deploy.md index 84480ea7..cba0d64f 100644 --- a/.claude/rules/deploy.md +++ b/.claude/rules/deploy.md @@ -23,10 +23,11 @@ paths: Reference incident: PR #346 (2026-05-18) deploy → user сам нашёл prod 500 на by-bbox, потом 400 на analyze, потом TypeError на poi-score, потом UI overlap. Каждое ловилось бы playwright smoke по `/site-finder/analysis/{cad}`. -## GHA path triggers +## Path triggers (Forgejo Actions, `.forgejo/workflows/`) -- `backend/**`, `frontend/**`, `Caddyfile`, `docker-compose.prod.yml`, `data/sql/*.sql` → `deploy.yml` (main stack) -- `docker-compose.obsidian.yml`, `scripts/setup-couchdb.sh` → `deploy-obsidian.yml` +- `backend/**`, `frontend/**`, `Caddyfile`, `caddy/**`, `docker-compose.prod.yml`, `data/sql/**`, `ops/glitchtip-auth-forwarder/**`, `.forgejo/workflows/deploy.yml` → `deploy.yml` (main Site Finder stack) +- trade-in изменения → `deploy-tradein.yml` (отдельный stack; paths-filter base = last deployed SHA → накопленный diff, fail-safe build-all) +- `docker-compose.obsidian.yml`, `scripts/setup-couchdb.sh` → `.github/workflows/deploy-obsidian.yml` - `docs/**` alone → НЕ триггерит деплой ## После изменения .env на VPS diff --git a/.claude/rules/frontend.md b/.claude/rules/frontend.md index 6aad4009..4c1a4f84 100644 --- a/.claude/rules/frontend.md +++ b/.claude/rules/frontend.md @@ -1,5 +1,7 @@ --- -paths: frontend/**/*.{ts,tsx,jsx,js} +paths: + - frontend/**/*.{ts,tsx,jsx,js} + - tradein-mvp/frontend/**/*.{ts,tsx,jsx,js} --- # Frontend conventions — Next.js 15 / React 19 / TypeScript 5 @@ -30,7 +32,7 @@ Reference: vault `Bug_RenderMarkdown_JavascriptUrl_May14` (`renderMarkdown.ts:sa cd frontend && npm run codegen ``` -Обновляет `src/types/openapi.ts` из live OpenAPI. Без этого frontend build упадёт на missing types. +Обновляет `src/lib/api-types.ts` из live OpenAPI (`npm run codegen` = `openapi-typescript … -o src/lib/api-types.ts`). Без этого frontend build упадёт на missing types. ## package.json + lockfile sync (CRITICAL — deploy aborts on mismatch) diff --git a/.claude/rules/git-pr.md b/.claude/rules/git-pr.md index eb1b0219..fddbd6dd 100644 --- a/.claude/rules/git-pr.md +++ b/.claude/rules/git-pr.md @@ -45,8 +45,8 @@ paths: - **Imperative**: "fix crash" не "fixed crash" - **Body — почему**, не что (что видно в diff) - **NO `Co-Authored-By: Claude ...`** — никогда (~/.claude/CLAUDE.md rule) -- Workers оставляют staged — main session коммитит -- **No auto-commit**: написать commit message в чат, user решает когда коммитить +- Workers (subagents) оставляют staged — main session коммитит +- **GenDesign (Mera/Ptica) = full self-service:** main/solo session коммитит → пушит → PR → **merge сам** (см. § Auto-merge policy). `balance_platform` — наоборот: только stage, commit message в чат, не коммитить/пушить/мержить ## PR body template @@ -85,11 +85,11 @@ Closes #N ## Auto-merge policy -**Любой scope** — bot мержит при `verdict=approve` + SHA match. Blocked-list снят 2026-05-16 ([Auto-merge any scope] memory rule). +**Self-merge разрешён (2026-06-27, Mera/Ptica).** Любая GenDesign-сессия — solo/foreground ИЛИ bot-pipeline — мержит свой PR сама (любой scope), когда checks зелёные. В bot-pipeline review остаётся (reviewer-окно ставит `verdict=approve` + SHA match), но merge-authority больше **не** эксклюзив reviewer'а — worker может смержить approved PR сам. Pre-merge gate: зелёный CI + (в pipeline) approve+SHA match. `balance_platform` — никогда не мержит (stage only). -**Жёсткие исключения** (даже при APPROVE — НЕ merge, ping user): -- Diff содержит литеральный secret/token/password/credential (40-char hex, API keys, JWT, и т.д.) — security tripwire -- PR меняет блок `## Auto-merge policy` в этом файле, `Critical workflow rules` в CLAUDE.md, `_autonomous_pickup.md` (claim/kill-switch/merge-FSM contract), `auto-code-reviewer.md` или любой `work-as-*.md` (self-extending guard, decided 2026-05-24, расширен 2026-05-29 — изменение правил пайплайна всегда через human, предотвращает bot-loop где bot сам расширяет свои merge права) +**Жёсткие исключения (даже при зелёном — НЕ merge, ping human):** +- Diff содержит литеральный secret/token/password/credential (40-char hex, API keys, JWT, и т.д.) — security tripwire. +- PR меняет правила самого пайплайна: блок `## Auto-merge policy` здесь, `Critical rules` в CLAUDE.md, `_autonomous_pickup.md` (claim/kill-switch/merge-FSM), `auto-code-reviewer.md` или любой `work-as-*.md` — **self-extending guard** (расширение/снятие собственных merge-прав всегда через human, предотвращает bot-loop). ## Sequential PRs @@ -122,7 +122,7 @@ Issues ≥ 1.5 day → 3-4 sub-PRs: **Foundation → Schema → Workers → Inte ## Запреты - ❌ `git push forgejo main` / direct push в main -- ❌ `mcp__forgejo__merge_pull_request` без approval (human "merge it" или bot verdict=approve + SHA match) +- ❌ merge PR с литеральным secret в diff ИЛИ PR меняющий правила пайплайна (self-extending guard) — это через human. Иначе self-merge OK (зелёный CI; в pipeline дополнительно approve+SHA) - ❌ `gh pr *` — bypassed 2026-05-16, используй Forgejo MCP или curl + `$FORGEJO_TOKEN` - ❌ `--no-verify` / `--amend` / `--no-edit` / `--force` без явного approval - ❌ `@claude` в PR comments — plain text only (`feedback_no_claude_mentions`) diff --git a/.claude/rules/sql.md b/.claude/rules/sql.md index b657593d..f221b4ef 100644 --- a/.claude/rules/sql.md +++ b/.claude/rules/sql.md @@ -1,5 +1,7 @@ --- -paths: data/sql/**/*.sql +paths: + - data/sql/**/*.sql + - tradein-mvp/backend/data/sql/**/*.sql --- # SQL conventions — PostgreSQL 16 / PostGIS 3.4 diff --git a/.claude/rules/tradein.md b/.claude/rules/tradein.md new file mode 100644 index 00000000..f54f8d34 --- /dev/null +++ b/.claude/rules/tradein.md @@ -0,0 +1,53 @@ +--- +paths: + - tradein-mvp/**/*.py + - tradein-mvp/**/*.sql + - tradein-mvp/frontend/**/*.{ts,tsx} +--- + +# trade-in (Mera) conventions — `tradein-mvp/` + +Отдельный продукт + отдельный стек от Site Finder. Backend `tradein-mvp/backend/app/**`, +SQL `tradein-mvp/backend/data/sql/NN_*.sql`, frontend `tradein-mvp/frontend/`. Backend Python +подчиняется `.claude/rules/backend.md` (psycopg v3, CAST, ruff-100), SQL — `.claude/rules/sql.md` +(NN naming, idempotency). Ниже — то, что СПЕЦИФИЧНО для trade-in. + +## Две БД — не путай + +- **`postgres-tradein`** (db=tradein) — скрейпленные листинги avito/cian/yandex, estimator, + coverage, houses. Для ЛЮБОЙ tradein-задачи метрики/схему бери отсюда (`mcp__postgres-tradein__*`). +- **`postgres-gendesign`** (db=gendesign) — Site Finder, НЕ trade-in. + +## Тестировать HTTP только ВНУТРИ контейнера + +SSH-туннель `localhost:8000` → `gendesign-backend` (Site Finder, db=gendesign, старый код), +**НЕ** tradein-backend (порт не опубликован на хост). curl на туннель:8000 по trade-in endpoint = +мусор / чужая БД (стоило ~2ч). Тест trade-in API только изнутри контейнера: + +```bash +ssh gendesign # затем: +docker exec tradein-backend curl -s localhost:8000/ # админ-роуты: -H "X-Authenticated-User: admin" +docker exec tradein-postgres psql -U -d tradein -c "..." +``` + +## Scheduler крутится в `tradein-scraper`, не `tradein-backend` + +In-app scheduler (`scrape_schedules`, tick 60s, `python -m app.scheduler_main`, +`SCHEDULER_ENABLE=true`) живёт в контейнере **`tradein-scraper`**; в `tradein-backend` намеренно +`false`. Статус scheduled-задач смотри в scraper-контейнере (logs/printenv), не в backend. +Ручной smoke: `UPDATE scrape_schedules SET next_run_at=now() WHERE source='X'` → подхват ≤60s. + +## SQL авто-применяется на ПРОД (strict) + +`tradein-mvp/backend/data/sql/NN_*.sql` применяется автоматически на деплое через `_schema_migrations` +в `.forgejo/workflows/deploy-tradein.yml` (НЕ init-only, strict exit-1). Idempotency критична — +деструктивный DDL хитит прод на деплое. NN-нумерация уже 3-значная и ИМЕЕТ коллизии (`108_*` ×2, +`084_*` ×2) → перед новым файлом `ls tradein-mvp/backend/data/sql | grep '^NN'` на дубль basename, +не доверяй `tail`. + +## Rapid-merge trap + +2 tradein-PR мержа за секунды → backend `test`-job cancelled → `build-backend` пропущен → +«deploy success» на СТАРОМ образе (нет нового кода/deps). Сверяй `:latest` Created-timestamp vs +время мержа + smoke в контейнере; не верь «деплой прошёл». Recovery: ручной `workflow_dispatch` +для `deploy-tradein.yml`. diff --git a/.forgejo/workflows/ci.yml b/.forgejo/workflows/ci.yml index ff1270b0..48a47220 100644 --- a/.forgejo/workflows/ci.yml +++ b/.forgejo/workflows/ci.yml @@ -276,14 +276,17 @@ jobs: working-directory: backend run: uv run python -c "import json; from app.main import app; print(json.dumps(app.openapi()))" > /tmp/openapi.json - - name: Regenerate api-types.ts + format (prettier defaults) + - name: Regenerate api-types.ts + format (project-local pinned prettier) working-directory: frontend # 1) openapi-typescript из дампнутого файла (эквивалент `npm run codegen`, - # который читает ту же схему по URL). 2) prettier (defaults, как - # pre-commit hook) → формат совпадает с committed-файлом. + # который читает ту же схему по URL). 2) ./node_modules/.bin/prettier — + # PROJECT-LOCAL, pinned (prettier 3.9.0 в devDependencies). НЕ `npx + # prettier` (тот резолвится в плавающий latest и расходится с pre-commit, + # ломая этот gate). Pre-commit hook гоняет тот же локальный prettier 3.9.0 + # → байт-в-байт идентичный формат. См. .pre-commit-config.yaml. run: | npx openapi-typescript /tmp/openapi.json -o src/lib/api-types.ts - npx prettier --write src/lib/api-types.ts + ./node_modules/.bin/prettier --write src/lib/api-types.ts - name: Assert api-types.ts is up-to-date working-directory: frontend diff --git a/.pre-commit-config.yaml b/.pre-commit-config.yaml index 08770073..f085b541 100644 --- a/.pre-commit-config.yaml +++ b/.pre-commit-config.yaml @@ -30,10 +30,17 @@ repos: - id: ruff-format files: ^(backend|tradein-mvp/backend)/ - # Frontend — prettier on TS/TSX/JSON + # Frontend — prettier on TS/TSX/JSON. + # additional_dependencies pins the EXACT prettier engine (3.9.0) so this hook + # and CI (`openapi-codegen-check` → ./node_modules/.bin/prettier, also 3.9.0 + # via frontend/package.json) format byte-for-byte identically. Без явного pin + # mirrors-prettier@v4.0.0-alpha.8 тянет CLI-обёртку, а CI `npx prettier` + # плавает в latest → расхождение формата → codegen-gate RED. Меняешь версию + # здесь — синхронно меняй prettier в frontend/package.json + lockfile. - repo: https://github.com/pre-commit/mirrors-prettier rev: v4.0.0-alpha.8 hooks: - id: prettier + additional_dependencies: ["prettier@3.9.0"] files: ^frontend/.*\.(ts|tsx|js|jsx|json|css|md)$ exclude: ^frontend/(node_modules|\.next|package-lock\.json)/ diff --git a/CLAUDE.md b/CLAUDE.md index 5d15a102..60980af9 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -41,6 +41,8 @@ Live: `https://gendsgn.ru/` — Свердловская обл. (ЕКБ, ПЗЗ | `database-expert` | `data/sql/**.sql`, Alembic, EXPLAIN ANALYZE | | `devops-engineer` | `docker-compose*.yml`, `Caddyfile`, `.github/workflows/**`, `.forgejo/workflows/**` | | `code-reviewer` | Pre-push lint (security, correctness, conventions) | +| `deep-code-reviewer` | Тщательный review критичных PR (миграции / auth / scrapers) + merge authority при ✅ APPROVE | +| `qa-tester` | Post-deploy smoke (playwright / curl / SQL) сразу после merge+deploy — rule #7 | **Routing:** тривиально (typo, 1-line) → main session. Single-domain clear → worker. Cross-domain / нечётко → `tech-analyst` first. Worker → `code-reviewer` → main commits → push → PR. diff --git a/auth/roles.yaml b/auth/roles.yaml index eaf32a08..54b719fb 100644 --- a/auth/roles.yaml +++ b/auth/roles.yaml @@ -54,6 +54,13 @@ roles: - "/admin/**" - "/api/v1/admin/**" - "/trade-in/api/v1/admin/**" + expired: + # Пробный доступ закончился — нет доступа ни к чему. Аккаунт остаётся в + # caddy/users.caddy.snippet (basic_auth), чтобы дойти до фронта и увидеть + # сообщение; /me отдаёт role=expired → RouteGuard рендерит trial-экран. + paths: [] + deny: + - "/**" # NB: реальные analyst-логины ДОЛЖНЫ быть добавлены и в # caddy/users.caddy.snippet (Caddy basic_auth) силами devops — иначе Caddy не @@ -72,7 +79,7 @@ users: user8: pilot user9: pilot user10: pilot - praktika: pilot # пилот-аккаунт агентства «Практика» 2026-05-29 + praktika: expired # пробный доступ закончился 2026-06-27 — см. NoAccessScreen variant="trial" admintest: admin # temp QA 2026-05-26 pilottest: pilot # temp QA 2026-05-26 analysttest: analyst # temp QA 2026-06-07 (#962) diff --git a/frontend/package-lock.json b/frontend/package-lock.json index 82b9b63f..ebe0ca27 100644 --- a/frontend/package-lock.json +++ b/frontend/package-lock.json @@ -40,6 +40,7 @@ "jsdom": "^25.0.1", "openapi-typescript": "^7.0.0", "postcss": "^8.4.0", + "prettier": "3.9.0", "tailwindcss": "^4.0.0", "typescript": "5.9.3", "vitest": "^2.1.9" @@ -14055,6 +14056,22 @@ "node": ">=0.10.0" } }, + "node_modules/prettier": { + "version": "3.9.0", + "resolved": "https://registry.npmjs.org/prettier/-/prettier-3.9.0.tgz", + "integrity": "sha512-LjIqSIC5VYLzs9WedVmJ2ljNAGnU+DteIClbahu4L/DBeWjZ6iT/k1lAYyu9JUh+1xINxWadaPw/Pl63y/agAw==", + "dev": true, + "license": "MIT", + "bin": { + "prettier": "bin/prettier.cjs" + }, + "engines": { + "node": ">=14" + }, + "funding": { + "url": "https://github.com/prettier/prettier?sponsor=1" + } + }, "node_modules/pretty-format": { "version": "27.5.1", "resolved": "https://registry.npmjs.org/pretty-format/-/pretty-format-27.5.1.tgz", diff --git a/frontend/package.json b/frontend/package.json index e6ef4dc6..1e6c5040 100644 --- a/frontend/package.json +++ b/frontend/package.json @@ -45,6 +45,7 @@ "jsdom": "^25.0.1", "openapi-typescript": "^7.0.0", "postcss": "^8.4.0", + "prettier": "3.9.0", "tailwindcss": "^4.0.0", "typescript": "5.9.3", "vitest": "^2.1.9" diff --git a/frontend/src/lib/api-types.ts b/frontend/src/lib/api-types.ts index a8ad8484..8e505843 100644 --- a/frontend/src/lib/api-types.ts +++ b/frontend/src/lib/api-types.ts @@ -2833,8 +2833,7 @@ export interface components { nspd_risk_zones?: components["schemas"]["RiskZone"][] | null; /** Nspd Opportunity Parcels */ nspd_opportunity_parcels?: - | components["schemas"]["OpportunityParcel"][] - | null; + components["schemas"]["OpportunityParcel"][] | null; /** Nspd Red Lines */ nspd_red_lines?: components["schemas"]["RedLine"][] | null; /** Nspd Dump */ @@ -3752,8 +3751,7 @@ export interface components { * @description Категория: competition|permitting|demand|risk|other */ category?: - | ("competition" | "permitting" | "demand" | "risk" | "other") - | null; + ("competition" | "permitting" | "demand" | "risk" | "other") | null; /** * Is Confidential * @description «Непублично» (§7.13): маркировка чувствительной заметки @@ -3846,8 +3844,7 @@ export interface components { body?: string | null; /** Category */ category?: - | ("competition" | "permitting" | "demand" | "risk" | "other") - | null; + ("competition" | "permitting" | "demand" | "risk" | "other") | null; /** Is Confidential */ is_confidential?: boolean | null; /** District */ @@ -4630,12 +4627,10 @@ export interface components { year_built?: number | null; /** House Type */ house_type?: - | ("panel" | "brick" | "monolith" | "monolith_brick" | "other") - | null; + ("panel" | "brick" | "monolith" | "monolith_brick" | "other") | null; /** Repair State */ repair_state?: - | ("needs_repair" | "standard" | "good" | "excellent") - | null; + ("needs_repair" | "standard" | "good" | "excellent") | null; /** Has Balcony */ has_balcony?: boolean | null; }; @@ -7877,8 +7872,7 @@ export interface operations { cad_num?: string | null; /** @description Фильтр по категории */ category?: - | ("competition" | "permitting" | "demand" | "risk" | "other") - | null; + ("competition" | "permitting" | "demand" | "risk" | "other") | null; /** @description Фильтр по пометке «непублично» */ is_confidential?: boolean | null; /** @description Фильтр по автору */ diff --git a/scripts/claude-hooks/check-secret-read.py b/scripts/claude-hooks/check-secret-read.py new file mode 100644 index 00000000..a7361e70 --- /dev/null +++ b/scripts/claude-hooks/check-secret-read.py @@ -0,0 +1,95 @@ +#!/usr/bin/env python3 +""" +Claude Code PreToolUse hook — blocks Bash commands that READ or TRANSMIT secret +files (.env*, .pgpass, ssh private keys). + +Why: `.claude/settings.json` permissions end with a blanket `Bash(*)` allow, which +sidesteps the `Read(./.env*)` deny rule — `cat backend/.env`, `Get-Content .env`, +`curl -F @backend/.env …` all exfiltrate prod DB passwords + FORGEJO/GLITCHTIP tokens +without a single prompt. This hook closes that bypass at the command layer. + +Wired in `.claude/settings.json` hooks.PreToolUse with matcher "Bash". + +Behavior: +- Reads hook input JSON from stdin; extracts tool_input.command. +- Blocks (exit 2, stderr -> Claude) iff the command references a REAL secret file + AND uses a read/transmit verb, OR `< file` redirection, OR `@file` upload, OR `open(`. +- Template files (.env.example / .sample / .template / .dist / .md) are allowed. +- `ls`/`stat`/`test -f` on a secret file are allowed (they don't read content). +- Fail-OPEN on parse errors (exit 0): this is a tripwire layered on top of the deny + list + (future) sandbox, not the sole guard — better to under-block than to wedge + every Bash call on a malformed payload. +""" +from __future__ import annotations + +import json +import re +import sys + +# A secret-file token: `.env` with any number of dotted segments, `.pgpass`, or an +# ssh private key. The leading delimiter ([start | space | quote | = : < @ ( / ]) +# anchors it as a filename/path component, not a substring of another word. +SECRET_RE = re.compile( + r"(?:^|[\s'\"=:<@(/])" + r"(\.env(?:\.[\w-]+)*|\.pgpass|id_rsa[\w.]*|id_ed25519[\w.]*|id_ecdsa[\w.]*)" + r"(?![\w-])", + re.IGNORECASE, +) +TEMPLATE_SUFFIXES = (".example", ".sample", ".template", ".dist", ".md") + +# Verbs that read a file's content or push it off-box. +READ_VERB_RE = re.compile( + r"(?i)(? bool: + low = token.lower() + return any(low.endswith(s) for s in TEMPLATE_SUFFIXES) + + +def main() -> int: + try: + payload = json.load(sys.stdin) + except Exception: + return 0 + + tool = payload.get("tool_name") or payload.get("tool") or "" + if tool != "Bash": + return 0 + + cmd = (payload.get("tool_input") or {}).get("command") or "" + if not cmd: + return 0 + + hits = [m.group(1) for m in SECRET_RE.finditer(cmd) if not _is_template(m.group(1))] + if not hits: + return 0 + + danger = ( + READ_VERB_RE.search(cmd) + or re.search(r"<\s*\S*\.env\b", cmd, re.IGNORECASE) + or re.search(r"@\S*\.env\b", cmd, re.IGNORECASE) + or re.search(r"open\(\s*['\"][^'\"]*\.env", cmd, re.IGNORECASE) + ) + if not danger: + return 0 + + msg = [ + f"Blocked: команда читает/передаёт секрет-файл ({hits[0]}).", + "Blanket `Bash(*)` обходит `Read(./.env*)` deny — этот PreToolUse-hook закрывает дыру.", + "Секрет-файлы содержат prod DB-пароли + FORGEJO/GLITCHTIP токены.", + "Если правда нужно прочитать — сделай это вручную вне Claude, либо используй `.env.example`.", + ] + print("\n".join(msg), file=sys.stderr) + return 2 + + +if __name__ == "__main__": + sys.exit(main()) diff --git a/scripts/claude-hooks/check-sql-pitfalls.py b/scripts/claude-hooks/check-sql-pitfalls.py new file mode 100644 index 00000000..6871733a --- /dev/null +++ b/scripts/claude-hooks/check-sql-pitfalls.py @@ -0,0 +1,87 @@ +#!/usr/bin/env python3 +""" +Claude Code PostToolUse hook — blocks `import psycopg2` in backend Python. + +GenDesign uses psycopg v3 (`psycopg[binary]>=3.2.0`); psycopg2 is NOT installed, +so `import psycopg2` is a guaranteed `ModuleNotFoundError` at runtime — a recurring +bug class (see `.claude/rules/backend.md`). This catches it at edit-time, not on CI. + +NOTE: the related `:param::type` cast trap is deliberately NOT checked here — the +codebase documents that anti-pattern in dozens of comments/test-docstrings +("never :param::type"), so a regex would false-positive constantly. That trap is +already covered by dedicated unit tests. + +Wired in `.claude/settings.json` hooks.PostToolUse with matcher "Edit|Write|MultiEdit". +Watches `backend/**/*.py` (also `tradein-mvp/backend/**/*.py`, normalized to `backend/`). +""" +from __future__ import annotations + +import json +import re +import sys +from pathlib import Path + +WATCH_PATH_PREFIX = "backend/" +# Real import statement at line start (after optional indent); not a comment or a +# docstring sentence like "never import psycopg2" (which starts with a word, not `import`). +PSYCOPG2_RE = re.compile( + r"(?m)^\s*(?:import\s+psycopg2|from\s+psycopg2(?:\.[\w.]+)?\s+import)\b" +) + + +def normalize_path(raw: str) -> str: + """Repo-relative POSIX path anchored on the first `backend` segment.""" + parts = Path(raw).parts + try: + idx = parts.index("backend") + except ValueError: + return Path(raw).as_posix() + return Path(*parts[idx:]).as_posix() + + +def main() -> int: + try: + payload = json.load(sys.stdin) + except Exception: + return 0 + + tool_name = payload.get("tool_name") or payload.get("tool") or "" + if tool_name not in {"Edit", "Write", "MultiEdit"}: + return 0 + + tool_input = payload.get("tool_input") or {} + file_path = tool_input.get("file_path") or tool_input.get("path") or "" + if not file_path: + return 0 + + norm = normalize_path(file_path) + if not norm.startswith(WATCH_PATH_PREFIX) or not norm.endswith(".py"): + return 0 + + edits = tool_input.get("edits") + if isinstance(edits, list): + new_text = "\n".join( + str(e.get("new_string") or "") for e in edits if isinstance(e, dict) + ) + else: + new_text = tool_input.get("new_string") or tool_input.get("content") or "" + if not new_text: + return 0 + + for lineno, line in enumerate(new_text.splitlines(), start=1): + if line.lstrip().startswith("#"): + continue + if PSYCOPG2_RE.match(line): + msg = [ + f"Blocked: `import psycopg2` в {norm} (line {lineno}).", + "psycopg2 НЕ установлен → ModuleNotFoundError. Используй `import psycopg` (v3).", + "Bulk INSERT: `cur.executemany()` / COPY (НЕ execute_values). См. .claude/rules/backend.md.", + ] + print("\n".join(msg), file=sys.stderr) + return 2 + + return 0 + + +if __name__ == "__main__": + sys.exit(main()) diff --git a/scripts/claude-hooks/session-preflight.py b/scripts/claude-hooks/session-preflight.py new file mode 100644 index 00000000..378315b3 --- /dev/null +++ b/scripts/claude-hooks/session-preflight.py @@ -0,0 +1,43 @@ +#!/usr/bin/env python3 +""" +Claude Code SessionStart hook — one-line preflight status printed into session context. + +Surfaces the session-setup facts that have caused real lost time: +- FORGEJO_ACCESS_TOKEN present? (goern forgejo-MCP needs it BEFORE launch; missing -> idle bot) +- OBSIDIAN_API_KEY present? (vault MCP) +- DB tunnel localhost:15432 reachable? ("tunnel :8000 is gendesign not tradein" / tunnel-down traps) + +Wired in `.claude/settings.json` hooks.SessionStart. ALWAYS exits 0 — informational only, +never blocks. Fast (<~0.6s): single non-blocking TCP probe with a short timeout. +""" +from __future__ import annotations + +import os +import socket +import sys + + +def _tcp_ok(host: str, port: int, timeout: float = 0.5) -> bool: + try: + with socket.create_connection((host, port), timeout=timeout): + return True + except OSError: + return False + + +def main() -> int: + try: + forgejo = "set" if os.environ.get("FORGEJO_ACCESS_TOKEN") else "MISSING" + obsidian = "set" if os.environ.get("OBSIDIAN_API_KEY") else "MISSING" + tunnel = "up" if _tcp_ok("localhost", 15432) else "DOWN" + print( + f"[preflight] FORGEJO_ACCESS_TOKEN={forgejo} · OBSIDIAN_API_KEY={obsidian} " + f"· db-tunnel(:15432)={tunnel}" + ) + except Exception: + pass + return 0 + + +if __name__ == "__main__": + sys.exit(main()) diff --git a/tradein-mvp/backend/app/core/auth.py b/tradein-mvp/backend/app/core/auth.py index 06f896b6..a548e9e6 100644 --- a/tradein-mvp/backend/app/core/auth.py +++ b/tradein-mvp/backend/app/core/auth.py @@ -31,7 +31,7 @@ import yaml logger = logging.getLogger(__name__) -Role = Literal["admin", "pilot", "analyst"] +Role = Literal["admin", "pilot", "analyst", "expired"] class UserScope(TypedDict): diff --git a/tradein-mvp/backend/app/services/estimator.py b/tradein-mvp/backend/app/services/estimator.py index 4b0742cc..0fb4ee94 100644 --- a/tradein-mvp/backend/app/services/estimator.py +++ b/tradein-mvp/backend/app/services/estimator.py @@ -26,6 +26,8 @@ import logging import math import re import time +from collections.abc import Callable +from dataclasses import dataclass from datetime import UTC, date, datetime, timedelta from typing import Any from uuid import uuid4 @@ -1936,6 +1938,645 @@ async def _with_budget(coro: Any, budget_s: float, *, label: str) -> Any: return None +# ── PricingResult dataclass (pure, no I/O) ────────────────────────────────── + + +@dataclass +class PricingResult: + """Return type of _price_from_inputs — все переменные, нужные estimate_quality после блока.""" + + median_ppm2: float + median_price: int + range_low: int + range_high: int + n_analogs: int + confidence: str + explanation: str + anchor_tier: str | None + anchor_comps_used: list[dict] + avito_imv_summary: AvitoImvSummary | None + dkp_corridor: DkpCorridor | None + expected_sold_per_m2: int | None + expected_sold_price: int | None + expected_sold_range_low: int | None + expected_sold_range_high: int | None + asking_to_sold_ratio: float | None + ratio_basis: str | None + sources_used_pre: list[str] + listings_clean: list[dict] + + +def _price_from_inputs( + *, + listings: list[dict], + area_m2: float, + rooms: int | None, + repair_state: str | None, + floor: int | None, + total_floors: int | None, + target_year: int | None, + analog_tier: str, + fallback_used: bool, + area_widened: bool, + anchor_comps: list[dict], + anchor_tier_fetched: str | None, + dkp_raw: dict | None, + imv_anchor: dict | None, + imv_eval: IMVEvaluation | None, + yandex_val_present: bool, + cian_val_present: bool, + ratio_resolver: Callable[[float | None], tuple[float | None, str | None]], + quarter_index_lookup: Callable[[str], tuple[float, int] | None], + quarter_indexes_lookup: Callable[[list[str]], dict[str, float]], + target_house_cadnum: str | None, + dadata_coarse: bool, + geo: GeocodeResult, + dadata_qc_geo: int | None, +) -> PricingResult: + """Deterministic pricing orchestration — pure, synchronous, zero I/O. + + Extracted from estimate_quality (#1966) to enable offline backtesting and + direct unit testing. All DB lookups are injected via callables or pre-fetched + values; behavior is byte-identical to the original block. + """ + # 3. Outlier filter + listings_clean = _filter_outliers(listings) + + # 4. Aggregation + if listings_clean: + prices_ppm2 = sorted(lot["price_per_m2"] for lot in listings_clean if lot["price_per_m2"]) + median_ppm2 = _percentile(prices_ppm2, 0.5) + q1_ppm2 = _percentile(prices_ppm2, 0.25) + q3_ppm2 = _percentile(prices_ppm2, 0.75) + median_price = int(median_ppm2 * area_m2) + range_low = int(q1_ppm2 * area_m2) + range_high = int(q3_ppm2 * area_m2) + # #2: n_analogs считается по prices_ppm2, а не len(listings_clean). + n_analogs = len(prices_ppm2) + else: + median_ppm2 = 0.0 + q1_ppm2 = 0.0 + q3_ppm2 = 0.0 + median_price = 0 + range_low = 0 + range_high = 0 + n_analogs = 0 + + # 4b. Repair coefficient + repair_coef = _repair_coefficient(repair_state) + repair_note = "" + if listings_clean and repair_coef != 1.0: + median_price = int(median_price * repair_coef) + range_low = int(range_low * repair_coef) + range_high = int(range_high * repair_coef) + median_ppm2 = median_ppm2 * repair_coef + pct = round((repair_coef - 1.0) * 100) + repair_note = ( + f" Цена скорректирована на состояние ремонта " + f"({_REPAIR_LABEL.get(repair_state, '')} {pct:+d}%)." + ) + + # Build sources_used_pre from listings + external sources + sources_used_pre = sorted({lot.get("source") for lot in listings_clean if lot.get("source")}) + if imv_eval is not None: + sources_used_pre = sorted(set(sources_used_pre) | {"avito_imv"}) + if yandex_val_present: + sources_used_pre = sorted(set(sources_used_pre) | {"yandex_valuation"}) + if cian_val_present: + sources_used_pre = sorted(set(sources_used_pre) | {"cian_valuation"}) + + # 4c. asking→sold variables — initialized; actual computation is after all mutations. + asking_to_sold_ratio: float | None = None + ratio_basis: str | None = None + expected_sold_per_m2: int | None = None + expected_sold_price: int | None = None + expected_sold_range_low: int | None = None + expected_sold_range_high: int | None = None + + confidence, explanation = _compute_confidence( + n_analogs, + median_ppm2, + q1_ppm2 if listings_clean else 0, + q3_ppm2 if listings_clean else 0, + fallback_used, + area_widened, + listings=listings_clean, + ) + + # Tier note — информируем пользователя о качестве house-match + tier_note = "" + if analog_tier == "S": + tier_note = " (аналоги из того же дома)" + elif analog_tier == "H": + tf_str = f"{total_floors}-эт." if total_floors is not None else "" + yr_str = f"{target_year}±15 г." if target_year else "" + parts_str = ", ".join(p for p in [yr_str, tf_str] if p) + tier_note = f" (аналоги из домов того же класса: {parts_str})" if parts_str else "" + else: + tier_note = " (нет аналогов в том же доме/классе — расширили поиск)" + + explanation = (explanation or "") + tier_note + repair_note + + # ── #651/#652 v2: same-building anchor ─────────────────────────────────── + anchor_tier: str | None = anchor_tier_fetched + anchor_comps_used: list[dict] = [] + + anchor = _compute_same_building_anchor( + anchor_comps, + area_target=area_m2, + rooms_target=rooms, + tier=anchor_tier or "", + sigma=settings.estimate_sb_area_sigma, + rooms_boost=settings.estimate_sb_rooms_match_boost, + floor_target=floor, + total_floors_target=total_floors, + floor_sigma=settings.estimate_sb_floor_sigma, + min_comps=settings.estimate_sb_min_comps, + mad_k=settings.estimate_sb_mad_k, + ) + + # #1795 шаг 3: гейт Tier C. + if ( + anchor is not None + and anchor_tier == "C" + and settings.estimate_anchor_tier_c_corridor_mult > 0 + ): + if dkp_raw is not None and dkp_raw.get("high_ppm2", 0) > 0: + corridor_high_for_gate = float(dkp_raw["high_ppm2"]) + else: + corridor_high_for_gate = (median_ppm2 / repair_coef) * 1.3 if repair_coef else 0.0 + gate_threshold = corridor_high_for_gate * settings.estimate_anchor_tier_c_corridor_mult + if gate_threshold > 0 and anchor["anchor_ppm2"] > gate_threshold: + logger.info( + "sb_anchor Tier C gate #1795: anchor_ppm2=%d > corridor_high×%.1f=%d" + " → keep radius median (anchor suppressed)", + int(anchor["anchor_ppm2"]), + settings.estimate_anchor_tier_c_corridor_mult, + int(gate_threshold), + ) + anchor = None + + # #audit-1: low-confidence gate. + if anchor is not None and settings.estimate_sb_low_conf_gate_enabled: + gate_low = anchor["confidence"] == "low" + gate_thin = ( + anchor["n"] < settings.estimate_sb_gate_min_n + and anchor["fsd"] > settings.estimate_sb_gate_max_fsd + ) + if gate_low or gate_thin: + logger.info( + "sb_anchor low-conf gate #audit-1: tier=%s n=%d fsd=%.3f conf=%s" + " → suppressed (gate_low=%s gate_thin=%s) → radius fallback", + anchor_tier, + anchor["n"], + anchor["fsd"], + anchor["confidence"], + gate_low, + gate_thin, + ) + anchor = None + anchor_tier = None + + if anchor is not None: + # #694: якорь мутирует headline — UI-аналоги должны отражать ЭТИ комплы. + anchor_comps_used = anchor_comps + est_ppm2 = anchor["anchor_ppm2"] + # PREMIUM GUARDRAIL (hard). + floor_ppm2 = anchor["comp_min_ppm2"] * (1.0 - settings.estimate_sb_guardrail_tol) + if est_ppm2 < floor_ppm2: + est_ppm2 = floor_ppm2 + new_ppm2 = est_ppm2 * repair_coef + point = int(new_ppm2 * area_m2) + # FSD-диапазон. + half = settings.estimate_fsd_k * anchor["fsd"] + new_range_low = int(point * max(0.0, 1.0 - half)) + new_range_high = int(point * (1.0 + half)) + # Спред комплов. + spread_low = int(anchor["comp_min_ppm2"] * repair_coef * area_m2) + spread_high = int(anchor["comp_max_ppm2"] * repair_coef * area_m2) + new_range_low = min(new_range_low, spread_low, point) + new_range_high = max(new_range_high, spread_high, point) + + logger.info( + "sb_anchor: tier=%s n=%d radius_median_ppm2=%d → anchor_asking_ppm2=%d" + " (uplift=%s haircut=%.2f) point %d → %d", + anchor_tier, + anchor["n"], + int(median_ppm2), + int(est_ppm2), + anchor["used_uplift"], + anchor["haircut"], + median_price, + point, + ) + median_ppm2 = new_ppm2 + median_price = point + range_low = new_range_low + range_high = new_range_high + confidence = anchor["confidence"] + tier_label = "того же дома" if anchor_tier == "A" else "ближайшего окружения (≤500 м)" + # #695: explanation описывает ИМЕННО якорные комплы. + explanation = ( + f"Оценка построена по {anchor['n']} аналогам из {tier_label}" + f"{' (топ-уровень в доме)' if anchor['used_uplift'] else ''}." + ) + repair_note + # #695 (QA fixup): n_analogs по anchor-популяции. + n_analogs = anchor["n"] + # #1871 P1: ghost-anchor guard. + if not listings_clean and confidence != "low": + logger.warning( + "estimator #1871 ghost-anchor guard: confidence %s→low " + "(anchor_n=%s, radius_analogs=0)", + confidence, + anchor["n"], + ) + confidence = "low" + explanation += ( + " Оценка опирается только на аналоги из того же дома — " + "сопоставимых предложений поблизости не найдено, поэтому " + "точность ориентировочная." + ) + # #1871 P2: split-дома wide-corridor disclosure. + if ( + settings.estimate_wide_corridor_disclosure_enabled + and anchor_tier == "A" + and median_price > 0 + ): + corridor_pct = (range_high - range_low) / median_price + if corridor_pct > settings.estimate_wide_corridor_threshold: + confidence = _downgrade_confidence(confidence) + explanation = (explanation or "") + ( + " Очень широкий ценовой диапазон по дому (вероятно, дом " + "разбит на секции разной этажности или разнородный фонд) — " + "оценка ориентировочная, уточните по конкретной секции." + ) + + # ── #651: IMV / Yandex blend (Tier D only, anchor_tier is None) ────────── + imv_anchor_present: bool = False + avito_imv_summary: AvitoImvSummary | None = None + if ( + anchor_tier is None + and settings.estimate_imv_blend_enabled + and listings_clean + and median_price > 0 + ): + anchor_total: int | None = None + anchor_higher: int | None = None + anchor_label: str | None = None + if imv_anchor is not None and imv_anchor.get("recommended_price"): + anchor_total = int(imv_anchor["recommended_price"]) + anchor_higher = ( + int(imv_anchor["higher_price"]) if imv_anchor.get("higher_price") else None + ) + anchor_label = "оценке Avito IMV" + _imv_mc = int(imv_anchor["market_count"]) if imv_anchor.get("market_count") else None + avito_imv_summary = AvitoImvSummary( + recommended_price=anchor_total, + lower_price=( + int(imv_anchor["lower_price"]) if imv_anchor.get("lower_price") else None + ), + higher_price=anchor_higher, + market_count=_imv_mc, + thin_market=( + _imv_mc is not None and _imv_mc < settings.avito_imv_thin_market_threshold + ), + ) + elif imv_eval is not None and imv_eval.recommended_price: + anchor_total = int(imv_eval.recommended_price) + anchor_higher = int(imv_eval.higher_price) if imv_eval.higher_price else None + anchor_label = "оценке Avito IMV" + avito_imv_summary = AvitoImvSummary( + recommended_price=anchor_total, + lower_price=(int(imv_eval.lower_price) if imv_eval.lower_price else None), + higher_price=anchor_higher, + market_count=imv_eval.market_count, + thin_market=( + imv_eval.market_count is not None + and imv_eval.market_count < settings.avito_imv_thin_market_threshold + ), + ) + + # #audit-5b: thin-market warning. + if avito_imv_summary is not None and avito_imv_summary.thin_market: + logger.warning( + "avito_imv thin_market #audit-5b: market_count=%s" + " (< avito_imv_thin_market_threshold=%d) — IMV reliability low", + avito_imv_summary.market_count, + settings.avito_imv_thin_market_threshold, + ) + + if anchor_total is not None: + imv_anchor_present = True + new_median, new_range_high, new_ppm2, blended, anchor_used = _apply_imv_blend( + median_price=median_price, + range_high=range_high, + median_ppm2=median_ppm2, + area=area_m2, + anchor_total=anchor_total, + anchor_higher=anchor_higher, + weight=settings.estimate_imv_blend_weight, + threshold=settings.estimate_imv_blend_threshold, + ) + if blended: + logger.info( + "imv_blend: median %d → %d (anchor=%d w=%.2f) range_high %d → %d", + median_price, + new_median, + anchor_used, + settings.estimate_imv_blend_weight, + range_high, + new_range_high, + ) + median_price = new_median + median_ppm2 = new_ppm2 + explanation = (explanation or "") + ( + f" Оценка скорректирована по {anchor_label} " + f"({anchor_used / 1_000_000:.1f} млн ₽)." + ) + sources_used_pre = sorted(set(sources_used_pre) | {"avito_imv"}) + # Диапазон расширяем даже если медиану не двигали. + range_high = new_range_high + + # Display-only IMV summary when headline built by same-building anchor. + if anchor_tier is not None and avito_imv_summary is None: + if imv_anchor is not None and imv_anchor.get("recommended_price"): + _disp_mc = int(imv_anchor["market_count"]) if imv_anchor.get("market_count") else None + avito_imv_summary = AvitoImvSummary( + recommended_price=int(imv_anchor["recommended_price"]), + lower_price=( + int(imv_anchor["lower_price"]) if imv_anchor.get("lower_price") else None + ), + higher_price=( + int(imv_anchor["higher_price"]) if imv_anchor.get("higher_price") else None + ), + market_count=_disp_mc, + thin_market=( + _disp_mc is not None and _disp_mc < settings.avito_imv_thin_market_threshold + ), + ) + + # ── #764: per-cadastral-quarter price index gap-correction ─────────────── + if ( + settings.estimate_quarter_index_enabled + and anchor_tier is None # Guard-1a + and not imv_anchor_present # Guard-1b + and median_price > 0 + and area_m2 + ): + target_quarter: str | None = _quarter_from_cadastre(target_house_cadnum) + if target_quarter is None: + for lot in listings_clean: + cq = _quarter_from_cadastre(lot.get("building_cadastral_number")) + if cq is not None: + target_quarter = cq + break + + if target_quarter is not None: + qindex_result = quarter_index_lookup(target_quarter) + if qindex_result is not None: + target_qi, target_n_deals = qindex_result + + # Bimodal/nominal guard (Guard-4). + if ( + target_qi > settings.estimate_quarter_index_max_for_small_n + and target_n_deals < settings.estimate_quarter_index_small_n_threshold + ): + logger.info( + "quarter_index: bimodal guard triggered " + "(index=%.3f n=%d < %d) for %s — no-op", + target_qi, + target_n_deals, + settings.estimate_quarter_index_small_n_threshold, + target_quarter, + ) + else: + lot_quarters_for_guard2: list[str] = [] + analog_quarters: list[tuple[str, float]] = [] + for lot in listings_clean: + lq = _quarter_from_cadastre(lot.get("building_cadastral_number")) + if lq is None: + continue + lot_quarters_for_guard2.append(lq) + lp = lot.get("price_per_m2") + if lp: + analog_quarters.append((lq, float(lp))) + + # Guard-2: same-quarter ratio. + same_quarter_count = sum( + 1 for lq in lot_quarters_for_guard2 if lq == target_quarter + ) + # #1385: знаменатель — только классифицируемые аналоги. + same_quarter_ratio = ( + same_quarter_count / len(lot_quarters_for_guard2) + if lot_quarters_for_guard2 + else 0.0 + ) + if same_quarter_ratio > settings.estimate_quarter_match_skip_ratio: + logger.info( + "quarter_index: Guard-2 skip (same-quarter ratio=%.2f > %.2f)" + " for %s", + same_quarter_ratio, + settings.estimate_quarter_match_skip_ratio, + target_quarter, + ) + else: + distinct_analog_quarters = list( + dict.fromkeys(lq for lq, _lp in analog_quarters) + ) + analog_index_map = quarter_indexes_lookup(distinct_analog_quarters) + weighted_sum = 0.0 + weight_total = 0.0 + for lq, lp in analog_quarters: + lot_qi = analog_index_map.get(lq) + if lot_qi is None: + continue + weighted_sum += lp * lot_qi + weight_total += lp + + avg_analog_index = weighted_sum / weight_total if weight_total > 0 else 1.0 + + ( + median_ppm2, + median_price, + range_low, + range_high, + qi_factor, + ) = _apply_quarter_index( + base_median_ppm2=median_ppm2, + base_median_price=median_price, + base_range_low=range_low, + base_range_high=range_high, + target_index=target_qi, + avg_analog_index=avg_analog_index, + min_factor=settings.estimate_quarter_index_factor_min, + max_factor=settings.estimate_quarter_index_factor_max, + ) + analogs_with_qi = sum( + 1 for lq, _lp in analog_quarters if lq in analog_index_map + ) + logger.info( + "quarter_index: applied target=%s target_qi=%.3f" + " avg_analog_qi=%.3f factor=%.3f" + " (same_quarter_ratio=%.2f analogs_with_qi=%d)", + target_quarter, + target_qi, + avg_analog_index, + qi_factor, + same_quarter_ratio, + analogs_with_qi, + ) + explanation = (explanation or "") + ( + f" Учтена локация квартала" f" (индекс цен квартала ×{qi_factor:.2f})." + ) + sources_used_pre = sorted(set(sources_used_pre) | {"quarter_index"}) + + # ── #1795 шаг 1: soft-кламп headline к коридору ДКП-сделок ────────────── + slack = settings.estimate_corridor_clamp_slack + if dkp_raw is not None and median_ppm2 > 0: + old_ppm2 = median_ppm2 + median_ppm2, median_price, range_low, range_high, clamped = _apply_corridor_clamp( + median_ppm2=median_ppm2, + median_price=median_price, + range_low=range_low, + range_high=range_high, + corridor_high_ppm2=dkp_raw["high_ppm2"], + corridor_count=dkp_raw["count"], + anchor_tier=anchor_tier, + slack=slack, + min_n=settings.estimate_corridor_clamp_min_n, + enabled=settings.estimate_corridor_clamp_enabled, + ) + if clamped: + logger.info( + "corridor clamp #1795: headline %d → %d ₽/м² (corridor_high=%d ×(1+%.2f)," + " count=%d, tier=%s)", + int(old_ppm2), + int(median_ppm2), + dkp_raw["high_ppm2"], + slack, + dkp_raw["count"], + anchor_tier, + ) + explanation = (explanation or "") + ( + " Оценка ограничена коридором реальных сделок Росреестра по улице." + ) + + # ── Radius-path нижний floor от DKP-коридора ───────────────────────────── + if ( + settings.estimate_radius_floor_enabled + and anchor_tier is None + and dkp_raw is not None + and dkp_raw.get("low_ppm2", 0) > 0 + and median_ppm2 > 0 + and dkp_raw.get("count", 0) >= settings.estimate_corridor_clamp_min_n + ): + radius_floor_ppm2 = float(dkp_raw["low_ppm2"]) * settings.estimate_radius_floor_factor + if median_ppm2 < radius_floor_ppm2: + floor_factor = radius_floor_ppm2 / median_ppm2 + logger.info( + "radius_floor: median_ppm2=%d < dkp_low=%d × factor=%.2f = floor=%d" + " → lifting (factor=%.3f)", + int(median_ppm2), + dkp_raw["low_ppm2"], + settings.estimate_radius_floor_factor, + int(radius_floor_ppm2), + floor_factor, + ) + median_ppm2 = radius_floor_ppm2 + median_price = round(median_price * floor_factor) + range_low = round(range_low * floor_factor) + range_high = round(range_high * floor_factor) + explanation = (explanation or "") + ( + " Оценка поднята до нижней границы коридора реальных сделок Росреестра." + ) + + # 4c (cont.). expected_sold AFTER all headline mutations. + if median_ppm2 > 0: + asking_to_sold_ratio, ratio_basis = ratio_resolver(median_ppm2) + if asking_to_sold_ratio is None: + ratio_basis = None + + if asking_to_sold_ratio is not None and median_price > 0: + effective_ratio = asking_to_sold_ratio + if settings.estimate_expected_sold_le_asking and effective_ratio > 1.0: + logger.info( + "expected_sold ratio clamped %.3f->1.0 (rooms=%s)", + effective_ratio, + rooms, + ) + effective_ratio = 1.0 + expected_sold_per_m2 = round(median_ppm2 * effective_ratio) + expected_sold_price = round(median_price * effective_ratio) + expected_sold_range_low = round(range_low * effective_ratio) + expected_sold_range_high = round(range_high * effective_ratio) + + # ── #652: ДКП-коридор реальных сделок (advisory) ───────────────────────── + dkp_corridor: DkpCorridor | None = None + if dkp_raw is not None: + dkp_corridor = DkpCorridor(**dkp_raw) + if median_ppm2 and dkp_raw["count"] >= 3: + if median_ppm2 > dkp_raw["high_ppm2"] * (1.0 + slack): + explanation = (explanation or "") + ( + " Оценка выше коридора реальных сделок Росреестра по улице." + ) + elif median_ppm2 < dkp_raw["low_ppm2"] * (1.0 - slack): + explanation = (explanation or "") + ( + " Оценка ниже коридора реальных сделок Росреестра по улице." + ) + + # ── #693: coarse-geo downgrade ─────────────────────────────────────────── + geo_coarse = _geocode_is_coarse(geo) + if (dadata_coarse or geo_coarse) and median_price > 0: + logger.info( + "coarse-geo gate #693: dadata_coarse=%s geo_coarse=%s anchor_tier=%s " + "median=%s geo.provider=%s geo.confidence=%s geo.full_address=%r geo=(%.5f,%.5f)", + dadata_coarse, + geo_coarse, + anchor_tier, + median_price, + geo.provider, + geo.confidence, + geo.full_address, + geo.lat, + geo.lon, + ) + if (dadata_coarse or geo_coarse) and anchor_tier != "A" and median_price > 0: + if dadata_coarse: + _coarse_label = {2: "населённого пункта", 3: "города", 4: "региона"}.get( + dadata_qc_geo, "населённого пункта" + ) + else: + _coarse_label = "населённого пункта" + confidence = "low" + explanation = (explanation or "") + ( + f" Адрес определён лишь до уровня {_coarse_label} — точные координаты " + "дома найти не удалось, поэтому оценка ориентировочная (аналоги взяты " + "по широкой окрестности)." + ) + + return PricingResult( + median_ppm2=median_ppm2, + median_price=median_price, + range_low=range_low, + range_high=range_high, + n_analogs=n_analogs, + confidence=confidence, + explanation=explanation, + anchor_tier=anchor_tier, + anchor_comps_used=anchor_comps_used, + avito_imv_summary=avito_imv_summary, + dkp_corridor=dkp_corridor, + expected_sold_per_m2=expected_sold_per_m2, + expected_sold_price=expected_sold_price, + expected_sold_range_low=expected_sold_range_low, + expected_sold_range_high=expected_sold_range_high, + asking_to_sold_ratio=asking_to_sold_ratio, + ratio_basis=ratio_basis, + sources_used_pre=sources_used_pre, + listings_clean=listings_clean, + ) + + # ── Public ─────────────────────────────────────────────────────────────────── async def estimate_quality( payload: TradeInEstimateInput, db: Session, created_by: str | None = None @@ -2128,52 +2769,9 @@ async def estimate_quality( area_widened = True analog_tier = analog_tier_wa - # 3. Outlier filter - listings_clean = _filter_outliers(listings) - - # 4. Aggregation - if listings_clean: - prices_ppm2 = sorted(lot["price_per_m2"] for lot in listings_clean if lot["price_per_m2"]) - median_ppm2 = _percentile(prices_ppm2, 0.5) - q1_ppm2 = _percentile(prices_ppm2, 0.25) - q3_ppm2 = _percentile(prices_ppm2, 0.75) - median_price = int(median_ppm2 * payload.area_m2) - range_low = int(q1_ppm2 * payload.area_m2) - range_high = int(q3_ppm2 * payload.area_m2) - # #2: n_analogs должен отражать число аналогов, реально внёсших цену в - # медиану, а не len(listings_clean). _filter_outliers СОХРАНЯЕТ листинги с - # price_per_m2 IS NULL, но prices_ppm2 их отбрасывает → len(listings_clean) - # завышал счётчик ("Найдено N аналогов" вводил в заблуждение; в пределе - # все price-less → median_ppm2=0, но n_analogs>0). Считаем по prices_ppm2. - # listings_clean НЕ фильтруем — downstream (analogs_lots, sources_used, - # repair_coef) по-прежнему работает с полным выживанием outlier-фильтра. - n_analogs = len(prices_ppm2) - else: - median_ppm2 = 0 - median_price = 0 - range_low = 0 - range_high = 0 - n_analogs = 0 - - # 4b. Поправка на состояние ремонта (встреча Птицы: ремонт влияет на цену). - # Аналоги — микс состояний; коэффициент сдвигает оценку под ремонт клиента. - repair_coef = _repair_coefficient(payload.repair_state) - repair_note = "" - if listings_clean and repair_coef != 1.0: - median_price = int(median_price * repair_coef) - range_low = int(range_low * repair_coef) - range_high = int(range_high * repair_coef) - median_ppm2 = median_ppm2 * repair_coef - pct = round((repair_coef - 1.0) * 100) - repair_note = ( - f" Цена скорректирована на состояние ремонта " - f"({_REPAIR_LABEL.get(payload.repair_state, '')} {pct:+d}%)." - ) - - # #1795: ДКП-коридор реальных сделок фетчим ЗДЕСЬ (раньше — после anchor/blend, - # ~стр. 2637), чтобы corridor_high был доступен для (а) Tier C-гейта при - # применении anchor (шаг 3) и (б) soft-клампа headline ДО expected_sold (шаг 1). - # Тот же объект переиспользуется в advisory-блоке ниже (без повторного фетча). + # ── PRE-FETCH: dkp_raw (hoisted before _price_from_inputs) ────────────── + # #1795: ДКП-коридор фетчим ДО вызова _price_from_inputs, чтобы + # corridor_high был доступен для Tier C-гейта и soft-клампа headline. dkp_raw = _fetch_dkp_corridor( db, address=geo.full_address, @@ -2181,62 +2779,10 @@ async def estimate_quality( area=payload.area_m2, ) - # 4c. Asking→sold коррекция (#648 Stage 3) — PURELY ADDITIVE. Headline - # median_price/range_*/median_ppm2 (ASKING активных объявлений) НЕ трогаем; - # вычисляем ПАРАЛЛЕЛЬНУЮ expected_sold цену = asking × per-rooms ratio - # (asking_to_sold_ratios, migration 080). Это релевантная для выкупа цена - # сделки (backtest #648 S1: bias asking-медианы +20% → −4% на held-out ДКП). - # NOTE: actual_deals (#564) остаётся ИНФОРМАЦИОННЫМ и НЕ подмешивается в - # headline — sold-коррекция здесь единственный sold-сигнал (без double-count). - # NOTE: expected_sold_* (= asking × ratio) выводятся НЕ здесь, а ПОСЛЕ #651 - # IMV-blend / SB-anchor / #1795 corridor-clamp / radius-floor (ниже), которые - # мутируют median_price/median_ppm2/range_high. Иначе expected_sold остаётся - # pre-blend → asking 75M / sold 45M (бессмысленная скидка в HeroSummary) и - # stale-значения persist'ятся в trade_in_estimates. - # FIX(ratio-tier-mismatch): _get_asking_sold_ratio вызывается ПОСЛЕ всех - # headline-мутаций (anchor/IMV-blend/quarter-index/corridor-clamp), передавая - # ФИНАЛЬНЫЙ median_ppm2 для tier-placement. Это устраняет tier-несовпадение - # когда anchor/quarter-index двигают headline в другой tier — ratio теперь - # соответствует тому ppm², к которому он будет применён. - # Инициализируем переменные здесь; реальный _get_asking_sold_ratio вызов — после - # corridor-clamp / radius-floor (ниже, ~строка 4c-cont). - asking_to_sold_ratio: float | None = None - ratio_basis: str | None = None - expected_sold_per_m2: int | None = None - expected_sold_price: int | None = None - expected_sold_range_low: int | None = None - expected_sold_range_high: int | None = None - - confidence, explanation = _compute_confidence( - n_analogs, - median_ppm2, - q1_ppm2 if listings_clean else 0, - q3_ppm2 if listings_clean else 0, - fallback_used, - area_widened, - listings=listings_clean, - ) - - # Tier note — информируем пользователя о качестве house-match - tier_note = "" - if analog_tier == "S": - tier_note = " (аналоги из того же дома)" - elif analog_tier == "H": - tf_str = f"{payload.total_floors}-эт." if payload.total_floors is not None else "" - yr_str = f"{target_year}±15 г." if target_year else "" - parts_str = ", ".join(p for p in [yr_str, tf_str] if p) - tier_note = f" (аналоги из домов того же класса: {parts_str})" if parts_str else "" - else: - tier_note = " (нет аналогов в том же доме/классе — расширили поиск)" - - explanation = (explanation or "") + tier_note + repair_note - # ── Stage 3: Avito IMV evaluation as 5-th source (on-demand cached) ── imv_eval: IMVEvaluation | None = None imv_house_type = _IMV_HOUSE_TYPE_MAP.get(target_house_type) imv_renovation = _IMV_REPAIR_MAP.get(payload.repair_state) - # IMV требует: address, rooms, area, floor, floor_at_home, house_type, renovation_type. - # Если payload не содержит required fields — skip IMV (graceful). if ( geo is not None and geo.full_address @@ -2257,21 +2803,10 @@ async def estimate_quality( house_type=imv_house_type, renovation_type=imv_renovation, has_balcony=bool(payload.has_balcony), - has_loggia=False, # payload не разделяет балкон/лоджия → дефолт False + has_loggia=False, ) - # Include IMV в sources_used если получили - sources_used_pre = sorted({lot.get("source") for lot in listings_clean if lot.get("source")}) - if imv_eval is not None: - sources_used_pre = sorted(set(sources_used_pre) | {"avito_imv"}) - # ── Stage 8: Yandex Valuation as on-demand source (anonymous, cached 24h) ── - # #654: главный латентность-подозреваемый. Этот источник UNGATED — бежит на - # КАЖДОЙ оценке (в т.ч. без floor/total_floors), а его внутренний httpx - # timeout 30s + curl_cffi impersonation + sleep_between_requests могут одни - # превысить gateway read timeout → opaque 502. Оборачиваем в budget: при - # превышении → None (cache-hit путь быстрый, timeout бьёт только по медленному - # cache-miss fetch). Деградация идентична сетевой ошибке внутри функции. yandex_val: YandexValuationResult | None = None if geo is not None and geo.full_address: yandex_val = await _with_budget( @@ -2280,7 +2815,6 @@ async def estimate_quality( label="yandex_valuation", ) if yandex_val is not None: - sources_used_pre = sorted(set(sources_used_pre) | {"yandex_valuation"}) saved_hist = _save_yandex_history_items(db, yandex_val) logger.info( "yandex_valuation: history items processed=%d saved=%d" @@ -2289,7 +2823,7 @@ async def estimate_quality( saved_hist, ) - # ── Stage 9: Cian Valuation as 7th source (on-demand, 24h cached, graceful if no cookies) ── + # ── Stage 9: Cian Valuation as 7th source (on-demand, 24h cached) ────── cian_val: CianValuationResult | None = None if ( geo is not None @@ -2316,7 +2850,6 @@ async def estimate_quality( label="cian_valuation", ) if cian_val is not None and cian_val.sale_price_rub: - sources_used_pre = sorted(set(sources_used_pre) | {"cian_valuation"}) logger.info( "cian_valuation: price=%s accuracy=%s house_id=%s", cian_val.sale_price_rub, @@ -2326,35 +2859,17 @@ async def estimate_quality( except Exception as exc: logger.warning("cian_valuation: lookup failed (graceful): %s", exc) - # ── #651/#652 v2: SAME-BUILDING ANCHOR (PRIMARY, validated) ────────────── - # Якорь из комплов ТОГО ЖЕ ДОМА (Tier A) / micro-radius (Tier C). Когда он - # сработал — он ЗАМЕНЯЕТ радиусную медиану для median_price/ppm²/range (premium - # больше не размывается массовой застройкой). За флагом; OFF ⇒ точно старое - # поведение. Сегмент в Tier C фиксирован guard'ом (#1186): вторичка + NULL-legacy. - anchor_tier: str | None = None - # #694: комплы того же дома, на которых РЕАЛЬНО построен headline-якорь. - # Заполняется только когда anchor мутировал median (ниже) — тогда UI-аналоги - # строятся из них, а не из радиусных listings_clean (cheaper/empty). - anchor_comps_used: list[dict[str, Any]] = [] - # #691: гейт НЕ требует радиусных аналогов (listings_clean) / median_price>0. - # На проде геокод часто = None → ST_DWithin не находит радиусные комплы → - # median=0, и same-building якорь скипался, ХОТЯ комплы того же дома есть - # (_fetch_anchor_comps резолвит дом по payload.address без гео — Ткачёва 13, - # Сакко 99). Запускаем по resolved-таргету (area + raw-адрес/geo); если комплов - # реально нет — _compute_same_building_anchor вернёт None и мутации не будет - # (поведение идентично старому). Новый гейт ⊇ старого: при непустом radius - # адрес всегда есть, так что ранее-якорившиеся кейсы якорятся по-прежнему. + # ── Pre-fetch: same-building anchor comps ───────────────────────────────── + # Guard mirrors the original in-block guard exactly; when false → ([], None). + _anchor_comps_pre: list[dict] + _anchor_tier_pre: str | None if ( settings.estimate_same_building_anchor_enabled and payload.area_m2 and (payload.address or (geo is not None and geo.full_address)) ): - comps, anchor_tier = _fetch_anchor_comps( + _anchor_comps_pre, _anchor_tier_pre = _fetch_anchor_comps( db, - # payload.address (сырой ввод) ПЕРВИЧЕН: на проде геокод часто - # возвращает None (lat/lon/canonical=None), и geo.full_address пуст — - # тогда same-building Tier A не находил дом и падал в радиус. Raw-адрес - # всегда есть и именно на нём валидирован нормализатор (#677/#679). address=payload.address or geo.full_address, target_house_id=target_house_id, lat=geo.lat, @@ -2362,640 +2877,89 @@ async def estimate_quality( rooms=payload.rooms, area=payload.area_m2, ) - anchor = _compute_same_building_anchor( - comps, - area_target=payload.area_m2, - rooms_target=payload.rooms, - tier=anchor_tier or "", - sigma=settings.estimate_sb_area_sigma, - rooms_boost=settings.estimate_sb_rooms_match_boost, - floor_target=payload.floor, - total_floors_target=payload.total_floors, - floor_sigma=settings.estimate_sb_floor_sigma, - min_comps=settings.estimate_sb_min_comps, - mad_k=settings.estimate_sb_mad_k, - ) - # #1795 шаг 3: гейт Tier C. Tier C = micro-radius (≤500 м), НЕ тот же дом — - # соседние элитные комплы тянут anchor вверх на право-скошенном премиум- - # распределении. Если anchor_ppm2 > corridor_high×mult — anchor НЕ заменяет - # консервативную радиусную медиану (anchor=None → fallback на radius). - # corridor_high из уже-зафетченного dkp_raw; если коридора нет — fallback - # radius_median×1.3 (чтобы гейт работал и без ДКП-данных). Tier A не гейтим. - if ( - anchor is not None - and anchor_tier == "C" - and settings.estimate_anchor_tier_c_corridor_mult > 0 - ): - if dkp_raw is not None and dkp_raw.get("high_ppm2", 0) > 0: - corridor_high_for_gate = float(dkp_raw["high_ppm2"]) - else: - corridor_high_for_gate = (median_ppm2 / repair_coef) * 1.3 if repair_coef else 0.0 - gate_threshold = corridor_high_for_gate * settings.estimate_anchor_tier_c_corridor_mult - if gate_threshold > 0 and anchor["anchor_ppm2"] > gate_threshold: - logger.info( - "sb_anchor Tier C gate #1795: anchor_ppm2=%d > corridor_high×%.1f=%d" - " → keep radius median (anchor suppressed)", - int(anchor["anchor_ppm2"]), - settings.estimate_anchor_tier_c_corridor_mult, - int(gate_threshold), - ) - anchor = None - # #audit-1: low-confidence gate — якорь с низкой уверенностью или малым n - # при высоком FSD НЕ заменяет headline; fallback на radius-median. - # Tier A (n≥4, FSD<0.15) проходит без изменений при дефолтных порогах. - if anchor is not None and settings.estimate_sb_low_conf_gate_enabled: - gate_low = anchor["confidence"] == "low" - gate_thin = ( - anchor["n"] < settings.estimate_sb_gate_min_n - and anchor["fsd"] > settings.estimate_sb_gate_max_fsd - ) - if gate_low or gate_thin: - logger.info( - "sb_anchor low-conf gate #audit-1: tier=%s n=%d fsd=%.3f conf=%s" - " → suppressed (gate_low=%s gate_thin=%s) → radius fallback", - anchor_tier, - anchor["n"], - anchor["fsd"], - anchor["confidence"], - gate_low, - gate_thin, - ) - anchor = None - anchor_tier = None - if anchor is not None: - # #694: якорь мутирует headline — UI-аналоги должны отражать ЭТИ комплы. - anchor_comps_used = comps - # Headline = recommended ASKING price (комплы — активные объявления; - # golden-реалы — asking). Берём anchor_ppm2 (PRE-haircut), НЕ - # anchor_sold_ppm2. asking→sold скидка применяется единственным - # механизмом — per-rooms asking_to_sold_ratio (migration 080) ниже, - # дающим distinct expected_sold. Двойная скидка (band-haircut здесь + - # ratio там) давала median == expected_sold (#677/#681 collision). - est_ppm2 = anchor["anchor_ppm2"] - # PREMIUM GUARDRAIL (hard): не ниже минимального same-building ppm² (−tol). - # Только Tier A/C (комплы реально из дома/микрорайона). Эконом — no-op - # (est уже ≥ floor), премиум — поднимает если mean занизил. ASKING-space. - floor_ppm2 = anchor["comp_min_ppm2"] * (1.0 - settings.estimate_sb_guardrail_tol) - if est_ppm2 < floor_ppm2: - est_ppm2 = floor_ppm2 - # POINT = anchor_asking × area × repair_coef (repair уже применён к старой - # median; здесь применяем к свежему якорю — заменяем headline целиком). - new_ppm2 = est_ppm2 * repair_coef - point = int(new_ppm2 * payload.area_m2) - # FSD-диапазон (tight): симметричный вокруг point, k·fsd полуширина. - half = settings.estimate_fsd_k * anchor["fsd"] - new_range_low = int(point * max(0.0, 1.0 - half)) - new_range_high = int(point * (1.0 + half)) - # Диапазон должен ПОКРЫВАТЬ same-building спред комплов (sold-adjusted) и - # удовлетворять low ≤ point ≤ high. Внутридомовая дисперсия (этаж/вид) — - # реальный разброс цены в доме; честный диапазон обязан её включать - # (иначе видовой топ-юнит вылетает за range_high — residual miss спека). - # comp spread в ASKING-пространстве (комплы — активные объявления). range_high - # покрывает RAW comp max — честно показываем верх дома (видовой/топ-юнит), - # иначе он вылетает за диапазон. range_low — RAW comp min (asking-space): - # headline теперь asking, band-haircut больше не применяется (sold-скидка — - # единственным механизмом per-rooms ratio ниже). - # #1518: repair_coef применён к point (new_ppm2), поэтому границы спреда - # тоже нормируем на ремонт — иначе центр и края в разных масштабах - # (для needs_repair coef<1 верх несимметрично завышен, для excellent — занижен). - spread_low = int(anchor["comp_min_ppm2"] * repair_coef * payload.area_m2) - spread_high = int(anchor["comp_max_ppm2"] * repair_coef * payload.area_m2) - new_range_low = min(new_range_low, spread_low, point) - new_range_high = max(new_range_high, spread_high, point) + else: + _anchor_comps_pre, _anchor_tier_pre = [], None - logger.info( - "sb_anchor: tier=%s n=%d radius_median_ppm2=%d → anchor_asking_ppm2=%d" - " (uplift=%s haircut=%.2f) point %d → %d", - anchor_tier, - anchor["n"], - int(median_ppm2), - int(est_ppm2), - anchor["used_uplift"], - anchor["haircut"], - median_price, - point, - ) - median_ppm2 = new_ppm2 - median_price = point - range_low = new_range_low - range_high = new_range_high - confidence = anchor["confidence"] - tier_label = "того же дома" if anchor_tier == "A" else "ближайшего окружения (≤500 м)" - # #695: когда якорь построил headline, explanation описывает ИМЕННО якорные - # комплы (anchor['n'] из tier_label). Радиусный base-текст («Найдено N из M - # разных адресов») и analog_tier tier_note относятся к ДРУГОЙ выборке и - # противоречат якорю (n_analogs≠anchor['n']; «разных адресов» vs «того же - # дома») — поэтому ЗАМЕНЯЕМ, а не конкатенируем. repair_note сохраняем - # (ортогонален — поправка на ремонт). confidence уже = anchor["confidence"]. - explanation = ( - f"Оценка построена по {anchor['n']} аналогам из {tier_label}" - f"{' (топ-уровень в доме)' if anchor['used_uplift'] else ''}." - ) + repair_note - # #695 (QA fixup): когда якорь подменяет headline и список аналогов на - # комплы дома, n_analogs ДОЛЖЕН считаться по anchor-популяции, а не по - # радиусу (listings_clean). Иначе все UI-счётчики («Аналогов N», - # «Показано N из M», «по N аналогам», гистограмма) расходятся: radius=4 vs - # anchor=5 → артефакт «Показано 5 из 4». anchor['n'] = число комплов, на - # которых построена оценка (= показываемый analogs, capped 10). Для anchor- - # пути это и есть «полное число найденных» по контракту n_analogs (#698). - n_analogs = anchor["n"] - # #1871 P1 (ghost-anchor): same-building anchor заменил headline, но - # радиусных аналогов ноль (listings_clean пуст) → нельзя отдавать - # высокую/среднюю уверенность. Downgrade до 'low' + честный caveat. - # _enforce_zero_analog_low ниже это не ловит: n_analogs уже перезаписан - # на anchor["n"] (>0), а не 0. - if not listings_clean and confidence != "low": - logger.warning( - "estimator #1871 ghost-anchor guard: confidence %s→low " - "(anchor_n=%s, radius_analogs=0)", - confidence, - anchor["n"], - ) - confidence = "low" - explanation += ( - " Оценка опирается только на аналоги из того же дома — " - "сопоставимых предложений поблизости не найдено, поэтому " - "точность ориентировочная." - ) - # ── #1871 P2: split-дома wide-corridor disclosure (за флагом, OFF) ── - # Tier A матчит по address-regex (намеренно НЕ house_id — дом дробится - # на несколько house_id). На split-доме разной этажности comp_min..max - # растягивается через несколько ценовых режимов → коридор 148%/170%. - # Коридор честно широкий (НЕ сужаем); только понижаем confidence на - # ступень и дописываем disclosure. НЕ трогаем point/median/range. - if ( - settings.estimate_wide_corridor_disclosure_enabled - and anchor_tier == "A" - and median_price > 0 - ): - corridor_pct = (range_high - range_low) / median_price - if corridor_pct > settings.estimate_wide_corridor_threshold: - confidence = _downgrade_confidence(confidence) - explanation = (explanation or "") + ( - " Очень широкий ценовой диапазон по дому (вероятно, дом " - "разбит на секции разной этажности или разнородный фонд) — " - "оценка ориентировочная, уточните по конкретной секции." - ) + # ── Pre-fetch: house IMV anchor ONCE (used for both blend and display) ─── + imv_anchor_data = _fetch_house_imv_anchor( + db, + target_house_id=target_house_id, + rooms=payload.rooms, + area=payload.area_m2, + ) - # ── #651: IMV / Yandex BLEND (killer accuracy fix) — SECONDARY, Tier D only ── - # Радиусная медиана системно занижает премиум/видовые квартиры (нет class/ - # segment-коррекции). Берём РЕАЛЬНЫЙ Avito IMV target-дома из house_imv_evaluations - # (avito_imv_evaluations пуст — keyed estimate_id, on-demand), используем как - # anchor: если IMV recommended_price > median × threshold — поднимаем медиану - # blend'ом и расширяем верх диапазона. Всё за флагом + null-guard (no-op без IMV). - # ВАЖНО (v2): IMV-blend выполняется ТОЛЬКО когда same-building anchor НЕ сработал - # (anchor_tier is None) — не накладываем blend поверх уже-построенного якоря дома. - # #764: imv_anchor_present — любой IMV-anchor повлиял на estimate (median OR range). - # Guard-1b использует этот флаг чтобы пропустить квартальный индекс при любом - # IMV-влиянии, не только при blended (range_high расширяется даже без blend). - imv_anchor_present: bool = False - avito_imv_summary: AvitoImvSummary | None = None - if ( - anchor_tier is None - and settings.estimate_imv_blend_enabled - and listings_clean - and median_price > 0 - ): - imv_anchor = _fetch_house_imv_anchor( - db, - target_house_id=target_house_id, - rooms=payload.rooms, - area=payload.area_m2, - ) - # Anchor chain: prefer Avito IMV recommended; fall back to on-demand imv_eval. - anchor_total: int | None = None - anchor_higher: int | None = None - anchor_label: str | None = None - if imv_anchor is not None and imv_anchor.get("recommended_price"): - anchor_total = int(imv_anchor["recommended_price"]) - anchor_higher = ( - int(imv_anchor["higher_price"]) if imv_anchor.get("higher_price") else None - ) - anchor_label = "оценке Avito IMV" - _imv_mc = int(imv_anchor["market_count"]) if imv_anchor.get("market_count") else None - avito_imv_summary = AvitoImvSummary( - recommended_price=anchor_total, - lower_price=( - int(imv_anchor["lower_price"]) if imv_anchor.get("lower_price") else None - ), - higher_price=anchor_higher, - market_count=_imv_mc, - # #audit-5b: тонкий рынок — малая выборка для IMV. - thin_market=( - _imv_mc is not None and _imv_mc < settings.avito_imv_thin_market_threshold - ), - ) - elif imv_eval is not None and imv_eval.recommended_price: - # on-demand IMV (avito_imv_evaluations) — fallback, обычно пуст - anchor_total = int(imv_eval.recommended_price) - anchor_higher = int(imv_eval.higher_price) if imv_eval.higher_price else None - anchor_label = "оценке Avito IMV" - avito_imv_summary = AvitoImvSummary( - recommended_price=anchor_total, - lower_price=int(imv_eval.lower_price) if imv_eval.lower_price else None, - higher_price=anchor_higher, - market_count=imv_eval.market_count, - # #audit-5b: тонкий рынок — market_count < threshold. - thin_market=( - imv_eval.market_count is not None - and imv_eval.market_count < settings.avito_imv_thin_market_threshold - ), - ) - - # #audit-5b: thin-market warning — IMV на малой выборке → reliability ↓. - if avito_imv_summary is not None and avito_imv_summary.thin_market: - logger.warning( - "avito_imv thin_market #audit-5b: market_count=%s" - " (< avito_imv_thin_market_threshold=%d) — IMV reliability low", - avito_imv_summary.market_count, - settings.avito_imv_thin_market_threshold, - ) - - if anchor_total is not None: - imv_anchor_present = True - new_median, new_range_high, new_ppm2, blended, anchor_used = _apply_imv_blend( - median_price=median_price, - range_high=range_high, - median_ppm2=median_ppm2, - area=payload.area_m2, - anchor_total=anchor_total, - anchor_higher=anchor_higher, - weight=settings.estimate_imv_blend_weight, - threshold=settings.estimate_imv_blend_threshold, - ) - if blended: - logger.info( - "imv_blend: median %d → %d (anchor=%d w=%.2f) range_high %d → %d", - median_price, - new_median, - anchor_used, - settings.estimate_imv_blend_weight, - range_high, - new_range_high, - ) - median_price = new_median - median_ppm2 = new_ppm2 - explanation = (explanation or "") + ( - f" Оценка скорректирована по {anchor_label} " - f"({anchor_used / 1_000_000:.1f} млн ₽)." - ) - sources_used_pre = sorted(set(sources_used_pre) | {"avito_imv"}) - # Диапазон расширяем даже если медиану не двигали (информативность). - range_high = new_range_high - - # Display-only Avito IMV summary, когда headline построен same-building якорем - # (IMV-blend выше пропущен). Якорь дома — primary; IMV остаётся cross-check в UI. - if anchor_tier is not None and avito_imv_summary is None: - imv_anchor_disp = _fetch_house_imv_anchor( - db, - target_house_id=target_house_id, - rooms=payload.rooms, - area=payload.area_m2, - ) - if imv_anchor_disp is not None and imv_anchor_disp.get("recommended_price"): - _disp_mc = ( - int(imv_anchor_disp["market_count"]) - if imv_anchor_disp.get("market_count") - else None - ) - avito_imv_summary = AvitoImvSummary( - recommended_price=int(imv_anchor_disp["recommended_price"]), - lower_price=( - int(imv_anchor_disp["lower_price"]) - if imv_anchor_disp.get("lower_price") - else None - ), - higher_price=( - int(imv_anchor_disp["higher_price"]) - if imv_anchor_disp.get("higher_price") - else None - ), - market_count=_disp_mc, - # #audit-5b: тонкий рынок. - thin_market=( - _disp_mc is not None and _disp_mc < settings.avito_imv_thin_market_threshold - ), - ) - - # ── #764: per-cadastral-quarter price index gap-correction ────────────────── - # Применяется ТОЛЬКО в pure-radius пути (Guard-1): когда same-building anchor - # не сработал И IMV-blend не поднял медиану. Оба механизма уже учитывают - # location пространственно — наложение индекса сверху даёт double-count. - # Формула: adjusted_ppm2 = base_ppm2 × (target_index / avg_analog_index), - # где avg_analog_index = взвешенная по ppm² медиана аналогов, чьи кварталы - # известны. Если аналоги без кадастрового номера — avg_analog_index=1.0 (no-op). - if ( - settings.estimate_quarter_index_enabled - and anchor_tier is None # Guard-1a: same-building anchor не сработал - and not imv_anchor_present # Guard-1b: IMV-anchor не повлиял (median или range) - and median_price > 0 - and payload.area_m2 - ): - # Резолвим квартал target'а: Primary — DaData house_cadnum. - target_quarter: str | None = _quarter_from_cadastre( - dadata.house_cadnum if dadata is not None else None - ) - # Fallback: building_cadastral_number из самих аналогов (если все в 1 доме - # — Tier S path; тогда кадастровый номер квартала тот же). Не применяем - # PostGIS point-in-quarter: нет готовой geometry-таблицы кварталов в tradein DB. - if target_quarter is None: - for lot in listings_clean: - cq = _quarter_from_cadastre(lot.get("building_cadastral_number")) - if cq is not None: - target_quarter = cq - break - - if target_quarter is not None: - qindex_result = _lookup_quarter_index( - db, - quarter_cad_number=target_quarter, - min_n_deals=settings.estimate_quarter_index_min_n_deals, - ) - if qindex_result is not None: - target_qi, target_n_deals = qindex_result - - # Bimodal/nominal guard (Guard-4): структурно неоднородный квартал - # при малой выборке → no-op (regr. Радищева 66:41:0401017 et al.) - if ( - target_qi > settings.estimate_quarter_index_max_for_small_n - and target_n_deals < settings.estimate_quarter_index_small_n_threshold - ): - logger.info( - "quarter_index: bimodal guard triggered " - "(index=%.3f n=%d < %d) for %s — no-op", - target_qi, - target_n_deals, - settings.estimate_quarter_index_small_n_threshold, - target_quarter, - ) - else: - # Вычисляем квартал каждого аналога ОДИН РАЗ — переиспользуем - # для Guard-2 (same-quarter ratio) и avg_analog_index weighting. - # lot_quarters_for_guard2: все лоты с известным кварталом (как раньше). - # analog_quarters: только лоты с известным кварталом И ценой (для весов). - lot_quarters_for_guard2: list[str] = [] - analog_quarters: list[tuple[str, float]] = [] - for lot in listings_clean: - lq = _quarter_from_cadastre(lot.get("building_cadastral_number")) - if lq is None: - continue - lot_quarters_for_guard2.append(lq) - lp = lot.get("price_per_m2") - if lp: - analog_quarters.append((lq, float(lp))) - - # Guard-2: доля аналогов ИЗ ТОГО ЖЕ квартала > skip_ratio → - # location уже в медиане — пропускаем. - same_quarter_count = sum( - 1 for lq in lot_quarters_for_guard2 if lq == target_quarter - ) - # #1385: знаменатель — ТОЛЬКО классифицируемые аналоги (с разрешённым - # кварталом), как и числитель. Аналоги без кадастра (типичные анонимные - # Avito) не классифицируемы и не должны разбавлять ratio — иначе при - # низком покрытии кадастром ratio структурно занижен, Guard-2 не - # срабатывает, и quarter-index применяется к медиане повторно. - same_quarter_ratio = ( - same_quarter_count / len(lot_quarters_for_guard2) - if lot_quarters_for_guard2 - else 0.0 - ) - if same_quarter_ratio > settings.estimate_quarter_match_skip_ratio: - logger.info( - "quarter_index: Guard-2 skip (same-quarter ratio=%.2f > %.2f)" - " for %s", - same_quarter_ratio, - settings.estimate_quarter_match_skip_ratio, - target_quarter, - ) - else: - # Вычисляем avg_analog_index — ppm²-взвешенное среднее по - # аналогам, чьи кварталы известны И присутствуют в индексе. - # Один батч-запрос вместо N последовательных FDW roundtrips. - # Аналоги без кадастрового номера — игнорируем (не штрафуем). - distinct_analog_quarters = list( - dict.fromkeys(lq for lq, _lp in analog_quarters) - ) - analog_index_map = _lookup_quarter_indexes( - db, - quarter_cad_numbers=distinct_analog_quarters, - min_n_deals=settings.estimate_quarter_index_min_n_deals, - ) - weighted_sum = 0.0 - weight_total = 0.0 - for lq, lp in analog_quarters: - lot_qi = analog_index_map.get(lq) - if lot_qi is None: - continue - weighted_sum += lp * lot_qi - weight_total += lp - - avg_analog_index = weighted_sum / weight_total if weight_total > 0 else 1.0 - - ( - median_ppm2, - median_price, - range_low, - range_high, - qi_factor, - ) = _apply_quarter_index( - base_median_ppm2=median_ppm2, - base_median_price=median_price, - base_range_low=range_low, - base_range_high=range_high, - target_index=target_qi, - avg_analog_index=avg_analog_index, - min_factor=settings.estimate_quarter_index_factor_min, - max_factor=settings.estimate_quarter_index_factor_max, - ) - analogs_with_qi = sum( - 1 for lq, _lp in analog_quarters if lq in analog_index_map - ) - logger.info( - "quarter_index: applied target=%s target_qi=%.3f" - " avg_analog_qi=%.3f factor=%.3f" - " (same_quarter_ratio=%.2f analogs_with_qi=%d)", - target_quarter, - target_qi, - avg_analog_index, - qi_factor, - same_quarter_ratio, - analogs_with_qi, - ) - explanation = (explanation or "") + ( - f" Учтена локация квартала" f" (индекс цен квартала ×{qi_factor:.2f})." - ) - sources_used_pre = sorted(set(sources_used_pre) | {"quarter_index"}) - - # ── #1795 шаг 1: soft-кламп headline к коридору ДКП-сделок (ДО expected_sold) ── - # Применяем кламп РАНЬШЕ вывода expected_sold, чтобы asking↔sold↔range остались - # консистентны автоматически (expected_sold берётся от уже-склампленного headline). - # Tier A (тот же дом) EXEMPT; эконом/комфорт (median в коридоре) → no-op. - slack = settings.estimate_corridor_clamp_slack - if dkp_raw is not None and median_ppm2 > 0: - old_ppm2 = median_ppm2 - median_ppm2, median_price, range_low, range_high, clamped = _apply_corridor_clamp( - median_ppm2=median_ppm2, - median_price=median_price, - range_low=range_low, - range_high=range_high, - corridor_high_ppm2=dkp_raw["high_ppm2"], - corridor_count=dkp_raw["count"], - anchor_tier=anchor_tier, - slack=slack, - min_n=settings.estimate_corridor_clamp_min_n, - enabled=settings.estimate_corridor_clamp_enabled, - ) - if clamped: - logger.info( - "corridor clamp #1795: headline %d → %d ₽/м² (corridor_high=%d ×(1+%.2f)," - " count=%d, tier=%s)", - int(old_ppm2), - int(median_ppm2), - dkp_raw["high_ppm2"], - slack, - dkp_raw["count"], - anchor_tier, - ) - explanation = (explanation or "") + ( - " Оценка ограничена коридором реальных сделок Росреестра по улице." - ) - - # ── FIX: radius-path нижний floor от DKP-коридора (анти-undershoot) ──────── - # Anchor-путь имеет hard floor (comp_min_ppm2×(1-tol)) — radius-путь аналогичной - # защиты снизу не имел: quarter-index-вниз или corridor-clamp могли занизить - # median неоправданно (асимметрия с anchor-path). Если dkp_raw доступен и - # итоговый median_ppm2 < dkp_low_ppm2 × factor → поднимаем до floor и логируем. - # Только radius-путь (anchor_tier is None); без dkp_raw → no-op. - # За флагом estimate_radius_floor_enabled (дефолт True). - if ( - settings.estimate_radius_floor_enabled - and anchor_tier is None - and dkp_raw is not None - and dkp_raw.get("low_ppm2", 0) > 0 - and median_ppm2 > 0 - and dkp_raw.get("count", 0) >= settings.estimate_corridor_clamp_min_n - ): - radius_floor_ppm2 = float(dkp_raw["low_ppm2"]) * settings.estimate_radius_floor_factor - if median_ppm2 < radius_floor_ppm2: - floor_factor = radius_floor_ppm2 / median_ppm2 - logger.info( - "radius_floor: median_ppm2=%d < dkp_low=%d × factor=%.2f = floor=%d" - " → lifting (factor=%.3f)", - int(median_ppm2), - dkp_raw["low_ppm2"], - settings.estimate_radius_floor_factor, - int(radius_floor_ppm2), - floor_factor, - ) - median_ppm2 = radius_floor_ppm2 - median_price = round(median_price * floor_factor) - range_low = round(range_low * floor_factor) - range_high = round(range_high * floor_factor) - explanation = (explanation or "") + ( - " Оценка поднята до нижней границы коридора реальных сделок Росреестра." - ) - - # 4c (cont.). expected_sold_* выводим ЗДЕСЬ — ПОСЛЕ всех headline-мутаций: - # #651 IMV-blend / SB-anchor / quarter-index / #1795 corridor-clamp / radius-floor. - # FIX(ratio-tier-mismatch): _get_asking_sold_ratio вызывается с ФИНАЛЬНЫМ - # median_ppm2 (а не с pre-anchor radius-медианой). Это устраняет tier-несовпадение - # когда anchor/quarter-index двигают headline в другой ppm2-tier. - # Сохраняем per_rooms/global_fallback семантику ratio_basis из самой функции. - # Graceful: нет ratio (нет migration-080 строки / БД-ошибка) → expected_sold_* - # остаются None → UI не показывает «−N%» badge (не фабрикуем). - if median_ppm2 > 0: - asking_to_sold_ratio, ratio_basis = _get_asking_sold_ratio( - db, payload.rooms, anchor_ppm2=median_ppm2 - ) - # Не было ratio — не вводим в заблуждение пустым basis. - if asking_to_sold_ratio is None: - ratio_basis = None - - if asking_to_sold_ratio is not None and median_price > 0: - # FIX(expected_sold-le-asking): clamp ratio <= 1.0 чтобы expected_sold не - # превышала объявление. Smoke: 3к/27.0М -> expected_sold 31.4М > asking. - # Причина: high-price tier ratio > 1.0 (product artefact, не реальные сделки - # выше прайса). Флаг OFF -> старое поведение без clamp. - effective_ratio = asking_to_sold_ratio - if settings.estimate_expected_sold_le_asking and effective_ratio > 1.0: - logger.info( - "expected_sold ratio clamped %.3f->1.0 (rooms=%s)", - effective_ratio, - payload.rooms, - ) - effective_ratio = 1.0 - expected_sold_per_m2 = round(median_ppm2 * effective_ratio) - expected_sold_price = round(median_price * effective_ratio) - expected_sold_range_low = round(range_low * effective_ratio) - expected_sold_range_high = round(range_high * effective_ratio) - - # ── #652: ДКП-коридор реальных сделок (ADVISORY + soft sanity-bound) ───── - # dkp_raw уже зафетчен выше (#1795) — переиспользуем, не фетчим повторно. - dkp_corridor: DkpCorridor | None = None - if dkp_raw is not None: - dkp_corridor = DkpCorridor(**dkp_raw) - # Soft sanity-bound: если итоговая медиана ₽/м² заметно вне коридора — - # текстовая пометка. При сработавшем clamp (выше) median_ppm2 уже прижат к - # потолку → эта ветка не дублирует пометку. slack ±25% — широко. - if median_ppm2 and dkp_raw["count"] >= 3: - if median_ppm2 > dkp_raw["high_ppm2"] * (1.0 + slack): - explanation = (explanation or "") + ( - " Оценка выше коридора реальных сделок Росреестра по улице." - ) - elif median_ppm2 < dkp_raw["low_ppm2"] * (1.0 - slack): - explanation = (explanation or "") + ( - " Оценка ниже коридора реальных сделок Росреестра по улице." - ) - - # ── #693: coarse-geo downgrade ────────────────────────────────────────── - # Когда DaData дала только грубую точность (settlement/city/region/unknown, - # qc_geo >= 2), PostGIS-радиус крутится вокруг центроида НП и собирает - # аналоги из случайных частей города → уверенная «medium» вводит в - # заблуждение. Помечаем как ориентировочную (low). КОНСЕРВАТИВНО: - # 1) есть позитивный сигнал грубости (dadata.qc_geo >= 2 ИЛИ _geocode_is_coarse); - # 2) НЕ сработал якорь «ТОГО ЖЕ ДОМА» (Tier A) — он стоит на реальных комплах - # дома (#691 path) и геоцентроид не важен. ВАЖНО (#693 QA-fail 2/3): Tier C - # (micro-radius ≤500 м) — это НЕ «тот же дом», а случайные квартиры в 500 м - # вокруг геокода. Если геокод грубый (мусор → ЕКБ-центроид), Tier C набирает - # ≥5 комплов в плотном центре и РАНЬШЕ блокировал downgrade (guard был - # `anchor_tier is None`) → канонический кейс «фывапролд 999» оставался medium. - # Теперь защищаем ТОЛЬКО Tier A; Tier C при грубом геокоде — тоже понижаем. - # 3) есть радиусное число (median_price > 0) — что квалифицировать. - # ИСТОЧНИКИ сигнала грубости (OR): dadata.qc_geo>=2 (если DaData активна) ИЛИ - # _geocode_is_coarse(geo) — прод-фолбэк, работает БЕЗ DaData. + # ── Coarse-geo signals ──────────────────────────────────────────────────── dadata_coarse = dadata is not None and dadata.qc_geo is not None and dadata.qc_geo >= 2 - geo_coarse = _geocode_is_coarse(geo) - if (dadata_coarse or geo_coarse) and median_price > 0: - # #693 QA diag (2/3): на проде QA просил залогировать сигналы геокода для - # негеокодируемых адресов — фиксируем, что увидел гейт. - logger.info( - "coarse-geo gate #693: dadata_coarse=%s geo_coarse=%s anchor_tier=%s " - "median=%s geo.provider=%s geo.confidence=%s geo.full_address=%r geo=(%.5f,%.5f)", - dadata_coarse, - geo_coarse, - anchor_tier, - median_price, - geo.provider, - geo.confidence, - geo.full_address, - geo.lat, - geo.lon, + + # ── DB-callable wrappers injected into pure pricing ─────────────────────── + def _ratio_resolver( + appm2: float | None, + ) -> tuple[float | None, str | None]: + return _get_asking_sold_ratio(db, payload.rooms, anchor_ppm2=appm2) + + def _qi_lookup(q: str) -> tuple[float, int] | None: + return _lookup_quarter_index( + db, + quarter_cad_number=q, + min_n_deals=settings.estimate_quarter_index_min_n_deals, ) - if (dadata_coarse or geo_coarse) and anchor_tier != "A" and median_price > 0: - # Уровень грубости берём из DaData (точнее), иначе generic «населённого пункта». - if dadata_coarse: - _coarse_label = {2: "населённого пункта", 3: "города", 4: "региона"}.get( - dadata.qc_geo, "населённого пункта" - ) - else: - _coarse_label = "населённого пункта" - confidence = "low" - explanation = (explanation or "") + ( - f" Адрес определён лишь до уровня {_coarse_label} — точные координаты " - "дома найти не удалось, поэтому оценка ориентировочная (аналоги взяты " - "по широкой окрестности)." + + def _qis_lookup(qs: list[str]) -> dict[str, float]: + return _lookup_quarter_indexes( + db, + quarter_cad_numbers=qs, + min_n_deals=settings.estimate_quarter_index_min_n_deals, ) + # ── Deterministic pricing orchestration ────────────────────────────────── + pr = _price_from_inputs( + listings=listings, + area_m2=payload.area_m2, + rooms=payload.rooms, + repair_state=payload.repair_state, + floor=payload.floor, + total_floors=payload.total_floors, + target_year=target_year, + analog_tier=analog_tier, + fallback_used=fallback_used, + area_widened=area_widened, + anchor_comps=_anchor_comps_pre, + anchor_tier_fetched=_anchor_tier_pre, + dkp_raw=dkp_raw, + imv_anchor=imv_anchor_data, + imv_eval=imv_eval, + yandex_val_present=yandex_val is not None, + cian_val_present=cian_val is not None and bool(cian_val.sale_price_rub), + ratio_resolver=_ratio_resolver, + quarter_index_lookup=_qi_lookup, + quarter_indexes_lookup=_qis_lookup, + target_house_cadnum=dadata.house_cadnum if dadata else None, + dadata_coarse=dadata_coarse, + geo=geo, + dadata_qc_geo=dadata.qc_geo if dadata else None, + ) + + # Unpack pricing result + median_price = pr.median_price + median_ppm2 = pr.median_ppm2 + range_low = pr.range_low + range_high = pr.range_high + n_analogs = pr.n_analogs + confidence = pr.confidence + explanation = pr.explanation + anchor_tier = pr.anchor_tier + anchor_comps_used = pr.anchor_comps_used + avito_imv_summary = pr.avito_imv_summary + dkp_corridor = pr.dkp_corridor + expected_sold_per_m2 = pr.expected_sold_per_m2 + expected_sold_price = pr.expected_sold_price + expected_sold_range_low = pr.expected_sold_range_low + expected_sold_range_high = pr.expected_sold_range_high + asking_to_sold_ratio = pr.asking_to_sold_ratio + ratio_basis = pr.ratio_basis + sources_used_pre = pr.sources_used_pre + listings_clean = pr.listings_clean + # 5. Deals — ДКП-only sales (вторичка) из rosreestr_deals. # Importer фильтрует doc_type='ДКП' (PR-A 2026-05-24), ДДУ застройщиков # исключены — больше не скёюят median вторички ~110-120 К/м². diff --git a/tradein-mvp/backend/app/services/scrape_pipeline.py b/tradein-mvp/backend/app/services/scrape_pipeline.py index aa38b5ff..90aa11d7 100644 --- a/tradein-mvp/backend/app/services/scrape_pipeline.py +++ b/tradein-mvp/backend/app/services/scrape_pipeline.py @@ -3087,10 +3087,10 @@ async def run_avito_full_load( class DomClickCitySweepCounters: """Aggregate counters для DomClick citywide sweep run. - DomClick SERP — citywide (city_id), не geo-bbox: anchor-loop отсутствует. - Один проход fetch_city перебирает rooms × pages внутри scraper'а. - Поля симметричны sibling-counters, но без detail_*/houses_* (DomClick SERP-only) - и без anchors_* (нет anchor-фазы). pages_fetched = rooms × pages worst-case. + DomClick BFF — citywide (city_id), не geo-bbox: anchor-loop отсутствует. + Один проход fetch_city перебирает ROOM_BUCKETS × pages внутри scraper'а. + Поля симметричны sibling-counters, но без detail_*/houses_* (SERP-only) + и без anchors_* (нет anchor-фазы). pages_fetched = 6 buckets × pages worst-case. """ lots_fetched: int = 0 @@ -3098,17 +3098,21 @@ class DomClickCitySweepCounters: lots_updated: int = 0 pages_fetched: int = 0 errors_count: int = 0 + blocked: int = 0 # 1 если QRATOR-блок был во время sweep + geo_filtered: int = 0 # число офферов отфильтрованных geo-guard def to_dict(self) -> dict[str, int]: return {f.name: getattr(self, f.name) for f in fields(self)} -# Дефолтные параметры sweep'а (EKB city_id=4, все комнатности). +# Дефолтные параметры sweep'а (EKB city_id=4). DOMCLICK_DEFAULT_CITY_ID: int = 4 -DOMCLICK_DEFAULT_ROOMS: list[int] = [0, 1, 2, 3, 4] -# Оценка времени одного fetch'а (network + camoufox render + parse) для watchdog. +DOMCLICK_DEFAULT_ROOMS: list[int] = [0, 1, 2, 3, 4] # vestigial; scraper sweeps all buckets +# Число BFF-бакетов (st/1/2/3/4/5+) — фиксировано; используется для watchdog. +_DOMCLICK_NUM_BUCKETS: int = 6 +# Оценка времени одного fetch'а (network + parse) для watchdog. _DOMCLICK_PER_FETCH_S: float = 12.0 -# Буфер сверху расчётного бюджета (cold browser start, save-фаза). +# Буфер сверху расчётного бюджета (cold browser start, save-фаза, price-splits). _DOMCLICK_SWEEP_BUDGET_S: float = 300.0 @@ -3118,46 +3122,51 @@ async def run_domclick_city_sweep( run_id: int, city_id: int = DOMCLICK_DEFAULT_CITY_ID, rooms: list[int] | None = None, - pages: int = 5, + pages: int = 100, request_delay_sec: float | None = None, ) -> DomClickCitySweepCounters: - """DomClick citywide sweep: SERP по city_id × rooms × pages → save → house-match. + """DomClick citywide sweep через BFF JSON API. Структурно зеркалит run_cian_city_sweep / run_yandex_city_sweep, но CITYWIDE: - DomClick SERP не поддерживает geo-radius (fetch_around → NotImplementedError), - поэтому anchor-loop отсутствует. DomClickScraper.fetch_city уже перебирает - rooms × pages через внутренний BrowserFetcher (camoufox headless) и - дедуплицирует по source_id. + DomClick не поддерживает geo-radius (fetch_around → NotImplementedError), + поэтому anchor-loop отсутствует. DomClickScraper.fetch_city перебирает все + ROOM_BUCKETS (st/1/2/3/4/5+) через BrowserFetcher → shared mobile proxy → + обходит QRATOR. Фазы (единственный citywide проход): 1. SERP: DomClickScraper().fetch_city(city_id, rooms, pages). - 2. SAVE: save_listings(db, lots, run_id=run_id) — BaseScraper save path, - триггерит существующий house-match hook (адрес-fingerprint tier, т.к. - DomClick SERP не отдаёт lat/lon). + 2. SAVE: save_listings(db, lots, run_id=run_id) — house-match по lat/lon и + address-fingerprint (BFF отдаёт координаты). Watchdog: весь fetch_city оборачивается в asyncio.wait_for. Таймаут считается - из rooms × pages × per-fetch + buffer (минимум ANCHOR_TIMEOUT_SEC). + из 6 buckets × pages × per-fetch + buffer (минимум ANCHOR_TIMEOUT_SEC). + Реальный sweep ≈ 330 страниц (rooms=2 требует price-split). Cooperative cancel: is_cancelled(db, run_id) проверяется перед SERP-фазой. - mark_done вызывается ВСЕГДА (кроме cancel / fatal). lat=lon=None у всех lots → - house-match использует только address_fingerprint tier. + + ЧЕСТНЫЙ СТАТУС (#1968): если scraper сообщил QRATOR-блок И lots == 0 → + mark_failed с объяснением. Иначе mark_done (в т.ч. при lots==0 без блока: + genuinely empty run). Возвращает DomClickCitySweepCounters. """ - from app.services.scrapers.domclick import DomClickScraper + from app.services.scrapers.domclick import ROOM_BUCKETS, DomClickScraper - _rooms = rooms if rooms is not None else list(DOMCLICK_DEFAULT_ROOMS) - _resolved_delay = request_delay_sec if request_delay_sec is not None else 8.0 + _resolved_delay = request_delay_sec if request_delay_sec is not None else 6.0 counters = DomClickCitySweepCounters() - # Watchdog-таймаут: число fetch'ей = len(rooms) × pages (worst-case, без - # early-break на пустой странице). Каждый fetch ≈ delay + render/parse overhead. - _num_fetches = max(1, len(_rooms)) * max(1, pages) + # Watchdog: 6 buckets × pages × per_fetch + budget. + # Реальный worst-case (rooms=2 price-split) ≈ 330 страниц × (delay+12s). + # pages=100 → 600 fetch'ей × ~18s + 300 ≈ 11100s (~3ч) — намеренно generous. + _num_fetches = _DOMCLICK_NUM_BUCKETS * max(1, pages) _sweep_timeout = max( ANCHOR_TIMEOUT_SEC, int(_num_fetches * (_resolved_delay + _DOMCLICK_PER_FETCH_S) + _DOMCLICK_SWEEP_BUDGET_S), ) + # Мутируемый контейнер для захвата scraper-ссылки из замыкания. + _scraper_ref: list[DomClickScraper] = [] + try: # ── Cooperative cancel перед SERP-фазой ────────────────────────────── if scrape_runs.is_cancelled(db, run_id): @@ -3166,10 +3175,11 @@ async def run_domclick_city_sweep( return counters logger.info( - "domclick-sweep run_id=%d: citywide SERP city_id=%d rooms=%s pages=%d (watchdog %ds)", + "domclick-sweep run_id=%d: BFF citywide sweep city_id=%d " + "buckets=%d pages_cap=%d (watchdog %ds)", run_id, city_id, - _rooms, + len(ROOM_BUCKETS), pages, _sweep_timeout, ) @@ -3179,10 +3189,11 @@ async def run_domclick_city_sweep( async def _domclick_phase() -> None: """Единственная citywide-фаза: fetch_city + save.""" nonlocal lots - async with DomClickScraper() as scraper: + async with DomClickScraper() as _scraper: + _scraper_ref.append(_scraper) if request_delay_sec is not None: - scraper.request_delay_sec = _resolved_delay - lots = await scraper.fetch_city(city_id=city_id, rooms=_rooms, pages=pages) + _scraper.request_delay_sec = _resolved_delay + lots = await _scraper.fetch_city(city_id=city_id, rooms=rooms, pages=pages) counters.lots_fetched += len(lots) if lots: inserted, updated = save_listings(db, lots, run_id=run_id) @@ -3202,18 +3213,53 @@ async def run_domclick_city_sweep( logger.exception("domclick-sweep run_id=%d: SERP phase failed", run_id) counters.errors_count += 1 - # pages_fetched: worst-case число запрошенных страниц (rooms × pages). + # Перенести счётчики scraper'а в counters (если scraper был создан). + if _scraper_ref: + _s = _scraper_ref[0] + counters.blocked = 1 if _s.blocked else 0 + counters.geo_filtered = _s.geo_filtered + # Не-block fetch-ошибки скрейпера учитываем в errors_count: они должны + # влиять на honest-status (0 lots + fetch_errors → НЕ ложный mark_done). + counters.errors_count += _s.fetch_errors + + # pages_fetched: worst-case число страниц (buckets × pages cap). counters.pages_fetched = _num_fetches scrape_runs.update_heartbeat(db, run_id, counters.to_dict()) - scrape_runs.mark_done(db, run_id, counters.to_dict()) + + # ── ЧЕСТНЫЙ СТАТУС (#1968) ──────────────────────────────────────────── + # 0 lots + (block ИЛИ любая ошибка) → failed: run без результата по + # внешней причине не должен маскироваться как 'done'. Нераспознанный + # block-вариант (нет литерал-маркера) приходит как fetch_error, не blocked — + # поэтому errors_count тоже триггерит failed. Genuinely empty 0-error run + # → done. Partial run с lots>0 остаётся done даже если часть бакетов упала. + if counters.lots_fetched == 0 and (counters.blocked or counters.errors_count > 0): + logger.error( + "domclick-sweep run_id=%d: 0 listings with blocked=%d errors=%d " + "— marking failed", + run_id, + counters.blocked, + counters.errors_count, + ) + scrape_runs.mark_failed( + db, + run_id, + "QRATOR block or fetch errors — 0 listings", + counters.to_dict(), + ) + else: + scrape_runs.mark_done(db, run_id, counters.to_dict()) + logger.info( - "domclick-sweep run_id=%d done: lots=%d (ins=%d/upd=%d) pages=%d errors=%d", + "domclick-sweep run_id=%d done: lots=%d (ins=%d/upd=%d) " + "pages=%d errors=%d blocked=%d geo_filtered=%d", run_id, counters.lots_fetched, counters.lots_inserted, counters.lots_updated, counters.pages_fetched, counters.errors_count, + counters.blocked, + counters.geo_filtered, ) return counters diff --git a/tradein-mvp/backend/app/services/scrapers/domclick.py b/tradein-mvp/backend/app/services/scrapers/domclick.py index ccb71cdb..b6f3087b 100644 --- a/tradein-mvp/backend/app/services/scrapers/domclick.py +++ b/tradein-mvp/backend/app/services/scrapers/domclick.py @@ -1,28 +1,32 @@ -"""DomClick.ru scraper — вторичка через headless BrowserFetcher (#796). +"""DomClick.ru scraper — вторичка через JSON BFF API (#1968). -Стратегия: HTML scrape через camoufox (AsyncCamoufox) — единственный -рабочий путь: DataDome пропускает headless Playwright, curl_cffi -> 401. +Стратегия: GET https://bff-search-web.domclick.ru/api/offers/v1?... через +BrowserFetcher(source="domclick") (generic provider → shared mobile proxy). +QRATOR банит прямые datacenter-запросы, но пропускает через mobile proxy. -URL шаблон: - https://domclick.ru/search?deal_type=sale&category=living&offer_type=flat - &city_id={city_id}&rooms={r}&p={page} +Ответ BrowserFetcher содержит JSON, обёрнутый в HTML (
 или bare body).
+Парсинг: _extract_json() вытаскивает первый {...} из ответа.
 
-Оффер-карточки: `a[href*="/card/"]` с href-паттерном `/card/sale__flat__`.
-Координаты SERP не отдаёт (lat = lon = None).
+Стратегия нумерации комнат (ROOM_BUCKETS):
+  "st"  — студии (force rooms=0 в ScrapedLot)
+  "1"–"4" — соответственно
+  "5+"  — 5 и более комнат
 
-Важно: selectolax `.text()` объединяет все дочерние текстовые узлы без
-разделителей. Числа из адреса (номер дома) могут слипнуться с ценой.
-Поэтому цену извлекаем из каждого дочернего элемента отдельно (а не из
-суммарного card_text), где элемент содержит символ рубля.
+Пагинация: offset 0, 20, 40, …, cap=2000. Если snippetsCount > 2000
+для бакета, рекурсивно делим по цене (binary split) до тех пор пока
+каждая ветка укладывается в cap.
+
+# TODO Layer B (#1846 follow-up): backfill renovation/wallType/priceHistory
+через detail-эндпоинт после первоначального сбора.
 """
 
 from __future__ import annotations
 
+import json
 import logging
-import re
-from urllib.parse import urljoin
-
-from selectolax.parser import HTMLParser, Node
+from datetime import date
+from typing import Any
+from urllib.parse import urlencode
 
 from app.services.scraper_settings import get_scraper_delay
 from app.services.scrapers.base import BaseScraper, ScrapedLot
@@ -30,112 +34,179 @@ from app.services.scrapers.domclick_exceptions import DomClickBlockedError
 
 logger = logging.getLogger(__name__)
 
-# ── DataDome block detection ─────────────────────────────────────────────────
-_DATADOME_MARKERS = ("datadome", "blocked", "access denied", "bot detected")
+# ── API constants ─────────────────────────────────────────────────────────────
 
+_BFF_BASE = "https://bff-search-web.domclick.ru"
+_EKB_ADDRESS_GUID = "0d475b79-88de-4054-818c-37d8f9d0d440"
+_EKB_AREA_ID = "20561"
 
-def _is_blocked_page(html: str) -> bool:
-    head = html[:2048].lower()
-    return any(m in head for m in _DATADOME_MARKERS)
+# Buckets to sweep — порядок влияет на логи.
+ROOM_BUCKETS: tuple[str, ...] = ("st", "1", "2", "3", "4", "5+")
 
+OFFSET_CAP: int = 2000  # max offset принятый BFF API
+PAGE_SIZE: int = 20  # items per page (жёстко задан API)
+# Потолок цены для binary split. 1 млрд ₽ — безопасно выше любой ЕКБ-квартиры;
+# выбран так чтобы НЕ отбрасывать листинги дороже потолка при сплите unbounded-бакета
+# (родительский count не ограничен сверху → clamped _lte должен покрывать весь хвост).
+LTE_MAX: int = 1_000_000_000
+MIN_PRICE_SPAN: int = 100_000  # ниже этого span прекращаем делить
 
-# ── Regex helpers ────────────────────────────────────────────────────────────
-# _RE_ROOMS расширен относительно avito.py: DomClick пишет «2-комн. квартира»
-# (с «комн.») — avito.py-вариант ловит только «N-к. кв.» / «N-к квартира».
-_RE_ROOMS = re.compile(
-    r"(\d)-(?:комн\.?\s*(?:квартира|кв\.?)?|к\.?\s*(?:квартира|кв\.?))",
-    re.IGNORECASE,
-)
-_RE_STUDIO = re.compile(r"\bстуди[яиюей]\b", re.IGNORECASE)
-_RE_AREA = re.compile(r"(\d+[.,]?\d*)\s*м[2²]", re.IGNORECASE)
-_RE_FLOOR = re.compile(r"(\d+)\s*/\s*(\d+)\s*эт\.?", re.IGNORECASE)
+# ── QRATOR block detection ────────────────────────────────────────────────────
 
-# Цена: парсим из одного элемента DOM (не из суммарного card_text) —
-# иначе число дома может слипнуться с ценой. Цена всегда в элементе с «₽».
-_RE_PRICE_EL = re.compile(r"([\d\s  ]+)\s*[₽р](?:уб\.?)?", re.IGNORECASE)
-
-# source_id из href вида /card/sale__flat__2075671636
-_RE_SOURCE_ID = re.compile(r"sale__flat__(\d+)")
-
-# Адресные ключевые слова (эвристика)
-_RE_ADDR_KW = re.compile(
-    r"ул\.|улица|пер\.|переулок|пр-т|проспект"
-    r"|бул\.|бульвар|шоссе|наб\.|набережная"
-    r"|пл\.|площадь|тракт|д\.\s*\d",
-    re.IGNORECASE,
+_QRATOR_MARKERS: tuple[str, ...] = (
+    "qrator",
+    "bot_mitigation",
+    "система защиты",
+    "403 | домклик",
+    "captcha",
+    "access denied",
 )
 
+# ── EKB geo guard ─────────────────────────────────────────────────────────────
 
-# ── Вспомогательные функции ──────────────────────────────────────────────────
+_EKB_LAT_MIN: float = 56.6
+_EKB_LAT_MAX: float = 57.0
+_EKB_LON_MIN: float = 60.2
+_EKB_LON_MAX: float = 60.9
+_EKB_REGION_NAME: str = "Екатеринбург"
 
 
-def _extract_rooms(text: str) -> int | None:
-    if _RE_STUDIO.search(text):
-        return 0
-    m = _RE_ROOMS.search(text)
-    return int(m.group(1)) if m else None
+# ── JSON extraction ───────────────────────────────────────────────────────────
 
 
-def _extract_area(text: str) -> float | None:
-    m = _RE_AREA.search(text)
-    return float(m.group(1).replace(",", ".")) if m else None
+def _extract_json(html: str) -> dict[str, Any]:
+    """Извлекает JSON-объект из ответа BrowserFetcher.
 
+    BrowserFetcher оборачивает JSON в HTML-страницу двумя способами:
+    1. Bare body: ``{"result": ...}`` напрямую в .
+    2. 
-wrapped: ``
{"result": ...}
``.
 
-def _extract_floor(text: str) -> tuple[int | None, int | None]:
-    m = _RE_FLOOR.search(text)
-    if m:
-        return int(m.group(1)), int(m.group(2))
-    return None, None
+    Стратегия: найти первый '{' и последний '}' — работает для обоих вариантов.
 
-
-def _extract_price_from_element(text: str) -> int | None:
-    """Извлекаем цену из текста ОДНОГО элемента DOM.
-
-    Парсим из одного span/div — не из суммарного card_text. Это исключает
-    случай, когда адресный номер дома стоит вплотную перед ценой в
-    объединённом тексте карточки («Ленина, 503 100 000 руб.»).
+    Raises:
+        DomClickBlockedError: если ответ содержит QRATOR/captcha маркеры.
+        ValueError: если JSON не найден или не является dict.
     """
-    m = _RE_PRICE_EL.search(text)
-    if m:
-        # Нормализуем пробелы любого вида (обычный, неразрывный, узкий)
-        raw = re.sub(r"[\s  ]", "", m.group(1))
-        if raw.isdigit():
-            val = int(raw)
-            if val > 0:
-                return val
-    return None
+    # Сканируем ВЕСЬ ответ (а не только первые 4096B): block-маркер может
+    # стоять за пределами head в крупных challenge-страницах.
+    html_lower = html.lower()
+    if any(m in html_lower for m in _QRATOR_MARKERS):
+        raise DomClickBlockedError(
+            f"DomClick BFF: QRATOR block page detected (markers checked: {_QRATOR_MARKERS[:2]})"
+        )
+
+    start = html.find("{")
+    end = html.rfind("}")
+    if start == -1 or end == -1 or end <= start:
+        raise ValueError(f"No JSON object found in BFF response (len={len(html)})")
+
+    data = json.loads(html[start : end + 1])
+    if not isinstance(data, dict):
+        raise ValueError(f"BFF response JSON is not a dict: {type(data)}")
+    return data  # type: ignore[return-value]
 
 
-def _extract_source_id(href: str) -> str | None:
-    """Числовой ID из href типа /card/sale__flat__2075671636."""
-    m = _RE_SOURCE_ID.search(href)
-    return m.group(1) if m else None
+# ── URL builders ──────────────────────────────────────────────────────────────
+
+
+def _build_offers_url(
+    rooms: str,
+    price_gte: int | None,
+    price_lte: int | None,
+    offset: int,
+) -> str:
+    """Строит URL для GET /api/offers/v1 с пагинацией.
+
+    urlencode кодирует "5+" → "5%2B" (literal '+' в query string = space → reject).
+    """
+    params: list[tuple[str, str]] = [
+        ("address", _EKB_ADDRESS_GUID),
+        ("aids", _EKB_AREA_ID),
+        ("deal_type", "sale"),
+        ("category", "living"),
+        ("offer_type", "flat"),
+        ("rooms", rooms),
+        ("sort", "qi"),
+        ("sort_dir", "desc"),
+        ("offset", str(offset)),
+        ("limit", str(PAGE_SIZE)),
+    ]
+    if price_gte is not None:
+        params.append(("sale_price__gte", str(price_gte)))
+    if price_lte is not None:
+        params.append(("sale_price__lte", str(price_lte)))
+    return f"{_BFF_BASE}/api/offers/v1?{urlencode(params)}"
+
+
+def _build_count_url(
+    rooms: str,
+    price_gte: int | None,
+    price_lte: int | None,
+) -> str:
+    """Строит URL для GET /api/offers/count/v1 (без offset/limit)."""
+    params: list[tuple[str, str]] = [
+        ("address", _EKB_ADDRESS_GUID),
+        ("aids", _EKB_AREA_ID),
+        ("deal_type", "sale"),
+        ("category", "living"),
+        ("offer_type", "flat"),
+        ("rooms", rooms),
+        ("sort", "qi"),
+        ("sort_dir", "desc"),
+    ]
+    if price_gte is not None:
+        params.append(("sale_price__gte", str(price_gte)))
+    if price_lte is not None:
+        params.append(("sale_price__lte", str(price_lte)))
+    return f"{_BFF_BASE}/api/offers/count/v1?{urlencode(params)}"
+
+
+def _parse_publish_date(iso: str | None) -> date | None:
+    """Парсит ISO8601 дату публикации → date. None при ошибке."""
+    if not iso:
+        return None
+    try:
+        return date.fromisoformat(iso[:10])
+    except (ValueError, TypeError):
+        return None
 
 
 # ── DomClickScraper ──────────────────────────────────────────────────────────
 
 
 class DomClickScraper(BaseScraper):
-    """DomClick вторичка parser. Источник = 'domklik'.
+    """DomClick вторичка через JSON BFF API. Источник = 'domklik'.
 
-    Использует BrowserFetcher (camoufox headless Firefox) — DataDome пропускает
-    его без дополнительной stealth-настройки. curl_cffi -> 401 (DataDome блок).
+    Использует BrowserFetcher(source="domclick") (generic provider → shared mobile
+    proxy), который обходит QRATOR. Прямые curl/httpx-запросы с datacenter-IP
+    блокируются QRATOR.
 
     Основной метод: fetch_city(city_id, rooms, pages).
     fetch_around() не реализован: DomClick не поддерживает geo-radius в URL.
+
+    Counters (публичные после fetch_city):
+        parse_failures  — офферы с ошибкой маппинга
+        geo_filtered    — офферы вне ЕКБ bbox или с неверным offerRegionName
+        blocked         — True если sweep был прерван QRATOR-блоком
+        fetch_errors    — не-block ошибки извлечения JSON (truncated/garbled/bad shape)
     """
 
     name = "domklik"
     source = "domklik"
     base_url = "https://domclick.ru"
-    # DataDome — консервативная задержка между страницами (fallback до init)
+    # Консервативная задержка между страницами (fallback до init)
     request_delay_sec = 8.0
 
     def __init__(self) -> None:
         super().__init__()
         self.request_delay_sec = get_scraper_delay(self.name)
-        # Счётчик карточек с неожиданной DOM-структурой (для observability)
         self.parse_failures: int = 0
+        self.geo_filtered: int = 0
+        self.blocked: bool = False
+        # Не-block ошибки извлечения JSON (truncated/garbled response, неверная
+        # структура). В отличие от parse_failures (per-item), это per-fetch ошибки,
+        # которые ограничивают сбор бакета. Учитываются в honest-status pipeline.
+        self.fetch_errors: int = 0
 
     async def __aenter__(self) -> DomClickScraper:
         await super().__aenter__()
@@ -155,237 +226,361 @@ class DomClickScraper(BaseScraper):
         self,
         city_id: int,
         rooms: list[int] | None = None,
-        pages: int = 20,
+        pages: int = 100,
     ) -> list[ScrapedLot]:
-        """Citywide sweep: все страницы SERP для city_id.
+        """Citywide sweep через BFF JSON API.
+
+        Аргументы city_id и rooms принимаются для совместимости сигнатуры с
+        вызывающим кодом (run_domclick_city_sweep), но:
+        - city_id — vestigial (EКБ захардкожен через GUID).
+        - rooms — игнорируется; всегда обходятся все ROOM_BUCKETS внутри.
 
         Args:
-            city_id: числовой ID города в DomClick (например, 4 = Екатеринбург).
-            rooms: список значений комнатности (0=студия, 1, 2, 3, 4, ...).
-                   None -> без фильтра комнатности (все квартиры сразу).
-            pages: максимальное число страниц на один sweep. Break on empty.
+            city_id: игнорируется (EKB захардкожен).
+            rooms: игнорируется (ROOM_BUCKETS перебирается всегда).
+            pages: максимальное число страниц на бакет (safety cap).
 
         Returns:
             Дедуплицированный по source_id список ScrapedLot.
         """
+        # TODO Layer B (#1846 follow-up): backfill renovation/wallType/priceHistory
+
         from app.services.scrapers.browser_fetcher import BrowserFetcher
 
-        all_lots: list[ScrapedLot] = []
+        out_lots: list[ScrapedLot] = []
         seen_ids: set[str] = set()
 
-        # Формируем список room-значений для sweep'ов
-        room_values: list[int | None] = [None] if not rooms else list(rooms)
-
         async with BrowserFetcher(source="domclick") as fetcher:
-            for room_val in room_values:
-                room_label = f"rooms={room_val}" if room_val is not None else "all_rooms"
+            for bucket in ROOM_BUCKETS:
                 logger.info(
-                    "domklik: city_id=%d %s sweep (max %d pages)", city_id, room_label, pages
+                    "domklik: BFF sweep rooms=%r city_id=%d pages_cap=%d",
+                    bucket,
+                    city_id,
+                    pages,
                 )
-
-                for page_num in range(1, pages + 1):
-                    url = self._build_url(city_id, room_val, page_num)
-                    logger.debug("domklik: fetch %s", url)
-
-                    try:
-                        html = await fetcher.fetch(url)
-                    except Exception as exc:
-                        logger.error(
-                            "domklik: fetch failed city_id=%d page=%d: %s",
-                            city_id,
-                            page_num,
-                            exc,
-                        )
-                        break
-
-                    try:
-                        lots = self._parse_html(html)
-                    except DomClickBlockedError:
-                        logger.warning(
-                            "domklik: DataDome block at city_id=%d page=%d — stopping sweep",
-                            city_id,
-                            page_num,
-                        )
-                        break
-
-                    if not lots:
-                        logger.info(
-                            "domklik: empty page city_id=%d %s page=%d — stopping sweep",
-                            city_id,
-                            room_label,
-                            page_num,
-                        )
-                        break
-
-                    # Дедупликация по source_id (кросс-sweep и кросс-страница)
-                    new_lots: list[ScrapedLot] = []
-                    for lot in lots:
-                        key = lot.source_id or lot.source_url
-                        if key not in seen_ids:
-                            seen_ids.add(key)
-                            new_lots.append(lot)
-
-                    all_lots.extend(new_lots)
-                    logger.info(
-                        "domklik: city_id=%d %s page=%d -> %d new (total %d)",
-                        city_id,
-                        room_label,
-                        page_num,
-                        len(new_lots),
-                        len(all_lots),
+                try:
+                    await self._sweep_bucket(
+                        fetcher=fetcher,
+                        rooms=bucket,
+                        price_gte=None,
+                        price_lte=None,
+                        seen_ids=seen_ids,
+                        out_lots=out_lots,
+                        pages=pages,
                     )
-
-                    # Пауза между страницами (anti-DataDome)
-                    if page_num < pages:
-                        await self.sleep_between_requests()
+                except DomClickBlockedError:
+                    self.blocked = True
+                    logger.error(
+                        "domklik: QRATOR block during rooms=%r — aborting all buckets",
+                        bucket,
+                    )
+                    break
+                except (ValueError, TypeError) as exc:
+                    # Defensive: bucket-level ошибка не должна убивать весь sweep.
+                    # _count/_paginate уже глотают эти ошибки per-fetch (fetch_errors++),
+                    # но если что-то всё же всплыло — переходим к следующему бакету.
+                    self.fetch_errors += 1
+                    logger.warning(
+                        "domklik: bucket rooms=%r failed (%s) — skipping to next bucket",
+                        bucket,
+                        exc,
+                        exc_info=True,
+                    )
+                    continue
 
         logger.info(
-            "domklik: fetch_city done city_id=%d total=%d parse_failures=%d",
+            "domklik: fetch_city done city_id=%d total=%d "
+            "parse_failures=%d geo_filtered=%d fetch_errors=%d blocked=%s",
             city_id,
-            len(all_lots),
+            len(out_lots),
             self.parse_failures,
+            self.geo_filtered,
+            self.fetch_errors,
+            self.blocked,
         )
-        return all_lots
+        return out_lots
 
-    # ── URL builder ───────────────────────────────────────────────────────────
+    # ── Internal sweep helpers ────────────────────────────────────────────────
 
-    def _build_url(self, city_id: int, rooms: int | None, page: int) -> str:
-        """Строит URL SERP DomClick."""
-        url = (
-            f"{self.base_url}/search"
-            f"?deal_type=sale&category=living&offer_type=flat"
-            f"&city_id={city_id}"
-        )
-        if rooms is not None:
-            url += f"&rooms={rooms}"
-        url += f"&p={page}"
-        return url
+    async def _count(
+        self,
+        fetcher: Any,
+        rooms: str,
+        price_gte: int | None,
+        price_lte: int | None,
+    ) -> int:
+        """Запрашивает snippetsCount для данного бакета (rooms + price range).
 
-    # ── HTML parsing ──────────────────────────────────────────────────────────
-
-    def _parse_html(self, html: str) -> list[ScrapedLot]:
-        """Парсим карточки из HTML через selectolax.
-
-        Селектор: `a[href*="/card/"]` с паттерном `/card/sale__flat__`.
+        DomClickBlockedError пробрасывается наверх (abort sweep). Прочие ошибки
+        извлечения JSON (truncated/garbled/bad shape) → fetch_errors++ и return 0
+        (бакет пропускается, sweep продолжается).
         """
-        if _is_blocked_page(html):
-            logger.warning("domklik: DataDome block page detected, 0 cards returned")
-            raise DomClickBlockedError("DomClick returned DataDome block page")
-        tree = HTMLParser(html)
-        card_links = tree.css('a[href*="/card/"]')
+        url = _build_count_url(rooms, price_gte, price_lte)
+        logger.debug("domklik: count url=%s", url)
+        html = await fetcher.fetch(url)
+        try:
+            data = _extract_json(html)
+        except DomClickBlockedError:
+            raise
+        except (ValueError, TypeError, AttributeError):
+            self.fetch_errors += 1
+            logger.warning(
+                "domklik: _count JSON extract failed rooms=%r price=[%s,%s] — skip bucket",
+                rooms,
+                price_gte,
+                price_lte,
+                exc_info=True,
+            )
+            return 0
+        # {"result": null} → None, {"snippetsCount": null} → None: harden обе.
+        res = data.get("result") or {}
+        raw = res.get("snippetsCount")
+        return int(raw) if raw else 0
 
-        seen_hrefs: set[str] = set()
-        lots: list[ScrapedLot] = []
+    async def _sweep_bucket(
+        self,
+        fetcher: Any,
+        rooms: str,
+        price_gte: int | None,
+        price_lte: int | None,
+        seen_ids: set[str],
+        out_lots: list[ScrapedLot],
+        pages: int,
+    ) -> None:
+        """Рекурсивный sweep бакета (rooms, price_gte, price_lte).
 
-        for link in card_links:
-            href = link.attributes.get("href", "")
-            # Только карточки вторичного жилья
-            if "sale__flat__" not in href:
-                continue
-            if href in seen_hrefs:
-                continue
-            seen_hrefs.add(href)
+        Если snippetsCount > OFFSET_CAP — делим диапазон цен пополам и
+        рекурсируем в каждую половину. Остановка рекурсии:
+        - span <= MIN_PRICE_SPAN → пагинируем как есть (с предупреждением о truncation)
+        - snippetsCount == 0 → пропускаем
+        - snippetsCount <= OFFSET_CAP → пагинируем
 
-            lot = self._card_link_to_lot(link, href)
-            if lot is not None:
-                lots.append(lot)
+        Raises:
+            DomClickBlockedError: если QRATOR-блок — propagate наверх.
+        """
+        count = await self._count(fetcher, rooms, price_gte, price_lte)
+        if count == 0:
+            return
 
-        return lots
+        if count > OFFSET_CAP:
+            _gte = price_gte if price_gte is not None else 0
+            _lte = price_lte if price_lte is not None else LTE_MAX
+            span = _lte - _gte
+            if span <= MIN_PRICE_SPAN:
+                logger.warning(
+                    "domklik: rooms=%r price=[%s,%s] count=%d > cap=%d "
+                    "but span=%d <= min=%d — paginating as-is (bucket truncated at %d)",
+                    rooms,
+                    price_gte,
+                    price_lte,
+                    count,
+                    OFFSET_CAP,
+                    span,
+                    MIN_PRICE_SPAN,
+                    OFFSET_CAP,
+                )
+                await self._paginate(
+                    fetcher, rooms, price_gte, price_lte, seen_ids, out_lots, pages
+                )
+                return
+            mid = (_gte + _lte) // 2
+            logger.debug(
+                "domklik: rooms=%r count=%d > cap=%d — price-split [%d,%d] → [%d,%d]+[%d,%d]",
+                rooms,
+                count,
+                OFFSET_CAP,
+                _gte,
+                _lte,
+                _gte,
+                mid,
+                mid + 1,
+                _lte,
+            )
+            await self._sweep_bucket(fetcher, rooms, _gte, mid, seen_ids, out_lots, pages)
+            await self._sweep_bucket(fetcher, rooms, mid + 1, _lte, seen_ids, out_lots, pages)
+        else:
+            await self._paginate(fetcher, rooms, price_gte, price_lte, seen_ids, out_lots, pages)
 
-    def _card_link_to_lot(self, link: Node, href: str) -> ScrapedLot | None:
-        """Парсинг одной карточки-ссылки -> ScrapedLot.
+    async def _paginate(
+        self,
+        fetcher: Any,
+        rooms: str,
+        price_gte: int | None,
+        price_lte: int | None,
+        seen_ids: set[str],
+        out_lots: list[ScrapedLot],
+        pages: int,
+    ) -> None:
+        """Пагинирует один бакет (rooms, price range) до исчерпания или cap.
 
-        DomClick SERP оборачивает карточку в `` — вся
-        информация (title, цена, адрес) внутри этого элемента.
+        DomClickBlockedError пробрасывается наверх (abort sweep). Прочие ошибки
+        извлечения JSON (truncated/garbled/bad shape) → fetch_errors++ и break
+        (трактуем как конец бакета — уже собранные lots сохраняются).
 
-        Цену извлекаем из отдельных дочерних элементов (не из объединённого
-        card_text) чтобы исключить слипание числа дома с ценой.
+        Raises:
+            DomClickBlockedError: propagate из _extract_json при QRATOR-блоке.
+        """
+        force_rooms = 0 if rooms == "st" else None
+        page_idx = 0
+        while page_idx < pages:
+            offset = page_idx * PAGE_SIZE
+            if offset >= OFFSET_CAP:
+                break
+            url = _build_offers_url(rooms, price_gte, price_lte, offset)
+            logger.debug("domklik: offers url=%s", url)
+            html = await fetcher.fetch(url)
+            try:
+                data = _extract_json(html)
+            except DomClickBlockedError:
+                raise
+            except (ValueError, TypeError, AttributeError):
+                self.fetch_errors += 1
+                logger.warning(
+                    "domklik: _paginate JSON extract failed rooms=%r offset=%d "
+                    "— ending bucket (lots so far preserved)",
+                    rooms,
+                    offset,
+                    exc_info=True,
+                )
+                break
+
+            # {"result": null} → None; harden до .get("items").
+            items: list[dict[str, Any]] = (data.get("result") or {}).get("items") or []
+            if not items:
+                break
+
+            for item in items:
+                lot = self._map_item(item, force_rooms=force_rooms)
+                if lot is None:
+                    continue
+                if not self._is_geo_ok(item):
+                    self.geo_filtered += 1
+                    continue
+                key = lot.source_id or lot.source_url
+                if key in seen_ids:
+                    continue
+                seen_ids.add(key)
+                out_lots.append(lot)
+
+            page_idx += 1
+            if page_idx < pages and len(items) == PAGE_SIZE:
+                await self.sleep_between_requests()
+            elif not items or len(items) < PAGE_SIZE:
+                break
+
+    def _is_geo_ok(self, item: dict[str, Any]) -> bool:
+        """Гео-гард: пропускает только листинги ЕКБ в bbox.
+
+        aids=20561 даёт чистый ЕКБ, но гард оставляем как defensive проверку.
+        """
+        region = item.get("offerRegionName", "")
+        if region != _EKB_REGION_NAME:
+            return False
+        loc = item.get("location") or {}
+        lat = loc.get("lat")
+        lon = loc.get("lon")
+        if lat is None or lon is None:
+            return False
+        return (
+            _EKB_LAT_MIN <= float(lat) <= _EKB_LAT_MAX
+            and _EKB_LON_MIN <= float(lon) <= _EKB_LON_MAX
+        )
+
+    def _map_item(
+        self, item: dict[str, Any], *, force_rooms: int | None = None
+    ) -> ScrapedLot | None:
+        """Маппинг offer-item из BFF API → ScrapedLot.
+
+        Возвращает None если price <= 0 или при ошибке парсинга.
         """
         try:
-            source_id = _extract_source_id(href)
-            source_url = urljoin(self.base_url, href)
-
-            # Суммарный текст — для rooms/area/floor (эти поля не подвержены
-            # проблеме слипания, т.к. используют специфичные маркеры: «м²», «эт.»)
-            card_text = link.text(strip=True)
-            if not card_text:
+            price = item.get("price", 0) or 0
+            if price <= 0:
                 return None
 
-            rooms = _extract_rooms(card_text)
-            area = _extract_area(card_text)
-            floor, total_floors = _extract_floor(card_text)
+            item_id = item.get("id")
+            source_id = str(item_id) if item_id is not None else None
+            source_url: str = item.get("path", "") or ""
 
-            # Цена: ищем в каждом дочернем элементе отдельно чтобы не слипались
-            # числа дома с ценой при конкатенации card_text
-            price = self._extract_price_from_children(link)
-            if not price or price <= 0:
-                return None
+            loc = item.get("location") or {}
+            lat: float | None = loc.get("lat")
+            lon: float | None = loc.get("lon")
 
-            address = self._extract_address(link)
+            address_obj = item.get("address") or {}
+            address: str | None = address_obj.get("displayName")
+
+            # 0 — sentinel «неизвестно» для area/floor/total_floors/buildYear
+            # (DomClick эмитит 0 для under-construction / unknown) → None.
+            obj_info = item.get("objectInfo") or {}
+            area_raw = obj_info.get("area")
+            area_m2: float | None = float(area_raw) if area_raw else None
+            floor_raw = obj_info.get("floor")
+            floor: int | None = int(floor_raw) if floor_raw else None
+
+            house = item.get("house") or {}
+            floors_raw = house.get("floors")
+            total_floors: int | None = int(floors_raw) if floors_raw else None
+            year_raw = house.get("buildYear")
+            year_built: int | None = int(year_raw) if year_raw else None
+
+            # rooms: 0 — ВАЛИДНОЕ значение (студия), поэтому проверка is not None,
+            # а не truthy. force_rooms=0 (st-бакет) перебивает objectInfo.
+            if force_rooms is not None:
+                rooms: int | None = force_rooms
+            else:
+                rooms_raw = obj_info.get("rooms")
+                rooms = int(rooms_raw) if rooms_raw is not None else None
+
+            square_price_raw = item.get("squarePrice")
+            price_per_m2: int | None = int(square_price_raw) if square_price_raw else None
+            if price_per_m2 is None and area_m2 and area_m2 > 0:
+                price_per_m2 = int(price / area_m2)
+
+            flat_complex = item.get("flatComplex") or {}
+            raw_payload: dict[str, Any] = {
+                "isRosreestrApproved": item.get("isRosreestrApproved"),
+                "squarePrice": square_price_raw,
+                "lastPriceHistoryState": item.get("lastPriceHistoryState"),
+                "flatComplex": {
+                    "name": flat_complex.get("name"),
+                    "id": flat_complex.get("id"),
+                    "slug": flat_complex.get("slug"),
+                }
+                if flat_complex
+                else None,
+                "updatedDate": item.get("updatedDate"),
+            }
+
+            publish_date = _parse_publish_date(item.get("publishedDate"))
 
             return ScrapedLot(
                 source="domklik",
                 source_url=source_url,
                 source_id=source_id,
                 address=address,
-                lat=None,  # DomClick SERP координаты не отдаёт
-                lon=None,
+                lat=lat,
+                lon=lon,
                 rooms=rooms,
-                area_m2=area,
+                area_m2=area_m2,
                 floor=floor,
                 total_floors=total_floors,
-                price_rub=price,
+                year_built=year_built,
+                price_rub=int(price),
+                price_per_m2=price_per_m2,
                 listing_segment="vtorichka",
-                raw_payload={"card_text": card_text[:500]},
+                publish_date=publish_date,
+                raw_payload=raw_payload,
             )
         except Exception:
             self.parse_failures += 1
             logger.warning(
-                "domclick _card_link_to_lot: parse failed href=%r (parse_failures=%d)",
-                href,
+                "domklik: _map_item failed for item id=%r (parse_failures=%d)",
+                item.get("id"),
                 self.parse_failures,
                 exc_info=True,
             )
             return None
 
-    def _extract_price_from_children(self, link: Node) -> int | None:
-        """Ищем цену в дочерних элементах карточки по символу рубля.
-
-        Итерируем по всем потомкам: первый элемент с рублёвым символом
-        в тексте — источник цены.
-        """
-        for selector in ("span", "div", "p", "strong", "b"):
-            for el in link.css(selector):
-                text = el.text(strip=True)
-                price = _extract_price_from_element(text)
-                if price:
-                    return price
-        # Fallback: суммарный текст карточки (если структура нестандартная)
-        return _extract_price_from_element(link.text(strip=True))
-
-    def _extract_address(self, link: Node) -> str | None:
-        """Эвристика извлечения адреса из DOM карточки.
-
-        DomClick помещает адрес в отдельный текстовый блок внутри карточки.
-        Обходим вложенные элементы и ищем первый текст с адресными ключевыми
-        словами, пропуская строки с площадью/этажом/ценой.
-
-        Возвращает None если адрес не найден.
-        """
-        for selector in ("span", "p", "div"):
-            for el in link.css(selector):
-                text = el.text(strip=True)
-                if not text or len(text) < 5 or len(text) > 200:
-                    continue
-                # Пропускаем блоки с площадью / этажом / ценой
-                if _RE_AREA.search(text) or _RE_FLOOR.search(text):
-                    continue
-                if _RE_PRICE_EL.search(text):
-                    continue
-                if _RE_ADDR_KW.search(text):
-                    return text
-        return None
-
 
 # ── Convenience runner (для Celery tasks) ────────────────────────────────────
 
@@ -393,14 +588,14 @@ class DomClickScraper(BaseScraper):
 async def scrape_domclick_city(
     city_id: int,
     rooms: list[int] | None = None,
-    pages: int = 20,
+    pages: int = 100,
 ) -> list[ScrapedLot]:
     """Удобная точка входа для вызова из Celery tasks.
 
     Пример::
 
         import asyncio
-        lots = asyncio.run(scrape_domclick_city(city_id=4, rooms=[1, 2, 3]))
+        lots = asyncio.run(scrape_domclick_city(city_id=4, rooms=None))
     """
     async with DomClickScraper() as scraper:
         return await scraper.fetch_city(city_id=city_id, rooms=rooms, pages=pages)
diff --git a/tradein-mvp/backend/app/services/scrapers/domclick_exceptions.py b/tradein-mvp/backend/app/services/scrapers/domclick_exceptions.py
index 118adb72..796251cc 100644
--- a/tradein-mvp/backend/app/services/scrapers/domclick_exceptions.py
+++ b/tradein-mvp/backend/app/services/scrapers/domclick_exceptions.py
@@ -2,4 +2,12 @@
 
 
 class DomClickBlockedError(Exception):
-    """DomClick вернул DataDome block-страницу (HTTP 200 + block HTML)."""
+    """DomClick BFF вернул QRATOR block-страницу (HTTP 200 + block HTML).
+
+    QRATOR (qrator.net) — WAF/DDoS-защита domclick.ru. Блокирует datacenter-IP
+    и возвращает HTML с маркерами: "qrator", "bot_mitigation", "система защиты",
+    "403 | домклик", "captcha", "access denied".
+
+    Обходится через shared mobile proxy (BrowserFetcher(source="domclick") →
+    generic provider → мобильный egress).
+    """
diff --git a/tradein-mvp/backend/data/sql/138_domclick_bff_rewrite_schedule.sql b/tradein-mvp/backend/data/sql/138_domclick_bff_rewrite_schedule.sql
new file mode 100644
index 00000000..5e053e34
--- /dev/null
+++ b/tradein-mvp/backend/data/sql/138_domclick_bff_rewrite_schedule.sql
@@ -0,0 +1,33 @@
+-- 138_domclick_bff_rewrite_schedule.sql
+--
+-- Обновляет параметры расписания domclick_city_sweep после переписки скрейпера
+-- на BFF JSON API (https://bff-search-web.domclick.ru/api/offers/v1).
+--
+-- DORMANT: enabled остаётся false до тех пор пока оператор не верифицирует
+-- прод-smoke нового BFF-скрейпера вручную:
+--   UPDATE scrape_schedules SET enabled = true WHERE source = 'domclick_city_sweep';
+--
+-- Ключевые изменения относительно 122_enable_domclick_city_sweep.sql:
+--   - pages_per_anchor поднят до 100 (каждый бакет до 2000 листингов,
+--     rooms=2 нуждается в price-split → реально ≈330 страниц)
+--   - rooms убран из default_params — скрейпер всегда обходит все ROOM_BUCKETS
+--     (st/1/2/3/4/5+) внутри независимо от этого параметра
+--   - request_delay_sec оставлен 6 (как в 122; BFF JSON легче HTML-рендера)
+--
+-- Зависимости: 052_scrape_schedules.sql (таблица), 122_enable_domclick_city_sweep.sql
+--   (строка должна существовать — INSERT ON CONFLICT DO NOTHING в 122).
+-- Idempotent: UPDATE не упадёт если строка уже обновлена (runs повторно безопасно).
+
+BEGIN;
+
+UPDATE scrape_schedules
+   SET enabled = false,
+       default_params = jsonb_build_object(
+           'city_id', 4,
+           'pages_per_anchor', 100,
+           'request_delay_sec', 6
+       ),
+       updated_at = now()
+ WHERE source = 'domclick_city_sweep';
+
+COMMIT;
diff --git a/tradein-mvp/backend/scripts/backtest_estimator.py b/tradein-mvp/backend/scripts/backtest_estimator.py
index e9496665..0508c85a 100644
--- a/tradein-mvp/backend/scripts/backtest_estimator.py
+++ b/tradein-mvp/backend/scripts/backtest_estimator.py
@@ -1,74 +1,85 @@
-"""Backtest harness — measures the estimator's asking-median accuracy vs real ДКП sold prices.
+"""Backtest harness — measures the estimator's accuracy vs real ДКП sold prices.
 
-Forgejo issue #648. **STRICTLY READ-ONLY**: this script issues only SELECT
-queries against prod. It never INSERTs, UPDATEs, or runs DDL.
+Forgejo issues #648 (asking-core) + #1966 (full spine). **STRICTLY READ-ONLY**:
+this script issues only SELECT queries against prod. It never INSERTs, UPDATEs,
+or runs DDL.
 
-WHAT IT MEASURES
-----------------
-The estimator predicts a квартира's value from the median of *active asking*
-prices of nearby analog listings (Tukey 1.5×IQR filter → median ppm²). Ground
-truth is `deals` — registered rosreestr ДКП sales (source='rosreestr') that
-carry geom + price_per_m2 + rooms. For a held-out sample of ДКП deals we:
+TWO ENGINES (``--engine``)
+--------------------------
+  * ``full`` (DEFAULT, #1966) — runs the **full deterministic pricing spine**
+    via ``estimator._price_from_inputs``: the same analog tier ladder
+    (Tier 0 cohort → Tier A 1 km → wide 2 km → widearea ±25 %), same-building
+    anchor comps, ДКП corridor clamp/floor, house Avito-IMV anchor, quarter
+    price index, and the asking→sold ratio that yields ``expected_sold``. The
+    PRODUCT-relevant prediction is ``expected_sold_per_m2`` (we also keep the
+    asking headline ``median_ppm2`` so both are reported).
+  * ``asking-core`` (#648) — the legacy asking-median + Tukey-IQR CORE only,
+    plus the in-/out-of-sample per-rooms asking→sold correction block. Kept for
+    comparison against the full spine.
 
-  1. Sample deals (id, lon, lat, rooms, sold_ppm2, deal_date).
-  2. For each deal, fetch nearby *active* listings of the same rooms within
-     `--radius` metres (PostGIS ST_DWithin on geography).
-  3. Predict via the estimator's OWN pure functions for fidelity:
-     `_filter_outliers` (Tukey IQR) → `_percentile(sorted, 0.5)` (median).
-     This mirrors exactly what `estimate_quality` does to its analog pool.
-     Deals with <3 surviving candidates are skipped (no prediction).
-  4. Per deal: signed_error_pct = 100*(pred - sold)/sold; abs = |signed|.
-  5. Aggregate overall + per-rooms: n_matched, n_no_analogs, median_bias_pct
-     (systematic over/under), MAPE (median |error|), p25/p75 of signed error.
-     Plus a city-wide deal_median_ppm2 vs ask_median_ppm2 headline spread.
+WHAT IT MEASURES (full spine)
+-----------------------------
+Ground truth is `deals` — registered rosreestr ДКП sales (source='rosreestr')
+that carry geom + price_per_m2 + rooms + area_m2 + floor/total_floors/year/type.
+For a held-out sample of ДКП deals we, per deal:
 
-CORRECTION BLOCK (issue #648 Stage 1)
--------------------------------------
-On top of the raw ASKING metrics the harness now emits a second CORRECTED
-block. From the SAME matched sample it derives a per-rooms asking→sold ratio
-``ratio[bucket] = median(sold_ppm2) / median(pred_ask_ppm2)`` (global fallback
-for thin buckets), then re-scores ``pred_sold = pred_ask * ratio[bucket]``
-through the same `_compute_metrics`. This DEMONSTRATES that a per-rooms factor
-removes the systematic +29.6% asking→sold bias.
+  1. Replicate ``estimate_quality``'s analog tier ladder with ``_fetch_analogs``
+     (same constants: DEFAULT_RADIUS_M, FALLBACK_RADIUS_M, MIN_ANALOGS_TIER_0,
+     the <5/<3 widen thresholds) and pre-fetch the spine inputs
+     (``_fetch_anchor_comps``, ``_fetch_dkp_corridor``,
+     ``_fetch_house_imv_anchor``) + inject the DB callables
+     (``_get_asking_sold_ratio``, ``_lookup_quarter_index(es)``).
+  2. Call ``_price_from_inputs`` for a byte-identical headline + expected_sold.
+     Deals the spine cannot price (median<=0 / <3 analogs) are skipped.
+  3. Score ``expected_sold_per_m2`` vs the realised SOLD ppm².
 
-  !! HONESTY — this ratio is IN-SAMPLE by default: it is derived AND evaluated
-     on the same deals, so the corrected bias lands near zero BY CONSTRUCTION.
-     That proves the MECHANISM, NOT out-of-sample accuracy. Pass
-     ``--holdout-split`` to fit on even-id deals and evaluate on the odd-id
-     half (deterministic, no RNG) for an honest number. The production ratio
-     (Stage 2) is fit over a SEPARATE window and the real A/B is current-vs-
-     corrected on held-out data. This script changes NOTHING in prod — it only
-     proves the correction is worth building.
+METRICS (full spine)
+--------------------
+  * EXPECTED_SOLD signed error: median bias % + MAPE (median |%|), OVERALL +
+    per-rooms + per-price-segment (эконом/комфорт/бизнес/элит/премиум).
+  * RANGE COVERAGE: fraction of deals whose actual sold TOTAL (sold_ppm2 ×
+    area) falls inside the predicted expected_sold RUB range — OVERALL +
+    per-confidence bucket.
+  * CONFIDENCE CALIBRATION: n / coverage / MAPE per confidence bucket
+    (high should be tighter & more accurate — surfaces R2 risk).
+  * SHARPNESS: median relative range width (high-low)/point — guards against
+    gaming coverage with very wide ranges.
+  * City-wide deal_median_ppm2 vs ask_median_ppm2 headline spread.
 
 CAVEATS (read these before trusting the numbers)
 -----------------------------------------------
   (a) TIME MISMATCH — this compares **CURRENT** active listings against
       **PAST** sold deals. It is NOT a point-in-time backtest: a deal closed
-      in 2025-06 is being judged against listings active today. As the market
-      moves, asking prices drift away from the historical sold price, which
-      inflates the apparent bias. A faithful point-in-time backtest needs
-      `listing_source_snapshots` (#570) — once that table accumulates enough
-      history we can query the asking median *as of each deal's date*.
-  (b) PARTIAL ESTIMATOR — this exercises only the asking-median + IQR CORE.
-      It does NOT include the full estimator's tiering (same-house / cohort /
-      class-coef), DaData enrichment, Avito-IMV, Cian/Yandex valuation, or the
-      repair-state coefficient. Real estimate accuracy may differ.
+      in 2025-06 is judged against listings active today, so asking prices
+      drift away from the historical sold price as the market moves. Treat the
+      output as a REGRESSION BASELINE (relative change between runs / engines),
+      NOT absolute truth. A faithful point-in-time backtest needs
+      `listing_source_snapshots` (#570).
+  (b) NETWORK LAYERS EXCLUDED — the spine runs offline, so the on-demand
+      network valuation sources (Avito-IMV on-demand eval, Yandex valuation,
+      Cian valuation) are NOT exercised (``imv_eval=None``,
+      ``yandex/cian_val_present=False``). The populated `house_imv_evaluations`
+      anchor IS injected, but it keys on house_id which the harness does not
+      resolve (target_house_id=None) → in practice no IMV anchor fires. Real
+      estimate accuracy with the network layers may differ.
   (c) ДКП ≠ TRUE MARKET — a registered ДКП price is what the parties declared
       to rosreestr; it can diverge from the genuine transaction price (tax
       optimisation, related-party sales, etc.).
 
 PERFORMANCE
 -----------
-One spatial listings subquery runs per sampled deal, so runtime scales with
-`--sample`. The default (300) finishes quickly; large samples (thousands) are
-slow because of the per-deal PostGIS ST_DWithin scan. Bump `--sample` only
-when you need tighter per-rooms confidence intervals.
+The full spine issues several spatial SELECTs per sampled deal (the tier ladder
++ anchor/corridor/imv pre-fetches), so runtime scales with `--sample`. The
+default (300) is fine; large samples (thousands) are slow.
 
 USAGE
 -----
     DATABASE_URL=postgresql+psycopg://... \
         python -m scripts.backtest_estimator --sample 300 --since 2025-06-01
 
+    # legacy asking-median core + correction block:
+    python -m scripts.backtest_estimator --engine asking-core
+
     # machine-readable:
     python -m scripts.backtest_estimator --json
 """
@@ -81,6 +92,7 @@ import logging
 import statistics
 from dataclasses import dataclass
 from pathlib import Path
+from types import SimpleNamespace
 from typing import Any
 
 from sqlalchemy import text
@@ -107,6 +119,37 @@ def _import_estimator() -> tuple[Any, Any]:
     return _filter_outliers, _percentile
 
 
+def _import_estimator_full() -> SimpleNamespace:
+    """Lazy import of the FULL pricing spine + the helpers it needs (#1966).
+
+    Returns a namespace bundling the estimator module (``m``), its ``settings``
+    singleton, and the ``GeocodeResult`` dataclass. Deferred for the same reason
+    as _import_estimator — importing app.services.estimator pulls
+    app.core.config.Settings, which fail-fasts when DATABASE_URL is unset, so we
+    keep it out of `--help` / the pure-metric unit tests.
+
+    Everything the full-spine prediction path calls lives on ``ns.m``:
+    _price_from_inputs, _fetch_analogs, _fetch_anchor_comps, _fetch_dkp_corridor,
+    _fetch_house_imv_anchor, _get_asking_sold_ratio, _lookup_quarter_index(es),
+    _target_cohort_range, and the DEFAULT_RADIUS_M / FALLBACK_RADIUS_M /
+    MIN_ANALOGS_TIER_0 constants.
+    """
+    try:
+        from app.services import estimator as est_mod  # type: ignore[import-not-found]
+    except ImportError:  # pragma: no cover — fallback for adhoc invocation
+        import sys
+
+        sys.path.insert(0, str(Path(__file__).resolve().parents[1]))
+        from app.services import estimator as est_mod
+    # settings + GeocodeResult are re-exported as module attributes of estimator
+    # (imported there at module level), so we read them off the same object.
+    return SimpleNamespace(
+        m=est_mod,
+        settings=est_mod.settings,
+        GeocodeResult=est_mod.GeocodeResult,
+    )
+
+
 def _session() -> Session:
     """Lazy SessionLocal factory — see _import_estimator for why it's deferred."""
     try:
@@ -144,6 +187,26 @@ ROOM_BUCKETS: tuple[int, ...] = (0, 1, 2, 3, 4)
 # of deals. See _derive_room_ratios.
 MIN_BUCKET = 20
 
+# Confidence buckets the spine emits (estimator._compute_confidence /
+# anchor / coarse-geo gates only ever return one of these three). Used by the
+# range-coverage + calibration breakdowns. Any unexpected value → "other".
+CONFIDENCE_BUCKETS: tuple[str, ...] = ("high", "medium", "low")
+
+# Price-segment bands by ₽/m² (EKB вторичка). The estimator has NO reusable
+# price-tier constant — `listing_segment` is categorical (vtorichka/novostroyki)
+# and DEAL_MIN/MAX_PPM2 are sanity bounds, not class bands — so these are
+# defined here. TUNABLE: rough EKB market tiers (эконом < комфорт < бизнес <
+# элит < премиум). Each entry is (label, upper_bound_exclusive); a value lands
+# in the first band whose upper bound it is below; the last band catches the
+# tail (+inf). Verified against config notes: p99.9 deals ≈ 500k, премиум ~680k.
+PRICE_SEGMENTS_PPM2: tuple[tuple[str, float], ...] = (
+    ("эконом", 120_000.0),
+    ("комфорт", 160_000.0),
+    ("бизнес", 220_000.0),
+    ("элит", 300_000.0),
+    ("премиум", float("inf")),
+)
+
 
 # --------------------------------------------------------------------------- #
 # Data carriers
@@ -152,7 +215,12 @@ MIN_BUCKET = 20
 
 @dataclass
 class DealSample:
-    """One held-out ДКП deal to backtest against."""
+    """One held-out ДКП deal to backtest against.
+
+    The full spine (#1966) needs the extra unit/house fields (area, address,
+    floor, total_floors, year_built, house_type) to replicate ``_fetch_analogs``
+    + the anchor/corridor pre-fetches; the asking-core engine ignores them.
+    """
 
     id: int
     lon: float
@@ -160,6 +228,40 @@ class DealSample:
     rooms: int
     sold_ppm2: float
     deal_date: Any  # datetime.date | None — carried through for reporting only
+    area_m2: float = 0.0
+    address: str | None = None
+    floor: int | None = None
+    total_floors: int | None = None
+    year_built: int | None = None
+    house_type: str | None = None
+
+
+@dataclass
+class Prediction:
+    """One full-spine prediction for a deal that DID get priced (#1966).
+
+    Carries both the asking headline (``median_ppm2``) and the product-relevant
+    ``expected_sold_*`` outputs. Range fields are RUB TOTALS (the spine emits
+    expected_sold_range_low/high as totals, not ₽/m²). ``expected_sold_*`` may be
+    None when the spine produced a headline but no asking→sold ratio resolved.
+    """
+
+    deal_id: int
+    rooms: int
+    area_m2: float
+    sold_ppm2: float
+    median_ppm2: float  # asking headline ₽/m²
+    confidence: str
+    anchor_tier: str | None
+    expected_sold_ppm2: float | None
+    expected_sold_price: float | None  # RUB total point
+    range_low: float | None  # expected_sold_range_low — RUB total
+    range_high: float | None  # expected_sold_range_high — RUB total
+
+    @property
+    def sold_total(self) -> float:
+        """Realised sold price as a RUB total (sold ₽/m² × area)."""
+        return self.sold_ppm2 * self.area_m2
 
 
 # --------------------------------------------------------------------------- #
@@ -374,6 +476,206 @@ def _apply_ratios(
     return out
 
 
+# --------------------------------------------------------------------------- #
+# Full-spine metric helpers (#1966) — PURE, no DB. Unit-tested.
+# --------------------------------------------------------------------------- #
+
+
+def _bucketize_confidence(confidence: str) -> str:
+    """Map a confidence string to a calibration bucket (unknown → 'other')."""
+    return confidence if confidence in CONFIDENCE_BUCKETS else "other"
+
+
+def _segment_label(ppm2: float) -> str:
+    """Price-segment label for a ₽/m² value (see PRICE_SEGMENTS_PPM2 bands)."""
+    for label, upper in PRICE_SEGMENTS_PPM2:
+        if ppm2 < upper:
+            return label
+    return PRICE_SEGMENTS_PPM2[-1][0]  # +inf tail — unreachable, defensive
+
+
+def _segment_metrics(rows: list[tuple[float, float]]) -> dict[str, dict[str, Any]]:
+    """Per-price-segment signed-error summary, bucketed by the SOLD ₽/m².
+
+    Each input row is ``(pred_ppm2, sold_ppm2)``. We bucket by the segment of the
+    SOLD price (ground truth), compute signed_error_pct = 100*(pred-sold)/sold,
+    and run `_errors_summary` per band. Rows with sold<=0 are dropped (can't
+    divide). Every band in PRICE_SEGMENTS_PPM2 is present (n=0 when empty) so the
+    report renders a stable table. Pure: no DB.
+    """
+    by_seg: dict[str, list[float]] = {label: [] for label, _ in PRICE_SEGMENTS_PPM2}
+    for pred, sold in rows:
+        if sold <= 0:
+            continue
+        by_seg[_segment_label(sold)].append(100.0 * (pred - sold) / sold)
+    out: dict[str, dict[str, Any]] = {}
+    for label, _ in PRICE_SEGMENTS_PPM2:
+        out[label] = _errors_summary(by_seg[label])
+    return out
+
+
+def _range_coverage(rows: list[tuple[float, float, float]]) -> dict[str, Any]:
+    """Coverage = fraction of (actual, low, high) rows with low<=actual<=high.
+
+    Boundary INCLUSIVE (a deal sitting exactly on the range edge counts as
+    covered). Empty input → coverage_pct None. Pure: no DB.
+    """
+    n = len(rows)
+    if n == 0:
+        return {"n": 0, "n_covered": 0, "coverage_pct": None}
+    covered = sum(1 for actual, lo, hi in rows if lo <= actual <= hi)
+    return {"n": n, "n_covered": covered, "coverage_pct": round(100.0 * covered / n, 2)}
+
+
+def _sharpness(rows: list[tuple[float, float, float]]) -> dict[str, Any]:
+    """Median relative range width = median((high-low)/point) over (point,lo,hi).
+
+    Guards against gaming coverage with arbitrarily wide ranges: a model can
+    always cover 100% by predicting an infinite range, so we report how WIDE the
+    ranges are relative to the point estimate. Rows with point<=0 are dropped.
+    Empty → median_rel_width None. Pure: no DB.
+    """
+    widths = [(hi - lo) / point for point, lo, hi in rows if point > 0]
+    if not widths:
+        return {"n": 0, "median_rel_width": None}
+    _, _percentile = _import_estimator()  # reuse estimator's interpolation percentile
+    return {"n": len(widths), "median_rel_width": round(_percentile(sorted(widths), 0.5), 4)}
+
+
+def _calibration_metrics(
+    rows: list[tuple[str, float | None, bool | None]],
+    *,
+    bucket_order: list[str] | None = None,
+) -> dict[str, dict[str, Any]]:
+    """Per-confidence n / coverage% / MAPE% — surfaces the R2 risk.
+
+    Each input row is ``(confidence, signed_error_pct | None, covered | None)``:
+      - ``n``           = predictions in this confidence bucket (always counted).
+      - ``coverage_pct``= 100 * mean(covered) over rows where covered is not None.
+      - ``mape_pct``    = median(|signed_error_pct|) over rows where signed is not
+                          None (the brief's median-abs-error MAPE definition).
+
+    A well-calibrated estimator shows HIGH confidence = higher coverage AND lower
+    MAPE than LOW. The canonical high/medium/low buckets are always present (n=0
+    when empty); an "other" bucket appears only if an off-enum confidence shows
+    up. Pure: no DB.
+    """
+    order = list(bucket_order) if bucket_order is not None else list(CONFIDENCE_BUCKETS)
+    coll: dict[str, dict[str, Any]] = {}
+    for confidence, signed, covered in rows:
+        b = _bucketize_confidence(confidence)
+        slot = coll.setdefault(b, {"n": 0, "abs_errors": [], "covered": []})
+        slot["n"] += 1
+        if signed is not None:
+            slot["abs_errors"].append(abs(signed))
+        if covered is not None:
+            slot["covered"].append(covered)
+        if b not in order:
+            order.append(b)
+
+    out: dict[str, dict[str, Any]] = {}
+    for b in order:
+        slot = coll.get(b, {"n": 0, "abs_errors": [], "covered": []})
+        cov = slot["covered"]
+        abs_errors = slot["abs_errors"]
+        n_covered = sum(1 for c in cov if c)
+        out[b] = {
+            "n": slot["n"],
+            "n_covered": n_covered,
+            "coverage_pct": (round(100.0 * n_covered / len(cov), 2) if cov else None),
+            "mape_pct": (round(statistics.median(abs_errors), 2) if abs_errors else None),
+        }
+    return out
+
+
+def _expected_sold_metrics(
+    rows: list[tuple[float, float, int]],
+    *,
+    n_no_prediction: int = 0,
+    per_rooms_no_prediction: dict[int, int] | None = None,
+) -> dict[str, Any]:
+    """expected_sold error block: overall + per-rooms (via _compute_metrics) + per-segment.
+
+    Each input row is ``(expected_sold_ppm2, sold_ppm2, rooms)``. Reuses the
+    existing `_compute_metrics` for the overall/per-rooms split (so the no-data
+    counts ride along as ``n_no_analogs`` — here meaning "no prediction") and
+    attaches a per-price-segment breakdown keyed by the SOLD ₽/m². Pure: no DB.
+    """
+    m = _compute_metrics(
+        rows,
+        n_no_analogs=n_no_prediction,
+        per_rooms_no_analogs=per_rooms_no_prediction,
+    )
+    m["per_segment"] = _segment_metrics([(pred, sold) for pred, sold, _ in rows])
+    return m
+
+
+def _compute_full_metrics(
+    predictions: list[Prediction],
+    *,
+    n_no_prediction: int = 0,
+    per_rooms_no_prediction: dict[int, int] | None = None,
+) -> dict[str, Any]:
+    """Aggregate full-spine Prediction records into the complete metrics dict.
+
+    Blocks (all over deals that DID get an expected_sold, except where noted):
+      - ``expected_sold``  : signed bias / MAPE overall + per-rooms + per-segment.
+      - ``range_coverage`` : sold-total ∈ expected_sold range, overall +
+                             per-confidence (counts ALL priced deals; a deal with
+                             no expected_sold range contributes to n but not to
+                             coverage).
+      - ``calibration``    : per-confidence n / coverage% / MAPE%.
+      - ``sharpness``      : median relative range width (high-low)/point.
+
+    Pure: no DB. Safe on an empty list (every block renders with None/0).
+    """
+    # Confidence bucket order: canonical three, then any extras encountered.
+    conf_order = list(CONFIDENCE_BUCKETS)
+    for p in predictions:
+        b = _bucketize_confidence(p.confidence)
+        if b not in conf_order:
+            conf_order.append(b)
+
+    es_rows: list[tuple[float, float, int]] = []  # (expected_sold_ppm2, sold_ppm2, rooms)
+    cov_rows: list[tuple[float, float, float]] = []  # (sold_total, range_low, range_high)
+    cov_by_conf: dict[str, list[tuple[float, float, float]]] = {b: [] for b in conf_order}
+    sharp_rows: list[tuple[float, float, float]] = []  # (point, range_low, range_high)
+    calib_rows: list[tuple[str, float | None, bool | None]] = []
+
+    for p in predictions:
+        signed: float | None = None
+        if p.expected_sold_ppm2 is not None and p.sold_ppm2 > 0:
+            signed = 100.0 * (p.expected_sold_ppm2 - p.sold_ppm2) / p.sold_ppm2
+            es_rows.append((p.expected_sold_ppm2, p.sold_ppm2, p.rooms))
+
+        covered: bool | None = None
+        if p.range_low is not None and p.range_high is not None:
+            covered = p.range_low <= p.sold_total <= p.range_high
+            cov_rows.append((p.sold_total, p.range_low, p.range_high))
+            cov_by_conf[_bucketize_confidence(p.confidence)].append(
+                (p.sold_total, p.range_low, p.range_high)
+            )
+            if p.expected_sold_price is not None:
+                sharp_rows.append((p.expected_sold_price, p.range_low, p.range_high))
+
+        calib_rows.append((p.confidence, signed, covered))
+
+    return {
+        "expected_sold": _expected_sold_metrics(
+            es_rows,
+            n_no_prediction=n_no_prediction,
+            per_rooms_no_prediction=per_rooms_no_prediction,
+        ),
+        "range_coverage": {
+            "overall": _range_coverage(cov_rows),
+            "per_confidence": {b: _range_coverage(cov_by_conf[b]) for b in conf_order},
+        },
+        "calibration": _calibration_metrics(calib_rows, bucket_order=conf_order),
+        "sharpness": _sharpness(sharp_rows),
+        "confidence_order": conf_order,
+    }
+
+
 def _render_table(metrics: dict[str, Any], headline: dict[str, Any]) -> str:
     """Render the aggregated metrics as a plain-text stdout report."""
     lines: list[str] = []
@@ -393,14 +695,8 @@ def _render_table(metrics: dict[str, Any], headline: dict[str, Any]) -> str:
 
     # ASKING block — the estimator's raw asking-median prediction vs SOLD.
     lines.append("")
-    lines.append(
-        "PER-DEAL ERROR (signed = 100*(pred-sold)/sold; +ve = over-predict):"
-    )
-    lines.extend(
-        _render_metrics_block(
-            "[ASKING] estimator asking-median (uncorrected)", metrics
-        )
-    )
+    lines.append("PER-DEAL ERROR (signed = 100*(pred-sold)/sold; +ve = over-predict):")
+    lines.extend(_render_metrics_block("[ASKING] estimator asking-median (uncorrected)", metrics))
 
     # CORRECTED block — asking-median × per-rooms asking→sold ratio (#648 S1).
     corrected = metrics.get("corrected")
@@ -423,24 +719,18 @@ def _render_table(metrics: dict[str, Any], headline: dict[str, Any]) -> str:
             )
         else:
             lines.append(
-                "!! IN-SAMPLE: ratios were derived AND evaluated on the same "
-                "deals, so the"
+                "!! IN-SAMPLE: ratios were derived AND evaluated on the same deals, so the"
             )
             lines.append(
-                "   corrected bias is near-zero BY CONSTRUCTION. This proves "
-                "the MECHANISM"
+                "   corrected bias is near-zero BY CONSTRUCTION. This proves the MECHANISM"
             )
             lines.append(
-                "   (a per-rooms ratio removes the systematic asking→sold "
-                "gap), NOT out-of-"
+                "   (a per-rooms ratio removes the systematic asking→sold gap), NOT out-of-"
             )
             lines.append(
-                "   sample accuracy. Re-run with --holdout-split for an honest "
-                "number; the"
-            )
-            lines.append(
-                "   real A/B (Stage 2) fits the ratio on a separate window."
+                "   sample accuracy. Re-run with --holdout-split for an honest number; the"
             )
+            lines.append("   real A/B (Stage 2) fits the ratio on a separate window.")
 
     lines.append("")
     lines.append("Caveats: CURRENT listings vs PAST deals (not point-in-time);")
@@ -449,11 +739,17 @@ def _render_table(metrics: dict[str, Any], headline: dict[str, Any]) -> str:
     return "\n".join(lines)
 
 
-def _render_metrics_block(title: str, metrics: dict[str, Any]) -> list[str]:
-    """Render one OVERALL + per-rooms metrics table (shared by both blocks)."""
+def _render_metrics_block(
+    title: str, metrics: dict[str, Any], *, no_col: str = "no_analog"
+) -> list[str]:
+    """Render one OVERALL + per-rooms metrics table (shared by all blocks).
+
+    ``no_col`` labels the skipped-deal column header — "no_analog" for the
+    asking-core block, "no_pred" for the full-spine expected_sold block (where it
+    counts deals the spine could not price).
+    """
     header = (
-        f"  {'bucket':<8} {'n':>5} {'no_analog':>10} "
-        f"{'bias%':>8} {'MAPE%':>8} {'p25%':>8} {'p75%':>8}"
+        f"  {'bucket':<8} {'n':>5} {no_col:>10} {'bias%':>8} {'MAPE%':>8} {'p25%':>8} {'p75%':>8}"
     )
     out: list[str] = [title, header, "  " + "-" * (len(header) - 2)]
     out.append(_fmt_row("OVERALL", metrics["overall"]))
@@ -501,6 +797,119 @@ def _fmt_ppm2(v: float | None) -> str:
     return "n/a" if v is None else f"{round(v):,}".replace(",", " ")
 
 
+def _fmt_cov(v: float | None) -> str:
+    """Coverage percent — non-signed (0..100), n/a for None."""
+    return "n/a" if v is None else f"{v:.1f}"
+
+
+def _fmt_ratio(v: float | None) -> str:
+    """Relative-width / ratio formatter (n/a for None)."""
+    return "n/a" if v is None else f"{v:.3f}"
+
+
+# --------------------------------------------------------------------------- #
+# Full-spine renderer (#1966)
+# --------------------------------------------------------------------------- #
+
+
+def _render_full_table(metrics: dict[str, Any]) -> str:
+    """Render the full-spine metrics dict (#1966) as a plain-text stdout report."""
+    headline = metrics.get("headline") or {}
+    lines: list[str] = []
+    lines.append("=" * 78)
+    lines.append("BACKTEST [full spine]: _price_from_inputs expected_sold vs ДКП sold")
+    lines.append("=" * 78)
+
+    dm = headline.get("deal_median_ppm2")
+    am = headline.get("ask_median_ppm2")
+    spread = headline.get("spread_pct")
+    lines.append("")
+    lines.append("CITY-WIDE HEADLINE (sample medians, ₽/m²):")
+    lines.append(f"  deal_median_ppm2 (SOLD):        {_fmt_ppm2(dm)}")
+    lines.append(f"  ask_median_ppm2  (ASKING head): {_fmt_ppm2(am)}")
+    lines.append(f"  spread (asking vs deal):        {_fmt_pct(spread)}")
+
+    es = metrics["expected_sold"]
+    lines.append("")
+    lines.append("EXPECTED_SOLD ERROR (signed = 100*(expected_sold-sold)/sold; +ve = over):")
+    lines.extend(_render_metrics_block("[EXPECTED_SOLD] per-rooms", es, no_col="no_pred"))
+    lines.append("")
+    lines.extend(_render_segment_block(es["per_segment"]))
+
+    conf_order = metrics.get("confidence_order") or list(CONFIDENCE_BUCKETS)
+    lines.append("")
+    lines.extend(_render_coverage_block(metrics["range_coverage"], conf_order))
+    lines.append("")
+    lines.extend(_render_calibration_block(metrics["calibration"], conf_order))
+
+    sh = metrics["sharpness"]
+    lines.append("")
+    lines.append(
+        f"SHARPNESS: median relative range width (high-low)/point = "
+        f"{_fmt_ratio(sh.get('median_rel_width'))} (n={sh.get('n', 0)})"
+    )
+
+    lines.append("")
+    lines.append("Caveats: CURRENT listings vs PAST deals (time-mismatched) →")
+    lines.append("regression baseline, not absolute truth; network valuation")
+    lines.append("layers (Avito-IMV on-demand / Yandex / Cian) excluded.")
+    lines.append("=" * 78)
+    return "\n".join(lines)
+
+
+def _render_segment_block(per_segment: dict[str, Any]) -> list[str]:
+    """Render the per-price-segment expected_sold error table."""
+    header = f"  {'segment':<10} {'n':>5} {'bias%':>8} {'MAPE%':>8} {'p25%':>8} {'p75%':>8}"
+    out: list[str] = [
+        "[EXPECTED_SOLD] per price-segment (band by SOLD ₽/m²):",
+        header,
+        "  " + "-" * (len(header) - 2),
+    ]
+    for label, _ in PRICE_SEGMENTS_PPM2:
+        m = per_segment[label]
+        out.append(
+            f"  {label:<10} {m.get('n', 0):>5} "
+            f"{_fmt_pct(m.get('median_bias_pct')):>8} {_fmt_pct(m.get('mape_pct')):>8} "
+            f"{_fmt_pct(m.get('p25_pct')):>8} {_fmt_pct(m.get('p75_pct')):>8}"
+        )
+    return out
+
+
+def _render_coverage_block(range_coverage: dict[str, Any], conf_order: list[str]) -> list[str]:
+    """Render range-coverage: overall + per-confidence (sold_total ∈ range)."""
+    ov = range_coverage["overall"]
+    out: list[str] = ["RANGE COVERAGE (sold total within predicted expected_sold range):"]
+    out.append(
+        f"  OVERALL    n={ov['n']:>5}  covered={ov['n_covered']:>5}  "
+        f"coverage={_fmt_cov(ov['coverage_pct'])}%"
+    )
+    per_conf = range_coverage.get("per_confidence") or {}
+    for b in conf_order:
+        c = per_conf.get(b, {"n": 0, "n_covered": 0, "coverage_pct": None})
+        out.append(
+            f"  {b:<10} n={c['n']:>5}  covered={c['n_covered']:>5}  "
+            f"coverage={_fmt_cov(c['coverage_pct'])}%"
+        )
+    return out
+
+
+def _render_calibration_block(calibration: dict[str, Any], conf_order: list[str]) -> list[str]:
+    """Render confidence-calibration: per bucket n / coverage% / MAPE%."""
+    header = f"  {'conf':<8} {'n':>5} {'coverage%':>10} {'MAPE%':>8}"
+    out: list[str] = [
+        "CONFIDENCE CALIBRATION (high should be tighter & more accurate):",
+        header,
+        "  " + "-" * (len(header) - 2),
+    ]
+    for b in conf_order:
+        c = calibration.get(b, {"n": 0, "coverage_pct": None, "mape_pct": None})
+        out.append(
+            f"  {b:<8} {c.get('n', 0):>5} "
+            f"{_fmt_cov(c.get('coverage_pct')):>10} {_fmt_pct(c.get('mape_pct')):>8}"
+        )
+    return out
+
+
 # --------------------------------------------------------------------------- #
 # DB layer — READ-ONLY SELECTs only.
 # --------------------------------------------------------------------------- #
@@ -516,12 +925,20 @@ _SAMPLE_SQL = text(
         ST_Y(geom::geometry) AS lat,
         rooms,
         price_per_m2 AS sold_ppm2,
-        deal_date
+        deal_date,
+        area_m2,
+        address,
+        floor,
+        total_floors,
+        year_built,
+        house_type
     FROM deals
     WHERE source = 'rosreestr'
       AND geom IS NOT NULL
       AND price_per_m2 BETWEEN CAST(:ppm2_min AS numeric) AND CAST(:ppm2_max AS numeric)
       AND rooms IS NOT NULL
+      AND area_m2 IS NOT NULL
+      AND area_m2 > 0
       AND deal_date >= CAST(:since AS date)
     ORDER BY id DESC
     LIMIT CAST(:sample AS integer)
@@ -569,6 +986,8 @@ def _load_sample(db: Session, *, sample: int, since: str) -> list[DealSample]:
     for r in rows:
         if r["lon"] is None or r["lat"] is None or r["sold_ppm2"] is None:
             continue
+        if r["area_m2"] is None or float(r["area_m2"]) <= 0:
+            continue
         out.append(
             DealSample(
                 id=r["id"],
@@ -577,6 +996,12 @@ def _load_sample(db: Session, *, sample: int, since: str) -> list[DealSample]:
                 rooms=int(r["rooms"]),
                 sold_ppm2=float(r["sold_ppm2"]),
                 deal_date=r["deal_date"],
+                area_m2=float(r["area_m2"]),
+                address=r["address"],
+                floor=(int(r["floor"]) if r["floor"] is not None else None),
+                total_floors=(int(r["total_floors"]) if r["total_floors"] is not None else None),
+                year_built=(int(r["year_built"]) if r["year_built"] is not None else None),
+                house_type=r["house_type"],
             )
         )
     return out
@@ -622,6 +1047,217 @@ def _predict_for_deal(
     return _percentile(prices, 0.5)
 
 
+def _select_analogs_full(
+    db: Session, deal: DealSample, est: SimpleNamespace
+) -> tuple[list[dict[str, Any]], str, bool, bool]:
+    """Replicate estimate_quality's analog tier ladder for one deal (#1966).
+
+    Mirrors the Tier0(cohort)→A(1 km)→wide(2 km)→widearea(±25 %) sequence
+    EXACTLY, including the MIN_ANALOGS_TIER_0 gate and the <5 / <3 widen
+    thresholds and the "only adopt the wider tier when it returns MORE lots"
+    guards. target_house_id is always None here (the harness does not resolve
+    canonical houses), so Tier S(canonical) never fires — same as a fresh
+    estimate without a house match.
+
+    Returns ``(listings, analog_tier, fallback_used, area_widened)``.
+    """
+    m = est.m
+    cohort_range = m._target_cohort_range(deal.year_built)
+    if cohort_range is not None:
+        cy_min, cy_max = cohort_range
+        listings_tier0, _, analog_tier = m._fetch_analogs(
+            db,
+            lat=deal.lat,
+            lon=deal.lon,
+            rooms=deal.rooms,
+            area=deal.area_m2,
+            radius_m=m.DEFAULT_RADIUS_M,
+            full_address=deal.address,
+            target_house_id=None,
+            year_built=deal.year_built,
+            house_type=deal.house_type,
+            total_floors=deal.total_floors,
+            cohort_year_min=cy_min,
+            cohort_year_max=cy_max,
+        )
+    else:
+        listings_tier0 = []
+        analog_tier = "W"
+
+    if len(listings_tier0) >= m.MIN_ANALOGS_TIER_0:
+        listings = listings_tier0
+        fallback_used = False
+    else:
+        listings, fallback_used, analog_tier = m._fetch_analogs(
+            db,
+            lat=deal.lat,
+            lon=deal.lon,
+            rooms=deal.rooms,
+            area=deal.area_m2,
+            radius_m=m.DEFAULT_RADIUS_M,
+            full_address=deal.address,
+            target_house_id=None,
+            year_built=deal.year_built,
+            house_type=deal.house_type,
+            total_floors=deal.total_floors,
+        )
+    area_widened = False
+
+    if len(listings) < 5:
+        listings_wide, _, analog_tier_wide = m._fetch_analogs(
+            db,
+            lat=deal.lat,
+            lon=deal.lon,
+            rooms=deal.rooms,
+            area=deal.area_m2,
+            radius_m=m.FALLBACK_RADIUS_M,
+            full_address=deal.address,
+            target_house_id=None,
+            year_built=deal.year_built,
+            house_type=deal.house_type,
+            total_floors=deal.total_floors,
+        )
+        if len(listings_wide) > len(listings):
+            listings = listings_wide
+            fallback_used = True
+            analog_tier = analog_tier_wide
+
+    if len(listings) < 3:
+        listings_widearea, _, analog_tier_wa = m._fetch_analogs(
+            db,
+            lat=deal.lat,
+            lon=deal.lon,
+            rooms=deal.rooms,
+            area=deal.area_m2,
+            radius_m=m.FALLBACK_RADIUS_M,
+            area_tolerance=0.25,
+            full_address=deal.address,
+            target_house_id=None,
+            year_built=deal.year_built,
+            house_type=deal.house_type,
+            total_floors=deal.total_floors,
+        )
+        if len(listings_widearea) > len(listings):
+            listings = listings_widearea
+            fallback_used = True
+            area_widened = True
+            analog_tier = analog_tier_wa
+
+    return listings, analog_tier, fallback_used, area_widened
+
+
+def _predict_full_spine(db: Session, deal: DealSample, est: SimpleNamespace) -> Prediction | None:
+    """Predict one deal through the FULL deterministic spine (#1966).
+
+    Selects analogs via the replicated tier ladder, pre-fetches the spine inputs
+    (anchor comps, ДКП corridor, house IMV anchor) and injects the DB callables,
+    then calls ``estimator._price_from_inputs`` with parameters that mirror
+    ``estimate_quality`` for a sold unit whose repair state is unknown and with
+    the network valuation layers excluded (imv_eval=None, yandex/cian absent).
+
+    Returns a Prediction, or None when the spine cannot price the deal
+    (median<=0 or  tuple[float | None, str | None]:
+        return m._get_asking_sold_ratio(db, deal.rooms, anchor_ppm2=appm2)
+
+    def _qi_lookup(q: str) -> tuple[float, int] | None:
+        return m._lookup_quarter_index(
+            db,
+            quarter_cad_number=q,
+            min_n_deals=settings.estimate_quarter_index_min_n_deals,
+        )
+
+    def _qis_lookup(qs: list[str]) -> dict[str, float]:
+        return m._lookup_quarter_indexes(
+            db,
+            quarter_cad_numbers=qs,
+            min_n_deals=settings.estimate_quarter_index_min_n_deals,
+        )
+
+    pr = m._price_from_inputs(
+        listings=listings,
+        area_m2=deal.area_m2,
+        rooms=deal.rooms,
+        repair_state=None,
+        floor=deal.floor,
+        total_floors=deal.total_floors,
+        target_year=deal.year_built,
+        analog_tier=analog_tier,
+        fallback_used=fallback_used,
+        area_widened=area_widened,
+        anchor_comps=anchor_comps,
+        anchor_tier_fetched=anchor_tier,
+        dkp_raw=dkp_raw,
+        imv_anchor=imv_anchor,
+        imv_eval=None,
+        yandex_val_present=False,
+        cian_val_present=False,
+        ratio_resolver=_ratio_resolver,
+        quarter_index_lookup=_qi_lookup,
+        quarter_indexes_lookup=_qis_lookup,
+        target_house_cadnum=None,
+        dadata_coarse=False,
+        geo=geo,
+        dadata_qc_geo=None,
+    )
+
+    # Skip when the spine couldn't price it — mirror the asking-core skip.
+    if pr.median_price <= 0 or pr.n_analogs < MIN_CANDIDATES:
+        return None
+
+    es_ppm2 = float(pr.expected_sold_per_m2) if pr.expected_sold_per_m2 is not None else None
+    es_price = float(pr.expected_sold_price) if pr.expected_sold_price is not None else None
+    r_low = float(pr.expected_sold_range_low) if pr.expected_sold_range_low is not None else None
+    r_high = float(pr.expected_sold_range_high) if pr.expected_sold_range_high is not None else None
+    return Prediction(
+        deal_id=deal.id,
+        rooms=deal.rooms,
+        area_m2=deal.area_m2,
+        sold_ppm2=deal.sold_ppm2,
+        median_ppm2=float(pr.median_ppm2),
+        confidence=pr.confidence,
+        anchor_tier=pr.anchor_tier,
+        expected_sold_ppm2=es_ppm2,
+        expected_sold_price=es_price,
+        range_low=r_low,
+        range_high=r_high,
+    )
+
+
 def _attach_correction(
     metrics: dict[str, Any],
     matched_rows: list[tuple[float, float, int]],
@@ -703,9 +1339,7 @@ def run_backtest(
     pred_ppm2_all: list[float] = []
 
     for i, deal in enumerate(deals, start=1):
-        pred = _predict_for_deal(
-            db, deal, radius=radius, rooms_tolerance=rooms_tolerance
-        )
+        pred = _predict_for_deal(db, deal, radius=radius, rooms_tolerance=rooms_tolerance)
         if pred is None:
             n_no_analogs += 1
             per_rooms_no_analogs[_bucketize_rooms(deal.rooms)] += 1
@@ -729,8 +1363,88 @@ def run_backtest(
         per_rooms_no_analogs=per_rooms_no_analogs,
     )
 
-    _attach_correction(
-        metrics, matched_rows, matched_ids, holdout_split=holdout_split
+    _attach_correction(metrics, matched_rows, matched_ids, holdout_split=holdout_split)
+
+    deal_median = statistics.median(sold_ppm2_all) if sold_ppm2_all else None
+    ask_median = statistics.median(pred_ppm2_all) if pred_ppm2_all else None
+    spread_pct: float | None = None
+    if deal_median and ask_median and deal_median > 0:
+        spread_pct = round(100.0 * (ask_median - deal_median) / deal_median, 2)
+
+    metrics["headline"] = {
+        "deal_median_ppm2": deal_median,
+        "ask_median_ppm2": ask_median,
+        "spread_pct": spread_pct,
+    }
+    metrics["params"] = {
+        "engine": "asking-core",
+        "sample_requested": sample,
+        "sample_loaded": len(deals),
+        "since": since,
+        "radius_m": radius,
+        "rooms_tolerance": rooms_tolerance,
+        "n_matched": len(matched_rows),
+        "n_no_analogs": n_no_analogs,
+        "holdout_split": holdout_split,
+    }
+    return metrics
+
+
+def run_backtest_full(db: Session, *, sample: int, since: str) -> dict[str, Any]:
+    """Drive the FULL-spine read-only backtest and return a metrics dict (#1966).
+
+    Per deal: load sample → ``_predict_full_spine`` (replicate the analog tier
+    ladder + pre-fetch spine inputs → ``_price_from_inputs``) → collect Prediction
+    records + no-prediction counts → ``_compute_full_metrics`` (expected_sold
+    error overall/per-rooms/per-segment + range-coverage + calibration +
+    sharpness) + a city-wide asking-vs-deal headline spread. No writes.
+
+    ``--radius`` / ``--rooms-tolerance`` / ``--holdout-split`` do NOT apply here —
+    the tier ladder uses the estimator's OWN constants (DEFAULT_RADIUS_M /
+    FALLBACK_RADIUS_M) and there is no per-rooms correction block (the spine
+    already emits expected_sold via the asking→sold ratio).
+    """
+    est = _import_estimator_full()
+    deals = _load_sample(db, sample=sample, since=since)
+    logger.info("loaded sample: %d ДКП deals (since=%s) [full spine]", len(deals), since)
+
+    predictions: list[Prediction] = []
+    n_no_prediction = 0
+    per_rooms_no_prediction: dict[int, int] = {b: 0 for b in ROOM_BUCKETS}
+
+    sold_ppm2_all: list[float] = [d.sold_ppm2 for d in deals]
+    pred_ppm2_all: list[float] = []  # asking headline median_ppm2 (priced deals)
+
+    for i, deal in enumerate(deals, start=1):
+        try:
+            pr = _predict_full_spine(db, deal, est)
+        except Exception as exc:
+            # Read-only: a failed SELECT can poison the tx → rollback so the next
+            # deal's queries run clean. Skip this deal (counts as no-prediction).
+            logger.warning("deal %s spine failed (graceful, skipped): %s", deal.id, exc)
+            db.rollback()
+            pr = None
+
+        if pr is None:
+            n_no_prediction += 1
+            per_rooms_no_prediction[_bucketize_rooms(deal.rooms)] += 1
+        else:
+            predictions.append(pr)
+            pred_ppm2_all.append(pr.median_ppm2)
+
+        if i % 50 == 0:
+            logger.info(
+                "progress %d/%d (priced=%d, no_pred=%d)",
+                i,
+                len(deals),
+                len(predictions),
+                n_no_prediction,
+            )
+
+    metrics = _compute_full_metrics(
+        predictions,
+        n_no_prediction=n_no_prediction,
+        per_rooms_no_prediction=per_rooms_no_prediction,
     )
 
     deal_median = statistics.median(sold_ppm2_all) if sold_ppm2_all else None
@@ -745,14 +1459,13 @@ def run_backtest(
         "spread_pct": spread_pct,
     }
     metrics["params"] = {
+        "engine": "full",
         "sample_requested": sample,
         "sample_loaded": len(deals),
         "since": since,
-        "radius_m": radius,
-        "rooms_tolerance": rooms_tolerance,
-        "n_matched": len(matched_rows),
-        "n_no_analogs": n_no_analogs,
-        "holdout_split": holdout_split,
+        "n_matched": len(predictions),
+        "n_no_prediction": n_no_prediction,
+        "price_segments_ppm2": [list(seg) for seg in PRICE_SEGMENTS_PPM2],
     }
     return metrics
 
@@ -766,10 +1479,21 @@ def _parse_args(argv: list[str] | None = None) -> argparse.Namespace:
     """argparse setup, factored out for testability."""
     p = argparse.ArgumentParser(
         description=(
-            "READ-ONLY backtest of the estimator's asking-median accuracy "
-            "against rosreestr ДКП sold prices (issue #648)."
+            "READ-ONLY backtest of the estimator against rosreestr ДКП sold "
+            "prices. Default engine 'full' runs the deterministic pricing spine "
+            "(_price_from_inputs); 'asking-core' is the legacy median+IQR core "
+            "(issues #648 / #1966)."
         ),
     )
+    p.add_argument(
+        "--engine",
+        choices=("full", "asking-core"),
+        default="full",
+        help="Prediction engine: 'full' (default) = the deterministic pricing "
+        "spine via _price_from_inputs (expected_sold + coverage/calibration/"
+        "segment metrics); 'asking-core' = legacy asking-median+IQR core with "
+        "the per-rooms asking→sold correction block.",
+    )
     p.add_argument(
         "--sample",
         type=int,
@@ -814,8 +1538,9 @@ def main(argv: list[str] | None = None) -> int:
     """CLI entry point. Returns the count of matched (predicted) deals."""
     args = _parse_args(argv)
     logger.info(
-        "backtest start: sample=%d since=%s radius=%dm rooms_tolerance=%d "
-        "holdout_split=%s",
+        "backtest start: engine=%s sample=%d since=%s radius=%dm "
+        "rooms_tolerance=%d holdout_split=%s",
+        args.engine,
         args.sample,
         args.since,
         args.radius,
@@ -825,19 +1550,24 @@ def main(argv: list[str] | None = None) -> int:
 
     db = _session()
     try:
-        metrics = run_backtest(
-            db,
-            sample=args.sample,
-            since=args.since,
-            radius=args.radius,
-            rooms_tolerance=args.rooms_tolerance,
-            holdout_split=args.holdout_split,
-        )
+        if args.engine == "full":
+            metrics = run_backtest_full(db, sample=args.sample, since=args.since)
+        else:
+            metrics = run_backtest(
+                db,
+                sample=args.sample,
+                since=args.since,
+                radius=args.radius,
+                rooms_tolerance=args.rooms_tolerance,
+                holdout_split=args.holdout_split,
+            )
     finally:
         db.close()
 
     if args.json:
         print(json.dumps(metrics, ensure_ascii=False, indent=2, default=str))
+    elif args.engine == "full":
+        print(_render_full_table(metrics))
     else:
         print(_render_table(metrics, metrics["headline"]))
 
diff --git a/tradein-mvp/backend/tests/scrapers/test_domclick.py b/tradein-mvp/backend/tests/scrapers/test_domclick.py
index 04d0557d..4653d796 100644
--- a/tradein-mvp/backend/tests/scrapers/test_domclick.py
+++ b/tradein-mvp/backend/tests/scrapers/test_domclick.py
@@ -1,252 +1,526 @@
-"""Unit-тесты для DomClick scraper (#796).
+"""Unit-тесты для DomClick BFF scraper (#1968).
 
 Тестируем:
-- Парсинг title-паттерна (rooms/area/floor) из текста карточки
-- Парсинг цены из отдельного DOM-элемента
-- Извлечение source_id из href
-- Полный цикл _parse_html -> list[ScrapedLot] на mocked HTML
+- _extract_json: bare + 
-wrapped → dict; QRATOR-маркер → DomClickBlockedError
+- _build_offers_url / _build_count_url: корректные параметры, "5+" → "5%2B",
+  offers имеет offset/limit, count — нет
+- _map_item: маппинг realistic offer dict → ScrapedLot; price=0 → None; studio
+- _is_geo_ok: гео-гард по bbox + offerRegionName
+- _sweep_bucket: price-split при count > OFFSET_CAP
+- DomClickScraper: source/name, fetch_around raises NotImplementedError
 """
 
 from __future__ import annotations
 
 import asyncio
+from typing import Any
 
 import pytest
 
 from app.services.scrapers.domclick import (
+    LTE_MAX,
+    OFFSET_CAP,
     DomClickScraper,
-    _extract_area,
-    _extract_floor,
-    _extract_price_from_element,
-    _extract_rooms,
-    _extract_source_id,
+    _build_count_url,
+    _build_offers_url,
+    _extract_json,
 )
 from app.services.scrapers.domclick_exceptions import DomClickBlockedError
 
-# ── Rooms / studio ───────────────────────────────────────────────────────────
+# ── _extract_json ─────────────────────────────────────────────────────────────
 
 
-@pytest.mark.parametrize(
-    "text, expected",
-    [
-        ("2-комн. квартира 53,5 м² 5/15 эт.", 2),
-        ("1-к. квартира 32,1 м² 3/9 эт.", 1),
-        ("3-комн. кв. 78 м² 10/12 эт.", 3),
-        ("4-комн. квартира 95 м² 2/5 эт.", 4),
-        ("студия 24 м² 1/10 эт.", 0),
-        ("Студия 30,5 м² 8/22 эт.", 0),
-        ("Нет комнат — просто текст", None),
-    ],
-)
-def test_extract_rooms(text: str, expected: int | None) -> None:
-    assert _extract_rooms(text) == expected
+def test_extract_json_bare_body() -> None:
+    """Bare JSON body (без HTML-обёртки) корректно парсится."""
+    html = '{"result": {"items": [], "snippetsCount": 42}}'
+    data = _extract_json(html)
+    assert data["result"]["snippetsCount"] == 42
 
 
-# ── Area ─────────────────────────────────────────────────────────────────────
+def test_extract_json_pre_wrapped() -> None:
+    """JSON в 
-теге корректно парсится."""
+    html = (
+        "DomClick"
+        '
{"result": {"items": [{"id": 1}]}}
'
+    )
+    data = _extract_json(html)
+    assert data["result"]["items"][0]["id"] == 1
 
 
-@pytest.mark.parametrize(
-    "text, expected",
-    [
-        ("2-комн. квартира 53,5 м² 5/15 эт.", 53.5),
-        ("студия 24 м² 1/10 эт.", 24.0),
-        ("78м2 на этаже", 78.0),
-        ("площадь 101.7 м²", 101.7),
-        ("Нет площади", None),
-    ],
-)
-def test_extract_area(text: str, expected: float | None) -> None:
-    assert _extract_area(text) == expected
+def test_extract_json_qrator_marker_raises() -> None:
+    """HTML с QRATOR-маркером вызывает DomClickBlockedError."""
+    blocked_html = (
+        "QRATOR protection"
+        "Access denied by bot_mitigation system"
+    )
+    with pytest.raises(DomClickBlockedError):
+        _extract_json(blocked_html)
 
 
-# ── Floor ─────────────────────────────────────────────────────────────────────
+def test_extract_json_captcha_raises() -> None:
+    """HTML с captcha-маркером вызывает DomClickBlockedError."""
+    html = "Please solve the captcha to continue"
+    with pytest.raises(DomClickBlockedError):
+        _extract_json(html)
 
 
-@pytest.mark.parametrize(
-    "text, floor, total",
-    [
-        ("2-комн. квартира 53,5 м² 5/15 эт.", 5, 15),
-        ("студия 24 м² 1/10 эт.", 1, 10),
-        ("12/24 эт.", 12, 24),
-        ("нет этажа", None, None),
-    ],
-)
-def test_extract_floor(text: str, floor: int | None, total: int | None) -> None:
-    assert _extract_floor(text) == (floor, total)
+def test_extract_json_no_json_raises() -> None:
+    """HTML без JSON-объекта вызывает ValueError."""
+    with pytest.raises(ValueError, match="No JSON"):
+        _extract_json("plain text")
 
 
-# ── Price (per-element) ───────────────────────────────────────────────────────
-# Цена парсится из отдельного DOM-элемента (не из суммарного card_text),
-# поэтому числа адреса не загрязняют парсинг цены.
+# ── _build_offers_url ────────────────────────────────────────────────────────
 
 
-@pytest.mark.parametrize(
-    "text, expected",
-    [
-        ("5 200 000 ₽", 5_200_000),
-        ("12 345 678 ₽", 12_345_678),
-        ("3 990 000 руб.", 3_990_000),
-        ("от 4 500 000 ₽", 4_500_000),
-        ("без цены", None),
-        ("0 ₽", None),
-    ],
-)
-def test_extract_price_from_element(text: str, expected: int | None) -> None:
-    assert _extract_price_from_element(text) == expected
+def test_build_offers_url_basic_params() -> None:
+    url = _build_offers_url("1", None, None, 0)
+    assert "deal_type=sale" in url
+    assert "category=living" in url
+    assert "offer_type=flat" in url
+    assert "rooms=1" in url
+    assert "offset=0" in url
+    assert "limit=20" in url
+    assert "sale_price__gte" not in url
+    assert "sale_price__lte" not in url
 
 
-# ── source_id from href ───────────────────────────────────────────────────────
+def test_build_offers_url_5plus_encoded() -> None:
+    """rooms='5+' должен кодироваться как '5%2B' в URL."""
+    url = _build_offers_url("5+", None, None, 0)
+    assert "5%2B" in url
+    # literal '+' в URL не должно быть для значения rooms
+    assert "rooms=5+" not in url
 
 
-@pytest.mark.parametrize(
-    "href, expected",
-    [
-        ("/card/sale__flat__2075671636", "2075671636"),
-        ("https://domclick.ru/card/sale__flat__123456789", "123456789"),
-        ("/card/sale__flat__1", "1"),
-        ("/card/something_else", None),
-        ("", None),
-    ],
-)
-def test_extract_source_id(href: str, expected: str | None) -> None:
-    assert _extract_source_id(href) == expected
+def test_build_offers_url_with_price_range() -> None:
+    url = _build_offers_url("2", 3_000_000, 6_000_000, 40)
+    assert "sale_price__gte=3000000" in url
+    assert "sale_price__lte=6000000" in url
+    assert "offset=40" in url
 
 
-# ── _parse_html -> list[ScrapedLot] ──────────────────────────────────────────
-# Цена в каждой карточке вынесена в отдельный  с символом рубля,
-# чтобы не сливаться с адресным номером дома при конкатенации card_text.
-
-_MOCK_HTML = """\
-
-
-
-  
-  
-    2-комн. квартира 53,5 м² 5/15 эт.
-    ул. Малышева, 1
-    5 200 000 ₽
-  
-
-  
-  
-    Студия 28 м² 3/10 эт.
-    пр-т Ленина, 50
-    3 100 000 ₽
-  
-
-  
-  
-    1-комн. квартира 32 м² 2/9 эт.
-  
-
-  
-  
-    что-то другое
-  
-
-  
-  
-    2-комн. квартира 53,5 м² 5/15 эт.
-    5 200 000 ₽
-  
-
-
-"""
+def test_build_offers_url_ekb_guid_present() -> None:
+    url = _build_offers_url("3", None, None, 0)
+    assert "0d475b79-88de-4054-818c-37d8f9d0d440" in url
+    assert "aids=20561" in url
 
 
-def test_parse_html_returns_lots() -> None:
-    scraper = DomClickScraper()
-    lots = scraper._parse_html(_MOCK_HTML)
-
-    # Карточки 1 и 2 парсятся; карточка 3 (без цены) отфильтрована; дубль — дедуплицирован
-    assert len(lots) == 2
+def test_build_offers_url_bff_host() -> None:
+    url = _build_offers_url("1", None, None, 0)
+    assert url.startswith("https://bff-search-web.domclick.ru/api/offers/v1?")
 
 
-def test_parse_html_card1_fields() -> None:
-    scraper = DomClickScraper()
-    lots = scraper._parse_html(_MOCK_HTML)
-
-    card1 = next(lot for lot in lots if lot.source_id == "2075671636")
-    assert card1.source == "domklik"
-    assert card1.source_url == "https://domclick.ru/card/sale__flat__2075671636"
-    assert card1.rooms == 2
-    assert card1.area_m2 == 53.5
-    assert card1.floor == 5
-    assert card1.total_floors == 15
-    assert card1.price_rub == 5_200_000
-    assert card1.lat is None
-    assert card1.lon is None
-    assert card1.listing_segment == "vtorichka"
+# ── _build_count_url ─────────────────────────────────────────────────────────
 
 
-def test_parse_html_studio() -> None:
-    scraper = DomClickScraper()
-    lots = scraper._parse_html(_MOCK_HTML)
-
-    studio = next(lot for lot in lots if lot.source_id == "1111111111")
-    assert studio.rooms == 0  # студия
-    assert studio.area_m2 == 28.0
-    assert studio.price_rub == 3_100_000
+def test_build_count_url_no_offset_limit() -> None:
+    """count url не должен содержать offset и limit."""
+    url = _build_count_url("2", None, None)
+    assert "offset" not in url
+    assert "limit" not in url
 
 
-def test_parse_html_no_price_filtered() -> None:
-    scraper = DomClickScraper()
-    lots = scraper._parse_html(_MOCK_HTML)
-
-    source_ids = {lot.source_id for lot in lots}
-    assert "9999999999" not in source_ids
+def test_build_count_url_5plus_encoded() -> None:
+    url = _build_count_url("5+", None, None)
+    assert "5%2B" in url
+    assert "rooms=5+" not in url
 
 
-def test_parse_html_dedup_same_href() -> None:
-    """Дубликат href на одной странице должен давать одну карточку."""
-    scraper = DomClickScraper()
-    lots = scraper._parse_html(_MOCK_HTML)
+def test_build_count_url_with_price() -> None:
+    url = _build_count_url("1", 1_000_000, 4_000_000)
+    assert "sale_price__gte=1000000" in url
+    assert "sale_price__lte=4000000" in url
 
-    ids_2075 = [lot for lot in lots if lot.source_id == "2075671636"]
-    assert len(ids_2075) == 1
+
+def test_build_count_url_bff_host() -> None:
+    url = _build_count_url("1", None, None)
+    assert url.startswith("https://bff-search-web.domclick.ru/api/offers/count/v1?")
+
+
+# ── _map_item → ScrapedLot ────────────────────────────────────────────────────
+
+_REALISTIC_ITEM: dict[str, Any] = {
+    "id": 2075671636,
+    "path": "https://domclick.ru/card/sale__flat__2075671636",
+    "location": {"lat": 56.838, "lon": 60.612},
+    "address": {"displayName": "Екатеринбург, улица Малышева, 1"},
+    "objectInfo": {"area": 53.5, "rooms": 2, "floor": 5, "isApartment": False},
+    "house": {"floors": 15, "buildYear": 2003},
+    "price": 5_200_000,
+    "squarePrice": 97196,
+    "flatComplex": {"id": 12345, "name": "ЖК Тест", "slug": "zhk-test"},
+    "isRosreestrApproved": True,
+    "publishedDate": "2026-06-01T10:00:00+05:00",
+    "updatedDate": "2026-06-15T12:00:00+05:00",
+    "lastPriceHistoryState": {"direction": "DECREASED"},
+    "offerRegionName": "Екатеринбург",
+}
+
+
+def test_map_item_basic_fields() -> None:
+    scraper = DomClickScraper.__new__(DomClickScraper)
+    scraper.parse_failures = 0
+    lot = scraper._map_item(_REALISTIC_ITEM)
+    assert lot is not None
+    assert lot.source == "domklik"
+    assert lot.source_id == "2075671636"
+    assert lot.source_url == "https://domclick.ru/card/sale__flat__2075671636"
+    assert lot.lat == pytest.approx(56.838)
+    assert lot.lon == pytest.approx(60.612)
+    assert lot.address == "Екатеринбург, улица Малышева, 1"
+    assert lot.rooms == 2
+    assert lot.area_m2 == pytest.approx(53.5)
+    assert lot.floor == 5
+    assert lot.total_floors == 15
+    assert lot.year_built == 2003
+    assert lot.price_rub == 5_200_000
+    assert lot.price_per_m2 == 97196
+    assert lot.listing_segment == "vtorichka"
+
+
+def test_map_item_publish_date() -> None:
+    scraper = DomClickScraper.__new__(DomClickScraper)
+    scraper.parse_failures = 0
+    from datetime import date
+
+    lot = scraper._map_item(_REALISTIC_ITEM)
+    assert lot is not None
+    assert lot.publish_date == date(2026, 6, 1)
+
+
+def test_map_item_raw_payload() -> None:
+    scraper = DomClickScraper.__new__(DomClickScraper)
+    scraper.parse_failures = 0
+    lot = scraper._map_item(_REALISTIC_ITEM)
+    assert lot is not None
+    assert lot.raw_payload is not None
+    assert lot.raw_payload["isRosreestrApproved"] is True
+    assert lot.raw_payload["flatComplex"]["name"] == "ЖК Тест"
+    assert lot.raw_payload["lastPriceHistoryState"]["direction"] == "DECREASED"
+
+
+def test_map_item_studio_force_rooms_zero() -> None:
+    """force_rooms=0 (студия-бакет) перебивает objectInfo.rooms."""
+    scraper = DomClickScraper.__new__(DomClickScraper)
+    scraper.parse_failures = 0
+    item = {**_REALISTIC_ITEM, "objectInfo": {"area": 28.0, "rooms": 1, "floor": 3}}
+    lot = scraper._map_item(item, force_rooms=0)
+    assert lot is not None
+    assert lot.rooms == 0
+
+
+def test_map_item_zero_price_returns_none() -> None:
+    scraper = DomClickScraper.__new__(DomClickScraper)
+    scraper.parse_failures = 0
+    item = {**_REALISTIC_ITEM, "price": 0}
+    assert scraper._map_item(item) is None
+
+
+def test_map_item_zero_sentinels_become_none() -> None:
+    """buildYear/area/floor/total_floors == 0 (sentinel «неизвестно») → None.
+
+    rooms=0 НЕ затрагивается (валидная студия), но здесь force_rooms не задан и
+    objectInfo.rooms=2, так что rooms остаётся 2.
+    """
+    scraper = DomClickScraper.__new__(DomClickScraper)
+    scraper.parse_failures = 0
+    item = {
+        **_REALISTIC_ITEM,
+        "objectInfo": {"area": 0, "rooms": 2, "floor": 0},
+        "house": {"floors": 0, "buildYear": 0},
+    }
+    lot = scraper._map_item(item)
+    assert lot is not None
+    assert lot.area_m2 is None
+    assert lot.floor is None
+    assert lot.total_floors is None
+    assert lot.year_built is None
+    assert lot.rooms == 2  # rooms 0-sentinel НЕ применяется
+
+
+def test_map_item_rooms_zero_studio_preserved() -> None:
+    """force_rooms=0 даёт rooms=0 (студия) — 0 здесь валиден, не sentinel."""
+    scraper = DomClickScraper.__new__(DomClickScraper)
+    scraper.parse_failures = 0
+    item = {**_REALISTIC_ITEM, "objectInfo": {"area": 24.0, "rooms": 0, "floor": 2}}
+    lot = scraper._map_item(item, force_rooms=0)
+    assert lot is not None
+    assert lot.rooms == 0
+
+
+def test_map_item_price_per_m2_fallback() -> None:
+    """Если squarePrice отсутствует, price_per_m2 вычисляется из price / area_m2."""
+    scraper = DomClickScraper.__new__(DomClickScraper)
+    scraper.parse_failures = 0
+    item = {**_REALISTIC_ITEM, "squarePrice": None}
+    lot = scraper._map_item(item)
+    assert lot is not None
+    assert lot.price_per_m2 == int(5_200_000 / 53.5)
+
+
+# ── _is_geo_ok — гео-гард ────────────────────────────────────────────────────
+
+
+def test_is_geo_ok_valid_ekb_item() -> None:
+    scraper = DomClickScraper.__new__(DomClickScraper)
+    assert scraper._is_geo_ok(_REALISTIC_ITEM) is True
+
+
+def test_is_geo_ok_wrong_region_name() -> None:
+    scraper = DomClickScraper.__new__(DomClickScraper)
+    item = {**_REALISTIC_ITEM, "offerRegionName": "Москва"}
+    assert scraper._is_geo_ok(item) is False
+
+
+def test_is_geo_ok_out_of_bbox_lat() -> None:
+    scraper = DomClickScraper.__new__(DomClickScraper)
+    item = {**_REALISTIC_ITEM, "location": {"lat": 55.5, "lon": 60.6}}
+    assert scraper._is_geo_ok(item) is False
+
+
+def test_is_geo_ok_out_of_bbox_lon() -> None:
+    scraper = DomClickScraper.__new__(DomClickScraper)
+    item = {**_REALISTIC_ITEM, "location": {"lat": 56.8, "lon": 61.5}}
+    assert scraper._is_geo_ok(item) is False
+
+
+def test_is_geo_ok_missing_coords() -> None:
+    scraper = DomClickScraper.__new__(DomClickScraper)
+    item = {**_REALISTIC_ITEM, "location": {"lat": None, "lon": None}}
+    assert scraper._is_geo_ok(item) is False
+
+
+# ── _sweep_bucket price-split ────────────────────────────────────────────────
+
+
+async def test_sweep_bucket_price_split_boundaries(monkeypatch: pytest.MonkeyPatch) -> None:
+    """_sweep_bucket: top-level unbounded split даёт contiguous non-overlapping ranges.
+
+    rooms="2" count>cap один раз, потом ≤cap → ровно 2 leaf-бакета, покрывающих
+    [0, LTE_MAX] без перекрытия и без зазора: [(0, mid), (mid+1, LTE_MAX)].
+    """
+    call_count: dict[str, int] = {"n": 0}
+    paginate_calls: list[tuple[str, int | None, int | None]] = []
+
+    async def fake_count(
+        self: DomClickScraper,
+        fetcher: object,
+        rooms: str,
+        price_gte: int | None,
+        price_lte: int | None,
+    ) -> int:
+        call_count["n"] += 1
+        # Первый вызов (unbounded) → over cap → price-split; остальные → под cap
+        return OFFSET_CAP + 1 if call_count["n"] == 1 else 1
+
+    async def fake_paginate(
+        self: DomClickScraper,
+        fetcher: object,
+        rooms: str,
+        price_gte: int | None,
+        price_lte: int | None,
+        seen_ids: set[str],
+        out_lots: list,
+        pages: int,
+    ) -> None:
+        paginate_calls.append((rooms, price_gte, price_lte))
+
+    monkeypatch.setattr(DomClickScraper, "_count", fake_count)
+    monkeypatch.setattr(DomClickScraper, "_paginate", fake_paginate)
+
+    scraper = DomClickScraper.__new__(DomClickScraper)
+    scraper.parse_failures = 0
+    scraper.geo_filtered = 0
+    scraper.blocked = False
+    scraper.fetch_errors = 0
+    scraper.request_delay_sec = 0.0
+
+    await scraper._sweep_bucket(
+        fetcher=object(),
+        rooms="2",
+        price_gte=None,
+        price_lte=None,
+        seen_ids=set(),
+        out_lots=[],
+        pages=10,
+    )
+
+    # 1 count call (over cap) + 2 recursive count calls (≤ cap) = 3 total
+    assert call_count["n"] == 3
+    # Both sub-buckets paginated, с точными границами split'а unbounded-диапазона.
+    mid = (0 + LTE_MAX) // 2
+    assert paginate_calls == [("2", 0, mid), ("2", mid + 1, LTE_MAX)]
+    # Sanity: contiguous (нет зазора) + non-overlapping + покрывают весь хвост.
+    assert paginate_calls[0][2] + 1 == paginate_calls[1][1]  # mid + 1
+    assert paginate_calls[0][1] == 0
+    assert paginate_calls[1][2] == LTE_MAX
+
+
+async def test_sweep_bucket_zero_count_no_paginate(monkeypatch: pytest.MonkeyPatch) -> None:
+    """_sweep_bucket при count==0 не вызывает _paginate."""
+    paginate_called = False
+
+    async def fake_count(self: DomClickScraper, *_args: object, **_kwargs: object) -> int:
+        return 0
+
+    async def fake_paginate(self: DomClickScraper, *_args: object, **_kwargs: object) -> None:
+        nonlocal paginate_called
+        paginate_called = True
+
+    monkeypatch.setattr(DomClickScraper, "_count", fake_count)
+    monkeypatch.setattr(DomClickScraper, "_paginate", fake_paginate)
+
+    scraper = DomClickScraper.__new__(DomClickScraper)
+    scraper.parse_failures = 0
+    scraper.geo_filtered = 0
+    scraper.blocked = False
+    scraper.fetch_errors = 0
+    scraper.request_delay_sec = 0.0
+
+    await scraper._sweep_bucket(
+        fetcher=object(),
+        rooms="3",
+        price_gte=None,
+        price_lte=None,
+        seen_ids=set(),
+        out_lots=[],
+        pages=5,
+    )
+    assert paginate_called is False
+
+
+# ── _paginate resilience — mid-bucket parse error ────────────────────────────
+
+
+class _FakeFetcher:
+    """Стаб BrowserFetcher: возвращает заранее заданные ответы по порядку вызовов."""
+
+    def __init__(self, responses: list[str]) -> None:
+        self._responses = responses
+        self._idx = 0
+
+    async def fetch(self, _url: str) -> str:
+        resp = self._responses[min(self._idx, len(self._responses) - 1)]
+        self._idx += 1
+        return resp
+
+
+async def test_paginate_mid_bucket_parse_error_preserves_lots() -> None:
+    """Garbled JSON на page 2 → bucket breaks, page-1 lots survive, fetch_errors++.
+
+    Page 1 — валидный JSON с 20 items (PAGE_SIZE) → продолжаем; page 2 — мусор
+    (ValueError из _extract_json) → fetch_errors++ и break, уже собранные lots целы.
+    """
+    import json
+
+    page1_items = [
+        {
+            "id": 1000 + i,
+            "path": f"https://domclick.ru/card/sale__flat__{1000 + i}",
+            "location": {"lat": 56.84, "lon": 60.6},
+            "address": {"displayName": "Екатеринбург, тест"},
+            "objectInfo": {"area": 40.0, "rooms": 1, "floor": 3},
+            "house": {"floors": 9, "buildYear": 2005},
+            "price": 4_000_000 + i,
+            "squarePrice": 100000,
+            "offerRegionName": "Екатеринбург",
+        }
+        for i in range(20)
+    ]
+    page1 = json.dumps({"result": {"items": page1_items}})
+    garbled = "
{not valid json at all
"
+    fetcher = _FakeFetcher([page1, garbled])
+
+    scraper = DomClickScraper.__new__(DomClickScraper)
+    scraper.parse_failures = 0
+    scraper.geo_filtered = 0
+    scraper.blocked = False
+    scraper.fetch_errors = 0
+    scraper.request_delay_sec = 0.0
+
+    out_lots: list = []
+    await scraper._paginate(
+        fetcher=fetcher,
+        rooms="1",
+        price_gte=None,
+        price_lte=None,
+        seen_ids=set(),
+        out_lots=out_lots,
+        pages=10,
+    )
+
+    assert len(out_lots) == 20  # page-1 lots сохранены
+    assert scraper.fetch_errors == 1  # page-2 garbled посчитан
+
+
+async def test_fetch_city_bucket_error_continues_to_next_bucket(
+    monkeypatch: pytest.MonkeyPatch,
+) -> None:
+    """Ошибка в одном бакете НЕ убивает весь sweep — остальные бакеты отрабатывают."""
+    swept: list[str] = []
+
+    async def fake_sweep_bucket(
+        self: DomClickScraper,
+        fetcher: object,
+        rooms: str,
+        price_gte: int | None,
+        price_lte: int | None,
+        seen_ids: set[str],
+        out_lots: list,
+        pages: int,
+    ) -> None:
+        swept.append(rooms)
+        if rooms == "2":
+            raise ValueError("garbled bucket")
+
+    # Заглушка BrowserFetcher через async context manager
+    class _BFStub:
+        async def __aenter__(self) -> _BFStub:
+            return self
+
+        async def __aexit__(self, *_a: object) -> None:
+            return None
+
+        async def fetch(self, _url: str) -> str:
+            return "{}"
+
+    import app.services.scrapers.browser_fetcher as bf_mod
+
+    monkeypatch.setattr(DomClickScraper, "_sweep_bucket", fake_sweep_bucket)
+    monkeypatch.setattr(bf_mod, "BrowserFetcher", lambda **_k: _BFStub())
+
+    scraper = DomClickScraper.__new__(DomClickScraper)
+    scraper.parse_failures = 0
+    scraper.geo_filtered = 0
+    scraper.blocked = False
+    scraper.fetch_errors = 0
+    scraper.request_delay_sec = 0.0
+
+    lots = await scraper.fetch_city(city_id=4, rooms=None, pages=5)
+
+    # Все 6 бакетов посещены несмотря на ошибку в "2"
+    assert swept == ["st", "1", "2", "3", "4", "5+"]
+    assert scraper.fetch_errors == 1
+    assert scraper.blocked is False
+    assert lots == []
+
+
+# ── DomClickScraper — class attrs ─────────────────────────────────────────────
 
 
 def test_scraper_source_name() -> None:
-    scraper = DomClickScraper()
+    scraper = DomClickScraper.__new__(DomClickScraper)
+    scraper.parse_failures = 0
+    scraper.geo_filtered = 0
+    scraper.blocked = False
+    scraper.request_delay_sec = 0.0
     assert scraper.name == "domklik"
     assert scraper.source == "domklik"
     assert scraper.base_url == "https://domclick.ru"
-    # request_delay_sec установлен из get_scraper_delay (DB или fallback 8.0)
-    assert scraper.request_delay_sec >= 0.0
 
 
 def test_fetch_around_raises() -> None:
-    scraper = DomClickScraper()
+    scraper = DomClickScraper.__new__(DomClickScraper)
     with pytest.raises(NotImplementedError, match="geo-radius"):
         asyncio.run(scraper.fetch_around(56.8, 60.6))
-
-
-def test_build_url_no_rooms() -> None:
-    scraper = DomClickScraper()
-    url = scraper._build_url(city_id=4, rooms=None, page=1)
-    assert "city_id=4" in url
-    assert "rooms" not in url
-    assert "p=1" in url
-
-
-def test_build_url_with_rooms() -> None:
-    scraper = DomClickScraper()
-    url = scraper._build_url(city_id=4, rooms=2, page=3)
-    assert "rooms=2" in url
-    assert "p=3" in url
-
-
-def test_parse_failures_counter() -> None:
-    """parse_failures счётчик начинается с 0."""
-    scraper = DomClickScraper()
-    assert scraper.parse_failures == 0
-
-
-def test_blocked_page_raises_domclick_blocked_error() -> None:
-    """HTML с datadome в начале вызывает DomClickBlockedError."""
-    scraper = DomClickScraper()
-    blocked_html = (
-        "datadome protectionblocked"
-    )
-    with pytest.raises(DomClickBlockedError):
-        scraper._parse_html(blocked_html)
diff --git a/tradein-mvp/backend/tests/test_backtest_estimator.py b/tradein-mvp/backend/tests/test_backtest_estimator.py
index 306137e2..788407d6 100644
--- a/tradein-mvp/backend/tests/test_backtest_estimator.py
+++ b/tradein-mvp/backend/tests/test_backtest_estimator.py
@@ -217,9 +217,8 @@ def _rows_for_bucket(
 def test_derive_ratios_per_bucket_exact() -> None:
     # Each bucket ≥ MIN_BUCKET deals so every bucket gets its OWN ratio.
     # bucket 1: sold/ask = 80k/100k = 0.80 ; bucket 2: 150k/200k = 0.75.
-    rows = (
-        _rows_for_bucket(1, n=bt.MIN_BUCKET, ask=100_000.0, sold=80_000.0)
-        + _rows_for_bucket(2, n=bt.MIN_BUCKET, ask=200_000.0, sold=150_000.0)
+    rows = _rows_for_bucket(1, n=bt.MIN_BUCKET, ask=100_000.0, sold=80_000.0) + _rows_for_bucket(
+        2, n=bt.MIN_BUCKET, ask=200_000.0, sold=150_000.0
     )
     ratios, meta = bt._derive_room_ratios(rows)
     assert ratios[1] == pytest.approx(0.80)
@@ -328,9 +327,7 @@ def test_corrected_metrics_cancel_plus_30_pct_bias_to_zero() -> None:
     # in-sample and re-applying it MUST collapse the corrected bias to ~0.
     rows: list[tuple[float, float, int]] = []
     for bucket, sold in ((0, 80_000.0), (1, 100_000.0), (2, 150_000.0)):
-        rows += _rows_for_bucket(
-            bucket, n=bt.MIN_BUCKET, ask=1.30 * sold, sold=sold
-        )
+        rows += _rows_for_bucket(bucket, n=bt.MIN_BUCKET, ask=1.30 * sold, sold=sold)
 
     # sanity: the ASKING block really is +30%.
     asking = bt._compute_metrics(rows)
@@ -346,9 +343,7 @@ def test_corrected_metrics_cancel_plus_30_pct_bias_to_zero() -> None:
     assert corrected["overall"]["median_bias_pct"] == pytest.approx(0.0, abs=1e-6)
     assert corrected["overall"]["mape_pct"] == pytest.approx(0.0, abs=1e-6)
     for bucket in (0, 1, 2):
-        assert corrected["per_rooms"][bucket]["median_bias_pct"] == pytest.approx(
-            0.0, abs=1e-6
-        )
+        assert corrected["per_rooms"][bucket]["median_bias_pct"] == pytest.approx(0.0, abs=1e-6)
 
 
 def test_corrected_metrics_global_fallback_cancels_uniform_bias() -> None:
@@ -469,8 +464,18 @@ def test_argparse_defaults() -> None:
 
 def test_argparse_overrides() -> None:
     ns = bt._parse_args(
-        ["--sample", "50", "--since", "2024-01-01", "--radius", "2000",
-         "--rooms-tolerance", "1", "--holdout-split", "--json"]
+        [
+            "--sample",
+            "50",
+            "--since",
+            "2024-01-01",
+            "--radius",
+            "2000",
+            "--rooms-tolerance",
+            "1",
+            "--holdout-split",
+            "--json",
+        ]
     )
     assert ns.sample == 50
     assert ns.since == "2024-01-01"
@@ -478,3 +483,328 @@ def test_argparse_overrides() -> None:
     assert ns.rooms_tolerance == 1
     assert ns.holdout_split is True
     assert ns.json is True
+
+
+# --------------------------------------------------------------------------- #
+# #1966 full spine — --engine flag.
+# --------------------------------------------------------------------------- #
+
+
+def test_argparse_engine_defaults_to_full() -> None:
+    assert bt._parse_args([]).engine == "full"
+
+
+def test_argparse_engine_asking_core_override() -> None:
+    assert bt._parse_args(["--engine", "asking-core"]).engine == "asking-core"
+    assert bt._parse_args(["--engine", "full"]).engine == "full"
+
+
+def test_argparse_engine_rejects_unknown() -> None:
+    with pytest.raises(SystemExit):
+        bt._parse_args(["--engine", "bogus"])
+
+
+# --------------------------------------------------------------------------- #
+# #1966 _bucketize_confidence / _segment_label — pure bucketing.
+# --------------------------------------------------------------------------- #
+
+
+def test_bucketize_confidence_canonical_passthrough() -> None:
+    assert bt._bucketize_confidence("high") == "high"
+    assert bt._bucketize_confidence("medium") == "medium"
+    assert bt._bucketize_confidence("low") == "low"
+
+
+def test_bucketize_confidence_unknown_maps_to_other() -> None:
+    assert bt._bucketize_confidence("weird") == "other"
+    assert bt._bucketize_confidence("") == "other"
+
+
+def test_segment_label_bands_and_boundaries() -> None:
+    # Boundaries are upper-exclusive: 120k falls into комфорт, not эконом.
+    assert bt._segment_label(100_000) == "эконом"
+    assert bt._segment_label(119_999) == "эконом"
+    assert bt._segment_label(120_000) == "комфорт"
+    assert bt._segment_label(150_000) == "комфорт"
+    assert bt._segment_label(160_000) == "бизнес"
+    assert bt._segment_label(219_999) == "бизнес"
+    assert bt._segment_label(220_000) == "элит"
+    assert bt._segment_label(300_000) == "премиум"
+    assert bt._segment_label(2_000_000) == "премиум"  # +inf tail catches the top
+
+
+# --------------------------------------------------------------------------- #
+# #1966 _segment_metrics — per-price-segment signed error (band by SOLD ppm²).
+# --------------------------------------------------------------------------- #
+
+
+def test_segment_metrics_buckets_by_sold_price() -> None:
+    rows = [
+        (110_000.0, 100_000.0),  # sold эконом, +10
+        (132_000.0, 110_000.0),  # sold эконом, +20  → median эконом bias +15
+        (165_000.0, 150_000.0),  # sold комфорт, +10
+        (330_000.0, 300_000.0),  # sold премиум, +10
+    ]
+    seg = bt._segment_metrics(rows)
+    assert set(seg.keys()) == {label for label, _ in bt.PRICE_SEGMENTS_PPM2}
+    assert seg["эконом"]["n"] == 2
+    assert seg["эконом"]["median_bias_pct"] == pytest.approx(15.0)
+    assert seg["комфорт"]["n"] == 1
+    assert seg["комфорт"]["median_bias_pct"] == pytest.approx(10.0)
+    assert seg["премиум"]["n"] == 1
+    # bands with no rows are still present with n=0.
+    assert seg["бизнес"]["n"] == 0
+    assert seg["элит"]["n"] == 0
+
+
+def test_segment_metrics_drops_nonpositive_sold() -> None:
+    seg = bt._segment_metrics([(100_000.0, 0.0), (100_000.0, -1.0)])
+    assert all(seg[label]["n"] == 0 for label, _ in bt.PRICE_SEGMENTS_PPM2)
+
+
+# --------------------------------------------------------------------------- #
+# #1966 _range_coverage — inside / outside / boundary inclusive.
+# --------------------------------------------------------------------------- #
+
+
+def test_range_coverage_inside_outside_and_boundary_inclusive() -> None:
+    rows = [
+        (100.0, 90.0, 110.0),  # inside
+        (80.0, 90.0, 110.0),  # below low → outside
+        (120.0, 90.0, 110.0),  # above high → outside
+        (90.0, 90.0, 110.0),  # exactly on low → covered (inclusive)
+        (110.0, 90.0, 110.0),  # exactly on high → covered (inclusive)
+    ]
+    cov = bt._range_coverage(rows)
+    assert cov["n"] == 5
+    assert cov["n_covered"] == 3
+    assert cov["coverage_pct"] == pytest.approx(60.0)
+
+
+def test_range_coverage_empty_returns_none_pct() -> None:
+    cov = bt._range_coverage([])
+    assert cov["n"] == 0
+    assert cov["n_covered"] == 0
+    assert cov["coverage_pct"] is None
+
+
+def test_range_coverage_full_and_zero() -> None:
+    assert bt._range_coverage([(100.0, 50.0, 150.0)])["coverage_pct"] == pytest.approx(100.0)
+    assert bt._range_coverage([(10.0, 50.0, 150.0)])["coverage_pct"] == pytest.approx(0.0)
+
+
+# --------------------------------------------------------------------------- #
+# #1966 _sharpness — median relative range width (guards coverage gaming).
+# --------------------------------------------------------------------------- #
+
+
+def test_sharpness_median_relative_width() -> None:
+    rows = [
+        (100.0, 90.0, 110.0),  # width 20 / point 100 = 0.20
+        (200.0, 150.0, 250.0),  # width 100 / point 200 = 0.50
+    ]
+    sh = bt._sharpness(rows)
+    assert sh["n"] == 2
+    assert sh["median_rel_width"] == pytest.approx(0.35)  # median(0.20, 0.50)
+
+
+def test_sharpness_drops_nonpositive_point_and_empty() -> None:
+    assert bt._sharpness([(0.0, 1.0, 2.0)])["median_rel_width"] is None
+    assert bt._sharpness([(-5.0, 1.0, 2.0)])["n"] == 0
+    assert bt._sharpness([])["median_rel_width"] is None
+
+
+# --------------------------------------------------------------------------- #
+# #1966 _calibration_metrics — per-confidence n / coverage% / MAPE%.
+# --------------------------------------------------------------------------- #
+
+
+def test_calibration_metrics_per_confidence_n_coverage_mape() -> None:
+    rows = [
+        ("high", 5.0, True),
+        ("high", 15.0, True),  # high: n=2, covered 2/2=100%, mape median(5,15)=10
+        ("low", 40.0, False),
+        ("low", 60.0, True),  # low: n=2, covered 1/2=50%, mape median(40,60)=50
+    ]
+    cal = bt._calibration_metrics(rows)
+    # canonical buckets always present.
+    assert set(("high", "medium", "low")).issubset(cal.keys())
+    assert cal["high"]["n"] == 2
+    assert cal["high"]["coverage_pct"] == pytest.approx(100.0)
+    assert cal["high"]["mape_pct"] == pytest.approx(10.0)
+    assert cal["low"]["n"] == 2
+    assert cal["low"]["coverage_pct"] == pytest.approx(50.0)
+    assert cal["low"]["mape_pct"] == pytest.approx(50.0)
+    # empty canonical bucket renders with n=0 / None metrics, not missing.
+    assert cal["medium"]["n"] == 0
+    assert cal["medium"]["coverage_pct"] is None
+    assert cal["medium"]["mape_pct"] is None
+
+
+def test_calibration_metrics_handles_none_signed_and_covered() -> None:
+    # A prediction with no expected_sold (signed None) and no range (covered None)
+    # still counts toward n but not toward coverage/MAPE.
+    rows: list[tuple[str, float | None, bool | None]] = [
+        ("high", None, None),
+        ("high", 10.0, True),
+    ]
+    cal = bt._calibration_metrics(rows)
+    assert cal["high"]["n"] == 2
+    assert cal["high"]["coverage_pct"] == pytest.approx(100.0)  # only the 1 with covered
+    assert cal["high"]["mape_pct"] == pytest.approx(10.0)  # only the 1 with signed
+
+
+def test_calibration_metrics_appends_other_bucket() -> None:
+    cal = bt._calibration_metrics([("exotic", 5.0, True)])
+    assert "other" in cal
+    assert cal["other"]["n"] == 1
+    # canonical three still present even though empty.
+    assert cal["high"]["n"] == 0
+
+
+# --------------------------------------------------------------------------- #
+# #1966 Prediction + _compute_full_metrics — integration of the new blocks.
+# --------------------------------------------------------------------------- #
+
+
+def _pred(
+    *,
+    rooms: int = 2,
+    area: float = 50.0,
+    sold_ppm2: float = 100_000.0,
+    median_ppm2: float = 120_000.0,
+    confidence: str = "high",
+    es_ppm2: float | None = 100_000.0,
+    es_price: float | None = 5_000_000.0,
+    range_low: float | None = 4_500_000.0,
+    range_high: float | None = 5_500_000.0,
+    anchor_tier: str | None = None,
+    deal_id: int = 1,
+) -> bt.Prediction:
+    return bt.Prediction(
+        deal_id=deal_id,
+        rooms=rooms,
+        area_m2=area,
+        sold_ppm2=sold_ppm2,
+        median_ppm2=median_ppm2,
+        confidence=confidence,
+        anchor_tier=anchor_tier,
+        expected_sold_ppm2=es_ppm2,
+        expected_sold_price=es_price,
+        range_low=range_low,
+        range_high=range_high,
+    )
+
+
+def test_prediction_sold_total_property() -> None:
+    p = _pred(sold_ppm2=100_000.0, area=50.0)
+    assert p.sold_total == pytest.approx(5_000_000.0)
+
+
+def test_compute_full_metrics_structure_and_blocks() -> None:
+    preds = [
+        # sold_total = 100k*50 = 5.0M, range [4.5M, 5.5M] → covered; es +0%
+        _pred(deal_id=1, confidence="high", es_ppm2=100_000.0, sold_ppm2=100_000.0),
+        # sold_total = 200k*50 = 10.0M, range [4.5M,5.5M] → NOT covered; es -50%
+        _pred(
+            deal_id=3,
+            confidence="low",
+            es_ppm2=100_000.0,
+            sold_ppm2=200_000.0,
+            median_ppm2=120_000.0,
+        ),
+    ]
+    m = bt._compute_full_metrics(preds, n_no_prediction=4, per_rooms_no_prediction={2: 4})
+
+    # expected_sold block: overall + per_rooms + per_segment, carries no-pred count.
+    assert m["expected_sold"]["overall"]["n"] == 2
+    assert m["expected_sold"]["overall"]["n_no_analogs"] == 4  # repurposed = no_pred
+    assert "per_segment" in m["expected_sold"]
+    assert set(m["expected_sold"]["per_segment"].keys()) == {
+        label for label, _ in bt.PRICE_SEGMENTS_PPM2
+    }
+
+    # range coverage: 1 of 2 inside → 50% overall.
+    assert m["range_coverage"]["overall"]["n"] == 2
+    assert m["range_coverage"]["overall"]["n_covered"] == 1
+    assert m["range_coverage"]["overall"]["coverage_pct"] == pytest.approx(50.0)
+    # per-confidence: high covered 100%, low covered 0%.
+    assert m["range_coverage"]["per_confidence"]["high"]["coverage_pct"] == pytest.approx(100.0)
+    assert m["range_coverage"]["per_confidence"]["low"]["coverage_pct"] == pytest.approx(0.0)
+
+    # calibration: high tighter/accurate, low not.
+    assert m["calibration"]["high"]["n"] == 1
+    assert m["calibration"]["high"]["coverage_pct"] == pytest.approx(100.0)
+    assert m["calibration"]["high"]["mape_pct"] == pytest.approx(0.0)
+    assert m["calibration"]["low"]["coverage_pct"] == pytest.approx(0.0)
+    assert m["calibration"]["low"]["mape_pct"] == pytest.approx(50.0)
+
+    # sharpness present.
+    assert m["sharpness"]["n"] == 2
+    assert m["sharpness"]["median_rel_width"] is not None
+
+    # confidence order: canonical three first.
+    assert m["confidence_order"][:3] == ["high", "medium", "low"]
+
+
+def test_compute_full_metrics_empty_is_safe() -> None:
+    m = bt._compute_full_metrics([])
+    assert m["expected_sold"]["overall"]["n"] == 0
+    assert m["range_coverage"]["overall"]["coverage_pct"] is None
+    assert m["calibration"]["high"]["n"] == 0
+    assert m["sharpness"]["median_rel_width"] is None
+
+
+def test_compute_full_metrics_no_expected_sold_counts_in_calibration_only() -> None:
+    # A priced deal with no expected_sold (ratio unresolved) and no range:
+    # counts in calibration n but contributes nothing to expected_sold / coverage.
+    preds = [
+        _pred(
+            deal_id=1,
+            confidence="medium",
+            es_ppm2=None,
+            es_price=None,
+            range_low=None,
+            range_high=None,
+        )
+    ]
+    m = bt._compute_full_metrics(preds)
+    assert m["expected_sold"]["overall"]["n"] == 0  # no es row
+    assert m["range_coverage"]["overall"]["n"] == 0  # no range row
+    assert m["calibration"]["medium"]["n"] == 1  # still counted
+    assert m["calibration"]["medium"]["coverage_pct"] is None
+
+
+# --------------------------------------------------------------------------- #
+# #1966 _render_full_table — smoke (must not crash, renders all blocks).
+# --------------------------------------------------------------------------- #
+
+
+def test_render_full_table_runs_on_real_metrics() -> None:
+    preds = [
+        _pred(deal_id=1, confidence="high", es_ppm2=100_000.0, sold_ppm2=100_000.0),
+        _pred(deal_id=3, confidence="low", es_ppm2=100_000.0, sold_ppm2=200_000.0),
+    ]
+    m = bt._compute_full_metrics(preds, n_no_prediction=1, per_rooms_no_prediction={2: 1})
+    m["headline"] = {
+        "deal_median_ppm2": 100_000.0,
+        "ask_median_ppm2": 120_000.0,
+        "spread_pct": 20.0,
+    }
+    out = bt._render_full_table(m)
+    assert "full spine" in out
+    assert "EXPECTED_SOLD" in out
+    assert "RANGE COVERAGE" in out
+    assert "CONFIDENCE CALIBRATION" in out
+    assert "SHARPNESS" in out
+    assert "per price-segment" in out
+    assert "эконом" in out  # segment band rendered
+    assert "regression baseline" in out  # caveat present
+
+
+def test_render_full_table_handles_empty_sample() -> None:
+    m = bt._compute_full_metrics([])
+    m["headline"] = {"deal_median_ppm2": None, "ask_median_ppm2": None, "spread_pct": None}
+    out = bt._render_full_table(m)
+    assert "n/a" in out  # None metrics render as n/a, no crash
+    assert "BACKTEST" in out
diff --git a/tradein-mvp/backend/tests/test_domclick_sweep.py b/tradein-mvp/backend/tests/test_domclick_sweep.py
index e6f9200c..d51191d2 100644
--- a/tradein-mvp/backend/tests/test_domclick_sweep.py
+++ b/tradein-mvp/backend/tests/test_domclick_sweep.py
@@ -78,6 +78,9 @@ def _stub_domclick_lifecycle(monkeypatch: pytest.MonkeyPatch) -> None:
     def _init(self: DomClickScraper) -> None:
         self.request_delay_sec = 0.0
         self.parse_failures = 0
+        self.geo_filtered = 0
+        self.blocked = False
+        self.fetch_errors = 0
 
     async def _aenter(self: DomClickScraper) -> DomClickScraper:
         return self
@@ -109,6 +112,8 @@ def test_domclick_sweep_counters_defaults() -> None:
     assert c.lots_updated == 0
     assert c.pages_fetched == 0
     assert c.errors_count == 0
+    assert c.blocked == 0
+    assert c.geo_filtered == 0
 
 
 def test_domclick_sweep_counters_to_dict_all_keys() -> None:
@@ -233,8 +238,13 @@ async def test_cancel_before_serp_skips_fetch(monkeypatch: pytest.MonkeyPatch) -
     assert fake.done is None
 
 
-async def test_fetch_city_exception_graceful(monkeypatch: pytest.MonkeyPatch) -> None:
-    """Исключение в fetch_city → errors_count++, sweep завершается mark_done (не fatal)."""
+async def test_fetch_city_exception_no_reraise(monkeypatch: pytest.MonkeyPatch) -> None:
+    """Исключение в fetch_city → errors_count++, sweep НЕ ре-райзит (graceful).
+
+    Honest-status (#1968): полный крах SERP-фазы (errors_count=1, 0 lots) → mark_failed,
+    НЕ mark_done. Run без единого листинга по внешней причине — это реальный failure,
+    а не успешный пустой прогон. Re-raise при этом НЕ происходит (task завершается чисто).
+    """
 
     async def fake_fetch_fail(
         self: DomClickScraper,
@@ -242,7 +252,7 @@ async def test_fetch_city_exception_graceful(monkeypatch: pytest.MonkeyPatch) ->
         rooms: list[int] | None = None,
         pages: int = 20,
     ) -> list[ScrapedLot]:
-        raise RuntimeError("camoufox crashed")
+        raise RuntimeError("browser crashed")
 
     monkeypatch.setattr(DomClickScraper, "fetch_city", fake_fetch_fail)
     monkeypatch.setattr(scrape_pipeline, "save_listings", lambda *a, **k: (0, 0))
@@ -256,36 +266,107 @@ async def test_fetch_city_exception_graceful(monkeypatch: pytest.MonkeyPatch) ->
 
     assert counters.errors_count == 1
     assert counters.lots_fetched == 0
-    assert fake.done is not None, "graceful: mark_done вызывается несмотря на ошибку SERP"
-    assert fake.failed is None
+    # 0 lots + error → honest mark_failed (не маскируем как done)
+    assert fake.failed is not None
+    assert fake.done is None
 
 
-# ── _parse_html parser unit test ────────────────────────────────────────────
+async def test_fetch_errors_and_no_lots_marks_failed(monkeypatch: pytest.MonkeyPatch) -> None:
+    """Scraper fetch_errors>0 + 0 лотов (без block) → mark_failed (FIX 3 / closes #1968).
 
-
-def test_parse_html_minimal_card() -> None:
-    """Минимальная DomClick карточка → ScrapedLot с rooms/area/floor/price/source_id."""
-    # __init__ застаблен фикстурой _stub_domclick_lifecycle → нет DB/browser
-    scraper = DomClickScraper()
-    html = """
-    
-      
-        
2-комн. квартира

-        52,3 м²
-        5 / 16 эт.
-        Екатеринбург, улица Ленина, 10
-        6 500 000 ₽
-      
-    
+    Нераспознанный block-вариант приходит как fetch_error (не blocked=True), но
+    honest-status всё равно обязан пометить run failed при пустом результате.
     """
-    lots = scraper._parse_html(html)
-    assert len(lots) == 1
-    lot = lots[0]
+
+    async def fake_fetch_errs(
+        self: DomClickScraper,
+        city_id: int,
+        rooms: list[int] | None = None,
+        pages: int = 100,
+    ) -> list[ScrapedLot]:
+        # Имитируем не-block ошибки извлечения JSON: blocked остаётся False
+        self.fetch_errors = 3
+        return []
+
+    monkeypatch.setattr(DomClickScraper, "fetch_city", fake_fetch_errs)
+    monkeypatch.setattr(scrape_pipeline, "save_listings", lambda *a, **k: (0, 0))
+
+    fake = _FakeRuns()
+    _install_runs(monkeypatch, fake)
+
+    counters = await scrape_pipeline.run_domclick_city_sweep(
+        db=MagicMock(), run_id=6, request_delay_sec=0.0
+    )
+
+    assert counters.lots_fetched == 0
+    assert counters.blocked == 0
+    assert counters.errors_count >= 3  # fetch_errors свёрнуты в errors_count
+    assert fake.failed is not None
+    assert fake.done is None
+
+
+async def test_blocked_and_no_lots_marks_failed(monkeypatch: pytest.MonkeyPatch) -> None:
+    """QRATOR-блок + 0 лотов → mark_failed (честный статус, closes #1968)."""
+
+    async def fake_fetch_blocked(
+        self: DomClickScraper,
+        city_id: int,
+        rooms: list[int] | None = None,
+        pages: int = 100,
+    ) -> list[ScrapedLot]:
+        # Имитируем QRATOR-блок: установим флаг на scraper и вернём пустой список
+        self.blocked = True
+        return []
+
+    monkeypatch.setattr(DomClickScraper, "fetch_city", fake_fetch_blocked)
+    monkeypatch.setattr(scrape_pipeline, "save_listings", lambda *a, **k: (0, 0))
+
+    fake = _FakeRuns()
+    _install_runs(monkeypatch, fake)
+
+    counters = await scrape_pipeline.run_domclick_city_sweep(
+        db=MagicMock(), run_id=5, request_delay_sec=0.0
+    )
+
+    assert counters.lots_fetched == 0
+    assert counters.blocked == 1
+    # Честный статус: mark_failed должен быть вызван, mark_done — нет
+    assert fake.failed is not None
+    assert "QRATOR" in fake.failed[0]
+    assert fake.done is None
+
+
+# ── _map_item via sweep — quick integration smoke ────────────────────────────
+
+
+def test_map_item_basic_mapping() -> None:
+    """DomClickScraper._map_item маппит BFF offer-item → ScrapedLot корректно."""
+    scraper = DomClickScraper()  # __init__ stubbed
+    item = {
+        "id": 111222333,
+        "path": "https://domclick.ru/card/sale__flat__111222333",
+        "location": {"lat": 56.838, "lon": 60.612},
+        "address": {"displayName": "Екатеринбург, улица Ленина, 10"},
+        "objectInfo": {"area": 52.3, "rooms": 2, "floor": 5},
+        "house": {"floors": 16, "buildYear": 2010},
+        "price": 6_500_000,
+        "squarePrice": 124282,
+        "flatComplex": None,
+        "isRosreestrApproved": False,
+        "publishedDate": "2026-06-01T10:00:00+05:00",
+        "updatedDate": None,
+        "lastPriceHistoryState": None,
+        "offerRegionName": "Екатеринбург",
+    }
+    lot = scraper._map_item(item)
+    assert lot is not None
     assert lot.source == "domklik"
-    assert lot.source_id == "2075671636"
+    assert lot.source_id == "111222333"
     assert lot.rooms == 2
     assert lot.area_m2 == pytest.approx(52.3)
     assert lot.floor == 5
     assert lot.total_floors == 16
     assert lot.price_rub == 6_500_000
-    assert lot.lat is None and lot.lon is None
+    assert lot.listing_segment == "vtorichka"
+    assert lot.lat == pytest.approx(56.838)
+    assert lot.lon == pytest.approx(60.612)
diff --git a/tradein-mvp/backend/tests/test_estimator_price_spine.py b/tradein-mvp/backend/tests/test_estimator_price_spine.py
new file mode 100644
index 00000000..cebee96b
--- /dev/null
+++ b/tradein-mvp/backend/tests/test_estimator_price_spine.py
@@ -0,0 +1,401 @@
+"""Hermetic unit tests for _price_from_inputs (#1966).
+
+Calls the pure synchronous pricing function directly with stub callables and
+hand-built inputs — no DB, no network, no mocks. Verifies that the extraction
+preserved the pricing logic identically to the original block in estimate_quality.
+
+NOTE: importing app.services.estimator pulls app.core.config.Settings which
+requires DATABASE_URL. Set it BEFORE importing app modules.
+"""
+
+import os
+
+import pytest
+
+os.environ.setdefault("DATABASE_URL", "postgresql+psycopg://test:test@localhost:5432/test")
+
+from app.services import estimator
+from app.services.geocoder import GeocodeResult
+
+# ── helpers ──────────────────────────────────────────────────────────────────
+
+
+def _geo(coarse: bool = False) -> GeocodeResult:
+    """Minimal GeocodeResult for test injection."""
+    full_address = "Екатеринбург" if coarse else "ул. Тестовая, 1"
+    return GeocodeResult(
+        lat=56.838,
+        lon=60.597,
+        full_address=full_address,
+        provider="nominatim",
+        confidence="approximate",
+    )
+
+
+def _lot(ppm2: float, address: str = "ул. Тестовая, 1", source: str = "avito") -> dict:
+    return {"price_per_m2": ppm2, "address": address, "source": source}
+
+
+def _lots(ppm2: float, n: int = 7) -> list[dict]:
+    """n unique-address lots all at the same ppm2."""
+    return [_lot(ppm2, address=f"ул. Тестовая, {i + 1}") for i in range(n)]
+
+
+def _dkp_raw(
+    low: int = 80_000,
+    median: int = 120_000,
+    high: int = 150_000,
+    count: int = 20,
+    period_months: int = 12,
+) -> dict:
+    return {
+        "low_ppm2": low,
+        "median_ppm2": median,
+        "high_ppm2": high,
+        "count": count,
+        "period_months": period_months,
+    }
+
+
+def _anchor_comp(ppm2: float, area: float = 50.0, rooms: int = 2) -> dict:
+    return {"price_per_m2": ppm2, "area_m2": area, "rooms": rooms}
+
+
+# Stub callables — returned in each test via closure.
+def _ratio_stub(
+    ratio: float | None,
+    basis: str | None = "per_rooms",
+) -> "tuple[float | None, str | None]":
+    return ratio, basis if ratio is not None else None
+
+
+def _qi_stub_none(q: str) -> "tuple[float, int] | None":
+    return None
+
+
+def _qis_stub_empty(qs: list[str]) -> dict[str, float]:
+    return {}
+
+
+def _call(
+    *,
+    listings: list[dict] | None = None,
+    area_m2: float = 50.0,
+    rooms: int | None = 2,
+    repair_state: str | None = None,
+    floor: int | None = 5,
+    total_floors: int | None = 10,
+    target_year: int | None = None,
+    analog_tier: str = "W",
+    fallback_used: bool = False,
+    area_widened: bool = False,
+    anchor_comps: list[dict] | None = None,
+    anchor_tier_fetched: str | None = None,
+    dkp_raw: dict | None = None,
+    imv_anchor: dict | None = None,
+    imv_eval=None,
+    yandex_val_present: bool = False,
+    cian_val_present: bool = False,
+    ratio: float | None = None,
+    quarter_index_lookup=None,
+    quarter_indexes_lookup=None,
+    target_house_cadnum: str | None = None,
+    dadata_coarse: bool = False,
+    geo: GeocodeResult | None = None,
+    dadata_qc_geo: int | None = None,
+) -> estimator.PricingResult:
+    if listings is None:
+        listings = _lots(100_000)
+    if anchor_comps is None:
+        anchor_comps = []
+    if geo is None:
+        geo = _geo(coarse=dadata_coarse)
+    if quarter_index_lookup is None:
+        quarter_index_lookup = _qi_stub_none
+    if quarter_indexes_lookup is None:
+        quarter_indexes_lookup = _qis_stub_empty
+
+    _ratio = ratio
+    _basis = "per_rooms" if ratio is not None else None
+
+    def ratio_resolver(appm2: float | None) -> tuple[float | None, str | None]:
+        return _ratio, _basis if _ratio is not None else None
+
+    return estimator._price_from_inputs(
+        listings=listings,
+        area_m2=area_m2,
+        rooms=rooms,
+        repair_state=repair_state,
+        floor=floor,
+        total_floors=total_floors,
+        target_year=target_year,
+        analog_tier=analog_tier,
+        fallback_used=fallback_used,
+        area_widened=area_widened,
+        anchor_comps=anchor_comps,
+        anchor_tier_fetched=anchor_tier_fetched,
+        dkp_raw=dkp_raw,
+        imv_anchor=imv_anchor,
+        imv_eval=imv_eval,
+        yandex_val_present=yandex_val_present,
+        cian_val_present=cian_val_present,
+        ratio_resolver=ratio_resolver,
+        quarter_index_lookup=quarter_index_lookup,
+        quarter_indexes_lookup=quarter_indexes_lookup,
+        target_house_cadnum=target_house_cadnum,
+        dadata_coarse=dadata_coarse,
+        geo=geo,
+        dadata_qc_geo=dadata_qc_geo,
+    )
+
+
+# ── Tests ────────────────────────────────────────────────────────────────────
+
+
+def test_radius_only_median_and_expected_sold() -> None:
+    """Pure radius path: 7 uniform lots → correct median, n_analogs, expected_sold."""
+    pr = _call(listings=_lots(100_000, n=7), ratio=0.95)
+
+    assert pr.median_price == int(100_000 * 50.0)  # 5_000_000
+    assert pr.median_ppm2 == 100_000.0
+    assert pr.n_analogs == 7
+    assert pr.anchor_tier is None
+    assert pr.dkp_corridor is None
+    assert pr.asking_to_sold_ratio == 0.95
+    assert pr.expected_sold_price == round(5_000_000 * 0.95)  # 4_750_000
+    assert pr.expected_sold_per_m2 == round(100_000 * 0.95)  # 95_000
+    assert "avito" in pr.sources_used_pre
+    assert len(pr.listings_clean) == 7
+
+
+def test_same_building_anchor_tier_a_mutates_headline() -> None:
+    """Tier A same-building anchor replaces radius median with higher price.
+
+    5 radius lots at 100k ppm2 (5M total). 5 anchor comps at 200k ppm2 (10M total).
+    After anchor fires: median_price >> radius median, n_analogs == anchor count.
+    """
+    comps = [_anchor_comp(200_000) for _ in range(5)]
+    pr = _call(
+        listings=_lots(100_000, n=5),
+        anchor_comps=comps,
+        anchor_tier_fetched="A",
+        ratio=None,
+    )
+
+    # Anchor must have fired (not suppressed).
+    assert pr.anchor_tier == "A"
+    # Headline is anchor-derived — must be above radius median (5_000_000).
+    assert pr.median_price > 5_000_000
+    # n_analogs resets to anchor population.
+    assert pr.n_analogs == 5
+    # anchor_comps_used is the injected comps list.
+    assert len(pr.anchor_comps_used) == 5
+    # No ratio → expected_sold is None.
+    assert pr.expected_sold_price is None
+
+
+def test_tier_c_corridor_gate_suppresses_anchor() -> None:
+    """Tier C anchor ppm2 >> corridor_high × mult → anchor suppressed.
+
+    anchor_tier remains "C" in the result (gate sets anchor=None but doesn't
+    reset anchor_tier); headline stays at the radius median.
+    """
+    # 5 comps at 300k ppm2; corridor_high=150k; gate threshold=150k×1.5=225k.
+    # 300k > 225k → suppressed.
+    comps = [_anchor_comp(300_000) for _ in range(5)]
+    radius_median_price = int(100_000 * 50.0)
+    pr = _call(
+        listings=_lots(100_000, n=5),
+        anchor_comps=comps,
+        anchor_tier_fetched="C",
+        dkp_raw=_dkp_raw(high=150_000, count=15),
+        ratio=None,
+    )
+
+    # Tier C gate sets anchor=None but leaves anchor_tier="C".
+    assert pr.anchor_tier == "C"
+    # Headline was NOT mutated by the suppressed anchor — stays at radius median.
+    assert pr.median_price == radius_median_price
+    # anchor_comps_used stays empty (anchor didn't fire).
+    assert pr.anchor_comps_used == []
+
+
+def test_low_conf_gate_suppresses_anchor() -> None:
+    """Low-confidence anchor is suppressed; anchor_tier reset to None.
+
+    4 comps with wide spread → high cv → fsd > 0.20 → confidence='low' → suppressed.
+    """
+    # 4 comps at [100k, 200k, 300k, 400k] → cv≈0.45, fsd≈0.20 → "low".
+    comps = [_anchor_comp(p) for p in [100_000, 200_000, 300_000, 400_000]]
+    radius_median_price = int(100_000 * 50.0)
+    pr = _call(
+        listings=_lots(100_000, n=5),
+        anchor_comps=comps,
+        anchor_tier_fetched="A",  # starts as A, gate resets to None
+        ratio=None,
+    )
+
+    # Gate resets anchor_tier to None on suppression.
+    assert pr.anchor_tier is None
+    # Headline stays at radius median.
+    assert pr.median_price == radius_median_price
+    assert pr.anchor_comps_used == []
+
+
+def test_imv_blend_raises_median_when_anchor_tier_none() -> None:
+    """IMV blend pushes radius median up when IMV >> median × threshold.
+
+    radius median=5M, IMV recommended=7M, area=50, weight=0.5, threshold=1.15.
+    IMV/radius = 7M/5M = 1.4 > 1.15 → blend: new_median = round(5M×0.5 + 7M×0.5).
+    """
+    imv_anchor = {
+        "recommended_price": 7_000_000,
+        "lower_price": 6_000_000,
+        "higher_price": 8_000_000,
+        "market_count": 50,
+    }
+    pr = _call(
+        listings=_lots(100_000, n=5),
+        imv_anchor=imv_anchor,
+        ratio=None,
+    )
+
+    expected_median = round(5_000_000 * 0.5 + 7_000_000 * 0.5)  # 6_000_000
+    assert pr.median_price == expected_median
+    assert pr.range_high == 8_000_000  # from anchor_higher
+    assert pr.avito_imv_summary is not None
+    assert pr.avito_imv_summary.recommended_price == 7_000_000
+    assert "avito_imv" in pr.sources_used_pre
+
+
+def test_quarter_index_guard2_skip_when_all_analogs_in_target_quarter() -> None:
+    """Guard-2: when all analogs are in the target quarter, index is NOT applied.
+
+    same_quarter_ratio=1.0 > skip_ratio=0.6 → Guard-2 fires → median unchanged.
+    """
+    target_cadnum = "66:41:0204016:350"
+    target_quarter = "66:41:0204016"
+    # Analogs are all in the SAME quarter as the target.
+    lots = [
+        {
+            "price_per_m2": 100_000,
+            "address": f"ул. Тестовая, {i + 1}",
+            "source": "avito",
+            "building_cadastral_number": f"{target_quarter}:{100 + i}",
+        }
+        for i in range(5)
+    ]
+
+    qi_called: list[str] = []
+
+    def qi_lookup(q: str) -> tuple[float, int] | None:
+        qi_called.append(q)
+        return (1.5, 100) if q == target_quarter else None  # high index — would change price
+
+    pr = _call(
+        listings=lots,
+        target_house_cadnum=target_cadnum,
+        quarter_index_lookup=qi_lookup,
+        quarter_indexes_lookup=_qis_stub_empty,
+        ratio=None,
+    )
+
+    # Guard-2 fired: median must remain unchanged (5M, not ×1.5).
+    assert pr.median_price == int(100_000 * 50.0)
+    # quarter_index was looked up but did NOT add "quarter_index" to sources.
+    assert "quarter_index" not in pr.sources_used_pre
+
+
+def test_quarter_index_applied_when_analogs_in_different_quarter() -> None:
+    """Quarter-index IS applied when analogs are in a different quarter from target.
+
+    target_qi=1.2, avg_analog_qi=1.0 → factor=1.2 → median_price×1.2.
+    Guard-2 skips (same_quarter_ratio=0.0 < 0.6).
+    """
+    target_cadnum = "66:41:0204016:350"
+    target_quarter = "66:41:0204016"
+    analog_quarter = "66:41:0999999"
+    lots = [
+        {
+            "price_per_m2": 100_000,
+            "address": f"ул. Иная, {i + 1}",
+            "source": "avito",
+            "building_cadastral_number": f"{analog_quarter}:{i + 1}",
+        }
+        for i in range(5)
+    ]
+
+    def qi_lookup(q: str) -> tuple[float, int] | None:
+        return (1.2, 100) if q == target_quarter else None
+
+    def qis_lookup(qs: list[str]) -> dict[str, float]:
+        return {q: 1.0 for q in qs if q == analog_quarter}
+
+    pr = _call(
+        listings=lots,
+        target_house_cadnum=target_cadnum,
+        quarter_index_lookup=qi_lookup,
+        quarter_indexes_lookup=qis_lookup,
+        ratio=None,
+    )
+
+    # factor=1.2/1.0=1.2; original radius=5M → adjusted=6M (±rounding via _apply_quarter_index)
+    assert pr.median_price > int(100_000 * 50.0)  # index pushed price up
+    assert "quarter_index" in pr.sources_used_pre
+
+
+def test_corridor_soft_clamp_headline_above_cap() -> None:
+    """Headline above corridor_high × (1+slack) is clamped down.
+
+    radius lots at 250k ppm2. corridor_high=150k, slack=0.40 →
+    cap=150k×1.40=210k. 250k > 210k → clamped to 210k.
+    """
+    pr = _call(
+        listings=_lots(250_000, n=7),
+        dkp_raw=_dkp_raw(low=80_000, median=120_000, high=150_000, count=15),
+        ratio=None,
+    )
+
+    # cap = 150_000 × 1.40 = 210_000
+    # Clamped: new ppm2 == 210_000, new_price = round(210_000 × 50)
+    assert pr.median_ppm2 == pytest.approx(210_000.0, rel=1e-4)
+    assert pr.median_price == round(210_000 * 50.0)
+    # DKP corridor present in result.
+    assert pr.dkp_corridor is not None
+    assert pr.dkp_corridor.high_ppm2 == 150_000
+
+
+def test_expected_sold_from_ratio_and_none_when_ratio_none() -> None:
+    """expected_sold = headline × ratio; when ratio is None, all expected_sold fields None."""
+    # Case A: ratio=0.90 → expected_sold fields filled.
+    pr_ratio = _call(listings=_lots(100_000, n=5), ratio=0.90)
+    assert pr_ratio.asking_to_sold_ratio == 0.90
+    assert pr_ratio.expected_sold_price is not None
+    assert pr_ratio.expected_sold_price == round(pr_ratio.median_price * 0.90)
+    assert pr_ratio.expected_sold_per_m2 is not None
+    assert pr_ratio.expected_sold_range_low is not None
+    assert pr_ratio.expected_sold_range_high is not None
+
+    # Case B: ratio=None → all expected_sold fields None.
+    pr_none = _call(listings=_lots(100_000, n=5), ratio=None)
+    assert pr_none.asking_to_sold_ratio is None
+    assert pr_none.ratio_basis is None
+    assert pr_none.expected_sold_price is None
+    assert pr_none.expected_sold_per_m2 is None
+    assert pr_none.expected_sold_range_low is None
+    assert pr_none.expected_sold_range_high is None
+
+
+def test_coarse_geo_downgrades_confidence_to_low() -> None:
+    """dadata_coarse=True with qc_geo=2 → confidence='low' with settlement label."""
+    pr = _call(
+        listings=_lots(100_000, n=7),
+        dadata_coarse=True,
+        dadata_qc_geo=2,
+        ratio=None,
+        # No anchor, no IMV → radius path → anchor_tier is None (not "A" → downgrade applies)
+        geo=_geo(coarse=False),  # geo itself not coarse; using dadata_coarse signal
+    )
+
+    assert pr.confidence == "low"
+    assert "населённого пункта" in pr.explanation
diff --git a/tradein-mvp/frontend/src/app/page.tsx b/tradein-mvp/frontend/src/app/page.tsx
index 18a2efe9..d5c343f0 100644
--- a/tradein-mvp/frontend/src/app/page.tsx
+++ b/tradein-mvp/frontend/src/app/page.tsx
@@ -182,7 +182,7 @@ export default function TradeInPage() {
           

             
Оценка квартиры на вторичке

             

-              Агрегируем данные из 7 источников + аналоги в продаже + фактические сделки.
+              Агрегируем 4 источника объявлений (Авито, Циан, Яндекс, N1) и реальные сделки Росреестра.
               Время сбора — 10–30 сек.
             

           

@@ -336,7 +336,7 @@ export default function TradeInPage() {