feat(22d): admin endpoint POST /admin/scrape/kn-catalog-objects #338

Merged

lekss361 merged 1 commit from feat/22d-catalog-objects-admin-endpoint into main

2026-05-17 21:41:54 +00:00

lekss361 commented

2026-05-17 21:39:14 +00:00

Owner

Summary

Manual trigger endpoint для catalog-object scraper из PR #335. Без него ad-hoc запуск возможен только через docker exec celery call на VPS.

Pattern follows existing scrape triggers (POST /kn, POST /pzz-sync, POST /ekburg-permits).

POST /api/v1/admin/scrape/kn-catalog-objects
Header: X-Admin-Token: <SCRAPE_ADMIN_TOKEN>
Body: {"region_code": 66, "max_objects": 3}

max_objects=null → берёт default из task (300). max_objects=3 → smoke-тест.

Test plan

Ruff clean

Post-deploy curl:

curl -X POST -H "X-Admin-Token: $TOKEN" -H "Content-Type: application/json" \
  -d '{"region_code":66,"max_objects":3}' \
  https://gendsgn.ru/api/v1/admin/scrape/kn-catalog-objects

Verify task accepted (returns task_id)
SELECT wall_type, energy_eff, ceiling_height_m FROM domrf_kn_objects WHERE catalog_scraped_at IS NOT NULL LIMIT 3 returns data

Follow-up: PR #335 merged (catalog-object scraper). Migration 118 applied 21:29:52Z.

## Summary Manual trigger endpoint для catalog-object scraper из PR #335. Без него ad-hoc запуск возможен только через `docker exec celery call` на VPS. Pattern follows existing scrape triggers (`POST /kn`, `POST /pzz-sync`, `POST /ekburg-permits`). ``` POST /api/v1/admin/scrape/kn-catalog-objects Header: X-Admin-Token: <SCRAPE_ADMIN_TOKEN> Body: {"region_code": 66, "max_objects": 3} ``` `max_objects=null` → берёт default из task (300). `max_objects=3` → smoke-тест. ## Test plan - [x] Ruff clean - [ ] Post-deploy curl: ``` curl -X POST -H "X-Admin-Token: $TOKEN" -H "Content-Type: application/json" \ -d '{"region_code":66,"max_objects":3}' \ https://gendsgn.ru/api/v1/admin/scrape/kn-catalog-objects ``` - [ ] Verify task accepted (returns `task_id`) - [ ] `SELECT wall_type, energy_eff, ceiling_height_m FROM domrf_kn_objects WHERE catalog_scraped_at IS NOT NULL LIMIT 3` returns data Follow-up: PR #335 merged (catalog-object scraper). Migration 118 applied 21:29:52Z.

lekss361 added 1 commit 2026-05-17 21:39:15 +00:00

feat(22d): admin endpoint POST /admin/scrape/kn-catalog-objects 1e08cd0c2e

Manual trigger для catalog-object scraper (PR #335). Smoke-тест и ad-hoc
повторные runs до next beat fire (Tuesday 04:00 UTC).

Body: {"region_code": 66, "max_objects": 3?}
Header: X-Admin-Token: <SCRAPE_ADMIN_TOKEN>

lekss361 commented

2026-05-17 21:41:48 +00:00

Author

Owner

Deep Code Review — verdict APPROVE

Summary

Status: APPROVE
Files: 1 (P1: backend/app/api/v1/admin_scrape.py)
Lines: +35 / -0
PR: #338, head 1e08cd0c

Pattern consistency

Mirrors trigger_ekburg_permits (admin_scrape.py:1229) exactly: BaseModel body, AdminTokenAuth dep, lazy import of task module, apply_async then return {task_id, ..., queued_at: "now"}.
AdminTokenAuth shared dependency (used in 20+ endpoints in same file) — auth check is centralized, not bypassed.

Conventions

No psycopg2, no requests, no print, no hardcoded secrets — clean.
Pydantic body uses only int and int | None with Field(ge=, le=) bounds — no EmailStr/HttpUrl/AnyUrl, so no risk of the email-validator dep crash that hit #330 (fixed in #337).
No SQL in handler — no ::type injection vector.
Lazy from app.workers.tasks.scrape_kn_catalog_objects import scrape_kn_catalog_objects — target module exists on main from PR #335.

Cross-file impact

Import target verified: backend/app/workers/tasks/scrape_kn_catalog_objects.py present on forgejo/main @ 7611287.
Task accepts region_code + optional max_objects kwargs — signature matches handler kwargs build.
New route /kn-catalog-objects does not collide with existing siblings (/kn, /pzz-sync, /ekburg-permits).

Risk

Low. Admin-gated, behind X-Admin-Token. max_objects bounded [1, 2000]. region_code bounded [1, 99]. No DB writes from handler — only Celery enqueue.

Merging.

## Deep Code Review — verdict APPROVE ### Summary - Status: APPROVE - Files: 1 (P1: `backend/app/api/v1/admin_scrape.py`) - Lines: +35 / -0 - PR: #338, head `1e08cd0c` ### Pattern consistency - Mirrors `trigger_ekburg_permits` (admin_scrape.py:1229) exactly: `BaseModel` body, `AdminTokenAuth` dep, lazy import of task module, `apply_async` then return `{task_id, ..., queued_at: "now"}`. - `AdminTokenAuth` shared dependency (used in 20+ endpoints in same file) — auth check is centralized, not bypassed. ### Conventions - No `psycopg2`, no `requests`, no `print`, no hardcoded secrets — clean. - Pydantic body uses only `int` and `int | None` with `Field(ge=, le=)` bounds — no `EmailStr`/`HttpUrl`/`AnyUrl`, so no risk of the email-validator dep crash that hit #330 (fixed in #337). - No SQL in handler — no `::type` injection vector. - Lazy `from app.workers.tasks.scrape_kn_catalog_objects import scrape_kn_catalog_objects` — target module exists on `main` from PR #335. ### Cross-file impact - Import target verified: `backend/app/workers/tasks/scrape_kn_catalog_objects.py` present on `forgejo/main` @ 7611287. - Task accepts `region_code` + optional `max_objects` kwargs — signature matches handler kwargs build. - New route `/kn-catalog-objects` does not collide with existing siblings (`/kn`, `/pzz-sync`, `/ekburg-permits`). ### Risk - Low. Admin-gated, behind `X-Admin-Token`. `max_objects` bounded `[1, 2000]`. `region_code` bounded `[1, 99]`. No DB writes from handler — only Celery enqueue. Merging.