fix(22d): register scrape_kn_catalog_objects in celery include list #339

Merged

lekss361 merged 2 commits from fix/22d-register-catalog-objects-task into main

2026-05-17 21:54:14 +00:00

lekss361 commented

2026-05-17 21:50:34 +00:00

Owner

Summary

PR #335 добавил task scrape_kn_catalog_objects, но celery_app.py имеет explicit include=[] список (а не autodiscover). Новый модуль не был добавлен → worker не импортировал task → apply_async из admin endpoint валился с:

KeyError: 'tasks.scrape_kn_catalog_objects.scrape_kn_catalog_objects'

One-line add to include list.

Test plan

Ruff/format pass
Post-deploy: re-trigger POST /admin/scrape/kn-catalog-objects {"max_objects":3} → должно вернуть task_id и выполниться (не упасть)
Verify SELECT wall_type, energy_eff FROM domrf_kn_objects WHERE catalog_scraped_at IS NOT NULL LIMIT 3 returns data

## Summary PR #335 добавил task `scrape_kn_catalog_objects`, но `celery_app.py` имеет **explicit `include=[]`** список (а не autodiscover). Новый модуль не был добавлен → worker не импортировал task → `apply_async` из admin endpoint валился с: ``` KeyError: 'tasks.scrape_kn_catalog_objects.scrape_kn_catalog_objects' ``` One-line add to `include` list. ## Test plan - [x] Ruff/format pass - [ ] Post-deploy: re-trigger `POST /admin/scrape/kn-catalog-objects {"max_objects":3}` → должно вернуть task_id и **выполниться** (не упасть) - [ ] Verify `SELECT wall_type, energy_eff FROM domrf_kn_objects WHERE catalog_scraped_at IS NOT NULL LIMIT 3` returns data

lekss361 added 2 commits 2026-05-17 21:50:35 +00:00

feat(22d): admin endpoint POST /admin/scrape/kn-catalog-objects 1e08cd0c2e

Manual trigger для catalog-object scraper (PR #335). Smoke-тест и ad-hoc
повторные runs до next beat fire (Tuesday 04:00 UTC).

Body: {"region_code": 66, "max_objects": 3?}
Header: X-Admin-Token: <SCRAPE_ADMIN_TOKEN>

fix(22d): register scrape_kn_catalog_objects in celery include list f3707712e5

PR #335 добавил task, но celery_app.py имеет explicit `include=[]` (а не
autodiscover), поэтому worker не импортировал новый модуль и не зарегистрировал
task name. apply_async из admin endpoint падал с
KeyError: 'tasks.scrape_kn_catalog_objects.scrape_kn_catalog_objects'.

One-line add to include list.

lekss361 commented

2026-05-17 21:54:10 +00:00

Author

Owner

Deep Code Review — verdict ✅ APPROVE

Net effect vs main

backend/app/workers/celery_app.py | 1 +

One-line surgical fix.

Diff size note

PR diff shows +36/-0, but that's against stale merge_base 7611287 (forked before #338). The branch contains 2 commits:

1e08cd0 — admin endpoint (duplicate of #338 already on main)
f370771 — the include-list fix

Verified via git merge-tree forgejo/main forgejo/fix/22d-... → resulting tree differs from main by exactly +1 line in celery_app.py. 3-way merge correctly deduplicates the endpoint hunk (identical text) and preserves landing.py added on main (#331).

Verification

app.workers.tasks.scrape_kn_catalog_objects — module exists on main (c3b96d2, added by #335)
Task decorator confirmed: @celery_app.task → def scrape_kn_catalog_objects(...)
Module path string matches actual file
No other includes accidentally removed (whole list preserved)
Insertion ordering reasonable (right after scrape_kn, before unrelated tasks)

Conventions

No psycopg2 / requests / print / f-string SQL / hardcoded secrets touched
100-char line OK

Why this is needed

PR #335 added the task module, but celery_app.py uses explicit include=[...] (not autodiscover). Worker silently doesn't import → apply_async from #338 endpoint hits KeyError: 'tasks.scrape_kn_catalog_objects...'. This PR closes the loop.

Post-merge test plan (from PR body)

Re-deploy worker
POST /admin/scrape/kn-catalog-objects {"max_objects":3} → expect 200 + task_id, no KeyError
SELECT wall_type, energy_eff FROM domrf_kn_objects WHERE catalog_scraped_at IS NOT NULL LIMIT 3

Merging via deep-code-reviewer.

## Deep Code Review — verdict ✅ APPROVE ### Net effect vs main ``` backend/app/workers/celery_app.py | 1 + ``` One-line surgical fix. ### Diff size note PR diff shows +36/-0, but that's against stale `merge_base 7611287` (forked before #338). The branch contains 2 commits: 1. `1e08cd0` — admin endpoint (duplicate of #338 already on main) 2. `f370771` — the include-list fix Verified via `git merge-tree forgejo/main forgejo/fix/22d-...` → resulting tree differs from main by **exactly +1 line in celery_app.py**. 3-way merge correctly deduplicates the endpoint hunk (identical text) and preserves `landing.py` added on main (#331). ### Verification - `app.workers.tasks.scrape_kn_catalog_objects` — module exists on main (`c3b96d2`, added by #335) - Task decorator confirmed: `@celery_app.task` → `def scrape_kn_catalog_objects(...)` - Module path string matches actual file - No other includes accidentally removed (whole list preserved) - Insertion ordering reasonable (right after `scrape_kn`, before unrelated tasks) ### Conventions - No psycopg2 / requests / print / f-string SQL / hardcoded secrets touched - 100-char line OK ### Why this is needed PR #335 added the task module, but `celery_app.py` uses **explicit `include=[...]`** (not autodiscover). Worker silently doesn't import → `apply_async` from #338 endpoint hits `KeyError: 'tasks.scrape_kn_catalog_objects...'`. This PR closes the loop. ### Post-merge test plan (from PR body) 1. Re-deploy worker 2. `POST /admin/scrape/kn-catalog-objects {"max_objects":3}` → expect 200 + task_id, no KeyError 3. `SELECT wall_type, energy_eff FROM domrf_kn_objects WHERE catalog_scraped_at IS NOT NULL LIMIT 3` Merging via deep-code-reviewer.