lekss361/gendesign
1
0
Fork 0
Code Issues 57 Pull requests Projects Releases Packages Wiki Activity Actions
gendesign/backend
History
lekss361 4875ef15c3 fix(scrape-objective): stream-parse lots_pf via ijson to avoid OOM 
httpx.Response.json() загружает 607 МБ payload в Python list[dict] -> 2-3 ГБ RAM -> host OOM-killer SIGKILL ForkPoolWorker.

Изменения:
- ObjectiveClient._open_stream() / stream_report() — httpx stream=True
- parse_lots_pf_stream() — ijson.items() с batched UPSERT + commit/5 batch
- _save_raw() принимает payload=None для streamed reports
- objective_raw_reports.payload allowed NULL (миграция 79)
- ijson>=3.2.0 добавлен в pyproject.toml, uv.lock обновлён

Glitchtip BACKEND-1H/-1I (release fce48a8) — OOM на live trigger.
2026-05-17 19:33:43 +03:00
..
alembic ops: alembic baseline, pre-commit, TIGER cleanup, pg_dump scripts 2026-04-26 13:08:51 +03:00
app fix(scrape-objective): stream-parse lots_pf via ijson to avoid OOM 2026-05-17 19:33:43 +03:00
db/init ops: alembic baseline, pre-commit, TIGER cleanup, pg_dump scripts 2026-04-26 13:08:51 +03:00
output feat(geo): NSPD bulk-fetcher без Playwright + resume-friendly UI 2026-05-11 08:53:28 +03:00
tests fix(sf-09): capacity-aware redistribute (round 2) — bounded + correct invariant 2026-05-17 15:30:50 +03:00
.dockerignore fix(scraper): диагностируем «кнопка не работает» — Redis lock + task_received log 2026-04-28 23:17:42 +03:00
.env.example feat(backend): sentry-sdk init для FastAPI + Celery (#204 backend) (#207) 2026-05-16 15:09:16 +00:00
.env.runtime.example sprint1: nspd scraper industrialization, per-bucket elasticity, cadastre cross-check, sentry releases 2026-04-30 21:51:19 +03:00
alembic.ini ops: alembic baseline, pre-commit, TIGER cleanup, pg_dump scripts 2026-04-26 13:08:51 +03:00
debug.log feat(geo): NSPD bulk-fetcher без Playwright + resume-friendly UI 2026-05-11 08:53:28 +03:00
Dockerfile fix(scraper): диагностируем «кнопка не работает» — Redis lock + task_received log 2026-04-28 23:17:42 +03:00
pyproject.toml fix(scrape-objective): stream-parse lots_pf via ijson to avoid OOM 2026-05-17 19:33:43 +03:00
uv.lock fix(scrape-objective): stream-parse lots_pf via ijson to avoid OOM 2026-05-17 19:33:43 +03:00