httpx.Response.json() загружает 607 МБ payload в Python list[dict] -> 2-3 ГБ RAM -> host OOM-killer SIGKILL ForkPoolWorker.
Изменения:
- ObjectiveClient._open_stream() / stream_report() — httpx stream=True
- parse_lots_pf_stream() — ijson.items() с batched UPSERT + commit/5 batch
- _save_raw() принимает payload=None для streamed reports
- objective_raw_reports.payload allowed NULL (миграция 79)
- ijson>=3.2.0 добавлен в pyproject.toml, uv.lock обновлён
Glitchtip BACKEND-1H/-1I (release
|
||
|---|---|---|
| .. | ||
| __init__.py | ||
| ekburg_permits_sync.py | ||
| noise_sync.py | ||
| nspd_denorm_backfill.py | ||
| nspd_geo.py | ||
| nspd_sync.py | ||
| objective_etl.py | ||
| poi_sync.py | ||
| pzz_sync.py | ||
| refresh_analytics.py | ||
| scrape_cadastre.py | ||
| scrape_kn.py | ||
| scrape_objective.py | ||