Two related fixes: 1. worker_process_init handler disposes the SQLAlchemy engine in each prefork child. Without this, child processes inherit open psycopg sockets from the parent. First use in a child raises ProgrammingError: can't change 'autocommit' now: connection in transaction status INTRANS. This was killing 1 of every 5 parallel geo jobs on cold start (job 13 in latest bulk run). 2. Add logger.info at start/end of worker_ready resume handler so we can see in worker logs whether it actually fired and how many jobs it resumed. |
||
|---|---|---|
| .. | ||
| tasks | ||
| __init__.py | ||
| celery_app.py | ||