proxy-pool P2: сервис proxy_pool (acquire/release/health) + health-checker #2162

New issue

Closed

opened 2026-07-02 16:25:33 +00:00 by lekss361 · 1 comment

lekss361 commented

2026-07-02 16:25:33 +00:00

Owner

Часть Epic #2160, шаг P2. ⛔ Depends P1 (таблица). Аддитивно (не подключён к бою).

Goal

app/services/proxy_pool.py: acquire(db, provider) -> Proxy | None (SELECT enabled AND healthy AND affinity IN (provider,'any') AND leased_by IS NULL ORDER BY last_ok_at NULLS LAST FOR UPDATE SKIP LOCKED LIMIT 1 → UPDATE leased_by/leased_at); release(db, proxy_id); mark_health(db, proxy_id, ok, exit_ip?, latency_ms?) (ok → consecutive_fails=0, last_ok_at=now; fail → consecutive_fails+1, авто-enabled=false при ≥ порога). healthy = consecutive_fails < N (конфиг).
Health-checker: scheduler source proxy_healthcheck (в scheduler.py + seed в scrape_schedules) — периодически для каждого enabled прокси curl ipify через него → mark_health. Освобождает stale-lease (leased_at старше threshold → release, на случай упавшего sweep).

Acceptance

unit-тесты acquire (SKIP LOCKED конкурентность — два acquire дают РАЗНЫЕ прокси), release, mark_health (fail→disable), stale-lease reaper
health-checker задача регистрируется в scheduler dispatch
НЕ подключать к боевым scrapers (P3/P4)
psycopg v3, SAVEPOINT где нужно

Agent: backend-engineer

Часть Epic #2160, шаг P2. ⛔ Depends P1 (таблица). Аддитивно (не подключён к бою). ## Goal - `app/services/proxy_pool.py`: `acquire(db, provider) -> Proxy | None` (SELECT enabled AND healthy AND affinity IN (provider,'any') AND leased_by IS NULL ORDER BY last_ok_at NULLS LAST FOR UPDATE SKIP LOCKED LIMIT 1 → UPDATE leased_by/leased_at); `release(db, proxy_id)`; `mark_health(db, proxy_id, ok, exit_ip?, latency_ms?)` (ok → consecutive_fails=0, last_ok_at=now; fail → consecutive_fails+1, авто-`enabled=false` при ≥ порога). healthy = consecutive_fails < N (конфиг). - Health-checker: scheduler source `proxy_healthcheck` (в `scheduler.py` + seed в `scrape_schedules`) — периодически для каждого enabled прокси curl ipify через него → mark_health. Освобождает stale-lease (leased_at старше threshold → release, на случай упавшего sweep). ## Acceptance - unit-тесты acquire (SKIP LOCKED конкурентность — два acquire дают РАЗНЫЕ прокси), release, mark_health (fail→disable), stale-lease reaper - health-checker задача регистрируется в scheduler dispatch - НЕ подключать к боевым scrapers (P3/P4) - psycopg v3, SAVEPOINT где нужно ## Agent: backend-engineer

lekss361 added the

labels 2026-07-02 16:26:05 +00:00

lekss361 added the

status/ready

status/wip

labels 2026-07-02 16:31:55 +00:00

bot-backend referenced this issue from a commit

2026-07-02 16:40:07 +00:00

feat(proxy-pool): pool service (acquire/release/health) + healthcheck scheduler task (#2162)

lekss361 referenced this issue

2026-07-02 16:41:29 +00:00

feat(proxy-pool): pool service (acquire/release/health) + healthcheck scheduler task (#2162) #2169

lekss361 referenced this issue from a commit

2026-07-02 16:41:41 +00:00

feat(proxy-pool): pool service (acquire/release/health) + healthcheck scheduler task (#2162)

lekss361 closed this issue

2026-07-02 16:42:13 +00:00

lekss361 commented

2026-07-02 16:42:16 +00:00

Author

Owner

Смержено (PR #2169). Сервис proxy_pool (acquire/release/mark_health/reap_stale_leases) + health-checker (scheduler source proxy_healthcheck, sub-hourly каденс, миграция 158) + 14 тестов, 40 регресс. Аддитивно — не подключено к боевому fetch (P3/P4). После деплоя можно грузить прокси через POST /api/v1/admin/proxies/bulk, health начнёт пинговать через ipify.

Смержено (PR #2169). Сервис `proxy_pool` (acquire/release/mark_health/reap_stale_leases) + health-checker (scheduler source `proxy_healthcheck`, sub-hourly каденс, миграция 158) + 14 тестов, 40 регресс. Аддитивно — не подключено к боевому fetch (P3/P4). После деплоя можно грузить прокси через `POST /api/v1/admin/proxies/bulk`, health начнёт пинговать через ipify.