lekss361
|
34e8c72e6c
|
feat(tradein): cian_session cookies management + admin endpoints (Wave 4 Worker A)
- New: app/services/cian_session.py — load/save/verify encrypted Cian cookies
(pgp_sym_encrypt via COOKIE_ENCRYPTION_KEY, psycopg v3, httpx)
- Modified: app/api/v1/admin.py — POST /scrape/cian/upload-cookies +
GET /scrape/cian/test-auth (behind existing require_admin X-Admin-Token)
- Modified: app/core/config.py — add cookie_encryption_key setting (env COOKIE_ENCRYPTION_KEY)
- New: tests/test_cian_session.py — 16 unit tests (16/16 passed)
- New: scripts/upload-cian-cookies.sh — CLI helper for cookie upload flow
|
2026-05-23 16:13:10 +03:00 |
|
lekss361
|
f6ece38619
|
feat(tradein): cian_detail.py — detail page scraper (88 fields + BTI + priceChanges, Wave 4 Worker B)
- fetch_detail(): curl_cffi AsyncSession → HTML → extract_state(mfe='frontend-offer-card', key='defaultState')
- DetailEnrichment dataclass: cian_id, windows_view_type, wcs counts, ceiling_height, repair_type, views_total/today, bti_data, price_changes, agent_profile, newbuilding_data, raw_offer/sister_states
- extract_all_states() sister container stash: bti (houseData), newObject, plus raw dump for future extraction
- save_detail_enrichment(): UPDATE listings (all 019 cols present), INSERT offer_price_history (ON CONFLICT DO NOTHING), UPSERT agents + link agent_id_fk, psycopg v3 CAST(:x AS type) everywhere
- Tests: 19 tests (unit helpers + async mocked fetch_detail: no-state, 403, core fields, BTI sister state, newbuilding link)
- Ruff clean, asyncio_mode=auto
|
2026-05-23 16:12:43 +03:00 |
|