feat(scraper-kit): copy cian providers with protocol injection, strangler (#2133 cian) #2157

Merged

lekss361 merged 1 commit from feat/scraper-kit-provider-cian into main

2026-07-02 16:11:32 +00:00

lekss361 commented

2026-07-02 16:11:19 +00:00

Owner

Часть #2133 (Epic #2126, шаг D — cian). Strangler-copy — app/services/scrapers/cian* + app/services/cian_session.py НЕ тронуты, деплой инертен.

Скопировано в `scraper_kit/providers/cian/`

serp.py ← cian.py (CianScraper), detail.py, newbuilding.py, valuation.py, session.py ← cian_session.py
Additive в корне kit: house_type_normalizer.py (pure, нужен serp'у, в kit не было)
Additive в contracts.py: поле cookie_encryption_key: str в ScraperConfig (для session; Settings структурно удовлетворяет)

Развязка

settings → инжект ScraperConfig (+ delay_provider); glitchtip_dsn/cian_proxy_url/cookie_encryption_key из config
sentry_sdk → lazy import (не новый dep) — pyproject/lock НЕ тронуты
внутр. импорты → scraper_kit.* (base/browser_fetcher/cian_state_parser/price_brackets/repair_state_normalizer/snapshot_writer/house_type_normalizer)

Golden-parity ✅ (9 тестов)

SERP _parse_serp_html+_offer_to_lot (вторичка/новостройка/студия, model_dump идентичен), _format_address, extract_state (app≡kit), detail fetch_detail (asdict), valuation _parse_valuation_state, newbuilding экстракторы, strangler-guard.

Verify

боевые cian*/cian_session не тронуты; старые cian-тесты 172 passed
grep from app в providers/cian/ пусто; ruff clean

Часть #2133 (Epic #2126, шаг D — cian). **Strangler-copy** — `app/services/scrapers/cian*` + `app/services/cian_session.py` НЕ тронуты, деплой инертен. ## Скопировано в `scraper_kit/providers/cian/` - `serp.py` ← cian.py (`CianScraper`), `detail.py`, `newbuilding.py`, `valuation.py`, `session.py` ← cian_session.py - Additive в корне kit: `house_type_normalizer.py` (pure, нужен serp'у, в kit не было) - Additive в `contracts.py`: поле `cookie_encryption_key: str` в `ScraperConfig` (для session; `Settings` структурно удовлетворяет) ## Развязка - `settings` → инжект `ScraperConfig` (+ `delay_provider`); `glitchtip_dsn`/`cian_proxy_url`/`cookie_encryption_key` из config - `sentry_sdk` → lazy import (не новый dep) — pyproject/lock НЕ тронуты - внутр. импорты → `scraper_kit.*` (base/browser_fetcher/cian_state_parser/price_brackets/repair_state_normalizer/snapshot_writer/house_type_normalizer) ## Golden-parity ✅ (9 тестов) SERP `_parse_serp_html`+`_offer_to_lot` (вторичка/новостройка/студия, `model_dump` идентичен), `_format_address`, `extract_state` (app≡kit), detail `fetch_detail` (`asdict`), valuation `_parse_valuation_state`, newbuilding экстракторы, strangler-guard. ## Verify - боевые `cian*`/`cian_session` не тронуты; старые cian-тесты 172 passed - grep `from app` в providers/cian/ пусто; ruff clean

lekss361 added 1 commit 2026-07-02 16:11:20 +00:00