fix(tradein): CIAN_REQUIRED_COOKIES — add real auth cookies (DMIR_AUTH + Cian session) #480

Merged

lekss361 merged 1 commit from feat/tradein-cian-cookies-real-list into main

2026-05-23 14:52:25 +00:00

lekss361 commented

2026-05-23 14:49:07 +00:00

Owner

Issue

Stage 4 PR #457 added CIAN_REQUIRED_COOKIES filter без real DevTools probe — list был guessed. Реальные cookies от logged-in cian.ru session (2026-05-23 dump) показывают что critical auth cookie — DMIR_AUTH (httpOnly, lax SameSite). Filter его dropped → upload flow не пройдёт verify_session() check.

Real cookies (DevTools dump)

Cookie	Was in filter?	Purpose
`DMIR_AUTH`	❌	Critical Cian auth token (httpOnly)
`_CIAN_GK`	❌	Cian session GUID
`_yasc`	❌	Yandex anti-spam
`_ym_visorc`	❌	Yandex visit code
`uxfb_card_satisfaction`	❌	UX widget state
`_ym_uid`	✅	Yandex Metrika UID
`_ym_d`	✅	Yandex Metrika date
`_ym_isad`	✅	Yandex ad block detection

Fix

Updated CIAN_REQUIRED_COOKIES set в cian_session.py:

ADD: DMIR_AUTH, _CIAN_GK, _yasc, _ym_visorc, uxfb_card_satisfaction
KEEP legacy guesses as backward-compat fallback (если future Cian deployment изменит names)

Final set: 21 entries (5 new real + 3 existing real + 13 legacy fallback).

Tests

17/17 pass (added test_required_cookies_includes_real_auth regression test)
Verifies все real DevTools cookies в filter set

Push verify

LOCAL == REMOTE = f7a1ba37

After merge → user может upload cookies через:

CLI: bash tradein-mvp/scripts/upload-cian-cookies.sh cookies.json
UI: /admin/scrape/cian-cookies page (PR coming separately)

Filter теперь корректно пропустит DMIR_AUTH → verify_session() hit на Valuation Calculator → user.isAuthenticated=true → cookies saved encrypted в DB.

Refs

PR #457 (Stage 4 — original cookie management)
PR #465 (Stage 7 — cian_valuation.py uses load_session())

Test plan

17 unit tests pass
Ruff clean
Post-deploy: user uploads real cookies → /admin/scrape/cian/test-auth returns {authenticated: true}

## Issue Stage 4 PR #457 added `CIAN_REQUIRED_COOKIES` filter без real DevTools probe — list был guessed. Реальные cookies от logged-in cian.ru session (2026-05-23 dump) показывают что critical auth cookie — **`DMIR_AUTH`** (httpOnly, lax SameSite). Filter его dropped → upload flow не пройдёт `verify_session()` check. ## Real cookies (DevTools dump) | Cookie | Was in filter? | Purpose | |---|---|---| | `DMIR_AUTH` | ❌ | **Critical Cian auth token (httpOnly)** | | `_CIAN_GK` | ❌ | Cian session GUID | | `_yasc` | ❌ | Yandex anti-spam | | `_ym_visorc` | ❌ | Yandex visit code | | `uxfb_card_satisfaction` | ❌ | UX widget state | | `_ym_uid` | ✅ | Yandex Metrika UID | | `_ym_d` | ✅ | Yandex Metrika date | | `_ym_isad` | ✅ | Yandex ad block detection | ## Fix Updated `CIAN_REQUIRED_COOKIES` set в `cian_session.py`: - **ADD**: `DMIR_AUTH`, `_CIAN_GK`, `_yasc`, `_ym_visorc`, `uxfb_card_satisfaction` - **KEEP** legacy guesses as backward-compat fallback (если future Cian deployment изменит names) Final set: 21 entries (5 new real + 3 existing real + 13 legacy fallback). ## Tests - 17/17 pass (added `test_required_cookies_includes_real_auth` regression test) - Verifies все real DevTools cookies в filter set ## Push verify LOCAL == REMOTE = `f7a1ba37` ## Next After merge → user может upload cookies через: - CLI: `bash tradein-mvp/scripts/upload-cian-cookies.sh cookies.json` - UI: `/admin/scrape/cian-cookies` page (PR coming separately) Filter теперь корректно пропустит `DMIR_AUTH` → `verify_session()` hit на Valuation Calculator → `user.isAuthenticated=true` → cookies saved encrypted в DB. ## Refs - PR #457 (Stage 4 — original cookie management) - PR #465 (Stage 7 — cian_valuation.py uses `load_session()`) ## Test plan - [x] 17 unit tests pass - [x] Ruff clean - [ ] Post-deploy: user uploads real cookies → `/admin/scrape/cian/test-auth` returns `{authenticated: true}`

lekss361 added 1 commit 2026-05-23 14:49:08 +00:00

fix(tradein): CIAN_REQUIRED_COOKIES — include real auth cookies (DMIR_AUTH, _CIAN_GK, _yasc, ...) f7a1ba37a8

Original list (Stage 4 PR) was guessed without real DevTools probe; dropped DMIR_AUTH
(main Cian auth token) and several observed cookies. Updated from logged-in cian.ru
session dump (2026-05-23). Old entries kept as backward-compat fallback.
Adds regression test asserting all real DevTools cookies are present in the filter.

lekss361 merged commit 99dd403c45 into main

2026-05-23 14:52:25 +00:00

lekss361 referenced this pull request from a commit

2026-05-23 14:52:26 +00:00

fix(tradein): CIAN_REQUIRED_COOKIES — add real auth cookies (DMIR_AUTH + Cian session) (#480)

lekss361 commented

2026-05-23 14:52:58 +00:00

Author

Owner

Merged via deep-code-reviewer — verdict APPROVE (MINOR).

Verification:

Cookie names case-sensitivity OK: DMIR_AUTH uppercase exact, _CIAN_GK underscore-prefixed, all RFC 6265 compliant.
Consumer admin.py:265 uses exact set membership (if k in CIAN_REQUIRED_COOKIES) — case-sensitive match confirmed.
Regression test test_required_cookies_includes_real_auth enforces all 8 DevTools cookies present.
Backward compat preserved (legacy 13 entries kept as fallback, total 21).
No security issue: cookies still encrypted via pgp_sym_encrypt, never logged raw (verify_session line 67 logs only mfe key, not values).
Deploy trigger correct: paths: tradein-mvp/** matches both changed files.

MINOR notes (non-blocking):

Vault: no decisions/Cian_Cookie_DevTools_Probe_May23.md yet — рекомендуется создать с procedure (Cookie-Editor extension required for httpOnly DMIR_AUTH) и snapshot текущего real cookies set.
Старый test test_required_cookies_contains_key_names оставляет only 5 legacy cookies — было бы чище объединить с новым regression test (отдельный PR).
Integration test для actual upload flow с mock httpx response для DMIR_AUTH path (отдельный PR).

Post-merge validation:

deploy-tradein.yml должен trigger (paths match).
CI pytest 17/17 green.
User uploads via PR #481 UI → /admin/scrape/cian/test-auth returns {authenticated: true, userId: <int>}.

Merged via deep-code-reviewer — verdict APPROVE (MINOR). **Verification**: - Cookie names case-sensitivity OK: `DMIR_AUTH` uppercase exact, `_CIAN_GK` underscore-prefixed, all RFC 6265 compliant. - Consumer `admin.py:265` uses exact set membership (`if k in CIAN_REQUIRED_COOKIES`) — case-sensitive match confirmed. - Regression test `test_required_cookies_includes_real_auth` enforces all 8 DevTools cookies present. - Backward compat preserved (legacy 13 entries kept as fallback, total 21). - No security issue: cookies still encrypted via `pgp_sym_encrypt`, never logged raw (verify_session line 67 logs only mfe key, not values). - Deploy trigger correct: `paths: tradein-mvp/**` matches both changed files. **MINOR notes (non-blocking)**: 1. Vault: no `decisions/Cian_Cookie_DevTools_Probe_May23.md` yet — рекомендуется создать с procedure (Cookie-Editor extension required for httpOnly DMIR_AUTH) и snapshot текущего real cookies set. 2. Старый test `test_required_cookies_contains_key_names` оставляет only 5 legacy cookies — было бы чище объединить с новым regression test (отдельный PR). 3. Integration test для actual upload flow с mock httpx response для DMIR_AUTH path (отдельный PR). **Post-merge validation**: 1. deploy-tradein.yml должен trigger (paths match). 2. CI pytest 17/17 green. 3. User uploads via PR #481 UI → `/admin/scrape/cian/test-auth` returns `{authenticated: true, userId: <int>}`.

lekss361 referenced this pull request

2026-05-23 14:54:20 +00:00

feat(tradein-ui): Cian cookies upload admin page #481