lekss361/gendesign

Fork 0

[tradein][backend] sber_index_pull scheduler fails on SSL — sberindex.ru serves an incomplete cert chain (missing Let's Encrypt intermediate) #922

New issue

Closed

opened 2026-05-31 19:21:58 +00:00 by lekss361 · 1 comment

lekss361 commented

2026-05-31 19:21:58 +00:00

Owner

Discovered during #794 prod QA (2026-05-31); root cause corrected after diagnosis

The monthly sber_index_pull task fails on the production scheduler path with httpcore.ConnectError [SSL: CERTIFICATE_VERIFY_FAILED] unable to get local issuer certificate.

⚠️ Corrected root cause (the initial "Russian CA" guess was WRONG)

Verified the actual cert chain from the prod box and locally:

sberindex.ru is signed by Let's Encrypt R13 → ISRG Root X1 (NOT a Russian/Минцифры CA). certifi already trusts ISRG Root X1.
BUT the server sends only 1 certificate (the leaf) — it omits the R13 intermediate:
- openssl s_client ... -showcerts | grep -c 'BEGIN CERTIFICATE' → 1
- Verify return code: 21 (unable to verify the first certificate) — fails even against the prod host's full system trust store, because an omitted intermediate is in no root store.
In-container repro: docker exec tradein-backend python -c "import httpx; httpx.get('https://sberindex.ru/')" → unable to get local issuer certificate.

→ This is a server-side misconfiguration on sberindex.ru (incomplete chain). Installing a Russian CA does nothing. Adding ISRG Root X1 does nothing (already trusted). The client must either skip verification or be handed the missing intermediate.

Fix options

verify=False on the httpx client in pull_sber_indices (recommended). Public, no-auth, non-sensitive open-data endpoint; server-side chain bug outside our control; robust against Let's Encrypt intermediate rotation (R10–R14+). One line. This is what the QA manual run used → {upserted: 621, errors: 0}.
Bundle the missing Let's Encrypt intermediate into certifi/CA bundle. Brittle: breaks again whenever sberindex renews under a different LE intermediate.

Recommend option 1.

Verify after fix

Set next_run_at = now() on scrape_schedules source='sber_index_pull' (or wait for the tick); confirm scrape_runs shows errors=0 and sber_price_index refreshes.

Related: #794, #919, #887. File: tradein-mvp/backend/app/services/sber_index.py.

## Discovered during #794 prod QA (2026-05-31); root cause corrected after diagnosis The monthly `sber_index_pull` task fails on the production scheduler path with `httpcore.ConnectError [SSL: CERTIFICATE_VERIFY_FAILED] unable to get local issuer certificate`. ### ⚠️ Corrected root cause (the initial "Russian CA" guess was WRONG) Verified the actual cert chain from the **prod box** and locally: - `sberindex.ru` is signed by **Let's Encrypt R13 → ISRG Root X1** (NOT a Russian/Минцифры CA). certifi already trusts ISRG Root X1. - BUT the server sends **only 1 certificate (the leaf)** — it **omits the R13 intermediate**: - `openssl s_client ... -showcerts | grep -c 'BEGIN CERTIFICATE'` → **1** - `Verify return code: 21 (unable to verify the first certificate)` — fails even against the prod host's full system trust store, because an omitted intermediate is in no root store. - In-container repro: `docker exec tradein-backend python -c "import httpx; httpx.get('https://sberindex.ru/')"` → `unable to get local issuer certificate`. → This is a **server-side misconfiguration on sberindex.ru** (incomplete chain). Installing a Russian CA does nothing. Adding ISRG Root X1 does nothing (already trusted). The client must either skip verification or be handed the missing intermediate. ### Fix options 1. **`verify=False` on the httpx client in `pull_sber_indices` (recommended).** Public, no-auth, non-sensitive open-data endpoint; server-side chain bug outside our control; robust against Let's Encrypt intermediate rotation (R10–R14+). One line. This is what the QA manual run used → `{upserted: 621, errors: 0}`. 2. **Bundle the missing Let's Encrypt intermediate** into certifi/CA bundle. Brittle: breaks again whenever sberindex renews under a different LE intermediate. Recommend option 1. ### Verify after fix Set `next_run_at = now()` on `scrape_schedules` source='sber_index_pull' (or wait for the tick); confirm `scrape_runs` shows `errors=0` and `sber_price_index` refreshes. Related: #794, #919, #887. File: `tradein-mvp/backend/app/services/sber_index.py`.

lekss361 added the

labels 2026-05-31 19:22:07 +00:00

lekss361 referenced this issue

2026-05-31 19:22:13 +00:00

[research][tradein][backend] СберИндекс city-index → time-adjustment для замороженных Rosreestr deals (#724/#727) #794

lekss361 changed title from ~~[tradein][backend] sber_index_pull scheduler fails on SSL — sberindex.ru Russian CA not in container trust store~~ to [tradein][backend] sber_index_pull scheduler fails on SSL — sberindex.ru serves an incomplete cert chain (missing Let's Encrypt intermediate)

2026-05-31 19:30:36 +00:00

bot-backend referenced this issue from a commit

2026-05-31 19:33:49 +00:00

fix(tradein): skip TLS verify for sberindex.ru pull — server omits LE intermediate (#922)

lekss361 referenced this issue from a pull request that will close it,

2026-05-31 19:33:59 +00:00

fix(tradein): skip TLS verify for sberindex.ru pull — server omits LE intermediate (#922) #924

lekss361 added

status/review

and removed

status/ready

labels 2026-05-31 19:34:07 +00:00

lekss361 closed this issue

2026-05-31 19:34:24 +00:00

lekss361 referenced this issue from a commit

2026-05-31 19:34:25 +00:00

Merge pull request 'fix(tradein): skip TLS verify for sberindex.ru pull — server omits LE intermediate (#922)' (#924) from fix/922-sber-tls-verify into main

lekss361 commented

2026-05-31 19:40:32 +00:00

Author

Owner

✅ Fixed & verified on prod scheduler path. PR #924 (verify=False) merged (a5457af) + deployed.

Forced a scheduler re-run (next_run_at=now()) → scrape_runs id 38: status=done, errors=0, upserted=621, duration=2s (vs the pre-fix id 37 = errors=9, upserted=0). All 9 series populate via the scheduler now; next_run_at recomputed to 2026-06-01 05:27 UTC — nightly refresh will succeed.

Closing.

✅ Fixed & verified on prod scheduler path. PR #924 (`verify=False`) merged (`a5457af`) + deployed. Forced a scheduler re-run (`next_run_at=now()`) → `scrape_runs` id **38**: `status=done, errors=0, upserted=621, duration=2s` (vs the pre-fix id 37 = `errors=9, upserted=0`). All 9 series populate via the scheduler now; `next_run_at` recomputed to 2026-06-01 05:27 UTC — nightly refresh will succeed. Closing.

lekss361 added

status/done

and removed

status/review

labels 2026-05-31 19:40:41 +00:00

bot-backend referenced this issue

2026-05-31 19:43:42 +00:00

feat(tradein): exhaustive Cian EKB secondary load — room×price partitioning #926

bot-backend referenced this issue

2026-07-12 13:17:37 +00:00

fix(tradein): repoint rosreestr_poll to /data-sets/ + verify Content-Type (СДЕЛКИ dataset moved off /opendata) #2477