fix(observability): убрать 83% мусора из трекера ошибок #2906

Merged
lekss361 merged 3 commits from fix/tradein-glitchtip-noise into main 2026-08-15 17:24:03 +00:00

3 commits

Author SHA1 Message Date
bot-backend
349494a9df fix(tradein/observability): close RetryError-half of GlitchTip noise fix (round 2)
All checks were successful
CI Trade-In / changes (pull_request) Successful in 8s
CI / changes (pull_request) Successful in 9s
CI Trade-In / browser-tests (pull_request) Has been skipped
CI Trade-In / frontend-checks (pull_request) Has been skipped
CI / backend-tests (pull_request) Has been skipped
CI / frontend-tests (pull_request) Has been skipped
CI Trade-In / backend-tests (pull_request) Successful in 4m30s
CI / openapi-codegen-check (pull_request) Has been skipped
Ревью round 1 подтвердил basic_auth-часть, но нашёл 4 факта в RetryError-части:

1. reraise=True в geocoder.py не убирает шум, а переименовывает: наружу летит
   httpx.HTTPStatusError, чей str() содержит ПОЛНЫЙ request URL с query string
   (`for url '...search?q=<адрес>&...'`) — воспроизведено эмпирически. Тот же
   per-address issue-explosion, просто под другим типом исключения.
   Фикс: _HTTPX_ERROR_URL_QUERY_RE в sentry_scrub.scrub_pii_event режет query
   string из httpx-style "for url '...'" сообщений — стабилизирует ТЕКСТ, не
   только тип, независимо от того, уважает ли GlitchTip fingerprint-поле.

2. stabilize_retry_error_fingerprint затирал fingerprint целиком по (типу
   причины) — RetryError из НЕСВЯЗАННЫХ подсистем с одинаковым типом причины
   схлопнулись бы в один issue (geocoder vs scraper_kit оба ловят httpx-типы).
   Фикс: culprit = event["logger"] (LoggingIntegration ставит его = имя
   модуля-источника logger.exception) идёт первым компонентом fingerprint —
   разные подсистемы больше не сливаются.

3. Второй живой источник RetryError, пропущенный round 1 (грепали литерал
   "RetryError", не producers): BaseScraper._http_get в packages/scraper-kit —
   @retry БЕЗ reraise=True, живой путь через YandexDetailScraper.fetch_detail
   (yandex/serp.py и valuation.py переопределяют _http_get без retry — не
   затронуты). Оставлен на fingerprint-хук намеренно: detail-URL варьируются
   в ПУТИ (offer id), не в query — _HTTPX_ERROR_URL_QUERY_RE их не покрывает,
   а добавление reraise=True туда воспроизвело бы ту же проблему через
   HTTPStatusError с variable path вместо query.

4. type(exc).__name__ == "RetryError" (string-compare) → isinstance(exc,
   RetryError) с прямым импортом tenacity.RetryError — не матчит посторонние
   классы с тем же __name__, не промахивается мимо подклассов.

Полный backend suite (4479 passed, 21 skipped) + geocoder/scheduler/alerts
подмножества — без регрессий (reraise=True уже влит в main).

Не тронуто (вне scope round 2, подтверждено ревьюером как верное):
ops/glitchtip-auth-forwarder/* (basic_auth 401 дроп), массовая чистка
накопленных issue.
2026-08-15 20:02:44 +03:00
bot-backend
662720e1c8 Merge remote-tracking branch 'forgejo/main' into _cont-tradein-glitchtip-noise 2026-08-15 19:43:34 +03:00
bot-backend
8fcec9f12e fix(tradein/observability): stop basic_auth 401 and RetryError GlitchTip noise
83% of tracker issues (7460 total) were pure noise drowning real signal:
- basic_auth 401 (3738 issues, 2019 distinct titles) — ops/glitchtip-auth-
  forwarder sent EVERY 401 from bots scanning gendsgn.ru (GET /wp-admin/
  install.php etc.) as an individual GlitchTip event, remote_ip baked into
  message/tags inflated cardinality. Not an application error — expected
  bot-scan traffic against a basic_auth-protected site.
- RetryError (2462 issues) — geocoder.py's three tenacity @retry-wrapped
  Nominatim helpers (lookup/suggest/reverse) raised tenacity.RetryError on
  exhaustion without reraise=True; RetryError.__str__() embeds a Future
  repr() with a memory address that differs every call, so GlitchTip
  grouped each exhausted retry as a distinct issue instead of one.

Fix at the source, not post-hoc issue cleanup:
- forwarder.py: before_send drops events tagged event_type in
  {basic_auth_failed, basic_auth_storm}; forwarder's own capture_exception
  (real script bugs) carries no such tag and passes through untouched.
- geocoder.py: reraise=True on all three @retry decorators — propagates
  the real underlying exception (stable type + stacktrace) instead of the
  unstable RetryError wrapper.
- sentry_scrub.stabilize_retry_error_fingerprint: belt-and-suspenders
  before_send hook, composed into both app/main.py and scheduler_main.py
  (geocoder runs in both processes — FastAPI request path and the
  overnight geocode_missing_listings batch). Collapses any RetryError that
  still slips through into one persistent issue per cause-exception type
  name only — never IP/address/listing-id.

Content-ful categories (OperationalError, city-sweep, harvest_quarter,
cian/avito/yandex sweep failures, scrape_freshness_check — ~700 issues)
are untouched: filters key off event_type tag / exception type name only.
2026-08-15 18:08:04 +03:00