style(tradein): уложить перенесённые строки в лимит 100 символов
All checks were successful
CI Trade-In / changes (pull_request) Successful in 8s
CI / changes (pull_request) Successful in 10s
CI Trade-In / browser-tests (pull_request) Has been skipped
CI Trade-In / frontend-checks (pull_request) Has been skipped
CI / backend-tests (pull_request) Has been skipped
CI / frontend-tests (pull_request) Has been skipped
CI / openapi-codegen-check (pull_request) Has been skipped
CI Trade-In / backend-tests (pull_request) Successful in 4m48s
All checks were successful
CI Trade-In / changes (pull_request) Successful in 8s
CI / changes (pull_request) Successful in 10s
CI Trade-In / browser-tests (pull_request) Has been skipped
CI Trade-In / frontend-checks (pull_request) Has been skipped
CI / backend-tests (pull_request) Has been skipped
CI / frontend-tests (pull_request) Has been skipped
CI / openapi-codegen-check (pull_request) Has been skipped
CI Trade-In / backend-tests (pull_request) Successful in 4m48s
Ruff E501 на трёх строках, которые удлинились от замены литерала на `DEFAULT_IMPERSONATE` в докстроках. Абзацы перевёрстаны целиком, а не разорваны по месту переполнения — рваный перенос читался бы как опечатка. Прогон: `ruff check app tests` — All checks passed. Refs #3148
This commit is contained in:
parent
cf3d4850e2
commit
dd3a79b475
4 changed files with 11 additions and 10 deletions
|
|
@ -2197,9 +2197,9 @@ async def scrape_yandex_address_backfill(
|
|||
"""T10: Backfill listings.address для Yandex-листингов без номера дома.
|
||||
|
||||
Yandex SERP-карточки содержат только улицу («улица Горького»). Этот endpoint
|
||||
фетчит detail-страницы через curl_cffi (kit-профиль `DEFAULT_IMPERSONATE`), извлекает полный адрес
|
||||
из <title> («Екатеринбург, улица Горького, 36 — id …»), обновляет
|
||||
listings.address и сбрасывает geocode_tried_at для перегеокодирования.
|
||||
фетчит detail-страницы через curl_cffi (kit-профиль `DEFAULT_IMPERSONATE`),
|
||||
извлекает полный адрес из <title> («Екатеринбург, улица Горького, 36 — id …»),
|
||||
обновляет listings.address и сбрасывает geocode_tried_at для перегеокодирования.
|
||||
|
||||
limit: сколько листингов обработать за запуск (default 200).
|
||||
request_delay_sec: пауза между запросами (default 3.0s).
|
||||
|
|
|
|||
|
|
@ -5,8 +5,8 @@ The offer detail page <title> has the full address including house number:
|
|||
«Екатеринбург, улица Горького, 36 — id 12345».
|
||||
|
||||
This service finds active yandex listings without a house number in `address`,
|
||||
fetches each detail page via curl_cffi (kit-профиль `DEFAULT_IMPERSONATE`), extracts the full address
|
||||
from the <title> regex, and UPDATE listings.address.
|
||||
fetches each detail page via curl_cffi (kit-профиль `DEFAULT_IMPERSONATE`),
|
||||
extracts the full address from the <title> regex, and UPDATE listings.address.
|
||||
|
||||
Deliberately NOT wired into scheduler — manual trigger only.
|
||||
Triggered via POST /api/v1/admin/scrape/yandex-address-backfill.
|
||||
|
|
|
|||
|
|
@ -7,9 +7,9 @@ page <title> contains the full address including house number:
|
|||
|
||||
This task selects active listings that match the predicate (source='yandex', region_code=66,
|
||||
address IS NOT NULL, address has no house number, source_url IS NOT NULL), fetches each
|
||||
detail page via curl_cffi (kit-профиль `DEFAULT_IMPERSONATE`), extracts the full address
|
||||
from the <title>, and
|
||||
UPDATE listings.address ONLY when the extracted value differs from the current one.
|
||||
detail page via curl_cffi (kit-профиль `DEFAULT_IMPERSONATE`), extracts the full
|
||||
address from the <title>, and UPDATE listings.address ONLY when the extracted
|
||||
value differs from the current one.
|
||||
|
||||
Wired into the in-app scheduler as source='yandex_address_backfill'. Triggered nightly via
|
||||
the scrape_schedules seed (migration 091). Manual trigger also available via POST
|
||||
|
|
|
|||
|
|
@ -96,8 +96,9 @@ OFFER_URL_PATTERN = "/offer/[0-9]+"
|
|||
# него собирается — это инвариант #2235 (`_canonical_source_url` в
|
||||
# scraper_kit/providers/yandex/serp.py) и та же формула, которой миграция 164
|
||||
# чинила легаси-строки. Живая проба 2026-08-12 прод-трактом (тот же прокси,
|
||||
# curl_cffi с kit-профилем (на момент замера — Chrome 120), тот же parse): 6 из 6 — HTTP 200 и parse OK, включая
|
||||
# строки, чей сохранённый source_url — рекламный редирект na100.pro/go.php.
|
||||
# curl_cffi с kit-профилем (на момент замера — Chrome 120), тот же parse): 6 из 6
|
||||
# — HTTP 200 и parse OK, включая строки, чей сохранённый source_url — рекламный
|
||||
# редирект na100.pro/go.php.
|
||||
#
|
||||
# Откуда взялся стухший адрес: source_url пишется ТОЛЬКО при вставке — его нет ни
|
||||
# в `ON CONFLICT DO UPDATE`, ни в reconcile-UPDATE у `save_listings`. Значит #2235
|
||||
|
|
|
|||
Loading…
Add table
Reference in a new issue