lekss361/gendesign
1
0
Fork 0
Code Issues 57 Pull requests Projects Releases Packages Wiki Activity Actions

fix(scrapers): narrow domrf Level-1 status badge-class regex (#1609 follow-up)
Some checks failed
CI / changes (push) Successful in 8s
Details
CI / changes (pull_request) Has been cancelled
Details
CI / backend-tests (pull_request) Has been cancelled
Details
CI / frontend-tests (pull_request) Has been cancelled
Details
CI / openapi-codegen-check (pull_request) Has been cancelled
Details
CI / backend-tests (push) Has been cancelled
Details
CI / frontend-tests (push) Has been cancelled
Details
CI / openapi-codegen-check (push) Has been cancelled
Details

Browse source 
_STATUS_BADGE_CLS_RE was too broad: `status|badge|tag|chip|label` matched
generic UI elements (e.g. <span class="label">, <span class="chip">) that
are unrelated to the sale-status badge, risking picking the wrong block.

Narrowed to `(?<![a-z])status(?![a-z])` — requires the literal token
"status" as a hyphen-delimited component of the CSS class (matches
`status-badge`, `flat-status-tag`, `object-status` but not bare `label`,
`tag`, `chip`, `badge`).

Adds regression test: page with generic label/tag/chip/badge elements
containing "В продаже" must NOT activate Level-1; only the real
`status-badge` block ("Продана") should be returned → status=sold.
This commit is contained in:
bot-backend 2026-06-17 21:21:56 +03:00
parent f55e83a150
commit c3c89646ce
2 changed files with 31 additions and 1 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

7
backend/app/services/scrapers/domrf_catalog.py
View file

@ -80,7 +80,12 @@ _STATUS_KW_RE = re.compile(
re.IGNORECASE | re.UNICODE,
)
# CSS-классы, характерные для статус-бейджей в Next.js/Tailwind.
_STATUS_BADGE_CLS_RE = re.compile(r"status|badge|tag|chip|label", re.IGNORECASE)
# Требуем слово «status» как отдельный токен внутри hyphen-separated CSS-класса
# (напр. «status-badge», «flat-status-tag», «object-status»).
# Предыдущий паттерн «status|badge|tag|chip|label» был слишком широк —
# срабатывал на любой generic элемент с классом «tag»/«chip»/«label»/«badge»,
# не имеющим отношения к статусу продажи (#1609 follow-up, #1686).
_STATUS_BADGE_CLS_RE = re.compile(r"(?<![a-z])status(?![a-z])", re.IGNORECASE)
def _classify_status_kw(matched_text: str) -> str | None:

25
backend/tests/services/scrapers/test_domrf_catalog_parse.py
View file

@ -211,6 +211,31 @@ def test_prodana_feminine_classified_as_sold() -> None:
assert result.get("status") == STATUS_SOLD
def test_generic_label_tag_not_treated_as_badge() -> None:
"""Элемент с классом 'label' или 'tag' (без слова 'status') НЕ должен
активировать Уровень 1 (#1686 follow-up: regex был слишком широк).
Страница: generic <span class="label"> с текстом 'В продаже' в шапке,
реальный статус-бейдж с классом 'status-badge' содержит 'Продана'.
Уровень 1 должен выбрать только блок status-badge, а не label-шапку.
"""
html = (
"<!doctype html><html><body>"
# generic label в шапке — НЕ должен активировать Level-1 badge match
'<span class="label">В продаже</span>'
'<span class="tag">Новостройка</span>'
'<span class="chip">ЕКБ</span>'
'<span class="badge">Акция</span>'
# настоящий статус-бейдж — должен быть подхвачен Level-1
'<div class="status-badge">Продана</div>'
"</body></html>"
)
result = parse_catalog_flat(html)
# Level-1 должен взять status-badge (Продана → sold),
# generic label/tag/chip/badge не должны перебить правильный результат
assert result.get("status") == STATUS_SOLD
def test_full_page_sold_flat_no_false_free() -> None:
"""Реалистичный HTML проданной квартиры: навигация 'в продаже', бейдж 'продано'."""
html = (