bot-backend
d6dca24ea2
Investigated with a real curl_cffi capture (HTTP 200, 46 cards): the Avito SERP lazy-renders — data-marker="item-date" appears only on the top above-the-fold cards (~10/46), so ~79% of cards have no date in the raw HTML curl_cffi receives. The marker wasn't renamed; the field is just absent. But every listing carries sortTimeStamp (epoch-ms) in the embedded per-item JSON (56 occurrences/page) — the line-266 'state empty' note was stale. Map item_id (= DOM data-item-id) -> sortTimeStamp -> date as the primary source, keep the DOM item-date + _parse_relative_date as fallback for the top cards. Verified on the real capture: 42/46 = 91% coverage (was ~21% DOM-only), clearing the >80% DoD bar. Adds a distilled real-structure fixture (avito_serp_sample.html) + unit test covering both the JSON (lazy card) and DOM (top card) paths. Refs #726
32 lines
2 KiB
HTML
32 lines
2 KiB
HTML
<!doctype html>
|
||
<!--
|
||
#726 minimal Avito SERP fixture (apartment search), distilled from a real
|
||
captured page (curl_cffi chrome120, HTTP 200, 2026-05-30). Exercises both date
|
||
paths of avito.py::_parse_html:
|
||
- card A (8043936560): LAZY card — no DOM data-marker="item-date" (как у ~79%
|
||
карточек ниже первого экрана); дата берётся из per-item JSON sortTimeStamp.
|
||
- card B (8163084615): TOP card — несёт DOM data-marker="item-date" (fallback path).
|
||
Embedded JSON фрагмент повторяет реальную форму: "id":<id> ... "sortTimeStamp":<ms>.
|
||
-->
|
||
<html lang="ru"><head><meta charset="utf-8"><title>Avito SERP fixture</title></head>
|
||
<body>
|
||
<div class="items-items">
|
||
<div data-marker="item" data-item-id="8043936560" id="i8043936560">
|
||
<a data-marker="item-title" href="/ekaterinburg/kvartiry/2-k._kvartira_54m_59et._8043936560">2-к. квартира, 54 м², 5/9 эт.</a>
|
||
<meta itemprop="price" content="5500000">
|
||
<div data-marker="item-line"><div data-marker="item-address"><p>ул. Малышева, 30</p></div></div>
|
||
</div>
|
||
<div data-marker="item" data-item-id="8163084615" id="i8163084615">
|
||
<a data-marker="item-title" href="/ekaterinburg/kvartiry/1-k._kvartira_38m_316et._8163084615">1-к. квартира, 38 м², 3/16 эт.</a>
|
||
<meta itemprop="price" content="4200000">
|
||
<div data-marker="item-line"><div data-marker="item-address"><p>ул. Куйбышева, 21</p></div></div>
|
||
<p data-marker="item-date">1 час назад</p>
|
||
</div>
|
||
</div>
|
||
<script>
|
||
window.__avito_state_fragment = {"catalog":{"items":[
|
||
{"id":8043936560,"category":{"id":24,"slug":"kvartiry"},"addressDetailed":{"locationName":"Екатеринбург"},"sortTimeStamp":1700000000000,"allowTimeStamp":1700000000000},
|
||
{"id":8163084615,"category":{"id":24,"slug":"kvartiry"},"addressDetailed":{"locationName":"Екатеринбург"},"sortTimeStamp":1700086400000,"allowTimeStamp":1700086400000}
|
||
]}};
|
||
</script>
|
||
</body></html>
|