lekss361/gendesign

Fork 0

feat(tradein): Stage 4c — scrapers admin pages (Avito real + Cian/Yandex stubs) #472

Merged

lekss361 merged 1 commit from feat/tradein-frontend-scrapers-admin into main

2026-05-23 14:05:57 +00:00

lekss361 commented

2026-05-23 14:02:20 +00:00

Owner

Summary

Stage 4c follow-up — admin UI для AvitoScraper v2 endpoints (PR #460 Stage 4a). По пользовательскому запросу: top nav + admin page для триггеров скрапера + stub'ы для Cian/Yandex.

LOC: 6 files (4 new + 2 modify).

Changes

NEW `components/trade-in/Topbar.tsx`

Shared header с 6 nav items: Оценка · История · Кэш · Авито · Циан · Яндекс. Активная вкладка — по active prop.

NEW `/scrapers/avito` — full admin UI

4 trigger секции (по pattern frontend/src/app/admin/scrape/objective/page.tsx):

#	Section	Endpoint	Form fields
1	Search around point	`POST /admin/scrape`	lat, lon, radius_m, sources=[avito]
2	Houses Catalog enrichment	`POST /admin/scrape/avito-house`	house_url
3	Detail page enrichment	`POST /admin/scrape/avito-detail`	item_url
4	IMV evaluation debug	`POST /admin/scrape/avito-imv`	address, rooms, area_m2, floor, total_floors, house_type, renovation_type, has_balcony, has_loggia

Каждая section: TanStack useMutation + JSON ResultPanel (success / error / loading states).

NEW `/scrapers/cian` + `/scrapers/yandex` — stub pages

Placeholders с пояснением: "В разработке, пока используйте cron-scrape.sh или curl". Зайти на них из nav можно — не 404.

UPDATE `app/page.tsx`

Inline <header className="topbar"> (15 lines) заменён на <Topbar active="estimate" />. Убран import API_BASE_URL (теперь internal в Topbar).

CSS append `trade-in.css`

.scraper-page, .scraper-section, .scraper-form, .scraper-result — responsive grid layout, focus styles, error variant.

Verify

✅ tsc --noEmit clean
✅ next build pass — 9 routes (3 новых: /scrapers/avito, /scrapers/cian, /scrapers/yandex)
⏭ npm run lint skip — pre-existing no-config issue (out of scope)

Test plan

tsc strict
next build
Deploy → visit /trade-in/scrapers/avito → submit search form для ЕКБ anchor → verify JSON response с counters
Submit Houses Catalog form (/catalog/houses/ekaterinburg/ul_akademika_postovskogo_17a/3171365) → verify house_id + reviews counters
Submit IMV form для test address → verify recommended_price + range
Verify Cian/Yandex stubs не падают (просто placeholder)
Verify Topbar активный tab подсвечивается на каждой странице

Notes

Backend endpoints (Stage 4a, PR #460) live, защищены require_admin. В prod нужен X-Admin-Token — но per recent PR #437 он dropped, теперь Caddy basic_auth достаточно. Frontend полагается на browser session creds (basic_auth handled by Caddy).
Stub-страницы Cian/Yandex намеренно minimal — функциональность будет добавлена в следующих stages.

Refs

Plan: AvitoScraper_v2_Implementation_Plan Stage 4 (extends 4a/4b with admin UI per user feedback 2026-05-23)
Backend contract: PR #460 (Stage 4a — 3 avito-* endpoints + 1 legacy /scrape)
Reference pattern: frontend/src/app/admin/scrape/objective/page.tsx

## Summary **Stage 4c follow-up** — admin UI для AvitoScraper v2 endpoints (PR #460 Stage 4a). По пользовательскому запросу: top nav + admin page для триггеров скрапера + stub'ы для Cian/Yandex. LOC: 6 files (4 new + 2 modify). ## Changes ### NEW `components/trade-in/Topbar.tsx` Shared header с 6 nav items: **Оценка · История · Кэш · Авито · Циан · Яндекс**. Активная вкладка — по `active` prop. ### NEW `/scrapers/avito` — full admin UI 4 trigger секции (по pattern `frontend/src/app/admin/scrape/objective/page.tsx`): | # | Section | Endpoint | Form fields | |---|---|---|---| | 1 | Search around point | `POST /admin/scrape` | lat, lon, radius_m, sources=[avito] | | 2 | Houses Catalog enrichment | `POST /admin/scrape/avito-house` | house_url | | 3 | Detail page enrichment | `POST /admin/scrape/avito-detail` | item_url | | 4 | IMV evaluation debug | `POST /admin/scrape/avito-imv` | address, rooms, area_m2, floor, total_floors, house_type, renovation_type, has_balcony, has_loggia | Каждая section: TanStack `useMutation` + JSON `ResultPanel` (success / error / loading states). ### NEW `/scrapers/cian` + `/scrapers/yandex` — stub pages Placeholders с пояснением: "В разработке, пока используйте cron-scrape.sh или curl". Зайти на них из nav можно — не 404. ### UPDATE `app/page.tsx` Inline `<header className="topbar">` (15 lines) заменён на `<Topbar active="estimate" />`. Убран import `API_BASE_URL` (теперь internal в Topbar). ### CSS append `trade-in.css` `.scraper-page`, `.scraper-section`, `.scraper-form`, `.scraper-result` — responsive grid layout, focus styles, error variant. ## Verify - ✅ `tsc --noEmit` clean - ✅ `next build` pass — 9 routes (3 новых: `/scrapers/avito`, `/scrapers/cian`, `/scrapers/yandex`) - ⏭ `npm run lint` skip — pre-existing no-config issue (out of scope) ## Test plan - [x] tsc strict - [x] next build - [ ] Deploy → visit `/trade-in/scrapers/avito` → submit search form для ЕКБ anchor → verify JSON response с counters - [ ] Submit Houses Catalog form (`/catalog/houses/ekaterinburg/ul_akademika_postovskogo_17a/3171365`) → verify house_id + reviews counters - [ ] Submit IMV form для test address → verify recommended_price + range - [ ] Verify Cian/Yandex stubs не падают (просто placeholder) - [ ] Verify Topbar активный tab подсвечивается на каждой странице ## Notes - Backend endpoints (Stage 4a, PR #460) live, защищены `require_admin`. В prod нужен `X-Admin-Token` — но per recent PR #437 он dropped, теперь Caddy basic_auth достаточно. Frontend полагается на browser session creds (basic_auth handled by Caddy). - Stub-страницы Cian/Yandex намеренно minimal — функциональность будет добавлена в следующих stages. ## Refs - Plan: AvitoScraper_v2_Implementation_Plan Stage 4 (extends 4a/4b with admin UI per user feedback 2026-05-23) - Backend contract: PR #460 (Stage 4a — 3 avito-* endpoints + 1 legacy /scrape) - Reference pattern: `frontend/src/app/admin/scrape/objective/page.tsx`

lekss361 added 1 commit 2026-05-23 14:02:20 +00:00

feat(tradein): Stage 4c — scrapers admin pages (Avito real + Cian/Yandex stubs) af3da0f02e

- NEW Topbar component (shared header с 6 nav items: Оценка/История/Кэш/Авито/Циан/Яндекс)
- NEW /scrapers/avito — 4 trigger forms по pattern frontend/admin/scrape/objective:
  * Search around point (lat/lon/radius) → POST /admin/scrape
  * Houses Catalog enrichment (house_url) → POST /admin/scrape/avito-house
  * Detail page enrichment (item_url) → POST /admin/scrape/avito-detail
  * IMV evaluation debug (9 params) → POST /admin/scrape/avito-imv
  TanStack useMutation; JSON response panel; loading/error states
- NEW /scrapers/cian + /scrapers/yandex — stub pages с pointer на cron/curl
- UPDATE page.tsx — replace inline topbar with <Topbar active="estimate" />
- CSS: .scraper-page, .scraper-section, .scraper-form, .scraper-result

Бэкенд endpoints из PR #460 (Stage 4a). tsc + next build pass.

lekss361 merged commit 649034479f into main

2026-05-23 14:05:57 +00:00

lekss361 referenced this pull request from a commit

2026-05-23 14:05:57 +00:00

feat(tradein): Stage 4c — scrapers admin pages (Avito real + Cian/Yandex stubs) (#472)

lekss361 commented

2026-05-23 14:06:23 +00:00

Author

Owner

Merged via deep-code-reviewer — verdict APPROVE (squash → 6490344).

Verified

Backend contract sync with PR #460 admin.py — все 4 endpoints (/scrape, /scrape/avito-house?house_url=, /scrape/avito-detail?item_url=, /scrape/avito-imv?…) signature-matched. Frontend correctly uses floor_at_home (PR description опечатка total_floors — код правильный).
Auth strategy: frontend никогда не шлёт X-Admin-Token; apiFetch добавляет только X-Session-Id + Content-Type. Caddy basic_auth (per #437) handles auth — browser session creds passed automatically. В dev settings.admin_token=None → endpoints open, acceptable.
TS strict: no any/@ts-ignore. useMutation<TData,Error,TVariables> generics explicit. 4 mutation states isolated — concurrent submission OK.
URL encoding: encodeURIComponent(house_url) + URLSearchParams для IMV — без manual concat.
CSS: новые .scraper-* classes без collisions; tokens с fallback var(--bg-card, #ffffff) graceful.
Topbar reuses existing .topbar/.top-nav/.is-active styles из старого header — visual continuity.

Minor follow-up (non-blocking, отдельным PR)

Topbar.tsx: добавить aria-current={item.key === active ? "page" : undefined} для screen-reader a11y.
avito/page.tsx: number inputs (lat/lon/area/rooms/floor) лучше с type="number" step="any" — parseFloat("abc") = NaN сейчас полагается на backend 422.
ResultPanel: добавить isPending rendering — long-running IMV (60+ sec) не имеет визуального feedback в panel, только disabled button.
cian/page.tsx + yandex/page.tsx: inline style={{ color: "var(--muted, …) }} дублирует .scraper-hint class — refactor.
Nav links на /history + /cache — pages не вижу в app/ (но это pre-existing, не regression этого PR).

Risk: Low — frontend-only, no backend/DB/auth changes. Easy revert.

Merged via deep-code-reviewer — verdict APPROVE (squash → 6490344). **Verified** - Backend contract sync with PR #460 admin.py — все 4 endpoints (`/scrape`, `/scrape/avito-house?house_url=`, `/scrape/avito-detail?item_url=`, `/scrape/avito-imv?…`) signature-matched. Frontend correctly uses `floor_at_home` (PR description опечатка `total_floors` — код правильный). - Auth strategy: frontend никогда не шлёт `X-Admin-Token`; `apiFetch` добавляет только `X-Session-Id` + `Content-Type`. Caddy basic_auth (per #437) handles auth — browser session creds passed automatically. В dev `settings.admin_token=None` → endpoints open, acceptable. - TS strict: no `any`/`@ts-ignore`. `useMutation<TData,Error,TVariables>` generics explicit. 4 mutation states isolated — concurrent submission OK. - URL encoding: `encodeURIComponent(house_url)` + `URLSearchParams` для IMV — без manual concat. - CSS: новые `.scraper-*` classes без collisions; tokens с fallback `var(--bg-card, #ffffff)` graceful. - Topbar reuses existing `.topbar/.top-nav/.is-active` styles из старого header — visual continuity. **Minor follow-up (non-blocking, отдельным PR)** - `Topbar.tsx`: добавить `aria-current={item.key === active ? "page" : undefined}` для screen-reader a11y. - `avito/page.tsx`: number inputs (lat/lon/area/rooms/floor) лучше с `type="number" step="any"` — `parseFloat("abc") = NaN` сейчас полагается на backend 422. - `ResultPanel`: добавить `isPending` rendering — long-running IMV (60+ sec) не имеет визуального feedback в panel, только disabled button. - `cian/page.tsx` + `yandex/page.tsx`: inline `style={{ color: "var(--muted, …) }}` дублирует `.scraper-hint` class — refactor. - Nav links на `/history` + `/cache` — pages не вижу в `app/` (но это pre-existing, не regression этого PR). **Risk**: Low — frontend-only, no backend/DB/auth changes. Easy revert.

lekss361 referenced this pull request

2026-05-23 14:09:34 +00:00

feat(tradein): cross-source matching service (3-tier: cadastr / fingerprint / geo / composite) #470

lekss361 referenced this pull request from a commit

2026-05-23 14:16:09 +00:00

fix(tradein): убрать require_admin полностью — Caddy basic_auth достаточен

lekss361 referenced this pull request

2026-05-23 14:17:24 +00:00

fix(tradein): убрать require_admin полностью — Caddy basic_auth достаточен #474