Commit Graph

4 Commits

Author SHA1 Message Date
wasrusgen
c97b8dce3c parsers: skip sponsored/ad URLs (cpc/sponsored=1) — they expire in 2-3 hours
User reported clicking matrix prices led to 'Произошла ошибка!' on OZON home page.
Cause: parsers captured /product/?sponsored=1&cpc=Jtiito95... links that died after few hours.

Fix:
- ozon.py: skip href with 'sponsored=1', '/promo/', 'cpc='. Strip query string from final URL.
- yamarket.py: skip 'sponsored=1', 'cpc=', 'advUuid' (Я.Маркет sponsored marker)
- citilink.py: strip query string from final URL (defensive)

Now matrix links go to canonical product pages that don't expire.
2026-05-11 17:20:59 +03:00
wasrusgen
44281b1e07 citilink: dedup by product ID + filter Next.js placeholder images 2026-05-11 13:59:07 +03:00
wasrusgen
c5f662f53d citilink: rewrite parser to walk up from a[href*=/product/] (CSS-in-JS resistant) 2026-05-11 13:57:18 +03:00
wasrusgen
e8b487891f backend: working parsers — OZON + Citilink (DOM via Playwright) + WB
DIAGNOSTIC RESULTS:
- OZON: 19 product links via Playwright on naked VPS-IP ✓
- Citilink: 112 data-meta-name Snippets ✓
- Wildberries: JSON API works with delays ✓
- Я.Маркет, DNS: blocked by ASN (need residential proxy)

OZON PARSER:
- Pure Playwright DOM (composer-api dropped — was blocked)
- Selects a[href*='/product/'], walks up to card div, extracts title/price/img
- Filters fake 'titles' like Распродажа, Скидка

CITILINK PARSER (new):
- Selects [data-meta-name*='Snippet'] or ProductCard markers
- Multiple title selectors fallback chain
- Filters out non-product hits

PARSERS/__init__.py:
- DEFAULT_SOURCES = (ozon, citilink, wb) — all work without proxy
- Я.Маркет, DNS kept but not default — usable when residential proxy added

NEW ENDPOINT:
- GET /api/parse_citilink?q=...&limit=N
2026-05-11 13:53:07 +03:00