Commit Graph

3 Commits

Author SHA1 Message Date
wasrusgen
3ee5275ea0 backend: PROXY_STATIC_LIST support (manual proxies without API token)
- proxy_pool now loads from both PROXY_STATIC_LIST (env, comma-separated) and PROXY6_TOKEN (API)
- Static list has priority, merged with API list (dedup by URL)
- /api/proxy_status returns masked proxy URLs for diagnostic (passwords hidden)
- Supports formats: 'http://user:pass@host:port' or 'host:port' (assumed http://)
2026-05-11 13:03:29 +03:00
wasrusgen
82425dbd88 backend: Proxy6 pool + parsers WB / OZON / Я.Маркет / DNS
PROXY POOL (app/proxy_pool.py):
- Loads active proxies from Proxy6.net API every 10 min
- Random rotation per request via proxied_client(timeout, headers)
- Graceful fallback to direct HTTP if PROXY6_TOKEN not set
- Config: PROXY6_TOKEN env var

PARSERS (app/parsers/):
- dns.py — refactored to use proxy_pool with retry+rotation on Qrator block
- wb.py — Wildberries JSON API (search.wb.ru), retries on 429
- ozon.py — OZON composer-api JSON (widgetStates extraction)
- yamarket.py — Я.Маркет HTML + embedded JSON parser
- __init__.py — enrich_one() fans out to all sources, aggregates min/max prices, max rating, sum reviews
- enrich_models() — batch enrich for AI by_category output

NEW DIAGNOSTIC ENDPOINTS (main.py):
- GET /api/parse_wb?q=...&limit=N
- GET /api/parse_ozon?q=...&limit=N
- GET /api/parse_yamarket?q=...&limit=N
- GET /api/parse_all?q=... — fan-out + aggregate
- GET /api/proxy_status — pool diagnostics (count, token configured, age)

PODBOR (main.py):
- _enrich_ai_with_dns -> _enrich_ai_marketplaces (uses all sources)

DEPLOY: needs PROXY6_TOKEN in /opt/zov-tech/deploy/.env on VPS, then docker compose build + up -d backend
2026-05-11 12:18:04 +03:00
wasrusgen
0e5895bdc4 feat(infra): Python FastAPI backend + Docker compose for VPS deploy (GigaChat with Russian root CA) 2026-05-10 17:44:21 +03:00