3 Commits

Author	SHA1	Message	Date
Mc Smog	3299cca574	feat: add product detail enrichment for Magnit products ... - Add isDetailsFetched field to Product model - Add fetchProductDetails() and fetchProductObjectInfo() methods to MagnitApiScraper - Add ProductParser methods for detail parsing - Add ProductService methods: getProductsNeedingDetails(), updateProductDetails(), markAsDetailsFetched() - Add enrich-product-details.ts script with statistics tracking - Update package.json with "enrich" script command - Add E2E_GUIDE.md documentation - Exclude debug scripts from tsconfig type-check (temporary) Co-Authored-By: Claude <noreply@anthropic.com>	2026-01-22 01:52:50 +05:00
Mc Smog	9164527f58	feat: enhanced Magnit scraper with streaming mode and retry logic ... - Add streaming mode for memory-efficient large catalog scraping - Implement retry logic with exponential backoff - Add auto session reinitialization on 403 errors - Add configurable options (pageSize, maxProducts, rateLimitDelay) - Add maxIterations protection against infinite loops - Add retry.ts utility module with withRetry and withRetryAndReinit - Update .env.example with new scraping options - Add pgAdmin and CloudBeaver to docker-compose Co-Authored-By: Claude <noreply@anthropic.com>	2026-01-21 22:14:04 +05:00
Mc Smog	19c0426cdc	Initial commit: Supermarket scraper MVP	2025-12-28 23:29:30 +05:00