- Create central project documentation with roadmap
- Add progress tracker with status tables
- Include architecture diagram and tech stack
- Add quick start guide and configuration reference
- Document recent changes and next steps
Co-Authored-By: Claude <noreply@anthropic.com>
- Add streaming mode for memory-efficient large catalog scraping
- Implement retry logic with exponential backoff
- Add auto session reinitialization on 403 errors
- Add configurable options (pageSize, maxProducts, rateLimitDelay)
- Add maxIterations protection against infinite loops
- Add retry.ts utility module with withRetry and withRetryAndReinit
- Update .env.example with new scraping options
- Add pgAdmin and CloudBeaver to docker-compose
Co-Authored-By: Claude <noreply@anthropic.com>
