- Add postgres-mcp service to docker-compose.yml (SSE mode on port 8000)
- Add .mcp.json.example with SSE configuration template
- Add .gitignore entries for .claude/settings.local.json and .mcp.json
- Add MCP_EXAMPLES.md with query examples for testing scraping results
- Add analysis scripts: analyze-category-nulls.ts, check-product-details.ts,
inspect-api-response.ts
Co-Authored-By: Claude <noreply@anthropic.com>
- Create central project documentation with roadmap
- Add progress tracker with status tables
- Include architecture diagram and tech stack
- Add quick start guide and configuration reference
- Document recent changes and next steps
Co-Authored-By: Claude <noreply@anthropic.com>
- Add streaming mode for memory-efficient large catalog scraping
- Implement retry logic with exponential backoff
- Add auto session reinitialization on 403 errors
- Add configurable options (pageSize, maxProducts, rateLimitDelay)
- Add maxIterations protection against infinite loops
- Add retry.ts utility module with withRetry and withRetryAndReinit
- Update .env.example with new scraping options
- Add pgAdmin and CloudBeaver to docker-compose
Co-Authored-By: Claude <noreply@anthropic.com>
