- Fix relative imports in experiments/ scripts (../ → ../../)
- Clean up tsconfig.json exclude list (remove non-existent paths)
- All debug scripts now work from their new location
Co-Authored-By: Claude <noreply@anthropic.com>
- Move debug/test scripts from src/scripts/ to experiments/
- Remove test-detail-endpoint from package.json
- Delete temp-product-page.html
- Move E2E_GUIDE.md to docs/
- Add experiments/README.md with documentation
- Keep only production scripts in src/scripts/
- Clean up tsconfig.json exclude list (experiments are now outside src/)
Co-Authored-By: Claude <noreply@anthropic.com>
- Add postgres-mcp service to docker-compose.yml (SSE mode on port 8000)
- Add .mcp.json.example with SSE configuration template
- Add .gitignore entries for .claude/settings.local.json and .mcp.json
- Add MCP_EXAMPLES.md with query examples for testing scraping results
- Add analysis scripts: analyze-category-nulls.ts, check-product-details.ts,
inspect-api-response.ts
Co-Authored-By: Claude <noreply@anthropic.com>
- Create central project documentation with roadmap
- Add progress tracker with status tables
- Include architecture diagram and tech stack
- Add quick start guide and configuration reference
- Document recent changes and next steps
Co-Authored-By: Claude <noreply@anthropic.com>
- Add streaming mode for memory-efficient large catalog scraping
- Implement retry logic with exponential backoff
- Add auto session reinitialization on 403 errors
- Add configurable options (pageSize, maxProducts, rateLimitDelay)
- Add maxIterations protection against infinite loops
- Add retry.ts utility module with withRetry and withRetryAndReinit
- Update .env.example with new scraping options
- Add pgAdmin and CloudBeaver to docker-compose
Co-Authored-By: Claude <noreply@anthropic.com>
