add AGENTS.md
This commit is contained in:
151
AGENTS.md
Normal file
151
AGENTS.md
Normal file
@@ -0,0 +1,151 @@
|
|||||||
|
# AGENTS.md
|
||||||
|
|
||||||
|
Guidelines for AI coding agents working on this repository.
|
||||||
|
|
||||||
|
## Project Overview
|
||||||
|
|
||||||
|
TypeScript-based scraper for Russian supermarkets (Magnit). Uses Playwright for sessions, Axios for API, PostgreSQL with Prisma ORM.
|
||||||
|
|
||||||
|
## Build & Run Commands
|
||||||
|
|
||||||
|
**Package Manager**: Use `pnpm` (not npm/yarn)
|
||||||
|
|
||||||
|
```bash
|
||||||
|
pnpm install # Install dependencies
|
||||||
|
pnpm exec playwright install chromium # Install browsers (once)
|
||||||
|
pnpm type-check # Type checking (validation)
|
||||||
|
pnpm build # Build TypeScript to dist/
|
||||||
|
pnpm dev # Run main scraper
|
||||||
|
pnpm enrich # Run product enrichment
|
||||||
|
pnpm test-db # Test database connection
|
||||||
|
```
|
||||||
|
|
||||||
|
### Prisma Commands
|
||||||
|
|
||||||
|
```bash
|
||||||
|
pnpm prisma:generate # Generate client after schema changes
|
||||||
|
pnpm prisma:migrate # Create and apply migrations
|
||||||
|
pnpm prisma:studio # Open database GUI
|
||||||
|
```
|
||||||
|
|
||||||
|
### Running Scripts Directly
|
||||||
|
|
||||||
|
```bash
|
||||||
|
tsx src/scripts/scrape-magnit-products.ts
|
||||||
|
MAGNIT_STORE_CODE=992301 tsx src/scripts/scrape-magnit-products.ts
|
||||||
|
```
|
||||||
|
|
||||||
|
## Testing
|
||||||
|
|
||||||
|
No test framework configured. Manual testing via `pnpm test-db`, `pnpm dev`, Prisma Studio.
|
||||||
|
|
||||||
|
## Code Style
|
||||||
|
|
||||||
|
### Imports
|
||||||
|
|
||||||
|
1. External packages first, then internal modules
|
||||||
|
2. **Always include `.js` extension** for local imports (ESM)
|
||||||
|
3. Use named imports from Prisma client
|
||||||
|
|
||||||
|
```typescript
|
||||||
|
import { chromium, Browser } from 'playwright';
|
||||||
|
import axios from 'axios';
|
||||||
|
import { Logger } from '../../../utils/logger.js';
|
||||||
|
import { PrismaClient } from '../../../../generated/prisma/client.js';
|
||||||
|
```
|
||||||
|
|
||||||
|
### Naming Conventions
|
||||||
|
|
||||||
|
| Type | Convention | Example |
|
||||||
|
|------|------------|---------|
|
||||||
|
| Classes/Interfaces | PascalCase | `MagnitApiScraper`, `CreateProductData` |
|
||||||
|
| Functions/variables | camelCase | `scrapeAllProducts`, `deviceId` |
|
||||||
|
| Constants | UPPER_SNAKE_CASE | `ACTUAL_API_PAGE_SIZE` |
|
||||||
|
| Class files | PascalCase | `MagnitApiScraper.ts` |
|
||||||
|
| Util files | camelCase | `logger.ts`, `errors.ts` |
|
||||||
|
|
||||||
|
### TypeScript Patterns
|
||||||
|
|
||||||
|
- **Strict mode** - all types explicit
|
||||||
|
- Interfaces for data, optional props with `?`, `readonly` for constants
|
||||||
|
|
||||||
|
```typescript
|
||||||
|
export interface MagnitScraperConfig {
|
||||||
|
storeCode: string;
|
||||||
|
headless?: boolean;
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
### Error Handling
|
||||||
|
|
||||||
|
Use custom error classes from `src/utils/errors.ts`:
|
||||||
|
- `ScraperError` - scraping failures
|
||||||
|
- `DatabaseError` - database operations
|
||||||
|
- `APIError` - HTTP/API failures (includes statusCode)
|
||||||
|
|
||||||
|
```typescript
|
||||||
|
try {
|
||||||
|
// operation
|
||||||
|
} catch (error) {
|
||||||
|
Logger.error('Ошибка операции:', error);
|
||||||
|
throw new APIError(
|
||||||
|
`Не удалось: ${error instanceof Error ? error.message : String(error)}`,
|
||||||
|
statusCode
|
||||||
|
);
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
### Logging
|
||||||
|
|
||||||
|
Use static `Logger` class from `src/utils/logger.ts`:
|
||||||
|
|
||||||
|
```typescript
|
||||||
|
Logger.info('Message'); // Always shown
|
||||||
|
Logger.error('Error:', error); // Always shown
|
||||||
|
Logger.debug('Debug'); // Only when DEBUG=true
|
||||||
|
```
|
||||||
|
|
||||||
|
### Async/Class Patterns
|
||||||
|
|
||||||
|
- All async methods return `Promise<T>` with explicit return types
|
||||||
|
- Class order: private props -> constructor -> public methods -> private methods
|
||||||
|
- Lifecycle: `initialize()` -> operations -> `close()`
|
||||||
|
|
||||||
|
### Services Pattern
|
||||||
|
|
||||||
|
- Services receive `PrismaClient` via constructor (DI)
|
||||||
|
- Use `getOrCreate` for idempotent operations
|
||||||
|
- Never call Prisma directly from scrapers
|
||||||
|
|
||||||
|
### Database Patterns
|
||||||
|
|
||||||
|
- Upsert via composite unique `(externalId, storeId)`
|
||||||
|
- Batch processing: 50 items per batch
|
||||||
|
- Prices: Float (rubles), converted from kopecks
|
||||||
|
|
||||||
|
### Comments
|
||||||
|
|
||||||
|
- JSDoc for public methods, inline comments in Russian
|
||||||
|
|
||||||
|
```typescript
|
||||||
|
/** Инициализация сессии через Playwright */
|
||||||
|
async initialize(): Promise<void> { }
|
||||||
|
```
|
||||||
|
|
||||||
|
## Cursor Rules
|
||||||
|
|
||||||
|
### Requestly API Tests (`.requestly-supermarket/**/*.json`)
|
||||||
|
|
||||||
|
- Use `rq.test()` for tests, `rq.expect()` for assertions
|
||||||
|
- Access response via `rq.response.body` (parse as JSON)
|
||||||
|
- Prices in kopecks (24999 = 249.99 rubles)
|
||||||
|
|
||||||
|
See `.cursor/rules/requestly-test-rules.mdc` for full docs.
|
||||||
|
|
||||||
|
## Environment Variables
|
||||||
|
|
||||||
|
```bash
|
||||||
|
DATABASE_URL=postgresql://user:password@localhost:5432/supermarket
|
||||||
|
MAGNIT_STORE_CODE=992301
|
||||||
|
DEBUG=true
|
||||||
|
```
|
||||||
Reference in New Issue
Block a user