mozempk/insta-recipe
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

simplify

Browse Source
This commit is contained in:
Giancarmine Salucci
2026-02-18 01:21:44 +01:00
parent 54321fd7c9
commit 49bccf8f15
84 changed files with 14474 additions and 13925 deletions
Download Patch File Download Diff File Expand all files Collapse all files

5
.prettierrc
View File

@@ -3,10 +3,7 @@
"singleQuote": true, "singleQuote": true,
"trailingComma": "none", "trailingComma": "none",
"printWidth": 100, "printWidth": 100,
"plugins": [ "plugins": ["prettier-plugin-svelte", "prettier-plugin-tailwindcss"],
"prettier-plugin-svelte",
"prettier-plugin-tailwindcss"
],
"overrides": [ "overrides": [
{ {
"files": "*.svelte", "files": "*.svelte",

40
README.md
View File

@@ -5,6 +5,7 @@ A modern web application that extracts recipes from Instagram posts and saves th
## 🚀 Features ## 🚀 Features
### Core Functionality ### Core Functionality
- **Async Queue Processing**: Fire-and-forget recipe extraction with background processing - **Async Queue Processing**: Fire-and-forget recipe extraction with background processing
- **Real-time Updates**: Server-Sent Events for live progress tracking - **Real-time Updates**: Server-Sent Events for live progress tracking
- **Push Notifications**: Background notifications when recipes complete - **Push Notifications**: Background notifications when recipes complete
@@ -13,6 +14,7 @@ A modern web application that extracts recipes from Instagram posts and saves th
- **PWA Support**: Installable Progressive Web App with offline capabilities - **PWA Support**: Installable Progressive Web App with offline capabilities
### User Experience ### User Experience
- **Queue Dashboard**: Monitor all recipe extractions in real-time - **Queue Dashboard**: Monitor all recipe extractions in real-time
- **Share Integration**: Browser share target for easy URL submission - **Share Integration**: Browser share target for easy URL submission
- **Responsive Design**: Works on desktop, tablet, and mobile - **Responsive Design**: Works on desktop, tablet, and mobile
@@ -20,6 +22,7 @@ A modern web application that extracts recipes from Instagram posts and saves th
- **Progress Tracking**: Visual progress through extraction phases - **Progress Tracking**: Visual progress through extraction phases
### Technical Architecture ### Technical Architecture
- **SvelteKit Frontend**: Modern reactive UI with TypeScript - **SvelteKit Frontend**: Modern reactive UI with TypeScript
- **Hexagonal Architecture**: Clean separation of concerns - **Hexagonal Architecture**: Clean separation of concerns
- **In-Memory Queue**: High-performance processing with configurable concurrency - **In-Memory Queue**: High-performance processing with configurable concurrency
@@ -29,6 +32,7 @@ A modern web application that extracts recipes from Instagram posts and saves th
## 📋 API Endpoints ## 📋 API Endpoints
### Queue Management ### Queue Management
- `POST /api/queue` - Enqueue Instagram URL for processing - `POST /api/queue` - Enqueue Instagram URL for processing
- `GET /api/queue` - List queue items with filtering and pagination - `GET /api/queue` - List queue items with filtering and pagination
- `GET /api/queue/{id}` - Get specific queue item details - `GET /api/queue/{id}` - Get specific queue item details
@@ -36,18 +40,21 @@ A modern web application that extracts recipes from Instagram posts and saves th
- `GET /api/queue/stream` - Server-Sent Events for real-time updates - `GET /api/queue/stream` - Server-Sent Events for real-time updates
### Push Notifications ### Push Notifications
- `POST /api/notifications/subscribe` - Subscribe to push notifications - `POST /api/notifications/subscribe` - Subscribe to push notifications
- `DELETE /api/notifications/subscribe` - Unsubscribe from notifications - `DELETE /api/notifications/subscribe` - Unsubscribe from notifications
- `GET /api/notifications/vapid-key` - Get VAPID public key - `GET /api/notifications/vapid-key` - Get VAPID public key
### Legacy Endpoints (Deprecated) ### Legacy Endpoints (Deprecated)
- ~~`POST /api/extract`~~ - Use `/api/queue` instead - ~~`POST /api/extract`~~ - Use `/api/queue` instead
- ~~`GET /api/extract-stream`~~ - Use `/api/queue/stream` instead - ~~`GET /api/extract-stream`~~ - Use `/api/queue/stream` instead
## 🛠 Development Setup ## 🛠 Development Setup
### Prerequisites ### Prerequisites
- Node.js 18+
- Node.js 18+
- npm or pnpm - npm or pnpm
- Tandoor Recipe Manager instance (optional) - Tandoor Recipe Manager instance (optional)
- LLM API access (OpenAI, Anthropic, or local) - LLM API access (OpenAI, Anthropic, or local)
@@ -79,6 +86,7 @@ open https://localhost:5173
``` ```
The app runs on HTTPS by default for: The app runs on HTTPS by default for:
- Service worker support (required for PWA) - Service worker support (required for PWA)
- Push notifications - Push notifications
- Browser share target API - Browser share target API
@@ -89,6 +97,7 @@ The app runs on HTTPS by default for:
The application uses HTTPS in development with SSL certificates signed by an external Caddy CA container. The current certificate is valid until **December 20, 2035** (10 years). The application uses HTTPS in development with SSL certificates signed by an external Caddy CA container. The current certificate is valid until **December 20, 2035** (10 years).
**Certificate Information:** **Certificate Information:**
- Location: `.ssl/` directory - Location: `.ssl/` directory
- CA Certificate: `.ssl/root.crt` (already trusted on the system) - CA Certificate: `.ssl/root.crt` (already trusted on the system)
- Server Certificate: `.ssl/localhost.crt` - Server Certificate: `.ssl/localhost.crt`
@@ -97,18 +106,21 @@ The application uses HTTPS in development with SSL certificates signed by an ext
Since the Caddy CA is already trusted on the system, the certificate should work without additional trust steps. If you encounter browser warnings: Since the Caddy CA is already trusted on the system, the certificate should work without additional trust steps. If you encounter browser warnings:
**Linux (Ubuntu/Debian):** **Linux (Ubuntu/Debian):**
```bash ```bash
sudo cp .ssl/root.crt /usr/local/share/ca-certificates/caddy-local.crt sudo cp .ssl/root.crt /usr/local/share/ca-certificates/caddy-local.crt
sudo update-ca-certificates sudo update-ca-certificates
``` ```
**Chrome/Chromium:** **Chrome/Chromium:**
1. Go to `chrome://settings/certificates` 1. Go to `chrome://settings/certificates`
2. Click "Authorities" → "Import" 2. Click "Authorities" → "Import"
3. Select `.ssl/root.crt` 3. Select `.ssl/root.crt`
4. Check "Trust this certificate for identifying websites" 4. Check "Trust this certificate for identifying websites"
**Checking Certificate Expiration:** **Checking Certificate Expiration:**
```bash ```bash
openssl x509 -enddate -noout -in .ssl/localhost.crt openssl x509 -enddate -noout -in .ssl/localhost.crt
``` ```
@@ -220,6 +232,7 @@ To enable web push notifications:
## 🏗 Architecture Overview ## 🏗 Architecture Overview
### Queue System ### Queue System
``` ```
User submits URL → Queue Manager → Queue Processor User submits URL → Queue Manager → Queue Processor
↓ ↓
@@ -231,7 +244,7 @@ User submits URL → Queue Manager → Queue Processor
### Processing Pipeline ### Processing Pipeline
1. **Extraction Phase**: Browser automation extracts text and images 1. **Extraction Phase**: Browser automation extracts text and images
2. **Parsing Phase**: LLM converts text to structured recipe data 2. **Parsing Phase**: LLM converts text to structured recipe data
3. **Upload Phase**: Automatic upload to Tandoor (if configured) 3. **Upload Phase**: Automatic upload to Tandoor (if configured)
Each phase tracks progress and can fail independently with proper error handling. Each phase tracks progress and can fail independently with proper error handling.
@@ -247,9 +260,9 @@ Each phase tracks progress and can fail independently with proper error handling
# Run all tests # Run all tests
npm test npm test
# Run specific test suites # Run specific test suites
npm run test:unit # Unit tests only npm run test:unit # Unit tests only
npm run test:client # Browser tests only npm run test:client # Browser tests only
npm run test:server # Server tests only npm run test:server # Server tests only
# Run tests in watch mode # Run tests in watch mode
@@ -257,9 +270,10 @@ npm run test:watch
``` ```
Test Coverage: Test Coverage:
- **138 total tests** covering all major components - **138 total tests** covering all major components
- Queue Manager: 28 tests - Queue Manager: 28 tests
- Queue Processor: 5 integration tests - Queue Processor: 5 integration tests
- API Endpoints: 17 tests - API Endpoints: 17 tests
- SSE Streaming: 6 tests - SSE Streaming: 6 tests
- Frontend Components: Browser tests - Frontend Components: Browser tests
@@ -279,11 +293,13 @@ npm run preview
### Deployment ### Deployment
The app is built as a Node.js application with the following outputs: The app is built as a Node.js application with the following outputs:
- `/.svelte-kit/output/server/` - Server bundle - `/.svelte-kit/output/server/` - Server bundle
- `/.svelte-kit/output/client/` - Static assets - `/.svelte-kit/output/client/` - Static assets
- `/build/` - Adapter output - `/build/` - Adapter output
Deploy the server bundle with: Deploy the server bundle with:
```bash ```bash
node build/index.js node build/index.js
``` ```
@@ -307,13 +323,15 @@ CMD ["node", "build"]
The app was migrated from a synchronous extraction system to an async queue-based system: The app was migrated from a synchronous extraction system to an async queue-based system:
**Before (Synchronous)**: **Before (Synchronous)**:
- User waited for entire extraction process to complete - User waited for entire extraction process to complete
- No progress tracking during processing - No progress tracking during processing
- No retry capability for failures - No retry capability for failures
- Single-threaded processing - Single-threaded processing
- Limited error handling - Limited error handling
**After (Async Queue)**: **After (Async Queue)**:
- Fire-and-forget: submit URL and redirect immediately - Fire-and-forget: submit URL and redirect immediately
- Real-time progress tracking via SSE - Real-time progress tracking via SSE
- Comprehensive retry system for failures - Comprehensive retry system for failures
@@ -324,16 +342,18 @@ The app was migrated from a synchronous extraction system to an async queue-base
### API Migration ### API Migration
**Old Synchronous Endpoints** (Deprecated): **Old Synchronous Endpoints** (Deprecated):
```bash ```bash
POST /api/extract # Submit URL and wait for completion POST /api/extract # Submit URL and wait for completion
GET /api/extract-stream # Long-polling for progress GET /api/extract-stream # Long-polling for progress
``` ```
**New Queue Endpoints**: **New Queue Endpoints**:
```bash ```bash
POST /api/queue # Submit URL, get queue ID immediately POST /api/queue # Submit URL, get queue ID immediately
GET /api/queue # List all queue items GET /api/queue # List all queue items
GET /api/queue/{id} # Get specific item status GET /api/queue/{id} # Get specific item status
POST /api/queue/{id}/retry # Retry failed items POST /api/queue/{id}/retry # Retry failed items
GET /api/queue/stream # Real-time SSE updates GET /api/queue/stream # Real-time SSE updates
``` ```
@@ -344,13 +364,14 @@ If migrating from the old system:
1. **Update Client Code**: Replace `/api/extract` calls with `/api/queue` 1. **Update Client Code**: Replace `/api/extract` calls with `/api/queue`
2. **Handle Async Responses**: Process queue ID instead of waiting for completion 2. **Handle Async Responses**: Process queue ID instead of waiting for completion
3. **Add Progress Tracking**: Implement SSE listeners for real-time updates 3. **Add Progress Tracking**: Implement SSE listeners for real-time updates
4. **Update Error Handling**: Handle new error classification system 4. **Update Error Handling**: Handle new error classification system
5. **Add Retry Logic**: Implement retry functionality for failed items 5. **Add Retry Logic**: Implement retry functionality for failed items
### Backward Compatibility ### Backward Compatibility
The legacy endpoints are still available but deprecated: The legacy endpoints are still available but deprecated:
- They will return `410 Gone` status with migration instructions - They will return `410 Gone` status with migration instructions
- Support will be removed in a future version - Support will be removed in a future version
- All new development should use the queue endpoints - All new development should use the queue endpoints
@@ -383,4 +404,3 @@ This project is licensed under the MIT License - see the LICENSE file for detail
- [Tandoor Recipe Manager](https://docs.tandoor.dev/) - Recipe management system - [Tandoor Recipe Manager](https://docs.tandoor.dev/) - Recipe management system
- [Workbox](https://developers.google.com/web/tools/workbox) - PWA capabilities - [Workbox](https://developers.google.com/web/tools/workbox) - PWA capabilities
- [fastq](https://github.com/mcollina/fastq) - High-performance queue processing - [fastq](https://github.com/mcollina/fastq) - High-performance queue processing

24
docker-compose.yml
View File

@@ -4,34 +4,34 @@ services:
container_name: insta-recipe container_name: insta-recipe
network_mode: host network_mode: host
ports: ports:
- "3000:3000" - '3000:3000'
environment: environment:
# LLM Configuration (Required) # LLM Configuration (Required)
- OPENAI_BASE_URL=${OPENAI_BASE_URL} - OPENAI_BASE_URL=${OPENAI_BASE_URL}
- OPENAI_API_KEY=${OPENAI_API_KEY} - OPENAI_API_KEY=${OPENAI_API_KEY}
- LLM_MODEL=${LLM_MODEL:-google/gemma-3-4b} - LLM_MODEL=${LLM_MODEL:-google/gemma-3-4b}
# Queue Configuration (Optional) # Queue Configuration (Optional)
- QUEUE_CONCURRENCY=${QUEUE_CONCURRENCY:-2} - QUEUE_CONCURRENCY=${QUEUE_CONCURRENCY:-2}
- QUEUE_MAX_RETRIES=${QUEUE_MAX_RETRIES:-3} - QUEUE_MAX_RETRIES=${QUEUE_MAX_RETRIES:-3}
# Tandoor Integration (Optional) # Tandoor Integration (Optional)
- TANDOOR_ENABLED=${TANDOOR_ENABLED:-false} - TANDOOR_ENABLED=${TANDOOR_ENABLED:-false}
- TANDOOR_SERVER_URL=${TANDOOR_SERVER_URL} - TANDOOR_SERVER_URL=${TANDOOR_SERVER_URL}
- TANDOOR_SPACE=${TANDOOR_SPACE:-1} - TANDOOR_SPACE=${TANDOOR_SPACE:-1}
- TANDOOR_TOKEN=${TANDOOR_TOKEN} - TANDOOR_TOKEN=${TANDOOR_TOKEN}
# Push Notifications (Optional) # Push Notifications (Optional)
- VAPID_PUBLIC_KEY=${VAPID_PUBLIC_KEY} - VAPID_PUBLIC_KEY=${VAPID_PUBLIC_KEY}
- VAPID_PRIVATE_KEY=${VAPID_PRIVATE_KEY} - VAPID_PRIVATE_KEY=${VAPID_PRIVATE_KEY}
# Authentication Scheduler (Optional) # Authentication Scheduler (Optional)
- AUTH_SCHEDULER_ENABLED=${AUTH_SCHEDULER_ENABLED:-false} - AUTH_SCHEDULER_ENABLED=${AUTH_SCHEDULER_ENABLED:-false}
- AUTH_SCHEDULER_INTERVAL_MINUTES=${AUTH_SCHEDULER_INTERVAL_MINUTES:-720} - AUTH_SCHEDULER_INTERVAL_MINUTES=${AUTH_SCHEDULER_INTERVAL_MINUTES:-720}
# Playwright Configuration # Playwright Configuration
- DISPLAY=:99 - DISPLAY=:99
# Node.js Environment # Node.js Environment
- NODE_ENV=production - NODE_ENV=production
security_opt: security_opt:
@@ -40,8 +40,14 @@ services:
- ./secrets:/app/secrets - ./secrets:/app/secrets
restart: unless-stopped restart: unless-stopped
healthcheck: healthcheck:
test: ["CMD", "node", "-e", "fetch('http://localhost:3000/api/health').then(r => r.ok ? process.exit(0) : process.exit(1)).catch(() => process.exit(1))"] test:
[
'CMD',
'node',
'-e',
"fetch('http://localhost:3000/api/health').then(r => r.ok ? process.exit(0) : process.exit(1)).catch(() => process.exit(1))"
]
interval: 30s interval: 30s
timeout: 10s timeout: 10s
retries: 3 retries: 3
start_period: 40s start_period: 40s

8
docker-compose_ori.yml
View File

@@ -2,7 +2,7 @@ services:
app: app:
build: . build: .
ports: ports:
- "5173:5173" - '5173:5173'
environment: environment:
- PLAYWRIGHT_WS_ENDPOINT=ws://playwright-service:3000 - PLAYWRIGHT_WS_ENDPOINT=ws://playwright-service:3000
- OPENAI_BASE_URL=http://ollama:11434/v1 - OPENAI_BASE_URL=http://ollama:11434/v1
@@ -18,7 +18,7 @@ services:
playwright-service: playwright-service:
build: ./playwright-service build: ./playwright-service
ipc: host ipc: host
ports: ["3000:3000"] ports: ['3000:3000']
environment: environment:
- DISPLAY=:99 - DISPLAY=:99
security_opt: security_opt:
@@ -26,9 +26,9 @@ services:
ollama: ollama:
image: ollama/ollama:latest image: ollama/ollama:latest
ports: ["11434:11434"] ports: ['11434:11434']
volumes: volumes:
- ollama_data:/root/.ollama - ollama_data:/root/.ollama
volumes: volumes:
ollama_data: ollama_data:

486
docs/API.md
View File

@@ -5,6 +5,7 @@ This document describes the InstaRecipe API endpoints for the async queue-based
## Base URL ## Base URL
All API endpoints are relative to your InstaRecipe instance: All API endpoints are relative to your InstaRecipe instance:
``` ```
https://your-instarecipe-instance.com/api https://your-instarecipe-instance.com/api
``` ```
@@ -23,13 +24,16 @@ All endpoints return standardized error responses:
```json ```json
{ {
"error": "Error type", "error": "Error type",
"message": "Human-readable error message", "message": "Human-readable error message",
"details": { /* Additional error context */ } "details": {
/* Additional error context */
}
} }
``` ```
HTTP status codes follow REST conventions: HTTP status codes follow REST conventions:
- `200` - Success - `200` - Success
- `201` - Created - `201` - Created
- `400` - Bad Request (invalid input) - `400` - Bad Request (invalid input)
@@ -45,13 +49,15 @@ HTTP status codes follow REST conventions:
Enqueue an Instagram URL for async processing. Enqueue an Instagram URL for async processing.
**Request:** **Request:**
```json ```json
{ {
"url": "https://instagram.com/p/abc123" "url": "https://instagram.com/p/abc123"
} }
``` ```
**Supported URL Formats:** **Supported URL Formats:**
- Posts: `https://instagram.com/p/{post-id}` - Posts: `https://instagram.com/p/{post-id}`
- Posts (www): `https://www.instagram.com/p/{post-id}` - Posts (www): `https://www.instagram.com/p/{post-id}`
- Reels: `https://instagram.com/reel/{reel-id}` - Reels: `https://instagram.com/reel/{reel-id}`
@@ -59,12 +65,14 @@ Enqueue an Instagram URL for async processing.
- With query parameters: `https://instagram.com/reel/{reel-id}?utm_source=share` - With query parameters: `https://instagram.com/reel/{reel-id}?utm_source=share`
**URL Requirements:** **URL Requirements:**
- Must use HTTPS protocol - Must use HTTPS protocol
- Hostname must be `instagram.com` or `www.instagram.com` - Hostname must be `instagram.com` or `www.instagram.com`
- Any Instagram path is accepted (posts, reels, IGTV, etc.) - Any Instagram path is accepted (posts, reels, IGTV, etc.)
- Query parameters and hash fragments are allowed - Query parameters and hash fragments are allowed
**Examples:** **Examples:**
```json ```json
// Post URL // Post URL
{ "url": "https://instagram.com/p/ABC123" } { "url": "https://instagram.com/p/ABC123" }
@@ -77,34 +85,36 @@ Enqueue an Instagram URL for async processing.
``` ```
**Response (201 Created):** **Response (201 Created):**
```json ```json
{ {
"id": "550e8400-e29b-41d4-a716-446655440000", "id": "550e8400-e29b-41d4-a716-446655440000",
"url": "https://instagram.com/reel/DSevV5CDcNm/?utm_source=ig_web_copy_link", "url": "https://instagram.com/reel/DSevV5CDcNm/?utm_source=ig_web_copy_link",
"status": "pending", "status": "pending",
"phases": [ "phases": [
{ {
"name": "extraction", "name": "extraction",
"status": "pending", "status": "pending",
"progress": 0 "progress": 0
}, },
{ {
"name": "parsing", "name": "parsing",
"status": "pending", "status": "pending",
"progress": 0 "progress": 0
}, },
{ {
"name": "uploading", "name": "uploading",
"status": "pending", "status": "pending",
"progress": 0 "progress": 0
} }
], ],
"createdAt": "2024-12-21T10:30:00Z", "createdAt": "2024-12-21T10:30:00Z",
"updatedAt": "2024-12-21T10:30:00Z" "updatedAt": "2024-12-21T10:30:00Z"
} }
``` ```
**Errors:** **Errors:**
- `400` - Invalid URL format (not a valid URL) - `400` - Invalid URL format (not a valid URL)
- `400` - URL must use HTTPS protocol - `400` - URL must use HTTPS protocol
- `400` - URL must be from instagram.com domain - `400` - URL must be from instagram.com domain
@@ -115,6 +125,7 @@ Enqueue an Instagram URL for async processing.
List queue items with optional filtering, pagination, and sorting. List queue items with optional filtering, pagination, and sorting.
**Query Parameters:** **Query Parameters:**
- `status` (optional): Filter by status (`pending`, `in_progress`, `success`, `error`, `unhealthy`) - `status` (optional): Filter by status (`pending`, `in_progress`, `success`, `error`, `unhealthy`)
- `limit` (optional): Number of items to return (default: 50, max: 100) - `limit` (optional): Number of items to return (default: 50, max: 100)
- `offset` (optional): Number of items to skip (default: 0) - `offset` (optional): Number of items to skip (default: 0)
@@ -122,6 +133,7 @@ List queue items with optional filtering, pagination, and sorting.
- `order` (optional): Sort order (`asc`, `desc`) (default: `desc`) - `order` (optional): Sort order (`asc`, `desc`) (default: `desc`)
**Examples:** **Examples:**
```bash ```bash
GET /api/queue # All items GET /api/queue # All items
GET /api/queue?status=error # Failed items only GET /api/queue?status=error # Failed items only
@@ -130,67 +142,68 @@ GET /api/queue?sort=status&order=asc # Sort by status
``` ```
**Response (200 OK):** **Response (200 OK):**
```json ```json
{ {
"items": [ "items": [
{ {
"id": "550e8400-e29b-41d4-a716-446655440000", "id": "550e8400-e29b-41d4-a716-446655440000",
"url": "https://instagram.com/p/abc123", "url": "https://instagram.com/p/abc123",
"status": "success", "status": "success",
"phases": [ "phases": [
{ {
"name": "extraction", "name": "extraction",
"status": "completed", "status": "completed",
"startedAt": "2024-12-21T10:30:01Z", "startedAt": "2024-12-21T10:30:01Z",
"completedAt": "2024-12-21T10:30:15Z", "completedAt": "2024-12-21T10:30:15Z",
"progress": 100 "progress": 100
}, },
{ {
"name": "parsing", "name": "parsing",
"status": "completed", "status": "completed",
"startedAt": "2024-12-21T10:30:15Z", "startedAt": "2024-12-21T10:30:15Z",
"completedAt": "2024-12-21T10:30:25Z", "completedAt": "2024-12-21T10:30:25Z",
"progress": 100 "progress": 100
}, },
{ {
"name": "uploading", "name": "uploading",
"status": "completed", "status": "completed",
"startedAt": "2024-12-21T10:30:25Z", "startedAt": "2024-12-21T10:30:25Z",
"completedAt": "2024-12-21T10:30:30Z", "completedAt": "2024-12-21T10:30:30Z",
"progress": 100 "progress": 100
} }
], ],
"results": { "results": {
"recipe": { "recipe": {
"name": "Chocolate Chip Cookies", "name": "Chocolate Chip Cookies",
"description": "Delicious homemade cookies", "description": "Delicious homemade cookies",
"servings": 24, "servings": 24,
"ingredients": [ "ingredients": [
{ {
"food": "flour", "food": "flour",
"amount": 2.25, "amount": 2.25,
"unit": "cups" "unit": "cups"
} }
], ],
"steps": [ "steps": [
{ {
"instruction": "Preheat oven to 375°F", "instruction": "Preheat oven to 375°F",
"time": 5 "time": 5
} }
], ],
"keywords": ["cookies", "dessert", "chocolate"], "keywords": ["cookies", "dessert", "chocolate"],
"image": "https://instagram.com/image.jpg" "image": "https://instagram.com/image.jpg"
}, },
"tandoorUrl": "https://tandoor.example.com/recipe/123", "tandoorUrl": "https://tandoor.example.com/recipe/123",
"extractedText": "Raw extracted text...", "extractedText": "Raw extracted text...",
"thumbnail": "https://instagram.com/thumbnail.jpg" "thumbnail": "https://instagram.com/thumbnail.jpg"
}, },
"createdAt": "2024-12-21T10:30:00Z", "createdAt": "2024-12-21T10:30:00Z",
"updatedAt": "2024-12-21T10:30:30Z" "updatedAt": "2024-12-21T10:30:30Z"
} }
], ],
"total": 42, "total": 42,
"hasMore": true "hasMore": true
} }
``` ```
@@ -199,12 +212,14 @@ GET /api/queue?sort=status&order=asc # Sort by status
Get details for a specific queue item. Get details for a specific queue item.
**Path Parameters:** **Path Parameters:**
- `id`: Queue item UUID - `id`: Queue item UUID
**Response (200 OK):** **Response (200 OK):**
Returns the same queue item structure as in the list response. Returns the same queue item structure as in the list response.
**Errors:** **Errors:**
- `400` - Invalid UUID format - `400` - Invalid UUID format
- `404` - Queue item not found - `404` - Queue item not found
@@ -213,22 +228,25 @@ Returns the same queue item structure as in the list response.
Retry a failed queue item. Only items with `error` or `unhealthy` status can be retried. Retry a failed queue item. Only items with `error` or `unhealthy` status can be retried.
**Path Parameters:** **Path Parameters:**
- `id`: Queue item UUID - `id`: Queue item UUID
**Response (200 OK):** **Response (200 OK):**
```json ```json
{ {
"success": true, "success": true,
"message": "Item queued for retry", "message": "Item queued for retry",
"item": { "item": {
"id": "550e8400-e29b-41d4-a716-446655440000", "id": "550e8400-e29b-41d4-a716-446655440000",
"status": "pending", "status": "pending",
"updatedAt": "2024-12-21T11:00:00Z" "updatedAt": "2024-12-21T11:00:00Z"
} }
} }
``` ```
**Errors:** **Errors:**
- `400` - Invalid UUID format - `400` - Invalid UUID format
- `404` - Queue item not found - `404` - Queue item not found
- `409` - Cannot retry item with current status (only `error` and `unhealthy` can be retried) - `409` - Cannot retry item with current status (only `error` and `unhealthy` can be retried)
@@ -240,10 +258,12 @@ Retry a failed queue item. Only items with `error` or `unhealthy` status can be
Server-Sent Events (SSE) endpoint for real-time queue updates. Server-Sent Events (SSE) endpoint for real-time queue updates.
**Query Parameters:** **Query Parameters:**
- `itemId` (optional): Filter updates for specific item - `itemId` (optional): Filter updates for specific item
- `status` (optional): Filter updates by status - `status` (optional): Filter updates by status
**Headers:** **Headers:**
``` ```
Accept: text/event-stream Accept: text/event-stream
Cache-Control: no-cache Cache-Control: no-cache
@@ -253,19 +273,23 @@ Cache-Control: no-cache
SSE stream with the following event types: SSE stream with the following event types:
#### connection #### connection
Sent when connection is established: Sent when connection is established:
``` ```
event: connection event: connection
data: {"message": "Connected to queue stream", "timestamp": "2024-12-21T10:30:00Z"} data: {"message": "Connected to queue stream", "timestamp": "2024-12-21T10:30:00Z"}
``` ```
#### queue-update #### queue-update
Sent when queue item status changes: Sent when queue item status changes:
``` ```
event: queue-update event: queue-update
data: { data: {
"itemId": "550e8400-e29b-41d4-a716-446655440000", "itemId": "550e8400-e29b-41d4-a716-446655440000",
"status": "in_progress", "status": "in_progress",
"timestamp": "2024-12-21T10:30:01Z", "timestamp": "2024-12-21T10:30:01Z",
"progress": [ "progress": [
{ {
@@ -279,7 +303,9 @@ data: {
``` ```
#### ping #### ping
Keep-alive ping sent every 30 seconds: Keep-alive ping sent every 30 seconds:
``` ```
event: ping event: ping
data: {"timestamp": "2024-12-21T10:30:30Z"} data: {"timestamp": "2024-12-21T10:30:30Z"}
@@ -288,30 +314,32 @@ data: {"timestamp": "2024-12-21T10:30:30Z"}
**Usage Examples:** **Usage Examples:**
**JavaScript:** **JavaScript:**
```javascript ```javascript
const eventSource = new EventSource('/api/queue/stream'); const eventSource = new EventSource('/api/queue/stream');
eventSource.addEventListener('connection', (event) => { eventSource.addEventListener('connection', (event) => {
console.log('Connected:', JSON.parse(event.data)); console.log('Connected:', JSON.parse(event.data));
}); });
eventSource.addEventListener('queue-update', (event) => { eventSource.addEventListener('queue-update', (event) => {
const update = JSON.parse(event.data); const update = JSON.parse(event.data);
console.log('Queue update:', update); console.log('Queue update:', update);
updateUI(update); updateUI(update);
}); });
eventSource.addEventListener('ping', (event) => { eventSource.addEventListener('ping', (event) => {
console.log('Keep-alive ping'); console.log('Keep-alive ping');
}); });
eventSource.onerror = (error) => { eventSource.onerror = (error) => {
console.error('SSE error:', error); console.error('SSE error:', error);
// Reconnect logic here // Reconnect logic here
}; };
``` ```
**curl:** **curl:**
```bash ```bash
curl -N -H "Accept: text/event-stream" \ curl -N -H "Accept: text/event-stream" \
"https://your-instance.com/api/queue/stream?itemId=550e8400-e29b-41d4-a716-446655440000" "https://your-instance.com/api/queue/stream?itemId=550e8400-e29b-41d4-a716-446655440000"
@@ -324,10 +352,11 @@ curl -N -H "Accept: text/event-stream" \
Get the VAPID public key required for push notification subscriptions. Get the VAPID public key required for push notification subscriptions.
**Response (200 OK):** **Response (200 OK):**
```json ```json
{ {
"publicKey": "BDummyPublicKeyForDevelopment...", "publicKey": "BDummyPublicKeyForDevelopment...",
"applicationServerKey": "BDummyPublicKeyForDevelopment..." "applicationServerKey": "BDummyPublicKeyForDevelopment..."
} }
``` ```
@@ -336,29 +365,32 @@ Get the VAPID public key required for push notification subscriptions.
Subscribe to push notifications for queue processing updates. Subscribe to push notifications for queue processing updates.
**Request:** **Request:**
```json ```json
{ {
"subscription": { "subscription": {
"endpoint": "https://fcm.googleapis.com/fcm/send/...", "endpoint": "https://fcm.googleapis.com/fcm/send/...",
"keys": { "keys": {
"p256dh": "BIBn3E_YUVpW5f6_Eq_GH...", "p256dh": "BIBn3E_YUVpW5f6_Eq_GH...",
"auth": "tBiH_Y1nPSuVh7TRMhcf..." "auth": "tBiH_Y1nPSuVh7TRMhcf..."
} }
}, },
"clientId": "unique-client-identifier" "clientId": "unique-client-identifier"
} }
``` ```
**Response (200 OK):** **Response (200 OK):**
```json ```json
{ {
"success": true, "success": true,
"message": "Successfully subscribed to push notifications", "message": "Successfully subscribed to push notifications",
"subscriptionCount": 5 "subscriptionCount": 5
} }
``` ```
**Errors:** **Errors:**
- `400` - Invalid subscription object or missing clientId - `400` - Invalid subscription object or missing clientId
### DELETE /api/notifications/subscribe ### DELETE /api/notifications/subscribe
@@ -366,18 +398,20 @@ Subscribe to push notifications for queue processing updates.
Unsubscribe from push notifications. Unsubscribe from push notifications.
**Request:** **Request:**
```json ```json
{ {
"clientId": "unique-client-identifier" "clientId": "unique-client-identifier"
} }
``` ```
**Response (200 OK):** **Response (200 OK):**
```json ```json
{ {
"success": true, "success": true,
"message": "Successfully unsubscribed from push notifications", "message": "Successfully unsubscribed from push notifications",
"subscriptionCount": 4 "subscriptionCount": 4
} }
``` ```
@@ -390,18 +424,19 @@ Unsubscribe from push notifications.
This synchronous extraction endpoint is deprecated. Use `POST /api/queue` instead. This synchronous extraction endpoint is deprecated. Use `POST /api/queue` instead.
**Migration:** **Migration:**
```javascript ```javascript
// ❌ Old synchronous approach // ❌ Old synchronous approach
const response = await fetch('/api/extract', { const response = await fetch('/api/extract', {
method: 'POST', method: 'POST',
body: JSON.stringify({ url }) body: JSON.stringify({ url })
}); });
const result = await response.json(); // Wait 30-60 seconds const result = await response.json(); // Wait 30-60 seconds
// ✅ New async queue approach // ✅ New async queue approach
const response = await fetch('/api/queue', { const response = await fetch('/api/queue', {
method: 'POST', method: 'POST',
body: JSON.stringify({ url }) body: JSON.stringify({ url })
}); });
const queueItem = await response.json(); // Immediate response const queueItem = await response.json(); // Immediate response
``` ```
@@ -413,17 +448,18 @@ const queueItem = await response.json(); // Immediate response
This SSE endpoint is deprecated. Use `POST /api/queue` + `GET /api/queue/stream` instead. This SSE endpoint is deprecated. Use `POST /api/queue` + `GET /api/queue/stream` instead.
**Migration:** **Migration:**
```javascript ```javascript
// ❌ Old approach // ❌ Old approach
const response = await fetch('/api/extract-stream', { const response = await fetch('/api/extract-stream', {
method: 'POST', method: 'POST',
body: JSON.stringify({ url }) body: JSON.stringify({ url })
}); });
// ✅ New approach // ✅ New approach
const queueResponse = await fetch('/api/queue', { const queueResponse = await fetch('/api/queue', {
method: 'POST', method: 'POST',
body: JSON.stringify({ url }) body: JSON.stringify({ url })
}); });
const item = await queueResponse.json(); const item = await queueResponse.json();
@@ -436,28 +472,28 @@ const eventSource = new EventSource(`/api/queue/stream?itemId=${item.id}`);
```typescript ```typescript
interface QueueItem { interface QueueItem {
id: string; // UUID v4 id: string; // UUID v4
url: string; // Instagram URL url: string; // Instagram URL
status: 'pending' | 'in_progress' | 'success' | 'error' | 'unhealthy'; status: 'pending' | 'in_progress' | 'success' | 'error' | 'unhealthy';
phases: Array<{ phases: Array<{
name: 'extraction' | 'parsing' | 'uploading'; name: 'extraction' | 'parsing' | 'uploading';
status: 'pending' | 'in_progress' | 'completed' | 'error'; status: 'pending' | 'in_progress' | 'completed' | 'error';
startedAt?: string; // ISO 8601 timestamp startedAt?: string; // ISO 8601 timestamp
completedAt?: string; // ISO 8601 timestamp completedAt?: string; // ISO 8601 timestamp
progress?: number; // 0-100 progress?: number; // 0-100
}>; }>;
results?: { results?: {
recipe?: Recipe; // Structured recipe data recipe?: Recipe; // Structured recipe data
tandoorUrl?: string; // Tandoor recipe URL tandoorUrl?: string; // Tandoor recipe URL
extractedText?: string; // Raw extracted text extractedText?: string; // Raw extracted text
thumbnail?: string; // Image URL thumbnail?: string; // Image URL
}; };
error?: string; // Error message error?: string; // Error message
createdAt: string; // ISO 8601 timestamp createdAt: string; // ISO 8601 timestamp
updatedAt: string; // ISO 8601 timestamp updatedAt: string; // ISO 8601 timestamp
} }
``` ```
@@ -465,32 +501,33 @@ interface QueueItem {
```typescript ```typescript
interface Recipe { interface Recipe {
name: string; name: string;
description?: string; description?: string;
servings?: number; servings?: number;
prepTime?: number; // Minutes prepTime?: number; // Minutes
cookTime?: number; // Minutes cookTime?: number; // Minutes
totalTime?: number; // Minutes totalTime?: number; // Minutes
ingredients: Array<{ ingredients: Array<{
food: string; food: string;
amount?: number; amount?: number;
unit?: string; unit?: string;
}>; }>;
steps: Array<{ steps: Array<{
instruction: string; instruction: string;
time?: number; // Minutes time?: number; // Minutes
}>; }>;
keywords?: string[]; // Recipe tags keywords?: string[]; // Recipe tags
image?: string; // Image URL image?: string; // Image URL
nutrition?: { // Nutritional information nutrition?: {
calories?: number; // Nutritional information
protein?: number; calories?: number;
carbs?: number; protein?: number;
fat?: number; carbs?: number;
}; fat?: number;
};
} }
``` ```
@@ -517,59 +554,58 @@ When implementing clients, consider these error recovery strategies:
```javascript ```javascript
async function processInstagramUrl(url) { async function processInstagramUrl(url) {
try { try {
// 1. Enqueue URL // 1. Enqueue URL
const queueResponse = await fetch('/api/queue', { const queueResponse = await fetch('/api/queue', {
method: 'POST', method: 'POST',
headers: { 'Content-Type': 'application/json' }, headers: { 'Content-Type': 'application/json' },
body: JSON.stringify({ url }) body: JSON.stringify({ url })
}); });
const queueItem = await queueResponse.json(); const queueItem = await queueResponse.json();
console.log('Enqueued:', queueItem.id); console.log('Enqueued:', queueItem.id);
// 2. Listen for real-time updates // 2. Listen for real-time updates
const eventSource = new EventSource(`/api/queue/stream?itemId=${queueItem.id}`); const eventSource = new EventSource(`/api/queue/stream?itemId=${queueItem.id}`);
return new Promise((resolve, reject) => { return new Promise((resolve, reject) => {
eventSource.addEventListener('queue-update', (event) => { eventSource.addEventListener('queue-update', (event) => {
const update = JSON.parse(event.data); const update = JSON.parse(event.data);
if (update.status === 'success') { if (update.status === 'success') {
eventSource.close(); eventSource.close();
resolve(update.results); resolve(update.results);
} else if (update.status === 'error') { } else if (update.status === 'error') {
eventSource.close(); eventSource.close();
reject(new Error(update.error)); reject(new Error(update.error));
} }
// Handle progress updates // Handle progress updates
console.log('Progress:', update.progress); console.log('Progress:', update.progress);
}); });
eventSource.onerror = (error) => { eventSource.onerror = (error) => {
eventSource.close(); eventSource.close();
reject(error); reject(error);
}; };
}); });
} catch (error) {
} catch (error) { console.error('Processing failed:', error);
console.error('Processing failed:', error); throw error;
throw error; }
}
} }
// Usage // Usage
processInstagramUrl('https://instagram.com/p/abc123') processInstagramUrl('https://instagram.com/p/abc123')
.then(results => { .then((results) => {
console.log('Recipe extracted:', results.recipe); console.log('Recipe extracted:', results.recipe);
if (results.tandoorUrl) { if (results.tandoorUrl) {
console.log('Uploaded to Tandoor:', results.tandoorUrl); console.log('Uploaded to Tandoor:', results.tandoorUrl);
} }
}) })
.catch(error => { .catch((error) => {
console.error('Extraction failed:', error.message); console.error('Extraction failed:', error.message);
}); });
``` ```
For more examples and integration guides, see the [Migration Documentation](/docs/MIGRATION.md). For more examples and integration guides, see the [Migration Documentation](/docs/MIGRATION.md).

51
docs/ARCHITECTURE.md
View File

@@ -91,21 +91,27 @@ insta-recipe/
## Key Directories ## Key Directories
### `/src/lib/server/` ### `/src/lib/server/`
Server-side business logic following Hexagonal Architecture principles. Contains domain logic, adapters for external systems (Instagram, Tandoor, LLM), and port definitions. Server-side business logic following Hexagonal Architecture principles. Contains domain logic, adapters for external systems (Instagram, Tandoor, LLM), and port definitions.
### `/src/lib/client/` ### `/src/lib/client/`
Client-side utilities for PWA features (push notifications, install prompts, service worker messaging). Client-side utilities for PWA features (push notifications, install prompts, service worker messaging).
### `/src/routes/api/` ### `/src/routes/api/`
RESTful API endpoints implemented as SvelteKit server routes. Each directory contains `+server.ts` files exporting HTTP verb handlers. RESTful API endpoints implemented as SvelteKit server routes. Each directory contains `+server.ts` files exporting HTTP verb handlers.
### `/src/routes/share/` ### `/src/routes/share/`
Share target page allowing users to share Instagram URLs directly from their browser or mobile apps. Share target page allowing users to share Instagram URLs directly from their browser or mobile apps.
### `/src/lib/server/queue/` ### `/src/lib/server/queue/`
Queue management system with in-memory storage, processor workers, and type definitions. Queue management system with in-memory storage, processor workers, and type definitions.
### `/docs/` ### `/docs/`
Comprehensive documentation including plans, outcomes, API specs, and migration guides. Comprehensive documentation including plans, outcomes, API specs, and migration guides.
--- ---
@@ -113,33 +119,43 @@ Comprehensive documentation including plans, outcomes, API specs, and migration
## Design Patterns ## Design Patterns
### Singleton Pattern ### Singleton Pattern
Used for shared service instances: Used for shared service instances:
- `QueueManager` (`queueManager` exported instance) - `QueueManager` (`queueManager` exported instance)
- `QueueProcessor` (`queueProcessor` exported instance) - `QueueProcessor` (`queueProcessor` exported instance)
- `PushNotificationService` (`pushNotificationService` exported instance) - `PushNotificationService` (`pushNotificationService` exported instance)
- `ServiceWorkerMessageHandler` (`serviceWorkerMessageHandler` exported instance) - `ServiceWorkerMessageHandler` (`serviceWorkerMessageHandler` exported instance)
### Factory Pattern ### Factory Pattern
Used for creating configured instances: Used for creating configured instances:
- `createLLM()` - Creates OpenAI client with environment configuration - `createLLM()` - Creates OpenAI client with environment configuration
- `createBrowserContext()` - Creates Playwright browser context with options - `createBrowserContext()` - Creates Playwright browser context with options
- `initializeBrowser()` - Initializes Chromium browser instance - `initializeBrowser()` - Initializes Chromium browser instance
### Observer Pattern ### Observer Pattern
Implemented in QueueManager for real-time updates: Implemented in QueueManager for real-time updates:
- Subscribers receive notifications on queue item changes - Subscribers receive notifications on queue item changes
- Server-Sent Events (SSE) stream queue updates to clients - Server-Sent Events (SSE) stream queue updates to clients
- Push notifications notify users of completion events - Push notifications notify users of completion events
### Adapter Pattern (Hexagonal Architecture) ### Adapter Pattern (Hexagonal Architecture)
External systems accessed via adapters: External systems accessed via adapters:
- **Instagram Adapter**: `extraction.ts` - Extracts content via Playwright - **Instagram Adapter**: `extraction.ts` - Extracts content via Playwright
- **LLM Adapter**: `llm.ts`, `parser.ts` - Recipe parsing via OpenAI - **LLM Adapter**: `llm.ts`, `parser.ts` - Recipe parsing via OpenAI
- **Tandoor Adapter**: `tandoor.ts` - Recipe management system integration - **Tandoor Adapter**: `tandoor.ts` - Recipe management system integration
- **Browser Adapter**: `browser.ts` - Playwright browser automation - **Browser Adapter**: `browser.ts` - Playwright browser automation
### Strategy Pattern ### Strategy Pattern
Multiple extraction strategies with fallback: Multiple extraction strategies with fallback:
1. Embedded JSON extraction 1. Embedded JSON extraction
2. DOM selector extraction 2. DOM selector extraction
3. GraphQL API extraction 3. GraphQL API extraction
@@ -150,28 +166,34 @@ Multiple extraction strategies with fallback:
## Key Components ## Key Components
### Queue Management System ### Queue Management System
**Location**: `src/lib/server/queue/` **Location**: `src/lib/server/queue/`
Three-phase processing pipeline: Three-phase processing pipeline:
1. **Extraction Phase**: Extract text and thumbnail from Instagram 1. **Extraction Phase**: Extract text and thumbnail from Instagram
2. **Parsing Phase**: Parse recipe using LLM 2. **Parsing Phase**: Parse recipe using LLM
3. **Uploading Phase**: Upload to Tandoor (if enabled) 3. **Uploading Phase**: Upload to Tandoor (if enabled)
**Components**: **Components**:
- `QueueManager`: In-memory FIFO queue with CRUD operations - `QueueManager`: In-memory FIFO queue with CRUD operations
- `QueueProcessor`: Worker that processes items with configurable concurrency - `QueueProcessor`: Worker that processes items with configurable concurrency
- `types.ts`: Comprehensive type definitions for queue items and updates - `types.ts`: Comprehensive type definitions for queue items and updates
### API Layer ### API Layer
**Location**: `src/routes/api/` **Location**: `src/routes/api/`
RESTful endpoints for: RESTful endpoints for:
- Queue operations (`POST /api/queue`, `GET /api/queue`, `GET /api/queue/[id]`) - Queue operations (`POST /api/queue`, `GET /api/queue`, `GET /api/queue/[id]`)
- Real-time updates (`GET /api/queue/stream` - SSE) - Real-time updates (`GET /api/queue/stream` - SSE)
- Push notifications (`POST /api/notifications/subscribe`) - Push notifications (`POST /api/notifications/subscribe`)
- Health checks (`GET /api/health`, `GET /api/llm-health`) - Health checks (`GET /api/health`, `GET /api/llm-health`)
### Client-Side Services ### Client-Side Services
**Location**: `src/lib/client/` **Location**: `src/lib/client/`
- **PushNotificationManager**: Manages Web Push API subscriptions - **PushNotificationManager**: Manages Web Push API subscriptions
@@ -179,14 +201,17 @@ RESTful endpoints for:
- **ServiceWorkerMessageHandler**: Processes service worker messages - **ServiceWorkerMessageHandler**: Processes service worker messages
### Instagram Extraction ### Instagram Extraction
**Location**: `src/lib/server/extraction.ts` **Location**: `src/lib/server/extraction.ts`
Multi-method extraction with intelligent fallback: Multi-method extraction with intelligent fallback:
- Progress callbacks for real-time feedback - Progress callbacks for real-time feedback
- Retry logic with configurable attempts - Retry logic with configurable attempts
- Thumbnail extraction and validation - Thumbnail extraction and validation
### LLM Integration ### LLM Integration
**Location**: `src/lib/server/parser.ts`, `src/lib/server/llm.ts` **Location**: `src/lib/server/parser.ts`, `src/lib/server/llm.ts`
- Recipe detection endpoint - Recipe detection endpoint
@@ -198,6 +223,7 @@ Multi-method extraction with intelligent fallback:
## Dependencies ## Dependencies
### Production Dependencies ### Production Dependencies
- **@types/uuid** (^10.0.0) - UUID type definitions - **@types/uuid** (^10.0.0) - UUID type definitions
- **date-fns** (^4.1.0) - Date utility library - **date-fns** (^4.1.0) - Date utility library
- **openai** (^4.20.0) - OpenAI API client - **openai** (^4.20.0) - OpenAI API client
@@ -206,6 +232,7 @@ Multi-method extraction with intelligent fallback:
- **zod** (^3.23.0) - Schema validation - **zod** (^3.23.0) - Schema validation
### Development Dependencies ### Development Dependencies
- **@sveltejs/kit** (^2.48.5) - SvelteKit framework - **@sveltejs/kit** (^2.48.5) - SvelteKit framework
- **@sveltejs/adapter-node** (^5.4.0) - Node.js adapter - **@sveltejs/adapter-node** (^5.4.0) - Node.js adapter
- **svelte** (^5.43.8) - Svelte 5 framework - **svelte** (^5.43.8) - Svelte 5 framework
@@ -223,12 +250,14 @@ Multi-method extraction with intelligent fallback:
## Module Organization ## Module Organization
### SvelteKit Path Aliases ### SvelteKit Path Aliases
- `$lib` → `src/lib/` - `$lib` → `src/lib/`
- `$lib/*` → `src/lib/*` - `$lib/*` → `src/lib/*`
- `$app/*` → SvelteKit app imports - `$app/*` → SvelteKit app imports
- `$env/dynamic/private` → Environment variables (server-side) - `$env/dynamic/private` → Environment variables (server-side)
### Directory Structure Conventions ### Directory Structure Conventions
- **Server-only code**: `src/lib/server/` (not bundled to client) - **Server-only code**: `src/lib/server/` (not bundled to client)
- **Client-only code**: `src/lib/client/` (not executed on server) - **Client-only code**: `src/lib/client/` (not executed on server)
- **Shared code**: `src/lib/` (available to both) - **Shared code**: `src/lib/` (available to both)
@@ -240,6 +269,7 @@ Multi-method extraction with intelligent fallback:
## Data Flow ## Data Flow
### Recipe Extraction Flow ### Recipe Extraction Flow
``` ```
User submits URL User submits URL
↓ ↓
@@ -261,6 +291,7 @@ SSE updates notify client
``` ```
### Real-time Updates Flow ### Real-time Updates Flow
``` ```
Client connects to GET /api/queue/stream (SSE) Client connects to GET /api/queue/stream (SSE)
↓ ↓
@@ -274,6 +305,7 @@ Client updates UI reactively
``` ```
### Push Notification Flow ### Push Notification Flow
``` ```
Client requests permission Client requests permission
↓ ↓
@@ -295,37 +327,44 @@ Notification displayed to user
## Build System ## Build System
### Build Command ### Build Command
```bash ```bash
npm run build npm run build
``` ```
Generates production-ready build in `build/` directory using: Generates production-ready build in `build/` directory using:
- Vite for bundling - Vite for bundling
- `@sveltejs/adapter-node` for Node.js deployment - `@sveltejs/adapter-node` for Node.js deployment
- TypeScript compilation - TypeScript compilation
- SvelteKit prerendering and optimization - SvelteKit prerendering and optimization
### Test Command ### Test Command
```bash ```bash
npm test npm test
``` ```
Runs test suite using Vitest with two projects: Runs test suite using Vitest with two projects:
1. **Server tests**: Node environment for server-side code 1. **Server tests**: Node environment for server-side code
2. **Client tests**: Playwright browser for Svelte components 2. **Client tests**: Playwright browser for Svelte components
### Development Server ### Development Server
```bash ```bash
npm run dev npm run dev
``` ```
Starts Vite dev server with: Starts Vite dev server with:
- HTTPS enabled (certificates in `.ssl/`) - HTTPS enabled (certificates in `.ssl/`)
- Hot module replacement - Hot module replacement
- TypeScript checking - TypeScript checking
- File watching - File watching
### Linting & Formatting ### Linting & Formatting
```bash ```bash
npm run lint # ESLint + Prettier check npm run lint # ESLint + Prettier check
npm run format # Prettier write npm run format # Prettier write
@@ -336,19 +375,24 @@ npm run format # Prettier write
## Deployment ## Deployment
### Docker Deployment ### Docker Deployment
Dockerfile includes: Dockerfile includes:
- Node.js 22 Alpine base image - Node.js 22 Alpine base image
- Playwright Chromium installation - Playwright Chromium installation
- Production build - Production build
- Port 3000 exposure - Port 3000 exposure
Run with: Run with:
```bash ```bash
docker-compose up docker-compose up
``` ```
### Environment Variables ### Environment Variables
Required configuration: Required configuration:
- `OPENAI_API_KEY` - LLM API access - `OPENAI_API_KEY` - LLM API access
- `TANDOOR_URL` - Tandoor instance URL (optional) - `TANDOOR_URL` - Tandoor instance URL (optional)
- `TANDOOR_TOKEN` - Tandoor API token (optional) - `TANDOOR_TOKEN` - Tandoor API token (optional)
@@ -360,13 +404,16 @@ Required configuration:
## Testing Architecture ## Testing Architecture
### Test Categories ### Test Categories
1. **Unit Tests**: Individual function testing 1. **Unit Tests**: Individual function testing
2. **Integration Tests**: Multi-component workflows 2. **Integration Tests**: Multi-component workflows
3. **API Tests**: Endpoint behavior validation 3. **API Tests**: Endpoint behavior validation
4. **Browser Tests**: Svelte component rendering 4. **Browser Tests**: Svelte component rendering
### Test Coverage ### Test Coverage
138 tests covering: 138 tests covering:
- Queue management operations - Queue management operations
- Instagram URL validation - Instagram URL validation
- SSE streaming - SSE streaming
@@ -375,6 +422,7 @@ Required configuration:
- Notification service - Notification service
### Test Configuration ### Test Configuration
- **Server tests**: Node environment with mocked dependencies - **Server tests**: Node environment with mocked dependencies
- **Client tests**: Playwright Chromium browser with Svelte testing library - **Client tests**: Playwright Chromium browser with Svelte testing library
@@ -383,15 +431,18 @@ Required configuration:
## Security Considerations ## Security Considerations
### SSL/TLS ### SSL/TLS
- Development uses local SSL certificates signed by external Caddy CA - Development uses local SSL certificates signed by external Caddy CA
- Certificates stored in `.ssl/` (git-ignored) - Certificates stored in `.ssl/` (git-ignored)
- Required for PWA features (Service Worker, Push API) - Required for PWA features (Service Worker, Push API)
### Authentication ### Authentication
- Basic auth for scheduled tasks (username/password from environment) - Basic auth for scheduled tasks (username/password from environment)
- Tandoor integration uses bearer token authentication - Tandoor integration uses bearer token authentication
### Input Validation ### Input Validation
- Instagram URL validation with regex patterns - Instagram URL validation with regex patterns
- Zod schema validation for API payloads - Zod schema validation for API payloads
- Error handling with custom error classes - Error handling with custom error classes

346
docs/CODE_STYLE.md
View File

@@ -19,12 +19,14 @@
### Files & Directories ### Files & Directories
#### SvelteKit Route Files #### SvelteKit Route Files
- Route pages: `+page.svelte` - Route pages: `+page.svelte`
- Route servers: `+server.ts` - Route servers: `+server.ts`
- Route layouts: `+layout.svelte` - Route layouts: `+layout.svelte`
- Type definitions: `$types.ts` (auto-generated) - Type definitions: `$types.ts` (auto-generated)
**Example:** **Example:**
``` ```
src/routes/api/queue/ src/routes/api/queue/
├── [id]/ ├── [id]/
@@ -37,19 +39,23 @@ src/routes/api/queue/
``` ```
#### Library Files #### Library Files
- **PascalCase** for classes and managers: `QueueManager.ts`, `PushNotificationService.ts` - **PascalCase** for classes and managers: `QueueManager.ts`, `PushNotificationService.ts`
- **kebab-case** for utilities and configs: `tandoor-config.ts`, `instagram-url.ts` - **kebab-case** for utilities and configs: `tandoor-config.ts`, `instagram-url.ts`
- **lowercase** for general modules: `browser.ts`, `extraction.ts`, `parser.ts` - **lowercase** for general modules: `browser.ts`, `extraction.ts`, `parser.ts`
**Examples from codebase:** **Examples from codebase:**
- `src/lib/server/queue/QueueManager.ts` - `src/lib/server/queue/QueueManager.ts`
- `src/lib/server/tandoor-config.ts` - `src/lib/server/tandoor-config.ts`
- `src/lib/client/PushNotificationManager.ts` - `src/lib/client/PushNotificationManager.ts`
#### Test Files #### Test Files
Pattern: `<name>.spec.ts` or `<name>.test.ts` Pattern: `<name>.spec.ts` or `<name>.test.ts`
**Examples:** **Examples:**
- `queue-manager.spec.ts` - `queue-manager.spec.ts`
- `instagram-url-validation.spec.ts` - `instagram-url-validation.spec.ts`
- `page.svelte.spec.ts` - `page.svelte.spec.ts`
@@ -57,10 +63,12 @@ Pattern: `<name>.spec.ts` or `<name>.test.ts`
### Variables & Functions ### Variables & Functions
#### Variables #### Variables
- **camelCase** for local variables and parameters - **camelCase** for local variables and parameters
- **SCREAMING_SNAKE_CASE** for constants - **SCREAMING_SNAKE_CASE** for constants
**Examples:** **Examples:**
```typescript ```typescript
// From QueueManager.ts // From QueueManager.ts
private items: Map<string, QueueItem> = new Map(); private items: Map<string, QueueItem> = new Map();
@@ -76,10 +84,12 @@ const unsubscribe = queueManager.subscribe(callback);
``` ```
#### Functions #### Functions
- **camelCase** for function names - **camelCase** for function names
- **Descriptive action verbs**: `enqueue`, `extractRecipe`, `uploadRecipeImage` - **Descriptive action verbs**: `enqueue`, `extractRecipe`, `uploadRecipeImage`
**Examples:** **Examples:**
```typescript ```typescript
// From QueueManager.ts // From QueueManager.ts
enqueue(url: string): QueueItem { ... } enqueue(url: string): QueueItem { ... }
@@ -99,62 +109,62 @@ export async function extractRecipe(text: string): Promise<Recipe> { ... }
### Types & Interfaces ### Types & Interfaces
#### Interfaces & Types #### Interfaces & Types
- **PascalCase** for interface names - **PascalCase** for interface names
- Prefix with `I` is **NOT** used - Prefix with `I` is **NOT** used
- Exported types use `export type` or `export interface` - Exported types use `export type` or `export interface`
**Examples:** **Examples:**
```typescript ```typescript
// From queue/types.ts // From queue/types.ts
export interface QueueItem { export interface QueueItem {
id: string; id: string;
url: string; url: string;
status: QueueItemStatus; status: QueueItemStatus;
enqueuedAt: string; enqueuedAt: string;
// ... // ...
} }
export interface QueueStatusUpdate { export interface QueueStatusUpdate {
type: string; type: string;
itemId: string; itemId: string;
status: QueueItemStatus; status: QueueItemStatus;
// ... // ...
} }
export type QueueItemStatus = export type QueueItemStatus = 'pending' | 'in_progress' | 'completed' | 'failed';
| 'pending'
| 'in_progress'
| 'completed'
| 'failed';
// From extraction.ts // From extraction.ts
export interface ExtractedContent { export interface ExtractedContent {
text: string; text: string;
thumbnailUrl?: string; thumbnailUrl?: string;
} }
export type ProgressCallback = (event: ProgressEvent) => void; export type ProgressCallback = (event: ProgressEvent) => void;
``` ```
#### Zod Schemas #### Zod Schemas
- **PascalCase** with `Schema` suffix - **PascalCase** with `Schema` suffix
- Inferred types without suffix - Inferred types without suffix
**Examples:** **Examples:**
```typescript ```typescript
// From parser.ts // From parser.ts
const RecipeSchema = z.object({ const RecipeSchema = z.object({
name: z.string(), name: z.string(),
description: z.string(), description: z.string(),
servings: z.number(), servings: z.number()
// ... // ...
}); });
export type Recipe = z.infer<typeof RecipeSchema>; export type Recipe = z.infer<typeof RecipeSchema>;
// From tandoor.ts // From tandoor.ts
const TandoorRecipeSchema = z.object({ const TandoorRecipeSchema = z.object({
// ... // ...
}); });
export type TandoorRecipe = z.infer<typeof TandoorRecipeSchema>; export type TandoorRecipe = z.infer<typeof TandoorRecipeSchema>;
@@ -163,35 +173,38 @@ export type TandoorRecipe = z.infer<typeof TandoorRecipeSchema>;
### Classes ### Classes
#### Class Names #### Class Names
- **PascalCase** for class names - **PascalCase** for class names
- Descriptive suffixes: `Manager`, `Service`, `Processor`, `Handler` - Descriptive suffixes: `Manager`, `Service`, `Processor`, `Handler`
**Examples:** **Examples:**
```typescript ```typescript
// From QueueManager.ts // From QueueManager.ts
export class QueueManager { export class QueueManager {
private items: Map<string, QueueItem> = new Map(); private items: Map<string, QueueItem> = new Map();
// ... // ...
} }
// From QueueProcessor.ts // From QueueProcessor.ts
export class QueueProcessor { export class QueueProcessor {
private processing: Set<string> = new Set(); private processing: Set<string> = new Set();
// ... // ...
} }
// From PushNotificationService.ts // From PushNotificationService.ts
class PushNotificationService { class PushNotificationService {
private subscriptions: Map<string, PushSubscription> = new Map(); private subscriptions: Map<string, PushSubscription> = new Map();
// ... // ...
} }
``` ```
#### Singleton Export Pattern #### Singleton Export Pattern
```typescript ```typescript
// Class definition // Class definition
export class QueueManager { export class QueueManager {
// Implementation // Implementation
} }
// Singleton instance export // Singleton instance export
@@ -203,6 +216,7 @@ export const queueManager = new QueueManager();
## Indentation & Formatting ## Indentation & Formatting
### General Rules ### General Rules
- **Indentation:** 2 spaces (enforced by Prettier) - **Indentation:** 2 spaces (enforced by Prettier)
- **No tabs** - **No tabs**
- **Max line length:** 100 characters (soft limit, not enforced) - **Max line length:** 100 characters (soft limit, not enforced)
@@ -213,6 +227,7 @@ export const queueManager = new QueueManager();
### Code Examples ### Code Examples
#### Function Declarations #### Function Declarations
```typescript ```typescript
// From QueueManager.ts // From QueueManager.ts
enqueue(url: string): QueueItem { enqueue(url: string): QueueItem {
@@ -234,43 +249,45 @@ enqueue(url: string): QueueItem {
retryCount: 0, retryCount: 0,
maxRetries: 3 maxRetries: 3
}; };
this.items.set(item.id, item); this.items.set(item.id, item);
return item; return item;
} }
``` ```
#### Async Functions #### Async Functions
```typescript ```typescript
// From extraction.ts // From extraction.ts
export async function extractTextAndThumbnail( export async function extractTextAndThumbnail(
url: string, url: string,
onProgress?: ProgressCallback onProgress?: ProgressCallback
): Promise<ExtractedContent> { ): Promise<ExtractedContent> {
const browser = await getBrowser(); const browser = await getBrowser();
const context = await createBrowserContext(browser); const context = await createBrowserContext(browser);
const page = await context.newPage(); const page = await context.newPage();
try { try {
await page.goto(url, { waitUntil: 'networkidle' }); await page.goto(url, { waitUntil: 'networkidle' });
// ... // ...
} finally { } finally {
await context.close(); await context.close();
} }
} }
``` ```
#### Object Destructuring #### Object Destructuring
```typescript ```typescript
// From route handlers // From route handlers
export const POST: RequestHandler = async ({ request }) => { export const POST: RequestHandler = async ({ request }) => {
const { url } = await request.json(); const { url } = await request.json();
// ... // ...
}; };
export const GET: RequestHandler = async ({ params }) => { export const GET: RequestHandler = async ({ params }) => {
const { id } = params; const { id } = params;
// ... // ...
}; };
``` ```
@@ -279,12 +296,14 @@ export const GET: RequestHandler = async ({ params }) => {
## Import Patterns ## Import Patterns
### Import Order ### Import Order
1. External dependencies (Node.js built-ins, npm packages) 1. External dependencies (Node.js built-ins, npm packages)
2. SvelteKit imports (`$lib`, `$app`, `$env`) 2. SvelteKit imports (`$lib`, `$app`, `$env`)
3. Relative imports (`./ `, `../`) 3. Relative imports (`./ `, `../`)
4. Type imports (separate from value imports when beneficial) 4. Type imports (separate from value imports when beneficial)
**Example:** **Example:**
```typescript ```typescript
// From QueueProcessor.ts // From QueueProcessor.ts
@@ -307,6 +326,7 @@ import type { QueueItem } from './types';
### Import Styles ### Import Styles
#### Named Imports (Preferred) #### Named Imports (Preferred)
```typescript ```typescript
import { json } from '@sveltejs/kit'; import { json } from '@sveltejs/kit';
import { queueManager } from '$lib/server/queue/QueueManager'; import { queueManager } from '$lib/server/queue/QueueManager';
@@ -314,12 +334,14 @@ import { validateInstagramUrl } from '$lib/server/validation/instagram-url';
``` ```
#### Type-Only Imports #### Type-Only Imports
```typescript ```typescript
import type { RequestHandler } from './$types'; import type { RequestHandler } from './$types';
import type { QueueItem, QueueItemStatus } from './types'; import type { QueueItem, QueueItemStatus } from './types';
``` ```
#### Default Imports #### Default Imports
```typescript ```typescript
import OpenAI from 'openai'; import OpenAI from 'openai';
import fs from 'fs'; import fs from 'fs';
@@ -329,6 +351,7 @@ import path from 'path';
### Export Patterns ### Export Patterns
#### Named Exports (Preferred) #### Named Exports (Preferred)
```typescript ```typescript
// Export functions // Export functions
export async function extractRecipe(text: string): Promise<Recipe> { ... } export async function extractRecipe(text: string): Promise<Recipe> { ... }
@@ -345,6 +368,7 @@ export interface QueueItem { ... }
``` ```
#### Singleton Pattern Export #### Singleton Pattern Export
```typescript ```typescript
// Define class // Define class
export class QueueManager { ... } export class QueueManager { ... }
@@ -358,16 +382,18 @@ export const queueManager = new QueueManager();
## Comments & Documentation ## Comments & Documentation
### JSDoc Style ### JSDoc Style
Used extensively for public APIs and exported functions. Used extensively for public APIs and exported functions.
**Function Documentation:** **Function Documentation:**
```typescript
````typescript
/** /**
* Add URL to processing queue * Add URL to processing queue
* *
* @param url - Instagram URL to process * @param url - Instagram URL to process
* @returns Newly created queue item * @returns Newly created queue item
* *
* @example * @example
* ```typescript * ```typescript
* const item = queueManager.enqueue('https://instagram.com/p/abc123'); * const item = queueManager.enqueue('https://instagram.com/p/abc123');
@@ -377,41 +403,43 @@ Used extensively for public APIs and exported functions.
enqueue(url: string): QueueItem { enqueue(url: string): QueueItem {
// Implementation // Implementation
} }
``` ````
**Class Documentation:** **Class Documentation:**
```typescript
````typescript
/** /**
* Singleton queue manager for processing Instagram URLs * Singleton queue manager for processing Instagram URLs
* *
* Features: * Features:
* - FIFO queue with unique IDs * - FIFO queue with unique IDs
* - Status tracking and updates * - Status tracking and updates
* - Progress event accumulation * - Progress event accumulation
* - Retry support for failed items * - Retry support for failed items
* - Pub/sub for real-time updates * - Pub/sub for real-time updates
* *
* @example * @example
* ```typescript * ```typescript
* import { queueManager } from './QueueManager'; * import { queueManager } from './QueueManager';
* *
* // Add item to queue * // Add item to queue
* const item = queueManager.enqueue('https://instagram.com/p/abc123'); * const item = queueManager.enqueue('https://instagram.com/p/abc123');
* ``` * ```
*/ */
export class QueueManager { export class QueueManager {
// Implementation // Implementation
} }
``` ````
**Module-Level Documentation:** **Module-Level Documentation:**
```typescript ```typescript
/** /**
* Queue Manager - Core queue operations and event management * Queue Manager - Core queue operations and event management
* *
* Manages an in-memory queue of Instagram URL processing jobs. * Manages an in-memory queue of Instagram URL processing jobs.
* Provides CRUD operations and pub/sub mechanism for queue updates. * Provides CRUD operations and pub/sub mechanism for queue updates.
* *
* Architecture: Domain Layer (Hexagonal Architecture) * Architecture: Domain Layer (Hexagonal Architecture)
* - Port: Defines queue operations interface * - Port: Defines queue operations interface
* - Implementation: In-memory Map-based storage * - Implementation: In-memory Map-based storage
@@ -421,19 +449,21 @@ export class QueueManager {
### Inline Comments ### Inline Comments
#### Single-line Comments #### Single-line Comments
```typescript ```typescript
// Set restrictive permissions // Set restrictive permissions
fs.chmodSync(authFile, 0o600); fs.chmodSync(authFile, 0o600);
// FIFO order - get oldest pending item // FIFO order - get oldest pending item
const pendingItems = Array.from(this.items.values()) const pendingItems = Array.from(this.items.values()).filter((item) => item.status === 'pending');
.filter(item => item.status === 'pending');
``` ```
#### Block Comments (Avoided) #### Block Comments (Avoided)
Single-line comments preferred. Block comments used only for large comment blocks or temporarily disabling code during development. Single-line comments preferred. Block comments used only for large comment blocks or temporarily disabling code during development.
### TODO Comments ### TODO Comments
```typescript ```typescript
// TODO: Add retry logic with exponential backoff // TODO: Add retry logic with exponential backoff
// FIXME: Handle race condition when multiple workers dequeue // FIXME: Handle race condition when multiple workers dequeue
@@ -446,17 +476,19 @@ Single-line comments preferred. Block comments used only for large comment block
### Type Safety ### Type Safety
#### Strict Mode Enabled #### Strict Mode Enabled
```json ```json
// tsconfig.json // tsconfig.json
{ {
"compilerOptions": { "compilerOptions": {
"strict": true, "strict": true,
"forceConsistentCasingInFileNames": true "forceConsistentCasingInFileNames": true
} }
} }
``` ```
#### Type Annotations #### Type Annotations
```typescript ```typescript
// Explicit return types for public functions // Explicit return types for public functions
export async function extractRecipe(text: string): Promise<Recipe> { ... } export async function extractRecipe(text: string): Promise<Recipe> { ... }
@@ -469,35 +501,24 @@ const items = queueManager.getAll(); // Type inferred
``` ```
### Union Types ### Union Types
```typescript ```typescript
export type QueueItemStatus = export type QueueItemStatus = 'pending' | 'in_progress' | 'completed' | 'failed';
| 'pending'
| 'in_progress'
| 'completed'
| 'failed';
export type ProcessingPhase = export type ProcessingPhase = 'extraction' | 'parsing' | 'uploading';
| 'extraction'
| 'parsing'
| 'uploading';
export type ProgressEventType = export type ProgressEventType = 'status' | 'method' | 'retry' | 'error' | 'thumbnail' | 'complete';
| 'status'
| 'method'
| 'retry'
| 'error'
| 'thumbnail'
| 'complete';
``` ```
### Generics ### Generics
```typescript ```typescript
// Generic function // Generic function
async function fetchFromTandoor<T>( async function fetchFromTandoor<T>(
url: string, url: string,
options: Partial<RequestInit> = { method: 'GET' } options: Partial<RequestInit> = { method: 'GET' }
): Promise<{ ok: boolean; data?: T; error?: string }> { ): Promise<{ ok: boolean; data?: T; error?: string }> {
// Implementation // Implementation
} }
``` ```
@@ -508,6 +529,7 @@ async function fetchFromTandoor<T>(
### Runes (Reactivity) ### Runes (Reactivity)
#### $state (Reactive Variables) #### $state (Reactive Variables)
```svelte ```svelte
<script lang="ts"> <script lang="ts">
let count = $state(0); let count = $state(0);
@@ -516,13 +538,14 @@ async function fetchFromTandoor<T>(
``` ```
#### $props (Component Props) #### $props (Component Props)
```svelte ```svelte
<script lang="ts"> <script lang="ts">
let { let {
recipe = null, recipe = null,
tandoorEnabled = false, tandoorEnabled = false,
onRetry, onRetry,
onImportToTandoor onImportToTandoor
} = $props<{ } = $props<{
recipe: Recipe | null; recipe: Recipe | null;
tandoorEnabled: boolean; tandoorEnabled: boolean;
@@ -533,6 +556,7 @@ async function fetchFromTandoor<T>(
``` ```
#### $derived (Computed Values) #### $derived (Computed Values)
```svelte ```svelte
<script lang="ts"> <script lang="ts">
let count = $state(0); let count = $state(0);
@@ -541,10 +565,11 @@ async function fetchFromTandoor<T>(
``` ```
#### $effect (Side Effects) #### $effect (Side Effects)
```svelte ```svelte
<script lang="ts"> <script lang="ts">
let url = $state(''); let url = $state('');
$effect(() => { $effect(() => {
console.log('URL changed:', url); console.log('URL changed:', url);
}); });
@@ -552,25 +577,26 @@ async function fetchFromTandoor<T>(
``` ```
### Component Structure ### Component Structure
```svelte ```svelte
<script lang="ts"> <script lang="ts">
// Imports // Imports
import { onMount } from 'svelte'; import { onMount } from 'svelte';
// Props // Props
let { items } = $props<{ items: Item[] }>(); let { items } = $props<{ items: Item[] }>();
// State // State
let loading = $state(false); let loading = $state(false);
// Derived state // Derived state
let count = $derived(items.length); let count = $derived(items.length);
// Functions // Functions
function handleClick() { function handleClick() {
// ... // ...
} }
// Effects // Effects
$effect(() => { $effect(() => {
// Side effects // Side effects
@@ -593,46 +619,47 @@ async function fetchFromTandoor<T>(
## Error Handling ## Error Handling
### Custom Error Classes ### Custom Error Classes
```typescript ```typescript
// From api/errors.ts // From api/errors.ts
export class ValidationError extends Error { export class ValidationError extends Error {
constructor(message: string) { constructor(message: string) {
super(message); super(message);
this.name = 'ValidationError'; this.name = 'ValidationError';
} }
} }
export class NotFoundError extends Error { export class NotFoundError extends Error {
constructor(resource: string) { constructor(resource: string) {
super(`${resource} not found`); super(`${resource} not found`);
this.name = 'NotFoundError'; this.name = 'NotFoundError';
} }
} }
export class ConflictError extends Error { export class ConflictError extends Error {
constructor(message: string) { constructor(message: string) {
super(message); super(message);
this.name = 'ConflictError'; this.name = 'ConflictError';
} }
} }
``` ```
### Try-Catch Pattern ### Try-Catch Pattern
```typescript ```typescript
export const POST: RequestHandler = async ({ request }) => { export const POST: RequestHandler = async ({ request }) => {
try { try {
const { url } = await request.json(); const { url } = await request.json();
if (!url) { if (!url) {
throw new ValidationError('URL is required'); throw new ValidationError('URL is required');
} }
const item = queueManager.enqueue(url); const item = queueManager.enqueue(url);
return json(item, { status: 201 }); return json(item, { status: 201 });
} catch (error) {
} catch (error) { return handleApiError(error);
return handleApiError(error); }
}
}; };
``` ```
@@ -641,14 +668,16 @@ export const POST: RequestHandler = async ({ request }) => {
## Linting Configuration ## Linting Configuration
### ESLint ### ESLint
**Config:** `eslint.config.js` **Config:** `eslint.config.js`
- Base: `@eslint/js` recommended - Base: `@eslint/js` recommended
- TypeScript: `typescript-eslint` recommended - TypeScript: `typescript-eslint` recommended
- Svelte: `eslint-plugin-svelte` recommended - Svelte: `eslint-plugin-svelte` recommended
- Formatting: `eslint-config-prettier` - Formatting: `eslint-config-prettier`
**Rules:** **Rules:**
```javascript ```javascript
{ {
rules: { rules: {
@@ -658,15 +687,16 @@ export const POST: RequestHandler = async ({ request }) => {
``` ```
### Prettier ### Prettier
**Config:** `.prettierrc` **Config:** `.prettierrc`
```json ```json
{ {
"useTabs": false, "useTabs": false,
"singleQuote": true, "singleQuote": true,
"trailingComma": "es5", "trailingComma": "es5",
"printWidth": 100, "printWidth": 100,
"plugins": ["prettier-plugin-svelte", "prettier-plugin-tailwindcss"] "plugins": ["prettier-plugin-svelte", "prettier-plugin-tailwindcss"]
} }
``` ```
@@ -675,38 +705,40 @@ export const POST: RequestHandler = async ({ request }) => {
## Testing Conventions ## Testing Conventions
### Test Structure ### Test Structure
```typescript ```typescript
import { describe, it, expect, beforeEach, afterEach } from 'vitest'; import { describe, it, expect, beforeEach, afterEach } from 'vitest';
describe('QueueManager', () => { describe('QueueManager', () => {
let manager: QueueManager; let manager: QueueManager;
beforeEach(() => { beforeEach(() => {
manager = new QueueManager(); manager = new QueueManager();
}); });
it('should enqueue items', () => { it('should enqueue items', () => {
const item = manager.enqueue('https://instagram.com/p/test'); const item = manager.enqueue('https://instagram.com/p/test');
expect(item.status).toBe('pending'); expect(item.status).toBe('pending');
}); });
it('should dequeue items in FIFO order', () => { it('should dequeue items in FIFO order', () => {
manager.enqueue('url1'); manager.enqueue('url1');
manager.enqueue('url2'); manager.enqueue('url2');
const first = manager.dequeue(); const first = manager.dequeue();
expect(first?.url).toBe('url1'); expect(first?.url).toBe('url1');
}); });
}); });
``` ```
### Mock Pattern ### Mock Pattern
```typescript ```typescript
vi.mock('$lib/server/extraction', () => ({ vi.mock('$lib/server/extraction', () => ({
extractTextAndThumbnail: vi.fn().mockResolvedValue({ extractTextAndThumbnail: vi.fn().mockResolvedValue({
text: 'Mock text', text: 'Mock text',
thumbnailUrl: 'https://example.com/thumb.jpg' thumbnailUrl: 'https://example.com/thumb.jpg'
}) })
})); }));
``` ```
@@ -715,14 +747,15 @@ vi.mock('$lib/server/extraction', () => ({
## File Headers ## File Headers
### Module Documentation Pattern ### Module Documentation Pattern
Every major module includes a header comment: Every major module includes a header comment:
```typescript ```typescript
/** /**
* Module Name - Brief Description * Module Name - Brief Description
* *
* Detailed description of the module's purpose and functionality. * Detailed description of the module's purpose and functionality.
* *
* Architecture: Layer Name (Hexagonal Architecture) * Architecture: Layer Name (Hexagonal Architecture)
* - Port: Description of port interface * - Port: Description of port interface
* - Implementation: Description of concrete implementation * - Implementation: Description of concrete implementation
@@ -730,13 +763,14 @@ Every major module includes a header comment:
``` ```
**Example:** **Example:**
```typescript ```typescript
/** /**
* Queue Manager - Core queue operations and event management * Queue Manager - Core queue operations and event management
* *
* Manages an in-memory queue of Instagram URL processing jobs. * Manages an in-memory queue of Instagram URL processing jobs.
* Provides CRUD operations and pub/sub mechanism for queue updates. * Provides CRUD operations and pub/sub mechanism for queue updates.
* *
* Architecture: Domain Layer (Hexagonal Architecture) * Architecture: Domain Layer (Hexagonal Architecture)
* - Port: Defines queue operations interface * - Port: Defines queue operations interface
* - Implementation: In-memory Map-based storage * - Implementation: In-memory Map-based storage
@@ -748,6 +782,7 @@ Every major module includes a header comment:
## Additional Conventions ## Additional Conventions
### Environment Variables ### Environment Variables
```typescript ```typescript
import { env } from '$env/dynamic/private'; import { env } from '$env/dynamic/private';
@@ -756,32 +791,37 @@ const tandoorUrl = env.TANDOOR_URL || null;
``` ```
### Date Handling ### Date Handling
ISO8601 strings throughout the application: ISO8601 strings throughout the application:
```typescript ```typescript
const now = new Date().toISOString(); const now = new Date().toISOString();
// Output: "2026-02-15T12:30:45.123Z" // Output: "2026-02-15T12:30:45.123Z"
``` ```
### Null vs Undefined ### Null vs Undefined
- `null`: Intentional absence of value - `null`: Intentional absence of value
- `undefined`: Not yet initialized or optional parameters - `undefined`: Not yet initialized or optional parameters
- Prefer `null` for API responses and data structures - Prefer `null` for API responses and data structures
### Async/Await ### Async/Await
Always preferred over Promise chains: Always preferred over Promise chains:
```typescript ```typescript
// Preferred // Preferred
async function fetchData() { async function fetchData() {
const response = await fetch(url); const response = await fetch(url);
const data = await response.json(); const data = await response.json();
return data; return data;
} }
// Avoid // Avoid
function fetchData() { function fetchData() {
return fetch(url) return fetch(url)
.then(response => response.json()) .then((response) => response.json())
.then(data => data); .then((data) => data);
} }
``` ```

611
docs/FINDINGS.md
View File

File diff suppressed because it is too large Load Diff

135
docs/MIGRATION.md
View File

@@ -11,6 +11,7 @@ The migration transformed InstaRecipe from a blocking, synchronous extraction sy
### Architecture Transformation ### Architecture Transformation
**Before: Synchronous System** **Before: Synchronous System**
``` ```
User Request → Direct Processing → Response (wait 30-60s) User Request → Direct Processing → Response (wait 30-60s)
↓ ↓ ↓ ↓ ↓ ↓
@@ -18,6 +19,7 @@ User Request → Direct Processing → Response (wait 30-60s)
``` ```
**After: Async Queue System** **After: Async Queue System**
``` ```
User Request → Queue Item Created → Immediate Response User Request → Queue Item Created → Immediate Response
↓ ↓ ↓ ↓ ↓ ↓
@@ -60,6 +62,7 @@ User Request → Queue Item Created → Immediate Response
### New Endpoints ### New Endpoints
#### Queue Management #### Queue Management
```typescript ```typescript
// Enqueue URL for processing // Enqueue URL for processing
POST /api/queue POST /api/queue
@@ -84,6 +87,7 @@ Events: connection, queue-update, ping
``` ```
#### Push Notifications #### Push Notifications
```typescript ```typescript
// Subscribe to push notifications // Subscribe to push notifications
POST /api/notifications/subscribe POST /api/notifications/subscribe
@@ -101,11 +105,11 @@ These endpoints are marked for removal and should not be used in new code:
```typescript ```typescript
// ❌ DEPRECATED: Synchronous extraction // ❌ DEPRECATED: Synchronous extraction
POST /api/extract POST / api / extract;
// 👉 Use: POST /api/queue // 👉 Use: POST /api/queue
// ❌ DEPRECATED: Long-polling progress // ❌ DEPRECATED: Long-polling progress
GET /api/extract-stream GET / api / extract - stream;
// 👉 Use: GET /api/queue/stream // 👉 Use: GET /api/queue/stream
``` ```
@@ -117,33 +121,33 @@ New queue items follow this structure:
```typescript ```typescript
interface QueueItem { interface QueueItem {
id: string; // UUID v4 id: string; // UUID v4
url: string; // Instagram URL url: string; // Instagram URL
status: 'pending' | 'in_progress' | 'success' | 'error' | 'unhealthy'; status: 'pending' | 'in_progress' | 'success' | 'error' | 'unhealthy';
// Processing phases with individual progress // Processing phases with individual progress
phases: Array<{ phases: Array<{
name: 'extraction' | 'parsing' | 'uploading'; name: 'extraction' | 'parsing' | 'uploading';
status: 'pending' | 'in_progress' | 'completed' | 'error'; status: 'pending' | 'in_progress' | 'completed' | 'error';
startedAt?: string; startedAt?: string;
completedAt?: string; completedAt?: string;
progress?: number; // 0-100 progress?: number; // 0-100
}>; }>;
// Results (populated on success) // Results (populated on success)
results?: { results?: {
recipe?: Recipe; // Extracted recipe data recipe?: Recipe; // Extracted recipe data
tandoorUrl?: string; // Link to uploaded recipe tandoorUrl?: string; // Link to uploaded recipe
extractedText?: string; // Raw extracted text extractedText?: string; // Raw extracted text
thumbnail?: string; // Image URL thumbnail?: string; // Image URL
}; };
// Error information // Error information
error?: string; error?: string;
// Timestamps // Timestamps
createdAt: string; createdAt: string;
updatedAt: string; updatedAt: string;
} }
``` ```
@@ -167,33 +171,35 @@ interface QueueStatusUpdate {
### For Frontend Applications ### For Frontend Applications
1. **Replace Synchronous Calls** 1. **Replace Synchronous Calls**
```typescript ```typescript
// ❌ Old synchronous approach // ❌ Old synchronous approach
const response = await fetch('/api/extract', { const response = await fetch('/api/extract', {
method: 'POST', method: 'POST',
body: JSON.stringify({ url }) body: JSON.stringify({ url })
}); });
const result = await response.json(); // Wait 30-60 seconds const result = await response.json(); // Wait 30-60 seconds
// ✅ New async queue approach // ✅ New async queue approach
const response = await fetch('/api/queue', { const response = await fetch('/api/queue', {
method: 'POST', method: 'POST',
body: JSON.stringify({ url }) body: JSON.stringify({ url })
}); });
const queueItem = await response.json(); // Immediate response const queueItem = await response.json(); // Immediate response
// Navigate to dashboard for real-time updates // Navigate to dashboard for real-time updates
window.location.href = `/?highlight=${queueItem.id}`; window.location.href = `/?highlight=${queueItem.id}`;
``` ```
2. **Add Real-time Updates** 2. **Add Real-time Updates**
```typescript ```typescript
// Setup Server-Sent Events for progress tracking // Setup Server-Sent Events for progress tracking
const eventSource = new EventSource(`/api/queue/stream?itemId=${itemId}`); const eventSource = new EventSource(`/api/queue/stream?itemId=${itemId}`);
eventSource.addEventListener('queue-update', (event) => { eventSource.addEventListener('queue-update', (event) => {
const update = JSON.parse(event.data); const update = JSON.parse(event.data);
updateUI(update); updateUI(update);
}); });
``` ```
@@ -201,36 +207,37 @@ interface QueueStatusUpdate {
```typescript ```typescript
// Handle different queue statuses // Handle different queue statuses
switch (item.status) { switch (item.status) {
case 'pending': case 'pending':
showPendingState(); showPendingState();
break; break;
case 'in_progress': case 'in_progress':
showProgressBar(item.phases); showProgressBar(item.phases);
break; break;
case 'success': case 'success':
showResults(item.results); showResults(item.results);
break; break;
case 'error': case 'error':
showErrorWithRetry(item.error, item.id); showErrorWithRetry(item.error, item.id);
break; break;
case 'unhealthy': case 'unhealthy':
showRetryableError(item.error, item.id); showRetryableError(item.error, item.id);
break; break;
} }
``` ```
### For Backend Integrations ### For Backend Integrations
1. **Update API Calls** 1. **Update API Calls**
```python ```python
# ❌ Old synchronous API # ❌ Old synchronous API
response = requests.post('/api/extract', json={'url': url}) response = requests.post('/api/extract', json={'url': url})
# This would block for 30-60 seconds # This would block for 30-60 seconds
# ✅ New async queue API # ✅ New async queue API
response = requests.post('/api/queue', json={'url': url}) response = requests.post('/api/queue', json={'url': url})
queue_item = response.json() queue_item = response.json()
# Poll or use SSE for updates # Poll or use SSE for updates
while True: while True:
item = requests.get(f'/api/queue/{queue_item["id"]}').json() item = requests.get(f'/api/queue/{queue_item["id"]}').json()
@@ -240,9 +247,10 @@ interface QueueStatusUpdate {
``` ```
2. **Implement SSE Client** (Python example) 2. **Implement SSE Client** (Python example)
```python ```python
import sseclient import sseclient
def listen_to_queue_updates(item_id): def listen_to_queue_updates(item_id):
messages = sseclient.SSEClient(f'/api/queue/stream?itemId={item_id}') messages = sseclient.SSEClient(f'/api/queue/stream?itemId={item_id}')
for msg in messages: for msg in messages:
@@ -266,7 +274,7 @@ QUEUE_TIMEOUT_MS=30000 # Processing timeout
QUEUE_RETRY_ATTEMPTS=3 # Maximum retry attempts QUEUE_RETRY_ATTEMPTS=3 # Maximum retry attempts
# Push notification settings (optional) # Push notification settings (optional)
VAPID_PUBLIC_KEY=BDummyPublicKey... VAPID_PUBLIC_KEY=BDummyPublicKey...
VAPID_PRIVATE_KEY=DummyPrivateKey... VAPID_PRIVATE_KEY=DummyPrivateKey...
# Existing LLM and Tandoor settings remain the same # Existing LLM and Tandoor settings remain the same
@@ -306,7 +314,7 @@ npm test
# Test specific components # Test specific components
npm test queue-manager npm test queue-manager
npm test queue-processor npm test queue-processor
npm test queue-api npm test queue-api
npm test queue-sse npm test queue-sse
``` ```
@@ -314,18 +322,21 @@ npm test queue-sse
## Performance Considerations ## Performance Considerations
### Before Migration ### Before Migration
- **Blocking Operations**: Each request blocked a server thread - **Blocking Operations**: Each request blocked a server thread
- **Single Processing**: One extraction at a time - **Single Processing**: One extraction at a time
- **No Progress**: Users waited without feedback - **No Progress**: Users waited without feedback
- **Memory Usage**: High memory usage during long operations - **Memory Usage**: High memory usage during long operations
### After Migration ### After Migration
- **Non-blocking**: Requests return immediately - **Non-blocking**: Requests return immediately
- **Concurrent Processing**: Multiple extractions in parallel - **Concurrent Processing**: Multiple extractions in parallel
- **Real-time Feedback**: Live progress updates - **Real-time Feedback**: Live progress updates
- **Efficient Memory**: Event-driven, minimal memory footprint - **Efficient Memory**: Event-driven, minimal memory footprint
### Performance Metrics ### Performance Metrics
- **Response Time**: 50ms (queue) vs 30-60s (synchronous) - **Response Time**: 50ms (queue) vs 30-60s (synchronous)
- **Throughput**: 2x concurrent processing vs 1x sequential - **Throughput**: 2x concurrent processing vs 1x sequential
- **User Experience**: Immediate feedback vs long waiting - **User Experience**: Immediate feedback vs long waiting
@@ -336,11 +347,13 @@ npm test queue-sse
If issues arise, the system can be rolled back by: If issues arise, the system can be rolled back by:
1. **Disable Queue Processing** 1. **Disable Queue Processing**
```env ```env
QUEUE_PROCESSING_ENABLED=false QUEUE_PROCESSING_ENABLED=false
``` ```
2. **Re-enable Legacy Endpoints** (if preserved) 2. **Re-enable Legacy Endpoints** (if preserved)
```typescript ```typescript
// Temporary fallback to synchronous processing // Temporary fallback to synchronous processing
app.post('/api/extract', legacyExtractHandler); app.post('/api/extract', legacyExtractHandler);
@@ -389,10 +402,10 @@ curl -X POST https://localhost:5173/api/notifications/vapid-key
The migration to an async queue system represents a significant architectural improvement that provides: The migration to an async queue system represents a significant architectural improvement that provides:
- **Better User Experience**: Immediate responses and real-time progress - **Better User Experience**: Immediate responses and real-time progress
- **Improved Reliability**: Error recovery and retry mechanisms - **Improved Reliability**: Error recovery and retry mechanisms
- **Enhanced Performance**: Concurrent processing and resource efficiency - **Enhanced Performance**: Concurrent processing and resource efficiency
- **Modern Features**: Push notifications and PWA capabilities - **Modern Features**: Push notifications and PWA capabilities
The new system maintains backward compatibility during the transition period while providing a clear migration path for all integrations. The new system maintains backward compatibility during the transition period while providing a clear migration path for all integrations.
For questions or issues during migration, refer to the comprehensive test suite and documentation, or open an issue in the project repository. For questions or issues during migration, refer to the comprehensive test suite and documentation, or open an issue in the project repository.

103
docs/SVELTEKIT_SSR_GUIDE.md
View File

@@ -3,6 +3,7 @@
This guide documents SSR-safe patterns and anti-patterns for InstaRecipe development. All developers should follow these guidelines to prevent SSR errors. This guide documents SSR-safe patterns and anti-patterns for InstaRecipe development. All developers should follow these guidelines to prevent SSR errors.
## Table of Contents ## Table of Contents
- [Core Principle](#core-principle) - [Core Principle](#core-principle)
- [Browser API Detection](#browser-api-detection) - [Browser API Detection](#browser-api-detection)
- [Lifecycle Hooks](#lifecycle-hooks) - [Lifecycle Hooks](#lifecycle-hooks)
@@ -18,6 +19,7 @@ This guide documents SSR-safe patterns and anti-patterns for InstaRecipe develop
**SvelteKit renders components on both the server and client.** Any browser-only APIs must be guarded or used in browser-only contexts. **SvelteKit renders components on both the server and client.** Any browser-only APIs must be guarded or used in browser-only contexts.
### Browser-Only APIs (Require Guards) ### Browser-Only APIs (Require Guards)
- `window.*` - `window.*`
- `document.*` - `document.*`
- `localStorage`, `sessionStorage` - `localStorage`, `sessionStorage`
@@ -36,8 +38,8 @@ This guide documents SSR-safe patterns and anti-patterns for InstaRecipe develop
import { browser } from '$app/environment'; import { browser } from '$app/environment';
if (browser) { if (browser) {
// Safe: only runs in browser // Safe: only runs in browser
const data = localStorage.getItem('key'); const data = localStorage.getItem('key');
} }
``` ```
@@ -49,14 +51,14 @@ if (browser) {
<script lang="ts"> <script lang="ts">
import { browser } from '$app/environment'; import { browser } from '$app/environment';
import { onMount } from 'svelte'; import { onMount } from 'svelte';
let eventSource = $state<EventSource | null>(null); let eventSource = $state<EventSource | null>(null);
function startSSEConnection() { function startSSEConnection() {
if (!browser) return; // ✅ Guard if (!browser) return; // ✅ Guard
eventSource = new EventSource('/api/stream'); eventSource = new EventSource('/api/stream');
} }
onMount(() => { onMount(() => {
if (browser) { // ✅ Explicit guard if (browser) { // ✅ Explicit guard
startSSEConnection(); startSSEConnection();
@@ -72,6 +74,7 @@ if (browser) {
### `onMount` - Browser-Only Lifecycle ### `onMount` - Browser-Only Lifecycle
**Use `onMount` for:** **Use `onMount` for:**
- Browser API initialization - Browser API initialization
- Timer setup (`setInterval`, `setTimeout`) - Timer setup (`setInterval`, `setTimeout`)
- Event listener registration - Event listener registration
@@ -81,12 +84,12 @@ if (browser) {
import { onMount } from 'svelte'; import { onMount } from 'svelte';
onMount(() => { onMount(() => {
// ✅ Only runs in browser (built-in SSR guard) // ✅ Only runs in browser (built-in SSR guard)
const interval = setInterval(() => { const interval = setInterval(() => {
// Polling logic // Polling logic
}, 1000); }, 1000);
return () => clearInterval(interval); // Cleanup return () => clearInterval(interval); // Cleanup
}); });
``` ```
@@ -96,8 +99,8 @@ onMount(() => {
import { onDestroy } from 'svelte'; import { onDestroy } from 'svelte';
onDestroy(() => { onDestroy(() => {
// ✅ Safe for cleanup // ✅ Safe for cleanup
eventSource?.close(); eventSource?.close();
}); });
``` ```
@@ -117,7 +120,7 @@ let stored = $state(localStorage.getItem('key')); // SSR crash!
// ✅ DO: Load in onMount // ✅ DO: Load in onMount
let stored = $state<string | null>(null); let stored = $state<string | null>(null);
onMount(() => { onMount(() => {
stored = localStorage.getItem('key'); stored = localStorage.getItem('key');
}); });
``` ```
@@ -142,29 +145,31 @@ let userAgent = $derived(navigator.userAgent); // SSR crash!
```typescript ```typescript
// ❌ BAD: No browser guard // ❌ BAD: No browser guard
$effect(() => { $effect(() => {
setInterval(() => checkHealth(), 1000); // SSR crash! setInterval(() => checkHealth(), 1000); // SSR crash!
}); });
// ✅ GOOD: With browser guard // ✅ GOOD: With browser guard
$effect(() => { $effect(() => {
if (!browser) return; if (!browser) return;
const interval = setInterval(() => checkHealth(), 1000); const interval = setInterval(() => checkHealth(), 1000);
return () => clearInterval(interval); return () => clearInterval(interval);
}); });
// ✅ BETTER: Use onMount for initialization instead // ✅ BETTER: Use onMount for initialization instead
onMount(() => { onMount(() => {
const interval = setInterval(() => checkHealth(), 1000); const interval = setInterval(() => checkHealth(), 1000);
return () => clearInterval(interval); return () => clearInterval(interval);
}); });
``` ```
**When to use `$effect`:** **When to use `$effect`:**
- Synchronizing derived state - Synchronizing derived state
- DOM manipulation (with browser guard) - DOM manipulation (with browser guard)
- Reactive cleanup - Reactive cleanup
**When NOT to use `$effect`:** **When NOT to use `$effect`:**
- Initialization (use `onMount`) - Initialization (use `onMount`)
- API calls on mount (use `onMount`) - API calls on mount (use `onMount`)
- Timer setup (use `onMount`) - Timer setup (use `onMount`)
@@ -189,11 +194,13 @@ if (browser && eventSource?.readyState === EventSource.OPEN)
``` ```
**EventSource States:** **EventSource States:**
- `EventSource.CONNECTING = 0` - `EventSource.CONNECTING = 0`
- `EventSource.OPEN = 1` - `EventSource.OPEN = 1`
- `EventSource.CLOSED = 2` - `EventSource.CLOSED = 2`
**WebSocket States:** **WebSocket States:**
- `WebSocket.CONNECTING = 0` - `WebSocket.CONNECTING = 0`
- `WebSocket.OPEN = 1` - `WebSocket.OPEN = 1`
- `WebSocket.CLOSING = 2` - `WebSocket.CLOSING = 2`
@@ -220,8 +227,8 @@ const interval = setInterval(() => {}, 1000); // SSR crash!
// ✅ GOOD: In onMount // ✅ GOOD: In onMount
onMount(() => { onMount(() => {
const interval = setInterval(() => {}, 1000); const interval = setInterval(() => {}, 1000);
return () => clearInterval(interval); return () => clearInterval(interval);
}); });
``` ```
@@ -260,22 +267,23 @@ onMount(() => {
import { browser } from '$app/environment'; import { browser } from '$app/environment';
export class PushNotificationManager { export class PushNotificationManager {
private static instance: PushNotificationManager | null = null; private static instance: PushNotificationManager | null = null;
static getInstance() { static getInstance() {
if (!browser) return null; // ✅ Early return for SSR if (!browser) return null; // ✅ Early return for SSR
// ... rest of implementation // ... rest of implementation
} }
private loadStoredSubscription() { private loadStoredSubscription() {
if (!browser) return null; // ✅ Guard localStorage if (!browser) return null; // ✅ Guard localStorage
const stored = localStorage.getItem('pushSubscription'); const stored = localStorage.getItem('pushSubscription');
return stored ? JSON.parse(stored) : null; return stored ? JSON.parse(stored) : null;
} }
} }
``` ```
**Why it's good:** **Why it's good:**
- Guards all browser API access - Guards all browser API access
- Early returns prevent unnecessary code execution during SSR - Early returns prevent unnecessary code execution during SSR
- Defensive programming with null checks - Defensive programming with null checks
@@ -288,16 +296,16 @@ export class PushNotificationManager {
<script lang="ts"> <script lang="ts">
import { browser } from '$app/environment'; import { browser } from '$app/environment';
import { onMount } from 'svelte'; import { onMount } from 'svelte';
let eventSource = $state<EventSource | null>(null); let eventSource = $state<EventSource | null>(null);
onMount(async () => { onMount(async () => {
await loadQueueItems(); await loadQueueItems();
if (browser) { // ✅ Guard if (browser) { // ✅ Guard
startSSEConnection(); startSSEConnection();
} }
}); });
function startSSEConnection() { function startSSEConnection() {
if (!browser) return; // ✅ Double guard for safety if (!browser) return; // ✅ Double guard for safety
eventSource = new EventSource('/api/queue/stream'); eventSource = new EventSource('/api/queue/stream');
@@ -316,7 +324,7 @@ export class PushNotificationManager {
```svelte ```svelte
<script lang="ts"> <script lang="ts">
import { onMount } from 'svelte'; import { onMount } from 'svelte';
onMount(() => { onMount(() => {
// ✅ onMount only runs in browser // ✅ onMount only runs in browser
checkHealth(); // Initial check checkHealth(); // Initial check
@@ -327,6 +335,7 @@ export class PushNotificationManager {
``` ```
**Why it's good:** **Why it's good:**
- Uses `onMount` instead of `$effect` for initialization - Uses `onMount` instead of `$effect` for initialization
- Timer setup in browser-only context - Timer setup in browser-only context
- Proper cleanup with return function - Proper cleanup with return function
@@ -344,7 +353,7 @@ let theme = $derived(localStorage.getItem('theme'));
// ✅ DO // ✅ DO
let theme = $state<string | null>(null); let theme = $state<string | null>(null);
onMount(() => { onMount(() => {
theme = localStorage.getItem('theme'); theme = localStorage.getItem('theme');
}); });
``` ```
@@ -353,19 +362,19 @@ onMount(() => {
```typescript ```typescript
// ❌ DON'T // ❌ DON'T
$effect(() => { $effect(() => {
// Runs during SSR! // Runs during SSR!
fetch('/api/data'); fetch('/api/data');
}); });
// ✅ DO: Guard browser-specific side effects // ✅ DO: Guard browser-specific side effects
$effect(() => { $effect(() => {
if (!browser) return; if (!browser) return;
fetch('/api/data'); fetch('/api/data');
}); });
// ✅ BETTER: Use onMount for initialization // ✅ BETTER: Use onMount for initialization
onMount(() => { onMount(() => {
fetch('/api/data'); fetch('/api/data');
}); });
``` ```
@@ -387,8 +396,8 @@ const interval = setInterval(() => {}, 1000);
// ✅ DO // ✅ DO
onMount(() => { onMount(() => {
const interval = setInterval(() => {}, 1000); const interval = setInterval(() => {}, 1000);
return () => clearInterval(interval); return () => clearInterval(interval);
}); });
``` ```
@@ -408,6 +417,7 @@ Watch server console for errors during build and preview.
### 2. Check for Hydration Warnings ### 2. Check for Hydration Warnings
Open browser DevTools console and look for: Open browser DevTools console and look for:
- "Hydration failed" - "Hydration failed"
- "The server response doesn't match the client content" - "The server response doesn't match the client content"
@@ -422,6 +432,7 @@ grep -r "navigator\." src/routes --include="*.svelte"
``` ```
Then verify each usage is either: Then verify each usage is either:
- In an event handler (safe) - In an event handler (safe)
- In `onMount` (safe) - In `onMount` (safe)
- Guarded with `if (browser)` (safe) - Guarded with `if (browser)` (safe)

158
docs/TESTING.md
View File

@@ -7,7 +7,7 @@ This guide explains how to properly mock dependencies when testing SvelteKit app
SvelteKit has a unique architecture where code can run on both server and client. This affects how we mock: SvelteKit has a unique architecture where code can run on both server and client. This affects how we mock:
1. **Server-only modules** (`$lib/server/*`, `*.server.ts`) - Only run on server 1. **Server-only modules** (`$lib/server/*`, `*.server.ts`) - Only run on server
2. **Universal modules** - Can run on both server and client 2. **Universal modules** - Can run on both server and client
3. **Environment variables** - Different modules for static vs dynamic access 3. **Environment variables** - Different modules for static vs dynamic access
## Key Principles ## Key Principles
@@ -32,12 +32,12 @@ SvelteKit has a unique architecture where code can run on both server and client
import { env } from '$env/dynamic/private'; import { env } from '$env/dynamic/private';
export const queueConfig = { export const queueConfig = {
concurrency: parseInt(env.QUEUE_CONCURRENCY || '2', 10), concurrency: parseInt(env.QUEUE_CONCURRENCY || '2', 10),
maxRetries: parseInt(env.QUEUE_MAX_RETRIES || '3', 10), maxRetries: parseInt(env.QUEUE_MAX_RETRIES || '3', 10),
tandoor: { tandoor: {
enabled: !!env.TANDOOR_TOKEN, enabled: !!env.TANDOOR_TOKEN,
token: env.TANDOOR_TOKEN || null token: env.TANDOOR_TOKEN || null
} }
}; };
``` ```
@@ -49,21 +49,21 @@ import * as queueConfigModule from '$lib/server/queue/config';
// Mock the config module // Mock the config module
vi.mock('$lib/server/queue/config', () => ({ vi.mock('$lib/server/queue/config', () => ({
queueConfig: { queueConfig: {
concurrency: 2, concurrency: 2,
maxRetries: 3, maxRetries: 3,
tandoor: { enabled: true, token: 'test-token' } tandoor: { enabled: true, token: 'test-token' }
} }
})); }));
describe('QueueProcessor', () => { describe('QueueProcessor', () => {
beforeEach(() => { beforeEach(() => {
vi.clearAllMocks(); vi.clearAllMocks();
}); });
afterEach(() => { afterEach(() => {
vi.restoreAllMocks(); vi.restoreAllMocks();
}); });
}); });
``` ```
@@ -78,10 +78,10 @@ import { vi } from 'vitest';
// IMPORTANT: Mock BEFORE importing the module that uses it // IMPORTANT: Mock BEFORE importing the module that uses it
vi.mock('$lib/server/extraction', () => ({ vi.mock('$lib/server/extraction', () => ({
extractTextAndThumbnail: vi.fn().mockResolvedValue({ extractTextAndThumbnail: vi.fn().mockResolvedValue({
bodyText: 'Mock recipe text', bodyText: 'Mock recipe text',
thumbnail: 'https://mock.com/image.jpg' thumbnail: 'https://mock.com/image.jpg'
}) })
})); }));
// NOW import the module that depends on these // NOW import the module that depends on these
@@ -89,15 +89,15 @@ import { queueProcessor } from '$lib/server/queue/QueueProcessor';
import { extractTextAndThumbnail } from '$lib/server/extraction'; import { extractTextAndThumbnail } from '$lib/server/extraction';
describe('QueueProcessor', () => { describe('QueueProcessor', () => {
it('should use mocked services', async () => { it('should use mocked services', async () => {
const item = queueManager.enqueue('https://instagram.com/p/test'); const item = queueManager.enqueue('https://instagram.com/p/test');
// Verify mock was called // Verify mock was called
expect(extractTextAndThumbnail).toHaveBeenCalledWith( expect(extractTextAndThumbnail).toHaveBeenCalledWith(
'https://instagram.com/p/test', 'https://instagram.com/p/test',
expect.any(Function) expect.any(Function)
); );
}); });
}); });
``` ```
@@ -112,22 +112,22 @@ import { describe, it, expect } from 'vitest';
import { POST } from '../routes/api/queue/+server'; import { POST } from '../routes/api/queue/+server';
describe('POST /api/queue', () => { describe('POST /api/queue', () => {
it('should reject invalid URLs', async () => { it('should reject invalid URLs', async () => {
const request = new Request('http://localhost/api/queue', { const request = new Request('http://localhost/api/queue', {
method: 'POST', method: 'POST',
headers: { 'Content-Type': 'application/json' }, headers: { 'Content-Type': 'application/json' },
body: JSON.stringify({ url: 'invalid-url' }) body: JSON.stringify({ url: 'invalid-url' })
}); });
const response = await POST({ request } as any); const response = await POST({ request } as any);
// ✅ CORRECT - Check status first // ✅ CORRECT - Check status first
expect(response.status).toBe(400); expect(response.status).toBe(400);
// ✅ CORRECT - Properly await error response // ✅ CORRECT - Properly await error response
const data = await response.json(); const data = await response.json();
expect(data.message).toContain('Invalid'); expect(data.message).toContain('Invalid');
}); });
}); });
``` ```
@@ -136,17 +136,17 @@ describe('POST /api/queue', () => {
```typescript ```typescript
// ❌ WRONG - This will fail // ❌ WRONG - This will fail
it('should reject invalid input', async () => { it('should reject invalid input', async () => {
const response = await endpoint({ request } as any); const response = await endpoint({ request } as any);
const data = response.json(); // Missing await! const data = response.json(); // Missing await!
expect(data.message).toBe('Error'); // data is a Promise expect(data.message).toBe('Error'); // data is a Promise
}); });
// ✅ CORRECT // ✅ CORRECT
it('should reject invalid input', async () => { it('should reject invalid input', async () => {
const response = await endpoint({ request } as any); const response = await endpoint({ request } as any);
expect(response.status).toBe(400); expect(response.status).toBe(400);
const data = await response.json(); // Properly awaited const data = await response.json(); // Properly awaited
expect(data.message).toBe('Error'); expect(data.message).toBe('Error');
}); });
``` ```
@@ -173,11 +173,11 @@ import { queueProcessor } from './QueueProcessor';
import { beforeEach, afterEach } from 'vitest'; import { beforeEach, afterEach } from 'vitest';
beforeEach(() => { beforeEach(() => {
vi.clearAllMocks(); // Clear call history vi.clearAllMocks(); // Clear call history
}); });
afterEach(() => { afterEach(() => {
vi.restoreAllMocks(); // Restore original implementations vi.restoreAllMocks(); // Restore original implementations
}); });
``` ```
@@ -203,16 +203,16 @@ const mockFn = vi.fn() as Mock<() => Promise<string>>;
```typescript ```typescript
it('should process item', async () => { it('should process item', async () => {
const item = queueManager.enqueue('https://instagram.com/p/test'); const item = queueManager.enqueue('https://instagram.com/p/test');
// Wait for processing with timeout // Wait for processing with timeout
await vi.waitFor( await vi.waitFor(
() => { () => {
const updated = queueManager.get(item.id); const updated = queueManager.get(item.id);
expect(updated?.status).toBe('success'); expect(updated?.status).toBe('success');
}, },
{ timeout: 5000, interval: 100 } { timeout: 5000, interval: 100 }
); );
}); });
``` ```
@@ -222,20 +222,20 @@ it('should process item', async () => {
import { vi } from 'vitest'; import { vi } from 'vitest';
beforeEach(() => { beforeEach(() => {
vi.useFakeTimers(); vi.useFakeTimers();
}); });
afterEach(() => { afterEach(() => {
vi.useRealTimers(); vi.useRealTimers();
}); });
it('should process after delay', async () => { it('should process after delay', async () => {
queueManager.enqueue('https://test.com'); queueManager.enqueue('https://test.com');
// Fast-forward time // Fast-forward time
await vi.advanceTimersByTimeAsync(1000); await vi.advanceTimersByTimeAsync(1000);
// Now check results // Now check results
}); });
``` ```
@@ -263,7 +263,7 @@ vi.mock('./module', () => ({ export: vi.fn() }));
// Mock with factory // Mock with factory
vi.mock('./module', () => { vi.mock('./module', () => {
return { dynamicExport: () => 'value' }; return { dynamicExport: () => 'value' };
}); });
// Spy on existing export // Spy on existing export
@@ -285,9 +285,9 @@ expect(mockFn).toHaveBeenCalledWith('arg1', 'arg2');
expect(mockFn).toHaveBeenLastCalledWith('arg'); expect(mockFn).toHaveBeenLastCalledWith('arg');
// Reset/restore // Reset/restore
vi.clearAllMocks(); // Clear call history vi.clearAllMocks(); // Clear call history
vi.resetAllMocks(); // + Reset implementations vi.resetAllMocks(); // + Reset implementations
vi.restoreAllMocks(); // + Restore original implementations vi.restoreAllMocks(); // + Restore original implementations
// Environment variables // Environment variables
vi.stubEnv('VAR_NAME', 'value'); vi.stubEnv('VAR_NAME', 'value');

8788
docs/Tandoor (2.3.6).yaml
View File

File diff suppressed because it is too large Load Diff

14
eslint.config.js
View File

@@ -21,16 +21,14 @@ export default defineConfig(
languageOptions: { languageOptions: {
globals: { ...globals.browser, ...globals.node } globals: { ...globals.browser, ...globals.node }
}, },
rules: { // typescript-eslint strongly recommend that you do not use the no-undef lint rule on TypeScript projects. rules: {
// see: https://typescript-eslint.io/troubleshooting/faqs/eslint/#i-get-errors-from-the-no-undef-rule-about-global-variables-not-being-defined-even-though-there-are-no-typescript-errors // typescript-eslint strongly recommend that you do not use the no-undef lint rule on TypeScript projects.
"no-undef": 'off' } // see: https://typescript-eslint.io/troubleshooting/faqs/eslint/#i-get-errors-from-the-no-undef-rule-about-global-variables-not-being-defined-even-though-there-are-no-typescript-errors
'no-undef': 'off'
}
}, },
{ {
files: [ files: ['**/*.svelte', '**/*.svelte.ts', '**/*.svelte.js'],
'**/*.svelte',
'**/*.svelte.ts',
'**/*.svelte.js'
],
languageOptions: { languageOptions: {
parserOptions: { parserOptions: {
projectService: true, projectService: true,

122
package.json
View File

@@ -1,63 +1,63 @@
{ {
"name": "insta-recipe", "name": "insta-recipe",
"private": true, "private": true,
"version": "0.0.1", "version": "0.0.1",
"type": "module", "type": "module",
"scripts": { "scripts": {
"dev": "vite dev", "dev": "vite dev",
"dev:host": "vite dev --host", "dev:host": "vite dev --host",
"build": "vite build", "build": "vite build",
"preview": "vite preview", "preview": "vite preview",
"prepare": "svelte-kit sync || echo ''", "prepare": "svelte-kit sync || echo ''",
"check": "svelte-kit sync && svelte-check --tsconfig ./tsconfig.json", "check": "svelte-kit sync && svelte-check --tsconfig ./tsconfig.json",
"check:watch": "svelte-kit sync && svelte-check --tsconfig ./tsconfig.json --watch", "check:watch": "svelte-kit sync && svelte-check --tsconfig ./tsconfig.json --watch",
"format": "prettier --write .", "format": "prettier --write .",
"lint": "prettier --check . && eslint .", "lint": "prettier --check . && eslint .",
"test:unit": "vitest", "test:unit": "vitest",
"test:e2e": "playwright test", "test:e2e": "playwright test",
"test": "npm run test:unit -- --run" "test": "npm run test:unit -- --run"
}, },
"devDependencies": { "devDependencies": {
"@eslint/compat": "^1.4.0", "@eslint/compat": "^1.4.0",
"@eslint/js": "^9.39.1", "@eslint/js": "^9.39.1",
"@playwright/test": "^1.58.2", "@playwright/test": "^1.58.2",
"@sveltejs/adapter-node": "^5.4.0", "@sveltejs/adapter-node": "^5.4.0",
"@sveltejs/kit": "^2.48.5", "@sveltejs/kit": "^2.48.5",
"@sveltejs/vite-plugin-svelte": "^6.2.1", "@sveltejs/vite-plugin-svelte": "^6.2.1",
"@tailwindcss/vite": "^4.1.17", "@tailwindcss/vite": "^4.1.17",
"@types/node": "^22", "@types/node": "^22",
"@vitest/browser-playwright": "^4.0.10", "@vitest/browser-playwright": "^4.0.10",
"eslint": "^9.39.1", "eslint": "^9.39.1",
"eslint-config-prettier": "^10.1.8", "eslint-config-prettier": "^10.1.8",
"eslint-plugin-svelte": "^3.13.0", "eslint-plugin-svelte": "^3.13.0",
"fast-glob": "^3.3.3", "fast-glob": "^3.3.3",
"globals": "^16.5.0", "globals": "^16.5.0",
"prettier": "^3.6.2", "prettier": "^3.6.2",
"prettier-plugin-svelte": "^3.4.0", "prettier-plugin-svelte": "^3.4.0",
"prettier-plugin-tailwindcss": "^0.7.1", "prettier-plugin-tailwindcss": "^0.7.1",
"svelte": "^5.43.8", "svelte": "^5.43.8",
"svelte-check": "^4.3.4", "svelte-check": "^4.3.4",
"tailwindcss": "^4.1.17", "tailwindcss": "^4.1.17",
"typescript": "^5.9.3", "typescript": "^5.9.3",
"typescript-eslint": "^8.47.0", "typescript-eslint": "^8.47.0",
"vite": "^6.0.0", "vite": "^6.0.0",
"vitest": "^4.0.10", "vitest": "^4.0.10",
"vitest-browser-svelte": "^2.0.1" "vitest-browser-svelte": "^2.0.1"
}, },
"dependencies": { "dependencies": {
"@types/uuid": "^10.0.0", "@types/uuid": "^10.0.0",
"date-fns": "^4.1.0", "date-fns": "^4.1.0",
"openai": "^4.20.0", "openai": "^4.20.0",
"playwright": "^1.56.1", "playwright": "^1.56.1",
"playwright-extra": "^4.3.6", "playwright-extra": "^4.3.6",
"puppeteer-extra-plugin-stealth": "^2.11.2", "puppeteer-extra-plugin-stealth": "^2.11.2",
"sharp": "^0.34.5", "sharp": "^0.34.5",
"uuid": "^13.0.0", "uuid": "^13.0.0",
"web-push": "^3.6.7", "web-push": "^3.6.7",
"zod": "^3.23.0" "zod": "^3.23.0"
}, },
"overrides": { "overrides": {
"cookie": "^0.7.0", "cookie": "^0.7.0",
"ajv": "^8.18.0" "ajv": "^8.18.0"
} }
} }

48
playwright.config.ts
View File

@@ -2,33 +2,33 @@ import { defineConfig, devices } from '@playwright/test';
/** /**
* Playwright configuration for E2E tests * Playwright configuration for E2E tests
* *
* See https://playwright.dev/docs/test-configuration * See https://playwright.dev/docs/test-configuration
*/ */
export default defineConfig({ export default defineConfig({
testDir: './src/tests', testDir: './src/tests',
testMatch: '**/*.e2e.spec.ts', testMatch: '**/*.e2e.spec.ts',
fullyParallel: true, fullyParallel: true,
forbidOnly: !!process.env.CI, forbidOnly: !!process.env.CI,
retries: process.env.CI ? 2 : 0, retries: process.env.CI ? 2 : 0,
workers: process.env.CI ? 1 : undefined, workers: process.env.CI ? 1 : undefined,
reporter: 'list', reporter: 'list',
use: { use: {
baseURL: 'http://localhost:5173', baseURL: 'http://localhost:5173',
trace: 'on-first-retry', trace: 'on-first-retry'
}, },
projects: [ projects: [
{ {
name: 'chromium', name: 'chromium',
use: { ...devices['Desktop Chrome'] }, use: { ...devices['Desktop Chrome'] }
}, }
], ],
webServer: { webServer: {
command: 'npm run dev', command: 'npm run dev',
url: 'http://localhost:5173', url: 'http://localhost:5173',
reuseExistingServer: !process.env.CI, reuseExistingServer: !process.env.CI,
timeout: 120000, timeout: 120000
}, }
}); });

40
scripts/gen-auth.js
View File

@@ -4,23 +4,23 @@ import fs from 'fs';
import path from 'path'; import path from 'path';
(async () => { (async () => {
const browser = await chromium.launch({ headless: false }); const browser = await chromium.launch({ headless: false });
const context = await browser.newContext(); const context = await browser.newContext();
const page = await context.newPage(); const page = await context.newPage();
console.log('🔹 Navigating to Instagram...'); console.log('🔹 Navigating to Instagram...');
await page.goto('https://www.instagram.com/'); await page.goto('https://www.instagram.com/');
console.log('⏳ Please log in manually. Waiting for "Home" icon...'); console.log('⏳ Please log in manually. Waiting for "Home" icon...');
try { try {
await page.waitForSelector('svg[aria-label="Home"]', { timeout: 120000 }); await page.waitForSelector('svg[aria-label="Home"]', { timeout: 120000 });
const secretsDir = path.resolve('../secrets'); const secretsDir = path.resolve('../secrets');
if (!fs.existsSync(secretsDir)) fs.mkdirSync(secretsDir); if (!fs.existsSync(secretsDir)) fs.mkdirSync(secretsDir);
await context.storageState({ path: path.join(secretsDir, 'auth.json') }); await context.storageState({ path: path.join(secretsDir, 'auth.json') });
console.log('🎉 Session saved to secrets/auth.json'); console.log('🎉 Session saved to secrets/auth.json');
} catch (e) { } catch (e) {
console.error('❌ Timeout or error:', e); console.error('❌ Timeout or error:', e);
} }
await browser.close(); await browser.close();
})(); })();

76
scripts/gen-favicon-ico.js
View File

@@ -7,50 +7,50 @@ const __filename = fileURLToPath(import.meta.url);
const __dirname = path.dirname(__filename); const __dirname = path.dirname(__filename);
async function generateFaviconIco() { async function generateFaviconIco() {
const sourceIcon = path.join(__dirname, '..', 'static', 'icon-source.png'); const sourceIcon = path.join(__dirname, '..', 'static', 'icon-source.png');
const outputIcon = path.join(__dirname, '..', 'static', 'favicon.ico'); const outputIcon = path.join(__dirname, '..', 'static', 'favicon.ico');
console.log('Generating favicon.ico from icon-source.png...'); console.log('Generating favicon.ico from icon-source.png...');
// Verify source file exists
if (!fs.existsSync(sourceIcon)) {
console.error('Error: icon-source.png not found at', sourceIcon);
process.exit(1);
}
// Resize to 32x32 with transparent background // Verify source file exists
await sharp(sourceIcon) if (!fs.existsSync(sourceIcon)) {
.resize(32, 32, { console.error('Error: icon-source.png not found at', sourceIcon);
fit: 'contain', process.exit(1);
background: { r: 0, g: 0, b: 0, alpha: 0 } }
})
.ensureAlpha()
.png()
.toFile(outputIcon);
// Verify output file // Resize to 32x32 with transparent background
const metadata = await sharp(outputIcon).metadata(); await sharp(sourceIcon)
const stats = fs.statSync(outputIcon); .resize(32, 32, {
fit: 'contain',
background: { r: 0, g: 0, b: 0, alpha: 0 }
})
.ensureAlpha()
.png()
.toFile(outputIcon);
console.log(`✓ favicon.ico generated successfully`); // Verify output file
console.log(` Dimensions: ${metadata.width}x${metadata.height}`); const metadata = await sharp(outputIcon).metadata();
console.log(` Format: ${metadata.format}`); const stats = fs.statSync(outputIcon);
console.log(` Size: ${(stats.size / 1024).toFixed(1)}KB`);
// Validate success criteria console.log(`✓ favicon.ico generated successfully`);
if (metadata.width !== 32 || metadata.height !== 32) { console.log(` Dimensions: ${metadata.width}x${metadata.height}`);
console.error('Error: Invalid dimensions'); console.log(` Format: ${metadata.format}`);
process.exit(1); console.log(` Size: ${(stats.size / 1024).toFixed(1)}KB`);
}
if (metadata.format !== 'png') {
console.error('Error: Invalid format');
process.exit(1);
}
console.log('✓ All validation checks passed'); // Validate success criteria
if (metadata.width !== 32 || metadata.height !== 32) {
console.error('Error: Invalid dimensions');
process.exit(1);
}
if (metadata.format !== 'png') {
console.error('Error: Invalid format');
process.exit(1);
}
console.log('✓ All validation checks passed');
} }
generateFaviconIco().catch(err => { generateFaviconIco().catch((err) => {
console.error('Error generating favicon.ico:', err); console.error('Error generating favicon.ico:', err);
process.exit(1); process.exit(1);
}); });

84
scripts/gen-favicon.js
View File

@@ -7,54 +7,54 @@ const __filename = fileURLToPath(import.meta.url);
const __dirname = path.dirname(__filename); const __dirname = path.dirname(__filename);
async function generateFavicon() { async function generateFavicon() {
const sourceIcon = path.join(__dirname, '..', 'static', 'icon-source.png'); const sourceIcon = path.join(__dirname, '..', 'static', 'icon-source.png');
const outputIcon = path.join(__dirname, '..', 'static', 'favicon.png'); const outputIcon = path.join(__dirname, '..', 'static', 'favicon.png');
console.log('Generating favicon.png from icon-source.png...'); console.log('Generating favicon.png from icon-source.png...');
// Verify source file exists
if (!fs.existsSync(sourceIcon)) {
console.error('Error: icon-source.png not found at', sourceIcon);
process.exit(1);
}
// Resize to 192x192 with transparent background // Verify source file exists
await sharp(sourceIcon) if (!fs.existsSync(sourceIcon)) {
.resize(192, 192, { console.error('Error: icon-source.png not found at', sourceIcon);
fit: 'contain', process.exit(1);
background: { r: 0, g: 0, b: 0, alpha: 0 } }
})
.ensureAlpha()
.png()
.toFile(outputIcon);
// Verify output file // Resize to 192x192 with transparent background
const metadata = await sharp(outputIcon).metadata(); await sharp(sourceIcon)
const stats = fs.statSync(outputIcon); .resize(192, 192, {
fit: 'contain',
background: { r: 0, g: 0, b: 0, alpha: 0 }
})
.ensureAlpha()
.png()
.toFile(outputIcon);
console.log(`✓ favicon.png generated successfully`); // Verify output file
console.log(` Dimensions: ${metadata.width}x${metadata.height}`); const metadata = await sharp(outputIcon).metadata();
console.log(` Format: ${metadata.format}`); const stats = fs.statSync(outputIcon);
console.log(` Size: ${(stats.size / 1024).toFixed(1)}KB`);
// Validate success criteria console.log(`✓ favicon.png generated successfully`);
if (metadata.width !== 192 || metadata.height !== 192) { console.log(` Dimensions: ${metadata.width}x${metadata.height}`);
console.error('Error: Invalid dimensions'); console.log(` Format: ${metadata.format}`);
process.exit(1); console.log(` Size: ${(stats.size / 1024).toFixed(1)}KB`);
}
if (metadata.format !== 'png') {
console.error('Error: Invalid format');
process.exit(1);
}
if (stats.size > 100 * 1024) {
console.error('Error: File size exceeds 100KB');
process.exit(1);
}
console.log('✓ All validation checks passed'); // Validate success criteria
if (metadata.width !== 192 || metadata.height !== 192) {
console.error('Error: Invalid dimensions');
process.exit(1);
}
if (metadata.format !== 'png') {
console.error('Error: Invalid format');
process.exit(1);
}
if (stats.size > 100 * 1024) {
console.error('Error: File size exceeds 100KB');
process.exit(1);
}
console.log('✓ All validation checks passed');
} }
generateFavicon().catch(err => { generateFavicon().catch((err) => {
console.error('Error generating favicon:', err); console.error('Error generating favicon:', err);
process.exit(1); process.exit(1);
}); });

96
scripts/generate-icon-512.js
View File

@@ -2,54 +2,54 @@ const sharp = require('sharp');
const fs = require('fs'); const fs = require('fs');
async function generateIcon512() { async function generateIcon512() {
try { try {
console.log('Generating icon-512.png from icon-source.png...'); console.log('Generating icon-512.png from icon-source.png...');
// Check if source file exists // Check if source file exists
if (!fs.existsSync('static/icon-source.png')) { if (!fs.existsSync('static/icon-source.png')) {
console.error('Error: static/icon-source.png does not exist'); console.error('Error: static/icon-source.png does not exist');
process.exit(1); process.exit(1);
} }
// Generate 512x512 icon // Generate 512x512 icon
await sharp('static/icon-source.png') await sharp('static/icon-source.png')
.resize(512, 512, { .resize(512, 512, {
fit: 'contain', fit: 'contain',
background: { r: 0, g: 0, b: 0, alpha: 0 } background: { r: 0, g: 0, b: 0, alpha: 0 }
}) })
.png() .png()
.toFile('static/icon-512.png'); .toFile('static/icon-512.png');
console.log('✓ Generated static/icon-512.png'); console.log('✓ Generated static/icon-512.png');
// Verify the result // Verify the result
const metadata = await sharp('static/icon-512.png').metadata(); const metadata = await sharp('static/icon-512.png').metadata();
const stats = fs.statSync('static/icon-512.png'); const stats = fs.statSync('static/icon-512.png');
console.log(` Dimensions: ${metadata.width}x${metadata.height}`); console.log(` Dimensions: ${metadata.width}x${metadata.height}`);
console.log(` Format: ${metadata.format}`); console.log(` Format: ${metadata.format}`);
console.log(` Size: ${Math.round(stats.size / 1024)}KB`); console.log(` Size: ${Math.round(stats.size / 1024)}KB`);
// Validate // Validate
if (metadata.width !== 512 || metadata.height !== 512) { if (metadata.width !== 512 || metadata.height !== 512) {
console.error('Error: Invalid dimensions'); console.error('Error: Invalid dimensions');
process.exit(1); process.exit(1);
} }
if (metadata.format !== 'png') { if (metadata.format !== 'png') {
console.error('Error: Invalid format'); console.error('Error: Invalid format');
process.exit(1); process.exit(1);
} }
if (stats.size > 200 * 1024) { if (stats.size > 200 * 1024) {
console.error('Error: File size exceeds 200KB'); console.error('Error: File size exceeds 200KB');
process.exit(1); process.exit(1);
} }
console.log('✓ Validation passed'); console.log('✓ Validation passed');
process.exit(0); process.exit(0);
} catch (error) { } catch (error) {
console.error('Error generating icon:', error.message); console.error('Error generating icon:', error.message);
process.exit(1); process.exit(1);
} }
} }
generateIcon512(); generateIcon512();

268
secrets/auth.json
View File

@@ -1,135 +1,135 @@
{ {
"cookies": [ "cookies": [
{ {
"name": "csrftoken", "name": "csrftoken",
"value": "SDRORLyWEsWWty2ZoVGdER", "value": "SDRORLyWEsWWty2ZoVGdER",
"domain": ".instagram.com", "domain": ".instagram.com",
"path": "/", "path": "/",
"expires": 1805933297.800746, "expires": 1805933297.800746,
"httpOnly": false, "httpOnly": false,
"secure": true, "secure": true,
"sameSite": "Lax" "sameSite": "Lax"
}, },
{ {
"name": "datr", "name": "datr",
"value": "isQuaeXe5-2mFvFSOdcgVq0u", "value": "isQuaeXe5-2mFvFSOdcgVq0u",
"domain": ".instagram.com", "domain": ".instagram.com",
"path": "/", "path": "/",
"expires": 1799232653.525143, "expires": 1799232653.525143,
"httpOnly": true, "httpOnly": true,
"secure": true, "secure": true,
"sameSite": "None" "sameSite": "None"
}, },
{ {
"name": "ig_did", "name": "ig_did",
"value": "5650C8B9-B8D8-4102-9B49-F0668CE34202", "value": "5650C8B9-B8D8-4102-9B49-F0668CE34202",
"domain": ".instagram.com", "domain": ".instagram.com",
"path": "/", "path": "/",
"expires": 1796208680.653147, "expires": 1796208680.653147,
"httpOnly": true, "httpOnly": true,
"secure": true, "secure": true,
"sameSite": "Lax" "sameSite": "Lax"
}, },
{ {
"name": "mid", "name": "mid",
"value": "aS7EigALAAHxXAxrkYg18Fzi-SR7", "value": "aS7EigALAAHxXAxrkYg18Fzi-SR7",
"domain": ".instagram.com", "domain": ".instagram.com",
"path": "/", "path": "/",
"expires": 1799232653.525191, "expires": 1799232653.525191,
"httpOnly": true, "httpOnly": true,
"secure": true, "secure": true,
"sameSite": "None" "sameSite": "None"
}, },
{ {
"name": "ds_user_id", "name": "ds_user_id",
"value": "59661903731", "value": "59661903731",
"domain": ".instagram.com", "domain": ".instagram.com",
"path": "/", "path": "/",
"expires": 1779149297.800838, "expires": 1779149297.800838,
"httpOnly": false, "httpOnly": false,
"secure": true, "secure": true,
"sameSite": "None" "sameSite": "None"
}, },
{ {
"name": "sessionid", "name": "sessionid",
"value": "59661903731%3AbekaIlo4nn7x2n%3A29%3AAYhv1LJUsfRtBSH-WmDLVrxiM7T9UotIOM3XY3iHKQ", "value": "59661903731%3AbekaIlo4nn7x2n%3A29%3AAYhv1LJUsfRtBSH-WmDLVrxiM7T9UotIOM3XY3iHKQ",
"domain": ".instagram.com", "domain": ".instagram.com",
"path": "/", "path": "/",
"expires": 1797910987.674116, "expires": 1797910987.674116,
"httpOnly": true, "httpOnly": true,
"secure": true, "secure": true,
"sameSite": "Lax" "sameSite": "Lax"
}, },
{ {
"name": "wd", "name": "wd",
"value": "1280x720", "value": "1280x720",
"domain": ".instagram.com", "domain": ".instagram.com",
"path": "/", "path": "/",
"expires": 1771978099, "expires": 1771978099,
"httpOnly": false, "httpOnly": false,
"secure": true, "secure": true,
"sameSite": "Lax" "sameSite": "Lax"
}, },
{ {
"name": "rur", "name": "rur",
"value": "\"CLN\\05459661903731\\0541802909297:01fe3b61178e8a40cb655e8d1aa137841487d29a8caf122f877ce2955e92fb4307b53ced\"", "value": "\"CLN\\05459661903731\\0541802909297:01fe3b61178e8a40cb655e8d1aa137841487d29a8caf122f877ce2955e92fb4307b53ced\"",
"domain": ".instagram.com", "domain": ".instagram.com",
"path": "/", "path": "/",
"expires": -1, "expires": -1,
"httpOnly": true, "httpOnly": true,
"secure": true, "secure": true,
"sameSite": "Lax" "sameSite": "Lax"
} }
], ],
"origins": [ "origins": [
{ {
"origin": "https://www.instagram.com", "origin": "https://www.instagram.com",
"localStorage": [ "localStorage": [
{ {
"name": "chatd-deviceid", "name": "chatd-deviceid",
"value": "9b4abdd3-f38f-473e-9e3b-2ae722acc64d" "value": "9b4abdd3-f38f-473e-9e3b-2ae722acc64d"
}, },
{ {
"name": "hb_timestamp", "name": "hb_timestamp",
"value": "1771370599886" "value": "1771370599886"
}, },
{ {
"name": "IGSession", "name": "IGSession",
"value": "k75336:1771375099770" "value": "k75336:1771375099770"
}, },
{ {
"name": "mutex_polaris_banzai", "name": "mutex_polaris_banzai",
"value": "4eic7h:1771373300769" "value": "4eic7h:1771373300769"
}, },
{ {
"name": "pixel_fire_ts", "name": "pixel_fire_ts",
"value": "1771121302843" "value": "1771121302843"
}, },
{ {
"name": "signal_flush_timestamp", "name": "signal_flush_timestamp",
"value": "1771371499888" "value": "1771371499888"
}, },
{ {
"name": "Session", "name": "Session",
"value": "t5cu8b:1771373334770" "value": "t5cu8b:1771373334770"
}, },
{ {
"name": "has_interop_upgraded", "name": "has_interop_upgraded",
"value": "{\"lastCheckedAt\":1766366944051,\"status\":false}" "value": "{\"lastCheckedAt\":1766366944051,\"status\":false}"
}, },
{ {
"name": "ig_boost_on_web_campaign_upsell_shown", "name": "ig_boost_on_web_campaign_upsell_shown",
"value": "false" "value": "false"
}, },
{ {
"name": "mutex_banzai", "name": "mutex_banzai",
"value": "4eic7h:1771373300769" "value": "4eic7h:1771373300769"
}, },
{ {
"name": "banzai:last_storage_flush", "name": "banzai:last_storage_flush",
"value": "1771366998859.2" "value": "1771366998859.2"
} }
] ]
} }
] ]
} }

2
src/app.html
View File

@@ -3,7 +3,7 @@
<head> <head>
<meta charset="utf-8" /> <meta charset="utf-8" />
<meta name="viewport" content="width=device-width, initial-scale=1" /> <meta name="viewport" content="width=device-width, initial-scale=1" />
<link rel="manifest" href="/manifest.json"> <link rel="manifest" href="/manifest.json" />
%sveltekit.head% %sveltekit.head%
</head> </head>
<body data-sveltekit-preload-data="hover"> <body data-sveltekit-preload-data="hover">

54
src/app.server.ts
View File

@@ -1,27 +1,27 @@
import { initializeBrowser, closeBrowser } from '$lib/server/browser'; import { initializeBrowser, closeBrowser } from '$lib/server/browser';
// Initialize browser when server starts // Initialize browser when server starts
export async function init() { export async function init() {
try { try {
await initializeBrowser(); await initializeBrowser();
} catch (error) { } catch (error) {
console.error('Failed to initialize browser:', error); console.error('Failed to initialize browser:', error);
process.exit(1); process.exit(1);
} }
// Graceful shutdown // Graceful shutdown
process.on('SIGTERM', async () => { process.on('SIGTERM', async () => {
console.log('SIGTERM received, shutting down gracefully...'); console.log('SIGTERM received, shutting down gracefully...');
await closeBrowser(); await closeBrowser();
process.exit(0); process.exit(0);
}); });
process.on('SIGINT', async () => { process.on('SIGINT', async () => {
console.log('SIGINT received, shutting down gracefully...'); console.log('SIGINT received, shutting down gracefully...');
await closeBrowser(); await closeBrowser();
process.exit(0); process.exit(0);
}); });
} }
// Run initialization immediately // Run initialization immediately
init().catch(console.error); init().catch(console.error);

64
src/hooks.server.ts
View File

@@ -1,32 +1,32 @@
import { startScheduler, stopScheduler } from '$lib/server/scheduler'; import { startScheduler, stopScheduler } from '$lib/server/scheduler';
import '$lib/server/queue/QueueProcessor'; // Trigger QueueProcessor auto-start import '$lib/server/queue/QueueProcessor'; // Trigger QueueProcessor auto-start
import type { ServerInit } from '@sveltejs/kit'; import type { ServerInit } from '@sveltejs/kit';
/** /**
* Initialize server-wide functionality * Initialize server-wide functionality
* Runs once when the server starts * Runs once when the server starts
* *
* Environment variables: * Environment variables:
* - AUTH_SCHEDULER_ENABLED: Set to 'true' to enable periodic auth renewal * - AUTH_SCHEDULER_ENABLED: Set to 'true' to enable periodic auth renewal
* - AUTH_SCHEDULER_INTERVAL_MINUTES: Minutes between each renewal (default: 720) * - AUTH_SCHEDULER_INTERVAL_MINUTES: Minutes between each renewal (default: 720)
*/ */
export const init: ServerInit = async () => { export const init: ServerInit = async () => {
console.log('[Server Init] Starting SvelteKit server...'); console.log('[Server Init] Starting SvelteKit server...');
console.log('[Server Init] QueueProcessor auto-started via import'); console.log('[Server Init] QueueProcessor auto-started via import');
// The scheduler will renew the Instagram session by loading the existing auth.json // The scheduler will renew the Instagram session by loading the existing auth.json
// and refreshing it with Instagram (requires initial setup via gen-auth.js) // and refreshing it with Instagram (requires initial setup via gen-auth.js)
await startScheduler(); await startScheduler();
}; };
/** /**
* Listen for graceful shutdown * Listen for graceful shutdown
* Clean up resources when the server is shutting down * Clean up resources when the server is shutting down
*/ */
process.on('sveltekit:shutdown', async (reason) => { process.on('sveltekit:shutdown', async (reason) => {
console.log(`[Server Shutdown] Shutdown triggered by: ${reason}`); console.log(`[Server Shutdown] Shutdown triggered by: ${reason}`);
// Stop the scheduler gracefully // Stop the scheduler gracefully
await stopScheduler(); await stopScheduler();
console.log('[Server Shutdown] Cleanup complete'); console.log('[Server Shutdown] Cleanup complete');
}); });

336
src/lib/client/PWAInstallManager.ts
View File

@@ -1,6 +1,6 @@
/** /**
* PWA Installation Manager * PWA Installation Manager
* *
* Handles PWA installation flow with cross-browser support. * Handles PWA installation flow with cross-browser support.
* Provides beforeinstallprompt event handling, user engagement detection, * Provides beforeinstallprompt event handling, user engagement detection,
* and dismissal state management for the install prompt. * and dismissal state management for the install prompt.
@@ -9,193 +9,193 @@
import { browser } from '$app/environment'; import { browser } from '$app/environment';
interface BeforeInstallPromptEvent extends Event { interface BeforeInstallPromptEvent extends Event {
prompt(): Promise<void>; prompt(): Promise<void>;
userChoice: Promise<{ outcome: 'accepted' | 'dismissed' }>; userChoice: Promise<{ outcome: 'accepted' | 'dismissed' }>;
} }
export class PWAInstallManager { export class PWAInstallManager {
private deferredPrompt: BeforeInstallPromptEvent | null = null; private deferredPrompt: BeforeInstallPromptEvent | null = null;
private listeners: Array<(canInstall: boolean) => void> = []; private listeners: Array<(canInstall: boolean) => void> = [];
private installable = false; private installable = false;
constructor() { constructor() {
if (browser) { if (browser) {
this.initializeInstallPrompt(); this.initializeInstallPrompt();
} }
} }
/** /**
* Initialize PWA install prompt event listeners * Initialize PWA install prompt event listeners
*/ */
private initializeInstallPrompt(): void { private initializeInstallPrompt(): void {
// Listen for beforeinstallprompt event (Chrome, Edge) // Listen for beforeinstallprompt event (Chrome, Edge)
window.addEventListener('beforeinstallprompt', (e: Event) => { window.addEventListener('beforeinstallprompt', (e: Event) => {
e.preventDefault(); e.preventDefault();
this.deferredPrompt = e as BeforeInstallPromptEvent; this.deferredPrompt = e as BeforeInstallPromptEvent;
this.installable = true; this.installable = true;
this.notifyListeners(true); this.notifyListeners(true);
console.log('[PWA] Install prompt available'); console.log('[PWA] Install prompt available');
}); });
// Listen for app installation completion // Listen for app installation completion
window.addEventListener('appinstalled', () => { window.addEventListener('appinstalled', () => {
console.log('[PWA] App was installed'); console.log('[PWA] App was installed');
this.installable = false; this.installable = false;
this.deferredPrompt = null; this.deferredPrompt = null;
this.notifyListeners(false); this.notifyListeners(false);
// Clear dismissal state since user installed
this.clearDismissed();
});
// Check if already installed // Clear dismissal state since user installed
if (this.isStandalone()) { this.clearDismissed();
console.log('[PWA] App is already running in standalone mode'); });
}
}
/** // Check if already installed
* Check if PWA can be installed if (this.isStandalone()) {
*/ console.log('[PWA] App is already running in standalone mode');
public canInstall(): boolean { }
return this.installable && this.deferredPrompt !== null; }
}
/** /**
* Show the browser's install prompt * Check if PWA can be installed
* */
* @returns Promise resolving to user's choice or 'unavailable' if no prompt available public canInstall(): boolean {
*/ return this.installable && this.deferredPrompt !== null;
public async showInstallPrompt(): Promise<'accepted' | 'dismissed' | 'unavailable'> { }
if (!this.deferredPrompt) {
console.warn('[PWA] Install prompt not available');
return 'unavailable';
}
try { /**
await this.deferredPrompt.prompt(); * Show the browser's install prompt
const { outcome } = await this.deferredPrompt.userChoice; *
* @returns Promise resolving to user's choice or 'unavailable' if no prompt available
this.deferredPrompt = null; */
this.installable = false; public async showInstallPrompt(): Promise<'accepted' | 'dismissed' | 'unavailable'> {
this.notifyListeners(false); if (!this.deferredPrompt) {
console.warn('[PWA] Install prompt not available');
console.log(`[PWA] Install prompt ${outcome}`); return 'unavailable';
return outcome; }
} catch (error) {
console.error('[PWA] Install prompt failed:', error);
return 'dismissed';
}
}
/** try {
* Register a callback for install state changes await this.deferredPrompt.prompt();
* const { outcome } = await this.deferredPrompt.userChoice;
* @param callback Function to call when install state changes
* @returns Unsubscribe function
*/
public onInstallStateChange(callback: (canInstall: boolean) => void): () => void {
this.listeners.push(callback);
// Call immediately with current state
callback(this.canInstall());
return () => {
this.listeners = this.listeners.filter(cb => cb !== callback);
};
}
/** this.deferredPrompt = null;
* Notify all listeners of state change this.installable = false;
*/ this.notifyListeners(false);
private notifyListeners(canInstall: boolean): void {
this.listeners.forEach(callback => {
try {
callback(canInstall);
} catch (error) {
console.error('[PWA] Error in install state listener:', error);
}
});
}
/** console.log(`[PWA] Install prompt ${outcome}`);
* Check if app is running in standalone mode (already installed) return outcome;
*/ } catch (error) {
public isStandalone(): boolean { console.error('[PWA] Install prompt failed:', error);
if (!browser) return false; return 'dismissed';
}
return ( }
window.matchMedia('(display-mode: standalone)').matches ||
(window.navigator as any).standalone === true ||
document.referrer.includes('android-app://')
);
}
/** /**
* Check if user has dismissed the install prompt * Register a callback for install state changes
*/ *
public isDismissed(): boolean { * @param callback Function to call when install state changes
if (!browser) return false; * @returns Unsubscribe function
return localStorage.getItem('pwa-install-dismissed') === 'true'; */
} public onInstallStateChange(callback: (canInstall: boolean) => void): () => void {
this.listeners.push(callback);
/** // Call immediately with current state
* Mark install prompt as dismissed by user callback(this.canInstall());
*/
public setDismissed(): void {
if (browser) {
localStorage.setItem('pwa-install-dismissed', 'true');
console.log('[PWA] Install prompt dismissed by user');
}
}
/** return () => {
* Clear dismissal state (called when app is installed) this.listeners = this.listeners.filter((cb) => cb !== callback);
*/ };
public clearDismissed(): void { }
if (browser) {
localStorage.removeItem('pwa-install-dismissed');
}
}
/** /**
* Get browser-specific installation instructions * Notify all listeners of state change
*/ */
public getInstallInstructions(): string { private notifyListeners(canInstall: boolean): void {
if (!browser) return 'Install instructions not available'; this.listeners.forEach((callback) => {
try {
callback(canInstall);
} catch (error) {
console.error('[PWA] Error in install state listener:', error);
}
});
}
const userAgent = navigator.userAgent.toLowerCase(); /**
const isMobile = /android|iphone|ipad|ipod|blackberry|iemobile|opera mini/i.test(userAgent); * Check if app is running in standalone mode (already installed)
*/
if (isMobile && userAgent.includes('safari') && !userAgent.includes('chrome')) { public isStandalone(): boolean {
return 'Tap the Share button and select "Add to Home Screen"'; if (!browser) return false;
} else if (userAgent.includes('chrome') && !userAgent.includes('edg')) {
return 'Look for the install button in your browser address bar';
} else if (userAgent.includes('edg')) {
return 'Look for the install button in your browser address bar';
} else if (userAgent.includes('firefox')) {
return 'Firefox has limited PWA support. Try Chrome or Edge for the best experience';
}
return 'Check your browser menu for "Install App" or "Add to Home Screen" options';
}
/** return (
* Get current browser name for UI customization window.matchMedia('(display-mode: standalone)').matches ||
*/ (window.navigator as any).standalone === true ||
public getBrowserName(): string { document.referrer.includes('android-app://')
if (!browser) return 'unknown'; );
}
const userAgent = navigator.userAgent.toLowerCase(); /**
* Check if user has dismissed the install prompt
if (userAgent.includes('chrome') && !userAgent.includes('edg')) return 'chrome'; */
if (userAgent.includes('safari') && !userAgent.includes('chrome')) return 'safari'; public isDismissed(): boolean {
if (userAgent.includes('firefox')) return 'firefox'; if (!browser) return false;
if (userAgent.includes('edg')) return 'edge'; return localStorage.getItem('pwa-install-dismissed') === 'true';
}
return 'unknown';
} /**
* Mark install prompt as dismissed by user
*/
public setDismissed(): void {
if (browser) {
localStorage.setItem('pwa-install-dismissed', 'true');
console.log('[PWA] Install prompt dismissed by user');
}
}
/**
* Clear dismissal state (called when app is installed)
*/
public clearDismissed(): void {
if (browser) {
localStorage.removeItem('pwa-install-dismissed');
}
}
/**
* Get browser-specific installation instructions
*/
public getInstallInstructions(): string {
if (!browser) return 'Install instructions not available';
const userAgent = navigator.userAgent.toLowerCase();
const isMobile = /android|iphone|ipad|ipod|blackberry|iemobile|opera mini/i.test(userAgent);
if (isMobile && userAgent.includes('safari') && !userAgent.includes('chrome')) {
return 'Tap the Share button and select "Add to Home Screen"';
} else if (userAgent.includes('chrome') && !userAgent.includes('edg')) {
return 'Look for the install button in your browser address bar';
} else if (userAgent.includes('edg')) {
return 'Look for the install button in your browser address bar';
} else if (userAgent.includes('firefox')) {
return 'Firefox has limited PWA support. Try Chrome or Edge for the best experience';
}
return 'Check your browser menu for "Install App" or "Add to Home Screen" options';
}
/**
* Get current browser name for UI customization
*/
public getBrowserName(): string {
if (!browser) return 'unknown';
const userAgent = navigator.userAgent.toLowerCase();
if (userAgent.includes('chrome') && !userAgent.includes('edg')) return 'chrome';
if (userAgent.includes('safari') && !userAgent.includes('chrome')) return 'safari';
if (userAgent.includes('firefox')) return 'firefox';
if (userAgent.includes('edg')) return 'edge';
return 'unknown';
}
} }
// Singleton instance for application-wide use // Singleton instance for application-wide use
export const pwaInstallManager = new PWAInstallManager(); export const pwaInstallManager = new PWAInstallManager();

634
src/lib/client/PushNotificationManager.ts
View File

@@ -1,379 +1,371 @@
/** /**
* Client-side Push Notification Manager * Client-side Push Notification Manager
* *
* Handles push notification subscription/unsubscription * Handles push notification subscription/unsubscription
* and permission management in the browser. * and permission management in the browser.
* *
* SSR-Safe: All browser API access is guarded and lazily initialized * SSR-Safe: All browser API access is guarded and lazily initialized
*/ */
import { browser } from '$app/environment'; import { browser } from '$app/environment';
interface NotificationState { interface NotificationState {
supported: boolean; supported: boolean;
permission: NotificationPermission; permission: NotificationPermission;
subscribed: boolean; subscribed: boolean;
loading: boolean; loading: boolean;
error: string | null; error: string | null;
} }
class PushNotificationManager { class PushNotificationManager {
private state: NotificationState = { private state: NotificationState = {
supported: false, supported: false,
permission: 'default', permission: 'default',
subscribed: false, subscribed: false,
loading: false, loading: false,
error: null error: null
}; };
private listeners: Array<(state: NotificationState) => void> = []; private listeners: Array<(state: NotificationState) => void> = [];
private registration: ServiceWorkerRegistration | null = null; private registration: ServiceWorkerRegistration | null = null;
private _clientId: string | null = null; private _clientId: string | null = null;
private _initialized = false; private _initialized = false;
constructor() { constructor() {
// SSR-safe constructor: no browser API access // SSR-safe constructor: no browser API access
// Initialization happens lazily when needed // Initialization happens lazily when needed
} }
/** /**
* Lazy initialization - only runs in browser context * Lazy initialization - only runs in browser context
*/ */
private ensureInitialized(): void { private ensureInitialized(): void {
if (this._initialized || !browser) return; if (this._initialized || !browser) return;
this._initialized = true;
this.checkSupport();
this.initializeServiceWorker();
}
/** this._initialized = true;
* Get clientId lazily - only generates in browser context this.checkSupport();
*/ this.initializeServiceWorker();
private get clientId(): string { }
if (!this._clientId && browser) {
this._clientId = this.generateClientId();
}
return this._clientId || 'ssr-fallback';
}
/** /**
* Subscribe to state changes * Get clientId lazily - only generates in browser context
*/ */
onStateChange(callback: (state: NotificationState) => void): () => void { private get clientId(): string {
this.ensureInitialized(); // Ensure initialized before sending state if (!this._clientId && browser) {
this._clientId = this.generateClientId();
this.listeners.push(callback); }
callback(this.state); // Send initial state return this._clientId || 'ssr-fallback';
}
return () => {
this.listeners = this.listeners.filter(cb => cb !== callback);
};
}
/** /**
* Get current state * Subscribe to state changes
*/ */
getState(): NotificationState { onStateChange(callback: (state: NotificationState) => void): () => void {
this.ensureInitialized(); this.ensureInitialized(); // Ensure initialized before sending state
return { ...this.state };
}
/** this.listeners.push(callback);
* Check if push notifications are supported callback(this.state); // Send initial state
* SSR-safe: guarded with browser check
*/
private checkSupport(): void {
if (!browser) {
this.state.supported = false;
this.state.permission = 'denied';
return;
}
this.state.supported = (
'serviceWorker' in navigator &&
'PushManager' in window &&
'Notification' in window
);
this.state.permission = this.state.supported ? Notification.permission : 'denied';
}
/** return () => {
* Initialize service worker registration this.listeners = this.listeners.filter((cb) => cb !== callback);
* SSR-safe: guarded with browser and support checks };
*/ }
private async initializeServiceWorker(): Promise<void> {
if (!browser || !this.state.supported) return;
try { /**
// Wait for service worker to be ready * Get current state
this.registration = await navigator.serviceWorker.ready; */
console.log('[PushManager] Service worker ready'); getState(): NotificationState {
this.ensureInitialized();
// Check if already subscribed return { ...this.state };
const subscription = await this.registration.pushManager.getSubscription(); }
this.state.subscribed = !!subscription;
this.notifyListeners();
} catch (error) {
console.error('[PushManager] Service worker initialization failed:', error);
this.state.error = 'Service worker not available';
this.notifyListeners();
}
}
/** /**
* Request notification permission * Check if push notifications are supported
*/ * SSR-safe: guarded with browser check
async requestPermission(): Promise<boolean> { */
this.ensureInitialized(); private checkSupport(): void {
if (!browser) {
if (!browser || !this.state.supported) { this.state.supported = false;
this.state.error = 'Push notifications not supported'; this.state.permission = 'denied';
this.notifyListeners(); return;
return false; }
}
if (this.state.permission === 'granted') { this.state.supported =
return true; 'serviceWorker' in navigator && 'PushManager' in window && 'Notification' in window;
}
try { this.state.permission = this.state.supported ? Notification.permission : 'denied';
this.state.loading = true; }
this.notifyListeners();
const permission = await Notification.requestPermission(); /**
this.state.permission = permission; * Initialize service worker registration
this.state.error = permission === 'denied' ? 'Permission denied' : null; * SSR-safe: guarded with browser and support checks
*/
this.state.loading = false; private async initializeServiceWorker(): Promise<void> {
this.notifyListeners(); if (!browser || !this.state.supported) return;
return permission === 'granted';
} catch (error) {
console.error('[PushManager] Permission request failed:', error);
this.state.error = 'Failed to request permission';
this.state.loading = false;
this.notifyListeners();
return false;
}
}
/** try {
* Subscribe to push notifications // Wait for service worker to be ready
*/ this.registration = await navigator.serviceWorker.ready;
async subscribe(): Promise<boolean> { console.log('[PushManager] Service worker ready');
if (!await this.requestPermission()) {
return false;
}
if (!this.registration) { // Check if already subscribed
this.state.error = 'Service worker not ready'; const subscription = await this.registration.pushManager.getSubscription();
this.notifyListeners(); this.state.subscribed = !!subscription;
return false;
}
try { this.notifyListeners();
this.state.loading = true; } catch (error) {
this.state.error = null; console.error('[PushManager] Service worker initialization failed:', error);
this.notifyListeners(); this.state.error = 'Service worker not available';
this.notifyListeners();
}
}
// Get VAPID public key from server /**
const vapidResponse = await fetch('/api/notifications/vapid-key'); * Request notification permission
if (!vapidResponse.ok) { */
throw new Error('Failed to get VAPID key'); async requestPermission(): Promise<boolean> {
} this.ensureInitialized();
const { publicKey } = await vapidResponse.json();
// Create push subscription
const subscription = await this.registration.pushManager.subscribe({
userVisibleOnly: true,
applicationServerKey: this.urlBase64ToUint8Array(publicKey)
});
// Send subscription to server if (!browser || !this.state.supported) {
const subscribeResponse = await fetch('/api/notifications/subscribe', { this.state.error = 'Push notifications not supported';
method: 'POST', this.notifyListeners();
headers: { return false;
'Content-Type': 'application/json' }
},
body: JSON.stringify({
subscription: subscription.toJSON(),
clientId: this.clientId
})
});
if (!subscribeResponse.ok) { if (this.state.permission === 'granted') {
throw new Error('Failed to register subscription with server'); return true;
} }
this.state.subscribed = true; try {
this.state.loading = false; this.state.loading = true;
this.notifyListeners(); this.notifyListeners();
console.log('[PushManager] Successfully subscribed to push notifications');
return true;
} catch (error) { const permission = await Notification.requestPermission();
console.error('[PushManager] Subscription failed:', error); this.state.permission = permission;
this.state.error = 'Failed to subscribe to notifications'; this.state.error = permission === 'denied' ? 'Permission denied' : null;
this.state.loading = false;
this.notifyListeners();
return false;
}
}
/** this.state.loading = false;
* Unsubscribe from push notifications this.notifyListeners();
*/
async unsubscribe(): Promise<boolean> {
if (!this.registration) {
this.state.error = 'Service worker not ready';
this.notifyListeners();
return false;
}
try { return permission === 'granted';
this.state.loading = true; } catch (error) {
this.state.error = null; console.error('[PushManager] Permission request failed:', error);
this.notifyListeners(); this.state.error = 'Failed to request permission';
this.state.loading = false;
this.notifyListeners();
return false;
}
}
// Get current subscription /**
const subscription = await this.registration.pushManager.getSubscription(); * Subscribe to push notifications
*/
if (subscription) { async subscribe(): Promise<boolean> {
// Unsubscribe from push service if (!(await this.requestPermission())) {
await subscription.unsubscribe(); return false;
}
// Remove from server
await fetch('/api/notifications/subscribe', {
method: 'DELETE',
headers: {
'Content-Type': 'application/json'
},
body: JSON.stringify({
clientId: this.clientId
})
});
}
this.state.subscribed = false; if (!this.registration) {
this.state.loading = false; this.state.error = 'Service worker not ready';
this.notifyListeners(); this.notifyListeners();
return false;
console.log('[PushManager] Successfully unsubscribed from push notifications'); }
return true;
} catch (error) { try {
console.error('[PushManager] Unsubscription failed:', error); this.state.loading = true;
this.state.error = 'Failed to unsubscribe from notifications'; this.state.error = null;
this.state.loading = false; this.notifyListeners();
this.notifyListeners();
return false;
}
}
/** // Get VAPID public key from server
* Toggle subscription state const vapidResponse = await fetch('/api/notifications/vapid-key');
*/ if (!vapidResponse.ok) {
async toggleSubscription(): Promise<boolean> { throw new Error('Failed to get VAPID key');
if (this.state.subscribed) { }
return await this.unsubscribe();
} else {
return await this.subscribe();
}
}
/** const { publicKey } = await vapidResponse.json();
* Generate unique client ID
* SSR-safe: guarded with browser check, uses localStorage only in browser
*/
private generateClientId(): string {
if (!browser) return '';
const stored = localStorage.getItem('push-client-id');
if (stored) return stored;
const id = `client_${Date.now()}_${Math.random().toString(36).substr(2, 9)}`; // Create push subscription
localStorage.setItem('push-client-id', id); const subscription = await this.registration.pushManager.subscribe({
return id; userVisibleOnly: true,
} applicationServerKey: this.urlBase64ToUint8Array(publicKey)
});
/** // Send subscription to server
* Convert URL-safe base64 string to Uint8Array const subscribeResponse = await fetch('/api/notifications/subscribe', {
* Enhanced with validation and error handling for VAPID keys method: 'POST',
* SSR-safe: uses window.atob only in browser context headers: {
*/ 'Content-Type': 'application/json'
private urlBase64ToUint8Array(base64String: string): Uint8Array<ArrayBuffer> { },
if (!browser) { body: JSON.stringify({
return new Uint8Array(0); subscription: subscription.toJSON(),
} clientId: this.clientId
})
});
// Input validation if (!subscribeResponse.ok) {
if (!base64String || typeof base64String !== 'string') { throw new Error('Failed to register subscription with server');
console.error('[PushManager] Invalid VAPID key: empty or non-string'); }
return new Uint8Array(0);
}
// Remove whitespace and validate format this.state.subscribed = true;
const cleanKey = base64String.trim(); this.state.loading = false;
if (cleanKey.length === 0) { this.notifyListeners();
console.error('[PushManager] Invalid VAPID key: empty string');
return new Uint8Array(0);
}
// VAPID keys should be 65 characters (unpadded base64) console.log('[PushManager] Successfully subscribed to push notifications');
if (cleanKey.length !== 65) { return true;
console.warn(`[PushManager] VAPID key length ${cleanKey.length}, expected 65`); } catch (error) {
} console.error('[PushManager] Subscription failed:', error);
this.state.error = 'Failed to subscribe to notifications';
this.state.loading = false;
this.notifyListeners();
return false;
}
}
try { /**
// Add proper padding * Unsubscribe from push notifications
const padding = '='.repeat((4 - cleanKey.length % 4) % 4); */
const base64 = (cleanKey + padding) async unsubscribe(): Promise<boolean> {
.replace(/-/g, '+') if (!this.registration) {
.replace(/_/g, '/'); this.state.error = 'Service worker not ready';
this.notifyListeners();
return false;
}
// Validate base64 format before decoding try {
const base64Regex = /^[A-Za-z0-9+\/]*={0,2}$/; this.state.loading = true;
if (!base64Regex.test(base64)) { this.state.error = null;
throw new Error('Invalid base64 characters'); this.notifyListeners();
}
const rawData = window.atob(base64); // Get current subscription
const outputArray = new Uint8Array(rawData.length); const subscription = await this.registration.pushManager.getSubscription();
for (let i = 0; i < rawData.length; ++i) { if (subscription) {
outputArray[i] = rawData.charCodeAt(i); // Unsubscribe from push service
} await subscription.unsubscribe();
console.log(`[PushManager] Successfully decoded VAPID key (${outputArray.length} bytes)`); // Remove from server
return outputArray; await fetch('/api/notifications/subscribe', {
method: 'DELETE',
headers: {
'Content-Type': 'application/json'
},
body: JSON.stringify({
clientId: this.clientId
})
});
}
} catch (error) { this.state.subscribed = false;
const errorMessage = error instanceof Error ? error.message : String(error); this.state.loading = false;
console.error('[PushManager] Failed to decode VAPID key:', error, 'Key:', cleanKey); this.notifyListeners();
throw new Error(`Invalid VAPID key format: ${errorMessage}`);
}
}
/** console.log('[PushManager] Successfully unsubscribed from push notifications');
* Notify all listeners of state change return true;
*/ } catch (error) {
private notifyListeners(): void { console.error('[PushManager] Unsubscription failed:', error);
this.listeners.forEach(callback => { this.state.error = 'Failed to unsubscribe from notifications';
try { this.state.loading = false;
callback({ ...this.state }); this.notifyListeners();
} catch (error) { return false;
console.error('[PushManager] Listener error:', error); }
} }
});
} /**
* Toggle subscription state
*/
async toggleSubscription(): Promise<boolean> {
if (this.state.subscribed) {
return await this.unsubscribe();
} else {
return await this.subscribe();
}
}
/**
* Generate unique client ID
* SSR-safe: guarded with browser check, uses localStorage only in browser
*/
private generateClientId(): string {
if (!browser) return '';
const stored = localStorage.getItem('push-client-id');
if (stored) return stored;
const id = `client_${Date.now()}_${Math.random().toString(36).substr(2, 9)}`;
localStorage.setItem('push-client-id', id);
return id;
}
/**
* Convert URL-safe base64 string to Uint8Array
* Enhanced with validation and error handling for VAPID keys
* SSR-safe: uses window.atob only in browser context
*/
private urlBase64ToUint8Array(base64String: string): Uint8Array<ArrayBuffer> {
if (!browser) {
return new Uint8Array(0);
}
// Input validation
if (!base64String || typeof base64String !== 'string') {
console.error('[PushManager] Invalid VAPID key: empty or non-string');
return new Uint8Array(0);
}
// Remove whitespace and validate format
const cleanKey = base64String.trim();
if (cleanKey.length === 0) {
console.error('[PushManager] Invalid VAPID key: empty string');
return new Uint8Array(0);
}
// VAPID keys should be 65 characters (unpadded base64)
if (cleanKey.length !== 65) {
console.warn(`[PushManager] VAPID key length ${cleanKey.length}, expected 65`);
}
try {
// Add proper padding
const padding = '='.repeat((4 - (cleanKey.length % 4)) % 4);
const base64 = (cleanKey + padding).replace(/-/g, '+').replace(/_/g, '/');
// Validate base64 format before decoding
const base64Regex = /^[A-Za-z0-9+\/]*={0,2}$/;
if (!base64Regex.test(base64)) {
throw new Error('Invalid base64 characters');
}
const rawData = window.atob(base64);
const outputArray = new Uint8Array(rawData.length);
for (let i = 0; i < rawData.length; ++i) {
outputArray[i] = rawData.charCodeAt(i);
}
console.log(`[PushManager] Successfully decoded VAPID key (${outputArray.length} bytes)`);
return outputArray;
} catch (error) {
const errorMessage = error instanceof Error ? error.message : String(error);
console.error('[PushManager] Failed to decode VAPID key:', error, 'Key:', cleanKey);
throw new Error(`Invalid VAPID key format: ${errorMessage}`);
}
}
/**
* Notify all listeners of state change
*/
private notifyListeners(): void {
this.listeners.forEach((callback) => {
try {
callback({ ...this.state });
} catch (error) {
console.error('[PushManager] Listener error:', error);
}
});
}
} }
// Singleton instance // Singleton instance
export const pushNotificationManager = new PushNotificationManager(); export const pushNotificationManager = new PushNotificationManager();
export type { NotificationState }; export type { NotificationState };

340
src/lib/client/ServiceWorkerMessageHandler.ts
View File

@@ -1,201 +1,201 @@
/** /**
* Service Worker Message Handler * Service Worker Message Handler
* *
* Handles messages from service worker (like notification actions) * Handles messages from service worker (like notification actions)
* and coordinates with the main application. * and coordinates with the main application.
*/ */
import { pushState } from "$app/navigation"; import { pushState } from '$app/navigation';
interface ServiceWorkerMessage { interface ServiceWorkerMessage {
type: string; type: string;
action?: string; action?: string;
data?: any; data?: any;
} }
class ServiceWorkerMessageHandler { class ServiceWorkerMessageHandler {
private retryCallbacks = new Map<string, () => void>(); private retryCallbacks = new Map<string, () => void>();
constructor() { constructor() {
this.initializeMessageListener(); this.initializeMessageListener();
} }
/** /**
* Listen for messages from service worker * Listen for messages from service worker
*/ */
private initializeMessageListener(): void { private initializeMessageListener(): void {
if ('serviceWorker' in navigator) { if ('serviceWorker' in navigator) {
navigator.serviceWorker.addEventListener('message', (event) => { navigator.serviceWorker.addEventListener('message', (event) => {
this.handleMessage(event.data); this.handleMessage(event.data);
}); });
} }
} }
/** /**
* Handle messages from service worker * Handle messages from service worker
*/ */
private handleMessage(message: ServiceWorkerMessage): void { private handleMessage(message: ServiceWorkerMessage): void {
console.log('[SW-Handler] Message received:', message); console.log('[SW-Handler] Message received:', message);
switch (message.type) { switch (message.type) {
case 'notification-action': case 'notification-action':
this.handleNotificationAction(message.action, message.data); this.handleNotificationAction(message.action, message.data);
break; break;
default:
console.log('[SW-Handler] Unknown message type:', message.type);
}
}
/** default:
* Handle notification action clicks console.log('[SW-Handler] Unknown message type:', message.type);
*/ }
private handleNotificationAction(action: string | undefined, data: any): void { }
if (!action || !data?.itemId) {
console.warn('[SW-Handler] Invalid notification action:', { action, data });
return;
}
switch (action) { /**
case 'view': * Handle notification action clicks
this.handleViewAction(data.itemId); */
break; private handleNotificationAction(action: string | undefined, data: any): void {
if (!action || !data?.itemId) {
case 'retry': console.warn('[SW-Handler] Invalid notification action:', { action, data });
this.handleRetryAction(data.itemId); return;
break; }
default:
console.log('[SW-Handler] Unknown notification action:', action);
}
}
/** switch (action) {
* Handle "view" action - scroll to item and highlight case 'view':
*/ this.handleViewAction(data.itemId);
private handleViewAction(itemId: string): void { break;
console.log('[SW-Handler] View action for item:', itemId);
// Find the queue item card and scroll to it
const element = document.querySelector(`[data-queue-item="${itemId}"]`);
if (element) {
element.scrollIntoView({
behavior: 'smooth',
block: 'center'
});
// Add temporary highlight effect
element.classList.add('ring-2', 'ring-blue-500');
setTimeout(() => {
element.classList.remove('ring-2', 'ring-blue-500');
}, 3000);
} else {
// If not found, navigate to homepage with highlight
const url = new URL(window.location.href);
url.searchParams.set('highlight', itemId);
pushState(url, {});
// Refresh page to show the item
//window.location.reload();
}
}
/** case 'retry':
* Handle "retry" action - trigger retry for failed item this.handleRetryAction(data.itemId);
*/ break;
private async handleRetryAction(itemId: string): Promise<void> {
console.log('[SW-Handler] Retry action for item:', itemId);
// Check if there's a registered callback
const callback = this.retryCallbacks.get(itemId);
if (callback) {
callback();
return;
}
// Fallback: direct API call default:
try { console.log('[SW-Handler] Unknown notification action:', action);
const response = await fetch(`/api/queue/${itemId}/retry`, { }
method: 'POST' }
});
if (response.ok) {
console.log('[SW-Handler] Retry initiated via API');
// Show user feedback
this.showRetryFeedback(true);
} else {
throw new Error('Retry request failed');
}
} catch (error) {
console.error('[SW-Handler] Retry failed:', error);
this.showRetryFeedback(false);
}
}
/** /**
* Register retry callback for a queue item * Handle "view" action - scroll to item and highlight
*/ */
registerRetryCallback(itemId: string, callback: () => void): void { private handleViewAction(itemId: string): void {
this.retryCallbacks.set(itemId, callback); console.log('[SW-Handler] View action for item:', itemId);
}
/** // Find the queue item card and scroll to it
* Unregister retry callback const element = document.querySelector(`[data-queue-item="${itemId}"]`);
*/ if (element) {
unregisterRetryCallback(itemId: string): void { element.scrollIntoView({
this.retryCallbacks.delete(itemId); behavior: 'smooth',
} block: 'center'
});
/** // Add temporary highlight effect
* Show retry feedback to user element.classList.add('ring-2', 'ring-blue-500');
*/ setTimeout(() => {
private showRetryFeedback(success: boolean): void { element.classList.remove('ring-2', 'ring-blue-500');
// Create temporary toast notification }, 3000);
const toast = document.createElement('div'); } else {
toast.className = `fixed bottom-4 left-4 px-4 py-2 rounded-lg text-white text-sm font-medium z-50 ${ // If not found, navigate to homepage with highlight
success ? 'bg-green-600' : 'bg-red-600' const url = new URL(window.location.href);
}`; url.searchParams.set('highlight', itemId);
toast.textContent = success pushState(url, {});
? 'Retry initiated - check the queue for updates'
: 'Failed to retry - please try again manually';
document.body.appendChild(toast);
// Auto-remove after 5 seconds
setTimeout(() => {
document.body.removeChild(toast);
}, 5000);
}
/** // Refresh page to show the item
* Send message to service worker //window.location.reload();
*/ }
async sendMessageToSW(message: any): Promise<any> { }
if (!('serviceWorker' in navigator)) {
throw new Error('Service worker not supported');
}
const registration = await navigator.serviceWorker.ready; /**
if (!registration.active) { * Handle "retry" action - trigger retry for failed item
throw new Error('Service worker not active'); */
} private async handleRetryAction(itemId: string): Promise<void> {
console.log('[SW-Handler] Retry action for item:', itemId);
return new Promise((resolve, reject) => { // Check if there's a registered callback
const channel = new MessageChannel(); const callback = this.retryCallbacks.get(itemId);
channel.port1.onmessage = (event) => { if (callback) {
resolve(event.data); callback();
}; return;
}
registration.active?.postMessage(message, [channel.port2]); // Fallback: direct API call
try {
const response = await fetch(`/api/queue/${itemId}/retry`, {
method: 'POST'
});
// Timeout after 5 seconds if (response.ok) {
setTimeout(() => { console.log('[SW-Handler] Retry initiated via API');
reject(new Error('Service worker message timeout'));
}, 5000); // Show user feedback
}); this.showRetryFeedback(true);
} } else {
throw new Error('Retry request failed');
}
} catch (error) {
console.error('[SW-Handler] Retry failed:', error);
this.showRetryFeedback(false);
}
}
/**
* Register retry callback for a queue item
*/
registerRetryCallback(itemId: string, callback: () => void): void {
this.retryCallbacks.set(itemId, callback);
}
/**
* Unregister retry callback
*/
unregisterRetryCallback(itemId: string): void {
this.retryCallbacks.delete(itemId);
}
/**
* Show retry feedback to user
*/
private showRetryFeedback(success: boolean): void {
// Create temporary toast notification
const toast = document.createElement('div');
toast.className = `fixed bottom-4 left-4 px-4 py-2 rounded-lg text-white text-sm font-medium z-50 ${
success ? 'bg-green-600' : 'bg-red-600'
}`;
toast.textContent = success
? 'Retry initiated - check the queue for updates'
: 'Failed to retry - please try again manually';
document.body.appendChild(toast);
// Auto-remove after 5 seconds
setTimeout(() => {
document.body.removeChild(toast);
}, 5000);
}
/**
* Send message to service worker
*/
async sendMessageToSW(message: any): Promise<any> {
if (!('serviceWorker' in navigator)) {
throw new Error('Service worker not supported');
}
const registration = await navigator.serviceWorker.ready;
if (!registration.active) {
throw new Error('Service worker not active');
}
return new Promise((resolve, reject) => {
const channel = new MessageChannel();
channel.port1.onmessage = (event) => {
resolve(event.data);
};
registration.active?.postMessage(message, [channel.port2]);
// Timeout after 5 seconds
setTimeout(() => {
reject(new Error('Service worker message timeout'));
}, 5000);
});
}
} }
// Singleton instance // Singleton instance
export const serviceWorkerMessageHandler = new ServiceWorkerMessageHandler(); export const serviceWorkerMessageHandler = new ServiceWorkerMessageHandler();

88
src/lib/server/api/errorHandler.ts
View File

@@ -1,15 +1,15 @@
/** /**
* API Error Handler * API Error Handler
* *
* Centralizes error handling for API endpoints by converting * Centralizes error handling for API endpoints by converting
* application errors into appropriate HTTP responses. * application errors into appropriate HTTP responses.
* *
* Maps error types to status codes: * Maps error types to status codes:
* - ValidationError → 400 Bad Request * - ValidationError → 400 Bad Request
* - NotFoundError → 404 Not Found * - NotFoundError → 404 Not Found
* - ConflictError → 409 Conflict * - ConflictError → 409 Conflict
* - Other errors → 500 Internal Server Error * - Other errors → 500 Internal Server Error
* *
* Provides consistent error response format across all API endpoints. * Provides consistent error response format across all API endpoints.
*/ */
@@ -19,46 +19,56 @@ import { logError } from '../utils/logger';
/** /**
* Handle API errors and convert to appropriate HTTP responses * Handle API errors and convert to appropriate HTTP responses
* *
* @param error - Error to handle (can be any type) * @param error - Error to handle (can be any type)
* @returns JSON response with appropriate status code and error message * @returns JSON response with appropriate status code and error message
*/ */
export function handleApiError(error: unknown): Response { export function handleApiError(error: unknown): Response {
// Log all errors for debugging // Log all errors for debugging
logError('[API Error]', error); logError('[API Error]', error);
// Handle known error types with specific status codes // Handle known error types with specific status codes
if (error instanceof ValidationError) { if (error instanceof ValidationError) {
return json({ return json(
message: error.message, {
type: 'validation_error' message: error.message,
}, { status: 400 }); type: 'validation_error'
} },
{ status: 400 }
if (error instanceof NotFoundError) { );
return json({ }
message: error.message,
type: 'not_found_error'
}, { status: 404 });
}
if (error instanceof ConflictError) {
return json({
message: error.message,
type: 'conflict_error'
}, { status: 409 });
}
// Handle generic errors if (error instanceof NotFoundError) {
const message = error instanceof Error ? error.message : 'Unknown error occurred'; return json(
{
// Don't expose internal error details in production message: error.message,
const publicMessage = process.env.NODE_ENV === 'production' type: 'not_found_error'
? 'Internal server error' },
: message; { status: 404 }
);
}
return json({ if (error instanceof ConflictError) {
message: publicMessage, return json(
type: 'server_error' {
}, { status: 500 }); message: error.message,
} type: 'conflict_error'
},
{ status: 409 }
);
}
// Handle generic errors
const message = error instanceof Error ? error.message : 'Unknown error occurred';
// Don't expose internal error details in production
const publicMessage = process.env.NODE_ENV === 'production' ? 'Internal server error' : message;
return json(
{
message: publicMessage,
type: 'server_error'
},
{ status: 500 }
);
}

32
src/lib/server/api/errors.ts
View File

@@ -1,11 +1,11 @@
/** /**
* Custom Error Classes for API Error Handling * Custom Error Classes for API Error Handling
* *
* Defines specific error types that map to HTTP status codes: * Defines specific error types that map to HTTP status codes:
* - ValidationError → 400 Bad Request * - ValidationError → 400 Bad Request
* - NotFoundError → 404 Not Found * - NotFoundError → 404 Not Found
* - ConflictError → 409 Conflict * - ConflictError → 409 Conflict
* *
* Used by API endpoints to throw meaningful errors that are * Used by API endpoints to throw meaningful errors that are
* caught and converted to proper HTTP responses by errorHandler.ts * caught and converted to proper HTTP responses by errorHandler.ts
*/ */
@@ -15,10 +15,10 @@
* Thrown when request data is invalid or malformed * Thrown when request data is invalid or malformed
*/ */
export class ValidationError extends Error { export class ValidationError extends Error {
constructor(message: string) { constructor(message: string) {
super(message); super(message);
this.name = 'ValidationError'; this.name = 'ValidationError';
} }
} }
/** /**
@@ -26,10 +26,10 @@ export class ValidationError extends Error {
* Thrown when requested resource does not exist * Thrown when requested resource does not exist
*/ */
export class NotFoundError extends Error { export class NotFoundError extends Error {
constructor(message: string) { constructor(message: string) {
super(message); super(message);
this.name = 'NotFoundError'; this.name = 'NotFoundError';
} }
} }
/** /**
@@ -37,8 +37,8 @@ export class NotFoundError extends Error {
* Thrown when operation conflicts with current resource state * Thrown when operation conflicts with current resource state
*/ */
export class ConflictError extends Error { export class ConflictError extends Error {
constructor(message: string) { constructor(message: string) {
super(message); super(message);
this.name = 'ConflictError'; this.name = 'ConflictError';
} }
} }

240
src/lib/server/browser.ts
View File

@@ -1,120 +1,120 @@
import { chromium } from 'playwright-extra'; import { chromium } from 'playwright-extra';
import type { Browser, BrowserContext } from 'playwright'; import type { Browser, BrowserContext } from 'playwright';
import StealthPlugin from 'puppeteer-extra-plugin-stealth'; import StealthPlugin from 'puppeteer-extra-plugin-stealth';
import fs from 'fs'; import fs from 'fs';
// Apply stealth plugin with all evasion techniques // Apply stealth plugin with all evasion techniques
chromium.use(StealthPlugin()); chromium.use(StealthPlugin());
let browser: Browser | null = null; let browser: Browser | null = null;
interface BrowserOptions { interface BrowserOptions {
userAgent?: string; userAgent?: string;
viewport?: { width: number; height: number }; viewport?: { width: number; height: number };
locale?: string; locale?: string;
timezone?: string; timezone?: string;
} }
export async function initializeBrowser(): Promise<Browser> { export async function initializeBrowser(): Promise<Browser> {
if (browser) { if (browser) {
return browser; return browser;
} }
console.log('Initializing Playwright browser...'); console.log('Initializing Playwright browser...');
// Use environment variable or let Playwright use its bundled browser // Use environment variable or let Playwright use its bundled browser
const executablePath = process.env.CHROMIUM_EXECUTABLE_PATH || '/usr/bin/google-chrome'; const executablePath = process.env.CHROMIUM_EXECUTABLE_PATH || '/usr/bin/google-chrome';
const launchOptions: Parameters<typeof chromium.launch>[0] = { const launchOptions: Parameters<typeof chromium.launch>[0] = {
headless: true, headless: true,
args: [ args: [
'--disable-blink-features=AutomationControlled', '--disable-blink-features=AutomationControlled',
'--disable-dev-shm-usage', '--disable-dev-shm-usage',
'--no-sandbox', '--no-sandbox',
'--disable-setuid-sandbox', '--disable-setuid-sandbox',
'--disable-gpu' '--disable-gpu'
] ]
}; };
// In test environment, let Playwright use bundled browser // In test environment, let Playwright use bundled browser
if (process.env.NODE_ENV !== 'test' && process.env.VITEST !== 'true') { if (process.env.NODE_ENV !== 'test' && process.env.VITEST !== 'true') {
launchOptions.executablePath = executablePath; launchOptions.executablePath = executablePath;
} }
browser = await chromium.launch(launchOptions); browser = await chromium.launch(launchOptions);
console.log('Browser initialized successfully'); console.log('Browser initialized successfully');
return browser; return browser;
} }
export async function getBrowser(): Promise<Browser> { export async function getBrowser(): Promise<Browser> {
if (!browser || !browser.isConnected()) { if (!browser || !browser.isConnected()) {
if (browser) { if (browser) {
console.warn('Browser is disconnected. Re-initializing...'); console.warn('Browser is disconnected. Re-initializing...');
try { try {
await browser.close(); await browser.close();
} catch (e) { } catch (e) {
/* ignore */ /* ignore */
} }
browser = null; browser = null;
} }
return initializeBrowser(); return initializeBrowser();
} }
return browser; return browser;
} }
export async function createBrowserContext( export async function createBrowserContext(
authStoragePath?: string, authStoragePath?: string,
options?: BrowserOptions options?: BrowserOptions
): Promise<BrowserContext> { ): Promise<BrowserContext> {
const browserInstance = await getBrowser(); const browserInstance = await getBrowser();
// Default stealth options // Default stealth options
const defaultOptions: BrowserOptions = { const defaultOptions: BrowserOptions = {
userAgent: userAgent:
'Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.0.0 Safari/537.36', 'Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.0.0 Safari/537.36',
viewport: { width: 1080, height: 1920 }, viewport: { width: 1080, height: 1920 },
locale: 'en-US', locale: 'en-US',
timezone: 'America/New_York' timezone: 'America/New_York'
}; };
const finalOptions = { ...defaultOptions, ...options }; const finalOptions = { ...defaultOptions, ...options };
// Load auth if available // Load auth if available
let context: BrowserContext; let context: BrowserContext;
const contextOptions = { const contextOptions = {
storageState: authStoragePath && fs.existsSync(authStoragePath) ? authStoragePath : undefined, storageState: authStoragePath && fs.existsSync(authStoragePath) ? authStoragePath : undefined,
userAgent: finalOptions.userAgent, userAgent: finalOptions.userAgent,
viewport: finalOptions.viewport, viewport: finalOptions.viewport,
locale: finalOptions.locale, locale: finalOptions.locale,
timezoneId: finalOptions.timezone, timezoneId: finalOptions.timezone,
permissions: [], permissions: [],
colorScheme: 'light' as const colorScheme: 'light' as const
}; };
if (authStoragePath && fs.existsSync(authStoragePath)) { if (authStoragePath && fs.existsSync(authStoragePath)) {
console.log('Loading authentication from:', authStoragePath); console.log('Loading authentication from:', authStoragePath);
} else { } else {
console.warn('No auth storage found. Running as guest.'); console.warn('No auth storage found. Running as guest.');
} }
context = await browserInstance.newContext(contextOptions); context = await browserInstance.newContext(contextOptions);
// Note: Anti-detection scripts are now handled automatically by the stealth plugin // Note: Anti-detection scripts are now handled automatically by the stealth plugin
// The plugin applies 15+ evasion techniques including: // The plugin applies 15+ evasion techniques including:
// - navigator.webdriver masking // - navigator.webdriver masking
// - chrome.runtime mocking // - chrome.runtime mocking
// - User-Agent override // - User-Agent override
// - WebGL fingerprinting evasion // - WebGL fingerprinting evasion
// - And many more... // - And many more...
return context; return context;
} }
export async function closeBrowser(): Promise<void> { export async function closeBrowser(): Promise<void> {
if (browser) { if (browser) {
console.log('Closing Playwright browser...'); console.log('Closing Playwright browser...');
await browser.close(); await browser.close();
browser = null; browser = null;
} }
} }

3326
src/lib/server/extraction.ts
View File

File diff suppressed because it is too large Load Diff

6
src/lib/server/llm.ts
View File

@@ -56,9 +56,9 @@ export async function checkModelAvailability(
const { client } = createLLM(); const { client } = createLLM();
const response = await client.models.list(); const response = await client.models.list();
const models = response.data || []; const models = response.data || [];
const foundModel = models.find((m) => m.id === model); const foundModel = models.find((m) => m.id === model);
if (foundModel) { if (foundModel) {
console.log('[LLM] Model available:', model); console.log('[LLM] Model available:', model);
return { available: true }; return { available: true };
@@ -78,4 +78,4 @@ export async function checkModelAvailability(
message: `Failed to check model availability: ${(e as Error).message}` message: `Failed to check model availability: ${(e as Error).message}`
}; };
} }
} }

406
src/lib/server/notifications/PushNotificationService.ts
View File

@@ -1,6 +1,6 @@
/** /**
* Push Notification Service for InstaRecipe Queue System * Push Notification Service for InstaRecipe Queue System
* *
* Handles web push notifications for background processing updates * Handles web push notifications for background processing updates
* when users are not actively viewing the application. * when users are not actively viewing the application.
*/ */
@@ -10,233 +10,237 @@ import webpush from 'web-push';
import { queueConfig } from '../queue/config'; import { queueConfig } from '../queue/config';
interface PushSubscription { interface PushSubscription {
endpoint: string; endpoint: string;
keys: { keys: {
p256dh: string; p256dh: string;
auth: string; auth: string;
}; };
} }
interface NotificationPayload { interface NotificationPayload {
title?: string; title?: string;
body: string; body: string;
type: 'success' | 'error' | 'progress'; type: 'success' | 'error' | 'progress';
itemId: string; itemId: string;
recipeName?: string; recipeName?: string;
tag?: string; tag?: string;
requireInteraction?: boolean; requireInteraction?: boolean;
analytics?: any; analytics?: any;
} }
class PushNotificationService { class PushNotificationService {
private subscriptions = new Map<string, PushSubscription>(); private subscriptions = new Map<string, PushSubscription>();
private vapidKeys: { publicKey: string; privateKey: string } | null = null; private vapidKeys: { publicKey: string; privateKey: string } | null = null;
constructor() { constructor() {
this.loadVapidKeys(); this.loadVapidKeys();
// Configure web-push with VAPID details
if (this.vapidKeys) {
webpush.setVapidDetails(
queueConfig.push.vapidEmail,
this.vapidKeys.publicKey,
this.vapidKeys.privateKey
);
}
}
/** // Configure web-push with VAPID details
* Load VAPID keys for push notifications if (this.vapidKeys) {
* In production, these should be stored securely and loaded from environment webpush.setVapidDetails(
*/ queueConfig.push.vapidEmail,
private loadVapidKeys() { this.vapidKeys.publicKey,
// Load from config module which uses SvelteKit's $env/dynamic/private this.vapidKeys.privateKey
this.vapidKeys = { );
publicKey: queueConfig.push.vapidPublicKey, }
privateKey: queueConfig.push.vapidPrivateKey }
};
}
/** /**
* Get the public VAPID key for client-side subscription * Load VAPID keys for push notifications
*/ * In production, these should be stored securely and loaded from environment
getPublicVapidKey(): string | null { */
return this.vapidKeys?.publicKey || null; private loadVapidKeys() {
} // Load from config module which uses SvelteKit's $env/dynamic/private
this.vapidKeys = {
publicKey: queueConfig.push.vapidPublicKey,
privateKey: queueConfig.push.vapidPrivateKey
};
}
/** /**
* Subscribe a client to push notifications * Get the public VAPID key for client-side subscription
*/ */
async subscribe(clientId: string, subscription: PushSubscription): Promise<void> { getPublicVapidKey(): string | null {
console.log(`[PushService] Subscribing client ${clientId}`); return this.vapidKeys?.publicKey || null;
this.subscriptions.set(clientId, subscription); }
// In production, store subscriptions in database
// For development, we'll keep them in memory
}
/** /**
* Unsubscribe a client from push notifications * Subscribe a client to push notifications
*/ */
async unsubscribe(clientId: string): Promise<void> { async subscribe(clientId: string, subscription: PushSubscription): Promise<void> {
console.log(`[PushService] Unsubscribing client ${clientId}`); console.log(`[PushService] Subscribing client ${clientId}`);
this.subscriptions.delete(clientId); this.subscriptions.set(clientId, subscription);
}
/** // In production, store subscriptions in database
* Send notification to all subscribed clients // For development, we'll keep them in memory
*/ }
async sendNotification(payload: NotificationPayload): Promise<void> {
if (this.subscriptions.size === 0) {
console.log('[PushService] No subscriptions, skipping notification');
return;
}
console.log(`[PushService] Sending notification to ${this.subscriptions.size} subscribers`); /**
console.log(`[PushService] Notification payload:`, payload); * Unsubscribe a client from push notifications
*/
async unsubscribe(clientId: string): Promise<void> {
console.log(`[PushService] Unsubscribing client ${clientId}`);
this.subscriptions.delete(clientId);
}
// In a real implementation, this would use web-push library /**
// For development/demo purposes, we'll simulate the notification * Send notification to all subscribed clients
const notificationData = { */
...payload, async sendNotification(payload: NotificationPayload): Promise<void> {
timestamp: new Date().toISOString() if (this.subscriptions.size === 0) {
}; console.log('[PushService] No subscriptions, skipping notification');
return;
}
for (const [clientId, subscription] of this.subscriptions) { console.log(`[PushService] Sending notification to ${this.subscriptions.size} subscribers`);
try { console.log(`[PushService] Notification payload:`, payload);
await this.sendToSubscription(subscription, notificationData);
console.log(`[PushService] ✓ Sent notification to client ${clientId}`);
} catch (error) {
console.error(`[PushService] ✗ Failed to send to client ${clientId}:`, error);
// Remove invalid subscriptions
this.subscriptions.delete(clientId);
}
}
}
/** // In a real implementation, this would use web-push library
* Send notification to specific subscription // For development/demo purposes, we'll simulate the notification
*/ const notificationData = {
private async sendToSubscription(subscription: PushSubscription, data: any): Promise<void> { ...payload,
try { timestamp: new Date().toISOString()
const payload = JSON.stringify(data); };
await webpush.sendNotification(
{
endpoint: subscription.endpoint,
keys: {
p256dh: subscription.keys.p256dh,
auth: subscription.keys.auth
}
},
payload,
{
TTL: 60 * 60 * 24, // 24 hours
}
);
console.log(`[PushService] ✓ Sent notification to ${subscription.endpoint.substring(0, 50)}...`);
} catch (error) {
// Check if subscription is expired/invalid
if ((error as any).statusCode === 410) {
console.warn(`[PushService] Subscription expired: ${subscription.endpoint.substring(0, 50)}...`);
throw new Error('Subscription expired');
}
console.error('[PushService] Failed to send notification:', {
endpoint: subscription.endpoint.substring(0, 50) + '...',
error: error instanceof Error ? error.message : String(error)
});
throw error;
}
}
/** for (const [clientId, subscription] of this.subscriptions) {
* Send success notification when recipe extraction completes try {
*/ await this.sendToSubscription(subscription, notificationData);
async notifySuccess(itemId: string, recipeName?: string, tandoorUrl?: string): Promise<void> { console.log(`[PushService] ✓ Sent notification to client ${clientId}`);
const payload: NotificationPayload = { } catch (error) {
type: 'success', console.error(`[PushService] ✗ Failed to send to client ${clientId}:`, error);
itemId, // Remove invalid subscriptions
recipeName, this.subscriptions.delete(clientId);
body: recipeName }
? `Recipe "${recipeName}" has been extracted and saved successfully!` }
: 'Your recipe extraction is complete and ready to view.', }
tag: `recipe-success-${itemId}`,
requireInteraction: true,
analytics: {
event: 'recipe_extraction_complete',
itemId,
timestamp: Date.now()
}
};
if (tandoorUrl) { /**
payload.body += ' View it in Tandoor.'; * Send notification to specific subscription
} */
private async sendToSubscription(subscription: PushSubscription, data: any): Promise<void> {
try {
const payload = JSON.stringify(data);
await this.sendNotification(payload); await webpush.sendNotification(
} {
endpoint: subscription.endpoint,
keys: {
p256dh: subscription.keys.p256dh,
auth: subscription.keys.auth
}
},
payload,
{
TTL: 60 * 60 * 24 // 24 hours
}
);
/** console.log(
* Send error notification when recipe extraction fails `[PushService] ✓ Sent notification to ${subscription.endpoint.substring(0, 50)}...`
*/ );
async notifyError(itemId: string, error: string): Promise<void> { } catch (error) {
const payload: NotificationPayload = { // Check if subscription is expired/invalid
type: 'error', if ((error as any).statusCode === 410) {
itemId, console.warn(
body: `Recipe extraction failed: ${error}. Tap to retry.`, `[PushService] Subscription expired: ${subscription.endpoint.substring(0, 50)}...`
tag: `recipe-error-${itemId}`, );
requireInteraction: true, throw new Error('Subscription expired');
analytics: { }
event: 'recipe_extraction_failed',
itemId,
error,
timestamp: Date.now()
}
};
await this.sendNotification(payload); console.error('[PushService] Failed to send notification:', {
} endpoint: subscription.endpoint.substring(0, 50) + '...',
error: error instanceof Error ? error.message : String(error)
});
throw error;
}
}
/** /**
* Send progress notification for long-running extractions * Send success notification when recipe extraction completes
*/ */
async notifyProgress(itemId: string, phase: string): Promise<void> { async notifySuccess(itemId: string, recipeName?: string, tandoorUrl?: string): Promise<void> {
const payload: NotificationPayload = { const payload: NotificationPayload = {
type: 'progress', type: 'success',
itemId, itemId,
body: `Recipe extraction in progress: ${phase}`, recipeName,
tag: `recipe-progress-${itemId}`, body: recipeName
requireInteraction: false, ? `Recipe "${recipeName}" has been extracted and saved successfully!`
analytics: { : 'Your recipe extraction is complete and ready to view.',
event: 'recipe_extraction_progress', tag: `recipe-success-${itemId}`,
itemId, requireInteraction: true,
phase, analytics: {
timestamp: Date.now() event: 'recipe_extraction_complete',
} itemId,
}; timestamp: Date.now()
}
};
await this.sendNotification(payload); if (tandoorUrl) {
} payload.body += ' View it in Tandoor.';
}
/** await this.sendNotification(payload);
* Get subscription count for monitoring }
*/
getSubscriptionCount(): number {
return this.subscriptions.size;
}
/** /**
* Clear all subscriptions (for testing/cleanup) * Send error notification when recipe extraction fails
*/ */
clearAllSubscriptions(): void { async notifyError(itemId: string, error: string): Promise<void> {
console.log('[PushService] Clearing all subscriptions'); const payload: NotificationPayload = {
this.subscriptions.clear(); type: 'error',
} itemId,
body: `Recipe extraction failed: ${error}. Tap to retry.`,
tag: `recipe-error-${itemId}`,
requireInteraction: true,
analytics: {
event: 'recipe_extraction_failed',
itemId,
error,
timestamp: Date.now()
}
};
await this.sendNotification(payload);
}
/**
* Send progress notification for long-running extractions
*/
async notifyProgress(itemId: string, phase: string): Promise<void> {
const payload: NotificationPayload = {
type: 'progress',
itemId,
body: `Recipe extraction in progress: ${phase}`,
tag: `recipe-progress-${itemId}`,
requireInteraction: false,
analytics: {
event: 'recipe_extraction_progress',
itemId,
phase,
timestamp: Date.now()
}
};
await this.sendNotification(payload);
}
/**
* Get subscription count for monitoring
*/
getSubscriptionCount(): number {
return this.subscriptions.size;
}
/**
* Clear all subscriptions (for testing/cleanup)
*/
clearAllSubscriptions(): void {
console.log('[PushService] Clearing all subscriptions');
this.subscriptions.clear();
}
} }
// Singleton instance // Singleton instance
export const pushNotificationService = new PushNotificationService(); export const pushNotificationService = new PushNotificationService();
export type { PushSubscription, NotificationPayload }; export type { PushSubscription, NotificationPayload };

420
src/lib/server/parser.ts
View File

@@ -1,208 +1,212 @@
import { createLLM, checkModelAvailability } from './llm'; import { createLLM, checkModelAvailability } from './llm';
import { zodResponseFormat } from 'openai/helpers/zod'; import { zodResponseFormat } from 'openai/helpers/zod';
import { z } from 'zod'; import { z } from 'zod';
import { RECIPE_DETECTION_PROMPT, RECIPE_EXTRACTION_PROMPT } from './prompts/recipe-extraction'; import { RECIPE_DETECTION_PROMPT, RECIPE_EXTRACTION_PROMPT } from './prompts/recipe-extraction';
import { logError } from './utils/logger'; import { logError } from './utils/logger';
const RecipeSchema = z.object({ const RecipeSchema = z.object({
name: z.string(), name: z.string(),
servings: z.number().nullable(), servings: z.number().nullable(),
description: z.string().nullable(), description: z.string().nullable(),
ingredients: z.array( ingredients: z
z.object({ .array(
item: z.string(), z.object({
amount: z.string(), item: z.string(),
unit: z.string() amount: z.string(),
}) unit: z.string()
).nullable(), })
steps: z.array(z.string()).nullable(), )
image: z.string().nullable().optional() .nullable(),
}); steps: z.array(z.string()).nullable(),
image: z.string().nullable().optional()
export type Recipe = z.infer<typeof RecipeSchema>; });
/** export type Recipe = z.infer<typeof RecipeSchema>;
* Detect if the text contains a recipe using binary classification
* @param text - The text to analyze /**
* @returns True if a recipe is detected, false otherwise * Detect if the text contains a recipe using binary classification
*/ * @param text - The text to analyze
export async function detectRecipe(text: string): Promise<boolean> { * @returns True if a recipe is detected, false otherwise
try { */
const { client, model } = createLLM(); export async function detectRecipe(text: string): Promise<boolean> {
try {
console.log('[LLM] Starting recipe detection...'); const { client, model } = createLLM();
console.log('[LLM] Model:', model);
console.log('[LLM] Text length:', text.length); console.log('[LLM] Starting recipe detection...');
console.log('[LLM] Model:', model);
const detectionResponse = await client.chat.completions.create({ console.log('[LLM] Text length:', text.length);
model,
messages: [ const detectionResponse = await client.chat.completions.create({
{ model,
role: 'system', messages: [
content: RECIPE_DETECTION_PROMPT {
}, role: 'system',
{ content: RECIPE_DETECTION_PROMPT
role: 'user', },
content: `Does this text contain a recipe?\n\n${text}` {
} role: 'user',
], content: `Does this text contain a recipe?\n\n${text}`
max_tokens: 10, }
temperature: 0 ],
}); max_tokens: 10,
temperature: 0
const detectionResult = detectionResponse.choices[0].message.content?.toLowerCase() ?? ''; });
console.log('[LLM] Detection response:', detectionResult);
const detectionResult = detectionResponse.choices[0].message.content?.toLowerCase() ?? '';
return detectionResult.includes('yes'); console.log('[LLM] Detection response:', detectionResult);
} catch (e) {
logError('[LLM] Recipe detection error', e); return detectionResult.includes('yes');
} catch (e) {
// Check if this is a model-related error logError('[LLM] Recipe detection error', e);
const errorMessage = (e as Error).message || '';
const isModelError = errorMessage.includes('400') && // Check if this is a model-related error
(errorMessage.toLowerCase().includes('model') || const errorMessage = (e as Error).message || '';
errorMessage.toLowerCase().includes('load')); const isModelError =
errorMessage.includes('400') &&
if (isModelError) { (errorMessage.toLowerCase().includes('model') || errorMessage.toLowerCase().includes('load'));
const { model } = createLLM();
const modelCheck = await checkModelAvailability(model); if (isModelError) {
if (!modelCheck.available) { const { model } = createLLM();
throw new Error(modelCheck.message || `Model "${model}" is not available`); const modelCheck = await checkModelAvailability(model);
} if (!modelCheck.available) {
} throw new Error(modelCheck.message || `Model "${model}" is not available`);
}
throw new Error(`Failed to detect recipe: ${(e as Error).message}`); }
}
} throw new Error(`Failed to detect recipe: ${(e as Error).message}`);
}
/** }
* Extract recipe data from text using LLM structured output
* @param text - The text containing the recipe /**
* @returns Parsed recipe object * Extract recipe data from text using LLM structured output
*/ * @param text - The text containing the recipe
export async function parseRecipe(text: string): Promise<Recipe> { * @returns Parsed recipe object
try { */
const { client, model } = createLLM(); export async function parseRecipe(text: string): Promise<Recipe> {
try {
console.log('[LLM] Starting recipe parsing...'); const { client, model } = createLLM();
console.log('[LLM] Model:', model);
console.log('[LLM] Starting recipe parsing...');
const completion = await client.beta.chat.completions.parse({ console.log('[LLM] Model:', model);
model,
messages: [ const completion = await client.beta.chat.completions.parse({
{ model,
role: 'system', messages: [
content: RECIPE_EXTRACTION_PROMPT {
}, role: 'system',
{ content: RECIPE_EXTRACTION_PROMPT
role: 'user', },
content: `Extract the recipe from this text:\n\n${text}` {
} role: 'user',
], content: `Extract the recipe from this text:\n\n${text}`
response_format: zodResponseFormat(RecipeSchema, 'recipe'), }
temperature: 0.3 ],
}); response_format: zodResponseFormat(RecipeSchema, 'recipe'),
temperature: 0.3
const recipe = completion.choices[0].message.parsed; });
console.log('[LLM] Parse response:', recipe?.name);
const recipe = completion.choices[0].message.parsed;
if (!recipe || !recipe.name) { console.log('[LLM] Parse response:', recipe?.name);
throw new Error('Failed to extract recipe - missing name');
} if (!recipe || !recipe.name) {
throw new Error('Failed to extract recipe - missing name');
return recipe; }
} catch (e) {
logError('[LLM] Recipe parsing error', e); return recipe;
} catch (e) {
// Check if this is a model-related error logError('[LLM] Recipe parsing error', e);
const errorMessage = (e as Error).message || '';
const isModelError = errorMessage.includes('400') && // Check if this is a model-related error
(errorMessage.toLowerCase().includes('model') || const errorMessage = (e as Error).message || '';
errorMessage.toLowerCase().includes('load')); const isModelError =
errorMessage.includes('400') &&
if (isModelError) { (errorMessage.toLowerCase().includes('model') || errorMessage.toLowerCase().includes('load'));
const { model } = createLLM();
const modelCheck = await checkModelAvailability(model); if (isModelError) {
if (!modelCheck.available) { const { model } = createLLM();
throw new Error(modelCheck.message || `Model "${model}" is not available`); const modelCheck = await checkModelAvailability(model);
} if (!modelCheck.available) {
} throw new Error(modelCheck.message || `Model "${model}" is not available`);
}
// If structured output fails, try standard completion }
if ((e as any).message?.includes('response_format') ||
(e as any).message?.includes('structured output')) { // If structured output fails, try standard completion
console.warn('[LLM] Falling back to standard completion'); if (
return await parseRecipeWithStandardCompletion(text); (e as any).message?.includes('response_format') ||
} (e as any).message?.includes('structured output')
) {
throw new Error(`Failed to parse recipe: ${(e as Error).message}`); console.warn('[LLM] Falling back to standard completion');
} return await parseRecipeWithStandardCompletion(text);
} }
/** throw new Error(`Failed to parse recipe: ${(e as Error).message}`);
* Complete workflow: detect recipe and parse if found }
* @param text - The text to analyze }
* @returns Parsed recipe object if detected, null otherwise
*/ /**
export async function extractRecipe(text: string): Promise<Recipe | null> { * Complete workflow: detect recipe and parse if found
const isRecipe = await detectRecipe(text); * @param text - The text to analyze
* @returns Parsed recipe object if detected, null otherwise
if (!isRecipe) { */
return null; export async function extractRecipe(text: string): Promise<Recipe | null> {
} const isRecipe = await detectRecipe(text);
return parseRecipe(text); if (!isRecipe) {
} return null;
}
/**
* Fallback parser using standard completion (no structured output) return parseRecipe(text);
* Used when the model doesn't support beta.chat.completions.parse() }
*/
async function parseRecipeWithStandardCompletion(text: string): Promise<Recipe> { /**
const { client, model } = createLLM(); * Fallback parser using standard completion (no structured output)
* Used when the model doesn't support beta.chat.completions.parse()
console.log('[LLM] Using standard completion fallback'); */
async function parseRecipeWithStandardCompletion(text: string): Promise<Recipe> {
const completion = await client.chat.completions.create({ const { client, model } = createLLM();
model,
messages: [ console.log('[LLM] Using standard completion fallback');
{
role: 'system', const completion = await client.chat.completions.create({
content: `You are a recipe extractor. Return ONLY valid JSON matching this schema: model,
{ messages: [
"name": "recipe name in Italian", {
"servings": number or null, role: 'system',
"description": "description in Italian or null", content: `You are a recipe extractor. Return ONLY valid JSON matching this schema:
"ingredients": [{"item": "ingredient name", "amount": "quantity", "unit": "SI unit"}], {
"steps": ["First step", "Second step", ...] "name": "recipe name in Italian",
} "servings": number or null,
"description": "description in Italian or null",
Convert all measurements to SI units (g, mL, °C). "ingredients": [{"item": "ingredient name", "amount": "quantity", "unit": "SI unit"}],
Translate everything to Italian. "steps": ["First step", "Second step", ...]
Extract ONLY what's in the text.` }
},
{ Convert all measurements to SI units (g, mL, °C).
role: 'user', Translate everything to Italian.
content: `Extract the recipe from this text:\n\n${text}` Extract ONLY what's in the text.`
} },
], {
max_tokens: 2000, role: 'user',
temperature: 0.3 content: `Extract the recipe from this text:\n\n${text}`
}); }
],
const jsonResponse = completion.choices[0].message.content; max_tokens: 2000,
if (!jsonResponse) { temperature: 0.3
throw new Error('Empty response from LLM'); });
}
const jsonResponse = completion.choices[0].message.content;
console.log('[LLM] Standard completion raw response:', jsonResponse.substring(0, 200)); if (!jsonResponse) {
throw new Error('Empty response from LLM');
// Parse and validate JSON (remove code fences if present) }
const cleanedJson = jsonResponse.replace(/```json\n?|```\n?/g, '').trim();
const parsedData = JSON.parse(cleanedJson); console.log('[LLM] Standard completion raw response:', jsonResponse.substring(0, 200));
const recipe = RecipeSchema.parse(parsedData);
// Parse and validate JSON (remove code fences if present)
console.log('[LLM] Standard completion parsed recipe:', recipe.name); const cleanedJson = jsonResponse.replace(/```json\n?|```\n?/g, '').trim();
const parsedData = JSON.parse(cleanedJson);
return recipe; const recipe = RecipeSchema.parse(parsedData);
}
console.log('[LLM] Standard completion parsed recipe:', recipe.name);
return recipe;
}

799
src/lib/server/queue/QueueManager.ts
View File

@@ -1,9 +1,9 @@
/** /**
* Queue Manager - Core queue operations and event management * Queue Manager - Core queue operations and event management
* *
* Manages an in-memory queue of Instagram URL processing jobs. * Manages an in-memory queue of Instagram URL processing jobs.
* Provides CRUD operations and pub/sub mechanism for queue updates. * Provides CRUD operations and pub/sub mechanism for queue updates.
* *
* Architecture: Domain Layer (Hexagonal Architecture) * Architecture: Domain Layer (Hexagonal Architecture)
* - Port: Defines queue operations interface * - Port: Defines queue operations interface
* - Implementation: In-memory Map-based storage * - Implementation: In-memory Map-based storage
@@ -16,427 +16,428 @@ import type { QueueItem, QueueItemStatus, QueueStatusUpdate, QueueUpdateCallback
/** /**
* Singleton queue manager for processing Instagram URLs * Singleton queue manager for processing Instagram URLs
* *
* Features: * Features:
* - FIFO queue with unique IDs * - FIFO queue with unique IDs
* - Status tracking and updates * - Status tracking and updates
* - Progress event accumulation * - Progress event accumulation
* - Retry support for failed items * - Retry support for failed items
* - Pub/sub for real-time updates * - Pub/sub for real-time updates
* *
* @example * @example
* ```typescript * ```typescript
* import { queueManager } from './QueueManager'; * import { queueManager } from './QueueManager';
* *
* // Add item to queue * // Add item to queue
* const item = queueManager.enqueue('https://instagram.com/p/abc123'); * const item = queueManager.enqueue('https://instagram.com/p/abc123');
* *
* // Subscribe to updates * // Subscribe to updates
* const unsubscribe = queueManager.subscribe((update) => { * const unsubscribe = queueManager.subscribe((update) => {
* console.log('Item updated:', update); * console.log('Item updated:', update);
* }); * });
* *
* // Get all items * // Get all items
* const items = queueManager.getAll(); * const items = queueManager.getAll();
* ``` * ```
*/ */
export class QueueManager { export class QueueManager {
/** Map of queue items by ID */ /** Map of queue items by ID */
private items: Map<string, QueueItem> = new Map(); private items: Map<string, QueueItem> = new Map();
/** Set of subscriber callbacks */ /** Set of subscriber callbacks */
private subscribers: Set<QueueUpdateCallback> = new Set(); private subscribers: Set<QueueUpdateCallback> = new Set();
/** /**
* Add URL to processing queue * Add URL to processing queue
* *
* @param url - Instagram URL to process * @param url - Instagram URL to process
* @returns Newly created queue item * @returns Newly created queue item
* *
* @example * @example
* ```typescript * ```typescript
* const item = queueManager.enqueue('https://instagram.com/p/abc123'); * const item = queueManager.enqueue('https://instagram.com/p/abc123');
* console.log('Queued with ID:', item.id); * console.log('Queued with ID:', item.id);
* ``` * ```
*/ */
enqueue(url: string): QueueItem { enqueue(url: string): QueueItem {
const now = new Date().toISOString(); const now = new Date().toISOString();
const item: QueueItem = { const item: QueueItem = {
id: uuidv4(), id: uuidv4(),
url, url,
status: 'pending', status: 'pending',
enqueuedAt: now, enqueuedAt: now,
createdAt: now, createdAt: now,
updatedAt: now, updatedAt: now,
phases: [ phases: [
{ name: 'extraction', status: 'pending' }, { name: 'extraction', status: 'pending' },
{ name: 'parsing', status: 'pending' }, { name: 'parsing', status: 'pending' },
{ name: 'uploading', status: 'pending' } { name: 'uploading', status: 'pending' }
], ],
logs: [], logs: [],
progressEvents: [], progressEvents: [],
retryCount: 0, retryCount: 0,
maxRetries: 3 maxRetries: 3
}; };
this.items.set(item.id, item); this.items.set(item.id, item);
this.notifySubscribers({ this.notifySubscribers({
type: 'status_change', type: 'status_change',
itemId: item.id, itemId: item.id,
status: 'pending', status: 'pending',
url: item.url, url: item.url,
timestamp: now, timestamp: now,
progress: item.phases progress: item.phases
}); });
return item; return item;
} }
/** /**
* Get next pending item for processing (FIFO) * Get next pending item for processing (FIFO)
* *
* Automatically marks the item as in_progress when dequeued. * Automatically marks the item as in_progress when dequeued.
* *
* @returns Next pending item, or null if queue is empty * @returns Next pending item, or null if queue is empty
* *
* @example * @example
* ```typescript * ```typescript
* const item = queueManager.dequeue(); * const item = queueManager.dequeue();
* if (item) { * if (item) {
* // Process item * // Process item
* console.log('Processing:', item.url); * console.log('Processing:', item.url);
* } * }
* ``` * ```
*/ */
dequeue(): QueueItem | null { dequeue(): QueueItem | null {
for (const item of this.items.values()) { for (const item of this.items.values()) {
if (item.status === 'pending') { if (item.status === 'pending') {
this.updateStatus(item.id, 'in_progress', { phase: 'extraction' }); this.updateStatus(item.id, 'in_progress', { phase: 'extraction' });
return item; return item;
} }
} }
return null; return null;
} }
/** /**
* Update item status and optional data * Update item status and optional data
* *
* Handles status-specific logic: * Handles status-specific logic:
* - Sets startedAt when transitioning to in_progress * - Sets startedAt when transitioning to in_progress
* - Sets completedAt when transitioning to success/error * - Sets completedAt when transitioning to success/error
* - Updates currentPhase for in_progress status * - Updates currentPhase for in_progress status
* *
* @param itemId - ID of item to update * @param itemId - ID of item to update
* @param status - New status * @param status - New status
* @param data - Optional additional data to merge into item * @param data - Optional additional data to merge into item
* *
* @example * @example
* ```typescript * ```typescript
* queueManager.updateStatus(itemId, 'in_progress', { * queueManager.updateStatus(itemId, 'in_progress', {
* phase: 'parsing' * phase: 'parsing'
* }); * });
* *
* queueManager.updateStatus(itemId, 'success', { * queueManager.updateStatus(itemId, 'success', {
* recipe: parsedRecipe, * recipe: parsedRecipe,
* tandoorRecipeId: 123 * tandoorRecipeId: 123
* }); * });
* ``` * ```
*/ */
updateStatus( updateStatus(itemId: string, status: QueueItemStatus, data?: any): void {
itemId: string, const item = this.items.get(itemId);
status: QueueItemStatus, if (!item) return;
data?: any
): void { const now = new Date().toISOString();
const item = this.items.get(itemId); item.status = status;
if (!item) return; item.updatedAt = now;
const now = new Date().toISOString(); // Update phase progress
item.status = status; if (status === 'in_progress' && data?.phase) {
item.updatedAt = now; item.currentPhase = data.phase;
// Update phase progress if (!item.startedAt) {
if (status === 'in_progress' && data?.phase) { item.startedAt = now;
item.currentPhase = data.phase; }
if (!item.startedAt) { // Update phases array
item.startedAt = now; const phaseIndex = item.phases.findIndex((p) => p.name === data.phase);
} if (phaseIndex >= 0) {
// Mark previous phases as completed
// Update phases array for (let i = 0; i < phaseIndex; i++) {
const phaseIndex = item.phases.findIndex(p => p.name === data.phase); if (item.phases[i].status === 'in_progress') {
if (phaseIndex >= 0) { item.phases[i].status = 'completed';
// Mark previous phases as completed item.phases[i].completedAt = now;
for (let i = 0; i < phaseIndex; i++) { }
if (item.phases[i].status === 'in_progress') { }
item.phases[i].status = 'completed'; // Mark current phase as in progress
item.phases[i].completedAt = now; item.phases[phaseIndex].status = 'in_progress';
} item.phases[phaseIndex].startedAt = now;
} }
// Mark current phase as in progress }
item.phases[phaseIndex].status = 'in_progress';
item.phases[phaseIndex].startedAt = now; if (status === 'success') {
} item.completedAt = now;
} // Mark all phases as completed
item.phases.forEach((phase) => {
if (status === 'success') { if (phase.status !== 'completed') {
item.completedAt = now; phase.status = 'completed';
// Mark all phases as completed phase.completedAt = now;
item.phases.forEach(phase => { }
if (phase.status !== 'completed') { });
phase.status = 'completed'; }
phase.completedAt = now;
} if (status === 'error' || status === 'unhealthy') {
}); item.completedAt = now;
} // Mark current phase as error
if (item.currentPhase) {
if (status === 'error' || status === 'unhealthy') { const phaseIndex = item.phases.findIndex((p) => p.name === item.currentPhase);
item.completedAt = now; if (phaseIndex >= 0) {
// Mark current phase as error item.phases[phaseIndex].status = 'error';
if (item.currentPhase) { item.phases[phaseIndex].error = data?.error?.message;
const phaseIndex = item.phases.findIndex(p => p.name === item.currentPhase); }
if (phaseIndex >= 0) { }
item.phases[phaseIndex].status = 'error'; }
item.phases[phaseIndex].error = data?.error?.message;
} // Wrap results in results object
} if (
} data?.extractedText ||
data?.thumbnail !== undefined ||
// Wrap results in results object data?.recipe ||
if (data?.extractedText || data?.thumbnail !== undefined || data?.recipe || data?.tandoorRecipeId) { data?.tandoorRecipeId
if (!item.results) { ) {
item.results = {}; if (!item.results) {
} item.results = {};
}
if (data.extractedText) {
item.results.extractedText = data.extractedText; if (data.extractedText) {
item.extractedText = data.extractedText; // Keep legacy item.results.extractedText = data.extractedText;
} item.extractedText = data.extractedText; // Keep legacy
if (data.thumbnail !== undefined) { }
item.results.thumbnail = data.thumbnail; if (data.thumbnail !== undefined) {
item.thumbnail = data.thumbnail; // Keep legacy item.results.thumbnail = data.thumbnail;
} item.thumbnail = data.thumbnail; // Keep legacy
if (data.recipe) { }
item.results.recipe = data.recipe; if (data.recipe) {
item.recipe = data.recipe; // Keep legacy item.results.recipe = data.recipe;
} item.recipe = data.recipe; // Keep legacy
if (data.tandoorRecipeId) { }
item.results.tandoorRecipeId = data.tandoorRecipeId; if (data.tandoorRecipeId) {
item.tandoorRecipeId = data.tandoorRecipeId; // Keep legacy item.results.tandoorRecipeId = data.tandoorRecipeId;
item.tandoorRecipeId = data.tandoorRecipeId; // Keep legacy
// Construct Tandoor URL
if (tandoorConfig.serverUrl) { // Construct Tandoor URL
item.results.tandoorUrl = `${tandoorConfig.serverUrl}/view/recipe/${data.tandoorRecipeId}`; if (tandoorConfig.serverUrl) {
} item.results.tandoorUrl = `${tandoorConfig.serverUrl}/view/recipe/${data.tandoorRecipeId}`;
} }
} }
}
if (data?.error) {
item.error = data.error; if (data?.error) {
} item.error = data.error;
}
// Notify subscribers with enhanced update
this.notifySubscribers({ // Notify subscribers with enhanced update
type: 'status_change', this.notifySubscribers({
itemId, type: 'status_change',
status, itemId,
timestamp: now, status,
url: item.url, timestamp: now,
phase: item.currentPhase, url: item.url,
progress: item.phases, phase: item.currentPhase,
results: item.results, progress: item.phases,
error: item.error, results: item.results,
...data error: item.error,
}); ...data
} });
}
/**
* Add progress event to item's history /**
* * Add progress event to item's history
* Also extracts message into logs array for easy display. *
* * Also extracts message into logs array for easy display.
* @param itemId - ID of item *
* @param event - Progress event to add * @param itemId - ID of item
* * @param event - Progress event to add
* @example *
* ```typescript * @example
* queueManager.addProgressEvent(itemId, { * ```typescript
* type: 'status', * queueManager.addProgressEvent(itemId, {
* message: 'Extracting from Instagram...', * type: 'status',
* timestamp: new Date().toISOString() * message: 'Extracting from Instagram...',
* }); * timestamp: new Date().toISOString()
* ``` * });
*/ * ```
addProgressEvent(itemId: string, event: any): void { */
const item = this.items.get(itemId); addProgressEvent(itemId: string, event: any): void {
if (!item) return; const item = this.items.get(itemId);
if (!item) return;
item.progressEvents.push(event);
item.logs.push(event.message); item.progressEvents.push(event);
item.logs.push(event.message);
this.notifySubscribers({
type: 'progress', this.notifySubscribers({
itemId, type: 'progress',
status: item.status, itemId,
timestamp: new Date().toISOString(), status: item.status,
data: { event } timestamp: new Date().toISOString(),
}); data: { event }
} });
}
/**
* Remove item from queue /**
* * Remove item from queue
* @param itemId - ID of item to remove *
* @returns true if item was removed, false if not found * @param itemId - ID of item to remove
* * @returns true if item was removed, false if not found
* @example *
* ```typescript * @example
* const removed = queueManager.remove(itemId); * ```typescript
* if (removed) { * const removed = queueManager.remove(itemId);
* console.log('Item removed successfully'); * if (removed) {
* } * console.log('Item removed successfully');
* ``` * }
*/ * ```
remove(itemId: string): boolean { */
const deleted = this.items.delete(itemId); remove(itemId: string): boolean {
if (deleted) { const deleted = this.items.delete(itemId);
this.notifySubscribers({ if (deleted) {
type: 'status_change', this.notifySubscribers({
itemId, type: 'status_change',
status: 'error', // Use error to signal removal itemId,
timestamp: new Date().toISOString(), status: 'error', // Use error to signal removal
data: { removed: true } timestamp: new Date().toISOString(),
}); data: { removed: true }
} });
return deleted; }
} return deleted;
}
/**
* Retry a failed or unhealthy item /**
* * Retry a failed or unhealthy item
* Resets item to pending status and clears error state. *
* Cannot retry items currently in progress. * Resets item to pending status and clears error state.
* * Cannot retry items currently in progress.
* @param itemId - ID of item to retry *
* @returns true if retry was initiated, false otherwise * @param itemId - ID of item to retry
* * @returns true if retry was initiated, false otherwise
* @example *
* ```typescript * @example
* const retried = queueManager.retry(itemId); * ```typescript
* if (retried) { * const retried = queueManager.retry(itemId);
* console.log('Item queued for retry'); * if (retried) {
* } else { * console.log('Item queued for retry');
* console.log('Cannot retry (item in progress or not found)'); * } else {
* } * console.log('Cannot retry (item in progress or not found)');
* ``` * }
*/ * ```
retry(itemId: string): boolean { */
const item = this.items.get(itemId); retry(itemId: string): boolean {
if (!item || item.status === 'in_progress') return false; const item = this.items.get(itemId);
if (!item || item.status === 'in_progress') return false;
item.retryCount++;
item.status = 'pending'; item.retryCount++;
item.currentPhase = undefined; item.status = 'pending';
item.error = undefined; item.currentPhase = undefined;
item.startedAt = undefined; item.error = undefined;
item.completedAt = undefined; item.startedAt = undefined;
item.completedAt = undefined;
// Reset phases to pending
item.phases = [ // Reset phases to pending
{ name: 'extraction', status: 'pending' }, item.phases = [
{ name: 'parsing', status: 'pending' }, { name: 'extraction', status: 'pending' },
{ name: 'uploading', status: 'pending' } { name: 'parsing', status: 'pending' },
]; { name: 'uploading', status: 'pending' }
];
this.notifySubscribers({
type: 'status_change', this.notifySubscribers({
itemId, type: 'status_change',
status: 'pending', itemId,
timestamp: new Date().toISOString(), status: 'pending',
progress: item.phases, timestamp: new Date().toISOString(),
data: { retryCount: item.retryCount } progress: item.phases,
}); data: { retryCount: item.retryCount }
});
return true;
} return true;
}
/**
* Get all queue items /**
* * Get all queue items
* @returns Array of all queue items *
* * @returns Array of all queue items
* @example *
* ```typescript * @example
* const items = queueManager.getAll(); * ```typescript
* console.log(`Queue has ${items.length} items`); * const items = queueManager.getAll();
* ``` * console.log(`Queue has ${items.length} items`);
*/ * ```
getAll(): QueueItem[] { */
return Array.from(this.items.values()); getAll(): QueueItem[] {
} return Array.from(this.items.values());
}
/**
* Get single item by ID /**
* * Get single item by ID
* @param itemId - ID of item to retrieve *
* @returns Queue item or undefined if not found * @param itemId - ID of item to retrieve
* * @returns Queue item or undefined if not found
* @example *
* ```typescript * @example
* const item = queueManager.get(itemId); * ```typescript
* if (item) { * const item = queueManager.get(itemId);
* console.log('Status:', item.status); * if (item) {
* } * console.log('Status:', item.status);
* ``` * }
*/ * ```
get(itemId: string): QueueItem | undefined { */
return this.items.get(itemId); get(itemId: string): QueueItem | undefined {
} return this.items.get(itemId);
}
/**
* Subscribe to queue updates /**
* * Subscribe to queue updates
* Callback will be called whenever any item is updated. *
* * Callback will be called whenever any item is updated.
* @param callback - Function to call on each update *
* @returns Unsubscribe function * @param callback - Function to call on each update
* * @returns Unsubscribe function
* @example *
* ```typescript * @example
* const unsubscribe = queueManager.subscribe((update) => { * ```typescript
* console.log('Update:', update.itemId, update.status); * const unsubscribe = queueManager.subscribe((update) => {
* }); * console.log('Update:', update.itemId, update.status);
* * });
* // Later... *
* unsubscribe(); * // Later...
* ``` * unsubscribe();
*/ * ```
subscribe(callback: QueueUpdateCallback): () => void { */
this.subscribers.add(callback); subscribe(callback: QueueUpdateCallback): () => void {
return () => this.subscribers.delete(callback); this.subscribers.add(callback);
} return () => this.subscribers.delete(callback);
}
/**
* Notify all subscribers of an update /**
* * Notify all subscribers of an update
* Handles errors in individual subscribers to prevent one *
* bad subscriber from affecting others. * Handles errors in individual subscribers to prevent one
* * bad subscriber from affecting others.
* @param update - Update to broadcast *
*/ * @param update - Update to broadcast
private notifySubscribers(update: QueueStatusUpdate): void { */
for (const callback of this.subscribers) { private notifySubscribers(update: QueueStatusUpdate): void {
try { for (const callback of this.subscribers) {
callback(update); try {
} catch (err) { callback(update);
logError('[QueueManager] Subscriber error', err); } catch (err) {
} logError('[QueueManager] Subscriber error', err);
} }
} }
}
} }
/** /**
* Singleton instance of QueueManager * Singleton instance of QueueManager
* *
* Use this instance throughout the application to ensure * Use this instance throughout the application to ensure
* all components interact with the same queue. * all components interact with the same queue.
*/ */

794
src/lib/server/queue/QueueProcessor.ts
View File

@@ -1,11 +1,11 @@
/** /**
* Queue Processor - Orchestrates async processing of queue items * Queue Processor - Orchestrates async processing of queue items
* *
* Manages concurrent processing of Instagram URLs through three phases: * Manages concurrent processing of Instagram URLs through three phases:
* 1. Extraction - Browser automation to extract text and thumbnail * 1. Extraction - Browser automation to extract text and thumbnail
* 2. Parsing - LLM-based recipe extraction * 2. Parsing - LLM-based recipe extraction
* 3. Uploading - Automatic upload to Tandoor (if configured) * 3. Uploading - Automatic upload to Tandoor (if configured)
* *
* Architecture: Domain Layer (Hexagonal Architecture) * Architecture: Domain Layer (Hexagonal Architecture)
* - Domain Logic: Orchestrates processing workflow * - Domain Logic: Orchestrates processing workflow
* - Uses Ports: extraction.ts, parser.ts, tandoor.ts (secondary adapters) * - Uses Ports: extraction.ts, parser.ts, tandoor.ts (secondary adapters)
@@ -23,422 +23,424 @@ import type { QueueItem } from './types';
/** /**
* Queue processor with configurable concurrency * Queue processor with configurable concurrency
* *
* Features: * Features:
* - Concurrent processing (default: 2 simultaneous items) * - Concurrent processing (default: 2 simultaneous items)
* - Three-phase pipeline: extraction → parsing → uploading * - Three-phase pipeline: extraction → parsing → uploading
* - Error classification (recoverable vs non-recoverable) * - Error classification (recoverable vs non-recoverable)
* - Progress tracking via QueueManager * - Progress tracking via QueueManager
* - Automatic start on instantiation * - Automatic start on instantiation
* *
* @example * @example
* ```typescript * ```typescript
* import { queueProcessor } from './QueueProcessor'; * import { queueProcessor } from './QueueProcessor';
* *
* // Processor auto-starts on import * // Processor auto-starts on import
* // Add items to queue and they'll be processed automatically * // Add items to queue and they'll be processed automatically
* *
* // Stop processing (e.g., for maintenance) * // Stop processing (e.g., for maintenance)
* queueProcessor.stop(); * queueProcessor.stop();
* *
* // Resume processing * // Resume processing
* queueProcessor.start(); * queueProcessor.start();
* ``` * ```
*/ */
export class QueueProcessor { export class QueueProcessor {
/** Whether processor is actively running */ /** Whether processor is actively running */
private processing = false; private processing = false;
/** Maximum number of items to process simultaneously */ /** Maximum number of items to process simultaneously */
private concurrency = queueConfig.concurrency; private concurrency = queueConfig.concurrency;
/** Number of workers currently processing items */ /** Number of workers currently processing items */
private activeWorkers = 0; private activeWorkers = 0;
/** Unsubscribe function for queue manager subscription */ /** Unsubscribe function for queue manager subscription */
private unsubscribeFromQueue?: () => void; private unsubscribeFromQueue?: () => void;
constructor() { constructor() {
// Subscribe to queue updates to process new items immediately // Subscribe to queue updates to process new items immediately
this.unsubscribeFromQueue = queueManager.subscribe((update) => { this.unsubscribeFromQueue = queueManager.subscribe((update) => {
// Trigger processing when new items are enqueued (status_change to 'pending') // Trigger processing when new items are enqueued (status_change to 'pending')
if (update.type === 'status_change' && update.status === 'pending') { if (update.type === 'status_change' && update.status === 'pending') {
console.log(`[QueueProcessor] New item enqueued: ${update.itemId}, triggering processing`); console.log(`[QueueProcessor] New item enqueued: ${update.itemId}, triggering processing`);
// Use immediate processing (no timeout) for newly enqueued items // Use immediate processing (no timeout) for newly enqueued items
setTimeout(() => this.processNextBatch(), 0); setTimeout(() => this.processNextBatch(), 0);
} }
}); });
} }
/** /**
* Start processing queue * Start processing queue
* *
* Begins dequeuing and processing items up to concurrency limit. * Begins dequeuing and processing items up to concurrency limit.
* Safe to call multiple times - will not start duplicates. * Safe to call multiple times - will not start duplicates.
*/ */
start(): void { start(): void {
if (this.processing) return; if (this.processing) return;
this.processing = true; this.processing = true;
console.log(`[QueueProcessor] Started with concurrency ${this.concurrency}`); console.log(`[QueueProcessor] Started with concurrency ${this.concurrency}`);
this.processNextBatch(); this.processNextBatch();
} }
/** /**
* Stop processing queue * Stop processing queue
* *
* Prevents new items from being dequeued. * Prevents new items from being dequeued.
* Items currently in progress will complete. * Items currently in progress will complete.
*/ */
stop(): void { stop(): void {
this.processing = false; this.processing = false;
console.log('[QueueProcessor] Stopped'); console.log('[QueueProcessor] Stopped');
// Cleanup subscription when stopping // Cleanup subscription when stopping
if (this.unsubscribeFromQueue) { if (this.unsubscribeFromQueue) {
this.unsubscribeFromQueue(); this.unsubscribeFromQueue();
this.unsubscribeFromQueue = undefined; this.unsubscribeFromQueue = undefined;
} }
} }
/** /**
* Process items up to concurrency limit * Process items up to concurrency limit
* *
* Dequeues pending items and starts processing them. * Dequeues pending items and starts processing them.
* Automatically called recursively to maintain worker pool. * Automatically called recursively to maintain worker pool.
*/ */
private async processNextBatch(): Promise<void> { private async processNextBatch(): Promise<void> {
if (!this.processing) return; if (!this.processing) return;
// Start new workers up to concurrency limit // Start new workers up to concurrency limit
while (this.activeWorkers < this.concurrency) { while (this.activeWorkers < this.concurrency) {
const item = queueManager.dequeue(); const item = queueManager.dequeue();
if (!item) break; if (!item) break;
this.activeWorkers++; this.activeWorkers++;
console.log(`[QueueProcessor] Starting item ${item.id} (${this.activeWorkers}/${this.concurrency} active)`); console.log(
`[QueueProcessor] Starting item ${item.id} (${this.activeWorkers}/${this.concurrency} active)`
this.processItem(item) );
.finally(() => {
this.activeWorkers--; this.processItem(item).finally(() => {
console.log(`[QueueProcessor] Finished item ${item.id} (${this.activeWorkers}/${this.concurrency} active)`); this.activeWorkers--;
// Try to process next item immediately console.log(
setTimeout(() => this.processNextBatch(), 0); `[QueueProcessor] Finished item ${item.id} (${this.activeWorkers}/${this.concurrency} active)`
}); );
} // Try to process next item immediately
setTimeout(() => this.processNextBatch(), 0);
// Check again after shorter delay if still processing and no active workers });
if (this.processing && this.activeWorkers === 0) { }
setTimeout(() => this.processNextBatch(), 100); // Reduced from 1000ms to 100ms
} // Check again after shorter delay if still processing and no active workers
} if (this.processing && this.activeWorkers === 0) {
setTimeout(() => this.processNextBatch(), 100); // Reduced from 1000ms to 100ms
/** }
* Process a single queue item through all phases }
*
* Executes three phases sequentially: /**
* 1. Extraction - Extract content from Instagram * Process a single queue item through all phases
* 2. Parsing - Parse recipe from extracted text *
* 3. Uploading - Upload to Tandoor (if configured) * Executes three phases sequentially:
* * 1. Extraction - Extract content from Instagram
* On success: marks item as 'success' * 2. Parsing - Parse recipe from extracted text
* On error: marks item as 'unhealthy' (recoverable) or 'error' (non-recoverable) * 3. Uploading - Upload to Tandoor (if configured)
* *
* @param item - Queue item to process * On success: marks item as 'success'
*/ * On error: marks item as 'unhealthy' (recoverable) or 'error' (non-recoverable)
private async processItem(item: QueueItem): Promise<void> { *
try { * @param item - Queue item to process
console.log(`[QueueProcessor] Processing ${item.url}`); */
private async processItem(item: QueueItem): Promise<void> {
// Phase 1: Extraction try {
await this.extractionPhase(item); console.log(`[QueueProcessor] Processing ${item.url}`);
// Phase 2: Parsing // Phase 1: Extraction
await this.parsingPhase(item); await this.extractionPhase(item);
// Phase 3: Tandoor Upload (if enabled) // Phase 2: Parsing
await this.uploadPhase(item); await this.parsingPhase(item);
// Success // Phase 3: Tandoor Upload (if enabled)
queueManager.updateStatus(item.id, 'success'); await this.uploadPhase(item);
console.log(`[QueueProcessor] ✓ Success: ${item.id}`);
// Success
// Send push notification queueManager.updateStatus(item.id, 'success');
await this.sendPushNotification(item, 'success'); console.log(`[QueueProcessor] ✓ Success: ${item.id}`);
} catch (error) { // Send push notification
const errorMsg = error instanceof Error ? error.message : 'Unknown error'; await this.sendPushNotification(item, 'success');
const recoverable = this.isRecoverableError(error); } catch (error) {
const errorMsg = error instanceof Error ? error.message : 'Unknown error';
logError(`[QueueProcessor] ${recoverable ? 'Unhealthy' : 'Error'}: ${item.id}`, error); const recoverable = this.isRecoverableError(error);
queueManager.updateStatus(item.id, recoverable ? 'unhealthy' : 'error', { logError(`[QueueProcessor] ${recoverable ? 'Unhealthy' : 'Error'}: ${item.id}`, error);
error: {
phase: item.currentPhase || 'extraction', queueManager.updateStatus(item.id, recoverable ? 'unhealthy' : 'error', {
message: errorMsg, error: {
recoverable, phase: item.currentPhase || 'extraction',
timestamp: new Date().toISOString() message: errorMsg,
} recoverable,
}); timestamp: new Date().toISOString()
}
// Send push notification });
await this.sendPushNotification(item, recoverable ? 'unhealthy' : 'error');
} // Send push notification
} await this.sendPushNotification(item, recoverable ? 'unhealthy' : 'error');
}
/** }
* Phase 1: Extract text and thumbnail from Instagram
* /**
* Uses browser automation to load Instagram post and extract: * Phase 1: Extract text and thumbnail from Instagram
* - Recipe text (from caption, comments, etc.) *
* - Thumbnail image (from meta tags or screenshot) * Uses browser automation to load Instagram post and extract:
* * - Recipe text (from caption, comments, etc.)
* Progress events are captured and added to queue item. * - Thumbnail image (from meta tags or screenshot)
* *
* @param item - Queue item being processed * Progress events are captured and added to queue item.
* @throws Error if extraction fails *
*/ * @param item - Queue item being processed
private async extractionPhase(item: QueueItem): Promise<void> { * @throws Error if extraction fails
queueManager.updateStatus(item.id, 'in_progress', { */
phase: 'extraction' private async extractionPhase(item: QueueItem): Promise<void> {
}); queueManager.updateStatus(item.id, 'in_progress', {
phase: 'extraction'
const progressCallback = (event: ProgressEvent) => { });
queueManager.addProgressEvent(item.id, event);
}; const progressCallback = (event: ProgressEvent) => {
queueManager.addProgressEvent(item.id, event);
console.log(`[QueueProcessor] Extracting: ${item.url}`); };
const extracted = await extractTextAndThumbnail(item.url, progressCallback);
console.log(`[QueueProcessor] Extracting: ${item.url}`);
queueManager.updateStatus(item.id, 'in_progress', { const extracted = await extractTextAndThumbnail(item.url, progressCallback);
phase: 'extraction',
extractedText: extracted.bodyText, queueManager.updateStatus(item.id, 'in_progress', {
thumbnail: extracted.thumbnail phase: 'extraction',
}); extractedText: extracted.bodyText,
thumbnail: extracted.thumbnail
console.log(`[QueueProcessor] ✓ Extraction complete: ${item.id}`); });
}
console.log(`[QueueProcessor] ✓ Extraction complete: ${item.id}`);
/** }
* Phase 2: Parse recipe from extracted text
* /**
* Uses LLM to extract structured recipe data: * Phase 2: Parse recipe from extracted text
* - Recipe name *
* - Ingredients with amounts and units * Uses LLM to extract structured recipe data:
* - Instructions/steps * - Recipe name
* - Servings, times, etc. * - Ingredients with amounts and units
* * - Instructions/steps
* Enriches recipe with metadata (URL, thumbnail). * - Servings, times, etc.
* *
* @param item - Queue item being processed * Enriches recipe with metadata (URL, thumbnail).
* @throws Error if parsing fails or no recipe found *
*/ * @param item - Queue item being processed
private async parsingPhase(item: QueueItem): Promise<void> { * @throws Error if parsing fails or no recipe found
if (!item.extractedText) { */
throw new Error('No extracted text available for parsing'); private async parsingPhase(item: QueueItem): Promise<void> {
} if (!item.extractedText) {
throw new Error('No extracted text available for parsing');
queueManager.updateStatus(item.id, 'in_progress', { }
phase: 'parsing'
}); queueManager.updateStatus(item.id, 'in_progress', {
phase: 'parsing'
queueManager.addProgressEvent(item.id, { });
type: 'status',
message: 'Parsing recipe with LLM...', queueManager.addProgressEvent(item.id, {
timestamp: new Date().toISOString() type: 'status',
}); message: 'Parsing recipe with LLM...',
timestamp: new Date().toISOString()
console.log(`[QueueProcessor] Parsing recipe: ${item.id}`); });
const recipe = await extractRecipe(item.extractedText);
console.log(`[QueueProcessor] Parsing recipe: ${item.id}`);
if (!recipe) { const recipe = await extractRecipe(item.extractedText);
throw new Error('Failed to parse recipe from extracted text');
} if (!recipe) {
throw new Error('Failed to parse recipe from extracted text');
// Enrich recipe with metadata }
if (recipe.description) {
recipe.description += `\n\nLink: ${item.url}`; // Enrich recipe with metadata
} else { if (recipe.description) {
recipe.description = `Link: ${item.url}`; recipe.description += `\n\nLink: ${item.url}`;
} } else {
recipe.description = `Link: ${item.url}`;
if (item.thumbnail) { }
recipe.image = item.thumbnail;
} if (item.thumbnail) {
recipe.image = item.thumbnail;
queueManager.updateStatus(item.id, 'in_progress', { }
phase: 'parsing',
recipe queueManager.updateStatus(item.id, 'in_progress', {
}); phase: 'parsing',
recipe
console.log(`[QueueProcessor] ✓ Parsing complete: ${item.id} - ${recipe.name}`); });
}
console.log(`[QueueProcessor] ✓ Parsing complete: ${item.id} - ${recipe.name}`);
/** }
* Phase 3: Upload to Tandoor (automatic)
* /**
* If Tandoor is configured (TANDOOR_TOKEN env var set): * Phase 3: Upload to Tandoor (automatic)
* - Uploads recipe with ingredients and steps *
* - Attempts to upload thumbnail/image * If Tandoor is configured (TANDOOR_TOKEN env var set):
* - Image upload failure is non-fatal (logged but doesn't fail item) * - Uploads recipe with ingredients and steps
* * - Attempts to upload thumbnail/image
* If Tandoor not configured: skips silently * - Image upload failure is non-fatal (logged but doesn't fail item)
* *
* @param item - Queue item being processed * If Tandoor not configured: skips silently
* @throws Error if Tandoor upload fails *
*/ * @param item - Queue item being processed
private async uploadPhase(item: QueueItem): Promise<void> { * @throws Error if Tandoor upload fails
// Check if Tandoor is enabled */
if (!queueConfig.tandoor.enabled) { private async uploadPhase(item: QueueItem): Promise<void> {
// Skip if Tandoor not configured // Check if Tandoor is enabled
queueManager.addProgressEvent(item.id, { if (!queueConfig.tandoor.enabled) {
type: 'status', // Skip if Tandoor not configured
message: 'Tandoor not configured, skipping upload', queueManager.addProgressEvent(item.id, {
timestamp: new Date().toISOString() type: 'status',
}); message: 'Tandoor not configured, skipping upload',
console.log(`[QueueProcessor] Tandoor not configured, skipping: ${item.id}`); timestamp: new Date().toISOString()
return; });
} console.log(`[QueueProcessor] Tandoor not configured, skipping: ${item.id}`);
return;
if (!item.recipe) { }
throw new Error('No recipe available for upload');
} if (!item.recipe) {
throw new Error('No recipe available for upload');
queueManager.updateStatus(item.id, 'in_progress', { }
phase: 'uploading'
}); queueManager.updateStatus(item.id, 'in_progress', {
phase: 'uploading'
queueManager.addProgressEvent(item.id, { });
type: 'status',
message: 'Uploading recipe to Tandoor...', queueManager.addProgressEvent(item.id, {
timestamp: new Date().toISOString() type: 'status',
}); message: 'Uploading recipe to Tandoor...',
timestamp: new Date().toISOString()
console.log(`[QueueProcessor] Uploading to Tandoor: ${item.id}`); });
// Upload recipe console.log(`[QueueProcessor] Uploading to Tandoor: ${item.id}`);
const result = await uploadRecipeWithIngredientsDTO(item.recipe);
// Upload recipe
if (!result.success) { const result = await uploadRecipeWithIngredientsDTO(item.recipe);
throw new Error(`Tandoor upload failed: ${result.error}`);
} if (!result.success) {
throw new Error(`Tandoor upload failed: ${result.error}`);
queueManager.updateStatus(item.id, 'in_progress', { }
phase: 'uploading',
tandoorRecipeId: result.recipeId queueManager.updateStatus(item.id, 'in_progress', {
}); phase: 'uploading',
tandoorRecipeId: result.recipeId
console.log(`[QueueProcessor] ✓ Recipe uploaded: ${item.id} → Tandoor #${result.recipeId}`); });
// Upload image if available console.log(`[QueueProcessor] ✓ Recipe uploaded: ${item.id} → Tandoor #${result.recipeId}`);
if (result.recipeId && result.imageUrl) {
queueManager.addProgressEvent(item.id, { // Upload image if available
type: 'status', if (result.recipeId && result.imageUrl) {
message: 'Uploading recipe image to Tandoor...', queueManager.addProgressEvent(item.id, {
timestamp: new Date().toISOString() type: 'status',
}); message: 'Uploading recipe image to Tandoor...',
timestamp: new Date().toISOString()
const imageResult = await uploadRecipeImage(result.recipeId, result.imageUrl); });
if (!imageResult.success) { const imageResult = await uploadRecipeImage(result.recipeId, result.imageUrl);
// Image upload failure is recoverable - log but don't fail
console.warn(`[QueueProcessor] Image upload failed for ${item.id}: ${imageResult.error}`); if (!imageResult.success) {
queueManager.addProgressEvent(item.id, { // Image upload failure is recoverable - log but don't fail
type: 'status', console.warn(`[QueueProcessor] Image upload failed for ${item.id}: ${imageResult.error}`);
message: `Image upload failed: ${imageResult.error}`, queueManager.addProgressEvent(item.id, {
timestamp: new Date().toISOString() type: 'status',
}); message: `Image upload failed: ${imageResult.error}`,
} else { timestamp: new Date().toISOString()
console.log(`[QueueProcessor] ✓ Image uploaded: ${item.id}`); });
} } else {
} console.log(`[QueueProcessor] ✓ Image uploaded: ${item.id}`);
}
queueManager.addProgressEvent(item.id, { }
type: 'status',
message: 'Tandoor upload completed', queueManager.addProgressEvent(item.id, {
timestamp: new Date().toISOString() type: 'status',
}); message: 'Tandoor upload completed',
} timestamp: new Date().toISOString()
});
/** }
* Determine if error is recoverable
* /**
* Recoverable errors (unhealthy): * Determine if error is recoverable
* - Network timeouts *
* - Connection failures * Recoverable errors (unhealthy):
* - Image upload failures * - Network timeouts
* - Thumbnail extraction failures * - Connection failures
* * - Image upload failures
* Non-recoverable errors (error): * - Thumbnail extraction failures
* - Invalid URL format *
* - Authentication failures * Non-recoverable errors (error):
* - Parsing failures (no recipe found) * - Invalid URL format
* * - Authentication failures
* @param error - Error to classify * - Parsing failures (no recipe found)
* @returns true if error is recoverable, false otherwise *
*/ * @param error - Error to classify
private isRecoverableError(error: unknown): boolean { * @returns true if error is recoverable, false otherwise
if (!(error instanceof Error)) return false; */
private isRecoverableError(error: unknown): boolean {
const message = error.message.toLowerCase(); if (!(error instanceof Error)) return false;
// Recoverable errors const message = error.message.toLowerCase();
const recoverablePatterns = [
'timeout', // Recoverable errors
'network', const recoverablePatterns = [
'econnrefused', 'timeout',
'enotfound', 'network',
'image upload failed', 'econnrefused',
'thumbnail', 'enotfound',
'etimeout', 'image upload failed',
'fetch failed' 'thumbnail',
]; 'etimeout',
'fetch failed'
return recoverablePatterns.some(pattern => message.includes(pattern)); ];
}
return recoverablePatterns.some((pattern) => message.includes(pattern));
/** }
* Send Web Push notification for queue item completion
* /**
* Sends appropriate notification based on processing status: * Send Web Push notification for queue item completion
* - success: Recipe extraction complete with details *
* - error/unhealthy: Extraction failed with retry option * Sends appropriate notification based on processing status:
* * - success: Recipe extraction complete with details
* @param item - Queue item that completed * - error/unhealthy: Extraction failed with retry option
* @param status - Completion status (success, unhealthy, error) *
*/ * @param item - Queue item that completed
private async sendPushNotification( * @param status - Completion status (success, unhealthy, error)
item: QueueItem, */
status: 'success' | 'unhealthy' | 'error' private async sendPushNotification(
): Promise<void> { item: QueueItem,
try { status: 'success' | 'unhealthy' | 'error'
switch (status) { ): Promise<void> {
case 'success': try {
await pushNotificationService.notifySuccess( switch (status) {
item.id, case 'success':
item.results?.recipe?.name, await pushNotificationService.notifySuccess(
item.results?.tandoorUrl item.id,
); item.results?.recipe?.name,
break; item.results?.tandoorUrl
);
case 'error': break;
case 'unhealthy':
const errorMessage = item.error?.message || 'Processing failed'; case 'error':
await pushNotificationService.notifyError(item.id, errorMessage); case 'unhealthy':
break; const errorMessage = item.error?.message || 'Processing failed';
await pushNotificationService.notifyError(item.id, errorMessage);
default: break;
console.warn(`[QueueProcessor] Unknown status for push notification: ${status}`);
} default:
} catch (error) { console.warn(`[QueueProcessor] Unknown status for push notification: ${status}`);
logError('[QueueProcessor] Failed to send push notification', error); }
// Don't let notification failures break processing } catch (error) {
} logError('[QueueProcessor] Failed to send push notification', error);
} // Don't let notification failures break processing
}
}
} }
/** /**
* Singleton instance of QueueProcessor * Singleton instance of QueueProcessor
* *
* Auto-starts on module import to begin processing queue. * Auto-starts on module import to begin processing queue.
*/ */
export const queueProcessor = new QueueProcessor(); export const queueProcessor = new QueueProcessor();

6
src/lib/server/queue/config.ts
View File

@@ -3,7 +3,7 @@ import { env } from '$env/dynamic/private';
/** /**
* Server-side configuration for the async queue system * Server-side configuration for the async queue system
* Uses SvelteKit's $env/dynamic/private for runtime environment access * Uses SvelteKit's $env/dynamic/private for runtime environment access
* *
* Environment Variables: * Environment Variables:
* - QUEUE_CONCURRENCY: Number of items to process concurrently (default: 2) * - QUEUE_CONCURRENCY: Number of items to process concurrently (default: 2)
* - QUEUE_MAX_RETRIES: Maximum retry attempts for failed items (default: 3) * - QUEUE_MAX_RETRIES: Maximum retry attempts for failed items (default: 3)
@@ -29,7 +29,9 @@ export const queueConfig = {
/** Web Push notification settings */ /** Web Push notification settings */
push: { push: {
vapidPublicKey: env.VAPID_PUBLIC_KEY || 'BNextdcB_fQ0BVvyGioM5L8Tf9vKQjs-WnF-rUbnU8MdWIZQYfggIHxBnW21I-lq_0HykLCdMpYj8d5joavWdxQ', vapidPublicKey:
env.VAPID_PUBLIC_KEY ||
'BNextdcB_fQ0BVvyGioM5L8Tf9vKQjs-WnF-rUbnU8MdWIZQYfggIHxBnW21I-lq_0HykLCdMpYj8d5joavWdxQ',
vapidPrivateKey: env.VAPID_PRIVATE_KEY || 'JwxI_KcsBcehYcTOufMcbVWJjCq1QbH5FJmSyQuG680', vapidPrivateKey: env.VAPID_PRIVATE_KEY || 'JwxI_KcsBcehYcTOufMcbVWJjCq1QbH5FJmSyQuG680',
vapidEmail: env.VAPID_EMAIL || 'mailto:admin@example.com' vapidEmail: env.VAPID_EMAIL || 'mailto:admin@example.com'
} }

268
src/lib/server/queue/types.ts
View File

@@ -1,6 +1,6 @@
/** /**
* Type definitions for the async in-memory processing queue * Type definitions for the async in-memory processing queue
* *
* This module defines the core data structures for queue items, * This module defines the core data structures for queue items,
* status updates, and callbacks used throughout the queue system. * status updates, and callbacks used throughout the queue system.
*/ */
@@ -15,12 +15,7 @@ import type { ProgressEvent } from '$lib/server/extraction';
* - unhealthy: Recoverable error occurred, can be retried * - unhealthy: Recoverable error occurred, can be retried
* - error: Non-recoverable error occurred * - error: Non-recoverable error occurred
*/ */
export type QueueItemStatus = export type QueueItemStatus = 'pending' | 'in_progress' | 'success' | 'unhealthy' | 'error';
| 'pending'
| 'in_progress'
| 'success'
| 'unhealthy'
| 'error';
/** /**
* Processing phases for queue items * Processing phases for queue items
@@ -28,26 +23,23 @@ export type QueueItemStatus =
* - parsing: Parsing recipe from extracted text * - parsing: Parsing recipe from extracted text
* - uploading: Uploading recipe to Tandoor * - uploading: Uploading recipe to Tandoor
*/ */
export type ProcessingPhase = export type ProcessingPhase = 'extraction' | 'parsing' | 'uploading';
| 'extraction'
| 'parsing'
| 'uploading';
/** /**
* Phase progress information * Phase progress information
* Tracks the status of each processing phase * Tracks the status of each processing phase
*/ */
export interface PhaseProgress { export interface PhaseProgress {
/** Name of the phase */ /** Name of the phase */
name: ProcessingPhase; name: ProcessingPhase;
/** Current status of this phase */ /** Current status of this phase */
status: 'pending' | 'in_progress' | 'completed' | 'error'; status: 'pending' | 'in_progress' | 'completed' | 'error';
/** When phase started processing (ISO 8601 string) */ /** When phase started processing (ISO 8601 string) */
startedAt?: string; startedAt?: string;
/** When phase completed (ISO 8601 string) */ /** When phase completed (ISO 8601 string) */
completedAt?: string; completedAt?: string;
/** Error message if phase failed */ /** Error message if phase failed */
error?: string; error?: string;
} }
/** /**
@@ -55,135 +47,135 @@ export interface PhaseProgress {
* Contains all outputs from the processing pipeline * Contains all outputs from the processing pipeline
*/ */
export interface ProcessingResults { export interface ProcessingResults {
/** Extracted text from Instagram */ /** Extracted text from Instagram */
extractedText?: string; extractedText?: string;
/** Thumbnail URL or data URL */ /** Thumbnail URL or data URL */
thumbnail?: string | null; thumbnail?: string | null;
/** Parsed recipe object */ /** Parsed recipe object */
recipe?: any; recipe?: any;
/** Tandoor recipe ID */ /** Tandoor recipe ID */
tandoorRecipeId?: number; tandoorRecipeId?: number;
/** Tandoor recipe URL (constructed from ID) */ /** Tandoor recipe URL (constructed from ID) */
tandoorUrl?: string; tandoorUrl?: string;
} }
/** /**
* Queue item representing a single Instagram URL processing job * Queue item representing a single Instagram URL processing job
*/ */
export interface QueueItem { export interface QueueItem {
/** Unique identifier (UUID) */ /** Unique identifier (UUID) */
id: string; id: string;
/** Instagram URL to process */ /** Instagram URL to process */
url: string; url: string;
/** Current status of the item */ /** Current status of the item */
status: QueueItemStatus; status: QueueItemStatus;
// Phase tracking // Phase tracking
/** Current processing phase (only set when status is in_progress) */ /** Current processing phase (only set when status is in_progress) */
currentPhase?: ProcessingPhase; currentPhase?: ProcessingPhase;
/** Array of all phases with their progress status */ /** Array of all phases with their progress status */
phases: PhaseProgress[]; phases: PhaseProgress[];
// Timestamps // Timestamps
/** When item was added to queue (ISO 8601 string) */ /** When item was added to queue (ISO 8601 string) */
enqueuedAt: string; enqueuedAt: string;
/** Alias for enqueuedAt (frontend uses this) */ /** Alias for enqueuedAt (frontend uses this) */
createdAt: string; createdAt: string;
/** When processing started (ISO 8601 string) */ /** When processing started (ISO 8601 string) */
startedAt?: string; startedAt?: string;
/** When processing completed (ISO 8601 string) */ /** When processing completed (ISO 8601 string) */
completedAt?: string; completedAt?: string;
/** Last update timestamp (ISO 8601 string) */ /** Last update timestamp (ISO 8601 string) */
updatedAt?: string; updatedAt?: string;
// Results - wrapped in results object // Results - wrapped in results object
/** Processing results container */ /** Processing results container */
results?: ProcessingResults; results?: ProcessingResults;
// Legacy direct properties (kept for transition period) // Legacy direct properties (kept for transition period)
/** @deprecated Use results.extractedText instead */ /** @deprecated Use results.extractedText instead */
extractedText?: string; extractedText?: string;
/** @deprecated Use results.thumbnail instead */ /** @deprecated Use results.thumbnail instead */
thumbnail?: string | null; thumbnail?: string | null;
/** @deprecated Use results.recipe instead */ /** @deprecated Use results.recipe instead */
recipe?: any; recipe?: any;
/** @deprecated Use results.tandoorRecipeId instead */ /** @deprecated Use results.tandoorRecipeId instead */
tandoorRecipeId?: number; tandoorRecipeId?: number;
// Progress tracking // Progress tracking
/** User-facing log messages */ /** User-facing log messages */
logs: string[]; logs: string[];
/** All SSE progress events received */ /** All SSE progress events received */
progressEvents: ProgressEvent[]; progressEvents: ProgressEvent[];
// Error handling // Error handling
/** Error details if processing failed */ /** Error details if processing failed */
error?: { error?: {
/** Phase where error occurred */ /** Phase where error occurred */
phase: ProcessingPhase; phase: ProcessingPhase;
/** Error message */ /** Error message */
message: string; message: string;
/** Whether error is recoverable (can retry) */ /** Whether error is recoverable (can retry) */
recoverable: boolean; recoverable: boolean;
/** When error occurred (ISO 8601 string) */ /** When error occurred (ISO 8601 string) */
timestamp: string; timestamp: string;
}; };
// Retry tracking // Retry tracking
/** Number of times this item has been retried */ /** Number of times this item has been retried */
retryCount: number; retryCount: number;
/** Maximum number of retries allowed */ /** Maximum number of retries allowed */
maxRetries: number; maxRetries: number;
} }
/** /**
* Update notification sent to queue subscribers * Update notification sent to queue subscribers
*/ */
export interface QueueStatusUpdate { export interface QueueStatusUpdate {
/** Type of update */ /** Type of update */
type: 'status_change' | 'progress' | 'phase_complete'; type: 'status_change' | 'progress' | 'phase_complete';
/** ID of the item that was updated */ /** ID of the item that was updated */
itemId: string; itemId: string;
/** New status of the item */ /** New status of the item */
status: QueueItemStatus; status: QueueItemStatus;
/** When update occurred (ISO 8601 string) */ /** When update occurred (ISO 8601 string) */
timestamp: string; timestamp: string;
/** URL of the item */ /** URL of the item */
url?: string; url?: string;
// Phase information // Phase information
/** Current phase (if status is in_progress) */ /** Current phase (if status is in_progress) */
phase?: ProcessingPhase; phase?: ProcessingPhase;
/** Full phase progress array */ /** Full phase progress array */
progress?: PhaseProgress[]; progress?: PhaseProgress[];
// Results // Results
/** Processing results object */ /** Processing results object */
results?: ProcessingResults; results?: ProcessingResults;
// Error // Error
/** Error information */ /** Error information */
error?: any; error?: any;
/** Additional data related to the update (legacy) */ /** Additional data related to the update (legacy) */
data?: any; data?: any;
} }
/** /**

396
src/lib/server/scheduler.ts
View File

@@ -1,194 +1,202 @@
import fs from 'fs'; import fs from 'fs';
import path from 'path'; import path from 'path';
import { getBrowser } from './browser'; import { getBrowser } from './browser';
import { env } from '$env/dynamic/private'; import { env } from '$env/dynamic/private';
import { logError } from './utils/logger'; import { logError } from './utils/logger';
export interface SchedulerConfig { export interface SchedulerConfig {
enabled: boolean; enabled: boolean;
intervalMinutes: number; intervalMinutes: number;
} }
interface SchedulerState { interface SchedulerState {
intervalId: NodeJS.Timeout | null; intervalId: NodeJS.Timeout | null;
lastRenewalTime: number | null; lastRenewalTime: number | null;
isRenewing: boolean; isRenewing: boolean;
} }
const state: SchedulerState = { const state: SchedulerState = {
intervalId: null, intervalId: null,
lastRenewalTime: null, lastRenewalTime: null,
isRenewing: false isRenewing: false
}; };
/** /**
* Get scheduler configuration from environment variables * Get scheduler configuration from environment variables
*/ */
function getConfig(): SchedulerConfig { function getConfig(): SchedulerConfig {
const enabled = env.AUTH_SCHEDULER_ENABLED === 'true'; const enabled = env.AUTH_SCHEDULER_ENABLED === 'true';
let intervalMinutes = parseInt(env.AUTH_SCHEDULER_INTERVAL_MINUTES || '720', 10); let intervalMinutes = parseInt(env.AUTH_SCHEDULER_INTERVAL_MINUTES || '720', 10);
if (isNaN(intervalMinutes) || intervalMinutes < 5) { if (isNaN(intervalMinutes) || intervalMinutes < 5) {
console.warn( console.warn(
`[Scheduler] Invalid or too short interval '${env.AUTH_SCHEDULER_INTERVAL_MINUTES}'. Defaulting to 720 minutes.` `[Scheduler] Invalid or too short interval '${env.AUTH_SCHEDULER_INTERVAL_MINUTES}'. Defaulting to 720 minutes.`
); );
intervalMinutes = 720; intervalMinutes = 720;
} }
return { return {
enabled, enabled,
intervalMinutes intervalMinutes
}; };
} }
/** /**
* Resolve authentication storage path * Resolve authentication storage path
*/ */
function resolveAuthPath(): string { function resolveAuthPath(): string {
const authPathDocker = '/app/secrets/auth.json'; const authPathDocker = '/app/secrets/auth.json';
const authPathLocal = './secrets/auth.json'; const authPathLocal = './secrets/auth.json';
if (fs.existsSync(authPathDocker)) { if (fs.existsSync(authPathDocker)) {
return authPathDocker; return authPathDocker;
} }
if (fs.existsSync(authPathLocal)) { if (fs.existsSync(authPathLocal)) {
return authPathLocal; return authPathLocal;
} }
// Default to local path if neither exists yet // Default to local path if neither exists yet
return authPathLocal; return authPathLocal;
} }
/** /**
* Renew Instagram authentication by loading existing auth and refreshing the session * Renew Instagram authentication by loading existing auth and refreshing the session
* Inspired by gen-auth.js - reuses existing stored credentials without manual input * Inspired by gen-auth.js - reuses existing stored credentials without manual input
*/ */
async function renewInstagramAuth(): Promise<boolean> { async function renewInstagramAuth(): Promise<boolean> {
if (state.isRenewing) { if (state.isRenewing) {
console.log('[Scheduler] Auth renewal already in progress, skipping'); console.log('[Scheduler] Auth renewal already in progress, skipping');
return false; return false;
} }
const authPath = resolveAuthPath(); const authPath = resolveAuthPath();
if (!fs.existsSync(authPath)) { if (!fs.existsSync(authPath)) {
console.warn('[Scheduler] No existing auth.json found. Run gen-auth.js first to set up initial authentication.'); console.warn(
return false; '[Scheduler] No existing auth.json found. Run gen-auth.js first to set up initial authentication.'
} );
return false;
state.isRenewing = true; }
let context = null; state.isRenewing = true;
let page = null;
let context = null;
try { let page = null;
console.log('[Scheduler] Starting Instagram authentication renewal...');
console.log(`[Scheduler] Loading existing auth from: ${authPath}`); try {
console.log('[Scheduler] Starting Instagram authentication renewal...');
const browser = await getBrowser(); console.log(`[Scheduler] Loading existing auth from: ${authPath}`);
// Load existing authentication state
context = await browser.newContext({ storageState: authPath }); const browser = await getBrowser();
page = await context.newPage(); // Load existing authentication state
context = await browser.newContext({ storageState: authPath });
// Navigate to Instagram homepage - the existing auth will be used automatically page = await context.newPage();
await page.goto('https://www.instagram.com/', { waitUntil: 'domcontentloaded' });
// Navigate to Instagram homepage - the existing auth will be used automatically
// Wait for the "Home" icon to appear (indicates successful login) await page.goto('https://www.instagram.com/', { waitUntil: 'domcontentloaded' });
try {
await page.waitForSelector('svg[aria-label="Home"]', { timeout: 30000 }); // Wait for the "Home" icon to appear (indicates successful login)
console.log('[Scheduler] Successfully authenticated with Instagram'); try {
} catch (e) { await page.waitForSelector('svg[aria-label="Home"]', { timeout: 30000 });
logError('[Scheduler] Home icon not found - session may be expired or invalid', e); console.log('[Scheduler] Successfully authenticated with Instagram');
return false; } catch (e) {
} logError('[Scheduler] Home icon not found - session may be expired or invalid', e);
return false;
// Save the refreshed authentication state }
const authDir = path.dirname(authPath);
// Save the refreshed authentication state
// Ensure directory exists const authDir = path.dirname(authPath);
if (!fs.existsSync(authDir)) {
fs.mkdirSync(authDir, { recursive: true }); // Ensure directory exists
} if (!fs.existsSync(authDir)) {
fs.mkdirSync(authDir, { recursive: true });
// Update auth.json with refreshed session }
await context.storageState({ path: authPath });
// Update auth.json with refreshed session
state.lastRenewalTime = Date.now(); await context.storageState({ path: authPath });
console.log(`[Scheduler] Instagram authentication renewed successfully at ${new Date().toISOString()}`);
console.log(`[Scheduler] Auth state updated at: ${authPath}`); state.lastRenewalTime = Date.now();
console.log(
return true; `[Scheduler] Instagram authentication renewed successfully at ${new Date().toISOString()}`
} catch (error) { );
logError('[Scheduler] Instagram authentication renewal failed', error); console.log(`[Scheduler] Auth state updated at: ${authPath}`);
return false;
} finally { return true;
if (page) { } catch (error) {
await page.close().catch(() => {}); logError('[Scheduler] Instagram authentication renewal failed', error);
} return false;
if (context) { } finally {
await context.close().catch(() => {}); if (page) {
} await page.close().catch(() => {});
state.isRenewing = false; }
} if (context) {
} await context.close().catch(() => {});
}
/** state.isRenewing = false;
* Start the authentication renewal scheduler }
*/ }
export async function startScheduler(): Promise<void> {
const config = getConfig(); /**
* Start the authentication renewal scheduler
if (!config.enabled) { */
console.log('[Scheduler] Authentication scheduler is disabled (set AUTH_SCHEDULER_ENABLED=true to enable)'); export async function startScheduler(): Promise<void> {
return; const config = getConfig();
}
if (!config.enabled) {
if (state.intervalId !== null) { console.log(
console.warn('[Scheduler] Scheduler is already running'); '[Scheduler] Authentication scheduler is disabled (set AUTH_SCHEDULER_ENABLED=true to enable)'
return; );
} return;
}
const intervalMs = config.intervalMinutes * 60 * 1000;
if (state.intervalId !== null) {
console.log(`[Scheduler] Starting authentication scheduler with ${config.intervalMinutes}min interval`); console.warn('[Scheduler] Scheduler is already running');
return;
// Schedule periodic renewals }
state.intervalId = setInterval(async () => {
await renewInstagramAuth(); const intervalMs = config.intervalMinutes * 60 * 1000;
}, intervalMs);
console.log(
// Ensure interval is not blocking (set it as unreferenceable so it doesn't keep the process alive) `[Scheduler] Starting authentication scheduler with ${config.intervalMinutes}min interval`
if (state.intervalId.unref) { );
state.intervalId.unref();
} // Schedule periodic renewals
state.intervalId = setInterval(async () => {
// Optional: Perform initial renewal on startup (uncomment to enable) await renewInstagramAuth();
// await renewInstagramAuth(); }, intervalMs);
}
// Ensure interval is not blocking (set it as unreferenceable so it doesn't keep the process alive)
/** if (state.intervalId.unref) {
* Stop the authentication renewal scheduler state.intervalId.unref();
*/ }
export async function stopScheduler(): Promise<void> {
if (state.intervalId === null) { // Optional: Perform initial renewal on startup (uncomment to enable)
console.log('[Scheduler] Scheduler is not running'); // await renewInstagramAuth();
return; }
}
/**
console.log('[Scheduler] Stopping authentication scheduler...'); * Stop the authentication renewal scheduler
clearInterval(state.intervalId); */
state.intervalId = null; export async function stopScheduler(): Promise<void> {
} if (state.intervalId === null) {
console.log('[Scheduler] Scheduler is not running');
/** return;
* Get scheduler status information }
*/
export function getSchedulerStatus() { console.log('[Scheduler] Stopping authentication scheduler...');
return { clearInterval(state.intervalId);
running: state.intervalId !== null, state.intervalId = null;
lastRenewalTime: state.lastRenewalTime ? new Date(state.lastRenewalTime).toISOString() : null, }
isRenewing: state.isRenewing,
config: getConfig() /**
}; * Get scheduler status information
} */
export function getSchedulerStatus() {
return {
running: state.intervalId !== null,
lastRenewalTime: state.lastRenewalTime ? new Date(state.lastRenewalTime).toISOString() : null,
isRenewing: state.isRenewing,
config: getConfig()
};
}

24
src/lib/server/tandoor-config.ts
View File

@@ -1,12 +1,12 @@
import { env } from '$env/dynamic/private'; import { env } from '$env/dynamic/private';
/** /**
* Server-side environment configuration for Tandoor integration * Server-side environment configuration for Tandoor integration
* These variables should be set in your .env file or as environment variables * These variables should be set in your .env file or as environment variables
*/ */
export const tandoorConfig = { export const tandoorConfig = {
enabled: env.TANDOOR_ENABLED === 'true', enabled: env.TANDOOR_ENABLED === 'true',
serverUrl: (env.TANDOOR_SERVER_URL || '').replace(/\/$/, ''), serverUrl: (env.TANDOOR_SERVER_URL || '').replace(/\/$/, ''),
space: env.TANDOOR_SPACE ? parseInt(env.TANDOOR_SPACE, 10) : 1, space: env.TANDOOR_SPACE ? parseInt(env.TANDOOR_SPACE, 10) : 1,
token: env.TANDOOR_TOKEN || null token: env.TANDOOR_TOKEN || null
}; };

1010
src/lib/server/tandoor.ts
View File

File diff suppressed because it is too large Load Diff

100
src/lib/server/utils/logger.ts
View File

@@ -1,10 +1,10 @@
/** /**
* Logging Utilities * Logging Utilities
* *
* Provides error serialization and structured logging utilities to prevent * Provides error serialization and structured logging utilities to prevent
* [object Object] logs in production. All functions handle circular references * [object Object] logs in production. All functions handle circular references
* and properly serialize Error objects with their properties. * and properly serialize Error objects with their properties.
* *
* Features: * Features:
* - Error serialization with stack traces * - Error serialization with stack traces
* - Circular reference detection and handling * - Circular reference detection and handling
@@ -15,10 +15,10 @@
/** /**
* Serializes an error object to a JSON string. * Serializes an error object to a JSON string.
* Handles both Error instances and plain objects. * Handles both Error instances and plain objects.
* *
* @param error - Error object or unknown value to serialize * @param error - Error object or unknown value to serialize
* @returns JSON string representation of the error * @returns JSON string representation of the error
* *
* @example * @example
* ```typescript * ```typescript
* const err = new Error('Something went wrong'); * const err = new Error('Something went wrong');
@@ -27,34 +27,34 @@
* ``` * ```
*/ */
export function serializeError(error: unknown): string { export function serializeError(error: unknown): string {
if (error instanceof Error) { if (error instanceof Error) {
const errorObject: Record<string, any> = { const errorObject: Record<string, any> = {
name: error.name, name: error.name,
message: error.message, message: error.message,
stack: error.stack stack: error.stack
}; };
// Add custom properties from the error object // Add custom properties from the error object
for (const key of Object.keys(error)) { for (const key of Object.keys(error)) {
if (!(key in errorObject)) { if (!(key in errorObject)) {
errorObject[key] = (error as any)[key]; errorObject[key] = (error as any)[key];
} }
} }
return JSON.stringify(errorObject, null, 2); return JSON.stringify(errorObject, null, 2);
} }
return JSON.stringify(error, null, 2); return JSON.stringify(error, null, 2);
} }
/** /**
* Serializes an object to a JSON string with circular reference handling. * Serializes an object to a JSON string with circular reference handling.
* Prevents "Converting circular structure to JSON" errors. * Prevents "Converting circular structure to JSON" errors.
* *
* @param obj - Object to serialize * @param obj - Object to serialize
* @param maxDepth - Maximum depth for nested objects (default: 10) * @param maxDepth - Maximum depth for nested objects (default: 10)
* @returns JSON string representation of the object * @returns JSON string representation of the object
* *
* @example * @example
* ```typescript * ```typescript
* const circular: any = { a: 1 }; * const circular: any = { a: 1 };
@@ -64,28 +64,28 @@ export function serializeError(error: unknown): string {
* ``` * ```
*/ */
export function serializeObject(obj: unknown, maxDepth: number = 10): string { export function serializeObject(obj: unknown, maxDepth: number = 10): string {
const seen = new WeakSet(); const seen = new WeakSet();
const replacer = (key: string, value: any): any => { const replacer = (key: string, value: any): any => {
if (typeof value === 'object' && value !== null) { if (typeof value === 'object' && value !== null) {
if (seen.has(value)) { if (seen.has(value)) {
return '[Circular]'; return '[Circular]';
} }
seen.add(value); seen.add(value);
} }
return value; return value;
}; };
return JSON.stringify(obj, replacer, 2); return JSON.stringify(obj, replacer, 2);
} }
/** /**
* Logs an error to console.error with proper serialization. * Logs an error to console.error with proper serialization.
* Convenience wrapper around serializeError(). * Convenience wrapper around serializeError().
* *
* @param prefix - Log prefix (e.g., '[ComponentName]') * @param prefix - Log prefix (e.g., '[ComponentName]')
* @param error - Error object or unknown value to log * @param error - Error object or unknown value to log
* *
* @example * @example
* ```typescript * ```typescript
* try { * try {
@@ -96,23 +96,23 @@ export function serializeObject(obj: unknown, maxDepth: number = 10): string {
* ``` * ```
*/ */
export function logError(prefix: string, error: unknown): void { export function logError(prefix: string, error: unknown): void {
if (error instanceof Error) { if (error instanceof Error) {
console.error(prefix, error.message); console.error(prefix, error.message);
if (error.stack) { if (error.stack) {
console.error('Stack:', error.stack); console.error('Stack:', error.stack);
} }
} else { } else {
console.error(prefix, serializeError(error)); console.error(prefix, serializeError(error));
} }
} }
/** /**
* Logs an object to console.log with proper serialization. * Logs an object to console.log with proper serialization.
* Handles circular references automatically. * Handles circular references automatically.
* *
* @param prefix - Log prefix (e.g., '[ComponentName]') * @param prefix - Log prefix (e.g., '[ComponentName]')
* @param obj - Object to log * @param obj - Object to log
* *
* @example * @example
* ```typescript * ```typescript
* const config = { url: 'https://example.com', timeout: 5000 }; * const config = { url: 'https://example.com', timeout: 5000 };
@@ -120,5 +120,5 @@ export function logError(prefix: string, error: unknown): void {
* ``` * ```
*/ */
export function logObject(prefix: string, obj: unknown): void { export function logObject(prefix: string, obj: unknown): void {
console.log(prefix, serializeObject(obj)); console.log(prefix, serializeObject(obj));
} }

10
src/lib/server/validation/instagram-url.ts
View File

@@ -1,6 +1,6 @@
/** /**
* Instagram URL Validation Utility * Instagram URL Validation Utility
* *
* Validates that a URL is from Instagram's domain and uses HTTPS. * Validates that a URL is from Instagram's domain and uses HTTPS.
* Accepts all Instagram URL formats (posts, reels, IGTV, etc.). * Accepts all Instagram URL formats (posts, reels, IGTV, etc.).
*/ */
@@ -12,23 +12,23 @@ export interface ValidationResult {
/** /**
* Validate Instagram URL * Validate Instagram URL
* *
* Accepts: * Accepts:
* - https://instagram.com/p/{post-id} * - https://instagram.com/p/{post-id}
* - https://www.instagram.com/p/{post-id} * - https://www.instagram.com/p/{post-id}
* - https://instagram.com/reel/{reel-id} * - https://instagram.com/reel/{reel-id}
* - https://instagram.com/tv/{tv-id} * - https://instagram.com/tv/{tv-id}
* - Any Instagram URL with query parameters * - Any Instagram URL with query parameters
* *
* Rejects: * Rejects:
* - Non-HTTPS URLs (http://) * - Non-HTTPS URLs (http://)
* - Non-Instagram domains * - Non-Instagram domains
* - Invalid URL format * - Invalid URL format
* - Subdomains other than www * - Subdomains other than www
* *
* @param url - The URL to validate * @param url - The URL to validate
* @returns Validation result with valid flag and optional error message * @returns Validation result with valid flag and optional error message
* *
* @example * @example
* ```typescript * ```typescript
* const result = validateInstagramUrl('https://instagram.com/reel/ABC123?utm_source=share'); * const result = validateInstagramUrl('https://instagram.com/reel/ABC123?utm_source=share');

10
src/routes/api/extract/+server.ts
View File

@@ -1,12 +1,12 @@
/** /**
* DEPRECATED: Legacy synchronous extraction endpoint * DEPRECATED: Legacy synchronous extraction endpoint
* *
* This endpoint is deprecated and will be removed in a future version. * This endpoint is deprecated and will be removed in a future version.
* Use the new async queue system instead: * Use the new async queue system instead:
* *
* POST /api/queue - Submit URL for async processing * POST /api/queue - Submit URL for async processing
* GET /api/queue/stream - Real-time progress updates via SSE * GET /api/queue/stream - Real-time progress updates via SSE
* *
* Migration Guide: /docs/MIGRATION.md * Migration Guide: /docs/MIGRATION.md
*/ */
@@ -31,7 +31,7 @@ export const POST: RequestHandler = async ({ request }) => {
removedIn: 'v2.0.0' removedIn: 'v2.0.0'
} }
}, },
{ {
status: 410, // 410 Gone - resource no longer available status: 410, // 410 Gone - resource no longer available
headers: { headers: {
'X-Deprecated': 'true', 'X-Deprecated': 'true',
@@ -40,4 +40,4 @@ export const POST: RequestHandler = async ({ request }) => {
} }
} }
); );
}; };

93
src/routes/api/health/+server.ts
View File

@@ -1,11 +1,11 @@
/** /**
* Health Check API Endpoint * Health Check API Endpoint
* *
* Provides status information about critical application services: * Provides status information about critical application services:
* - Queue processing status * - Queue processing status
* - Queue statistics (pending, in_progress, etc.) * - Queue statistics (pending, in_progress, etc.)
* - Server uptime information * - Server uptime information
* *
* Used for monitoring and debugging queue processor functionality. * Used for monitoring and debugging queue processor functionality.
*/ */
@@ -14,48 +14,51 @@ import { queueManager } from '$lib/server/queue/QueueManager';
import { queueProcessor } from '$lib/server/queue/QueueProcessor'; import { queueProcessor } from '$lib/server/queue/QueueProcessor';
export const GET = async () => { export const GET = async () => {
try { try {
// Get current queue items by status // Get current queue items by status
const allItems = queueManager.getAll(); const allItems = queueManager.getAll();
const statusCounts = { const statusCounts = {
pending: allItems.filter(item => item.status === 'pending').length, pending: allItems.filter((item) => item.status === 'pending').length,
in_progress: allItems.filter(item => item.status === 'in_progress').length, in_progress: allItems.filter((item) => item.status === 'in_progress').length,
success: allItems.filter(item => item.status === 'success').length, success: allItems.filter((item) => item.status === 'success').length,
error: allItems.filter(item => item.status === 'error').length, error: allItems.filter((item) => item.status === 'error').length,
unhealthy: allItems.filter(item => item.status === 'unhealthy').length unhealthy: allItems.filter((item) => item.status === 'unhealthy').length
}; };
const stats = {
total: allItems.length
};
const healthData = { const stats = {
timestamp: new Date().toISOString(), total: allItems.length
status: 'healthy', };
services: {
queueProcessor: {
status: 'running', // QueueProcessor auto-starts, so it's always running
description: 'Queue processing service is operational'
},
queueManager: {
status: 'healthy',
stats,
statusCounts
}
},
uptime: process.uptime(),
version: process.env.npm_package_version || 'unknown'
};
return json(healthData); const healthData = {
} catch (error) { timestamp: new Date().toISOString(),
console.error('[Health Check] Error retrieving health status:', error); status: 'healthy',
services: {
return json({ queueProcessor: {
timestamp: new Date().toISOString(), status: 'running', // QueueProcessor auto-starts, so it's always running
status: 'unhealthy', description: 'Queue processing service is operational'
error: error instanceof Error ? error.message : 'Unknown error', },
uptime: process.uptime() queueManager: {
}, { status: 500 }); status: 'healthy',
} stats,
}; statusCounts
}
},
uptime: process.uptime(),
version: process.env.npm_package_version || 'unknown'
};
return json(healthData);
} catch (error) {
console.error('[Health Check] Error retrieving health status:', error);
return json(
{
timestamp: new Date().toISOString(),
status: 'unhealthy',
error: error instanceof Error ? error.message : 'Unknown error',
uptime: process.uptime()
},
{ status: 500 }
);
}
};

28
src/routes/api/llm-health/+server.ts
View File

@@ -10,21 +10,27 @@ export async function GET() {
const isHealthy = await checkLLMHealth(); const isHealthy = await checkLLMHealth();
if (isHealthy) { if (isHealthy) {
return json({ return json({
status: 'healthy', status: 'healthy',
message: 'LLM service is accessible' message: 'LLM service is accessible'
}); });
} else { } else {
return json({ return json(
status: 'unhealthy', {
message: 'LLM service is not accessible' status: 'unhealthy',
}, { status: 503 }); message: 'LLM service is not accessible'
},
{ status: 503 }
);
} }
} catch (error) { } catch (error) {
const errorMessage = error instanceof Error ? error.message : 'Unknown error'; const errorMessage = error instanceof Error ? error.message : 'Unknown error';
return json({ return json(
status: 'error', {
message: errorMessage status: 'error',
}, { status: 500 }); message: errorMessage
},
{ status: 500 }
);
} }
} }

135
src/routes/api/notifications/subscribe/+server.ts
View File

@@ -1,6 +1,6 @@
/** /**
* Push Notification Subscription API * Push Notification Subscription API
* *
* Handles web push notification subscription/unsubscription * Handles web push notification subscription/unsubscription
* for queue processing updates. * for queue processing updates.
*/ */
@@ -11,9 +11,9 @@ import type { RequestHandler } from './$types.js';
/** /**
* Subscribe to push notifications * Subscribe to push notifications
* *
* POST /api/notifications/subscribe * POST /api/notifications/subscribe
* *
* Body: * Body:
* { * {
* "subscription": { * "subscription": {
@@ -27,87 +27,70 @@ import type { RequestHandler } from './$types.js';
* } * }
*/ */
export const POST: RequestHandler = async ({ request }) => { export const POST: RequestHandler = async ({ request }) => {
try { try {
const { subscription, clientId } = await request.json(); const { subscription, clientId } = await request.json();
// Validate required fields // Validate required fields
if (!subscription || !subscription.endpoint || !subscription.keys) { if (!subscription || !subscription.endpoint || !subscription.keys) {
return json( return json({ error: 'Invalid subscription object' }, { status: 400 });
{ error: 'Invalid subscription object' }, }
{ status: 400 }
); if (!clientId || typeof clientId !== 'string') {
} return json({ error: 'Client ID is required' }, { status: 400 });
}
if (!clientId || typeof clientId !== 'string') {
return json( // Subscribe client
{ error: 'Client ID is required' }, await pushNotificationService.subscribe(clientId, {
{ status: 400 } endpoint: subscription.endpoint,
); keys: {
} p256dh: subscription.keys.p256dh,
auth: subscription.keys.auth
// Subscribe client }
await pushNotificationService.subscribe(clientId, { });
endpoint: subscription.endpoint,
keys: { console.log(`[NotificationAPI] Client ${clientId} subscribed to push notifications`);
p256dh: subscription.keys.p256dh,
auth: subscription.keys.auth return json({
} success: true,
}); message: 'Successfully subscribed to push notifications',
subscriptionCount: pushNotificationService.getSubscriptionCount()
console.log(`[NotificationAPI] Client ${clientId} subscribed to push notifications`); });
} catch (error) {
return json({ console.error('[NotificationAPI] Subscription error:', error);
success: true, return json({ error: 'Failed to subscribe to notifications' }, { status: 500 });
message: 'Successfully subscribed to push notifications', }
subscriptionCount: pushNotificationService.getSubscriptionCount()
});
} catch (error) {
console.error('[NotificationAPI] Subscription error:', error);
return json(
{ error: 'Failed to subscribe to notifications' },
{ status: 500 }
);
}
}; };
/** /**
* Unsubscribe from push notifications * Unsubscribe from push notifications
* *
* DELETE /api/notifications/subscribe * DELETE /api/notifications/subscribe
* *
* Body: * Body:
* { * {
* "clientId": "unique-client-id" * "clientId": "unique-client-id"
* } * }
*/ */
export const DELETE: RequestHandler = async ({ request }) => { export const DELETE: RequestHandler = async ({ request }) => {
try { try {
const { clientId } = await request.json(); const { clientId } = await request.json();
if (!clientId || typeof clientId !== 'string') { if (!clientId || typeof clientId !== 'string') {
return json( return json({ error: 'Client ID is required' }, { status: 400 });
{ error: 'Client ID is required' }, }
{ status: 400 }
); // Unsubscribe client
} await pushNotificationService.unsubscribe(clientId);
// Unsubscribe client console.log(`[NotificationAPI] Client ${clientId} unsubscribed from push notifications`);
await pushNotificationService.unsubscribe(clientId);
return json({
console.log(`[NotificationAPI] Client ${clientId} unsubscribed from push notifications`); success: true,
message: 'Successfully unsubscribed from push notifications',
return json({ subscriptionCount: pushNotificationService.getSubscriptionCount()
success: true, });
message: 'Successfully unsubscribed from push notifications', } catch (error) {
subscriptionCount: pushNotificationService.getSubscriptionCount() console.error('[NotificationAPI] Unsubscription error:', error);
}); return json({ error: 'Failed to unsubscribe from notifications' }, { status: 500 });
}
} catch (error) { };
console.error('[NotificationAPI] Unsubscription error:', error);
return json(
{ error: 'Failed to unsubscribe from notifications' },
{ status: 500 }
);
}
};

120
src/routes/api/notifications/test/+server.ts
View File

@@ -1,6 +1,6 @@
/** /**
* Test Push Notification API * Test Push Notification API
* *
* Allows manual testing of push notifications with different payloads. * Allows manual testing of push notifications with different payloads.
* Sends notification to all subscribed clients. * Sends notification to all subscribed clients.
*/ */
@@ -11,71 +11,69 @@ import type { RequestHandler } from './$types.js';
/** /**
* Send test push notification * Send test push notification
* *
* POST /api/notifications/test * POST /api/notifications/test
* *
* Body: * Body:
* { * {
* "type": "success" | "error" | "progress" * "type": "success" | "error" | "progress"
* } * }
*/ */
export const POST: RequestHandler = async ({ request }) => { export const POST: RequestHandler = async ({ request }) => {
try { try {
const { type } = await request.json(); const { type } = await request.json();
if (!type || !['success', 'error', 'progress'].includes(type)) { if (!type || !['success', 'error', 'progress'].includes(type)) {
return json( return json(
{ error: 'Invalid notification type. Must be: success, error, or progress' }, { error: 'Invalid notification type. Must be: success, error, or progress' },
{ status: 400 } { status: 400 }
); );
} }
const testItemId = 'test_' + Date.now(); const testItemId = 'test_' + Date.now();
// Create test payloads for each type // Create test payloads for each type
const payloads = { const payloads = {
success: { success: {
type: 'success' as const, type: 'success' as const,
itemId: testItemId, itemId: testItemId,
body: 'Test recipe extraction completed successfully!', body: 'Test recipe extraction completed successfully!',
recipeName: 'Test Recipe', recipeName: 'Test Recipe',
tag: `recipe-success-${testItemId}`, tag: `recipe-success-${testItemId}`,
requireInteraction: false requireInteraction: false
}, },
error: { error: {
type: 'error' as const, type: 'error' as const,
itemId: testItemId, itemId: testItemId,
body: 'Test recipe extraction failed - this is a test error', body: 'Test recipe extraction failed - this is a test error',
tag: `recipe-error-${testItemId}`, tag: `recipe-error-${testItemId}`,
requireInteraction: true requireInteraction: true
}, },
progress: { progress: {
type: 'progress' as const, type: 'progress' as const,
itemId: testItemId, itemId: testItemId,
body: 'Test recipe extraction in progress: parsing phase', body: 'Test recipe extraction in progress: parsing phase',
tag: `recipe-progress-${testItemId}`, tag: `recipe-progress-${testItemId}`,
requireInteraction: false requireInteraction: false
} }
}; };
const payload = payloads[type as keyof typeof payloads]; const payload = payloads[type as keyof typeof payloads];
await pushNotificationService.sendNotification(payload); await pushNotificationService.sendNotification(payload);
console.log(`[NotificationTestAPI] Sent test ${type} notification`); console.log(`[NotificationTestAPI] Sent test ${type} notification`);
return json({ return json({
success: true, success: true,
message: `Test ${type} notification sent`, message: `Test ${type} notification sent`,
subscriberCount: pushNotificationService.getSubscriptionCount() subscriberCount: pushNotificationService.getSubscriptionCount()
}); });
} catch (error) {
} catch (error) { console.error(
console.error('[NotificationTestAPI] Error sending test notification:', '[NotificationTestAPI] Error sending test notification:',
error instanceof Error ? error.message : String(error)); error instanceof Error ? error.message : String(error)
return json( );
{ error: 'Failed to send test notification' }, return json({ error: 'Failed to send test notification' }, { status: 500 });
{ status: 500 } }
);
}
}; };

45
src/routes/api/notifications/vapid-key/+server.ts
View File

@@ -1,6 +1,6 @@
/** /**
* VAPID Public Key API * VAPID Public Key API
* *
* Returns the public key for web push notifications. * Returns the public key for web push notifications.
* Required by browsers to create push subscriptions. * Required by browsers to create push subscriptions.
*/ */
@@ -11,9 +11,9 @@ import type { RequestHandler } from './$types.js';
/** /**
* Get VAPID public key * Get VAPID public key
* *
* GET /api/notifications/vapid-key * GET /api/notifications/vapid-key
* *
* Response: * Response:
* { * {
* "publicKey": "BDummyPublicKeyForDevelopment", * "publicKey": "BDummyPublicKeyForDevelopment",
@@ -21,26 +21,19 @@ import type { RequestHandler } from './$types.js';
* } * }
*/ */
export const GET: RequestHandler = async () => { export const GET: RequestHandler = async () => {
try { try {
const publicKey = pushNotificationService.getPublicVapidKey(); const publicKey = pushNotificationService.getPublicVapidKey();
if (!publicKey) { if (!publicKey) {
return json( return json({ error: 'VAPID public key not configured' }, { status: 503 });
{ error: 'VAPID public key not configured' }, }
{ status: 503 }
); return json({
} publicKey,
applicationServerKey: publicKey // Alias for compatibility
return json({ });
publicKey, } catch (error) {
applicationServerKey: publicKey // Alias for compatibility console.error('[NotificationAPI] VAPID key error:', error);
}); return json({ error: 'Failed to get VAPID public key' }, { status: 500 });
}
} catch (error) { };
console.error('[NotificationAPI] VAPID key error:', error);
return json(
{ error: 'Failed to get VAPID public key' },
{ status: 500 }
);
}
};

230
src/routes/api/queue/+server.ts
View File

@@ -1,6 +1,6 @@
/** /**
* Queue API Endpoints * Queue API Endpoints
* *
* Provides HTTP interface for queue operations: * Provides HTTP interface for queue operations:
* - POST /api/queue - Enqueue Instagram URL for processing * - POST /api/queue - Enqueue Instagram URL for processing
* - GET /api/queue - List all queue items with optional status filtering * - GET /api/queue - List all queue items with optional status filtering
@@ -15,135 +15,133 @@ import type { RequestHandler } from './$types';
/** /**
* POST /api/queue - Enqueue Instagram URL * POST /api/queue - Enqueue Instagram URL
* *
* Body: { url: string } * Body: { url: string }
* Returns: { id: string, url: string, status: string, enqueuedAt: string } * Returns: { id: string, url: string, status: string, enqueuedAt: string }
* *
* Validates Instagram URL format and enqueues for processing. * Validates Instagram URL format and enqueues for processing.
* Returns 400 for invalid URLs, 500 for server errors. * Returns 400 for invalid URLs, 500 for server errors.
*/ */
export const POST: RequestHandler = async ({ request }) => { export const POST: RequestHandler = async ({ request }) => {
try { try {
// Parse JSON body with proper error handling // Parse JSON body with proper error handling
let body; let body;
try { try {
body = await request.json(); body = await request.json();
} catch (jsonError) { } catch (jsonError) {
throw new ValidationError('Invalid JSON in request body'); throw new ValidationError('Invalid JSON in request body');
} }
// Validate request body // Validate request body
if (!body || typeof body !== 'object') { if (!body || typeof body !== 'object') {
throw new ValidationError('Request body must be JSON object'); throw new ValidationError('Request body must be JSON object');
} }
const { url } = body; const { url } = body;
// Validate URL presence // Validate URL presence
if (!url || typeof url !== 'string') { if (!url || typeof url !== 'string') {
throw new ValidationError('URL is required and must be a string'); throw new ValidationError('URL is required and must be a string');
} }
// Validate Instagram URL format using utility // Validate Instagram URL format using utility
const validation = validateInstagramUrl(url); const validation = validateInstagramUrl(url);
if (!validation.valid) { if (!validation.valid) {
throw new ValidationError(validation.error || 'Invalid Instagram URL'); throw new ValidationError(validation.error || 'Invalid Instagram URL');
} }
// Enqueue the URL // Enqueue the URL
const queueItem = queueManager.enqueue(url); const queueItem = queueManager.enqueue(url);
// Return minimal response (full details available at GET /api/queue/{id}) // Return minimal response (full details available at GET /api/queue/{id})
return json({ return json({
id: queueItem.id, id: queueItem.id,
url: queueItem.url, url: queueItem.url,
status: queueItem.status, status: queueItem.status,
enqueuedAt: queueItem.enqueuedAt enqueuedAt: queueItem.enqueuedAt
}); });
} catch (error) {
} catch (error) { return handleApiError(error);
return handleApiError(error); }
}
}; };
/** /**
* GET /api/queue - List queue items * GET /api/queue - List queue items
* *
* Query params: * Query params:
* - status?: string - Filter by status (pending, in_progress, success, unhealthy, error) * - status?: string - Filter by status (pending, in_progress, success, unhealthy, error)
* - limit?: number - Maximum items to return (default: 50, max: 200) * - limit?: number - Maximum items to return (default: 50, max: 200)
* - offset?: number - Pagination offset (default: 0) * - offset?: number - Pagination offset (default: 0)
* *
* Returns: { items: QueueItem[], total: number, hasMore: boolean } * Returns: { items: QueueItem[], total: number, hasMore: boolean }
*/ */
export const GET: RequestHandler = async ({ url }) => { export const GET: RequestHandler = async ({ url }) => {
try { try {
const searchParams = url.searchParams; const searchParams = url.searchParams;
// Parse query parameters // Parse query parameters
const statusFilter = searchParams.get('status'); const statusFilter = searchParams.get('status');
const limitParam = searchParams.get('limit'); const limitParam = searchParams.get('limit');
const offsetParam = searchParams.get('offset'); const offsetParam = searchParams.get('offset');
// Validate and parse limit // Validate and parse limit
let limit = 50; // default let limit = 50; // default
if (limitParam) { if (limitParam) {
const parsedLimit = parseInt(limitParam, 10); const parsedLimit = parseInt(limitParam, 10);
if (isNaN(parsedLimit) || parsedLimit < 1) { if (isNaN(parsedLimit) || parsedLimit < 1) {
throw new ValidationError('Limit must be a positive integer'); throw new ValidationError('Limit must be a positive integer');
} }
if (parsedLimit > 200) { if (parsedLimit > 200) {
throw new ValidationError('Limit cannot exceed 200'); throw new ValidationError('Limit cannot exceed 200');
} }
limit = parsedLimit; limit = parsedLimit;
} }
// Validate and parse offset // Validate and parse offset
let offset = 0; // default let offset = 0; // default
if (offsetParam) { if (offsetParam) {
const parsedOffset = parseInt(offsetParam, 10); const parsedOffset = parseInt(offsetParam, 10);
if (isNaN(parsedOffset) || parsedOffset < 0) { if (isNaN(parsedOffset) || parsedOffset < 0) {
throw new ValidationError('Offset must be a non-negative integer'); throw new ValidationError('Offset must be a non-negative integer');
} }
offset = parsedOffset; offset = parsedOffset;
} }
// Validate status filter // Validate status filter
const validStatuses = ['pending', 'in_progress', 'success', 'unhealthy', 'error']; const validStatuses = ['pending', 'in_progress', 'success', 'unhealthy', 'error'];
if (statusFilter && !validStatuses.includes(statusFilter)) { if (statusFilter && !validStatuses.includes(statusFilter)) {
throw new ValidationError( throw new ValidationError(
`Invalid status filter. Must be one of: ${validStatuses.join(', ')}` `Invalid status filter. Must be one of: ${validStatuses.join(', ')}`
); );
} }
// Get all items // Get all items
let items = queueManager.getAll(); let items = queueManager.getAll();
const totalCount = items.length; const totalCount = items.length;
// Apply status filter // Apply status filter
if (statusFilter) { if (statusFilter) {
items = items.filter(item => item.status === statusFilter); items = items.filter((item) => item.status === statusFilter);
} }
// Sort by enqueued time (newest first) // Sort by enqueued time (newest first)
items.sort((a, b) => new Date(b.enqueuedAt).getTime() - new Date(a.enqueuedAt).getTime()); items.sort((a, b) => new Date(b.enqueuedAt).getTime() - new Date(a.enqueuedAt).getTime());
// Apply pagination // Apply pagination
const paginatedItems = items.slice(offset, offset + limit); const paginatedItems = items.slice(offset, offset + limit);
const hasMore = (offset + limit) < items.length; const hasMore = offset + limit < items.length;
return json({ return json({
items: paginatedItems, items: paginatedItems,
total: statusFilter ? items.length : totalCount, total: statusFilter ? items.length : totalCount,
hasMore, hasMore,
pagination: { pagination: {
offset, offset,
limit, limit,
count: paginatedItems.length count: paginatedItems.length
} }
}); });
} catch (error) {
} catch (error) { return handleApiError(error);
return handleApiError(error); }
} };
};

134
src/routes/api/queue/[id]/+server.ts
View File

@@ -1,6 +1,6 @@
/** /**
* Individual Queue Item API Endpoints * Individual Queue Item API Endpoints
* *
* Provides HTTP interface for individual queue item operations: * Provides HTTP interface for individual queue item operations:
* - GET /api/queue/[id] - Get specific queue item details * - GET /api/queue/[id] - Get specific queue item details
* - DELETE /api/queue/[id] - Remove queue item * - DELETE /api/queue/[id] - Remove queue item
@@ -14,84 +14,80 @@ import type { RequestHandler } from './$types';
/** /**
* GET /api/queue/[id] - Get queue item by ID * GET /api/queue/[id] - Get queue item by ID
* *
* Returns full queue item details including progress events and results. * Returns full queue item details including progress events and results.
* Returns 404 if item not found, 400 for invalid ID format. * Returns 404 if item not found, 400 for invalid ID format.
*/ */
export const GET: RequestHandler = async ({ params }) => { export const GET: RequestHandler = async ({ params }) => {
try { try {
const { id } = params; const { id } = params;
// Validate ID parameter // Validate ID parameter
if (!id || typeof id !== 'string') { if (!id || typeof id !== 'string') {
throw new ValidationError('Queue item ID is required'); throw new ValidationError('Queue item ID is required');
} }
// Validate UUID format (basic check) // Validate UUID format (basic check)
const uuidPattern = /^[0-9a-f]{8}-[0-9a-f]{4}-4[0-9a-f]{3}-[89ab][0-9a-f]{3}-[0-9a-f]{12}$/i; const uuidPattern = /^[0-9a-f]{8}-[0-9a-f]{4}-4[0-9a-f]{3}-[89ab][0-9a-f]{3}-[0-9a-f]{12}$/i;
if (!uuidPattern.test(id)) { if (!uuidPattern.test(id)) {
throw new ValidationError('Invalid queue item ID format'); throw new ValidationError('Invalid queue item ID format');
} }
// Get queue item // Get queue item
const queueItem = queueManager.get(id); const queueItem = queueManager.get(id);
if (!queueItem) { if (!queueItem) {
throw new NotFoundError('Queue item not found'); throw new NotFoundError('Queue item not found');
} }
// Return full item details // Return full item details
return json(queueItem); return json(queueItem);
} catch (error) {
} catch (error) { return handleApiError(error);
return handleApiError(error); }
}
}; };
/** /**
* DELETE /api/queue/[id] - Remove queue item * DELETE /api/queue/[id] - Remove queue item
* *
* Removes an item from the queue. * Removes an item from the queue.
* Returns 404 if item not found, 400 for invalid ID format, * Returns 404 if item not found, 400 for invalid ID format,
* 409 if item is currently being processed. * 409 if item is currently being processed.
*/ */
export const DELETE: RequestHandler = async ({ params }) => { export const DELETE: RequestHandler = async ({ params }) => {
try { try {
const { id } = params; const { id } = params;
// Validate ID parameter // Validate ID parameter
if (!id || typeof id !== 'string') { if (!id || typeof id !== 'string') {
throw new ValidationError('Queue item ID is required'); throw new ValidationError('Queue item ID is required');
} }
// Validate UUID format // Validate UUID format
const uuidPattern = /^[0-9a-f]{8}-[0-9a-f]{4}-4[0-9a-f]{3}-[89ab][0-9a-f]{3}-[0-9a-f]{12}$/i; const uuidPattern = /^[0-9a-f]{8}-[0-9a-f]{4}-4[0-9a-f]{3}-[89ab][0-9a-f]{3}-[0-9a-f]{12}$/i;
if (!uuidPattern.test(id)) { if (!uuidPattern.test(id)) {
throw new ValidationError('Invalid queue item ID format'); throw new ValidationError('Invalid queue item ID format');
} }
// Check if item exists // Check if item exists
const existingItem = queueManager.get(id); const existingItem = queueManager.get(id);
if (!existingItem) { if (!existingItem) {
throw new NotFoundError('Queue item not found'); throw new NotFoundError('Queue item not found');
} }
// Prevent deletion of in-progress items // Prevent deletion of in-progress items
if (existingItem.status === 'in_progress') { if (existingItem.status === 'in_progress') {
throw new ConflictError( throw new ConflictError('Cannot delete item that is currently being processed');
'Cannot delete item that is currently being processed' }
);
} // Remove the item
const success = queueManager.remove(id);
// Remove the item
const success = queueManager.remove(id); return json({
success,
return json({ message: 'Queue item removed successfully'
success, });
message: 'Queue item removed successfully' } catch (error) {
}); return handleApiError(error);
}
} catch (error) { };
return handleApiError(error);
}
};

99
src/routes/api/queue/[id]/retry/+server.ts
View File

@@ -1,6 +1,6 @@
/** /**
* Queue Item Retry API Endpoint * Queue Item Retry API Endpoint
* *
* Provides HTTP interface for retrying failed queue items: * Provides HTTP interface for retrying failed queue items:
* - POST /api/queue/[id]/retry - Retry failed/unhealthy queue item * - POST /api/queue/[id]/retry - Retry failed/unhealthy queue item
*/ */
@@ -13,58 +13,57 @@ import type { RequestHandler } from './$types';
/** /**
* POST /api/queue/[id]/retry - Retry queue item * POST /api/queue/[id]/retry - Retry queue item
* *
* Resets a failed or unhealthy queue item to pending status for reprocessing. * Resets a failed or unhealthy queue item to pending status for reprocessing.
* Only items with status 'error' or 'unhealthy' can be retried. * Only items with status 'error' or 'unhealthy' can be retried.
* *
* Returns the updated queue item on success. * Returns the updated queue item on success.
* Returns 404 if item not found, 400 for invalid operations, 409 for wrong status. * Returns 404 if item not found, 400 for invalid operations, 409 for wrong status.
*/ */
export const POST: RequestHandler = async ({ params }) => { export const POST: RequestHandler = async ({ params }) => {
try { try {
const { id } = params; const { id } = params;
// Validate ID parameter // Validate ID parameter
if (!id || typeof id !== 'string') { if (!id || typeof id !== 'string') {
throw new ValidationError('Queue item ID is required'); throw new ValidationError('Queue item ID is required');
} }
// Validate UUID format (basic check) // Validate UUID format (basic check)
const uuidPattern = /^[0-9a-f]{8}-[0-9a-f]{4}-4[0-9a-f]{3}-[89ab][0-9a-f]{3}-[0-9a-f]{12}$/i; const uuidPattern = /^[0-9a-f]{8}-[0-9a-f]{4}-4[0-9a-f]{3}-[89ab][0-9a-f]{3}-[0-9a-f]{12}$/i;
if (!uuidPattern.test(id)) { if (!uuidPattern.test(id)) {
throw new ValidationError('Invalid queue item ID format'); throw new ValidationError('Invalid queue item ID format');
} }
// Check if item exists // Check if item exists
const existingItem = queueManager.get(id); const existingItem = queueManager.get(id);
if (!existingItem) { if (!existingItem) {
throw new NotFoundError('Queue item not found'); throw new NotFoundError('Queue item not found');
} }
// Check if item can be retried // Check if item can be retried
if (existingItem.status !== 'error' && existingItem.status !== 'unhealthy') { if (existingItem.status !== 'error' && existingItem.status !== 'unhealthy') {
throw new ConflictError( throw new ConflictError(
`Cannot retry item with status '${existingItem.status}'. Only 'error' and 'unhealthy' items can be retried.` `Cannot retry item with status '${existingItem.status}'. Only 'error' and 'unhealthy' items can be retried.`
); );
} }
// Retry the item // Retry the item
const retryResult = queueManager.retry(id); const retryResult = queueManager.retry(id);
if (!retryResult) { if (!retryResult) {
// This shouldn't happen given our checks above, but handle it gracefully // This shouldn't happen given our checks above, but handle it gracefully
throw new Error('Failed to retry queue item'); throw new Error('Failed to retry queue item');
} }
// Return the updated item // Return the updated item
const updatedItem = queueManager.get(id); const updatedItem = queueManager.get(id);
return json({ return json({
success: true, success: true,
item: updatedItem, item: updatedItem,
message: 'Queue item has been reset and will be reprocessed' message: 'Queue item has been reset and will be reprocessed'
}); });
} catch (error) {
} catch (error) { return handleApiError(error);
return handleApiError(error); }
} };
};

394
src/routes/api/queue/stream/+server.ts
View File

@@ -1,6 +1,6 @@
/** /**
* Queue SSE Stream API Endpoint * Queue SSE Stream API Endpoint
* *
* Provides Server-Sent Events stream for real-time queue updates: * Provides Server-Sent Events stream for real-time queue updates:
* - GET /api/queue/stream - Stream queue status updates * - GET /api/queue/stream - Stream queue status updates
*/ */
@@ -11,209 +11,209 @@ import type { QueueStatusUpdate } from '$lib/server/queue/types';
/** /**
* GET /api/queue/stream - Server-Sent Events stream for queue updates * GET /api/queue/stream - Server-Sent Events stream for queue updates
* *
* Returns a continuous stream of queue status updates in SSE format. * Returns a continuous stream of queue status updates in SSE format.
* Supports optional query parameters: * Supports optional query parameters:
* - ?id={queue-item-id} - Stream updates only for specific item * - ?id={queue-item-id} - Stream updates only for specific item
* - ?status={status} - Stream updates only for items with specific status * - ?status={status} - Stream updates only for items with specific status
* *
* SSE Event Format: * SSE Event Format:
* - event: queue-update * - event: queue-update
* - data: JSON string with QueueStatusUpdate object * - data: JSON string with QueueStatusUpdate object
* *
* Connection is kept alive until client disconnects. * Connection is kept alive until client disconnects.
*/ */
export const GET: RequestHandler = async ({ url, request }) => { export const GET: RequestHandler = async ({ url, request }) => {
const searchParams = url.searchParams; const searchParams = url.searchParams;
const itemIdFilter = searchParams.get('id'); const itemIdFilter = searchParams.get('id');
const statusFilter = searchParams.get('status'); const statusFilter = searchParams.get('status');
// Validate status filter if provided // Validate status filter if provided
const validStatuses = ['pending', 'in_progress', 'success', 'unhealthy', 'error']; const validStatuses = ['pending', 'in_progress', 'success', 'unhealthy', 'error'];
if (statusFilter && !validStatuses.includes(statusFilter)) { if (statusFilter && !validStatuses.includes(statusFilter)) {
return new Response(`Invalid status filter. Must be one of: ${validStatuses.join(', ')}`, { return new Response(`Invalid status filter. Must be one of: ${validStatuses.join(', ')}`, {
status: 400, status: 400,
headers: { 'Content-Type': 'text/plain' } headers: { 'Content-Type': 'text/plain' }
}); });
} }
// Validate item ID filter if provided // Validate item ID filter if provided
if (itemIdFilter) { if (itemIdFilter) {
const uuidPattern = /^[0-9a-f]{8}-[0-9a-f]{4}-4[0-9a-f]{3}-[89ab][0-9a-f]{3}-[0-9a-f]{12}$/i; const uuidPattern = /^[0-9a-f]{8}-[0-9a-f]{4}-4[0-9a-f]{3}-[89ab][0-9a-f]{3}-[0-9a-f]{12}$/i;
if (!uuidPattern.test(itemIdFilter)) { if (!uuidPattern.test(itemIdFilter)) {
return new Response('Invalid queue item ID format', { return new Response('Invalid queue item ID format', {
status: 400, status: 400,
headers: { 'Content-Type': 'text/plain' } headers: { 'Content-Type': 'text/plain' }
}); });
} }
} }
// Track stream state to prevent "Controller already closed" errors // Track stream state to prevent "Controller already closed" errors
let isClosed = false; let isClosed = false;
let unsubscribe: (() => void) | null = null; let unsubscribe: (() => void) | null = null;
let keepAliveInterval: NodeJS.Timeout | null = null; let keepAliveInterval: NodeJS.Timeout | null = null;
// Unified cleanup function - prevents double cleanup // Unified cleanup function - prevents double cleanup
const cleanup = () => { const cleanup = () => {
if (isClosed) return; // Already cleaned up if (isClosed) return; // Already cleaned up
isClosed = true; isClosed = true;
console.log('[SSE] Cleaning up stream connection'); console.log('[SSE] Cleaning up stream connection');
// Unsubscribe from queue updates // Unsubscribe from queue updates
if (unsubscribe) { if (unsubscribe) {
unsubscribe(); unsubscribe();
unsubscribe = null; unsubscribe = null;
} }
// Clear keep-alive interval // Clear keep-alive interval
if (keepAliveInterval) { if (keepAliveInterval) {
clearInterval(keepAliveInterval); clearInterval(keepAliveInterval);
keepAliveInterval = null; keepAliveInterval = null;
} }
}; };
// Safe enqueue helper - checks stream state before enqueueing // Safe enqueue helper - checks stream state before enqueueing
const safeEnqueue = (controller: ReadableStreamDefaultController, message: string): boolean => { const safeEnqueue = (controller: ReadableStreamDefaultController, message: string): boolean => {
if (isClosed) { if (isClosed) {
return false; // Stream already closed, don't attempt to enqueue return false; // Stream already closed, don't attempt to enqueue
} }
try { try {
controller.enqueue(new TextEncoder().encode(message)); controller.enqueue(new TextEncoder().encode(message));
return true; return true;
} catch (error) { } catch (error) {
// Controller closed or errored - clean up and mark as closed // Controller closed or errored - clean up and mark as closed
console.error('[SSE] Error enqueueing message:', error); console.error('[SSE] Error enqueueing message:', error);
cleanup(); cleanup();
return false; return false;
} }
}; };
// Create SSE response stream // Create SSE response stream
const stream = new ReadableStream({ const stream = new ReadableStream({
start(controller) { start(controller) {
console.log('[SSE] Stream started'); console.log('[SSE] Stream started');
// Send initial connection message // Send initial connection message
const connectionMsg = `event: connection\ndata: {"type":"connection","timestamp":"${new Date().toISOString()}","message":"Connected to queue stream"}\n\n`; const connectionMsg = `event: connection\ndata: {"type":"connection","timestamp":"${new Date().toISOString()}","message":"Connected to queue stream"}\n\n`;
if (!safeEnqueue(controller, connectionMsg)) { if (!safeEnqueue(controller, connectionMsg)) {
return; return;
} }
// Send current queue state as initial data // Send current queue state as initial data
try { try {
const currentItems = queueManager.getAll(); const currentItems = queueManager.getAll();
let filteredItems = currentItems; let filteredItems = currentItems;
// Apply filters // Apply filters
if (itemIdFilter) { if (itemIdFilter) {
filteredItems = currentItems.filter(item => item.id === itemIdFilter); filteredItems = currentItems.filter((item) => item.id === itemIdFilter);
} }
if (statusFilter) { if (statusFilter) {
filteredItems = filteredItems.filter(item => item.status === statusFilter); filteredItems = filteredItems.filter((item) => item.status === statusFilter);
} }
// Send initial state for each matching item // Send initial state for each matching item
for (const item of filteredItems) { for (const item of filteredItems) {
if (isClosed) break; // Stop if stream was closed if (isClosed) break; // Stop if stream was closed
const update: QueueStatusUpdate = { const update: QueueStatusUpdate = {
type: 'status_change', type: 'status_change',
itemId: item.id, itemId: item.id,
status: item.status, status: item.status,
timestamp: new Date().toISOString(), timestamp: new Date().toISOString(),
url: item.url, url: item.url,
progress: item.phases, progress: item.phases,
results: item.results, results: item.results,
error: item.error error: item.error
}; };
const sseMessage = `event: queue-update\ndata: ${JSON.stringify(update)}\n\n`; const sseMessage = `event: queue-update\ndata: ${JSON.stringify(update)}\n\n`;
if (!safeEnqueue(controller, sseMessage)) { if (!safeEnqueue(controller, sseMessage)) {
break; // Stop if enqueue failed break; // Stop if enqueue failed
} }
} }
} catch (error) { } catch (error) {
console.error('[SSE] Error sending initial queue state:', error); console.error('[SSE] Error sending initial queue state:', error);
} }
// Subscribe to queue updates // Subscribe to queue updates
unsubscribe = queueManager.subscribe((update) => { unsubscribe = queueManager.subscribe((update) => {
if (isClosed) return; // Don't process if already closed if (isClosed) return; // Don't process if already closed
// Apply filters // Apply filters
let shouldSend = true; let shouldSend = true;
if (itemIdFilter && update.itemId !== itemIdFilter) { if (itemIdFilter && update.itemId !== itemIdFilter) {
shouldSend = false; shouldSend = false;
} }
if (statusFilter && update.status !== statusFilter) { if (statusFilter && update.status !== statusFilter) {
shouldSend = false; shouldSend = false;
} }
if (shouldSend) { if (shouldSend) {
const sseMessage = `event: queue-update\ndata: ${JSON.stringify(update)}\n\n`; const sseMessage = `event: queue-update\ndata: ${JSON.stringify(update)}\n\n`;
safeEnqueue(controller, sseMessage); safeEnqueue(controller, sseMessage);
} }
}); });
// Keep-alive ping every 30 seconds // Keep-alive ping every 30 seconds
keepAliveInterval = setInterval(() => { keepAliveInterval = setInterval(() => {
if (isClosed) { if (isClosed) {
// Stop pinging if closed // Stop pinging if closed
if (keepAliveInterval) { if (keepAliveInterval) {
clearInterval(keepAliveInterval); clearInterval(keepAliveInterval);
keepAliveInterval = null; keepAliveInterval = null;
} }
return; return;
} }
const pingMsg = `event: ping\ndata: {"type":"ping","timestamp":"${new Date().toISOString()}"}\n\n`; const pingMsg = `event: ping\ndata: {"type":"ping","timestamp":"${new Date().toISOString()}"}\n\n`;
if (!safeEnqueue(controller, pingMsg)) { if (!safeEnqueue(controller, pingMsg)) {
// Failed to send ping, clear interval // Failed to send ping, clear interval
if (keepAliveInterval) { if (keepAliveInterval) {
clearInterval(keepAliveInterval); clearInterval(keepAliveInterval);
keepAliveInterval = null; keepAliveInterval = null;
} }
} }
}, 30000); }, 30000);
// Handle client disconnect // Handle client disconnect
request.signal.addEventListener('abort', () => { request.signal.addEventListener('abort', () => {
console.log('[SSE] Client disconnected (abort signal)'); console.log('[SSE] Client disconnected (abort signal)');
cleanup(); cleanup();
// Try to send disconnect message (may fail if already closed) // Try to send disconnect message (may fail if already closed)
const disconnectMsg = `event: disconnect\ndata: {"type":"disconnect","timestamp":"${new Date().toISOString()}","message":"Connection closed"}\n\n`; const disconnectMsg = `event: disconnect\ndata: {"type":"disconnect","timestamp":"${new Date().toISOString()}","message":"Connection closed"}\n\n`;
safeEnqueue(controller, disconnectMsg); safeEnqueue(controller, disconnectMsg);
// Close the controller // Close the controller
try { try {
controller.close(); controller.close();
} catch (error) { } catch (error) {
// Already closed, ignore // Already closed, ignore
} }
}); });
}, },
cancel() { cancel() {
// This is called when the stream is cancelled by the client // This is called when the stream is cancelled by the client
console.log('[SSE] Stream cancelled by client'); console.log('[SSE] Stream cancelled by client');
cleanup(); cleanup();
} }
}); });
return new Response(stream, { return new Response(stream, {
status: 200, status: 200,
headers: { headers: {
'Content-Type': 'text/event-stream', 'Content-Type': 'text/event-stream',
'Cache-Control': 'no-cache', 'Cache-Control': 'no-cache',
// Connection header omitted - Node.js handles connection management automatically // Connection header omitted - Node.js handles connection management automatically
'Access-Control-Allow-Origin': '*', 'Access-Control-Allow-Origin': '*',
'Access-Control-Allow-Headers': 'Cache-Control', 'Access-Control-Allow-Headers': 'Cache-Control',
'Access-Control-Expose-Headers': 'Content-Type' 'Access-Control-Expose-Headers': 'Content-Type'
} }
}); });
}; };

10
src/routes/api/tandoor-config/+server.ts
View File

@@ -1,5 +1,5 @@
import { json } from '@sveltejs/kit'; import { json } from '@sveltejs/kit';
import {tandoorConfig} from '$lib/server/tandoor-config'; import { tandoorConfig } from '$lib/server/tandoor-config';
export async function GET() { export async function GET() {
return json({...tandoorConfig, token: ''}); return json({ ...tandoorConfig, token: '' });
} }

86
src/routes/api/tandoor/+server.ts
View File

@@ -1,43 +1,43 @@
import { json } from '@sveltejs/kit'; import { json } from '@sveltejs/kit';
import type { RequestHandler } from './$types'; import type { RequestHandler } from './$types';
import { uploadRecipeWithIngredientsDTO, uploadRecipeImage } from '$lib/server/tandoor'; import { uploadRecipeWithIngredientsDTO, uploadRecipeImage } from '$lib/server/tandoor';
export const POST: RequestHandler = async ({ request }) => { export const POST: RequestHandler = async ({ request }) => {
const { recipe } = await request.json(); const { recipe } = await request.json();
if (!recipe) { if (!recipe) {
return json({ error: 'No recipe provided' }, { status: 400 }); return json({ error: 'No recipe provided' }, { status: 400 });
} }
try { try {
const result = await uploadRecipeWithIngredientsDTO(recipe); const result = await uploadRecipeWithIngredientsDTO(recipe);
if (!result.success) { if (!result.success) {
return json({ error: result.error || 'Failed to upload recipe' }, { status: 500 }); return json({ error: result.error || 'Failed to upload recipe' }, { status: 500 });
} }
// Upload image if available // Upload image if available
let imageStatus = null; let imageStatus = null;
if (result.recipeId && result.imageUrl) { if (result.recipeId && result.imageUrl) {
imageStatus = await uploadRecipeImage(result.recipeId, result.imageUrl); imageStatus = await uploadRecipeImage(result.recipeId, result.imageUrl);
if (!imageStatus.success) { if (!imageStatus.success) {
console.warn('Image upload failed, but recipe created:', imageStatus.error); console.warn('Image upload failed, but recipe created:', imageStatus.error);
} }
} }
return json({ return json({
success: true, success: true,
message: 'Recipe successfully imported to Tandoor', message: 'Recipe successfully imported to Tandoor',
recipeId: result.recipeId, recipeId: result.recipeId,
imageUpload: imageStatus?.success ? 'successful' : 'failed' imageUpload: imageStatus?.success ? 'successful' : 'failed'
}); });
} catch (error) { } catch (error) {
console.error('Tandoor upload error:', error); console.error('Tandoor upload error:', error);
return json( return json(
{ {
error: error instanceof Error ? error.message : 'Unknown error occurred' error: error instanceof Error ? error.message : 'Unknown error occurred'
}, },
{ status: 500 } { status: 500 }
); );
} }
} };

2
src/routes/page.svelte.spec.ts
View File

@@ -6,7 +6,7 @@ import Page from './+page.svelte';
describe('/+page.svelte', () => { describe('/+page.svelte', () => {
it('should render h1', async () => { it('should render h1', async () => {
render(Page); render(Page);
const heading = page.getByRole('heading', { level: 1 }); const heading = page.getByRole('heading', { level: 1 });
await expect.element(heading).toBeInTheDocument(); await expect.element(heading).toBeInTheDocument();
}); });

427
src/service-worker.ts
View File

@@ -7,287 +7,284 @@ import { build, files, version } from '$service-worker';
declare let self: ServiceWorkerGlobalScope; declare let self: ServiceWorkerGlobalScope;
// Create a unique cache name for this deployment // Create a unique cache name for this deployment
const CACHE = `cache-${version}`; const CACHE = `cache-${version}`;
const ASSETS = [ const ASSETS = [
...build, // the app itself ...build, // the app itself
...files // everything in `static` ...files // everything in `static`
]; ];
// Global error handlers (preserve existing) // Global error handlers (preserve existing)
self.addEventListener('error', (event) => { self.addEventListener('error', (event) => {
console.error('[SW] Global error:', event.error); console.error('[SW] Global error:', event.error);
console.error('[SW] Error details:', { console.error('[SW] Error details:', {
message: event.message, message: event.message,
filename: event.filename, filename: event.filename,
lineno: event.lineno, lineno: event.lineno,
colno: event.colno, colno: event.colno,
error: event.error error: event.error
}); });
}); });
self.addEventListener('unhandledrejection', (event) => { self.addEventListener('unhandledrejection', (event) => {
console.error('[SW] Unhandled promise rejection:', event.reason); console.error('[SW] Unhandled promise rejection:', event.reason);
event.preventDefault(); // Prevent default browser behavior event.preventDefault(); // Prevent default browser behavior
}); });
console.log('[SW] Service worker script loading...'); console.log('[SW] Service worker script loading...');
// Install event - cache all assets // Install event - cache all assets
self.addEventListener('install', (event) => { self.addEventListener('install', (event) => {
console.log('[SW] Installing service worker...'); console.log('[SW] Installing service worker...');
async function addFilesToCache() {
const cache = await caches.open(CACHE);
await cache.addAll(ASSETS);
console.log(`[SW] Cached ${ASSETS.length} assets`);
}
event.waitUntil(addFilesToCache()); async function addFilesToCache() {
const cache = await caches.open(CACHE);
await cache.addAll(ASSETS);
console.log(`[SW] Cached ${ASSETS.length} assets`);
}
event.waitUntil(addFilesToCache());
}); });
// Activate event - clean up old caches // Activate event - clean up old caches
self.addEventListener('activate', (event) => { self.addEventListener('activate', (event) => {
console.log('[SW] Activating service worker...'); console.log('[SW] Activating service worker...');
async function deleteOldCaches() {
for (const key of await caches.keys()) {
if (key !== CACHE) {
console.log('[SW] Deleting old cache:', key);
await caches.delete(key);
}
}
}
event.waitUntil(deleteOldCaches()); async function deleteOldCaches() {
for (const key of await caches.keys()) {
if (key !== CACHE) {
console.log('[SW] Deleting old cache:', key);
await caches.delete(key);
}
}
}
event.waitUntil(deleteOldCaches());
}); });
// Fetch event - serve from cache with network fallback // Fetch event - serve from cache with network fallback
self.addEventListener('fetch', (event) => { self.addEventListener('fetch', (event) => {
// ignore POST requests etc // ignore POST requests etc
if (event.request.method !== 'GET') return; if (event.request.method !== 'GET') return;
async function respond() { async function respond() {
const url = new URL(event.request.url); const url = new URL(event.request.url);
const cache = await caches.open(CACHE); const cache = await caches.open(CACHE);
// `build`/`files` can always be served from the cache // `build`/`files` can always be served from the cache
if (ASSETS.includes(url.pathname)) { if (ASSETS.includes(url.pathname)) {
const response = await cache.match(url.pathname); const response = await cache.match(url.pathname);
if (response) { if (response) {
return response; return response;
} }
} }
// for everything else, try the network first, but // for everything else, try the network first, but
// fall back to the cache if we're offline // fall back to the cache if we're offline
try { try {
const response = await fetch(event.request); const response = await fetch(event.request);
// if we're offline, fetch can return a value that is not a Response
// instead of throwing - and we can't pass this non-Response to respondWith
if (!(response instanceof Response)) {
throw new Error('invalid response from fetch');
}
if (response.status === 200) { // if we're offline, fetch can return a value that is not a Response
cache.put(event.request, response.clone()); // instead of throwing - and we can't pass this non-Response to respondWith
} if (!(response instanceof Response)) {
return response; throw new Error('invalid response from fetch');
} catch (err) { }
const response = await cache.match(event.request);
if (response) {
return response;
}
// if there's no cache, then just error out
// as there is nothing we can do to respond to this request
throw err;
}
}
event.respondWith(respond()); if (response.status === 200) {
cache.put(event.request, response.clone());
}
return response;
} catch (err) {
const response = await cache.match(event.request);
if (response) {
return response;
}
// if there's no cache, then just error out
// as there is nothing we can do to respond to this request
throw err;
}
}
event.respondWith(respond());
}); });
// Push notification handling // Push notification handling
self.addEventListener('push', (event) => { self.addEventListener('push', (event) => {
console.log('[SW] Push event received:', event); console.log('[SW] Push event received:', event);
if (!event.data) {
console.log('[SW] Push event but no data');
return;
}
let data; if (!event.data) {
try { console.log('[SW] Push event but no data');
data = event.data.json(); return;
} catch (e) { }
console.error('[SW] Failed to parse push data:', e);
return;
}
console.log('[SW] Push data:', data); let data;
try {
data = event.data.json();
} catch (e) {
console.error('[SW] Failed to parse push data:', e);
return;
}
const options: NotificationOptions = { console.log('[SW] Push data:', data);
body: data.body || 'Recipe processing update',
icon: '/favicon.png',
badge: '/favicon.png',
data: data,
requireInteraction: data.requireInteraction || false,
silent: false,
tag: data.tag || 'recipe-update',
timestamp: Date.now(),
actions: []
};
// Add actions based on notification type const options: NotificationOptions = {
if (data.type === 'success' && data.itemId) { body: data.body || 'Recipe processing update',
options.actions = [ icon: '/favicon.png',
{ badge: '/favicon.png',
action: 'view', data: data,
title: 'View Recipe', requireInteraction: data.requireInteraction || false,
icon: '/favicon.png' silent: false,
}, tag: data.tag || 'recipe-update',
{ timestamp: Date.now(),
action: 'dismiss', actions: []
title: 'Dismiss' };
}
];
} else if (data.type === 'error' && data.itemId) {
options.actions = [
{
action: 'retry',
title: 'Retry',
icon: '/favicon.png'
},
{
action: 'view',
title: 'View Details'
}
];
}
const title = data.title || getNotificationTitle(data.type, data); // Add actions based on notification type
if (data.type === 'success' && data.itemId) {
options.actions = [
{
action: 'view',
title: 'View Recipe',
icon: '/favicon.png'
},
{
action: 'dismiss',
title: 'Dismiss'
}
];
} else if (data.type === 'error' && data.itemId) {
options.actions = [
{
action: 'retry',
title: 'Retry',
icon: '/favicon.png'
},
{
action: 'view',
title: 'View Details'
}
];
}
event.waitUntil( const title = data.title || getNotificationTitle(data.type, data);
self.registration.showNotification(title, options)
); event.waitUntil(self.registration.showNotification(title, options));
}); });
// Handle notification clicks // Handle notification clicks
self.addEventListener('notificationclick', (event) => { self.addEventListener('notificationclick', (event) => {
console.log('[SW] Notification click received:', event); console.log('[SW] Notification click received:', event);
event.notification.close();
const data = event.notification.data; event.notification.close();
const action = event.action;
let url = '/'; const data = event.notification.data;
const action = event.action;
if (action === 'view' && data?.itemId) { let url = '/';
url = `/?highlight=${data.itemId}`;
} else if (action === 'retry' && data?.itemId) {
// Navigate to dashboard and trigger retry via postMessage
url = `/?highlight=${data.itemId}&action=retry`;
} else if (data?.itemId) {
url = `/?highlight=${data.itemId}`;
}
event.waitUntil( if (action === 'view' && data?.itemId) {
clients.matchAll({ type: 'window', includeUncontrolled: true }) url = `/?highlight=${data.itemId}`;
.then((clientsList) => { } else if (action === 'retry' && data?.itemId) {
// Check if there's already a window/tab open // Navigate to dashboard and trigger retry via postMessage
for (const client of clientsList) { url = `/?highlight=${data.itemId}&action=retry`;
if (client.url.includes(self.location.origin) && 'focus' in client) { } else if (data?.itemId) {
return client.focus().then(() => { url = `/?highlight=${data.itemId}`;
// Send message to the client about the action }
return client.postMessage({
type: 'notification-action', event.waitUntil(
action: action, clients.matchAll({ type: 'window', includeUncontrolled: true }).then((clientsList) => {
data: data // Check if there's already a window/tab open
}); for (const client of clientsList) {
}); if (client.url.includes(self.location.origin) && 'focus' in client) {
} return client.focus().then(() => {
} // Send message to the client about the action
return client.postMessage({
// If no window is open, open a new one type: 'notification-action',
if (clients.openWindow) { action: action,
return clients.openWindow(url); data: data
} });
}) });
); }
}
// If no window is open, open a new one
if (clients.openWindow) {
return clients.openWindow(url);
}
})
);
}); });
// Handle notification close // Handle notification close
self.addEventListener('notificationclose', (event) => { self.addEventListener('notificationclose', (event) => {
console.log('[SW] Notification closed:', event); console.log('[SW] Notification closed:', event);
// Track notification dismissals if needed // Track notification dismissals if needed
const data = event.notification.data; const data = event.notification.data;
if (data?.analytics) { if (data?.analytics) {
// Could send analytics event here // Could send analytics event here
console.log('[SW] Notification dismissed:', data); console.log('[SW] Notification dismissed:', data);
} }
}); });
// Background sync for retry operations // Background sync for retry operations
self.addEventListener('sync', (event) => { self.addEventListener('sync', (event) => {
console.log('[SW] Background sync:', event.tag); console.log('[SW] Background sync:', event.tag);
if (event.tag === 'retry-queue-item') { if (event.tag === 'retry-queue-item') {
event.waitUntil(handleRetrySync()); event.waitUntil(handleRetrySync());
} }
}); });
// Helper functions // Helper functions
function getNotificationTitle(type: string, data: any): string { function getNotificationTitle(type: string, data: any): string {
switch (type) { switch (type) {
case 'success': case 'success':
return data.recipeName return data.recipeName
? `✅ Recipe Ready: ${data.recipeName}` ? `✅ Recipe Ready: ${data.recipeName}`
: '✅ Recipe extraction complete'; : '✅ Recipe extraction complete';
case 'error': case 'error':
return '❌ Recipe extraction failed'; return '❌ Recipe extraction failed';
case 'progress': case 'progress':
return `🔄 Processing recipe...`; return `🔄 Processing recipe...`;
default: default:
return '📱 InstaRecipe Update'; return '📱 InstaRecipe Update';
} }
} }
async function handleRetrySync() { async function handleRetrySync() {
try { try {
// Get retry items from IndexedDB or localStorage if needed // Get retry items from IndexedDB or localStorage if needed
console.log('[SW] Handling retry sync'); console.log('[SW] Handling retry sync');
// This could implement background retry logic // This could implement background retry logic
// For now, we'll let the main app handle retries // For now, we'll let the main app handle retries
return Promise.resolve(); return Promise.resolve();
} catch (error) { } catch (error) {
console.error('[SW] Retry sync failed:', error); console.error('[SW] Retry sync failed:', error);
throw error; throw error;
} }
} }
// Message handling for communication with main app // Message handling for communication with main app
self.addEventListener('message', (event) => { self.addEventListener('message', (event) => {
console.log('[SW] Message received:', event.data); console.log('[SW] Message received:', event.data);
const { type, data } = event.data; const { type, data } = event.data;
switch (type) { switch (type) {
case 'SKIP_WAITING': case 'SKIP_WAITING':
self.skipWaiting(); self.skipWaiting();
break; break;
case 'GET_VERSION': case 'GET_VERSION':
event.ports[0].postMessage({ version: '1.0.0' }); event.ports[0].postMessage({ version: '1.0.0' });
break; break;
case 'QUEUE_RETRY': case 'QUEUE_RETRY':
// Queue a background sync for retry // Queue a background sync for retry
self.registration.sync.register('retry-queue-item'); self.registration.sync.register('retry-queue-item');
break; break;
default: default:
console.log('[SW] Unknown message type:', type); console.log('[SW] Unknown message type:', type);
} }
}); });

130
src/tests/error-handler-logging.spec.ts
View File

@@ -4,77 +4,77 @@ import * as logger from '$lib/server/utils/logger';
import { ValidationError, NotFoundError, ConflictError } from '$lib/server/api/errors'; import { ValidationError, NotFoundError, ConflictError } from '$lib/server/api/errors';
describe('errorHandler logging', () => { describe('errorHandler logging', () => {
let logErrorSpy: any; let logErrorSpy: any;
beforeEach(() => { beforeEach(() => {
logErrorSpy = vi.spyOn(logger, 'logError').mockImplementation(() => {}); logErrorSpy = vi.spyOn(logger, 'logError').mockImplementation(() => {});
}); });
test('should use logError for standard errors', () => { test('should use logError for standard errors', () => {
const error = new Error('Test error'); const error = new Error('Test error');
handleApiError(error);
expect(logErrorSpy).toHaveBeenCalledWith('[API Error]', error);
});
test('should use logError for ValidationError', () => { handleApiError(error);
const error = new ValidationError('Invalid input');
const response = handleApiError(error);
expect(logErrorSpy).toHaveBeenCalledWith('[API Error]', error);
expect(response.status).toBe(400);
});
test('should use logError for NotFoundError', () => { expect(logErrorSpy).toHaveBeenCalledWith('[API Error]', error);
const error = new NotFoundError('Resource not found'); });
const response = handleApiError(error);
expect(logErrorSpy).toHaveBeenCalledWith('[API Error]', error);
expect(response.status).toBe(404);
});
test('should use logError for ConflictError', () => { test('should use logError for ValidationError', () => {
const error = new ConflictError('Resource conflict'); const error = new ValidationError('Invalid input');
const response = handleApiError(error);
expect(logErrorSpy).toHaveBeenCalledWith('[API Error]', error);
expect(response.status).toBe(409);
});
test('should serialize complex error objects', () => { const response = handleApiError(error);
const complexError = {
code: 'ERR_VALIDATION',
message: 'Invalid input',
details: { field: 'email', reason: 'invalid format' }
};
handleApiError(complexError);
expect(logErrorSpy).toHaveBeenCalledWith('[API Error]', complexError);
});
test('should handle unknown error types', () => { expect(logErrorSpy).toHaveBeenCalledWith('[API Error]', error);
const unknownError = 'String error'; expect(response.status).toBe(400);
});
handleApiError(unknownError);
expect(logErrorSpy).toHaveBeenCalledWith('[API Error]', unknownError);
});
test('logs should not use console.error directly', () => { test('should use logError for NotFoundError', () => {
const consoleErrorSpy = vi.spyOn(console, 'error').mockImplementation(() => {}); const error = new NotFoundError('Resource not found');
const error = new Error('Test'); const response = handleApiError(error);
handleApiError(error);
expect(logErrorSpy).toHaveBeenCalledWith('[API Error]', error);
// logError internally calls console.error, but handleApiError shouldn't call it directly expect(response.status).toBe(404);
// We're checking that handleApiError uses logError, not console.error });
expect(logErrorSpy).toHaveBeenCalledWith('[API Error]', error);
test('should use logError for ConflictError', () => {
consoleErrorSpy.mockRestore(); const error = new ConflictError('Resource conflict');
});
const response = handleApiError(error);
expect(logErrorSpy).toHaveBeenCalledWith('[API Error]', error);
expect(response.status).toBe(409);
});
test('should serialize complex error objects', () => {
const complexError = {
code: 'ERR_VALIDATION',
message: 'Invalid input',
details: { field: 'email', reason: 'invalid format' }
};
handleApiError(complexError);
expect(logErrorSpy).toHaveBeenCalledWith('[API Error]', complexError);
});
test('should handle unknown error types', () => {
const unknownError = 'String error';
handleApiError(unknownError);
expect(logErrorSpy).toHaveBeenCalledWith('[API Error]', unknownError);
});
test('logs should not use console.error directly', () => {
const consoleErrorSpy = vi.spyOn(console, 'error').mockImplementation(() => {});
const error = new Error('Test');
handleApiError(error);
// logError internally calls console.error, but handleApiError shouldn't call it directly
// We're checking that handleApiError uses logError, not console.error
expect(logErrorSpy).toHaveBeenCalledWith('[API Error]', error);
consoleErrorSpy.mockRestore();
});
}); });

51
src/tests/extraction-logging.spec.ts
View File

@@ -5,15 +5,15 @@ import fs from 'fs';
describe('extraction.ts logging', () => { describe('extraction.ts logging', () => {
let logErrorSpy: any; let logErrorSpy: any;
beforeEach(() => { beforeEach(() => {
logErrorSpy = vi.spyOn(logger, 'logError').mockImplementation(() => {}); logErrorSpy = vi.spyOn(logger, 'logError').mockImplementation(() => {});
}); });
afterEach(() => { afterEach(() => {
vi.restoreAllMocks(); vi.restoreAllMocks();
}); });
test('should use logError for extraction failures', async () => { test('should use logError for extraction failures', async () => {
// Trigger extraction error with invalid URL // Trigger extraction error with invalid URL
try { try {
@@ -22,66 +22,61 @@ describe('extraction.ts logging', () => {
} catch (error) { } catch (error) {
// Expected - extraction of invalid URL should fail // Expected - extraction of invalid URL should fail
} }
// logError should have been called during retry/error handling // logError should have been called during retry/error handling
expect(logErrorSpy).toHaveBeenCalled(); expect(logErrorSpy).toHaveBeenCalled();
const calls = logErrorSpy.mock.calls; const calls = logErrorSpy.mock.calls;
expect(calls.length).toBeGreaterThan(0); expect(calls.length).toBeGreaterThan(0);
// Verify at least one call has the expected format // Verify at least one call has the expected format
const errorCall = calls.find((call: any[]) => const errorCall = calls.find(
call[0]?.match(/\[.*\]/) && call[1] !== undefined (call: any[]) => call[0]?.match(/\[.*\]/) && call[1] !== undefined
); );
expect(errorCall).toBeDefined(); expect(errorCall).toBeDefined();
expect(errorCall[0]).toMatch(/\[.*\]/); // Has prefix like [Retry], [Extractor], etc expect(errorCall[0]).toMatch(/\[.*\]/); // Has prefix like [Retry], [Extractor], etc
expect(errorCall[1]).toBeDefined(); // Has error object expect(errorCall[1]).toBeDefined(); // Has error object
}); });
test('logs should not contain [object Object]', async () => { test('logs should not contain [object Object]', async () => {
const consoleWarnSpy = vi.spyOn(console, 'warn').mockImplementation(() => {}); const consoleWarnSpy = vi.spyOn(console, 'warn').mockImplementation(() => {});
const consoleErrorSpy = vi.spyOn(console, 'error').mockImplementation(() => {}); const consoleErrorSpy = vi.spyOn(console, 'error').mockImplementation(() => {});
// Trigger extraction error // Trigger extraction error
try { try {
await extractTextAndThumbnail('https://invalid-url-that-will-fail.com/test'); await extractTextAndThumbnail('https://invalid-url-that-will-fail.com/test');
} catch (e) { } catch (e) {
// Expected // Expected
} }
// Check all console.warn and console.error calls // Check all console.warn and console.error calls
const allCalls = [ const allCalls = [...consoleWarnSpy.mock.calls, ...consoleErrorSpy.mock.calls];
...consoleWarnSpy.mock.calls,
...consoleErrorSpy.mock.calls
];
const errorCalls = allCalls const errorCalls = allCalls
.map(call => call.join(' ')) .map((call) => call.join(' '))
.filter(msg => msg.includes('[object Object]')); .filter((msg) => msg.includes('[object Object]'));
expect(errorCalls).toHaveLength(0); expect(errorCalls).toHaveLength(0);
}); });
test('logError should serialize error objects properly', async () => { test('logError should serialize error objects properly', async () => {
// Create a mock error with complex structure // Create a mock error with complex structure
const mockError = new Error('Test error'); const mockError = new Error('Test error');
(mockError as any).customProp = { nested: 'value' }; (mockError as any).customProp = { nested: 'value' };
// Call logError directly to verify it handles complex errors // Call logError directly to verify it handles complex errors
logger.logError('[Test] Test message', mockError); logger.logError('[Test] Test message', mockError);
expect(logErrorSpy).toHaveBeenCalledWith('[Test] Test message', mockError); expect(logErrorSpy).toHaveBeenCalledWith('[Test] Test message', mockError);
// Verify the actual logger implementation doesn't produce [object Object] // Verify the actual logger implementation doesn't produce [object Object]
const consoleErrorSpy = vi.spyOn(console, 'error'); const consoleErrorSpy = vi.spyOn(console, 'error');
vi.restoreAllMocks(); vi.restoreAllMocks();
// Call real logError // Call real logError
logger.logError('[Test] Real test', mockError); logger.logError('[Test] Real test', mockError);
const output = consoleErrorSpy.mock.calls const output = consoleErrorSpy.mock.calls.map((call) => call.join(' ')).join(' ');
.map(call => call.join(' '))
.join(' ');
// Should not contain [object Object] // Should not contain [object Object]
expect(output).not.toContain('[object Object]'); expect(output).not.toContain('[object Object]');
}); });

18
src/tests/extraction-url-validation.integration.spec.ts
View File

@@ -2,7 +2,7 @@ import { describe, it, expect } from 'vitest';
/** /**
* Integration tests for thumbnail URL validation in the complete extraction flow * Integration tests for thumbnail URL validation in the complete extraction flow
* *
* These tests verify that URL validation works correctly in realistic scenarios: * These tests verify that URL validation works correctly in realistic scenarios:
* - Complete extraction flow with failing URLs falls back to screenshot * - Complete extraction flow with failing URLs falls back to screenshot
* - Valid URLs are successfully fetched and used * - Valid URLs are successfully fetched and used
@@ -184,21 +184,21 @@ describe('Thumbnail URL Validation Integration', () => {
/** /**
* Example of how integration tests could be structured with real mocking: * Example of how integration tests could be structured with real mocking:
* *
* import { chromium } from 'playwright'; * import { chromium } from 'playwright';
* import { extractTextAndThumbnail } from '$lib/server/extraction'; * import { extractTextAndThumbnail } from '$lib/server/extraction';
* *
* it('should validate URL and fall back', async () => { * it('should validate URL and fall back', async () => {
* const browser = await chromium.launch(); * const browser = await chromium.launch();
* const context = await browser.newContext(); * const context = await browser.newContext();
* const page = await context.newPage(); * const page = await context.newPage();
* *
* // Mock the page content * // Mock the page content
* await page.setContent(` * await page.setContent(`
* <meta property="og:image" content="https://example.com/invalid.jpg"> * <meta property="og:image" content="https://example.com/invalid.jpg">
* <video poster="https://example.com/also-invalid.jpg"></video> * <video poster="https://example.com/also-invalid.jpg"></video>
* `); * `);
* *
* // Mock fetch to return 404 for these URLs * // Mock fetch to return 404 for these URLs
* await page.route('**\/*', route => { * await page.route('**\/*', route => {
* if (route.request().url().includes('invalid.jpg')) { * if (route.request().url().includes('invalid.jpg')) {
@@ -207,23 +207,23 @@ describe('Thumbnail URL Validation Integration', () => {
* route.continue(); * route.continue();
* } * }
* }); * });
* *
* const progressEvents = []; * const progressEvents = [];
* const result = await extractTextAndThumbnail( * const result = await extractTextAndThumbnail(
* 'https://instagram.com/p/test', * 'https://instagram.com/p/test',
* (event) => progressEvents.push(event) * (event) => progressEvents.push(event)
* ); * );
* *
* // Verify screenshot fallback was used * // Verify screenshot fallback was used
* expect(result.thumbnail).toMatch(/^data:image\/jpeg;base64,/); * expect(result.thumbnail).toMatch(/^data:image\/jpeg;base64,/);
* *
* // Verify progress events show URL validation failures * // Verify progress events show URL validation failures
* expect(progressEvents).toContainEqual( * expect(progressEvents).toContainEqual(
* expect.objectContaining({ * expect.objectContaining({
* message: expect.stringContaining('HTTP 404') * message: expect.stringContaining('HTTP 404')
* }) * })
* ); * );
* *
* await browser.close(); * await browser.close();
* }); * });
*/ */

18
src/tests/favicon-ico.spec.ts
View File

@@ -8,19 +8,19 @@ const __filename = fileURLToPath(import.meta.url);
const __dirname = path.dirname(__filename); const __dirname = path.dirname(__filename);
test('favicon.ico should exist', () => { test('favicon.ico should exist', () => {
const icoPath = path.join(__dirname, '..', '..', 'static', 'favicon.ico'); const icoPath = path.join(__dirname, '..', '..', 'static', 'favicon.ico');
expect(fs.existsSync(icoPath)).toBe(true); expect(fs.existsSync(icoPath)).toBe(true);
}); });
test('favicon.ico should be 32x32', async () => { test('favicon.ico should be 32x32', async () => {
const icoPath = path.join(__dirname, '..', '..', 'static', 'favicon.ico'); const icoPath = path.join(__dirname, '..', '..', 'static', 'favicon.ico');
const metadata = await sharp(icoPath).metadata(); const metadata = await sharp(icoPath).metadata();
expect(metadata.width).toBe(32); expect(metadata.width).toBe(32);
expect(metadata.height).toBe(32); expect(metadata.height).toBe(32);
}); });
test('favicon.ico should be valid PNG format', async () => { test('favicon.ico should be valid PNG format', async () => {
const icoPath = path.join(__dirname, '..', '..', 'static', 'favicon.ico'); const icoPath = path.join(__dirname, '..', '..', 'static', 'favicon.ico');
const metadata = await sharp(icoPath).metadata(); const metadata = await sharp(icoPath).metadata();
expect(metadata.format).toBe('png'); expect(metadata.format).toBe('png');
}); });

42
src/tests/favicon.spec.ts
View File

@@ -8,30 +8,30 @@ const __filename = fileURLToPath(import.meta.url);
const __dirname = path.dirname(__filename); const __dirname = path.dirname(__filename);
describe('PWA Icon Generation - favicon.png', () => { describe('PWA Icon Generation - favicon.png', () => {
const faviconPath = path.join(__dirname, '..', '..', 'static', 'favicon.png'); const faviconPath = path.join(__dirname, '..', '..', 'static', 'favicon.png');
test('favicon.png should exist', () => { test('favicon.png should exist', () => {
expect(fs.existsSync(faviconPath)).toBe(true); expect(fs.existsSync(faviconPath)).toBe(true);
}); });
test('favicon.png should have exact 192x192 dimensions', async () => { test('favicon.png should have exact 192x192 dimensions', async () => {
const metadata = await sharp(faviconPath).metadata(); const metadata = await sharp(faviconPath).metadata();
expect(metadata.width).toBe(192); expect(metadata.width).toBe(192);
expect(metadata.height).toBe(192); expect(metadata.height).toBe(192);
}); });
test('favicon.png should be PNG format', async () => { test('favicon.png should be PNG format', async () => {
const metadata = await sharp(faviconPath).metadata(); const metadata = await sharp(faviconPath).metadata();
expect(metadata.format).toBe('png'); expect(metadata.format).toBe('png');
}); });
test('favicon.png should be less than 100KB', () => { test('favicon.png should be less than 100KB', () => {
const stats = fs.statSync(faviconPath); const stats = fs.statSync(faviconPath);
expect(stats.size).toBeLessThan(100 * 1024); expect(stats.size).toBeLessThan(100 * 1024);
}); });
test('favicon.png should have RGBA channels', async () => { test('favicon.png should have RGBA channels', async () => {
const metadata = await sharp(faviconPath).metadata(); const metadata = await sharp(faviconPath).metadata();
expect(metadata.channels).toBe(4); // RGBA expect(metadata.channels).toBe(4); // RGBA
}); });
}); });

328
src/tests/fixtures.ts
View File

@@ -1,164 +1,164 @@
import fs from 'fs'; import fs from 'fs';
import path from 'path'; import path from 'path';
/** /**
* Test utilities for scheduler testing * Test utilities for scheduler testing
*/ */
export const testFixtures = { export const testFixtures = {
/** /**
* Create a mock auth.json file with valid Instagram session * Create a mock auth.json file with valid Instagram session
*/ */
createMockAuthFile: (filePath: string) => { createMockAuthFile: (filePath: string) => {
const dir = path.dirname(filePath); const dir = path.dirname(filePath);
if (!fs.existsSync(dir)) { if (!fs.existsSync(dir)) {
fs.mkdirSync(dir, { recursive: true }); fs.mkdirSync(dir, { recursive: true });
} }
const mockAuth = { const mockAuth = {
cookies: [ cookies: [
{ {
name: 'sessionid', name: 'sessionid',
value: 'mock-session-' + Date.now(), value: 'mock-session-' + Date.now(),
domain: '.instagram.com', domain: '.instagram.com',
path: '/', path: '/',
expires: Math.floor(Date.now() / 1000) + 3600 * 24 * 30, expires: Math.floor(Date.now() / 1000) + 3600 * 24 * 30,
httpOnly: true, httpOnly: true,
secure: true, secure: true,
sameSite: 'Strict' sameSite: 'Strict'
}, },
{ {
name: 'ig_did', name: 'ig_did',
value: 'mock-did-' + Date.now(), value: 'mock-did-' + Date.now(),
domain: '.instagram.com', domain: '.instagram.com',
path: '/', path: '/',
expires: Math.floor(Date.now() / 1000) + 3600 * 24 * 365, expires: Math.floor(Date.now() / 1000) + 3600 * 24 * 365,
httpOnly: false, httpOnly: false,
secure: true, secure: true,
sameSite: 'Strict' sameSite: 'Strict'
} }
], ],
origins: [ origins: [
{ {
origin: 'https://www.instagram.com', origin: 'https://www.instagram.com',
localStorage: [ localStorage: [
{ {
name: 'ig_nrcb', name: 'ig_nrcb',
value: '1' value: '1'
} }
] ]
} }
] ]
}; };
fs.writeFileSync(filePath, JSON.stringify(mockAuth, null, 2)); fs.writeFileSync(filePath, JSON.stringify(mockAuth, null, 2));
return mockAuth; return mockAuth;
}, },
/** /**
* Clean up mock auth files * Clean up mock auth files
*/ */
cleanupMockAuthFile: (filePath: string) => { cleanupMockAuthFile: (filePath: string) => {
if (fs.existsSync(filePath)) { if (fs.existsSync(filePath)) {
fs.unlinkSync(filePath); fs.unlinkSync(filePath);
} }
const dir = path.dirname(filePath); const dir = path.dirname(filePath);
if (fs.existsSync(dir) && fs.readdirSync(dir).length === 0) { if (fs.existsSync(dir) && fs.readdirSync(dir).length === 0) {
fs.rmdirSync(dir); fs.rmdirSync(dir);
} }
}, },
/** /**
* Mock environment for scheduler testing * Mock environment for scheduler testing
*/ */
setupEnv: (config: Record<string, string | undefined>) => { setupEnv: (config: Record<string, string | undefined>) => {
const original: Record<string, string | undefined> = {}; const original: Record<string, string | undefined> = {};
for (const [key, value] of Object.entries(config)) { for (const [key, value] of Object.entries(config)) {
original[key] = process.env[key]; original[key] = process.env[key];
if (value === undefined) { if (value === undefined) {
delete process.env[key]; delete process.env[key];
} else { } else {
process.env[key] = value; process.env[key] = value;
} }
} }
return () => { return () => {
// Restore original env // Restore original env
for (const [key, value] of Object.entries(original)) { for (const [key, value] of Object.entries(original)) {
if (value === undefined) { if (value === undefined) {
delete process.env[key]; delete process.env[key];
} else { } else {
process.env[key] = value; process.env[key] = value;
} }
} }
}; };
}, },
/** /**
* Validate auth.json file structure * Validate auth.json file structure
*/ */
validateAuthFile: (filePath: string): boolean => { validateAuthFile: (filePath: string): boolean => {
try { try {
const content = JSON.parse(fs.readFileSync(filePath, 'utf-8')); const content = JSON.parse(fs.readFileSync(filePath, 'utf-8'));
// Check required fields // Check required fields
if (!Array.isArray(content.cookies)) return false; if (!Array.isArray(content.cookies)) return false;
if (!Array.isArray(content.origins)) return false; if (!Array.isArray(content.origins)) return false;
// Check cookie structure // Check cookie structure
for (const cookie of content.cookies) { for (const cookie of content.cookies) {
if (!cookie.name || !cookie.value || !cookie.domain) { if (!cookie.name || !cookie.value || !cookie.domain) {
return false; return false;
} }
} }
return true; return true;
} catch { } catch {
return false; return false;
} }
}, },
/** /**
* Get mock browser context for testing * Get mock browser context for testing
*/ */
createMockBrowserContext: () => { createMockBrowserContext: () => {
return { return {
newPage: async () => ({ newPage: async () => ({
goto: async () => {}, goto: async () => {},
waitForSelector: async () => {}, waitForSelector: async () => {},
evaluate: async () => 'Home', evaluate: async () => 'Home',
close: async () => {}, close: async () => {},
screenshot: async () => Buffer.from('mock-image') screenshot: async () => Buffer.from('mock-image')
}), }),
storageState: async (options: { path: string }) => { storageState: async (options: { path: string }) => {
const mockAuth = { const mockAuth = {
cookies: [{ name: 'sessionid', value: 'refreshed', domain: '.instagram.com' }], cookies: [{ name: 'sessionid', value: 'refreshed', domain: '.instagram.com' }],
origins: [] origins: []
}; };
fs.writeFileSync(options.path, JSON.stringify(mockAuth, null, 2)); fs.writeFileSync(options.path, JSON.stringify(mockAuth, null, 2));
}, },
close: async () => {} close: async () => {}
}; };
} }
}; };
/** /**
* Helper to create a spy for interval/timeout functions * Helper to create a spy for interval/timeout functions
*/ */
export const createTimerSpy = () => { export const createTimerSpy = () => {
let timers: NodeJS.Timeout[] = []; let timers: NodeJS.Timeout[] = [];
return { return {
setInterval: (callback: () => void, ms: number) => { setInterval: (callback: () => void, ms: number) => {
const timer = setInterval(callback, ms); const timer = setInterval(callback, ms);
timers.push(timer); timers.push(timer);
return timer; return timer;
}, },
cleanup: () => { cleanup: () => {
timers.forEach((timer) => clearInterval(timer)); timers.forEach((timer) => clearInterval(timer));
timers = []; timers = [];
} }
}; };
}; };

68
src/tests/icon-512.test.ts
View File

@@ -4,45 +4,45 @@ import fs from 'fs';
import path from 'path'; import path from 'path';
describe('Icon 512x512 Generation', () => { describe('Icon 512x512 Generation', () => {
const iconPath = path.resolve('static/icon-512.png'); const iconPath = path.resolve('static/icon-512.png');
it('should exist', () => { it('should exist', () => {
expect(fs.existsSync(iconPath)).toBe(true); expect(fs.existsSync(iconPath)).toBe(true);
}); });
it('should have correct dimensions (512x512)', async () => { it('should have correct dimensions (512x512)', async () => {
const metadata = await sharp(iconPath).metadata(); const metadata = await sharp(iconPath).metadata();
expect(metadata.width).toBe(512); expect(metadata.width).toBe(512);
expect(metadata.height).toBe(512); expect(metadata.height).toBe(512);
}); });
it('should be PNG format', async () => { it('should be PNG format', async () => {
const metadata = await sharp(iconPath).metadata(); const metadata = await sharp(iconPath).metadata();
expect(metadata.format).toBe('png'); expect(metadata.format).toBe('png');
}); });
it('should have valid RGBA encoding', async () => { it('should have valid RGBA encoding', async () => {
const metadata = await sharp(iconPath).metadata(); const metadata = await sharp(iconPath).metadata();
expect(metadata.channels).toBeGreaterThanOrEqual(3); // At least RGB expect(metadata.channels).toBeGreaterThanOrEqual(3); // At least RGB
}); });
it('should be less than 200KB', () => { it('should be less than 200KB', () => {
const stats = fs.statSync(iconPath); const stats = fs.statSync(iconPath);
const sizeInKB = stats.size / 1024; const sizeInKB = stats.size / 1024;
// Note: With current icon-source.png (672KB RGB), achieving both <200KB AND RGBA // Note: With current icon-source.png (672KB RGB), achieving both <200KB AND RGBA
// is not possible with lossless PNG compression. Trade-off: prioritize file size for web performance // is not possible with lossless PNG compression. Trade-off: prioritize file size for web performance
expect(sizeInKB).toBeLessThan(300); // Relaxed from 200KB due to source image constraints expect(sizeInKB).toBeLessThan(300); // Relaxed from 200KB due to source image constraints
}); });
it('should have transparency support (alpha channel)', async () => { it('should have transparency support (alpha channel)', async () => {
const metadata = await sharp(iconPath).metadata(); const metadata = await sharp(iconPath).metadata();
// Note: Source image is RGB without alpha. When using palette optimization for file size, // Note: Source image is RGB without alpha. When using palette optimization for file size,
// Sharp removes unused alpha channel. This is acceptable as transparency is not needed for this icon. // Sharp removes unused alpha channel. This is acceptable as transparency is not needed for this icon.
expect(metadata.channels).toBeGreaterThanOrEqual(3); // Accept RGB or RGBA expect(metadata.channels).toBeGreaterThanOrEqual(3); // Accept RGB or RGBA
}); });
it('should not be corrupted', async () => { it('should not be corrupted', async () => {
// Try to read the image - will throw if corrupted // Try to read the image - will throw if corrupted
await expect(sharp(iconPath).metadata()).resolves.toBeDefined(); await expect(sharp(iconPath).metadata()).resolves.toBeDefined();
}); });
}); });

97
src/tests/instagram-caption-extraction.e2e.spec.ts
View File

@@ -1,18 +1,18 @@
/** /**
* E2E Test for Instagram Caption Extraction * E2E Test for Instagram Caption Extraction
* *
* JIRA: RECIPE-0006 * JIRA: RECIPE-0006
* *
* CURRENT STATUS: Instagram actively prevents web scraping. * CURRENT STATUS: Instagram actively prevents web scraping.
* - All extraction methods (JSON, DOM, Internal State) return only truncated text (≤130 chars) * - All extraction methods (JSON, DOM, Internal State) return only truncated text (≤130 chars)
* - Full captions are loaded dynamically via GraphQL after user interaction * - Full captions are loaded dynamically via GraphQL after user interaction
* - "More" button expansion requires complex interaction simulation * - "More" button expansion requires complex interaction simulation
* *
* This test validates that: * This test validates that:
* 1. Multiple extraction strategies are attempted * 1. Multiple extraction strategies are attempted
* 2. The test fails if ALL strategies produce truncated output * 2. The test fails if ALL strategies produce truncated output
* 3. Anti-scraping detection is working * 3. Anti-scraping detection is working
* *
* To get full captions, consider: * To get full captions, consider:
* - Official Instagram Graph API (requires authentication) * - Official Instagram Graph API (requires authentication)
* - Manual user flow simulation with authenticated browser * - Manual user flow simulation with authenticated browser
@@ -29,19 +29,20 @@ describe('Instagram Caption Extraction E2E', () => {
const browser = await getBrowser(); const browser = await getBrowser();
const context = await createBrowserContext('./secrets/auth.json'); const context = await createBrowserContext('./secrets/auth.json');
const page = await context.newPage(); const page = await context.newPage();
try { try {
const testUrl = 'https://www.instagram.com/reel/DP6oN7JCEo8/?utm_source=ig_web_button_share_sheet'; const testUrl =
'https://www.instagram.com/reel/DP6oN7JCEo8/?utm_source=ig_web_button_share_sheet';
console.log('[DEBUG] Navigating to:', testUrl); console.log('[DEBUG] Navigating to:', testUrl);
await page.goto(testUrl, { waitUntil: 'domcontentloaded' }); await page.goto(testUrl, { waitUntil: 'domcontentloaded' });
await page.waitForTimeout(3000); await page.waitForTimeout(3000);
// Search for links in different ways // Search for links in different ways
const shortcode = 'DP6oN7JCEo8'; const shortcode = 'DP6oN7JCEo8';
console.log(`\n[DEBUG] Searching for links with shortcode: ${shortcode}`); console.log(`\n[DEBUG] Searching for links with shortcode: ${shortcode}`);
// Method 1: Contains shortcode anywhere // Method 1: Contains shortcode anywhere
const links1 = await page.locator(`a[href*="${shortcode}"]`).all(); const links1 = await page.locator(`a[href*="${shortcode}"]`).all();
console.log(`Method 1 - a[href*="${shortcode}"]: Found ${links1.length} links`); console.log(`Method 1 - a[href*="${shortcode}"]: Found ${links1.length} links`);
@@ -49,11 +50,11 @@ describe('Instagram Caption Extraction E2E', () => {
const href = await links1[i].getAttribute('href'); const href = await links1[i].getAttribute('href');
console.log(` [${i}] ${href}`); console.log(` [${i}] ${href}`);
} }
// Method 2: Get ALL links and filter // Method 2: Get ALL links and filter
const allLinks = await page.locator('a').all(); const allLinks = await page.locator('a').all();
console.log(`\n[DEBUG] Total links on page: ${allLinks.length}`); console.log(`\n[DEBUG] Total links on page: ${allLinks.length}`);
let matchingLinks = 0; let matchingLinks = 0;
for (const link of allLinks) { for (const link of allLinks) {
const href = await link.getAttribute('href'); const href = await link.getAttribute('href');
@@ -64,14 +65,13 @@ describe('Instagram Caption Extraction E2E', () => {
} }
} }
console.log(`Found ${matchingLinks} links containing shortcode`); console.log(`Found ${matchingLinks} links containing shortcode`);
//Method 3: Check page HTML directly //Method 3: Check page HTML directly
const html = await page.content(); const html = await page.content();
const htmlMatches = (html.match(new RegExp(shortcode, 'g')) || []).length; const htmlMatches = (html.match(new RegExp(shortcode, 'g')) || []).length;
console.log(`\n[DEBUG] Shortcode appears ${htmlMatches} times in page HTML`); console.log(`\n[DEBUG] Shortcode appears ${htmlMatches} times in page HTML`);
expect(true).toBe(true); expect(true).toBe(true);
} finally { } finally {
await page.close(); await page.close();
await context.close(); await context.close();
@@ -82,29 +82,33 @@ describe('Instagram Caption Extraction E2E', () => {
const browser = await getBrowser(); const browser = await getBrowser();
const context = await createBrowserContext('./secrets/auth.json'); const context = await createBrowserContext('./secrets/auth.json');
const page = await context.newPage(); const page = await context.newPage();
try { try {
const testUrl = 'https://www.instagram.com/reel/DP6oN7JCEo8/?utm_source=ig_web_button_share_sheet'; const testUrl =
'https://www.instagram.com/reel/DP6oN7JCEo8/?utm_source=ig_web_button_share_sheet';
console.log('[DEBUG] Navigating to:', testUrl); console.log('[DEBUG] Navigating to:', testUrl);
await page.goto(testUrl, { waitUntil: 'domcontentloaded' }); await page.goto(testUrl, { waitUntil: 'domcontentloaded' });
await page.waitForTimeout(3000); // Let page settle await page.waitForTimeout(3000); // Let page settle
// Take BEFORE screenshot // Take BEFORE screenshot
await page.screenshot({ path: 'debug_before.png', fullPage: true }); await page.screenshot({ path: 'debug_before.png', fullPage: true });
console.log('[DEBUG] BEFORE screenshot saved'); console.log('[DEBUG] BEFORE screenshot saved');
// Try to find and click "more" button // Try to find and click "more" button
console.log('[DEBUG] Looking for "more" button...'); console.log('[DEBUG] Looking for "more" button...');
const moreElements = await page.locator('span, div, button').filter({ hasText: /more/i }).all(); const moreElements = await page
.locator('span, div, button')
.filter({ hasText: /more/i })
.all();
console.log(`[DEBUG] Found ${moreElements.length} elements with "more"`); console.log(`[DEBUG] Found ${moreElements.length} elements with "more"`);
for (let i = 0; i < Math.min(moreElements.length, 10); i++) { for (let i = 0; i < Math.min(moreElements.length, 10); i++) {
const el = moreElements[i]; const el = moreElements[i];
const text = await el.textContent(); const text = await el.textContent();
const visible = await el.isVisible().catch(() => false); const visible = await el.isVisible().catch(() => false);
console.log(` [${i}] "${text}" visible:${visible}`); console.log(` [${i}] "${text}" visible:${visible}`);
if (visible && text && text.toLowerCase().includes('more')) { if (visible && text && text.toLowerCase().includes('more')) {
console.log(` -> Attempting to click element ${i}`); console.log(` -> Attempting to click element ${i}`);
try { try {
@@ -117,16 +121,16 @@ describe('Instagram Caption Extraction E2E', () => {
} }
} }
} }
// Take AFTER screenshot // Take AFTER screenshot
await page.screenshot({ path: 'debug_after.png', fullPage: true }); await page.screenshot({ path: 'debug_after.png', fullPage: true });
console.log('[DEBUG] AFTER screenshot saved'); console.log('[DEBUG] AFTER screenshot saved');
// Analyze spans again // Analyze spans again
const spanData = await page.evaluate(() => { const spanData = await page.evaluate(() => {
const spans = Array.from(document.querySelectorAll('span')); const spans = Array.from(document.querySelectorAll('span'));
return spans return spans
.filter(s => (s.textContent || '').length > 30) .filter((s) => (s.textContent || '').length > 30)
.map((s, idx) => ({ .map((s, idx) => ({
index: idx, index: idx,
text: (s.textContent || '').substring(0, 200), text: (s.textContent || '').substring(0, 200),
@@ -137,15 +141,16 @@ describe('Instagram Caption Extraction E2E', () => {
})) }))
.sort((a, b) => b.length - a.length); // Sort by text length .sort((a, b) => b.length - a.length); // Sort by text length
}); });
console.log('[DEBUG] Top spans by LENGTH after click attempt:'); console.log('[DEBUG] Top spans by LENGTH after click attempt:');
spanData.slice(0, 5).forEach(span => { spanData.slice(0, 5).forEach((span) => {
console.log(` [${span.index}] BR:${span.brCount} Links:${span.linkCount} Len:${span.length}`); console.log(
` [${span.index}] BR:${span.brCount} Links:${span.linkCount} Len:${span.length}`
);
console.log(` Text: "${span.text}"`); console.log(` Text: "${span.text}"`);
}); });
expect(true).toBe(true); // Dummy assertion expect(true).toBe(true); // Dummy assertion
} finally { } finally {
await page.close(); await page.close();
await context.close(); await context.close();
@@ -155,27 +160,28 @@ describe('Instagram Caption Extraction E2E', () => {
it('should extract complete recipe without metadata prefix (or at least try all methods)', async () => { it('should extract complete recipe without metadata prefix (or at least try all methods)', async () => {
// Instagram's current anti-scraping measures make full extraction difficult // Instagram's current anti-scraping measures make full extraction difficult
// This test validates that we try all available methods // This test validates that we try all available methods
const testUrl = 'https://www.instagram.com/reel/DP6oN7JCEo8/?utm_source=ig_web_button_share_sheet'; const testUrl =
'https://www.instagram.com/reel/DP6oN7JCEo8/?utm_source=ig_web_button_share_sheet';
const result = await extractTextAndThumbnail(testUrl); const result = await extractTextAndThumbnail(testUrl);
// Verify extraction succeeded // Verify extraction succeeded
expect(result).toBeDefined(); expect(result).toBeDefined();
expect(result.bodyText).toBeDefined(); expect(result.bodyText).toBeDefined();
console.log('[Test] Extracted text length:', result.bodyText.length); console.log('[Test] Extracted text length:', result.bodyText.length);
console.log('[Test] Full text:', result.bodyText); console.log('[Test] Full text:', result.bodyText);
// Verify no HTML tags remain in the extracted text // Verify no HTML tags remain in the extracted text
expect(result.bodyText).not.toMatch(/<[^>]+>/); expect(result.bodyText).not.toMatch(/<[^>]+>/);
expect(result.bodyText).not.toMatch(/&nbsp;/); expect(result.bodyText).not.toMatch(/&nbsp;/);
expect(result.bodyText).not.toMatch(/&amp;/); expect(result.bodyText).not.toMatch(/&amp;/);
// Verify line breaks are preserved (should have multiple lines) // Verify line breaks are preserved (should have multiple lines)
const lines = result.bodyText.split('\n'); const lines = result.bodyText.split('\n');
expect(lines.length).toBeGreaterThan(5); // Recipe should have multiple lines expect(lines.length).toBeGreaterThan(5); // Recipe should have multiple lines
// If we got more than 130 chars, great! If not, that's OK too (Instagram blocks us) // If we got more than 130 chars, great! If not, that's OK too (Instagram blocks us)
if (result.bodyText.length > 130) { if (result.bodyText.length > 130) {
// We succeeded! Validate quality // We succeeded! Validate quality
@@ -191,21 +197,22 @@ describe('Instagram Caption Extraction E2E', () => {
}, 30000); }, 30000);
it('should handle extraction attempt and return truncated text gracefully', async () => { it('should handle extraction attempt and return truncated text gracefully', async () => {
const testUrl = 'https://www.instagram.com/reel/DP6oN7JCEo8/?utm_source=ig_web_button_share_sheet'; const testUrl =
'https://www.instagram.com/reel/DP6oN7JCEo8/?utm_source=ig_web_button_share_sheet';
const result = await extractTextAndThumbnail(testUrl); const result = await extractTextAndThumbnail(testUrl);
// Verify extraction returns something // Verify extraction returns something
expect(result).toBeDefined(); expect(result).toBeDefined();
expect(result.bodyText).toBeDefined(); expect(result.bodyText).toBeDefined();
expect(result.bodyText.length).toBeGreaterThan(0); expect(result.bodyText.length).toBeGreaterThan(0);
// Should start with recipe title (even if truncated) // Should start with recipe title (even if truncated)
expect(result.bodyText).toMatch(/^La cacio e pepe/i); expect(result.bodyText).toMatch(/^La cacio e pepe/i);
// Should have thumbnail // Should have thumbnail
expect(result.thumbnail).toBeDefined(); expect(result.thumbnail).toBeDefined();
console.log(`[Test] Extracted ${result.bodyText.length} chars (Instagram limits scraping)`); console.log(`[Test] Extracted ${result.bodyText.length} chars (Instagram limits scraping)`);
}, 30000); }, 30000);
}); });

100
src/tests/instagram-caption-extraction.unit.spec.ts
View File

@@ -1,11 +1,11 @@
/** /**
* Unit tests for Instagram caption extraction and cleaning * Unit tests for Instagram caption extraction and cleaning
* JIRA: RECIPE-0006 * JIRA: RECIPE-0006
* *
* Tests the cleanText() and extractFromDOM() functions with mocked Playwright Page fixtures. * Tests the cleanText() and extractFromDOM() functions with mocked Playwright Page fixtures.
* Uses exact problematic output from real Instagram data to validate metadata prefix removal, * Uses exact problematic output from real Instagram data to validate metadata prefix removal,
* quote handling, and hashtag cleaning. * quote handling, and hashtag cleaning.
* *
* This replaces slow E2E tests (30s, flaky) with fast unit tests (<100ms, deterministic). * This replaces slow E2E tests (30s, flaky) with fast unit tests (<100ms, deterministic).
*/ */
@@ -17,7 +17,7 @@ describe('cleanText()', () => {
it('should remove hashtags from end of text', () => { it('should remove hashtags from end of text', () => {
const input = 'Recipe instructions here #cacio #pepe #recipe'; const input = 'Recipe instructions here #cacio #pepe #recipe';
const result = cleanText(input); const result = cleanText(input);
expect(result).toBe('Recipe instructions here'); expect(result).toBe('Recipe instructions here');
expect(result).not.toContain('#cacio'); expect(result).not.toContain('#cacio');
expect(result).not.toContain('#pepe'); expect(result).not.toContain('#pepe');
@@ -26,7 +26,7 @@ describe('cleanText()', () => {
it('should preserve hashtags in middle of text', () => { it('should preserve hashtags in middle of text', () => {
const input = 'Try this #amazing recipe for pasta'; const input = 'Try this #amazing recipe for pasta';
const result = cleanText(input); const result = cleanText(input);
expect(result).toContain('#amazing'); expect(result).toContain('#amazing');
expect(result).toBe('Try this #amazing recipe for pasta'); expect(result).toBe('Try this #amazing recipe for pasta');
}); });
@@ -37,7 +37,7 @@ Liked by user123 and others
View all 50 comments View all 50 comments
Add a comment...`; Add a comment...`;
const result = cleanText(input); const result = cleanText(input);
expect(result).toBe('Recipe text'); expect(result).toBe('Recipe text');
expect(result).not.toContain('Liked by'); expect(result).not.toContain('Liked by');
expect(result).not.toContain('View all'); expect(result).not.toContain('View all');
@@ -47,14 +47,14 @@ Add a comment...`;
it('should normalize excessive whitespace', () => { it('should normalize excessive whitespace', () => {
const input = 'Recipe with extra spaces'; const input = 'Recipe with extra spaces';
const result = cleanText(input); const result = cleanText(input);
expect(result).toBe('Recipe with extra spaces'); expect(result).toBe('Recipe with extra spaces');
}); });
it('should handle international characters in hashtags', () => { it('should handle international characters in hashtags', () => {
const input = 'Ricetta italiana #cacio #pepé #àncora'; const input = 'Ricetta italiana #cacio #pepé #àncora';
const result = cleanText(input); const result = cleanText(input);
expect(result).toBe('Ricetta italiana'); expect(result).toBe('Ricetta italiana');
}); });
}); });
@@ -64,12 +64,12 @@ describe('extractFromDOM() with mocked og:description', () => {
// Simulates what the browser's page.evaluate() would return after cleaning metadata // Simulates what the browser's page.evaluate() would return after cleaning metadata
const createMockPage = (ogContent: string | null) => { const createMockPage = (ogContent: string | null) => {
// Simulate the browser's metadata cleaning logic // Simulate the browser's metadata cleaning logic
const cleanedContent = ogContent const cleanedContent = ogContent
? ogContent.replace(/^\d+K?\s+likes,\s+\d+\s+comments\s+-\s+[\w.]+\s+on\s+[^:]+:\s*["']?/, '') ? ogContent.replace(/^\d+K?\s+likes,\s+\d+\s+comments\s+-\s+[\w.]+\s+on\s+[^:]+:\s*["']?/, '')
: null; : null;
let evaluateCallCount = 0; let evaluateCallCount = 0;
return { return {
evaluate: vi.fn().mockImplementation(async () => { evaluate: vi.fn().mockImplementation(async () => {
evaluateCallCount++; evaluateCallCount++;
@@ -91,12 +91,13 @@ describe('extractFromDOM() with mocked og:description', () => {
it('should remove metadata prefix from og:description fallback', async () => { it('should remove metadata prefix from og:description fallback', async () => {
// Exact fixture from context_compact.yaml // Exact fixture from context_compact.yaml
const ogContent = '16K likes, 325 comments - chef.antonio.la.cava on October 17, 2025: "La cacio e pepe infallibile di Luciano Monosilio 🍝'; const ogContent =
'16K likes, 325 comments - chef.antonio.la.cava on October 17, 2025: "La cacio e pepe infallibile di Luciano Monosilio 🍝';
const mockPage = createMockPage(ogContent); const mockPage = createMockPage(ogContent);
const result = await extractFromDOM(mockPage); const result = await extractFromDOM(mockPage);
expect(result).not.toBeNull(); expect(result).not.toBeNull();
expect(result?.bodyText).not.toContain('16K likes'); expect(result?.bodyText).not.toContain('16K likes');
expect(result?.bodyText).not.toContain('chef.antonio.la.cava'); expect(result?.bodyText).not.toContain('chef.antonio.la.cava');
@@ -104,12 +105,13 @@ describe('extractFromDOM() with mocked og:description', () => {
}); });
it('should remove opening quote after metadata prefix', async () => { it('should remove opening quote after metadata prefix', async () => {
const ogContent = '16K likes, 325 comments - chef.antonio.la.cava on October 17, 2025: "La cacio e pepe infallibile di Luciano Monosilio 🍝'; const ogContent =
'16K likes, 325 comments - chef.antonio.la.cava on October 17, 2025: "La cacio e pepe infallibile di Luciano Monosilio 🍝';
const mockPage = createMockPage(ogContent); const mockPage = createMockPage(ogContent);
const result = await extractFromDOM(mockPage); const result = await extractFromDOM(mockPage);
expect(result).not.toBeNull(); expect(result).not.toBeNull();
expect(result?.bodyText).not.toMatch(/^"/); expect(result?.bodyText).not.toMatch(/^"/);
expect(result?.bodyText).toMatch(/^La cacio e pepe/); expect(result?.bodyText).toMatch(/^La cacio e pepe/);
@@ -117,31 +119,31 @@ describe('extractFromDOM() with mocked og:description', () => {
it('should handle metadata prefix with various like counts (K suffix)', async () => { it('should handle metadata prefix with various like counts (K suffix)', async () => {
const ogContent = '1K likes, 50 comments - user.name on January 1, 2025: "Recipe text here'; const ogContent = '1K likes, 50 comments - user.name on January 1, 2025: "Recipe text here';
const mockPage = createMockPage(ogContent); const mockPage = createMockPage(ogContent);
const result = await extractFromDOM(mockPage); const result = await extractFromDOM(mockPage);
expect(result).not.toBeNull(); expect(result).not.toBeNull();
expect(result?.bodyText).toBe('Recipe text here'); expect(result?.bodyText).toBe('Recipe text here');
}); });
it('should handle metadata prefix without K suffix', async () => { it('should handle metadata prefix without K suffix', async () => {
const ogContent = '500 likes, 20 comments - username on May 5, 2024: Recipe content'; const ogContent = '500 likes, 20 comments - username on May 5, 2024: Recipe content';
const mockPage = createMockPage(ogContent); const mockPage = createMockPage(ogContent);
const result = await extractFromDOM(mockPage); const result = await extractFromDOM(mockPage);
expect(result).not.toBeNull(); expect(result).not.toBeNull();
expect(result?.bodyText).toBe('Recipe content'); expect(result?.bodyText).toBe('Recipe content');
}); });
it('should return null when no content available', async () => { it('should return null when no content available', async () => {
const mockPage = createMockPage(null); const mockPage = createMockPage(null);
const result = await extractFromDOM(mockPage); const result = await extractFromDOM(mockPage);
expect(result).toBeNull(); expect(result).toBeNull();
}); });
}); });
@@ -168,41 +170,43 @@ describe('Integration: Full extraction flow', () => {
it('should extract, clean metadata prefix, remove quotes, and clean hashtags', async () => { it('should extract, clean metadata prefix, remove quotes, and clean hashtags', async () => {
// Simulating what the browser's page.evaluate() would return AFTER cleaning metadata // Simulating what the browser's page.evaluate() would return AFTER cleaning metadata
// (the browser regex already strips the metadata prefix and quotes) // (the browser regex already strips the metadata prefix and quotes)
const browserCleanedContent = 'La cacio e pepe infallibile di Luciano Monosilio 🍝 #cacio #pepe #recipe'; const browserCleanedContent =
'La cacio e pepe infallibile di Luciano Monosilio 🍝 #cacio #pepe #recipe';
const mockPage = createMockPage(browserCleanedContent); const mockPage = createMockPage(browserCleanedContent);
const result = await extractFromDOM(mockPage); const result = await extractFromDOM(mockPage);
expect(result).not.toBeNull(); expect(result).not.toBeNull();
// Verify no metadata prefix // Verify no metadata prefix
expect(result?.bodyText).not.toContain('16K likes'); expect(result?.bodyText).not.toContain('16K likes');
expect(result?.bodyText).not.toContain('chef.antonio.la.cava'); expect(result?.bodyText).not.toContain('chef.antonio.la.cava');
// Verify no opening quote // Verify no opening quote
expect(result?.bodyText).not.toMatch(/^"/); expect(result?.bodyText).not.toMatch(/^"/);
// Verify starts with actual content // Verify starts with actual content
expect(result?.bodyText).toMatch(/^La cacio e pepe/); expect(result?.bodyText).toMatch(/^La cacio e pepe/);
// Verify hashtags removed from end // Verify hashtags removed from end
expect(result?.bodyText).not.toContain('#cacio'); expect(result?.bodyText).not.toContain('#cacio');
expect(result?.bodyText).not.toContain('#pepe'); expect(result?.bodyText).not.toContain('#pepe');
expect(result?.bodyText).not.toContain('#recipe'); expect(result?.bodyText).not.toContain('#recipe');
// Verify clean output // Verify clean output
expect(result?.bodyText).toBe('La cacio e pepe infallibile di Luciano Monosilio 🍝'); expect(result?.bodyText).toBe('La cacio e pepe infallibile di Luciano Monosilio 🍝');
}); });
it('should handle full real-world caption with multiline content', async () => { it('should handle full real-world caption with multiline content', async () => {
// Browser has already cleaned metadata, only hashtags remain // Browser has already cleaned metadata, only hashtags remain
const browserCleanedContent = 'La cacio e pepe\n\nIngredients:\n- Pasta\n- Cheese\n\n#recipe #pasta'; const browserCleanedContent =
'La cacio e pepe\n\nIngredients:\n- Pasta\n- Cheese\n\n#recipe #pasta';
const mockPage = createMockPage(browserCleanedContent); const mockPage = createMockPage(browserCleanedContent);
const result = await extractFromDOM(mockPage); const result = await extractFromDOM(mockPage);
expect(result).not.toBeNull(); expect(result).not.toBeNull();
expect(result?.bodyText).toMatch(/^La cacio e pepe/); expect(result?.bodyText).toMatch(/^La cacio e pepe/);
expect(result?.bodyText).toContain('Ingredients:'); expect(result?.bodyText).toContain('Ingredients:');
@@ -213,11 +217,11 @@ describe('Integration: Full extraction flow', () => {
it('should preserve emojis in extracted text', async () => { it('should preserve emojis in extracted text', async () => {
const browserCleanedContent = 'Recipe 🍝 with emojis 🙏🏻 📝'; const browserCleanedContent = 'Recipe 🍝 with emojis 🙏🏻 📝';
const mockPage = createMockPage(browserCleanedContent); const mockPage = createMockPage(browserCleanedContent);
const result = await extractFromDOM(mockPage); const result = await extractFromDOM(mockPage);
expect(result).not.toBeNull(); expect(result).not.toBeNull();
expect(result?.bodyText).toContain('🍝'); expect(result?.bodyText).toContain('🍝');
expect(result?.bodyText).toContain('🙏🏻'); expect(result?.bodyText).toContain('🙏🏻');
@@ -226,22 +230,22 @@ describe('Integration: Full extraction flow', () => {
it('should handle content without hashtags', async () => { it('should handle content without hashtags', async () => {
const browserCleanedContent = 'Simple recipe text'; const browserCleanedContent = 'Simple recipe text';
const mockPage = createMockPage(browserCleanedContent); const mockPage = createMockPage(browserCleanedContent);
const result = await extractFromDOM(mockPage); const result = await extractFromDOM(mockPage);
expect(result).not.toBeNull(); expect(result).not.toBeNull();
expect(result?.bodyText).toBe('Simple recipe text'); expect(result?.bodyText).toBe('Simple recipe text');
}); });
it('should handle single quote instead of double quote', async () => { it('should handle single quote instead of double quote', async () => {
const browserCleanedContent = 'Recipe with single quote'; const browserCleanedContent = 'Recipe with single quote';
const mockPage = createMockPage(browserCleanedContent); const mockPage = createMockPage(browserCleanedContent);
const result = await extractFromDOM(mockPage); const result = await extractFromDOM(mockPage);
expect(result).not.toBeNull(); expect(result).not.toBeNull();
expect(result?.bodyText).not.toMatch(/^'/); expect(result?.bodyText).not.toMatch(/^'/);
expect(result?.bodyText).toBe('Recipe with single quote'); expect(result?.bodyText).toBe('Recipe with single quote');

5
src/tests/llm-logging.spec.ts
View File

@@ -76,9 +76,6 @@ describe('llm.ts logging', () => {
await checkModelAvailability('test-model'); await checkModelAvailability('test-model');
expect(logErrorSpy).toHaveBeenCalledWith( expect(logErrorSpy).toHaveBeenCalledWith('[LLM] Model availability check failed', complexError);
'[LLM] Model availability check failed',
complexError
);
}); });
}); });

303
src/tests/logger.spec.ts
View File

@@ -2,157 +2,154 @@ import { describe, test, expect, vi, beforeEach, afterEach } from 'vitest';
import { serializeError, serializeObject, logError, logObject } from '$lib/server/utils/logger'; import { serializeError, serializeObject, logError, logObject } from '$lib/server/utils/logger';
describe('logger utilities', () => { describe('logger utilities', () => {
describe('serializeError', () => { describe('serializeError', () => {
test('handles Error objects', () => { test('handles Error objects', () => {
const error = new Error('Test error message'); const error = new Error('Test error message');
const result = serializeError(error); const result = serializeError(error);
expect(result).toContain('Test error message'); expect(result).toContain('Test error message');
expect(result).toContain('"name": "Error"'); expect(result).toContain('"name": "Error"');
expect(result).toContain('"message"'); expect(result).toContain('"message"');
}); });
test('handles plain objects', () => { test('handles plain objects', () => {
const obj = { code: 404, message: 'Not found' }; const obj = { code: 404, message: 'Not found' };
const result = serializeError(obj); const result = serializeError(obj);
expect(result).toContain('"code": 404'); expect(result).toContain('"code": 404');
expect(result).toContain('"message": "Not found"'); expect(result).toContain('"message": "Not found"');
}); });
test('includes stack trace for Error objects', () => { test('includes stack trace for Error objects', () => {
const error = new Error('Stack test'); const error = new Error('Stack test');
const result = serializeError(error); const result = serializeError(error);
expect(result).toContain('"stack"'); expect(result).toContain('"stack"');
}); });
test('handles Error with custom properties', () => { test('handles Error with custom properties', () => {
const error = new Error('Custom error') as any; const error = new Error('Custom error') as any;
error.statusCode = 500; error.statusCode = 500;
error.details = { info: 'extra data' }; error.details = { info: 'extra data' };
const result = serializeError(error); const result = serializeError(error);
expect(result).toContain('"statusCode": 500'); expect(result).toContain('"statusCode": 500');
expect(result).toContain('extra data'); expect(result).toContain('extra data');
}); });
}); });
describe('serializeObject', () => { describe('serializeObject', () => {
test('handles circular references', () => { test('handles circular references', () => {
const obj: any = { a: 1, b: 2 }; const obj: any = { a: 1, b: 2 };
obj.self = obj; obj.self = obj;
const result = serializeObject(obj); const result = serializeObject(obj);
expect(result).toContain('[Circular]'); expect(result).toContain('[Circular]');
expect(result).toContain('"a": 1'); expect(result).toContain('"a": 1');
}); });
test('handles deeply nested objects', () => { test('handles deeply nested objects', () => {
const obj = { const obj = {
level1: { level1: {
level2: { level2: {
level3: { level3: {
value: 'deep' value: 'deep'
} }
} }
} }
}; };
const result = serializeObject(obj); const result = serializeObject(obj);
expect(result).toContain('"value": "deep"'); expect(result).toContain('"value": "deep"');
}); });
test('handles arrays', () => { test('handles arrays', () => {
const obj = { items: [1, 2, 3] }; const obj = { items: [1, 2, 3] };
const result = serializeObject(obj); const result = serializeObject(obj);
expect(result).toContain('"items"'); expect(result).toContain('"items"');
expect(result).toContain('['); expect(result).toContain('[');
}); });
test('handles null and undefined', () => { test('handles null and undefined', () => {
const obj = { a: null, b: undefined }; const obj = { a: null, b: undefined };
const result = serializeObject(obj); const result = serializeObject(obj);
expect(result).toContain('"a": null'); expect(result).toContain('"a": null');
}); });
}); });
describe('logError', () => { describe('logError', () => {
let consoleErrorSpy: any; let consoleErrorSpy: any;
beforeEach(() => { beforeEach(() => {
consoleErrorSpy = vi.spyOn(console, 'error').mockImplementation(() => {}); consoleErrorSpy = vi.spyOn(console, 'error').mockImplementation(() => {});
}); });
afterEach(() => { afterEach(() => {
consoleErrorSpy.mockRestore(); consoleErrorSpy.mockRestore();
}); });
test('outputs to console.error', () => { test('outputs to console.error', () => {
const error = new Error('Test'); const error = new Error('Test');
logError('[Test]', error); logError('[Test]', error);
expect(consoleErrorSpy).toHaveBeenCalledWith('[Test]', 'Test'); expect(consoleErrorSpy).toHaveBeenCalledWith('[Test]', 'Test');
}); });
test('logs stack trace for Error objects', () => { test('logs stack trace for Error objects', () => {
const error = new Error('Stack error'); const error = new Error('Stack error');
logError('[Test]', error); logError('[Test]', error);
expect(consoleErrorSpy).toHaveBeenCalledWith( expect(consoleErrorSpy).toHaveBeenCalledWith(
expect.stringMatching(/Stack/), expect.stringMatching(/Stack/),
expect.any(String) expect.any(String)
); );
}); });
test('handles non-Error objects', () => { test('handles non-Error objects', () => {
const obj = { code: 500, message: 'Server error' }; const obj = { code: 500, message: 'Server error' };
logError('[Test]', obj); logError('[Test]', obj);
expect(consoleErrorSpy).toHaveBeenCalledWith( expect(consoleErrorSpy).toHaveBeenCalledWith(
'[Test]', '[Test]',
expect.stringContaining('"code": 500') expect.stringContaining('"code": 500')
); );
}); });
}); });
describe('logObject', () => { describe('logObject', () => {
let consoleLogSpy: any; let consoleLogSpy: any;
beforeEach(() => { beforeEach(() => {
consoleLogSpy = vi.spyOn(console, 'log').mockImplementation(() => {}); consoleLogSpy = vi.spyOn(console, 'log').mockImplementation(() => {});
}); });
afterEach(() => { afterEach(() => {
consoleLogSpy.mockRestore(); consoleLogSpy.mockRestore();
}); });
test('outputs to console.log', () => { test('outputs to console.log', () => {
const obj = { key: 'value' }; const obj = { key: 'value' };
logObject('[Test]', obj); logObject('[Test]', obj);
expect(consoleLogSpy).toHaveBeenCalledWith( expect(consoleLogSpy).toHaveBeenCalledWith(
'[Test]', '[Test]',
expect.stringContaining('"key": "value"') expect.stringContaining('"key": "value"')
); );
}); });
test('handles circular references', () => { test('handles circular references', () => {
const obj: any = { a: 1 }; const obj: any = { a: 1 };
obj.self = obj; obj.self = obj;
logObject('[Test]', obj); logObject('[Test]', obj);
expect(consoleLogSpy).toHaveBeenCalledWith( expect(consoleLogSpy).toHaveBeenCalledWith('[Test]', expect.stringContaining('[Circular]'));
'[Test]', });
expect.stringContaining('[Circular]') });
);
});
});
}); });

288
src/tests/notification-test-api.spec.ts
View File

@@ -1,6 +1,6 @@
/** /**
* Tests for Test Notification API Endpoint * Tests for Test Notification API Endpoint
* *
* Verifies /api/notifications/test endpoint functionality including: * Verifies /api/notifications/test endpoint functionality including:
* - Type validation * - Type validation
* - Payload structure * - Payload structure
@@ -12,179 +12,181 @@ import { POST } from '../routes/api/notifications/test/+server';
import { pushNotificationService } from '$lib/server/notifications/PushNotificationService'; import { pushNotificationService } from '$lib/server/notifications/PushNotificationService';
describe('POST /api/notifications/test', () => { describe('POST /api/notifications/test', () => {
let sendNotificationSpy: any; let sendNotificationSpy: any;
let getSubscriptionCountSpy: any; let getSubscriptionCountSpy: any;
beforeEach(() => { beforeEach(() => {
vi.clearAllMocks(); vi.clearAllMocks();
// Spy on pushNotificationService methods
sendNotificationSpy = vi.spyOn(pushNotificationService, 'sendNotification').mockResolvedValue();
getSubscriptionCountSpy = vi.spyOn(pushNotificationService, 'getSubscriptionCount').mockReturnValue(2);
});
test('should validate notification type - reject invalid type', async () => { // Spy on pushNotificationService methods
const request = new Request('http://localhost/api/notifications/test', { sendNotificationSpy = vi.spyOn(pushNotificationService, 'sendNotification').mockResolvedValue();
method: 'POST', getSubscriptionCountSpy = vi
headers: { 'Content-Type': 'application/json' }, .spyOn(pushNotificationService, 'getSubscriptionCount')
body: JSON.stringify({ type: 'invalid' }) .mockReturnValue(2);
}); });
const response = await POST({ request } as any); test('should validate notification type - reject invalid type', async () => {
const data = await response.json(); const request = new Request('http://localhost/api/notifications/test', {
method: 'POST',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify({ type: 'invalid' })
});
expect(response.status).toBe(400); const response = await POST({ request } as any);
expect(data.error).toContain('Invalid notification type'); const data = await response.json();
expect(sendNotificationSpy).not.toHaveBeenCalled();
});
test('should validate notification type - reject missing type', async () => { expect(response.status).toBe(400);
const request = new Request('http://localhost/api/notifications/test', { expect(data.error).toContain('Invalid notification type');
method: 'POST', expect(sendNotificationSpy).not.toHaveBeenCalled();
headers: { 'Content-Type': 'application/json' }, });
body: JSON.stringify({})
});
const response = await POST({ request } as any); test('should validate notification type - reject missing type', async () => {
const data = await response.json(); const request = new Request('http://localhost/api/notifications/test', {
method: 'POST',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify({})
});
expect(response.status).toBe(400); const response = await POST({ request } as any);
expect(data.error).toContain('Invalid notification type'); const data = await response.json();
expect(sendNotificationSpy).not.toHaveBeenCalled();
});
test('should send test success notification', async () => { expect(response.status).toBe(400);
const request = new Request('http://localhost/api/notifications/test', { expect(data.error).toContain('Invalid notification type');
method: 'POST', expect(sendNotificationSpy).not.toHaveBeenCalled();
headers: { 'Content-Type': 'application/json' }, });
body: JSON.stringify({ type: 'success' })
});
const response = await POST({ request } as any); test('should send test success notification', async () => {
const data = await response.json(); const request = new Request('http://localhost/api/notifications/test', {
method: 'POST',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify({ type: 'success' })
});
expect(response.status).toBe(200); const response = await POST({ request } as any);
expect(data.success).toBe(true); const data = await response.json();
expect(data.message).toContain('success');
expect(data.subscriberCount).toBe(2);
expect(sendNotificationSpy).toHaveBeenCalledWith( expect(response.status).toBe(200);
expect.objectContaining({ expect(data.success).toBe(true);
type: 'success', expect(data.message).toContain('success');
body: expect.stringContaining('Test recipe'), expect(data.subscriberCount).toBe(2);
recipeName: 'Test Recipe',
itemId: expect.stringMatching(/^test_\d+$/),
tag: expect.stringMatching(/^recipe-success-test_\d+$/),
requireInteraction: false
})
);
});
test('should send test error notification', async () => { expect(sendNotificationSpy).toHaveBeenCalledWith(
const request = new Request('http://localhost/api/notifications/test', { expect.objectContaining({
method: 'POST', type: 'success',
headers: { 'Content-Type': 'application/json' }, body: expect.stringContaining('Test recipe'),
body: JSON.stringify({ type: 'error' }) recipeName: 'Test Recipe',
}); itemId: expect.stringMatching(/^test_\d+$/),
tag: expect.stringMatching(/^recipe-success-test_\d+$/),
requireInteraction: false
})
);
});
const response = await POST({ request } as any); test('should send test error notification', async () => {
const data = await response.json(); const request = new Request('http://localhost/api/notifications/test', {
method: 'POST',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify({ type: 'error' })
});
expect(response.status).toBe(200); const response = await POST({ request } as any);
expect(data.success).toBe(true); const data = await response.json();
expect(data.message).toContain('error');
expect(sendNotificationSpy).toHaveBeenCalledWith( expect(response.status).toBe(200);
expect.objectContaining({ expect(data.success).toBe(true);
type: 'error', expect(data.message).toContain('error');
body: expect.stringContaining('test error'),
itemId: expect.stringMatching(/^test_\d+$/),
tag: expect.stringMatching(/^recipe-error-test_\d+$/),
requireInteraction: true
})
);
});
test('should send test progress notification', async () => { expect(sendNotificationSpy).toHaveBeenCalledWith(
const request = new Request('http://localhost/api/notifications/test', { expect.objectContaining({
method: 'POST', type: 'error',
headers: { 'Content-Type': 'application/json' }, body: expect.stringContaining('test error'),
body: JSON.stringify({ type: 'progress' }) itemId: expect.stringMatching(/^test_\d+$/),
}); tag: expect.stringMatching(/^recipe-error-test_\d+$/),
requireInteraction: true
})
);
});
const response = await POST({ request } as any); test('should send test progress notification', async () => {
const data = await response.json(); const request = new Request('http://localhost/api/notifications/test', {
method: 'POST',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify({ type: 'progress' })
});
expect(response.status).toBe(200); const response = await POST({ request } as any);
expect(data.success).toBe(true); const data = await response.json();
expect(data.message).toContain('progress');
expect(sendNotificationSpy).toHaveBeenCalledWith( expect(response.status).toBe(200);
expect.objectContaining({ expect(data.success).toBe(true);
type: 'progress', expect(data.message).toContain('progress');
body: expect.stringContaining('parsing phase'),
itemId: expect.stringMatching(/^test_\d+$/),
tag: expect.stringMatching(/^recipe-progress-test_\d+$/),
requireInteraction: false
})
);
});
test('should return subscriber count in response', async () => { expect(sendNotificationSpy).toHaveBeenCalledWith(
getSubscriptionCountSpy.mockReturnValue(5); expect.objectContaining({
type: 'progress',
body: expect.stringContaining('parsing phase'),
itemId: expect.stringMatching(/^test_\d+$/),
tag: expect.stringMatching(/^recipe-progress-test_\d+$/),
requireInteraction: false
})
);
});
const request = new Request('http://localhost/api/notifications/test', { test('should return subscriber count in response', async () => {
method: 'POST', getSubscriptionCountSpy.mockReturnValue(5);
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify({ type: 'success' })
});
const response = await POST({ request } as any); const request = new Request('http://localhost/api/notifications/test', {
const data = await response.json(); method: 'POST',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify({ type: 'success' })
});
expect(data.subscriberCount).toBe(5); const response = await POST({ request } as any);
expect(getSubscriptionCountSpy).toHaveBeenCalled(); const data = await response.json();
});
test('should handle sendNotification errors', async () => { expect(data.subscriberCount).toBe(5);
sendNotificationSpy.mockRejectedValue(new Error('Push service error')); expect(getSubscriptionCountSpy).toHaveBeenCalled();
});
const request = new Request('http://localhost/api/notifications/test', { test('should handle sendNotification errors', async () => {
method: 'POST', sendNotificationSpy.mockRejectedValue(new Error('Push service error'));
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify({ type: 'success' })
});
const response = await POST({ request } as any); const request = new Request('http://localhost/api/notifications/test', {
const data = await response.json(); method: 'POST',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify({ type: 'success' })
});
expect(response.status).toBe(500); const response = await POST({ request } as any);
expect(data.error).toContain('Failed to send test notification'); const data = await response.json();
});
test('should generate unique itemId for each request', async () => { expect(response.status).toBe(500);
const request1 = new Request('http://localhost/api/notifications/test', { expect(data.error).toContain('Failed to send test notification');
method: 'POST', });
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify({ type: 'success' })
});
const request2 = new Request('http://localhost/api/notifications/test', { test('should generate unique itemId for each request', async () => {
method: 'POST', const request1 = new Request('http://localhost/api/notifications/test', {
headers: { 'Content-Type': 'application/json' }, method: 'POST',
body: JSON.stringify({ type: 'success' }) headers: { 'Content-Type': 'application/json' },
}); body: JSON.stringify({ type: 'success' })
});
await POST({ request: request1 } as any); const request2 = new Request('http://localhost/api/notifications/test', {
const call1 = sendNotificationSpy.mock.calls[0][0]; method: 'POST',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify({ type: 'success' })
});
// Wait a bit to ensure different timestamp await POST({ request: request1 } as any);
await new Promise(resolve => setTimeout(resolve, 2)); const call1 = sendNotificationSpy.mock.calls[0][0];
await POST({ request: request2 } as any); // Wait a bit to ensure different timestamp
const call2 = sendNotificationSpy.mock.calls[1][0]; await new Promise((resolve) => setTimeout(resolve, 2));
expect(call1.itemId).not.toBe(call2.itemId); await POST({ request: request2 } as any);
expect(call1.tag).not.toBe(call2.tag); const call2 = sendNotificationSpy.mock.calls[1][0];
});
expect(call1.itemId).not.toBe(call2.itemId);
expect(call1.tag).not.toBe(call2.tag);
});
}); });

15
src/tests/parser-logging.spec.ts
View File

@@ -47,10 +47,7 @@ describe('parser.ts logging', () => {
// Expected to throw // Expected to throw
} }
expect(logErrorSpy).toHaveBeenCalledWith( expect(logErrorSpy).toHaveBeenCalledWith('[LLM] Recipe detection error', expect.any(Error));
'[LLM] Recipe detection error',
expect.any(Error)
);
}); });
test('parseRecipe should use logError on failure', async () => { test('parseRecipe should use logError on failure', async () => {
@@ -60,10 +57,7 @@ describe('parser.ts logging', () => {
// Expected to throw // Expected to throw
} }
expect(logErrorSpy).toHaveBeenCalledWith( expect(logErrorSpy).toHaveBeenCalledWith('[LLM] Recipe parsing error', expect.any(Error));
'[LLM] Recipe parsing error',
expect.any(Error)
);
}); });
test('should not log stack trace separately', async () => { test('should not log stack trace separately', async () => {
@@ -73,8 +67,9 @@ describe('parser.ts logging', () => {
// Expected to throw // Expected to throw
} }
const stackCalls = consoleErrorSpy.mock.calls const stackCalls = consoleErrorSpy.mock.calls.filter((call: any) =>
.filter((call: any) => call[0]?.includes('Stack trace')); call[0]?.includes('Stack trace')
);
expect(stackCalls).toHaveLength(0); expect(stackCalls).toHaveLength(0);
}); });

299
src/tests/push-notification-service.spec.ts
View File

@@ -4,190 +4,189 @@ import webpush from 'web-push';
// Mock web-push module BEFORE importing the service // Mock web-push module BEFORE importing the service
vi.mock('web-push', () => ({ vi.mock('web-push', () => ({
default: { default: {
setVapidDetails: vi.fn(), setVapidDetails: vi.fn(),
sendNotification: vi.fn() sendNotification: vi.fn()
} }
})); }));
// Import service AFTER mocking // Import service AFTER mocking
import { pushNotificationService } from '$lib/server/notifications/PushNotificationService'; import { pushNotificationService } from '$lib/server/notifications/PushNotificationService';
describe('PushNotificationService web-push integration', () => { describe('PushNotificationService web-push integration', () => {
beforeEach(() => { beforeEach(() => {
vi.clearAllMocks(); vi.clearAllMocks();
// Clear all subscriptions before each test // Clear all subscriptions before each test
pushNotificationService.clearAllSubscriptions(); pushNotificationService.clearAllSubscriptions();
}); });
test('should have VAPID public key configured', () => { test('should have VAPID public key configured', () => {
// Verify the service has a public VAPID key available // Verify the service has a public VAPID key available
const publicKey = pushNotificationService.getPublicVapidKey(); const publicKey = pushNotificationService.getPublicVapidKey();
expect(publicKey).toBeTruthy(); expect(publicKey).toBeTruthy();
expect(typeof publicKey).toBe('string'); expect(typeof publicKey).toBe('string');
expect(publicKey!.length).toBeGreaterThan(0); expect(publicKey!.length).toBeGreaterThan(0);
}); });
test('should send notification with web-push', async () => { test('should send notification with web-push', async () => {
const mockSubscription = { const mockSubscription = {
endpoint: 'https://push.example.com/test', endpoint: 'https://push.example.com/test',
keys: { p256dh: 'test-key', auth: 'test-auth' } keys: { p256dh: 'test-key', auth: 'test-auth' }
}; };
vi.mocked(webpush.sendNotification).mockResolvedValue({} as any); vi.mocked(webpush.sendNotification).mockResolvedValue({} as any);
await pushNotificationService.subscribe('client-1', mockSubscription); await pushNotificationService.subscribe('client-1', mockSubscription);
await pushNotificationService.sendNotification({ await pushNotificationService.sendNotification({
type: 'success', type: 'success',
itemId: 'test-123', itemId: 'test-123',
body: 'Test notification' body: 'Test notification'
}); });
expect(webpush.sendNotification).toHaveBeenCalledWith( expect(webpush.sendNotification).toHaveBeenCalledWith(
expect.objectContaining({ expect.objectContaining({
endpoint: mockSubscription.endpoint, endpoint: mockSubscription.endpoint,
keys: mockSubscription.keys keys: mockSubscription.keys
}), }),
expect.any(String), expect.any(String),
expect.objectContaining({ expect.objectContaining({
TTL: 60 * 60 * 24 TTL: 60 * 60 * 24
}) })
); );
}); });
test('should handle subscription expiration (410)', async () => { test('should handle subscription expiration (410)', async () => {
const mockError: any = new Error('Gone'); const mockError: any = new Error('Gone');
mockError.statusCode = 410; mockError.statusCode = 410;
vi.mocked(webpush.sendNotification).mockRejectedValue(mockError); vi.mocked(webpush.sendNotification).mockRejectedValue(mockError);
const mockSubscription = { const mockSubscription = {
endpoint: 'https://push.example.com/expired', endpoint: 'https://push.example.com/expired',
keys: { p256dh: 'test', auth: 'test' } keys: { p256dh: 'test', auth: 'test' }
}; };
await pushNotificationService.subscribe('client-1', mockSubscription); await pushNotificationService.subscribe('client-1', mockSubscription);
// Verify subscription exists before sending
expect(pushNotificationService.getSubscriptionCount()).toBe(1);
// sendNotification catches errors internally and removes invalid subscriptions // Verify subscription exists before sending
// It doesn't throw, so we just await it expect(pushNotificationService.getSubscriptionCount()).toBe(1);
await pushNotificationService.sendNotification({
type: 'error',
itemId: 'test',
body: 'Test'
});
// Verify the subscription was removed due to 410 error // sendNotification catches errors internally and removes invalid subscriptions
expect(pushNotificationService.getSubscriptionCount()).toBe(0); // It doesn't throw, so we just await it
}); await pushNotificationService.sendNotification({
type: 'error',
itemId: 'test',
body: 'Test'
});
test('should send notification with TTL of 24 hours', async () => { // Verify the subscription was removed due to 410 error
const mockSubscription = { expect(pushNotificationService.getSubscriptionCount()).toBe(0);
endpoint: 'https://push.example.com/test-ttl', });
keys: { p256dh: 'test-key', auth: 'test-auth' }
};
vi.mocked(webpush.sendNotification).mockResolvedValue({} as any); test('should send notification with TTL of 24 hours', async () => {
const mockSubscription = {
endpoint: 'https://push.example.com/test-ttl',
keys: { p256dh: 'test-key', auth: 'test-auth' }
};
await pushNotificationService.subscribe('client-2', mockSubscription); vi.mocked(webpush.sendNotification).mockResolvedValue({} as any);
await pushNotificationService.sendNotification({
type: 'progress',
itemId: 'test-456',
body: 'Progress update'
});
expect(webpush.sendNotification).toHaveBeenCalledWith( await pushNotificationService.subscribe('client-2', mockSubscription);
expect.any(Object), await pushNotificationService.sendNotification({
expect.any(String), type: 'progress',
{ TTL: 60 * 60 * 24 } itemId: 'test-456',
); body: 'Progress update'
}); });
test('should serialize notification data as JSON', async () => { expect(webpush.sendNotification).toHaveBeenCalledWith(expect.any(Object), expect.any(String), {
const mockSubscription = { TTL: 60 * 60 * 24
endpoint: 'https://push.example.com/test-json', });
keys: { p256dh: 'test-key', auth: 'test-auth' } });
};
vi.mocked(webpush.sendNotification).mockResolvedValue({} as any); test('should serialize notification data as JSON', async () => {
const mockSubscription = {
endpoint: 'https://push.example.com/test-json',
keys: { p256dh: 'test-key', auth: 'test-auth' }
};
const testPayload = { vi.mocked(webpush.sendNotification).mockResolvedValue({} as any);
type: 'success' as const,
itemId: 'test-789',
body: 'JSON test',
recipeName: 'Test Recipe'
};
await pushNotificationService.subscribe('client-3', mockSubscription); const testPayload = {
await pushNotificationService.sendNotification(testPayload); type: 'success' as const,
itemId: 'test-789',
body: 'JSON test',
recipeName: 'Test Recipe'
};
const sendCallArgs = vi.mocked(webpush.sendNotification).mock.calls[0]; await pushNotificationService.subscribe('client-3', mockSubscription);
const sentPayload = sendCallArgs[1]; await pushNotificationService.sendNotification(testPayload);
// Verify the payload is stringified JSON
expect(typeof sentPayload).toBe('string');
const parsedPayload = JSON.parse(sentPayload);
expect(parsedPayload).toMatchObject({
type: 'success',
itemId: 'test-789',
body: 'JSON test',
recipeName: 'Test Recipe'
});
});
test('should handle multiple subscriptions', async () => { const sendCallArgs = vi.mocked(webpush.sendNotification).mock.calls[0];
const mockSubscription1 = { const sentPayload = sendCallArgs[1];
endpoint: 'https://push.example.com/client1',
keys: { p256dh: 'key1', auth: 'auth1' }
};
const mockSubscription2 = {
endpoint: 'https://push.example.com/client2',
keys: { p256dh: 'key2', auth: 'auth2' }
};
vi.mocked(webpush.sendNotification).mockResolvedValue({} as any); // Verify the payload is stringified JSON
expect(typeof sentPayload).toBe('string');
const parsedPayload = JSON.parse(sentPayload);
expect(parsedPayload).toMatchObject({
type: 'success',
itemId: 'test-789',
body: 'JSON test',
recipeName: 'Test Recipe'
});
});
await pushNotificationService.subscribe('client-1', mockSubscription1); test('should handle multiple subscriptions', async () => {
await pushNotificationService.subscribe('client-2', mockSubscription2); const mockSubscription1 = {
endpoint: 'https://push.example.com/client1',
keys: { p256dh: 'key1', auth: 'auth1' }
};
const mockSubscription2 = {
endpoint: 'https://push.example.com/client2',
keys: { p256dh: 'key2', auth: 'auth2' }
};
await pushNotificationService.sendNotification({ vi.mocked(webpush.sendNotification).mockResolvedValue({} as any);
type: 'success',
itemId: 'test-multi',
body: 'Multi-subscriber test'
});
// Should have sent to both subscribers await pushNotificationService.subscribe('client-1', mockSubscription1);
expect(webpush.sendNotification).toHaveBeenCalledTimes(2); await pushNotificationService.subscribe('client-2', mockSubscription2);
});
test('should log endpoint prefix only (privacy)', async () => { await pushNotificationService.sendNotification({
const consoleSpy = vi.spyOn(console, 'log'); type: 'success',
itemId: 'test-multi',
const longEndpoint = 'https://push.example.com/very-long-endpoint-with-secret-tokens-that-should-not-be-fully-logged-12345678901234567890'; body: 'Multi-subscriber test'
const mockSubscription = { });
endpoint: longEndpoint,
keys: { p256dh: 'test-key', auth: 'test-auth' }
};
vi.mocked(webpush.sendNotification).mockResolvedValue({} as any); // Should have sent to both subscribers
expect(webpush.sendNotification).toHaveBeenCalledTimes(2);
});
await pushNotificationService.subscribe('client-privacy', mockSubscription); test('should log endpoint prefix only (privacy)', async () => {
await pushNotificationService.sendNotification({ const consoleSpy = vi.spyOn(console, 'log');
type: 'success',
itemId: 'test-privacy',
body: 'Privacy test'
});
// Find the log call with endpoint const longEndpoint =
const endpointLogCall = consoleSpy.mock.calls.find( 'https://push.example.com/very-long-endpoint-with-secret-tokens-that-should-not-be-fully-logged-12345678901234567890';
call => typeof call[0] === 'string' && call[0].includes('Sent notification to') const mockSubscription = {
); endpoint: longEndpoint,
keys: { p256dh: 'test-key', auth: 'test-auth' }
};
expect(endpointLogCall).toBeTruthy(); vi.mocked(webpush.sendNotification).mockResolvedValue({} as any);
// Should log only first 50 chars + ellipsis, not the full endpoint
expect(endpointLogCall![0]).toContain(longEndpoint.substring(0, 50)); await pushNotificationService.subscribe('client-privacy', mockSubscription);
expect(endpointLogCall![0]).not.toContain('secret-tokens'); await pushNotificationService.sendNotification({
}); type: 'success',
itemId: 'test-privacy',
body: 'Privacy test'
});
// Find the log call with endpoint
const endpointLogCall = consoleSpy.mock.calls.find(
(call) => typeof call[0] === 'string' && call[0].includes('Sent notification to')
);
expect(endpointLogCall).toBeTruthy();
// Should log only first 50 chars + ellipsis, not the full endpoint
expect(endpointLogCall![0]).toContain(longEndpoint.substring(0, 50));
expect(endpointLogCall![0]).not.toContain('secret-tokens');
});
}); });

294
src/tests/push-notifications.e2e.spec.ts
View File

@@ -1,6 +1,6 @@
/** /**
* E2E Tests for Push Notifications * E2E Tests for Push Notifications
* *
* Tests the complete push notification workflow using Playwright: * Tests the complete push notification workflow using Playwright:
* - Permission granting * - Permission granting
* - Subscription creation * - Subscription creation
@@ -8,197 +8,199 @@
* - Manual test notifications * - Manual test notifications
* - Unsubscribe flow * - Unsubscribe flow
* - localStorage persistence * - localStorage persistence
* *
* Note: These tests require the dev server to be running. * Note: These tests require the dev server to be running.
*/ */
import { test, expect, type BrowserContext } from '@playwright/test'; import { test, expect, type BrowserContext } from '@playwright/test';
test.describe('Push Notifications E2E', () => { test.describe('Push Notifications E2E', () => {
let context: BrowserContext; let context: BrowserContext;
test.beforeEach(async ({ browser }) => { test.beforeEach(async ({ browser }) => {
// Create new context with notification permissions granted // Create new context with notification permissions granted
context = await browser.newContext(); context = await browser.newContext();
await context.grantPermissions(['notifications']); await context.grantPermissions(['notifications']);
}); });
test.afterEach(async () => { test.afterEach(async () => {
await context?.close(); await context?.close();
}); });
test('should subscribe to push notifications', async () => { test('should subscribe to push notifications', async () => {
const page = await context.newPage(); const page = await context.newPage();
await page.goto('/'); await page.goto('/');
await page.waitForLoadState('networkidle'); await page.waitForLoadState('networkidle');
// Wait for service worker to be registered // Wait for service worker to be registered
await page.waitForFunction(() => 'serviceWorker' in navigator && 'PushManager' in window); await page.waitForFunction(() => 'serviceWorker' in navigator && 'PushManager' in window);
// Find the notification toggle button // Find the notification toggle button
const toggleButton = page.getByRole('button', { name: /enable notifications/i }); const toggleButton = page.getByRole('button', { name: /enable notifications/i });
await expect(toggleButton).toBeVisible(); await expect(toggleButton).toBeVisible();
// Click to enable notifications
await toggleButton.click();
// Wait for subscription to complete
await page.waitForTimeout(2000);
// Verify subscription was created in browser // Click to enable notifications
const subscription = await page.evaluate(async () => { await toggleButton.click();
const registration = await navigator.serviceWorker.ready;
const sub = await registration.pushManager.getSubscription();
return sub ? {
endpoint: sub.endpoint,
hasKeys: !!(sub as any).keys
} : null;
});
expect(subscription).not.toBeNull(); // Wait for subscription to complete
expect(subscription?.endpoint).toBeTruthy(); await page.waitForTimeout(2000);
expect(subscription?.endpoint).toContain('https://');
expect(subscription?.hasKeys).toBe(true);
// Verify button text changed to "Disable Notifications" // Verify subscription was created in browser
await expect(toggleButton).toHaveText(/disable notifications/i); const subscription = await page.evaluate(async () => {
const registration = await navigator.serviceWorker.ready;
await page.close(); const sub = await registration.pushManager.getSubscription();
}); return sub
? {
endpoint: sub.endpoint,
hasKeys: !!(sub as any).keys
}
: null;
});
test('should show test notification buttons when subscribed', async () => { expect(subscription).not.toBeNull();
const page = await context.newPage(); expect(subscription?.endpoint).toBeTruthy();
await page.goto('/'); expect(subscription?.endpoint).toContain('https://');
await page.waitForLoadState('networkidle'); expect(subscription?.hasKeys).toBe(true);
// Wait for service worker // Verify button text changed to "Disable Notifications"
await page.waitForFunction(() => 'serviceWorker' in navigator); await expect(toggleButton).toHaveText(/disable notifications/i);
// Enable notifications first await page.close();
const toggleButton = page.getByRole('button', { name: /enable notifications/i }); });
await toggleButton.click();
await page.waitForTimeout(2000);
// Verify test buttons are visible test('should show test notification buttons when subscribed', async () => {
const testSuccessButton = page.getByRole('button', { name: /test success/i }); const page = await context.newPage();
const testErrorButton = page.getByRole('button', { name: /test error/i }); await page.goto('/');
const testProgressButton = page.getByRole('button', { name: /test progress/i }); await page.waitForLoadState('networkidle');
await expect(testSuccessButton).toBeVisible(); // Wait for service worker
await expect(testErrorButton).toBeVisible(); await page.waitForFunction(() => 'serviceWorker' in navigator);
await expect(testProgressButton).toBeVisible();
await page.close(); // Enable notifications first
}); const toggleButton = page.getByRole('button', { name: /enable notifications/i });
await toggleButton.click();
await page.waitForTimeout(2000);
test('should send test notifications', async () => { // Verify test buttons are visible
const page = await context.newPage(); const testSuccessButton = page.getByRole('button', { name: /test success/i });
await page.goto('/'); const testErrorButton = page.getByRole('button', { name: /test error/i });
await page.waitForLoadState('networkidle'); const testProgressButton = page.getByRole('button', { name: /test progress/i });
// Wait for service worker await expect(testSuccessButton).toBeVisible();
await page.waitForFunction(() => 'serviceWorker' in navigator); await expect(testErrorButton).toBeVisible();
await expect(testProgressButton).toBeVisible();
// Enable notifications first await page.close();
const toggleButton = page.getByRole('button', { name: /enable notifications/i }); });
await toggleButton.click();
await page.waitForTimeout(2000);
// Mock the test notification API response test('should send test notifications', async () => {
await page.route('/api/notifications/test', async (route) => { const page = await context.newPage();
await route.fulfill({ await page.goto('/');
status: 200, await page.waitForLoadState('networkidle');
contentType: 'application/json',
body: JSON.stringify({ success: true, subscriberCount: 1 })
});
});
// Click test success button // Wait for service worker
const testSuccessButton = page.getByRole('button', { name: /test success/i }); await page.waitForFunction(() => 'serviceWorker' in navigator);
await testSuccessButton.click();
// Wait for and verify success message // Enable notifications first
const successMessage = page.getByText(/✓ test success notification sent/i); const toggleButton = page.getByRole('button', { name: /enable notifications/i });
await expect(successMessage).toBeVisible({ timeout: 5000 }); await toggleButton.click();
await page.waitForTimeout(2000);
// Verify message contains subscriber count // Mock the test notification API response
await expect(successMessage).toContainText('1 subscriber'); await page.route('/api/notifications/test', async (route) => {
await route.fulfill({
status: 200,
contentType: 'application/json',
body: JSON.stringify({ success: true, subscriberCount: 1 })
});
});
// Wait for auto-dismiss // Click test success button
await expect(successMessage).not.toBeVisible({ timeout: 4000 }); const testSuccessButton = page.getByRole('button', { name: /test success/i });
await testSuccessButton.click();
await page.close(); // Wait for and verify success message
}); const successMessage = page.getByText(/✓ test success notification sent/i);
await expect(successMessage).toBeVisible({ timeout: 5000 });
test('should unsubscribe from push notifications', async () => { // Verify message contains subscriber count
const page = await context.newPage(); await expect(successMessage).toContainText('1 subscriber');
await page.goto('/');
await page.waitForLoadState('networkidle');
// Wait for service worker // Wait for auto-dismiss
await page.waitForFunction(() => 'serviceWorker' in navigator); await expect(successMessage).not.toBeVisible({ timeout: 4000 });
// First subscribe await page.close();
const toggleButton = page.getByRole('button', { name: /enable notifications/i }); });
await toggleButton.click();
await page.waitForTimeout(2000);
// Verify subscribed test('should unsubscribe from push notifications', async () => {
await expect(toggleButton).toHaveText(/disable notifications/i); const page = await context.newPage();
await page.goto('/');
await page.waitForLoadState('networkidle');
// Now unsubscribe // Wait for service worker
await toggleButton.click(); await page.waitForFunction(() => 'serviceWorker' in navigator);
await page.waitForTimeout(2000);
// Verify subscription was removed // First subscribe
const subscription = await page.evaluate(async () => { const toggleButton = page.getByRole('button', { name: /enable notifications/i });
const registration = await navigator.serviceWorker.ready; await toggleButton.click();
return await registration.pushManager.getSubscription(); await page.waitForTimeout(2000);
});
expect(subscription).toBeNull(); // Verify subscribed
await expect(toggleButton).toHaveText(/disable notifications/i);
// Verify button text changed back // Now unsubscribe
await expect(toggleButton).toHaveText(/enable notifications/i); await toggleButton.click();
await page.waitForTimeout(2000);
// Verify test buttons are no longer visible // Verify subscription was removed
const testSuccessButton = page.getByRole('button', { name: /test success/i }); const subscription = await page.evaluate(async () => {
await expect(testSuccessButton).not.toBeVisible(); const registration = await navigator.serviceWorker.ready;
return await registration.pushManager.getSubscription();
});
await page.close(); expect(subscription).toBeNull();
});
test('should persist clientId in localStorage', async () => { // Verify button text changed back
const page = await context.newPage(); await expect(toggleButton).toHaveText(/enable notifications/i);
await page.goto('/');
await page.waitForLoadState('networkidle');
// Wait for service worker // Verify test buttons are no longer visible
await page.waitForFunction(() => 'serviceWorker' in navigator); const testSuccessButton = page.getByRole('button', { name: /test success/i });
await expect(testSuccessButton).not.toBeVisible();
// Enable notifications await page.close();
const toggleButton = page.getByRole('button', { name: /enable notifications/i }); });
await toggleButton.click();
await page.waitForTimeout(2000);
// Verify clientId is stored in localStorage test('should persist clientId in localStorage', async () => {
const clientId = await page.evaluate(() => { const page = await context.newPage();
return localStorage.getItem('push-client-id'); await page.goto('/');
}); await page.waitForLoadState('networkidle');
expect(clientId).toBeTruthy(); // Wait for service worker
expect(clientId).toMatch(/^client_[a-f0-9-]+$/); await page.waitForFunction(() => 'serviceWorker' in navigator);
// Reload page and verify clientId persists // Enable notifications
await page.reload(); const toggleButton = page.getByRole('button', { name: /enable notifications/i });
await page.waitForLoadState('networkidle'); await toggleButton.click();
await page.waitForTimeout(2000);
const persistedClientId = await page.evaluate(() => { // Verify clientId is stored in localStorage
return localStorage.getItem('push-client-id'); const clientId = await page.evaluate(() => {
}); return localStorage.getItem('push-client-id');
});
expect(persistedClientId).toBe(clientId); expect(clientId).toBeTruthy();
expect(clientId).toMatch(/^client_[a-f0-9-]+$/);
await page.close(); // Reload page and verify clientId persists
}); await page.reload();
await page.waitForLoadState('networkidle');
const persistedClientId = await page.evaluate(() => {
return localStorage.getItem('push-client-id');
});
expect(persistedClientId).toBe(clientId);
await page.close();
});
}); });

1192
src/tests/queue-api.spec.ts
View File

File diff suppressed because it is too large Load Diff

139
src/tests/queue-manager-logging.spec.ts
View File

@@ -1,6 +1,6 @@
/** /**
* Tests for QueueManager logging serialization * Tests for QueueManager logging serialization
* *
* Verifies that QueueManager uses logError utility for error serialization * Verifies that QueueManager uses logError utility for error serialization
* instead of console.error which outputs [object Object]. * instead of console.error which outputs [object Object].
*/ */
@@ -11,98 +11,89 @@ import * as logger from '$lib/server/utils/logger';
import type { QueueUpdateCallback } from '$lib/server/queue/types'; import type { QueueUpdateCallback } from '$lib/server/queue/types';
describe('QueueManager logging', () => { describe('QueueManager logging', () => {
let manager: QueueManager; let manager: QueueManager;
let logErrorSpy: any; let logErrorSpy: any;
beforeEach(() => { beforeEach(() => {
manager = new QueueManager(); manager = new QueueManager();
logErrorSpy = vi.spyOn(logger, 'logError').mockImplementation(() => {}); logErrorSpy = vi.spyOn(logger, 'logError').mockImplementation(() => {});
}); });
test('should use logError when subscriber throws error', () => { test('should use logError when subscriber throws error', () => {
const failingCallback: QueueUpdateCallback = () => { const failingCallback: QueueUpdateCallback = () => {
throw new Error('Subscriber failed'); throw new Error('Subscriber failed');
}; };
manager.subscribe(failingCallback); manager.subscribe(failingCallback);
// Enqueue an item (this will notify subscribers) // Enqueue an item (this will notify subscribers)
manager.enqueue('https://instagram.com/p/test123'); manager.enqueue('https://instagram.com/p/test123');
expect(logErrorSpy).toHaveBeenCalledWith( expect(logErrorSpy).toHaveBeenCalledWith('[QueueManager] Subscriber error', expect.any(Error));
'[QueueManager] Subscriber error', });
expect.any(Error)
);
});
test('should serialize complex error objects', () => { test('should serialize complex error objects', () => {
const complexError = { const complexError = {
code: 'ERR_SUBSCRIBER', code: 'ERR_SUBSCRIBER',
message: 'Callback failed', message: 'Callback failed',
details: { reason: 'Network timeout' } details: { reason: 'Network timeout' }
}; };
const failingCallback: QueueUpdateCallback = () => { const failingCallback: QueueUpdateCallback = () => {
throw complexError; throw complexError;
}; };
manager.subscribe(failingCallback); manager.subscribe(failingCallback);
manager.enqueue('https://instagram.com/p/test456'); manager.enqueue('https://instagram.com/p/test456');
expect(logErrorSpy).toHaveBeenCalledWith( expect(logErrorSpy).toHaveBeenCalledWith('[QueueManager] Subscriber error', complexError);
'[QueueManager] Subscriber error', });
complexError
);
});
test('should not prevent other subscribers from being notified on error', () => { test('should not prevent other subscribers from being notified on error', () => {
const consoleErrorSpy = vi.spyOn(console, 'error').mockImplementation(() => {}); const consoleErrorSpy = vi.spyOn(console, 'error').mockImplementation(() => {});
const failingCallback: QueueUpdateCallback = () => { const failingCallback: QueueUpdateCallback = () => {
throw new Error('First subscriber fails'); throw new Error('First subscriber fails');
}; };
const successCallback = vi.fn(); const successCallback = vi.fn();
manager.subscribe(failingCallback); manager.subscribe(failingCallback);
manager.subscribe(successCallback); manager.subscribe(successCallback);
manager.enqueue('https://instagram.com/p/test789'); manager.enqueue('https://instagram.com/p/test789');
// Error should be logged via logError // Error should be logged via logError
expect(logErrorSpy).toHaveBeenCalled(); expect(logErrorSpy).toHaveBeenCalled();
// Second subscriber should still be called // Second subscriber should still be called
expect(successCallback).toHaveBeenCalled(); expect(successCallback).toHaveBeenCalled();
// Should not contain [object Object] in console output // Should not contain [object Object] in console output
const errorMessages = consoleErrorSpy.mock.calls const errorMessages = consoleErrorSpy.mock.calls.map((call) => call.join(' '));
.map(call => call.join(' '));
const hasObjectObject = errorMessages.some(msg => const hasObjectObject = errorMessages.some((msg) => msg.includes('[object Object]'));
msg.includes('[object Object]')
);
expect(hasObjectObject).toBe(false); expect(hasObjectObject).toBe(false);
}); });
test('should handle Error instances with custom properties', () => { test('should handle Error instances with custom properties', () => {
const customError: any = new Error('Custom error'); const customError: any = new Error('Custom error');
customError.statusCode = 500; customError.statusCode = 500;
customError.details = { field: 'url', issue: 'invalid' }; customError.details = { field: 'url', issue: 'invalid' };
const failingCallback: QueueUpdateCallback = () => { const failingCallback: QueueUpdateCallback = () => {
throw customError; throw customError;
}; };
manager.subscribe(failingCallback); manager.subscribe(failingCallback);
manager.enqueue('https://instagram.com/p/custom'); manager.enqueue('https://instagram.com/p/custom');
expect(logErrorSpy).toHaveBeenCalledWith( expect(logErrorSpy).toHaveBeenCalledWith(
'[QueueManager] Subscriber error', '[QueueManager] Subscriber error',
expect.objectContaining({ expect.objectContaining({
message: 'Custom error', message: 'Custom error',
statusCode: 500, statusCode: 500,
details: { field: 'url', issue: 'invalid' } details: { field: 'url', issue: 'invalid' }
}) })
); );
}); });
}); });

692
src/tests/queue-manager.spec.ts
View File

@@ -1,6 +1,6 @@
/** /**
* Unit tests for QueueManager * Unit tests for QueueManager
* *
* Tests core queue operations, status management, and pub/sub functionality. * Tests core queue operations, status management, and pub/sub functionality.
*/ */
@@ -8,349 +8,349 @@ import { describe, it, expect, beforeEach, vi } from 'vitest';
import { QueueManager } from '$lib/server/queue/QueueManager'; import { QueueManager } from '$lib/server/queue/QueueManager';
describe('QueueManager', () => { describe('QueueManager', () => {
let queueManager: QueueManager; let queueManager: QueueManager;
beforeEach(() => { beforeEach(() => {
// Create fresh instance for each test // Create fresh instance for each test
queueManager = new QueueManager(); queueManager = new QueueManager();
}); });
describe('enqueue', () => { describe('enqueue', () => {
it('should enqueue items with unique IDs', () => { it('should enqueue items with unique IDs', () => {
const item1 = queueManager.enqueue('https://instagram.com/p/test1'); const item1 = queueManager.enqueue('https://instagram.com/p/test1');
const item2 = queueManager.enqueue('https://instagram.com/p/test2'); const item2 = queueManager.enqueue('https://instagram.com/p/test2');
expect(item1.id).toBeTruthy(); expect(item1.id).toBeTruthy();
expect(item2.id).toBeTruthy(); expect(item2.id).toBeTruthy();
expect(item1.id).not.toBe(item2.id); expect(item1.id).not.toBe(item2.id);
}); });
it('should create items with pending status', () => { it('should create items with pending status', () => {
const item = queueManager.enqueue('https://instagram.com/p/test'); const item = queueManager.enqueue('https://instagram.com/p/test');
expect(item.status).toBe('pending'); expect(item.status).toBe('pending');
expect(item.enqueuedAt).toBeTruthy(); expect(item.enqueuedAt).toBeTruthy();
expect(item.logs).toEqual([]); expect(item.logs).toEqual([]);
expect(item.progressEvents).toEqual([]); expect(item.progressEvents).toEqual([]);
expect(item.retryCount).toBe(0); expect(item.retryCount).toBe(0);
expect(item.maxRetries).toBe(3); expect(item.maxRetries).toBe(3);
}); });
it('should notify subscribers when enqueueing', () => { it('should notify subscribers when enqueueing', () => {
const callback = vi.fn(); const callback = vi.fn();
queueManager.subscribe(callback); queueManager.subscribe(callback);
const item = queueManager.enqueue('https://instagram.com/p/test'); const item = queueManager.enqueue('https://instagram.com/p/test');
expect(callback).toHaveBeenCalledWith( expect(callback).toHaveBeenCalledWith(
expect.objectContaining({ expect.objectContaining({
itemId: item.id, itemId: item.id,
status: 'pending' status: 'pending'
}) })
); );
}); });
}); });
describe('dequeue', () => { describe('dequeue', () => {
it('should dequeue oldest pending item first (FIFO)', () => { it('should dequeue oldest pending item first (FIFO)', () => {
const item1 = queueManager.enqueue('https://instagram.com/p/test1'); const item1 = queueManager.enqueue('https://instagram.com/p/test1');
const item2 = queueManager.enqueue('https://instagram.com/p/test2'); const item2 = queueManager.enqueue('https://instagram.com/p/test2');
const dequeued1 = queueManager.dequeue(); const dequeued1 = queueManager.dequeue();
expect(dequeued1?.id).toBe(item1.id); expect(dequeued1?.id).toBe(item1.id);
const dequeued2 = queueManager.dequeue(); const dequeued2 = queueManager.dequeue();
expect(dequeued2?.id).toBe(item2.id); expect(dequeued2?.id).toBe(item2.id);
}); });
it('should return null when queue is empty', () => { it('should return null when queue is empty', () => {
const item = queueManager.dequeue(); const item = queueManager.dequeue();
expect(item).toBeNull(); expect(item).toBeNull();
}); });
it('should mark dequeued item as in_progress', () => { it('should mark dequeued item as in_progress', () => {
const enqueuedItem = queueManager.enqueue('https://instagram.com/p/test'); const enqueuedItem = queueManager.enqueue('https://instagram.com/p/test');
const dequeuedItem = queueManager.dequeue(); const dequeuedItem = queueManager.dequeue();
expect(dequeuedItem?.status).toBe('in_progress'); expect(dequeuedItem?.status).toBe('in_progress');
expect(dequeuedItem?.currentPhase).toBe('extraction'); expect(dequeuedItem?.currentPhase).toBe('extraction');
expect(dequeuedItem?.startedAt).toBeTruthy(); expect(dequeuedItem?.startedAt).toBeTruthy();
}); });
it('should skip non-pending items', () => { it('should skip non-pending items', () => {
const item1 = queueManager.enqueue('https://instagram.com/p/test1'); const item1 = queueManager.enqueue('https://instagram.com/p/test1');
const item2 = queueManager.enqueue('https://instagram.com/p/test2'); const item2 = queueManager.enqueue('https://instagram.com/p/test2');
// Dequeue first item // Dequeue first item
queueManager.dequeue(); queueManager.dequeue();
// Second item should be next // Second item should be next
const dequeued = queueManager.dequeue(); const dequeued = queueManager.dequeue();
expect(dequeued?.id).toBe(item2.id); expect(dequeued?.id).toBe(item2.id);
}); });
}); });
describe('updateStatus', () => { describe('updateStatus', () => {
it('should update item status', () => { it('should update item status', () => {
const item = queueManager.enqueue('https://instagram.com/p/test'); const item = queueManager.enqueue('https://instagram.com/p/test');
queueManager.updateStatus(item.id, 'in_progress', { phase: 'parsing' }); queueManager.updateStatus(item.id, 'in_progress', { phase: 'parsing' });
const updated = queueManager.get(item.id); const updated = queueManager.get(item.id);
expect(updated?.status).toBe('in_progress'); expect(updated?.status).toBe('in_progress');
expect(updated?.currentPhase).toBe('parsing'); expect(updated?.currentPhase).toBe('parsing');
}); });
it('should set completedAt for terminal statuses', () => { it('should set completedAt for terminal statuses', () => {
const item = queueManager.enqueue('https://instagram.com/p/test'); const item = queueManager.enqueue('https://instagram.com/p/test');
queueManager.updateStatus(item.id, 'success'); queueManager.updateStatus(item.id, 'success');
const updated = queueManager.get(item.id); const updated = queueManager.get(item.id);
expect(updated?.completedAt).toBeTruthy(); expect(updated?.completedAt).toBeTruthy();
}); });
it('should merge additional data into item', () => { it('should merge additional data into item', () => {
const item = queueManager.enqueue('https://instagram.com/p/test'); const item = queueManager.enqueue('https://instagram.com/p/test');
queueManager.updateStatus(item.id, 'success', { queueManager.updateStatus(item.id, 'success', {
recipe: { name: 'Test Recipe' }, recipe: { name: 'Test Recipe' },
tandoorRecipeId: 123 tandoorRecipeId: 123
}); });
const updated = queueManager.get(item.id); const updated = queueManager.get(item.id);
expect(updated?.recipe).toEqual({ name: 'Test Recipe' }); expect(updated?.recipe).toEqual({ name: 'Test Recipe' });
expect(updated?.tandoorRecipeId).toBe(123); expect(updated?.tandoorRecipeId).toBe(123);
}); });
it('should handle error data', () => { it('should handle error data', () => {
const item = queueManager.enqueue('https://instagram.com/p/test'); const item = queueManager.enqueue('https://instagram.com/p/test');
const errorData = { const errorData = {
error: { error: {
phase: 'extraction' as const, phase: 'extraction' as const,
message: 'Failed to load page', message: 'Failed to load page',
recoverable: true, recoverable: true,
timestamp: new Date().toISOString() timestamp: new Date().toISOString()
} }
}; };
queueManager.updateStatus(item.id, 'unhealthy', errorData); queueManager.updateStatus(item.id, 'unhealthy', errorData);
const updated = queueManager.get(item.id); const updated = queueManager.get(item.id);
expect(updated?.error).toEqual(errorData.error); expect(updated?.error).toEqual(errorData.error);
}); });
}); });
describe('addProgressEvent', () => { describe('addProgressEvent', () => {
it('should add progress events to item', () => { it('should add progress events to item', () => {
const item = queueManager.enqueue('https://instagram.com/p/test'); const item = queueManager.enqueue('https://instagram.com/p/test');
const event = { const event = {
type: 'status', type: 'status',
message: 'Extracting...', message: 'Extracting...',
timestamp: new Date().toISOString() timestamp: new Date().toISOString()
}; };
queueManager.addProgressEvent(item.id, event); queueManager.addProgressEvent(item.id, event);
const updated = queueManager.get(item.id); const updated = queueManager.get(item.id);
expect(updated?.progressEvents).toHaveLength(1); expect(updated?.progressEvents).toHaveLength(1);
expect(updated?.progressEvents[0]).toEqual(event); expect(updated?.progressEvents[0]).toEqual(event);
}); });
it('should add event message to logs', () => { it('should add event message to logs', () => {
const item = queueManager.enqueue('https://instagram.com/p/test'); const item = queueManager.enqueue('https://instagram.com/p/test');
queueManager.addProgressEvent(item.id, { queueManager.addProgressEvent(item.id, {
type: 'status', type: 'status',
message: 'Test message', message: 'Test message',
timestamp: new Date().toISOString() timestamp: new Date().toISOString()
}); });
const updated = queueManager.get(item.id); const updated = queueManager.get(item.id);
expect(updated?.logs).toContain('Test message'); expect(updated?.logs).toContain('Test message');
}); });
it('should notify subscribers with event data', () => { it('should notify subscribers with event data', () => {
const callback = vi.fn(); const callback = vi.fn();
queueManager.subscribe(callback); queueManager.subscribe(callback);
const item = queueManager.enqueue('https://instagram.com/p/test'); const item = queueManager.enqueue('https://instagram.com/p/test');
callback.mockClear(); // Clear enqueue notification callback.mockClear(); // Clear enqueue notification
const event = { type: 'status', message: 'Test', timestamp: new Date().toISOString() }; const event = { type: 'status', message: 'Test', timestamp: new Date().toISOString() };
queueManager.addProgressEvent(item.id, event); queueManager.addProgressEvent(item.id, event);
expect(callback).toHaveBeenCalledWith( expect(callback).toHaveBeenCalledWith(
expect.objectContaining({ expect.objectContaining({
itemId: item.id, itemId: item.id,
data: { event } data: { event }
}) })
); );
}); });
}); });
describe('remove', () => { describe('remove', () => {
it('should remove items by ID', () => { it('should remove items by ID', () => {
const item = queueManager.enqueue('https://instagram.com/p/test'); const item = queueManager.enqueue('https://instagram.com/p/test');
const removed = queueManager.remove(item.id); const removed = queueManager.remove(item.id);
expect(removed).toBe(true); expect(removed).toBe(true);
expect(queueManager.get(item.id)).toBeUndefined(); expect(queueManager.get(item.id)).toBeUndefined();
}); });
it('should return false for non-existent items', () => { it('should return false for non-existent items', () => {
const removed = queueManager.remove('non-existent-id'); const removed = queueManager.remove('non-existent-id');
expect(removed).toBe(false); expect(removed).toBe(false);
}); });
it('should notify subscribers when removing', () => { it('should notify subscribers when removing', () => {
const callback = vi.fn(); const callback = vi.fn();
queueManager.subscribe(callback); queueManager.subscribe(callback);
const item = queueManager.enqueue('https://instagram.com/p/test'); const item = queueManager.enqueue('https://instagram.com/p/test');
callback.mockClear(); callback.mockClear();
queueManager.remove(item.id); queueManager.remove(item.id);
expect(callback).toHaveBeenCalledWith( expect(callback).toHaveBeenCalledWith(
expect.objectContaining({ expect.objectContaining({
itemId: item.id, itemId: item.id,
data: { removed: true } data: { removed: true }
}) })
); );
}); });
}); });
describe('retry', () => { describe('retry', () => {
it('should retry failed items', () => { it('should retry failed items', () => {
const item = queueManager.enqueue('https://instagram.com/p/test'); const item = queueManager.enqueue('https://instagram.com/p/test');
queueManager.updateStatus(item.id, 'error'); queueManager.updateStatus(item.id, 'error');
const retried = queueManager.retry(item.id); const retried = queueManager.retry(item.id);
expect(retried).toBe(true); expect(retried).toBe(true);
const updated = queueManager.get(item.id); const updated = queueManager.get(item.id);
expect(updated?.status).toBe('pending'); expect(updated?.status).toBe('pending');
expect(updated?.retryCount).toBe(1); expect(updated?.retryCount).toBe(1);
expect(updated?.error).toBeUndefined(); expect(updated?.error).toBeUndefined();
expect(updated?.currentPhase).toBeUndefined(); expect(updated?.currentPhase).toBeUndefined();
}); });
it('should not retry items in progress', () => { it('should not retry items in progress', () => {
const item = queueManager.enqueue('https://instagram.com/p/test'); const item = queueManager.enqueue('https://instagram.com/p/test');
queueManager.updateStatus(item.id, 'in_progress'); queueManager.updateStatus(item.id, 'in_progress');
const retried = queueManager.retry(item.id); const retried = queueManager.retry(item.id);
expect(retried).toBe(false); expect(retried).toBe(false);
expect(queueManager.get(item.id)?.status).toBe('in_progress'); expect(queueManager.get(item.id)?.status).toBe('in_progress');
}); });
it('should increment retry count', () => { it('should increment retry count', () => {
const item = queueManager.enqueue('https://instagram.com/p/test'); const item = queueManager.enqueue('https://instagram.com/p/test');
queueManager.updateStatus(item.id, 'error'); queueManager.updateStatus(item.id, 'error');
queueManager.retry(item.id); queueManager.retry(item.id);
queueManager.retry(item.id); queueManager.retry(item.id);
expect(queueManager.get(item.id)?.retryCount).toBe(2); expect(queueManager.get(item.id)?.retryCount).toBe(2);
}); });
}); });
describe('getAll', () => { describe('getAll', () => {
it('should return all queue items', () => { it('should return all queue items', () => {
queueManager.enqueue('https://instagram.com/p/test1'); queueManager.enqueue('https://instagram.com/p/test1');
queueManager.enqueue('https://instagram.com/p/test2'); queueManager.enqueue('https://instagram.com/p/test2');
queueManager.enqueue('https://instagram.com/p/test3'); queueManager.enqueue('https://instagram.com/p/test3');
const items = queueManager.getAll(); const items = queueManager.getAll();
expect(items).toHaveLength(3); expect(items).toHaveLength(3);
}); });
it('should return empty array when queue is empty', () => { it('should return empty array when queue is empty', () => {
const items = queueManager.getAll(); const items = queueManager.getAll();
expect(items).toEqual([]); expect(items).toEqual([]);
}); });
}); });
describe('get', () => { describe('get', () => {
it('should return item by ID', () => { it('should return item by ID', () => {
const item = queueManager.enqueue('https://instagram.com/p/test'); const item = queueManager.enqueue('https://instagram.com/p/test');
const retrieved = queueManager.get(item.id); const retrieved = queueManager.get(item.id);
expect(retrieved?.id).toBe(item.id); expect(retrieved?.id).toBe(item.id);
expect(retrieved?.url).toBe(item.url); expect(retrieved?.url).toBe(item.url);
}); });
it('should return undefined for non-existent ID', () => { it('should return undefined for non-existent ID', () => {
const item = queueManager.get('non-existent-id'); const item = queueManager.get('non-existent-id');
expect(item).toBeUndefined(); expect(item).toBeUndefined();
}); });
}); });
describe('subscribe', () => { describe('subscribe', () => {
it('should notify subscribers of updates', () => { it('should notify subscribers of updates', () => {
const callback = vi.fn(); const callback = vi.fn();
queueManager.subscribe(callback); queueManager.subscribe(callback);
queueManager.enqueue('https://instagram.com/p/test'); queueManager.enqueue('https://instagram.com/p/test');
expect(callback).toHaveBeenCalled(); expect(callback).toHaveBeenCalled();
}); });
it('should return unsubscribe function', () => { it('should return unsubscribe function', () => {
const callback = vi.fn(); const callback = vi.fn();
const unsubscribe = queueManager.subscribe(callback); const unsubscribe = queueManager.subscribe(callback);
queueManager.enqueue('https://instagram.com/p/test1'); queueManager.enqueue('https://instagram.com/p/test1');
expect(callback).toHaveBeenCalledTimes(1); expect(callback).toHaveBeenCalledTimes(1);
unsubscribe(); unsubscribe();
callback.mockClear(); callback.mockClear();
queueManager.enqueue('https://instagram.com/p/test2'); queueManager.enqueue('https://instagram.com/p/test2');
expect(callback).not.toHaveBeenCalled(); expect(callback).not.toHaveBeenCalled();
}); });
it('should handle subscriber errors gracefully', () => { it('should handle subscriber errors gracefully', () => {
const goodCallback = vi.fn(); const goodCallback = vi.fn();
const badCallback = vi.fn(() => { const badCallback = vi.fn(() => {
throw new Error('Subscriber error'); throw new Error('Subscriber error');
}); });
queueManager.subscribe(goodCallback); queueManager.subscribe(goodCallback);
queueManager.subscribe(badCallback); queueManager.subscribe(badCallback);
// Should not throw despite bad callback // Should not throw despite bad callback
expect(() => { expect(() => {
queueManager.enqueue('https://instagram.com/p/test'); queueManager.enqueue('https://instagram.com/p/test');
}).not.toThrow(); }).not.toThrow();
// Good callback should still be called // Good callback should still be called
expect(goodCallback).toHaveBeenCalled(); expect(goodCallback).toHaveBeenCalled();
}); });
it('should support multiple subscribers', () => { it('should support multiple subscribers', () => {
const callback1 = vi.fn(); const callback1 = vi.fn();
const callback2 = vi.fn(); const callback2 = vi.fn();
const callback3 = vi.fn(); const callback3 = vi.fn();
queueManager.subscribe(callback1); queueManager.subscribe(callback1);
queueManager.subscribe(callback2); queueManager.subscribe(callback2);
queueManager.subscribe(callback3); queueManager.subscribe(callback3);
queueManager.enqueue('https://instagram.com/p/test'); queueManager.enqueue('https://instagram.com/p/test');
expect(callback1).toHaveBeenCalled(); expect(callback1).toHaveBeenCalled();
expect(callback2).toHaveBeenCalled(); expect(callback2).toHaveBeenCalled();
expect(callback3).toHaveBeenCalled(); expect(callback3).toHaveBeenCalled();
}); });
}); });
}); });

156
src/tests/queue-processor-logging.spec.ts
View File

@@ -2,19 +2,19 @@ import { describe, test, expect, vi, beforeEach, afterEach } from 'vitest';
// Mock parser to avoid LLM calls // Mock parser to avoid LLM calls
vi.mock('$lib/server/parser', () => ({ vi.mock('$lib/server/parser', () => ({
extractRecipe: vi.fn().mockResolvedValue({ extractRecipe: vi.fn().mockResolvedValue({
name: 'Test Recipe', name: 'Test Recipe',
ingredients: [], ingredients: [],
instructions: 'Test instructions', instructions: 'Test instructions',
servings: 4 servings: 4
}), }),
detectRecipe: vi.fn().mockResolvedValue(true) detectRecipe: vi.fn().mockResolvedValue(true)
})); }));
// Mock tandoor to avoid API calls // Mock tandoor to avoid API calls
vi.mock('$lib/server/tandoor', () => ({ vi.mock('$lib/server/tandoor', () => ({
uploadRecipeWithIngredientsDTO: vi.fn().mockResolvedValue({ id: 123 }), uploadRecipeWithIngredientsDTO: vi.fn().mockResolvedValue({ id: 123 }),
uploadRecipeImage: vi.fn().mockResolvedValue(true) uploadRecipeImage: vi.fn().mockResolvedValue(true)
})); }));
import { queueManager } from '$lib/server/queue/QueueManager'; import { queueManager } from '$lib/server/queue/QueueManager';
@@ -22,72 +22,74 @@ import * as extraction from '$lib/server/extraction';
import { queueProcessor } from '$lib/server/queue/QueueProcessor'; import { queueProcessor } from '$lib/server/queue/QueueProcessor';
describe('QueueProcessor logging', () => { describe('QueueProcessor logging', () => {
let consoleErrorSpy: any;
let consoleErrorSpy: any;
beforeEach(async () => {
beforeEach(async () => { // Stop processor first
// Stop processor first queueProcessor.stop();
queueProcessor.stop();
// Clear queue
// Clear queue const items = queueManager.getAll();
const items = queueManager.getAll(); items.forEach((item) => queueManager.remove(item.id));
items.forEach(item => queueManager.remove(item.id));
// Setup console.error spy
// Setup console.error spy consoleErrorSpy = vi.spyOn(console, 'error').mockImplementation(() => {});
consoleErrorSpy = vi.spyOn(console, 'error').mockImplementation(() => {});
// Give time for cleanup
// Give time for cleanup await new Promise((resolve) => setTimeout(resolve, 100));
await new Promise(resolve => setTimeout(resolve, 100)); });
});
afterEach(() => {
afterEach(() => { queueProcessor.stop();
queueProcessor.stop(); consoleErrorSpy.mockRestore();
consoleErrorSpy.mockRestore(); });
});
test('error logs should be properly serialized (no [object Object])', async () => {
test('error logs should be properly serialized (no [object Object])', async () => { // Create complex error object
// Create complex error object const complexError = new Error('Test extraction error');
const complexError = new Error('Test extraction error'); (complexError as any).code = 'ERR_TEST';
(complexError as any).code = 'ERR_TEST'; (complexError as any).details = { phase: 'extraction', retries: 3 };
(complexError as any).details = { phase: 'extraction', retries: 3 };
// Mock extraction to fail BEFORE starting processor
// Mock extraction to fail BEFORE starting processor const extractSpy = vi.spyOn(extraction, 'extractTextAndThumbnail');
const extractSpy = vi.spyOn(extraction, 'extractTextAndThumbnail'); extractSpy.mockRejectedValueOnce(complexError);
extractSpy.mockRejectedValueOnce(complexError);
const item = queueManager.enqueue('https://instagram.com/p/TEST');
const item = queueManager.enqueue('https://instagram.com/p/TEST'); queueProcessor.start();
queueProcessor.start();
// Wait for error status
// Wait for error status await vi.waitFor(
await vi.waitFor(() => { () => {
const updated = queueManager.get(item.id); const updated = queueManager.get(item.id);
return updated?.status === 'error' || updated?.status === 'unhealthy'; return updated?.status === 'error' || updated?.status === 'unhealthy';
}, { timeout: 5000 }); },
{ timeout: 5000 }
// Stop processor );
queueProcessor.stop();
// Stop processor
// Wait a bit for all logs to finish queueProcessor.stop();
await new Promise(resolve => setTimeout(resolve, 100));
// Wait a bit for all logs to finish
// Check that console.error doesn't contain [object Object] await new Promise((resolve) => setTimeout(resolve, 100));
const allCalls = consoleErrorSpy.mock.calls.map((call: any[]) =>
call.map(arg => { // Check that console.error doesn't contain [object Object]
if (arg && typeof arg === 'object' && arg.message) { const allCalls = consoleErrorSpy.mock.calls.map((call: any[]) =>
return arg.message; // Handle Error objects call
} .map((arg) => {
return String(arg); if (arg && typeof arg === 'object' && arg.message) {
}).join(' ') return arg.message; // Handle Error objects
); }
return String(arg);
const hasObjectObject = allCalls.some((msg: string) => msg.includes('[object Object]')); })
expect(hasObjectObject).toBe(false); .join(' ')
);
// Verify QueueProcessor logs are present
const queueProcessorLogs = allCalls.filter((msg: string) => const hasObjectObject = allCalls.some((msg: string) => msg.includes('[object Object]'));
msg.includes('[QueueProcessor]') expect(hasObjectObject).toBe(false);
);
// Verify QueueProcessor logs are present
expect(queueProcessorLogs.length).toBeGreaterThan(0); const queueProcessorLogs = allCalls.filter((msg: string) => msg.includes('[QueueProcessor]'));
});
expect(queueProcessorLogs.length).toBeGreaterThan(0);
});
}); });

457
src/tests/queue-processor.spec.ts
View File

@@ -1,6 +1,6 @@
/** /**
* Integration tests for QueueProcessor * Integration tests for QueueProcessor
* *
* Tests the processor's ability to handle queue items through mocked dependencies. * Tests the processor's ability to handle queue items through mocked dependencies.
* The QueueProcessor auto-starts, so these tests verify actual processing behavior. * The QueueProcessor auto-starts, so these tests verify actual processing behavior.
*/ */
@@ -10,55 +10,56 @@ import { queueManager } from '$lib/server/queue/QueueManager';
// Mock web-push module BEFORE importing modules that depend on it // Mock web-push module BEFORE importing modules that depend on it
vi.mock('web-push', () => ({ vi.mock('web-push', () => ({
default: { default: {
setVapidDetails: vi.fn(), setVapidDetails: vi.fn(),
sendNotification: vi.fn().mockResolvedValue(undefined) sendNotification: vi.fn().mockResolvedValue(undefined)
} }
})); }));
// Mock queueConfig BEFORE importing QueueProcessor // Mock queueConfig BEFORE importing QueueProcessor
vi.mock('$lib/server/queue/config', () => ({ vi.mock('$lib/server/queue/config', () => ({
queueConfig: { queueConfig: {
concurrency: 2, concurrency: 2,
maxRetries: 3, maxRetries: 3,
tandoor: { tandoor: {
enabled: true, enabled: true,
token: 'test-token', token: 'test-token',
serverUrl: 'http://localhost:8080' serverUrl: 'http://localhost:8080'
}, },
push: { push: {
vapidPublicKey: 'BNextdcB_fQ0BVvyGioM5L8Tf9vKQjs-WnF-rUbnU8MdWIZQYfggIHxBnW21I-lq_0HykLCdMpYj8d5joavWdxQ', vapidPublicKey:
vapidPrivateKey: 'JwxI_KcsBcehYcTOufMcbVWJjCq1QbH5FJmSyQuG680', 'BNextdcB_fQ0BVvyGioM5L8Tf9vKQjs-WnF-rUbnU8MdWIZQYfggIHxBnW21I-lq_0HykLCdMpYj8d5joavWdxQ',
vapidEmail: 'mailto:test@example.com' vapidPrivateKey: 'JwxI_KcsBcehYcTOufMcbVWJjCq1QbH5FJmSyQuG680',
} vapidEmail: 'mailto:test@example.com'
} }
}
})); }));
// Mock external dependencies BEFORE importing QueueProcessor // Mock external dependencies BEFORE importing QueueProcessor
vi.mock('$lib/server/extraction', () => ({ vi.mock('$lib/server/extraction', () => ({
extractTextAndThumbnail: vi.fn().mockResolvedValue({ extractTextAndThumbnail: vi.fn().mockResolvedValue({
bodyText: 'Default recipe text', bodyText: 'Default recipe text',
thumbnail: null thumbnail: null
}) })
})); }));
vi.mock('$lib/server/parser', () => ({ vi.mock('$lib/server/parser', () => ({
extractRecipe: vi.fn().mockResolvedValue({ extractRecipe: vi.fn().mockResolvedValue({
name: 'Default Recipe', name: 'Default Recipe',
ingredients: ['ingredient 1'], ingredients: ['ingredient 1'],
steps: ['step 1'], steps: ['step 1'],
description: 'A default recipe' description: 'A default recipe'
}) })
})); }));
vi.mock('$lib/server/tandoor', () => ({ vi.mock('$lib/server/tandoor', () => ({
uploadRecipeWithIngredientsDTO: vi.fn().mockResolvedValue({ uploadRecipeWithIngredientsDTO: vi.fn().mockResolvedValue({
success: true, success: true,
recipeId: 999 recipeId: 999
}), }),
uploadRecipeImage: vi.fn().mockResolvedValue({ uploadRecipeImage: vi.fn().mockResolvedValue({
success: true success: true
}) })
})); }));
import { extractTextAndThumbnail } from '$lib/server/extraction'; import { extractTextAndThumbnail } from '$lib/server/extraction';
@@ -70,197 +71,195 @@ import * as configModule from '$lib/server/queue/config';
import '$lib/server/queue/QueueProcessor'; import '$lib/server/queue/QueueProcessor';
describe('QueueProcessor Integration Tests', () => { describe('QueueProcessor Integration Tests', () => {
beforeEach(async () => { beforeEach(async () => {
// Clear queue // Clear queue
queueManager.getAll().forEach(item => queueManager.remove(item.id)); queueManager.getAll().forEach((item) => queueManager.remove(item.id));
// Reset mocks and their implementations // Reset mocks and their implementations
vi.resetAllMocks(); vi.resetAllMocks();
// Set default mock implementations // Set default mock implementations
vi.mocked(extractTextAndThumbnail).mockResolvedValue({ vi.mocked(extractTextAndThumbnail).mockResolvedValue({
bodyText: 'Default recipe text', bodyText: 'Default recipe text',
thumbnail: null thumbnail: null
}); });
vi.mocked(extractRecipe).mockResolvedValue({ vi.mocked(extractRecipe).mockResolvedValue({
name: 'Default Recipe', name: 'Default Recipe',
servings: 2, servings: 2,
ingredients: [{ item: 'ingredient 1', amount: '1', unit: 'piece' }], ingredients: [{ item: 'ingredient 1', amount: '1', unit: 'piece' }],
steps: ['step 1'], steps: ['step 1'],
description: 'A default recipe' description: 'A default recipe'
}); });
vi.mocked(uploadRecipeWithIngredientsDTO).mockResolvedValue({ vi.mocked(uploadRecipeWithIngredientsDTO).mockResolvedValue({
success: true, success: true,
recipeId: 999 recipeId: 999
}); });
vi.mocked(uploadRecipeImage).mockResolvedValue({ vi.mocked(uploadRecipeImage).mockResolvedValue({
success: true success: true
}); });
}); });
afterEach(async () => { afterEach(async () => {
// Wait for any pending processing to complete // Wait for any pending processing to complete
await new Promise((resolve) => setTimeout(resolve, 100)); await new Promise((resolve) => setTimeout(resolve, 100));
}); });
it('should process item through all phases when Tandoor is configured', async () => { it('should process item through all phases when Tandoor is configured', async () => {
// Set up successful mocks // Set up successful mocks
vi.mocked(extractTextAndThumbnail).mockResolvedValue({ vi.mocked(extractTextAndThumbnail).mockResolvedValue({
bodyText: 'Recipe instructions here', bodyText: 'Recipe instructions here',
thumbnail: 'https://example.com/thumb.jpg' thumbnail: 'https://example.com/thumb.jpg'
}); });
vi.mocked(extractRecipe).mockResolvedValue({ vi.mocked(extractRecipe).mockResolvedValue({
name: 'Test Recipe', name: 'Test Recipe',
servings: 4, servings: 4,
ingredients: [ ingredients: [
{ item: 'flour', amount: '2', unit: 'cups' }, { item: 'flour', amount: '2', unit: 'cups' },
{ item: 'eggs', amount: '2', unit: 'pieces' } { item: 'eggs', amount: '2', unit: 'pieces' }
], ],
steps: ['mix', 'bake'], steps: ['mix', 'bake'],
description: 'test' description: 'test'
}); });
vi.mocked(uploadRecipeWithIngredientsDTO).mockResolvedValue({ vi.mocked(uploadRecipeWithIngredientsDTO).mockResolvedValue({
success: true, success: true,
recipeId: 123 recipeId: 123
}); });
// Enqueue (processor is already running from auto-start) // Enqueue (processor is already running from auto-start)
// Note: Tandoor is enabled in the mocked config // Note: Tandoor is enabled in the mocked config
const item = queueManager.enqueue('https://instagram.com/p/test-tandoor'); const item = queueManager.enqueue('https://instagram.com/p/test-tandoor');
// Wait for processing to complete - increased timeout // Wait for processing to complete - increased timeout
await new Promise((resolve) => setTimeout(resolve, 1000)); await new Promise((resolve) => setTimeout(resolve, 1000));
const updated = queueManager.get(item.id); const updated = queueManager.get(item.id);
// Verify success // Verify success
expect(updated?.status).toBe('success'); expect(updated?.status).toBe('success');
expect(updated?.extractedText).toBe('Recipe instructions here'); expect(updated?.extractedText).toBe('Recipe instructions here');
expect(updated?.recipe?.name).toBe('Test Recipe'); expect(updated?.recipe?.name).toBe('Test Recipe');
expect(updated?.tandoorRecipeId).toBe(123); expect(updated?.tandoorRecipeId).toBe(123);
// Verify all functions were called // Verify all functions were called
expect(extractTextAndThumbnail).toHaveBeenCalled(); expect(extractTextAndThumbnail).toHaveBeenCalled();
expect(extractRecipe).toHaveBeenCalled(); expect(extractRecipe).toHaveBeenCalled();
expect(uploadRecipeWithIngredientsDTO).toHaveBeenCalled(); expect(uploadRecipeWithIngredientsDTO).toHaveBeenCalled();
}, 10000); // Increase timeout for processing }, 10000); // Increase timeout for processing
it('should skip Tandoor upload when not configured', async () => { it('should skip Tandoor upload when not configured', async () => {
// Temporarily disable Tandoor for this test // Temporarily disable Tandoor for this test
const originalConfig = { ...configModule.queueConfig }; const originalConfig = { ...configModule.queueConfig };
vi.spyOn(configModule, 'queueConfig', 'get').mockReturnValue({ vi.spyOn(configModule, 'queueConfig', 'get').mockReturnValue({
...originalConfig, ...originalConfig,
tandoor: { tandoor: {
enabled: false, enabled: false,
token: null, token: null,
serverUrl: null serverUrl: null
} }
}); });
vi.mocked(extractTextAndThumbnail).mockResolvedValue({ vi.mocked(extractTextAndThumbnail).mockResolvedValue({
bodyText: 'Recipe text', bodyText: 'Recipe text',
thumbnail: null thumbnail: null
}); });
vi.mocked(extractRecipe).mockResolvedValue({ vi.mocked(extractRecipe).mockResolvedValue({
name: 'No Tandoor Recipe', name: 'No Tandoor Recipe',
servings: null, servings: null,
ingredients: [], ingredients: [],
steps: [], steps: [],
description: '' description: ''
}); });
const item = queueManager.enqueue('https://instagram.com/p/no-tandoor'); const item = queueManager.enqueue('https://instagram.com/p/no-tandoor');
await new Promise((resolve) => setTimeout(resolve, 800)); await new Promise((resolve) => setTimeout(resolve, 800));
const updated = queueManager.get(item.id); const updated = queueManager.get(item.id);
// Should still succeed without Tandoor // Should still succeed without Tandoor
expect(updated?.status).toBe('success'); expect(updated?.status).toBe('success');
expect(updated?.recipe?.name).toBe('No Tandoor Recipe'); expect(updated?.recipe?.name).toBe('No Tandoor Recipe');
expect(uploadRecipeWithIngredientsDTO).not.toHaveBeenCalled(); expect(uploadRecipeWithIngredientsDTO).not.toHaveBeenCalled();
// Restore mock // Restore mock
vi.restoreAllMocks(); vi.restoreAllMocks();
}, 10000); }, 10000);
it('should handle extraction errors', async () => { it('should handle extraction errors', async () => {
vi.mocked(extractTextAndThumbnail).mockRejectedValue( vi.mocked(extractTextAndThumbnail).mockRejectedValue(new Error('Network timeout'));
new Error('Network timeout')
); const item = queueManager.enqueue('https://instagram.com/p/error');
const item = queueManager.enqueue('https://instagram.com/p/error'); await new Promise((resolve) => setTimeout(resolve, 800));
await new Promise((resolve) => setTimeout(resolve, 800)); const updated = queueManager.get(item.id);
const updated = queueManager.get(item.id); // Should mark as unhealthy (recoverable)
expect(updated?.status).toBe('unhealthy');
// Should mark as unhealthy (recoverable) expect(updated?.error?.message).toContain('timeout');
expect(updated?.status).toBe('unhealthy'); }, 10000);
expect(updated?.error?.message).toContain('timeout');
}, 10000); it('should handle parsing failure', async () => {
vi.mocked(extractTextAndThumbnail).mockResolvedValue({
it('should handle parsing failure', async () => { bodyText: 'Not a recipe',
vi.mocked(extractTextAndThumbnail).mockResolvedValue({ thumbnail: null
bodyText: 'Not a recipe', });
thumbnail: null
}); vi.mocked(extractRecipe).mockResolvedValue(null);
vi.mocked(extractRecipe).mockResolvedValue(null); const item = queueManager.enqueue('https://instagram.com/p/not-recipe');
const item = queueManager.enqueue('https://instagram.com/p/not-recipe'); await new Promise((resolve) => setTimeout(resolve, 800));
await new Promise((resolve) => setTimeout(resolve, 800)); const updated = queueManager.get(item.id);
const updated = queueManager.get(item.id); // Should mark as error (non-recoverable - no recipe found)
expect(updated?.status).toBe('error');
// Should mark as error (non-recoverable - no recipe found) expect(updated?.error?.message).toContain('recipe');
expect(updated?.status).toBe('error'); }, 10000);
expect(updated?.error?.message).toContain('recipe');
}, 10000); it('should process multiple items respecting concurrency', async () => {
// Set up mocks with delay to observe concurrency
it('should process multiple items respecting concurrency', async () => { vi.mocked(extractTextAndThumbnail).mockImplementation(async () => {
// Set up mocks with delay to observe concurrency await new Promise((resolve) => setTimeout(resolve, 300));
vi.mocked(extractTextAndThumbnail).mockImplementation(async () => { return { bodyText: 'text', thumbnail: null };
await new Promise((resolve) => setTimeout(resolve, 300)); });
return { bodyText: 'text', thumbnail: null };
}); vi.mocked(extractRecipe).mockResolvedValue({
name: 'Concurrent Recipe',
vi.mocked(extractRecipe).mockResolvedValue({ servings: null,
name: 'Concurrent Recipe', ingredients: [],
servings: null, steps: [],
ingredients: [], description: ''
steps: [], });
description: ''
}); // Enqueue 3 items (Tandoor enabled by default in config mock)
queueManager.enqueue('https://instagram.com/p/item1');
// Enqueue 3 items (Tandoor enabled by default in config mock) queueManager.enqueue('https://instagram.com/p/item2');
queueManager.enqueue('https://instagram.com/p/item1'); queueManager.enqueue('https://instagram.com/p/item3');
queueManager.enqueue('https://instagram.com/p/item2');
queueManager.enqueue('https://instagram.com/p/item3'); // Wait a bit for processor to start working
await new Promise((resolve) => setTimeout(resolve, 150));
// Wait a bit for processor to start working
await new Promise((resolve) => setTimeout(resolve, 150)); const items = queueManager.getAll();
const inProgress = items.filter((i) => i.status === 'in_progress');
const items = queueManager.getAll();
const inProgress = items.filter(i => i.status === 'in_progress'); // With concurrency=2, should have max 2 in progress at once
expect(inProgress.length).toBeLessThanOrEqual(2);
// With concurrency=2, should have max 2 in progress at once
expect(inProgress.length).toBeLessThanOrEqual(2); // Wait for all to complete
await new Promise((resolve) => setTimeout(resolve, 2000));
// Wait for all to complete
await new Promise((resolve) => setTimeout(resolve, 2000)); const final = queueManager.getAll();
const completed = final.filter((i) => i.status === 'success');
const final = queueManager.getAll();
const completed = final.filter(i => i.status === 'success'); // All 3 should eventually complete
expect(completed.length).toBe(3);
// All 3 should eventually complete }, 15000);
expect(completed.length).toBe(3);
}, 15000);
}); });

262
src/tests/queue-sse.spec.ts
View File

@@ -1,6 +1,6 @@
/** /**
* Integration tests for Queue SSE Stream endpoint * Integration tests for Queue SSE Stream endpoint
* *
* Tests the Server-Sent Events stream for real-time queue updates. * Tests the Server-Sent Events stream for real-time queue updates.
*/ */
@@ -9,133 +9,133 @@ import { queueManager } from '$lib/server/queue/QueueManager';
import { GET as streamGET } from '../routes/api/queue/stream/+server.js'; import { GET as streamGET } from '../routes/api/queue/stream/+server.js';
describe('Queue SSE Stream Endpoint', () => { describe('Queue SSE Stream Endpoint', () => {
beforeEach(() => { beforeEach(() => {
// Clear queue between tests // Clear queue between tests
queueManager.getAll().forEach(item => queueManager.remove(item.id)); queueManager.getAll().forEach((item) => queueManager.remove(item.id));
}); });
afterEach(() => { afterEach(() => {
// Clean up after tests // Clean up after tests
queueManager.getAll().forEach(item => queueManager.remove(item.id)); queueManager.getAll().forEach((item) => queueManager.remove(item.id));
}); });
describe('GET /api/queue/stream', () => { describe('GET /api/queue/stream', () => {
it('should return SSE response with correct headers', async () => { it('should return SSE response with correct headers', async () => {
const url = new URL('http://localhost/api/queue/stream'); const url = new URL('http://localhost/api/queue/stream');
const request = new Request(url); const request = new Request(url);
const response = await streamGET({ const response = await streamGET({
url, url,
request: { request: {
...request, ...request,
signal: new AbortController().signal signal: new AbortController().signal
} }
} as any); } as any);
expect(response.status).toBe(200); expect(response.status).toBe(200);
expect(response.headers.get('Content-Type')).toBe('text/event-stream'); expect(response.headers.get('Content-Type')).toBe('text/event-stream');
expect(response.headers.get('Cache-Control')).toBe('no-cache'); expect(response.headers.get('Cache-Control')).toBe('no-cache');
// Connection header no longer manually set - managed automatically by Node.js // Connection header no longer manually set - managed automatically by Node.js
}); });
it('should reject invalid status filter', async () => { it('should reject invalid status filter', async () => {
const url = new URL('http://localhost/api/queue/stream?status=invalid'); const url = new URL('http://localhost/api/queue/stream?status=invalid');
const request = new Request(url); const request = new Request(url);
const response = await streamGET({ const response = await streamGET({
url, url,
request: { request: {
...request, ...request,
signal: new AbortController().signal signal: new AbortController().signal
} }
} as any); } as any);
expect(response.status).toBe(400); expect(response.status).toBe(400);
const text = await response.text(); const text = await response.text();
expect(text).toContain('Invalid status filter'); expect(text).toContain('Invalid status filter');
}); });
it('should reject invalid item ID format', async () => { it('should reject invalid item ID format', async () => {
const url = new URL('http://localhost/api/queue/stream?id=invalid-id'); const url = new URL('http://localhost/api/queue/stream?id=invalid-id');
const request = new Request(url); const request = new Request(url);
const response = await streamGET({ const response = await streamGET({
url, url,
request: { request: {
...request, ...request,
signal: new AbortController().signal signal: new AbortController().signal
} }
} as any); } as any);
expect(response.status).toBe(400); expect(response.status).toBe(400);
const text = await response.text(); const text = await response.text();
expect(text).toBe('Invalid queue item ID format'); expect(text).toBe('Invalid queue item ID format');
}); });
it('should accept valid status filter', async () => { it('should accept valid status filter', async () => {
const url = new URL('http://localhost/api/queue/stream?status=pending'); const url = new URL('http://localhost/api/queue/stream?status=pending');
const request = new Request(url); const request = new Request(url);
const response = await streamGET({ const response = await streamGET({
url, url,
request: { request: {
...request, ...request,
signal: new AbortController().signal signal: new AbortController().signal
} }
} as any); } as any);
expect(response.status).toBe(200); expect(response.status).toBe(200);
expect(response.headers.get('Content-Type')).toBe('text/event-stream'); expect(response.headers.get('Content-Type')).toBe('text/event-stream');
}); });
it('should accept valid item ID filter', async () => { it('should accept valid item ID filter', async () => {
// Add a test item first // Add a test item first
const item = queueManager.enqueue('https://instagram.com/p/TEST123'); const item = queueManager.enqueue('https://instagram.com/p/TEST123');
const url = new URL(`http://localhost/api/queue/stream?id=${item.id}`); const url = new URL(`http://localhost/api/queue/stream?id=${item.id}`);
const request = new Request(url); const request = new Request(url);
const response = await streamGET({ const response = await streamGET({
url, url,
request: { request: {
...request, ...request,
signal: new AbortController().signal signal: new AbortController().signal
} }
} as any); } as any);
expect(response.status).toBe(200); expect(response.status).toBe(200);
expect(response.headers.get('Content-Type')).toBe('text/event-stream'); expect(response.headers.get('Content-Type')).toBe('text/event-stream');
}); });
it('should handle stream initialization without errors', async () => { it('should handle stream initialization without errors', async () => {
// Add some test items // Add some test items
queueManager.enqueue('https://instagram.com/p/TEST1'); queueManager.enqueue('https://instagram.com/p/TEST1');
queueManager.enqueue('https://instagram.com/p/TEST2'); queueManager.enqueue('https://instagram.com/p/TEST2');
const url = new URL('http://localhost/api/queue/stream'); const url = new URL('http://localhost/api/queue/stream');
const abortController = new AbortController(); const abortController = new AbortController();
const request = new Request(url, { const request = new Request(url, {
signal: abortController.signal signal: abortController.signal
}); });
const response = await streamGET({ const response = await streamGET({
url, url,
request request
} as any); } as any);
expect(response.status).toBe(200); expect(response.status).toBe(200);
expect(response.body).toBeInstanceOf(ReadableStream); expect(response.body).toBeInstanceOf(ReadableStream);
// Abort the request to clean up // Abort the request to clean up
abortController.abort(); abortController.abort();
}); });
}); });
// Note: Full SSE stream testing would require more complex setup with // Note: Full SSE stream testing would require more complex setup with
// ReadableStream readers and async iteration, which is beyond the scope // ReadableStream readers and async iteration, which is beyond the scope
// of these basic endpoint validation tests. The above tests verify that: // of these basic endpoint validation tests. The above tests verify that:
// 1. The endpoint responds correctly // 1. The endpoint responds correctly
// 2. Headers are set properly for SSE // 2. Headers are set properly for SSE
// 3. Parameter validation works // 3. Parameter validation works
// 4. Stream initialization succeeds // 4. Stream initialization succeeds
}); });

268
src/tests/scheduler.integration.spec.ts
View File

@@ -1,134 +1,134 @@
import { describe, it, expect, beforeEach, afterEach } from 'vitest'; import { describe, it, expect, beforeEach, afterEach } from 'vitest';
import path from 'path'; import path from 'path';
import fs from 'fs'; import fs from 'fs';
/** /**
* Integration tests for the scheduler * Integration tests for the scheduler
* These tests verify the scheduler behavior with mocked browser contexts * These tests verify the scheduler behavior with mocked browser contexts
*/ */
describe('Scheduler Integration Tests', () => { describe('Scheduler Integration Tests', () => {
const mockAuthPath = path.join(__dirname, '../../__mocks__/auth.json'); const mockAuthPath = path.join(__dirname, '../../__mocks__/auth.json');
const mockAuthDir = path.dirname(mockAuthPath); const mockAuthDir = path.dirname(mockAuthPath);
beforeEach(() => { beforeEach(() => {
// Create mock directory structure // Create mock directory structure
if (!fs.existsSync(mockAuthDir)) { if (!fs.existsSync(mockAuthDir)) {
fs.mkdirSync(mockAuthDir, { recursive: true }); fs.mkdirSync(mockAuthDir, { recursive: true });
} }
// Create mock auth.json // Create mock auth.json
const mockAuth = { const mockAuth = {
cookies: [ cookies: [
{ {
name: 'sessionid', name: 'sessionid',
value: 'mock-session-id', value: 'mock-session-id',
domain: '.instagram.com', domain: '.instagram.com',
path: '/', path: '/',
expires: Date.now() / 1000 + 3600 * 24 * 30, // 30 days expires: Date.now() / 1000 + 3600 * 24 * 30, // 30 days
httpOnly: true, httpOnly: true,
secure: true, secure: true,
sameSite: 'Strict' sameSite: 'Strict'
} }
], ],
origins: [] origins: []
}; };
fs.writeFileSync(mockAuthPath, JSON.stringify(mockAuth, null, 2)); fs.writeFileSync(mockAuthPath, JSON.stringify(mockAuth, null, 2));
}); });
afterEach(() => { afterEach(() => {
// Cleanup mock files // Cleanup mock files
if (fs.existsSync(mockAuthPath)) { if (fs.existsSync(mockAuthPath)) {
fs.unlinkSync(mockAuthPath); fs.unlinkSync(mockAuthPath);
} }
if (fs.existsSync(mockAuthDir) && fs.readdirSync(mockAuthDir).length === 0) { if (fs.existsSync(mockAuthDir) && fs.readdirSync(mockAuthDir).length === 0) {
fs.rmdirSync(mockAuthDir); fs.rmdirSync(mockAuthDir);
} }
}); });
describe('Auth File Management', () => { describe('Auth File Management', () => {
it('should detect existing auth.json file', () => { it('should detect existing auth.json file', () => {
const exists = fs.existsSync(mockAuthPath); const exists = fs.existsSync(mockAuthPath);
expect(exists).toBe(true); expect(exists).toBe(true);
}); });
it('should preserve auth.json structure when renewed', () => { it('should preserve auth.json structure when renewed', () => {
const authContent = JSON.parse(fs.readFileSync(mockAuthPath, 'utf-8')); const authContent = JSON.parse(fs.readFileSync(mockAuthPath, 'utf-8'));
expect(authContent).toHaveProperty('cookies'); expect(authContent).toHaveProperty('cookies');
expect(authContent).toHaveProperty('origins'); expect(authContent).toHaveProperty('origins');
expect(Array.isArray(authContent.cookies)).toBe(true); expect(Array.isArray(authContent.cookies)).toBe(true);
}); });
it('should create secrets directory if it does not exist', () => { it('should create secrets directory if it does not exist', () => {
const secretsDir = path.join(__dirname, '../../__mocks__/secrets'); const secretsDir = path.join(__dirname, '../../__mocks__/secrets');
if (!fs.existsSync(secretsDir)) { if (!fs.existsSync(secretsDir)) {
fs.mkdirSync(secretsDir, { recursive: true }); fs.mkdirSync(secretsDir, { recursive: true });
} }
expect(fs.existsSync(secretsDir)).toBe(true); expect(fs.existsSync(secretsDir)).toBe(true);
// Cleanup // Cleanup
if (fs.readdirSync(secretsDir).length === 0) { if (fs.readdirSync(secretsDir).length === 0) {
fs.rmdirSync(secretsDir); fs.rmdirSync(secretsDir);
} }
}); });
}); });
describe('Scheduler Timing', () => { describe('Scheduler Timing', () => {
it('should calculate correct interval from hours', () => { it('should calculate correct interval from hours', () => {
const hours = 12; const hours = 12;
const expectedMs = hours * 60 * 60 * 1000; const expectedMs = hours * 60 * 60 * 1000;
expect(expectedMs).toBe(43200000); expect(expectedMs).toBe(43200000);
}); });
it('should support 6-hour renewal interval', () => { it('should support 6-hour renewal interval', () => {
const hours = 6; const hours = 6;
const expectedMs = hours * 60 * 60 * 1000; const expectedMs = hours * 60 * 60 * 1000;
expect(expectedMs).toBe(21600000); expect(expectedMs).toBe(21600000);
}); });
it('should support 24-hour renewal interval', () => { it('should support 24-hour renewal interval', () => {
const hours = 24; const hours = 24;
const expectedMs = hours * 60 * 60 * 1000; const expectedMs = hours * 60 * 60 * 1000;
expect(expectedMs).toBe(86400000); expect(expectedMs).toBe(86400000);
}); });
}); });
describe('Error Handling', () => { describe('Error Handling', () => {
it('should handle missing auth.json gracefully', () => { it('should handle missing auth.json gracefully', () => {
const nonExistentPath = path.join(__dirname, '../../__mocks__/nonexistent.json'); const nonExistentPath = path.join(__dirname, '../../__mocks__/nonexistent.json');
const exists = fs.existsSync(nonExistentPath); const exists = fs.existsSync(nonExistentPath);
expect(exists).toBe(false); expect(exists).toBe(false);
}); });
it('should validate auth.json structure', () => { it('should validate auth.json structure', () => {
const authContent = JSON.parse(fs.readFileSync(mockAuthPath, 'utf-8')); const authContent = JSON.parse(fs.readFileSync(mockAuthPath, 'utf-8'));
const hasRequiredFields = 'cookies' in authContent && 'origins' in authContent; const hasRequiredFields = 'cookies' in authContent && 'origins' in authContent;
expect(hasRequiredFields).toBe(true); expect(hasRequiredFields).toBe(true);
}); });
}); });
describe('Path Resolution', () => { describe('Path Resolution', () => {
it('should resolve Docker auth path when it exists', () => { it('should resolve Docker auth path when it exists', () => {
// This would be tested with actual file system mocks // This would be tested with actual file system mocks
const dockerPath = '/app/secrets/auth.json'; const dockerPath = '/app/secrets/auth.json';
const localPath = './secrets/auth.json'; const localPath = './secrets/auth.json';
// In real scenario, mock fs.existsSync to return true for dockerPath // In real scenario, mock fs.existsSync to return true for dockerPath
expect(dockerPath).toMatch(/\/app\/secrets\/auth\.json/); expect(dockerPath).toMatch(/\/app\/secrets\/auth\.json/);
}); });
it('should fall back to local path', () => { it('should fall back to local path', () => {
const localPath = './secrets/auth.json'; const localPath = './secrets/auth.json';
expect(localPath).toMatch(/\.\/secrets\/auth\.json/); expect(localPath).toMatch(/\.\/secrets\/auth\.json/);
}); });
}); });
}); });

410
src/tests/scheduler.spec.ts
View File

@@ -1,205 +1,205 @@
import { getSchedulerStatus, startScheduler, stopScheduler } from '$lib/server/scheduler'; import { getSchedulerStatus, startScheduler, stopScheduler } from '$lib/server/scheduler';
import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest'; import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest';
// Mock environment variables // Mock environment variables
const { mockEnv } = vi.hoisted(() => { const { mockEnv } = vi.hoisted(() => {
return { return {
mockEnv: { mockEnv: {
AUTH_SCHEDULER_ENABLED: 'false', AUTH_SCHEDULER_ENABLED: 'false',
AUTH_SCHEDULER_INTERVAL_MINUTES: '720' AUTH_SCHEDULER_INTERVAL_MINUTES: '720'
} }
}; };
}); });
vi.mock('$env/dynamic/private', () => ({ vi.mock('$env/dynamic/private', () => ({
env: mockEnv env: mockEnv
})); }));
// Mock the browser module // Mock the browser module
vi.mock('$lib/server/browser', () => ({ vi.mock('$lib/server/browser', () => ({
getBrowser: vi.fn(), getBrowser: vi.fn(),
initializeBrowser: vi.fn(), initializeBrowser: vi.fn(),
closeBrowser: vi.fn() closeBrowser: vi.fn()
})); }));
// Mock fs operations // Mock fs operations
const mockFs = { const mockFs = {
existsSync: vi.fn(), existsSync: vi.fn(),
mkdirSync: vi.fn(), mkdirSync: vi.fn(),
writeFileSync: vi.fn(), writeFileSync: vi.fn(),
readFileSync: vi.fn() readFileSync: vi.fn()
}; };
describe('Scheduler Service', () => { describe('Scheduler Service', () => {
beforeEach(() => { beforeEach(() => {
// Reset environment variables // Reset environment variables
mockEnv.AUTH_SCHEDULER_ENABLED = 'false'; mockEnv.AUTH_SCHEDULER_ENABLED = 'false';
mockEnv.AUTH_SCHEDULER_INTERVAL_MINUTES = '720'; mockEnv.AUTH_SCHEDULER_INTERVAL_MINUTES = '720';
// Clear all mocks // Clear all mocks
vi.clearAllMocks(); vi.clearAllMocks();
// Reset scheduler state by stopping if running // Reset scheduler state by stopping if running
try { try {
stopScheduler(); stopScheduler();
} catch { } catch {
// Ignore if not running // Ignore if not running
} }
}); });
afterEach(async () => { afterEach(async () => {
// Ensure scheduler is stopped after each test // Ensure scheduler is stopped after each test
await stopScheduler(); await stopScheduler();
}); });
describe('Configuration', () => { describe('Configuration', () => {
it('should use default interval when AUTH_SCHEDULER_INTERVAL_MINUTES is not set', async () => { it('should use default interval when AUTH_SCHEDULER_INTERVAL_MINUTES is not set', async () => {
mockEnv.AUTH_SCHEDULER_ENABLED = 'true'; mockEnv.AUTH_SCHEDULER_ENABLED = 'true';
mockEnv.AUTH_SCHEDULER_INTERVAL_MINUTES = ''; mockEnv.AUTH_SCHEDULER_INTERVAL_MINUTES = '';
const status = getSchedulerStatus(); const status = getSchedulerStatus();
expect(status.config.intervalMinutes).toBe(720); expect(status.config.intervalMinutes).toBe(720);
}); });
it('should parse custom interval minutes from environment', async () => { it('should parse custom interval minutes from environment', async () => {
mockEnv.AUTH_SCHEDULER_ENABLED = 'true'; mockEnv.AUTH_SCHEDULER_ENABLED = 'true';
mockEnv.AUTH_SCHEDULER_INTERVAL_MINUTES = '30'; mockEnv.AUTH_SCHEDULER_INTERVAL_MINUTES = '30';
const status = getSchedulerStatus(); const status = getSchedulerStatus();
expect(status.config.intervalMinutes).toBe(30); expect(status.config.intervalMinutes).toBe(30);
}); });
it('should disable scheduler when AUTH_SCHEDULER_ENABLED is not true', async () => { it('should disable scheduler when AUTH_SCHEDULER_ENABLED is not true', async () => {
mockEnv.AUTH_SCHEDULER_ENABLED = 'false'; mockEnv.AUTH_SCHEDULER_ENABLED = 'false';
const status = getSchedulerStatus(); const status = getSchedulerStatus();
expect(status.config.enabled).toBe(false); expect(status.config.enabled).toBe(false);
expect(status.running).toBe(false); expect(status.running).toBe(false);
}); });
it('should parse AUTH_SCHEDULER_ENABLED as true when set to "true"', async () => { it('should parse AUTH_SCHEDULER_ENABLED as true when set to "true"', async () => {
mockEnv.AUTH_SCHEDULER_ENABLED = 'true'; mockEnv.AUTH_SCHEDULER_ENABLED = 'true';
const status = getSchedulerStatus(); const status = getSchedulerStatus();
expect(status.config.enabled).toBe(true); expect(status.config.enabled).toBe(true);
}); });
}); });
describe('Scheduler Lifecycle', () => { describe('Scheduler Lifecycle', () => {
it('should not start when disabled', async () => { it('should not start when disabled', async () => {
mockEnv.AUTH_SCHEDULER_ENABLED = 'false'; mockEnv.AUTH_SCHEDULER_ENABLED = 'false';
await startScheduler(); await startScheduler();
const status = getSchedulerStatus(); const status = getSchedulerStatus();
expect(status.running).toBe(false); expect(status.running).toBe(false);
}); });
it('should start when enabled', async () => { it('should start when enabled', async () => {
mockEnv.AUTH_SCHEDULER_ENABLED = 'true'; mockEnv.AUTH_SCHEDULER_ENABLED = 'true';
mockFs.existsSync.mockReturnValue(true); mockFs.existsSync.mockReturnValue(true);
await startScheduler(); await startScheduler();
const status = getSchedulerStatus(); const status = getSchedulerStatus();
expect(status.running).toBe(true); expect(status.running).toBe(true);
}); });
it('should not start twice', async () => { it('should not start twice', async () => {
mockEnv.AUTH_SCHEDULER_ENABLED = 'true'; mockEnv.AUTH_SCHEDULER_ENABLED = 'true';
mockFs.existsSync.mockReturnValue(true); mockFs.existsSync.mockReturnValue(true);
await startScheduler(); await startScheduler();
const consoleSpy = vi.spyOn(console, 'warn'); const consoleSpy = vi.spyOn(console, 'warn');
await startScheduler(); await startScheduler();
expect(consoleSpy).toHaveBeenCalledWith('[Scheduler] Scheduler is already running'); expect(consoleSpy).toHaveBeenCalledWith('[Scheduler] Scheduler is already running');
}); });
it('should stop the scheduler', async () => { it('should stop the scheduler', async () => {
mockEnv.AUTH_SCHEDULER_ENABLED = 'true'; mockEnv.AUTH_SCHEDULER_ENABLED = 'true';
mockFs.existsSync.mockReturnValue(true); mockFs.existsSync.mockReturnValue(true);
await startScheduler(); await startScheduler();
expect(getSchedulerStatus().running).toBe(true); expect(getSchedulerStatus().running).toBe(true);
await stopScheduler(); await stopScheduler();
expect(getSchedulerStatus().running).toBe(false); expect(getSchedulerStatus().running).toBe(false);
}); });
it('should handle stopping when not running', async () => { it('should handle stopping when not running', async () => {
const consoleSpy = vi.spyOn(console, 'log'); const consoleSpy = vi.spyOn(console, 'log');
await stopScheduler(); await stopScheduler();
expect(consoleSpy).toHaveBeenCalledWith('[Scheduler] Scheduler is not running'); expect(consoleSpy).toHaveBeenCalledWith('[Scheduler] Scheduler is not running');
}); });
}); });
describe('Status Reporting', () => { describe('Status Reporting', () => {
it('should return scheduler status with default values', () => { it('should return scheduler status with default values', () => {
mockEnv.AUTH_SCHEDULER_ENABLED = 'false'; mockEnv.AUTH_SCHEDULER_ENABLED = 'false';
const status = getSchedulerStatus(); const status = getSchedulerStatus();
expect(status).toEqual({ expect(status).toEqual({
running: false, running: false,
lastRenewalTime: null, lastRenewalTime: null,
isRenewing: false, isRenewing: false,
config: { config: {
enabled: false, enabled: false,
intervalMinutes: 720 intervalMinutes: 720
} }
}); });
}); });
it('should report running state correctly', async () => { it('should report running state correctly', async () => {
mockEnv.AUTH_SCHEDULER_ENABLED = 'true'; mockEnv.AUTH_SCHEDULER_ENABLED = 'true';
mockFs.existsSync.mockReturnValue(true); mockFs.existsSync.mockReturnValue(true);
await startScheduler(); await startScheduler();
const status = getSchedulerStatus(); const status = getSchedulerStatus();
expect(status.running).toBe(true); expect(status.running).toBe(true);
expect(status.isRenewing).toBe(false); expect(status.isRenewing).toBe(false);
}); });
it('should track configuration', async () => { it('should track configuration', async () => {
mockEnv.AUTH_SCHEDULER_ENABLED = 'true'; mockEnv.AUTH_SCHEDULER_ENABLED = 'true';
mockEnv.AUTH_SCHEDULER_INTERVAL_MINUTES = '1440'; mockEnv.AUTH_SCHEDULER_INTERVAL_MINUTES = '1440';
const status = getSchedulerStatus(); const status = getSchedulerStatus();
expect(status.config.enabled).toBe(true); expect(status.config.enabled).toBe(true);
expect(status.config.intervalMinutes).toBe(1440); expect(status.config.intervalMinutes).toBe(1440);
}); });
}); });
describe('Auth Renewal', () => { describe('Auth Renewal', () => {
it('should skip renewal if no auth.json exists', async () => { it('should skip renewal if no auth.json exists', async () => {
mockFs.existsSync.mockReturnValue(false); mockFs.existsSync.mockReturnValue(false);
// Note: In a real test, you'd import and call the renewal function directly // Note: In a real test, you'd import and call the renewal function directly
// This test verifies the behavior when auth file is missing // This test verifies the behavior when auth file is missing
expect(mockFs.existsSync.mock.calls.length).toBeGreaterThanOrEqual(0); expect(mockFs.existsSync.mock.calls.length).toBeGreaterThanOrEqual(0);
}); });
it('should prevent concurrent renewal attempts', async () => { it('should prevent concurrent renewal attempts', async () => {
// This would be tested through integration tests with actual browser context // This would be tested through integration tests with actual browser context
// The scheduler maintains state.isRenewing flag to prevent concurrent calls // The scheduler maintains state.isRenewing flag to prevent concurrent calls
const status = getSchedulerStatus(); const status = getSchedulerStatus();
expect(status.isRenewing).toBe(false); expect(status.isRenewing).toBe(false);
}); });
}); });
describe('Environment Variables', () => { describe('Environment Variables', () => {
it('should handle empty AUTH_SCHEDULER_INTERVAL_MINUTES with default', () => { it('should handle empty AUTH_SCHEDULER_INTERVAL_MINUTES with default', () => {
mockEnv.AUTH_SCHEDULER_ENABLED = 'true'; mockEnv.AUTH_SCHEDULER_ENABLED = 'true';
mockEnv.AUTH_SCHEDULER_INTERVAL_MINUTES = ''; mockEnv.AUTH_SCHEDULER_INTERVAL_MINUTES = '';
const status = getSchedulerStatus(); const status = getSchedulerStatus();
// Empty string should fall back to default due to parseInt('', 10) returning NaN // Empty string should fall back to default due to parseInt('', 10) returning NaN
// and the || 720 fallback // and the || 720 fallback
expect(status.config.intervalMinutes).toBeDefined(); expect(status.config.intervalMinutes).toBeDefined();
}); });
}); });
}); });

34
src/tests/sse-extraction.spec.ts
View File

@@ -1,6 +1,6 @@
/** /**
* Integration tests for SSE extraction endpoint * Integration tests for SSE extraction endpoint
* *
* Tests the real-time progress streaming from extraction to frontend * Tests the real-time progress streaming from extraction to frontend
*/ */
@@ -11,31 +11,31 @@ describe('SSE Extraction Endpoint', () => {
it('should stream progress events for successful extraction', async () => { it('should stream progress events for successful extraction', async () => {
// Mock Instagram URL (would need real URL for full e2e test) // Mock Instagram URL (would need real URL for full e2e test)
const testUrl = 'https://www.instagram.com/p/test123/'; const testUrl = 'https://www.instagram.com/p/test123/';
const events: ProgressEvent[] = []; const events: ProgressEvent[] = [];
// Note: This is a structure test. Real testing requires: // Note: This is a structure test. Real testing requires:
// 1. Running server // 1. Running server
// 2. Valid Instagram URL // 2. Valid Instagram URL
// 3. Browser context available // 3. Browser context available
// Expected event flow // Expected event flow
const expectedEventTypes = [ const expectedEventTypes = [
'status', // Starting extraction 'status', // Starting extraction
'status', // Loading page 'status', // Loading page
'method', // Trying first method 'method', // Trying first method
'status', // Success or next method 'status', // Success or next method
'status', // Parsing recipe 'status', // Parsing recipe
'complete' // Final result 'complete' // Final result
]; ];
expect(expectedEventTypes).toBeDefined(); expect(expectedEventTypes).toBeDefined();
}); });
it('should handle errors gracefully', async () => { it('should handle errors gracefully', async () => {
// Test with invalid URL // Test with invalid URL
const invalidUrl = 'not-a-valid-url'; const invalidUrl = 'not-a-valid-url';
// Expected: error event should be sent // Expected: error event should be sent
expect(invalidUrl).toBeTruthy(); expect(invalidUrl).toBeTruthy();
}); });
@@ -92,14 +92,14 @@ describe('SSE Extraction Endpoint', () => {
describe('Frontend SSE Parser', () => { describe('Frontend SSE Parser', () => {
it('should parse SSE event format correctly', () => { it('should parse SSE event format correctly', () => {
const sseMessage = 'event: progress\ndata: {"type":"status","message":"test"}\n\n'; const sseMessage = 'event: progress\ndata: {"type":"status","message":"test"}\n\n';
const eventMatch = sseMessage.match(/^event: (\w+)\ndata: (.+)$/s); const eventMatch = sseMessage.match(/^event: (\w+)\ndata: (.+)$/s);
expect(eventMatch).toBeTruthy(); expect(eventMatch).toBeTruthy();
if (eventMatch) { if (eventMatch) {
const [, eventType, eventData] = eventMatch; const [, eventType, eventData] = eventMatch;
expect(eventType).toBe('progress'); expect(eventType).toBe('progress');
const parsed = JSON.parse(eventData.replace(/\n\n$/, '')); const parsed = JSON.parse(eventData.replace(/\n\n$/, ''));
expect(parsed.type).toBe('status'); expect(parsed.type).toBe('status');
expect(parsed.message).toBe('test'); expect(parsed.message).toBe('test');
@@ -112,7 +112,7 @@ describe('Frontend SSE Parser', () => {
'embedded-json': '📦', 'embedded-json': '📦',
'dom-selector': '🎯', 'dom-selector': '🎯',
'graphql-api': '🔌', 'graphql-api': '🔌',
'legacy': '📄' legacy: '📄'
}; };
return method ? icons[method] || '⚙️' : '⚙️'; return method ? icons[method] || '⚙️' : '⚙️';
}; };
@@ -128,7 +128,7 @@ describe('Frontend SSE Parser', () => {
/** /**
* Manual E2E Testing Checklist: * Manual E2E Testing Checklist:
* *
* □ Start dev server: npm run dev * □ Start dev server: npm run dev
* □ Open /share?url=<instagram-url> * □ Open /share?url=<instagram-url>
* □ Click "Extract Recipe" * □ Click "Extract Recipe"

26
src/tests/tandoor-logging.spec.ts
View File

@@ -24,18 +24,13 @@ describe('tandoor logging', () => {
name: 'Test Recipe', name: 'Test Recipe',
servings: 4, servings: 4,
description: 'Test description', description: 'Test description',
ingredients: [ ingredients: [{ item: 'Flour', amount: '2', unit: 'cups' }],
{ item: 'Flour', amount: '2', unit: 'cups' }
],
steps: ['Mix ingredients'] steps: ['Mix ingredients']
}; };
await uploadRecipeWithIngredientsDTO(recipe); await uploadRecipeWithIngredientsDTO(recipe);
expect(logErrorSpy).toHaveBeenCalledWith( expect(logErrorSpy).toHaveBeenCalledWith('[Tandoor] Fetch error', expect.any(Error));
'[Tandoor] Fetch error',
expect.any(Error)
);
}); });
test('should use logError on API error response', async () => { test('should use logError on API error response', async () => {
@@ -80,10 +75,7 @@ describe('tandoor logging', () => {
await uploadRecipeWithIngredientsDTO(recipe); await uploadRecipeWithIngredientsDTO(recipe);
expect(logErrorSpy).toHaveBeenCalledWith( expect(logErrorSpy).toHaveBeenCalledWith('[Tandoor] Fetch error', expect.any(Error));
'[Tandoor] Fetch error',
expect.any(Error)
);
}); });
test('should use logError on image upload failure', async () => { test('should use logError on image upload failure', async () => {
@@ -93,10 +85,7 @@ describe('tandoor logging', () => {
const result = await uploadRecipeImage(123, 'https://example.com/image.jpg'); const result = await uploadRecipeImage(123, 'https://example.com/image.jpg');
expect(result.success).toBe(false); expect(result.success).toBe(false);
expect(logErrorSpy).toHaveBeenCalledWith( expect(logErrorSpy).toHaveBeenCalledWith('[Tandoor Upload] Exception', error);
'[Tandoor Upload] Exception',
error
);
}); });
test('should use logError instead of manual error logging', async () => { test('should use logError instead of manual error logging', async () => {
@@ -112,11 +101,8 @@ describe('tandoor logging', () => {
}); });
// Verify logError was called (which handles stack trace serialization) // Verify logError was called (which handles stack trace serialization)
expect(logErrorSpy).toHaveBeenCalledWith( expect(logErrorSpy).toHaveBeenCalledWith('[Tandoor] Fetch error', error);
'[Tandoor] Fetch error',
error
);
// logError itself logs stack traces, which is expected behavior // logError itself logs stack traces, which is expected behavior
// The key is that tandoor.ts uses logError instead of manual logging // The key is that tandoor.ts uses logError instead of manual logging
}); });

2
src/tests/thumbnail-validation.spec.ts
View File

@@ -2,7 +2,7 @@ import { describe, it, expect, vi, beforeEach, afterEach } from 'vitest';
/** /**
* Unit tests for thumbnail URL validation in fetchImageAsBase64 * Unit tests for thumbnail URL validation in fetchImageAsBase64
* *
* These tests verify that the enhanced URL validation: * These tests verify that the enhanced URL validation:
* - Accepts only HTTP 200 status codes * - Accepts only HTTP 200 status codes
* - Validates content-type is image/* * - Validates content-type is image/*

2
svelte.config.js
View File

@@ -6,7 +6,7 @@ const config = {
// Consult https://svelte.dev/docs/kit/integrations // Consult https://svelte.dev/docs/kit/integrations
// for more information about preprocessors // for more information about preprocessors
preprocess: vitePreprocess(), preprocess: vitePreprocess(),
kit: { kit: {
adapter: adapter(), adapter: adapter(),
serviceWorker: { serviceWorker: {
register: true // Enable SvelteKit's native service worker registration register: true // Enable SvelteKit's native service worker registration

32
vite.config.ts
View File

@@ -5,22 +5,22 @@ import { sveltekit } from '@sveltejs/kit/vite';
import fs from 'fs'; import fs from 'fs';
export default defineConfig({ export default defineConfig({
define: { define: {
'process.env.NODE_ENV': process.env.NODE_ENV === 'production' ? '"production"' : '"development"' 'process.env.NODE_ENV': process.env.NODE_ENV === 'production' ? '"production"' : '"development"'
}, },
server: { server: {
watch: { watch: {
ignored: ['**/debug_page.txt', '**/.ssl/**', '**/docs/**', '**/secrets/**'] ignored: ['**/debug_page.txt', '**/.ssl/**', '**/docs/**', '**/secrets/**']
}, },
https: fs.existsSync('./.ssl/localhost.key') && fs.existsSync('./.ssl/localhost.crt') https:
? { fs.existsSync('./.ssl/localhost.key') && fs.existsSync('./.ssl/localhost.crt')
key: fs.readFileSync('./.ssl/localhost.key'), ? {
cert: fs.readFileSync('./.ssl/localhost.crt') key: fs.readFileSync('./.ssl/localhost.key'),
} cert: fs.readFileSync('./.ssl/localhost.crt')
: undefined }
}, : undefined
plugins: [ },
tailwindcss(), sveltekit()], plugins: [tailwindcss(), sveltekit()],
test: { test: {
expect: { requireAssertions: true }, expect: { requireAssertions: true },
projects: [ projects: [