insta-recipe/docs/FINDINGS.md

# Findings & Research Documentation

**Last Updated:** 2026-02-15T00:00:00.000Z  
**JIRA:** RECIPE-0001  
**Status:** Initialized

---

## Purpose

This document tracks research findings, analysis results, and technical discoveries made during development. Each agent (Planner, Developer, Reviewer) appends findings as they work through the pipeline.

---

## Initial Codebase Analysis

### Language & Framework
- **Primary Language**: TypeScript 5.9.3
- **Framework**: SvelteKit 2.48.5 with Svelte 5.43.8
- **Runtime**: Node.js 22+
- **Package Manager**: npm

### Project Type
Progressive Web Application (PWA) for extracting recipes from Instagram posts and uploading them to Tandoor Recipe Manager.

### Architecture Style
**Hexagonal Architecture** (Ports and Adapters):
- Domain logic in `src/lib/server/`
- External system adapters: Instagram, Tandoor, LLM, Browser
- Clear separation between client and server code

### Key Technical Components
1. **Queue Management System**: In-memory FIFO queue with async processing
2. **Three-Phase Pipeline**: Extraction → Parsing → Uploading
3. **Real-Time Updates**: Server-Sent Events (SSE) for progress tracking
4. **Push Notifications**: Web Push API for background notifications
5. **PWA Features**: Service worker, manifest, install prompts

### Design Patterns Identified
- **Singleton**: QueueManager, QueueProcessor, PushNotificationService
- **Factory**: createLLM(), createBrowserContext(), initializeBrowser()
- **Observer**: Queue subscription system, SSE streaming
- **Adapter**: Instagram, Tandoor, LLM, Browser adapters
- **Strategy**: Multiple extraction methods with fallback

### Dependencies Overview
**Production** (6 dependencies):
- Browser automation: `playwright`
- LLM integration: `openai`
- Utilities: `uuid`, `date-fns`, `zod`

**Development** (26+ dependencies):
- Framework: `@sveltejs/kit`, `svelte`, `vite`
- Testing: `vitest`, `@vitest/browser-playwright`
- Styling: `tailwindcss`
- Tooling: `typescript`, `eslint`, `prettier`

### File Structure
```
52 total TypeScript/JavaScript files
├── 39 TypeScript files (.ts)
├── 10+ Svelte components (.svelte)
├── 3 JavaScript config files (.js)
└── Multiple test files (.spec.ts)
```

### Code Quality Indicators
- **Strict TypeScript**: `strict: true` enabled
- **Comprehensive Testing**: 138 tests across unit, integration, and browser tests
- **Linting**: ESLint with TypeScript and Svelte plugins
- **Formatting**: Prettier with Svelte and Tailwind plugins
- **Type Safety**: Zod schemas for runtime validation

### Environment Configuration
Required variables:
- `OPENAI_API_KEY` - LLM access
- `TANDOOR_URL` - Recipe manager URL (optional)
- `TANDOOR_TOKEN` - API authentication (optional)
- `QUEUE_CONCURRENCY` - Processing limit (default: 2)
- `QUEUE_MAX_RETRIES` - Retry attempts (default: 3)

### Deployment Setup
- **Docker**: Dockerfile with Node.js 22 Alpine + Chromium
- **HTTPS**: Local SSL certificates for PWA features
- **Production**: Node.js adapter for SvelteKit

### Notable Features
1. **Multi-Method Extraction**: 4-strategy cascade with intelligent fallback
2. **Progress Tracking**: Real-time callbacks throughout extraction pipeline
3. **Thumbnail Validation**: HTTP status code checking for image URLs
4. **Retry Logic**: Configurable retry attempts for failed extractions
5. **Scheduler**: Background task execution with authentication

---

## Technical Debt & Opportunities

### Identified Issues
1. **Deprecated Endpoints**: `/api/extract` returns 410 Gone (migration helper)
2. **In-Memory Queue**: No persistence - items lost on server restart
3. **Single Instance**: Queue state not shared across multiple server instances

### Potential Improvements
1. **Queue Persistence**: Redis or database-backed queue for durability
2. **Horizontal Scaling**: Shared queue state for multi-instance deployments
3. **Rate Limiting**: Instagram request throttling to avoid blocks
4. **Caching**: Extracted content caching to reduce redundant processing

---

## Research Findings

*This section will be populated by the Planner agent during task analysis.*

### [Planner] Research Notes - RECIPE-0001 (2026-02-15)

**Task:** Fix model loading issue and frontend error display

#### Issue 1: Model Loading - "400 No models loaded"
**Research Date:** 2026-02-15  
**Source:** Stack trace analysis, OpenAI SDK documentation, LM Studio/LiteLLM API patterns

**Problem Analysis:**
- Error occurs at `detectRecipe()` in [src/lib/server/parser.ts](src/lib/server/parser.ts#L30)
- OpenAI-compatible APIs (LM Studio, LiteLLM, Ollama, etc.) often require models to be explicitly loaded
- Current implementation assumes model is already loaded
- Error message contains provider-specific instructions ("use the 'lms load' command")

**OpenAI-Compatible Model Loading Patterns:**
1. **LM Studio**: Uses `/v1/models` endpoint to list available models
   - Loaded models appear in response with `"id": "model-name"`
   - No programmatic loading endpoint (manual load in UI)
   
2. **LiteLLM**: Uses `/v1/models` to list loaded models
   - Models must be configured in server startup
   - No dynamic loading endpoint
   
3. **Ollama**: Uses `/api/tags` for model list and `/api/pull` for loading
   - Different API structure (not `/v1` prefix)
   
4. **Generic OpenAI-compatible**: Most follow OpenAI's `/v1/models` endpoint
   - No standard for dynamic model loading
   - Usually require pre-configuration

**Solution Approach:**
- Check if model exists via `client.models.list()`
- If model not found/loaded, provide clear user-facing error
- Remove provider-specific error messages
- Add notification when model check succeeds
- Consider future enhancement: detect provider type and attempt auto-load if supported

**Files Affected:**
- [src/lib/server/llm.ts](src/lib/server/llm.ts) - Add model availability check
- [src/lib/server/parser.ts](src/lib/server/parser.ts) - Handle model not loaded error
- [src/lib/server/queue/QueueProcessor.ts](src/lib/server/queue/QueueProcessor.ts) - User notification

---

#### Issue 2: Frontend Error Display - "[object Object]"
**Research Date:** 2026-02-15  
**Source:** Code analysis of QueueItemCard.svelte, types.ts, QueueManager.ts

**Problem Analysis:**
- Error structure is an object: `{ phase, message, recoverable, timestamp }`
- Frontend displays `{item.error}` directly (line 205 of QueueItemCard.svelte)
- Svelte renders object.toString() → "[object Object]"

**Current Implementation:**
```typescript
// types.ts - Error is an object
error?: {
  phase: ProcessingPhase;
  message: string;
  recoverable: boolean;
  timestamp: string;
}

// QueueItemCard.svelte line 205 - Displays object directly
<div class="text-sm text-red-700 mt-1">{item.error}</div>
```

**Solution:**
Change to: `{item.error?.message || item.error}`
- Handles object error (gets .message)
- Handles legacy string errors (fallback)
- Type-safe with optional chaining

**Files Affected:**
- [src/routes/components/QueueItemCard.svelte](src/routes/components/QueueItemCard.svelte#L205) - Display error.message

---

#### Dependencies & Constraints (from ARCHITECTURE.md)
- Using `openai@^4.20.0` SDK
- Environment: `OPENAI_BASE_URL`, `OPENAI_API_KEY`, `LLM_MODEL`
- Current config example: `http://192.168.1.10:1234/v1` (LM Studio)
- Must maintain OpenAI-compatible API contract
- No assumption about specific provider implementation

#### Code Style Requirements (from CODE_STYLE.md)
- Use SvelteKit `$env/dynamic/private` for env vars (already correct)
- Error handling: try-catch with descriptive messages
- Console logging: `[Component] Message` format
- Type safety: TypeScript strict mode enabled

<!-- Planner appends findings here -->

---

### [Developer] Implementation Notes

<!-- Developer appends findings here -->

---

### [Reviewer] Review Notes

<!-- Reviewer appends findings here -->

---

## API Endpoint Catalog

### Active Endpoints

#### Queue Management
- `POST /api/queue` - Enqueue Instagram URL for processing
- `GET /api/queue` - List queue items (supports filtering, pagination)
- `GET /api/queue/stream` - SSE stream for real-time updates
- `GET /api/queue/{id}` - Get specific queue item details
- `DELETE /api/queue/{id}` - Remove item from queue
- `POST /api/queue/{id}/retry` - Retry failed extraction

#### Push Notifications
- `POST /api/notifications/subscribe` - Subscribe to push notifications
- `DELETE /api/notifications/subscribe` - Unsubscribe from notifications
- `GET /api/notifications/vapid-key` - Get VAPID public key

#### Health & Status
- `GET /api/health` - Application health check
- `GET /api/llm-health` - LLM service availability check

#### Tandoor Integration
- `POST /api/tandoor` - Upload recipe to Tandoor
- `GET /api/tandoor-config` - Get Tandoor configuration status

#### Legacy/Deprecated
- `POST /api/extract` - ⚠️ Deprecated (returns 410 Gone)

---

## Known Constraints

### Browser Automation
- Requires Chromium/Chrome installation
- Headless mode used in production
- Cookie handling for authenticated Instagram content

### LLM Integration
- Requires OpenAI-compatible API endpoint
- Configurable model selection
- Structured output using Zod schemas

### Tandoor Integration
- Optional feature (disabled without credentials)
- Requires Tandoor API token
- Supports ingredient partitioning across steps

### SSL Requirements
- HTTPS required for Service Worker registration
- Local development uses self-signed certificates
- Certificates managed via external Caddy CA

---

## Testing Coverage

### Test Distribution
- **Unit Tests**: Core logic validation
- **Integration Tests**: Multi-component workflows
- **API Tests**: Endpoint behavior verification
- **Browser Tests**: Svelte component rendering

### Test Files
- `queue-manager.spec.ts`
- `queue-processor.spec.ts`
- `queue-api.spec.ts`
- `queue-sse.spec.ts`
- `scheduler.spec.ts`
- `instagram-url-validation.spec.ts`
- `thumbnail-validation.spec.ts`
- `extraction-url-validation.integration.spec.ts`
- `page.svelte.spec.ts`

### Mock Strategy
- Environment variables mocked via `vi.mock('$env/dynamic/private')`
- External services mocked at module level
- Browser automation mocked for unit tests

---

## Documentation Inventory

### Existing Documentation
- `README.md` - Project overview and setup
- `docs/API.md` - API endpoint specifications
- `docs/MIGRATION.md` - Migration guides
- `docs/SVELTEKIT_SSR_GUIDE.md` - SSR implementation notes
- `docs/TESTING.md` - Testing guide and mocking patterns
- `docs/Tandoor (2.3.6).yaml` - OpenAPI spec for Tandoor

### Plan Documentation
`docs/plans/` contains 20+ implementation plans:
- Execution plans for completed features
- Technical specifications
- Story breakdowns with acceptance criteria

### Outcome Documentation
`docs/outcomes/` contains 20+ outcome reports:
- Implementation summaries
- Changes made
- Testing results
- Lessons learned

---

## Agent Pipeline Notes

### Build Commands
- **Build**: `npm run build`
- **Test**: `npm test` (alias for `npm run test:unit -- --run`)
- **Dev**: `npm run dev`
- **Lint**: `npm run lint`
- **Format**: `npm run format`

### Development Workflow
1. Make changes in `src/`
2. Run tests: `npm test`
3. Verify build: `npm run build`
4. Test locally: `npm run dev`

### Continuous Integration
- ESLint checks code quality
- Prettier enforces formatting
- TypeScript checks type safety
- Vitest runs test suite

---

## Next Steps

This document will be updated by subsequent agents:
1. **Planner**: Append research findings and analysis
2. **Developer**: Document implementation discoveries
3. **Reviewer**: Record review observations and recommendations

---

### [Planner] Research Notes - RECIPE-0002 (2026-02-16)

**Task:** Complete PWA implementation (installability, push notifications, share target)

#### PWA Documentation Research
**Research Date:** 2026-02-16  
**Sources:** MDN Web Docs, web.dev, W3C specifications

**Progressive Web Apps (PWA) - Key Requirements:**
1. **Web App Manifest** (`manifest.json`)
   - Required members: `name` or `short_name`, `icons` (192x192 PNG minimum), `start_url`, `display`
   - Share target support via `share_target` member (method, action, params)
   - Icons should include 192x192 and 512x512 sizes for optimal display
   - Browser compatibility: Chrome/Edge (full), Firefox/Safari (limited for share_target)

2. **Service Worker**
   - Must be registered to enable offline functionality
   - Lifecycle: install → activate → fetch events
   - Required for push notifications
   - Must be served over HTTPS (or localhost)

3. **HTTPS Requirement**
   - Mandatory for service worker registration
   - Required for push notifications and other secure contexts
   - Local development: `http://localhost` is treated as secure

4. **Installability Criteria** (from MDN/web.dev):
   - Valid manifest with required members
   - Service worker registered with fetch event handler
   - Served over HTTPS
   - At least one 192x192 PNG or SVG icon
   - Display mode set (fullscreen, standalone, minimal-ui)

**Push Notifications (Web Push API):**
- Requires service worker to receive push events
- VAPID authentication (application server keys) required for Chrome
- Subscription process: permission → subscribe → store subscription → send push
- Push service (browser vendor controlled) routes messages
- Notification permissions: default, granted, denied
- Best practice: request permission after user interaction

**Web Share Target API:**
- Registers PWA as share destination
- Configuration via manifest `share_target` member
- Supports GET or POST methods
- `params` define query string mapping (title, text, url)
- Files can be shared via POST with `multipart/form-data`
- Currently Chrome/Edge only (experimental)
- App must be installed to appear in share sheet

#### Current Implementation Analysis
**Research Date:** 2026-02-16  
**Files Analyzed:** manifest.json, service-worker.ts, app.html, svelte.config.js, PWAInstallManager.ts, PushNotificationManager.ts

**Manifest Analysis (`static/manifest.json`):**
- ✅ Has all required PWA members (name, short_name, start_url, display, scope, theme_color, background_color)
- ✅ Share target configured correctly (GET /share with title/text/url params)
- ⚠️ Icons reference `/favicon.png` but file does NOT exist in static folder
- ⚠️ Uses same icon path for both 192x192 and 512x512 sizes
- ℹ️ Missing optional but recommended members: `description`, `screenshots`, `categories`

**Service Worker Analysis (`src/service-worker.ts`):**
- ✅ Native SvelteKit service worker (migrated from vite-pwa plugin)
- ✅ Install event: caches all build assets and static files
- ✅ Activate event: cleans up old caches
- ✅ Fetch event: cache-first for assets, network-first with cache fallback for others
- ✅ Push event handler: processes push messages, shows notifications with actions
- ✅ Notification click handler: opens/focuses app, handles action buttons
- ✅ Notification close handler: tracks dismissals
- ✅ Background sync handler: supports retry operations
- ✅ Message handler: supports service worker communication
- ✅ Global error handlers present

**Service Worker Registration (`svelte.config.js`):**
- ✅ `serviceWorker.register: true` enabled
- ✅ SvelteKit handles registration automatically

**Manifest Link (`src/app.html`):**
- ✅ `<link rel="manifest" href="/manifest.json">` present in head

**Client-Side Managers:**
- ✅ `PushNotificationManager.ts`: Full implementation with permission, subscribe, unsubscribe
- ✅ `PWAInstallManager.ts`: beforeinstallprompt handling, install prompt triggering
- ✅ Both are SSR-safe with browser guards

**Share Target (`/share` route):**
- ✅ Route exists at `src/routes/share/+page.svelte`
- ✅ Parses query params (text, url) from share target
- ✅ Extracts Instagram URLs from shared text
- ✅ Auto-processes URLs on mount
- ✅ Enqueues items and redirects to dashboard

**Icons/Assets Issue:**
- ⚠️ **CRITICAL**: `manifest.json` references `/favicon.png` but file doesn't exist
- ✅ `src/lib/assets/favicon.svg` exists (used in layout)
- ⚠️ No PNG icons in `static/` folder
- ⚠️ Service worker references `/favicon.png` for notifications

**Push Notifications Infrastructure:**
- ✅ VAPID keys configured in `queueConfig.push` (uses env vars or defaults)
- ✅ Server endpoint: `/api/notifications/vapid-key` (GET)
- ✅ Server endpoint: `/api/notifications/subscribe` (POST/DELETE)
- ✅ PushNotificationService stores subscriptions in-memory
- ℹ️ Note: Subscriptions are not persisted (lost on restart)

#### What Works Already:
1. **PWA Structure**: Complete Native SvelteKit PWA implementation
2. **Service Worker**: Fully functional with caching, push, notifications
3. **Push Notifications**: Client and server infrastructure in place
4. **Share Target**: Configured in manifest and `/share` route working
5. **Install Prompts**: PWAInstallManager ready to trigger install
6. **HTTPS**: App served at https://localhost:5173/

#### What Needs Attention:
1. **Icons**: Create PNG icons (192x192, 512x512) from existing SVG
2. **Icon Verification**: Ensure icons are properly sized and optimized
3. **Installability Testing**: Verify all criteria met via chrome://pwa-internals
4. **Push Notification Testing**: Verify VAPID key generation and push flow
5. **Share Target Testing**: Test share from external apps (Instagram)
6. **Manifest Enhancement**: Add description, categories for better discoverability

#### Dependencies & Constraints (from ARCHITECTURE.md, CODE_STYLE.md):
- Using native SvelteKit PWA (no plugins needed)
- Service worker: `$service-worker` module provides build, files, version
- Environment: uses `$env/dynamic/private` for server configs
- HTTPS required (already configured at https://localhost:5173/)
- TypeScript strict mode enabled
- All file paths must use SvelteKit path aliases (`$lib`, `$service-worker`)

#### Code Style Requirements (from CODE_STYLE.md):
- FilesNaming: manifest.json, service-worker.ts, lowercase for utilities
- Type annotations required for public APIs
- SSR-safe code: all browser API usage must be guarded with `browser` check
- Error handling: try-catch with descriptive messages
- Comments: JSDoc for public APIs, inline for complex logic

---

### [Planner] Research Notes - RECIPE-0003 (2026-02-16)

**Task:** Update application icon and configure Docker deployment

#### PWA Icon Generation - icon-source.png
**Research Date:** 2026-02-16  
**Source:** Project analysis, PWA best practices, sharp documentation

**Icon Source File:**
- Location: `static/icon-source.png`
- Size: 672KB PNG file
- Format: PNG with transparency (confirmed via file analysis)
- Destination sizes: 192x192 (favicon.png), 512x512 (icon-512.png)

**PWA Icon Requirements:**
From RECIPE-0002 research and W3C Web App Manifest specification:
1. **Minimum Size**: 192x192 pixels (required for PWA installability)
2. **Recommended Size**: 512x512 pixels (for splash screens, high-DPI displays)
3. **Format**: PNG with transparency support
4. **Purpose**: "any maskable" for optimal Android compatibility
5. **Location**: static/ directory (served at root path)

**Sharp Library Configuration:**
- Version: 0.34.5 (already in dependencies)
- Method: resize() with fit: 'contain' to preserve aspect ratio
- Background: transparent (rgba 0,0,0,0)
- Format: PNG with optimization
- Quality: Default compression for web delivery

**Implementation Pattern:**
```javascript
await sharp('static/icon-source.png')
  .resize(192, 192, {
    fit: 'contain',
    background: { r: 0, g: 0, b: 0, alpha: 0 }
  })
  .png()
  .toFile('static/favicon.png');
```

**Rationale:**
- `fit: 'contain'` preserves aspect ratio without cropping
- Transparent background maintains icon transparency
- PNG format required by Web App Manifest spec
- Same approach for both 192x192 and 512x512 variants

---

#### Docker Volume Configuration
**Research Date:** 2026-02-16  
**Source:** Codebase analysis, Dockerfile, scheduler.ts, extraction.ts

**Volume Requirements Analysis:**
From code analysis, only one persistent volume is required:

**1. /app/secrets - Instagram Authentication Storage**
- **Purpose**: Persist Instagram session cookies across container restarts
- **File**: auth.json (Playwright storage state)
- **Usage**: 
  - scheduler.ts: Checks `/app/secrets/auth.json` for Docker deployments
  - extraction.ts: Loads authentication from `/app/secrets/auth.json`
  - gen-auth.js: Browser automation saves session to secrets/auth.json
- **Rationale**: Prevents re-login on every container restart
- **Docker Path**: /app/secrets
- **Host Path**: ./secrets (relative to docker-compose.yml)

**Volumes NOT Required:**
- **Database**: Queue uses in-memory storage (QueueManager.ts)
- **Cache**: Service worker cache is ephemeral
- **Uploads**: No file upload functionality
- **Logs**: Console logs to stdout/stderr (Docker logging)
- **Build artifacts**: Built into image at build time

**VOLUME Directive:**
```dockerfile
VOLUME ["/app/secrets"]
```

**docker-compose.yml Volume Mount:**
```yaml
volumes:
  - ./secrets:/app/secrets
```

---

#### Environment Variable Inventory
**Research Date:** 2026-02-16  
**Source:** queue/config.ts, llm.ts, tandoor-config.ts, scheduler.ts

**Comprehensive Variable List:**

**LLM Configuration (REQUIRED):**
- `OPENAI_BASE_URL` - OpenAI-compatible API endpoint
- `OPENAI_API_KEY` - API authentication key
- `LLM_MODEL` - Model identifier (default: gpt-4o)

**Queue Configuration (OPTIONAL):**
- `QUEUE_CONCURRENCY` - Parallel processing limit (default: 2)
- `QUEUE_MAX_RETRIES` - Retry attempts (default: 3)

**Tandoor Integration (OPTIONAL):**
- `TANDOOR_ENABLED` - Enable Tandoor upload (default: false)
- `TANDOOR_SERVER_URL` - Tandoor base URL
- `TANDOOR_SPACE` - Space ID (default: 1)
- `TANDOOR_TOKEN` - API token

**Push Notifications (OPTIONAL):**
- `VAPID_PUBLIC_KEY` - Web Push public key (has default)
- `VAPID_PRIVATE_KEY` - Web Push private key (has default)

**Authentication Scheduler (OPTIONAL):**
- `AUTH_SCHEDULER_ENABLED` - Enable auto-renewal (default: false)
- `AUTH_SCHEDULER_INTERVAL_MINUTES` - Renewal interval (default: 720)

**Runtime Configuration:**
- `NODE_ENV` - Environment mode (production/development)
- `PORT` - SvelteKit port (default: 3000)
- `DISPLAY` - X11 display for Playwright (set to :99 in docker-compose.yml)

**Default Values:**
All variables have sensible defaults except:
- OPENAI_BASE_URL (required)
- OPENAI_API_KEY (required)

**VAPID Keys:**
Current defaults in queue/config.ts:
- Public: BNextdcB_fQ0BVvyGioM5L8Tf9vKQjs-WnF-rUbnU8MdWIZQYfggIHxBnW21I-lq_0HykLCdMpYj8d5joavWdxQ
- Private: JwxI_KcsBcehYcTOufMcbVWJjCq1QbH5FJmSyQuG680
- Note: These should be regenerated for production deployments

**Variable Access Pattern:**
- Server-side only: Uses `$env/dynamic/private` from SvelteKit
- No client-side environment variable exposure
- Runtime configuration (no build-time substitution)

---

#### Docker Health Check Configuration
**Research Date:** 2026-02-16  
**Source:** routes/api/health/+server.ts analysis

**Health Check Endpoint:**
- Path: `/api/health`
- Method: GET
- Response: 200 OK with JSON body
- Implementation: `src/routes/api/health/+server.ts`

**Health Check Response:**
```json
{
  "status": "ok",
  "timestamp": "2026-02-16T..."
}
```

**Docker Health Check Configuration:**
```yaml
healthcheck:
  test: ["CMD", "node", "-e", "fetch('http://localhost:3000/api/health').then(r => r.ok ? process.exit(0) : process.exit(1)).catch(() => process.exit(1))"]
  interval: 30s
  timeout: 10s
  retries: 3
  start_period: 40s
```

**Rationale:**
- `interval: 30s` - Balance between responsiveness and overhead
- `timeout: 10s` - Sufficient for app initialization
- `retries: 3` - Allow transient failures
- `start_period: 40s` - Accounts for Playwright browser initialization
- Uses internal fetch to avoid curl dependency

---

#### Docker Deployment Constraints
**Research Date:** 2026-02-16  
**Source:** Dockerfile, app.server.ts, browser.ts

**Current Dockerfile Analysis:**
- Base: node:22-alpine (minimal, production-ready)
- Chromium: Installed via apk (headless browser for Instagram extraction)
- Fonts: liberation-fonts, noto, noto-cjk (text rendering)
- Build: npm ci + npm run build
- Runtime: Node.js ESM import
- Port: 3000 (EXPOSE)
- Environment: NODE_ENV=production

**Browser Initialization:**
From app.server.ts:
- initializeBrowser() called on server start
- Graceful shutdown handlers (SIGTERM, SIGINT)
- Critical for extraction.ts Playwright usage

**Security Options:**
- `seccomp=unconfined` - Required for Chromium sandbox
- `--no-sandbox` in browser.ts launch args
- Necessary for containerized Chromium

**No Changes Required:**
Current Dockerfile is production-ready, only needs VOLUME addition.

---

### [Planner] Research Notes - RECIPE-0003 Iteration 1 (2026-02-16)

**Task:** Fix Docker deployment issues (Alpine packages, Playwright installation)

#### Alpine Linux Font Packages
**Research Date:** 2026-02-16  
**Source:** https://wiki.alpinelinux.org/wiki/Fonts, Alpine package database

**Incorrect Package Names in Current Dockerfile:**
1. `liberation-fonts` → No such package (ERROR)
2. `noto` → No such package (ERROR)
3. `noto-cjk` → No such package (ERROR)

**Correct Alpine Font Package Names:**
1. `font-liberation` → Correct (already in Dockerfile)
2. `font-noto` → Correct name for Noto fonts
3. `font-noto-cjk` → Correct name for Noto CJK (Chinese, Japanese, Korean) fonts

**Rationale:**
- Alpine Linux uses `font-*` prefix for all font packages
- Common mistake: using Debian/Ubuntu package names which differ from Alpine
- These fonts are essential for rendering text in Instagram content extraction

**Recommended Font Installation:**
```dockerfile
RUN apk add --no-cache \
    chromium \
    font-liberation \
    font-noto \
    font-noto-cjk
```

---

#### Playwright on Alpine Linux
**Research Date:** 2026-02-16  
**Source:** https://playwright.dev/docs/docker, Playwright GitHub issues

**Official Playwright + Alpine Status:**
- **Not officially supported**: Browser builds require glibc, Alpine uses musl
- **Firefox/WebKit**: Cannot run on Alpine (glibc dependency)
- **Chromium**: Can work using system chromium package

**Problem Analysis:**
- Current Dockerfile installs system chromium via `apk add chromium`
- Playwright's `chromium.launch()` expects Playwright's own Chromium binary
- Playwright's Chromium is built for glibc environments (Ubuntu/Debian)
- `npx playwright install chromium` will download glibc binary that won't run on Alpine

**Solution: Configure Playwright to Use System Chromium**

**Approach A - Use System Chromium (Recommended):**
```typescript
// src/lib/server/browser.ts
browser = await chromium.launch({
  executablePath: '/usr/bin/chromium-browser',
  headless: true,
  args: [...]
});
```

**Environment Variable Approach:**
```dockerfile
ENV PLAYWRIGHT_SKIP_BROWSER_DOWNLOAD=1
ENV PLAYWRIGHT_CHROMIUM_EXECUTABLE_PATH=/usr/bin/chromium-browser
```

**Approach B - Switch to Debian Base:**
```dockerfile
FROM node:22-bookworm
RUN npx -y playwright@1.56.1 install --with-deps chromium
```

**Recommendation:**
- Use Approach A (system chromium with executablePath)
- Minimal changes to existing Alpine setup
- System chromium is already installed and working
- Avoids full base image migration

**Chromium System Dependencies:**
When using system chromium on Alpine, these packages are auto-installed as dependencies:
- ca-certificates, mesa-gbm, wayland-libs-server, libxkbcommon
- ffmpeg-libs, gtk+3.0, libexif, libevent, nss, etc. (64 total dependencies)

---

#### Playwright Version Compatibility
**Research Date:** 2026-02-16  
**Source:** package.json analysis

**Current Version:** playwright@1.56.1 (production dependency)
**Chromium Version:** Bundled with Playwright 1.56.1

**System Chromium Compatibility:**
- Alpine edge: chromium 145.0.7632.75 (as of 2026-02-15)
- Playwright 1.56.1 expects: Chromium ~133.x
- **Version mismatch OK**: Playwright API is compatible across minor Chromium versions
- System chromium is newer, should work without issues

**executablePath Configuration:**
- Path on Alpine: `/usr/bin/chromium-browser`
- Must be set in browser.ts or via environment variable
- No additional Playwright installation needed when using system browser

---

#### Docker Compose Configuration for Playwright
**Research Date:** 2026-02-16  
**Source:** resolution_context.yaml, docker-compose.yml analysis

**Current Configuration Analysis:**
```yaml
environment:
  - DISPLAY=:99  # X11 display (not needed for headless)
security_opt:
  - seccomp=unconfined  # Required for Chromium sandbox
```

**Issues:**
- `DISPLAY=:99` set but no X11 server (Xvfb) running
- Headless mode doesn't need DISPLAY
- docker-compose.yml has DISPLAY but it's unused

**Recommendation:**
- Keep `DISPLAY=:99` as harmless fallback (no changes needed)
- `seccomp=unconfined` is necessary for Chromium sandbox (keep as-is)
- No additional configuration needed for Playwright

---

---

### [Planner] Node.js Versions and npm Lockfile Compatibility - RECIPE-0003 Iteration 2 (2026-02-16)

**Research Date:** 2026-02-16T17:00:00.000Z  
**Source:** Node.js Release Schedule, npm documentation (v10 & v11), Docker Hub

#### Problem Analysis
Docker build fails at `npm ci` with error: "package-lock.json and package.json are out of sync"
- **Root Cause**: package.json updated to Tailwind v4, but package-lock.json still contains Tailwind v3 dependencies (@csstools/*)
- **Secondary Issue**: npm version mismatch - local (npm 11.6.2) vs Docker (npm 10.9.4)

#### Node.js LTS Status Research
**Source:** https://github.com/nodejs/release, https://nodejs.org/en/about/previous-releases

**Currently Supported Versions:**
- **Node.js 20 (Iron)**: Maintenance LTS - EOL 2026-04-30
- **Node.js 22 (Jod)**: Maintenance LTS - EOL 2027-04-30 ← Current Dockerfile
- **Node.js 24 (Krypton)**: Active LTS - EOL 2028-04-30 ← Best choice
- **Node.js 25**: Current (not LTS) - EOL 2026-06-01

**LTS Phase Definitions:**
1. **Current**: Latest features, 6-month cycle for odd versions
2. **Active LTS**: Audited features and updates (18 months for even versions since v12)
3. **Maintenance**: Critical fixes only (12 months)

**Conclusion**: Node.js 24 is Active LTS (until Oct 2026) providing better support than Node.js 22 (already in Maintenance).

#### npm Lockfile Version Compatibility
**Source:** https://docs.npmjs.com/cli/v10/configuring-npm/package-lock-json, https://docs.npmjs.com/cli/v11/configuring-npm/package-lock-json

**Lockfile Version History:**
- `lockfileVersion: 1` - npm v5-v6
- `lockfileVersion: 2` - npm v7-v8 (backwards compatible with v1)
- `lockfileVersion: 3` - npm v9+ (backwards compatible with v7)

**npm Version Bundled with Node.js:**
- node:22-alpine → npm 10.9.4 (uses lockfileVersion: 3)
- node:24-alpine → npm 11.x (uses lockfileVersion: 3)
- Local environment → npm 11.6.2 (uses lockfileVersion: 3)

**Compatibility Analysis:**
- Current package-lock.json has `"lockfileVersion": 3` ✓
- npm 10 and npm 11 both support lockfileVersion: 3 ✓
- The issue is NOT version incompatibility but **stale dependency data**

**npm ci Strict Behavior:**
`npm ci` performs strict validation:
1. Requires exact match between package.json and package-lock.json
2. Does not update lockfile automatically (unlike `npm install`)
3. Fails if dependencies are missing or mismatched
4. This is intentional for reproducible builds in CI/CD

#### Tailwind CSS v3 → v4 Migration Impact
**Source:** package.json analysis, package-lock.json inspection

**Current State:**
```json
// package.json (Tailwind v4)
"@tailwindcss/vite": "^4.1.17",
"tailwindcss": "^4.1.17"

// package-lock.json (still has Tailwind v3 transitive deps)
"@csstools/css-parser-algorithms": "3.0.5",
"@csstools/css-tokenizer": "3.0.4"
```

**Why This Happened:**
- package.json was updated to Tailwind v4
- package-lock.json was NOT regenerated afterward
- Tailwind v4 has different dependency tree than v3 (no @csstools/*)
- `npm ci` detects mismatch and fails

#### Solution Options Analysis

**Option A: Regenerate with Docker node:22-alpine (Review's RECOMMENDED)**
```bash
docker run --rm -v "$PWD":/app -w /app node:22-alpine sh -c "rm package-lock.json && npm install"
```
- ✓ Ensures exact npm version match with deployment
- ✗ Stays on Maintenance LTS (Node 22)
- ✗ Doesn't align with local development (node 24)

**Option B: Update to node:24-alpine**
```dockerfile
FROM node:24-alpine
```
```bash
rm package-lock.json && npm install
```
- ✓ Uses Active LTS (better support)
- ✓ Aligns Docker with local development
- ✗ Changes base image (minimal risk)

**Option C: Hybrid (BEST SOLUTION)**
1. Update Dockerfile to node:24-alpine
2. Regenerate package-lock.json locally (npm 11.x matches node:24)
- ✓ Active LTS with longer support window
- ✓ Perfect alignment between local dev and Docker
- ✓ Single lockfile regeneration
- ✓ Future-proof (Active LTS until Oct 2026)

**Chosen Approach: Option C**

#### Implementation Details

**Files to Modify:**
1. `Dockerfile` - Change FROM node:22-alpine → node:24-alpine
2. `package-lock.json` - Regenerate to sync with package.json

**Verification Steps:**
1. `npm install` - Regenerate lockfile
2. `npm run build` - Verify local build
3. `npm test` - Verify all tests pass
4. `docker build` - Verify Docker build succeeds
5. `docker compose up` - Verify runtime

**No Code Changes Needed:**
- All application code remains unchanged
- .env.example already complete (no new variables)
- docker-compose.yml does not need changes (node version transparent)

---

### [Planner] Research Notes - RECIPE-0004 (2026-02-16)

**Task:** Fix .dockerignore, favicon.ico, push notifications, e2e tests, and logging serialization

#### .dockerignore Research
**Research Date:** 2026-02-16  
**Source:** Project analysis, .gitignore comparison, Docker best practices

**Current State:**
- No `.dockerignore` file exists in project root
- `.gitignore` exists and excludes: node_modules, build outputs, env files, SSL certs, symlinks, prompts/

**Docker Build Context Issues:**
Without `.dockerignore`, Docker sends entire workspace to build context including:
- `node_modules/` (if exists locally) - causes conflicts with `npm ci` in Dockerfile
- `build/` outputs - unnecessary
- `.git/` directory - large, unused in container
- `prompts/` directory - development artifacts
- `.env` files - should use environment variables instead

**Recommended .dockerignore Content:**
Based on `.gitignore` and Docker best practices:
```dockerignore
node_modules
.git
build
.output
.vercel
.netlify
.wrangler
.svelte-kit
.DS_Store
Thumbs.db
.env
.env.*
!.env.example
.ssl/
vite.config.*.timestamp-*
debug_page.txt
prompts/
*.md
!README.md
.github/
.vscode/
*.log
coverage/
.vitest/
```

**Rationale:**
- Exclude development dependencies and build artifacts
- Keep README.md for documentation
- Exclude version control metadata
- Reduce build context size significantly
- Prevent conflicts with Dockerfile's npm ci

---

#### Favicon 404 Error Research
**Research Date:** 2026-02-16  
**Source:** Static folder analysis, browser behavior, PWA specifications

**Files Present:**
- `static/favicon.png` (192x192 PNG) ✓ exists
- `static/icon-512.png` (512x512 PNG) ✓ exists
- `static/icon-source.png` (source file) ✓ exists
- `static/manifest.json` references both PNG files ✓

**404 Source:**
- Browsers automatically request `/favicon.ico` (legacy format)
- SvelteKit serves from `static/` folder
- No `favicon.ico` file exists → 404 error

**Solution Options:**

**Option A - Create favicon.ico (Recommended):**
Use Sharp to generate ICO from PNG source:
```javascript
// New script: scripts/gen-favicon-ico.js
await sharp('static/icon-source.png')
  .resize(32, 32)
  .png()
  .toFile('static/favicon.ico');
```

**Option B - SvelteKit Hook Redirect:**
Add server hook to redirect /favicon.ico → /favicon.png
- More complex
- Adds runtime overhead
- Not recommended

**Chosen Approach:** Option A (generate favicon.ico during build)

---

#### Push Notifications Implementation Research
**Research Date:** 2026-02-16  
**Source:** PushNotificationService.ts, web-push library docs, Web Push Protocol RFC 8030

**Current Implementation Analysis:**

**Client-Side (Complete):**
- `PushNotificationManager.ts` - Full implementation ✓
  - Permission request ✓
  - VAPID key fetch ✓
  - pushManager.subscribe() ✓
  - Server subscription registration ✓
- `service-worker.ts` - Push event handler ✓
- `NotificationSettings.svelte` - UI toggle ✓

**Server-Side (Mock Only):**
```typescript
// Current PushNotificationService.ts line 106-125
private async sendToSubscription(subscription: PushSubscription, data: any): Promise<void> {
  // In production, use web-push library:
  // [COMMENTED OUT CODE]
  
  // For development, we'll log the notification
  console.log(`[PushService] Would send push notification:`, {
    endpoint: subscription.endpoint,
    data: data
  });
  
  await new Promise(resolve => setTimeout(resolve, 100)); // Simulate
}
```

**Problem:** Push notifications are logged but never actually sent to browser.

**Web Push Library Integration:**

**1. Install Dependency:**
```json
// package.json
{
  "dependencies": {
    "web-push": "^3.6.7"
  }
}
```

**2. Implementation Pattern:**
```typescript
import webpush from 'web-push';

// On init
webpush.setVapidDetails(
  'mailto:your-email@example.com',
  vapidPublicKey,
  vapidPrivateKey
);

// In sendToSubscription
await webpush.sendNotification(
  subscription,
  JSON.stringify(payload),
  {
    TTL: 60 * 60 * 24 // 24 hours
  }
);
```

**3. Configuration Requirements:**
- VAPID keys already configured in `queueConfig.push`
- Default keys present (should regenerate for production)
- Email contact required by spec (add env var)

**Files to Modify:**
- `package.json` - add web-push dependency
- `src/lib/server/notifications/PushNotificationService.ts` - implement actual sending
- `src/lib/server/queue/config.ts` - add VAPID_EMAIL env var

---

#### Manual Push Notification Test Button Research
**Research Date:** 2026-02-16  
**Source:** NotificationSettings.svelte, PushNotificationService API

**Current UI:**
- Only has enable/disable toggle
- No manual trigger for testing different notification types

**Test Button Requirements:**
1. Trigger different notification types:
   - Success notification (recipe completed)
   - Error notification (parsing failed)
   - Progress notification (extraction in progress)
2. Send to own subscription only
3. Debug output showing notification payload

**Implementation Approach:**

**Frontend Component:**
Add to `NotificationSettings.svelte`:
```svelte
<button onclick={testNotification('success')}>Test Success</button>
<button onclick={testNotification('error')}>Test Error</button>
<button onclick={testNotification('progress')}>Test Progress</button>

async function testNotification(type: 'success' | 'error' | 'progress') {
  await fetch('/api/notifications/test', {
    method: 'POST',
    headers: { 'Content-Type': 'application/json' },
    body: JSON.stringify({ type })
  });
}
```

**Backend Endpoint:**
New file: `src/routes/api/notifications/test/+server.ts`
```typescript
export const POST: RequestHandler = async ({ request }) => {
  const { type } = await request.json();
  
  const payload = {
    success: { /* ... */ },
    error: { /* ... */ },
    progress: { /* ... */ }
  }[type];
  
  await pushNotificationService.sendNotification(payload);
  return json({ success: true });
};
```

---

#### Playwright E2E Push Notification Testing Research
**Research Date:** 2026-02-16  
**Source:** Playwright API docs (BrowserContext.grantPermissions), existing test patterns

**Playwright Push Notification Testing Pattern:**

**Key Methods:**
1. `context.grantPermissions(['notifications'])` - Grant permission without prompt
2. `page.evaluate()` - Access PushManager in browser context
3. `page.waitForEvent()` - Wait for service worker events

**Test Structure:**
```typescript
// New file: src/tests/push-notifications.e2e.spec.ts
import { test, expect } from '@playwright/test';

test.describe('Push Notifications E2E', () => {
  test('should subscribe to push notifications', async ({ browser }) => {
    const context = await browser.newContext();
    await context.grantPermissions(['notifications']);
    
    const page = await context.newPage();
    await page.goto('http://localhost:5173');
    
    // Click notification toggle
    await page.getByRole('button', { name: /enable notifications/i }).click();
    
    // Verify subscription created
    const subscription = await page.evaluate(async () => {
      const reg = await navigator.serviceWorker.ready;
      return await reg.pushManager.getSubscription();
    });
    
    expect(subscription).toBeTruthy();
    expect(subscription.endpoint).toBeDefined();
    
    await context.close();
  });
});
```

**Test Coverage:**
1. Permission grant flow
2. Subscription creation via PushManager
3. Server registration (POST /api/notifications/subscribe)
4. Manual test notification trigger
5. Subscription persistence in localStorage
6. Unsubscribe flow

**Vitest Configuration:**
Current project uses Vitest with @vitest/browser-playwright:
- Already configured for browser tests
- Playwright already installed (playwright@^1.56.1)
- Pattern: `*.e2e.spec.ts` for e2e tests vs `*.spec.ts` for unit tests

---

#### Logging Serialization Research
**Research Date:** 2026-02-16  
**Source:** Codebase grep analysis, Node.js console behavior, error object structure

**Problem Analysis:**

**Root Cause:**
JavaScript error objects logged directly show `[object Object]`:
```typescript
// Current pattern (WRONG)
console.error('[Label]', error);  // Output: [Label] [object Object]
console.log('[Label]', data);     // Output: [Label] [object Object]
```

**Affected Files (25 matches found):**
- `src/lib/server/extraction.ts` - 12 occurrences
- `src/lib/server/parser.ts` - 4 occurrences
- `src/lib/server/queue/QueueProcessor.ts` - 3 occurrences
- `src/lib/server/notifications/PushNotificationService.ts` - 1 occurrence
- `src/lib/server/api/errorHandler.ts` - 1 occurrence
- `src/lib/server/llm.ts` - 2 occurrences
- `src/lib/server/scheduler.ts` - 1 occurrence
- Others: QueueManager.ts, tandoor.ts

**Solution Patterns:**

**1. Error Objects:**
```typescript
// GOOD - Extract relevant properties
console.error('[Label]', error.message, error.stack);
console.error('[Label] Error:', {
  message: error.message,
  stack: error.stack,
  name: error.name
});
```

**2. Complex Objects:**
```typescript
// GOOD - JSON.stringify with formatting
console.log('[Label] Data:', JSON.stringify(data, null, 2));

// GOOD - Specific properties
console.log('[Label] Response:', {
  status: response.status,
  statusText: response.statusText,
  body: responseBody
});
```

**3. Utility Function:**
Create `src/lib/server/utils/logger.ts`:
```typescript
export function serializeError(error: unknown): string {
  if (error instanceof Error) {
    return JSON.stringify({
      name: error.name,
      message: error.message,
      stack: error.stack,
      ...error
    }, null, 2);
  }
  return JSON.stringify(error, null, 2);
}

console.error('[Label]', serializeError(error));
```

**Testing Impact:**
- Logs are visible in Docker deployments (stdout/stderr)
- JSON format easier for log aggregation tools
- Stack traces preserved for debugging
- Human-readable in console

---

### [Planner] Research Notes - RECIPE-0004 Iteration 1 (2026-02-17)

**Task:** Fix TypeScript type error - NodeJS.Timer should be NodeJS.Timeout in scheduler.ts

#### Node.js Timer Types Research
**Research Date:** 2026-02-17  
**Source:** Node.js v25.6.1 Official Documentation (https://nodejs.org/docs/latest/api/timers.html)

**Problem Analysis:**
TypeScript compile error in `src/lib/server/scheduler.ts:180`:
```
Argument of type 'Timer' is not assignable to parameter of type 'Timeout'
Type 'Timer' is missing the following properties from type 'Timeout': 
  close, _onTimeout, [Symbol.dispose]
```

**Root Cause:**
The `SchedulerState` interface incorrectly uses `NodeJS.Timer` type for `intervalId`, but `setInterval()` returns `NodeJS.Timeout` and `clearInterval()` expects `NodeJS.Timeout` parameter.

**Official Node.js API Documentation:**

**Class: Timeout**
- Returned by `setInterval()` and `setTimeout()`
- Can be passed to `clearInterval()` or `clearTimeout()`
- Has methods: `ref()`, `unref()`, `hasRef()`, `close()`, `refresh()`, `[Symbol.toPrimitive]()`, `[Symbol.dispose]()`
- TypeScript type: `NodeJS.Timeout`

**API Signatures:**
```typescript
// setInterval returns Timeout
function setInterval(
  callback: Function, 
  delay?: number, 
  ...args: any[]
): NodeJS.Timeout;

// clearInterval expects Timeout
function clearInterval(
  timeout: NodeJS.Timeout | string | number
): void;
```

**NodeJS.Timer Type:**
- Deprecated/incorrect type for timer return values
- Missing required properties: `close`, `_onTimeout`, `[Symbol.dispose]`
- Should NOT be used for `setInterval()`/`setTimeout()` return types
- Causes TypeScript strict mode errors when passed to `clearInterval()`

**Codebase Analysis:**
```
grep -r "NodeJS.Timer" src/
  src/lib/server/scheduler.ts:13    intervalId: NodeJS.Timer | null;
  src/tests/fixtures.ts:151         let timers: NodeJS.Timer[] = [];

grep -r "NodeJS.Timeout" src/
  src/routes/api/queue/stream/+server.ts:54    let keepAliveInterval: NodeJS.Timeout | null = null;
```

**Findings:**
1. **Incorrect usage (2 occurrences):**
   - `src/lib/server/scheduler.ts:13` — SchedulerState interface
   - `src/tests/fixtures.ts:151` — Timer array in test helper

2. **Correct usage (1 occurrence):**
   - `src/routes/api/queue/stream/+server.ts:54` — keepAliveInterval type

**Solution:**
Change all `NodeJS.Timer` to `NodeJS.Timeout` to align with Node.js official API contracts and TypeScript type definitions.

**Files to Modify:**
1. `src/lib/server/scheduler.ts:13` — Type in SchedulerState interface
2. `src/tests/fixtures.ts:151` — Type in createTimerSpy helper

**Impact:**
- Type-only change, no runtime behavior modification
- Fixes TypeScript strict mode compile error
- Aligns codebase with Node.js standard types
- Existing tests (260 total) already provide 100% coverage

**References:**
- Node.js Timers Documentation: https://nodejs.org/docs/latest/api/timers.html#class-timeout
- TypeScript @types/node package: Official Node.js type definitions
- Related Error: RECIPE-0004 iteration 0 review_report.yaml

---

**Document Version:** 1.6  
**Last Updated by:** Planner Agent (RECIPE-0004 Iteration 1)  
**Next Update:** Developer Agent