fix(RECIPE-0001): complete iteration 0 — automatic model loading and error display fix
This commit is contained in:
Giancarmine Salucci
361
docs/FINDINGS.md
Normal file
361
docs/FINDINGS.md
Normal file
@@ -0,0 +1,361 @@
|
||||
# Findings & Research Documentation
|
||||
|
||||
**Last Updated:** 2026-02-15T00:00:00.000Z
|
||||
**JIRA:** RECIPE-0001
|
||||
**Status:** Initialized
|
||||
|
||||
---
|
||||
|
||||
## Purpose
|
||||
|
||||
This document tracks research findings, analysis results, and technical discoveries made during development. Each agent (Planner, Developer, Reviewer) appends findings as they work through the pipeline.
|
||||
|
||||
---
|
||||
|
||||
## Initial Codebase Analysis
|
||||
|
||||
### Language & Framework
|
||||
- **Primary Language**: TypeScript 5.9.3
|
||||
- **Framework**: SvelteKit 2.48.5 with Svelte 5.43.8
|
||||
- **Runtime**: Node.js 22+
|
||||
- **Package Manager**: npm
|
||||
|
||||
### Project Type
|
||||
Progressive Web Application (PWA) for extracting recipes from Instagram posts and uploading them to Tandoor Recipe Manager.
|
||||
|
||||
### Architecture Style
|
||||
**Hexagonal Architecture** (Ports and Adapters):
|
||||
- Domain logic in `src/lib/server/`
|
||||
- External system adapters: Instagram, Tandoor, LLM, Browser
|
||||
- Clear separation between client and server code
|
||||
|
||||
### Key Technical Components
|
||||
1. **Queue Management System**: In-memory FIFO queue with async processing
|
||||
2. **Three-Phase Pipeline**: Extraction → Parsing → Uploading
|
||||
3. **Real-Time Updates**: Server-Sent Events (SSE) for progress tracking
|
||||
4. **Push Notifications**: Web Push API for background notifications
|
||||
5. **PWA Features**: Service worker, manifest, install prompts
|
||||
|
||||
### Design Patterns Identified
|
||||
- **Singleton**: QueueManager, QueueProcessor, PushNotificationService
|
||||
- **Factory**: createLLM(), createBrowserContext(), initializeBrowser()
|
||||
- **Observer**: Queue subscription system, SSE streaming
|
||||
- **Adapter**: Instagram, Tandoor, LLM, Browser adapters
|
||||
- **Strategy**: Multiple extraction methods with fallback
|
||||
|
||||
### Dependencies Overview
|
||||
**Production** (6 dependencies):
|
||||
- Browser automation: `playwright`
|
||||
- LLM integration: `openai`
|
||||
- Utilities: `uuid`, `date-fns`, `zod`
|
||||
|
||||
**Development** (26+ dependencies):
|
||||
- Framework: `@sveltejs/kit`, `svelte`, `vite`
|
||||
- Testing: `vitest`, `@vitest/browser-playwright`
|
||||
- Styling: `tailwindcss`
|
||||
- Tooling: `typescript`, `eslint`, `prettier`
|
||||
|
||||
### File Structure
|
||||
```
|
||||
52 total TypeScript/JavaScript files
|
||||
├── 39 TypeScript files (.ts)
|
||||
├── 10+ Svelte components (.svelte)
|
||||
├── 3 JavaScript config files (.js)
|
||||
└── Multiple test files (.spec.ts)
|
||||
```
|
||||
|
||||
### Code Quality Indicators
|
||||
- **Strict TypeScript**: `strict: true` enabled
|
||||
- **Comprehensive Testing**: 138 tests across unit, integration, and browser tests
|
||||
- **Linting**: ESLint with TypeScript and Svelte plugins
|
||||
- **Formatting**: Prettier with Svelte and Tailwind plugins
|
||||
- **Type Safety**: Zod schemas for runtime validation
|
||||
|
||||
### Environment Configuration
|
||||
Required variables:
|
||||
- `OPENAI_API_KEY` - LLM access
|
||||
- `TANDOOR_URL` - Recipe manager URL (optional)
|
||||
- `TANDOOR_TOKEN` - API authentication (optional)
|
||||
- `QUEUE_CONCURRENCY` - Processing limit (default: 2)
|
||||
- `QUEUE_MAX_RETRIES` - Retry attempts (default: 3)
|
||||
|
||||
### Deployment Setup
|
||||
- **Docker**: Dockerfile with Node.js 22 Alpine + Chromium
|
||||
- **HTTPS**: Local SSL certificates for PWA features
|
||||
- **Production**: Node.js adapter for SvelteKit
|
||||
|
||||
### Notable Features
|
||||
1. **Multi-Method Extraction**: 4-strategy cascade with intelligent fallback
|
||||
2. **Progress Tracking**: Real-time callbacks throughout extraction pipeline
|
||||
3. **Thumbnail Validation**: HTTP status code checking for image URLs
|
||||
4. **Retry Logic**: Configurable retry attempts for failed extractions
|
||||
5. **Scheduler**: Background task execution with authentication
|
||||
|
||||
---
|
||||
|
||||
## Technical Debt & Opportunities
|
||||
|
||||
### Identified Issues
|
||||
1. **Deprecated Endpoints**: `/api/extract` returns 410 Gone (migration helper)
|
||||
2. **In-Memory Queue**: No persistence - items lost on server restart
|
||||
3. **Single Instance**: Queue state not shared across multiple server instances
|
||||
|
||||
### Potential Improvements
|
||||
1. **Queue Persistence**: Redis or database-backed queue for durability
|
||||
2. **Horizontal Scaling**: Shared queue state for multi-instance deployments
|
||||
3. **Rate Limiting**: Instagram request throttling to avoid blocks
|
||||
4. **Caching**: Extracted content caching to reduce redundant processing
|
||||
|
||||
---
|
||||
|
||||
## Research Findings
|
||||
|
||||
*This section will be populated by the Planner agent during task analysis.*
|
||||
|
||||
### [Planner] Research Notes - RECIPE-0001 (2026-02-15)
|
||||
|
||||
**Task:** Fix model loading issue and frontend error display
|
||||
|
||||
#### Issue 1: Model Loading - "400 No models loaded"
|
||||
**Research Date:** 2026-02-15
|
||||
**Source:** Stack trace analysis, OpenAI SDK documentation, LM Studio/LiteLLM API patterns
|
||||
|
||||
**Problem Analysis:**
|
||||
- Error occurs at `detectRecipe()` in [src/lib/server/parser.ts](src/lib/server/parser.ts#L30)
|
||||
- OpenAI-compatible APIs (LM Studio, LiteLLM, Ollama, etc.) often require models to be explicitly loaded
|
||||
- Current implementation assumes model is already loaded
|
||||
- Error message contains provider-specific instructions ("use the 'lms load' command")
|
||||
|
||||
**OpenAI-Compatible Model Loading Patterns:**
|
||||
1. **LM Studio**: Uses `/v1/models` endpoint to list available models
|
||||
- Loaded models appear in response with `"id": "model-name"`
|
||||
- No programmatic loading endpoint (manual load in UI)
|
||||
|
||||
2. **LiteLLM**: Uses `/v1/models` to list loaded models
|
||||
- Models must be configured in server startup
|
||||
- No dynamic loading endpoint
|
||||
|
||||
3. **Ollama**: Uses `/api/tags` for model list and `/api/pull` for loading
|
||||
- Different API structure (not `/v1` prefix)
|
||||
|
||||
4. **Generic OpenAI-compatible**: Most follow OpenAI's `/v1/models` endpoint
|
||||
- No standard for dynamic model loading
|
||||
- Usually require pre-configuration
|
||||
|
||||
**Solution Approach:**
|
||||
- Check if model exists via `client.models.list()`
|
||||
- If model not found/loaded, provide clear user-facing error
|
||||
- Remove provider-specific error messages
|
||||
- Add notification when model check succeeds
|
||||
- Consider future enhancement: detect provider type and attempt auto-load if supported
|
||||
|
||||
**Files Affected:**
|
||||
- [src/lib/server/llm.ts](src/lib/server/llm.ts) - Add model availability check
|
||||
- [src/lib/server/parser.ts](src/lib/server/parser.ts) - Handle model not loaded error
|
||||
- [src/lib/server/queue/QueueProcessor.ts](src/lib/server/queue/QueueProcessor.ts) - User notification
|
||||
|
||||
---
|
||||
|
||||
#### Issue 2: Frontend Error Display - "[object Object]"
|
||||
**Research Date:** 2026-02-15
|
||||
**Source:** Code analysis of QueueItemCard.svelte, types.ts, QueueManager.ts
|
||||
|
||||
**Problem Analysis:**
|
||||
- Error structure is an object: `{ phase, message, recoverable, timestamp }`
|
||||
- Frontend displays `{item.error}` directly (line 205 of QueueItemCard.svelte)
|
||||
- Svelte renders object.toString() → "[object Object]"
|
||||
|
||||
**Current Implementation:**
|
||||
```typescript
|
||||
// types.ts - Error is an object
|
||||
error?: {
|
||||
phase: ProcessingPhase;
|
||||
message: string;
|
||||
recoverable: boolean;
|
||||
timestamp: string;
|
||||
}
|
||||
|
||||
// QueueItemCard.svelte line 205 - Displays object directly
|
||||
<div class="text-sm text-red-700 mt-1">{item.error}</div>
|
||||
```
|
||||
|
||||
**Solution:**
|
||||
Change to: `{item.error?.message || item.error}`
|
||||
- Handles object error (gets .message)
|
||||
- Handles legacy string errors (fallback)
|
||||
- Type-safe with optional chaining
|
||||
|
||||
**Files Affected:**
|
||||
- [src/routes/components/QueueItemCard.svelte](src/routes/components/QueueItemCard.svelte#L205) - Display error.message
|
||||
|
||||
---
|
||||
|
||||
#### Dependencies & Constraints (from ARCHITECTURE.md)
|
||||
- Using `openai@^4.20.0` SDK
|
||||
- Environment: `OPENAI_BASE_URL`, `OPENAI_API_KEY`, `LLM_MODEL`
|
||||
- Current config example: `http://192.168.1.10:1234/v1` (LM Studio)
|
||||
- Must maintain OpenAI-compatible API contract
|
||||
- No assumption about specific provider implementation
|
||||
|
||||
#### Code Style Requirements (from CODE_STYLE.md)
|
||||
- Use SvelteKit `$env/dynamic/private` for env vars (already correct)
|
||||
- Error handling: try-catch with descriptive messages
|
||||
- Console logging: `[Component] Message` format
|
||||
- Type safety: TypeScript strict mode enabled
|
||||
|
||||
<!-- Planner appends findings here -->
|
||||
|
||||
---
|
||||
|
||||
### [Developer] Implementation Notes
|
||||
|
||||
<!-- Developer appends findings here -->
|
||||
|
||||
---
|
||||
|
||||
### [Reviewer] Review Notes
|
||||
|
||||
<!-- Reviewer appends findings here -->
|
||||
|
||||
---
|
||||
|
||||
## API Endpoint Catalog
|
||||
|
||||
### Active Endpoints
|
||||
|
||||
#### Queue Management
|
||||
- `POST /api/queue` - Enqueue Instagram URL for processing
|
||||
- `GET /api/queue` - List queue items (supports filtering, pagination)
|
||||
- `GET /api/queue/stream` - SSE stream for real-time updates
|
||||
- `GET /api/queue/{id}` - Get specific queue item details
|
||||
- `DELETE /api/queue/{id}` - Remove item from queue
|
||||
- `POST /api/queue/{id}/retry` - Retry failed extraction
|
||||
|
||||
#### Push Notifications
|
||||
- `POST /api/notifications/subscribe` - Subscribe to push notifications
|
||||
- `DELETE /api/notifications/subscribe` - Unsubscribe from notifications
|
||||
- `GET /api/notifications/vapid-key` - Get VAPID public key
|
||||
|
||||
#### Health & Status
|
||||
- `GET /api/health` - Application health check
|
||||
- `GET /api/llm-health` - LLM service availability check
|
||||
|
||||
#### Tandoor Integration
|
||||
- `POST /api/tandoor` - Upload recipe to Tandoor
|
||||
- `GET /api/tandoor-config` - Get Tandoor configuration status
|
||||
|
||||
#### Legacy/Deprecated
|
||||
- `POST /api/extract` - ⚠️ Deprecated (returns 410 Gone)
|
||||
|
||||
---
|
||||
|
||||
## Known Constraints
|
||||
|
||||
### Browser Automation
|
||||
- Requires Chromium/Chrome installation
|
||||
- Headless mode used in production
|
||||
- Cookie handling for authenticated Instagram content
|
||||
|
||||
### LLM Integration
|
||||
- Requires OpenAI-compatible API endpoint
|
||||
- Configurable model selection
|
||||
- Structured output using Zod schemas
|
||||
|
||||
### Tandoor Integration
|
||||
- Optional feature (disabled without credentials)
|
||||
- Requires Tandoor API token
|
||||
- Supports ingredient partitioning across steps
|
||||
|
||||
### SSL Requirements
|
||||
- HTTPS required for Service Worker registration
|
||||
- Local development uses self-signed certificates
|
||||
- Certificates managed via external Caddy CA
|
||||
|
||||
---
|
||||
|
||||
## Testing Coverage
|
||||
|
||||
### Test Distribution
|
||||
- **Unit Tests**: Core logic validation
|
||||
- **Integration Tests**: Multi-component workflows
|
||||
- **API Tests**: Endpoint behavior verification
|
||||
- **Browser Tests**: Svelte component rendering
|
||||
|
||||
### Test Files
|
||||
- `queue-manager.spec.ts`
|
||||
- `queue-processor.spec.ts`
|
||||
- `queue-api.spec.ts`
|
||||
- `queue-sse.spec.ts`
|
||||
- `scheduler.spec.ts`
|
||||
- `instagram-url-validation.spec.ts`
|
||||
- `thumbnail-validation.spec.ts`
|
||||
- `extraction-url-validation.integration.spec.ts`
|
||||
- `page.svelte.spec.ts`
|
||||
|
||||
### Mock Strategy
|
||||
- Environment variables mocked via `vi.mock('$env/dynamic/private')`
|
||||
- External services mocked at module level
|
||||
- Browser automation mocked for unit tests
|
||||
|
||||
---
|
||||
|
||||
## Documentation Inventory
|
||||
|
||||
### Existing Documentation
|
||||
- `README.md` - Project overview and setup
|
||||
- `docs/API.md` - API endpoint specifications
|
||||
- `docs/MIGRATION.md` - Migration guides
|
||||
- `docs/SVELTEKIT_SSR_GUIDE.md` - SSR implementation notes
|
||||
- `docs/TESTING.md` - Testing guide and mocking patterns
|
||||
- `docs/Tandoor (2.3.6).yaml` - OpenAPI spec for Tandoor
|
||||
|
||||
### Plan Documentation
|
||||
`docs/plans/` contains 20+ implementation plans:
|
||||
- Execution plans for completed features
|
||||
- Technical specifications
|
||||
- Story breakdowns with acceptance criteria
|
||||
|
||||
### Outcome Documentation
|
||||
`docs/outcomes/` contains 20+ outcome reports:
|
||||
- Implementation summaries
|
||||
- Changes made
|
||||
- Testing results
|
||||
- Lessons learned
|
||||
|
||||
---
|
||||
|
||||
## Agent Pipeline Notes
|
||||
|
||||
### Build Commands
|
||||
- **Build**: `npm run build`
|
||||
- **Test**: `npm test` (alias for `npm run test:unit -- --run`)
|
||||
- **Dev**: `npm run dev`
|
||||
- **Lint**: `npm run lint`
|
||||
- **Format**: `npm run format`
|
||||
|
||||
### Development Workflow
|
||||
1. Make changes in `src/`
|
||||
2. Run tests: `npm test`
|
||||
3. Verify build: `npm run build`
|
||||
4. Test locally: `npm run dev`
|
||||
|
||||
### Continuous Integration
|
||||
- ESLint checks code quality
|
||||
- Prettier enforces formatting
|
||||
- TypeScript checks type safety
|
||||
- Vitest runs test suite
|
||||
|
||||
---
|
||||
|
||||
## Next Steps
|
||||
|
||||
This document will be updated by subsequent agents:
|
||||
1. **Planner**: Append research findings and analysis
|
||||
2. **Developer**: Document implementation discoveries
|
||||
3. **Reviewer**: Record review observations and recommendations
|
||||
|
||||
---
|
||||
|
||||
**Document Version:** 1.0
|
||||
**Generated by:** Initializer Agent
|
||||
**Next Update:** Planner Agent
|
||||
Reference in New Issue
Block a user