diff --git a/docker-compose.yml b/docker-compose.yml index 8fe8341..43c57ab 100644 --- a/docker-compose.yml +++ b/docker-compose.yml @@ -2,13 +2,14 @@ services: app: build: . container_name: insta-recipe + network_mode: host ports: - "3000:3000" environment: # LLM Configuration (Required) - OPENAI_BASE_URL=${OPENAI_BASE_URL} - OPENAI_API_KEY=${OPENAI_API_KEY} - - LLM_MODEL=${LLM_MODEL:-gpt-4o} + - LLM_MODEL=${LLM_MODEL:-google/gemma-3-4b} # Queue Configuration (Optional) - QUEUE_CONCURRENCY=${QUEUE_CONCURRENCY:-2} diff --git a/docs/FINDINGS.md b/docs/FINDINGS.md index f2d97b4..6e278a7 100644 --- a/docs/FINDINGS.md +++ b/docs/FINDINGS.md @@ -1398,6 +1398,198 @@ Change all `NodeJS.Timer` to `NodeJS.Timeout` to align with Node.js official API --- -**Document Version:** 1.6 -**Last Updated by:** Planner Agent (RECIPE-0004 Iteration 1) +**Document Version:** 1.7 +**Last Updated by:** Planner Agent (RECIPE-0005 Iteration 0) +**Next Update:** Developer Agent + +--- + +### [Planner] Research Notes - RECIPE-0005 (2026-02-17) + +**Task:** Fix Playwright Docker dependencies and create LMStudio integration for E2E testing + +#### Playwright Alpine Linux Docker Integration - RECIPE-0005 +**Research Date:** 2026-02-17 +**Source:** FINDINGS.md (RECIPE-0003), Dockerfile analysis, browser.ts, Playwright documentation + +**Problem Analysis:** +- Container fails with: "Executable doesn't exist at /root/.cache/ms-playwright/chromium_headless_shell-1208/" +- Alpine Linux uses musl libc, Playwright's bundled browsers require glibc +- Current Dockerfile installs system chromium via `apk add chromium` but browser.ts doesn't specify executable path +- Playwright API defaults to searching for its own bundled browser binary (not present) + +**Solution (Already Researched in RECIPE-0003):** +Configure Playwright to use system chromium installed by Alpine APK: + +```typescript +// src/lib/server/browser.ts - initializeBrowser() +browser = await chromium.launch({ + executablePath: '/usr/bin/chromium-browser', // System chromium path + headless: true, + args: [ + '--disable-blink-features=AutomationControlled', + '--disable-dev-shm-usage', + '--no-sandbox', + '--disable-setuid-sandbox', + '--disable-gpu' + ] +}); +``` + +**Files to Modify:** +- `src/lib/server/browser.ts` - Add `executablePath: '/usr/bin/chromium-browser'` to launch options + +**No Changes Needed:** +- Dockerfile already has `chromium` and fonts installed correctly +- No need for `npx playwright install` (would fail on Alpine anyway) + +--- + +#### LMStudio Docker Networking - RECIPE-0005 +**Research Date:** 2026-02-17 +**Source:** Docker networking documentation, LMStudio API patterns, OpenAI-compatible endpoints + +**Problem:** +- LMStudio runs on host at `http://localhost:1234` +- Docker containers have isolated networking - `localhost` inside container != host `localhost` +- Container needs to access host services + +**Docker Networking Solutions:** + +**Option A - network_mode: host (Recommended for LMStudio):** +```yaml +services: + app: + network_mode: host +``` +- Container shares host network stack +- `localhost:1234` inside container = host's `localhost:1234` +- **Trade-off**: Loses container network isolation, port mapping ignored +- **Best for**: Local development/testing with host services + +**Option B - extra_hosts (Alternative):** +```yaml +services: + app: + extra_hosts: + - "host.docker.internal:host-gateway" + environment: + - OPENAI_BASE_URL=http://host.docker.internal:1234/v1 +``` +- Works on Docker Desktop (Mac/Windows) and Linux with Docker 20.10+ +- Maintains container network isolation +- **Trade-off**: Requires changing OPENAI_BASE_URL from localhost + +**Chosen Approach:** network_mode: host +- **Rationale**: Simplest for local LMStudio integration, no URL changes needed +- Tool mandate specifies "http://localhost:1234" must work +- Matches requirement for local development/testing setup + +--- + +#### LMStudio + Gemma 3 Configuration - RECIPE-0005 +**Research Date:** 2026-02-17 +**Source:** .env.example, llm.ts, prompt.yaml tool mandates + +**Current Configuration:** +```env +OPENAI_BASE_URL=http://localhost:1234/v1 +OPENAI_API_KEY=your-api-key-here +LLM_MODEL=google/gemma-3-4b +``` + +**LMStudio API Compatibility:** +- LMStudio provides OpenAI-compatible endpoint at `/v1` +- Uses same API client: openai@^4.20.0 +- Model identifiers match LMStudio's loaded model names +- API key can be any non-empty value (LMStudio doesn't validate in local mode) + +**Model Availability Check:** +From prior research (RECIPE-0001), `llm.ts` already implements: +- `checkModelAvailability(model: string)` - verifies model loaded via `client.models.list()` +- Returns available models if specified model not found +- User must manually load model in LMStudio UI before running container + +**No Code Changes Needed:** +- LLM integration already OpenAI-compatible +- Model check already implemented +- Only need environment variable configuration + +--- + +#### Docker Compose Complete Configuration - RECIPE-0005 +**Research Date:** 2026-02-17 +**Source:** docker-compose.yml, .env.example, queueConfig, tandoorConfig + +**Required Changes:** +1. Add `network_mode: host` for LMStudio access +2. Update LLM_MODEL default to `google/gemma-3-4b` +3. Update .env.example defaults to match tool mandates + +**Current docker-compose.yml:** +- Already has all environment variables configured +- Already has `./secrets:/app/secrets` volume mount +- Already has healthcheck configured +- Already has `seccomp=unconfined` for Chromium + +**Port Mapping with network_mode: host:** +- `ports:` section ignored when using `network_mode: host` +- App will bind directly to host port 3000 +- No conflicts expected (LMStudio uses 1234, app uses 3000, Tandoor external) + +--- + +#### End-to-End Testing Strategy - RECIPE-0005 +**Research Date:** 2026-02-17 +**Source:** Test URL from prompt, queue system architecture + +**Test URL:** https://www.instagram.com/reel/DP6oN7JCEo8/?utm_source=ig_web_button_share_sheet + +**Testing Workflow:** +1. Build Docker image: `docker-compose build` +2. Start container: `docker-compose up` +3. Verify LMStudio loaded Gemma 3 model: `http://localhost:1234/v1/models` +4. Verify app health: `http://localhost:3000/api/health` +5. Verify LLM health: `http://localhost:3000/api/llm-health` +6. Enqueue test URL: `POST http://localhost:3000/api/queue` +7. Monitor progress: `GET http://localhost:3000/api/queue/stream` +8. Verify extraction succeeds with Gemma 3 +9. Check Tandoor upload (if configured) + +**Success Indicators:** +- Chromium launches without "Executable doesn't exist" error +- LLM health check passes +- Extraction phase completes successfully +- Recipe parsing succeeds with Gemma 3 +- All existing tests pass (`npm test`) + +--- + +#### Files Summary - RECIPE-0005 + +**Modified Files:** +1. `src/lib/server/browser.ts` - Add executablePath for Alpine chromium +2. `docker-compose.yml` - Add network_mode: host, update LLM_MODEL default +3. `.env.example` - Update LLM_MODEL default to google/gemma-3-4b + +**No Changes:** +- `Dockerfile` - Already correct (chromium + fonts installed) +- `src/lib/server/llm.ts` - Already OpenAI-compatible +- `src/lib/server/queue/config.ts` - Already reads env vars correctly +- Test files - All existing tests should pass + +**Testing:** +- Manual E2E test with provided Instagram URL +- Verify in Docker container with LMStudio +- All unit tests must pass + +**Dependencies:** +- User must have LMStudio running on host at localhost:1234 +- User must manually load google/gemma-3-4b model in LMStudio +- Secrets volume must exist for Instagram auth (optional) + +--- + +**Document Version:** 1.7 +**Last Updated by:** Planner Agent (RECIPE-0005 Iteration 0) **Next Update:** Developer Agent diff --git a/src/lib/server/browser.ts b/src/lib/server/browser.ts index d2736d3..c07720d 100644 --- a/src/lib/server/browser.ts +++ b/src/lib/server/browser.ts @@ -17,6 +17,7 @@ export async function initializeBrowser(): Promise { console.log('Initializing Playwright browser...'); browser = await chromium.launch({ + executablePath: '/usr/bin/chromium-browser', headless: true, args: [ '--disable-blink-features=AutomationControlled',