mozempk/insta-recipe
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

fix(ssr): resolve EventSource SSR violations and implement best practices

Browse Source 
- Fix EventSource is not defined error in queue dashboard
- Add browser guards for all EventSource usage
- Replace static constants (EventSource.OPEN/CLOSED) with numeric values
- Fix setInterval SSR violation in LLM health indicator
- Replace $effect anti-pattern with onMount in share page
- Add comprehensive SvelteKit SSR best practices documentation
- Add SSR audit and testing verification

All changes follow SvelteKit best practices and are verified against
official documentation. Production build succeeds with no SSR errors.

Closes: FixEventSourceSSR
See: docs/outcomes/FixEventSourceSSR.md
This commit is contained in:
Giancarmine Salucci
2025-12-22 03:00:29 +01:00
parent 35d6f6e40a
commit 8545744bb1
47 changed files with 12827 additions and 363 deletions
Download Patch File Download Diff File Expand all files Collapse all files

1475
docs/plans/AsyncInMemoryProcessingQueue.md Normal file
View File

File diff suppressed because it is too large Load Diff

856
docs/plans/FixEventSourceSSR.md Normal file
View File

@@ -0,0 +1,856 @@
# Execution Plan: Fix SSR Violations and SvelteKit Best Practices
## Outcome Name
FixEventSourceSSRAndBestPractices
## Problem Analysis
### Primary Issue
`ReferenceError: EventSource is not defined` at `/home/moze/Sources/insta-recipe/src/routes/+page.svelte:299:66`
### Root Cause
The code is accessing `EventSource` during server-side rendering (SSR), but `EventSource` is a browser-only Web API that doesn't exist in Node.js. Additionally, comprehensive codebase analysis revealed multiple SSR violations and SvelteKit anti-patterns.
### Affected Files - Critical
1. **[src/routes/+page.svelte](src/routes/+page.svelte)** - EventSource accessed at L299, L82 without browser guards
2. **[src/routes/share/+page.svelte](src/routes/share/+page.svelte#L22-L25)** - `$effect` with side effects (calls `process()` function)
3. **[src/routes/share/components/LlmHealthIndicator.svelte](src/routes/share/components/LlmHealthIndicator.svelte#L36-L39)** - `$effect` with `setInterval` (no browser guard)
### Affected Files - Already Compliant (Good Examples)
1. **[src/lib/client/PushNotificationManager.ts](src/lib/client/PushNotificationManager.ts)** ✅
- Properly uses `browser` from `$app/environment` (L10)
- Guards `localStorage` access (L296, L300)
- Guards `window.atob` access (L318)
- Guards `navigator.serviceWorker` access (L111)
2. **[src/lib/client/ServiceWorkerMessageHandler.ts](src/lib/client/ServiceWorkerMessageHandler.ts)** ✅
- All browser APIs properly used in client-only context
- Not imported/used in SSR contexts
### SvelteKit Best Practices (from llms-full.txt documentation)
#### 1. Browser API Access
**Pattern:** Import `browser` from `$app/environment` and guard all browser-only APIs
```js
import { browser } from '$app/environment';
if (browser) {
// Browser-only code
}
```
**Browser-only APIs to guard:**
- `window.*`
- `document.*`
- `localStorage`, `sessionStorage`
- `navigator.*`
- `EventSource`, `WebSocket`
- `location.*`
#### 2. Lifecycle Hooks
**Pattern:** `onMount` only runs in browser (built-in SSR guard)
```js
import { onMount } from 'svelte';
onMount(() => {
// Automatically browser-only
// Still good practice to add explicit browser check for clarity
});
```
#### 3. Runes and Reactivity
**`$effect` gotcha:** Effects run during SSR AND hydration. Must guard browser APIs!
```js
$effect(() => {
if (!browser) return;
// Browser-only reactive code
});
```
**`$derived` gotcha:** Computed values run during SSR. Keep them pure!
```js
// ✅ GOOD - pure computation
let doubled = $derived(count * 2);
// ❌ BAD - side effects in derived
let value = $derived(localStorage.getItem('key')); // SSR crash!
```
#### 4. State Initialization
**Pattern:** Initialize with SSR-safe defaults, update in `onMount`
```js
let data = $state<Data | null>(null);
onMount(() => {
// Load browser-only data
data = JSON.parse(localStorage.getItem('key') || 'null');
});
```
#### 5. Static Constants
**Gotcha:** Accessing static properties of browser APIs causes SSR errors
```js
// ❌ BAD - EventSource.OPEN doesn't exist in Node
if (eventSource?.readyState === EventSource.OPEN)
// ✅ GOOD - Use numeric constants or guard
if (browser && eventSource?.readyState === EventSource.OPEN)
// OR
if (eventSource?.readyState === 1) // EventSource.OPEN = 1
```
### Codebase Analysis Results
#### ✅ Already Properly Guarded
- `PushNotificationManager.ts` - Excellent example of SSR-safe patterns
- `ServiceWorkerMessageHandler.ts` - Client-only, properly scoped
- All API routes in `src/routes/api/**` - Server-only contexts
- Service worker (`service-worker.ts`) - Runs in worker context only
#### ⚠️ Needs Fixing
**High Priority (Breaking SSR):**
1. **+page.svelte (Queue Dashboard)**
- L299: `eventSource?.readyState === EventSource.OPEN` - No browser guard
- L82: `eventSource?.readyState === EventSource.CLOSED` - No browser guard
- L67: `new EventSource()` - Inside `onMount` but needs explicit guard
- Missing `browser` import
2. **LlmHealthIndicator.svelte**
- L36-39: `$effect` with `setInterval` - No browser guard
- Should use `onMount` instead for timer setup
**Medium Priority (Anti-patterns):**
3. **share/+page.svelte**
- L22-25: `$effect` calling `process()` with side effects
- Should use `onMount` with conditional logic instead
- `$effect` is meant for synchronization, not side effects
#### 📋 Not Issues (Clarifications)
- `setTimeout` in components (L81 in +page.svelte, L53 in share/+page.svelte) - ✅ OK because inside `onMount` or event handlers
- `goto` from `$app/navigation` - ✅ SSR-safe (SvelteKit handles this)
- `$page` store from `$app/stores` - ✅ SSR-safe (available in both contexts)
- Server-side code (`lib/server/**`) using browser automation - ✅ OK (different context, uses Puppeteer)
## Stories
## Stories
### Story 1: Fix EventSource SSR in Queue Dashboard
**As a** developer
**I want** to guard all EventSource usage from SSR execution
**So that** the application doesn't crash with "EventSource is not defined"
**Acceptance Criteria:**
- Import `browser` from `$app/environment`
- Guard `EventSource` constructor in `startSSEConnection()`
- Replace `EventSource.OPEN` constant with numeric value `1` or add browser guard
- Replace `EventSource.CLOSED` constant with numeric value `2` or add browser guard
- Connection status works correctly after hydration
- No SSR errors in server logs
**Technical Details:**
**Lines to fix:**
- L2: Add `import { browser } from '$app/environment';`
- L67-68: Add browser guard before creating EventSource
- L82: Change `EventSource.CLOSED` to `2` or guard with `browser`
- L299: Change `EventSource.OPEN` to `1` or guard with `browser`
**Implementation:**
```typescript
import { browser } from '$app/environment';
function startSSEConnection() {
if (!browser) return; // ✅ Guard
try {
eventSource = new EventSource('/api/queue/stream');
// ... rest
}
}
// In reconnection logic (L82):
if (browser && eventSource?.readyState === 2) { // CLOSED = 2
startSSEConnection();
}
// In template (L299):
<div class="w-2 h-2 rounded-full {browser && eventSource?.readyState === 1 ? 'bg-green-400' : 'bg-red-400'}"></div>
```
**Files:**
- [src/routes/+page.svelte](src/routes/+page.svelte)
---
### Story 2: Fix $effect Anti-pattern in Share Page
**As a** developer
**I want** to replace `$effect` side effects with `onMount` pattern
**So that** the code follows SvelteKit best practices
**Acceptance Criteria:**
- Replace `$effect` with `onMount` for auto-processing shared URLs
- No side effects in reactive expressions
- Auto-processing still works when URL is shared
- No unnecessary re-triggering
**Technical Details:**
According to SvelteKit documentation, `$effect` should be used for synchronization, not side effects like API calls. Use `onMount` instead.
**Current problematic code (L22-25):**
```typescript
$effect(() => {
if (targetUrl && status === 'idle') {
process();
}
});
```
**Fixed code:**
```typescript
let hasProcessed = $state(false);
onMount(() => {
if (targetUrl && !hasProcessed) {
hasProcessed = true;
process();
}
});
```
**Files:**
- [src/routes/share/+page.svelte](src/routes/share/+page.svelte)
**SvelteKit Pattern Reference:**
> Use `$effect` for synchronizing derived state, DOM manipulation, or reactive cleanup.
> Use `onMount` for initialization, API calls, and browser-only setup.
---
### Story 3: Fix setInterval SSR in LLM Health Indicator
**As a** developer
**I want** to guard `setInterval` from SSR execution
**So that** the component doesn't break during server rendering
**Acceptance Criteria:**
- Add browser guard to `$effect` containing `setInterval`
- Health polling only runs in browser
- Component renders safely during SSR
- Cleanup still works correctly
**Technical Details:**
**Current code (L36-39):**
```typescript
$effect(() => {
checkHealth(); // Initial check
const interval = setInterval(checkHealth, pollInterval);
return () => clearInterval(interval);
});
```
**Fixed code:**
```typescript
$effect(() => {
if (!browser) return; // ✅ SSR guard
checkHealth(); // Initial check
const interval = setInterval(checkHealth, pollInterval);
return () => clearInterval(interval);
});
```
**Better alternative - use onMount:**
```typescript
onMount(() => {
checkHealth(); // Initial check
const interval = setInterval(checkHealth, pollInterval);
return () => clearInterval(interval);
});
```
**Files:**
- [src/routes/share/components/LlmHealthIndicator.svelte](src/routes/share/components/LlmHealthIndicator.svelte)
---
### Story 4: Add SSR Best Practices Documentation
**As a** developer
**I want** documentation on SSR best practices for this project
**So that** future development avoids these issues
**Acceptance Criteria:**
- Create or update developer documentation
- Include examples from the codebase
- Reference SvelteKit official documentation
- Add inline comments explaining SSR guards
**Technical Details:**
Create documentation covering:
1. Browser API detection with `$app/environment`
2. Lifecycle hook usage (`onMount` vs `$effect`)
3. Common gotchas (static constants, timers, storage APIs)
4. Good examples from our codebase (PushNotificationManager)
**Files to create/update:**
- `docs/SVELTEKIT_SSR_GUIDE.md` (new)
- Add inline comments to fixed files
---
### Story 5: Comprehensive SSR Audit and Testing
**As a** developer
**I want** to verify no other SSR violations exist
**So that** the application is fully SSR-safe
**Acceptance Criteria:**
- Manual SSR test: `npm run build && npm run preview`
- Check server logs for any SSR errors
- Test all routes with JavaScript disabled (progressive enhancement)
- Verify hydration works correctly
- No console warnings about hydration mismatches
**Technical Details:**
**Testing checklist:**
- [ ] Build production bundle: `npm run build`
- [ ] Preview production: `npm run preview`
- [ ] Navigate to all routes
- [ ] Check server console for errors
- [ ] Verify SSE connection works
- [ ] Test push notification UI
- [ ] Test queue dashboard
- [ ] Test share page with/without URL params
**Search patterns to verify:**
```bash
# Find any unguarded browser API usage
grep -r "window\." src/routes --include="*.svelte"
grep -r "document\." src/routes --include="*.svelte"
grep -r "localStorage" src/routes --include="*.svelte"
grep -r "navigator\." src/routes --include="*.svelte"
```
**Known safe patterns:**
- API routes (`src/routes/api/**`) - server-only
- Client libraries (`src/lib/client/**`) - properly guarded
- Event handlers (`onclick`, `onsubmit`) - run client-side
- `onMount` callbacks - run client-side
## Implementation Plan
### Phase 1: Critical Fixes (Blocks Production)
**Priority:** URGENT - Fixes SSR crashes
1. **Story 1** - Fix EventSource in Queue Dashboard
- Add `browser` import
- Guard EventSource creation
- Fix static constant references
- Test SSR rendering
2. **Story 3** - Fix setInterval in LLM Health Indicator
- Add browser guard to $effect OR convert to onMount
- Test component SSR
**Estimated Time:** 30 minutes
**Testing:** Build and preview, check server logs
---
### Phase 2: Best Practices (Improves Code Quality)
**Priority:** HIGH - Fixes anti-patterns
3. **Story 2** - Fix $effect anti-pattern in Share Page
- Replace $effect with onMount
- Add processed flag to prevent re-runs
- Test auto-processing behavior
**Estimated Time:** 20 minutes
**Testing:** Test share target flow
---
### Phase 3: Validation & Documentation (Prevents Future Issues)
**Priority:** MEDIUM - Long-term maintainability
4. **Story 5** - Comprehensive SSR Audit
- Run production build
- Test all routes
- Verify no SSR errors
5. **Story 4** - Documentation
- Create SSR best practices guide
- Add inline comments
- Document patterns from PushNotificationManager
**Estimated Time:** 1 hour
**Testing:** Full regression test
---
### Total Estimated Time
- Critical fixes: 30 min
- Best practices: 20 min
- Validation & docs: 1 hour
- **Total: ~2 hours**
## Technical Specifications
### SvelteKit Runes Reference
#### `$state` - Reactive State
```typescript
let count = $state(0); // Simple state
let obj = $state({ name: 'Alice' }); // Deep reactive proxy
```
- ✅ SSR-safe for primitive values
- ⚠️ Don't initialize with browser APIs
#### `$derived` - Computed Values
```typescript
let doubled = $derived(count * 2);
```
- ✅ Runs during SSR
- ⚠️ Must be pure (no side effects)
- ❌ Don't access browser APIs
#### `$effect` - Reactive Side Effects
```typescript
$effect(() => {
// Runs during SSR AND hydration
console.log('count changed:', count);
});
```
- ⚠️ Runs in both SSR and browser
- ✅ Use for synchronization
- ❌ Not for initialization or API calls
- **Must guard browser APIs**
#### `onMount` - Browser-Only Lifecycle
```typescript
onMount(() => {
// Only runs in browser
return () => {
// Cleanup
};
});
```
- ✅ Only runs in browser
- ✅ Use for initialization
- ✅ Use for browser API access
### Browser API Constants
Some browser APIs expose static constants that don't exist during SSR:
**EventSource:**
- `EventSource.CONNECTING = 0`
- `EventSource.OPEN = 1`
- `EventSource.CLOSED = 2`
**Solutions:**
```typescript
// ❌ BAD - Crashes SSR
if (es.readyState === EventSource.OPEN)
// ✅ GOOD - Use numeric value
if (es.readyState === 1)
// ✅ GOOD - Guard access
if (browser && es.readyState === EventSource.OPEN)
```
**WebSocket:** Similar issue with `WebSocket.OPEN`, etc.
### Dependencies
- `$app/environment` - Built-in SvelteKit module
- No new package dependencies required
### Files to Modify
**Critical (Phase 1):**
1. `src/routes/+page.svelte` - Queue dashboard
2. `src/routes/share/components/LlmHealthIndicator.svelte` - Health indicator
**Best Practices (Phase 2):**
3. `src/routes/share/+page.svelte` - Share page
**Documentation (Phase 3):**
4. `docs/SVELTEKIT_SSR_GUIDE.md` - New file
### Code Patterns Summary
#### Pattern 1: Browser API in Component State
```svelte
<script lang="ts">
import { browser } from '$app/environment';
import { onMount } from 'svelte';
let eventSource = $state<EventSource | null>(null);
onMount(() => {
if (browser) {
eventSource = new EventSource('/api/stream');
}
return () => {
eventSource?.close();
};
});
</script>
<!-- Use numeric constants or guard in template -->
<div>Status: {eventSource?.readyState === 1 ? 'Connected' : 'Disconnected'}</div>
```
#### Pattern 2: Timers and Intervals
```svelte
<script lang="ts">
import { onMount } from 'svelte';
onMount(() => {
const interval = setInterval(() => {
// Polling logic
}, 1000);
return () => clearInterval(interval);
});
</script>
```
#### Pattern 3: Auto-Processing on Mount
```svelte
<script lang="ts">
import { onMount } from 'svelte';
let hasProcessed = $state(false);
let data = $derived(computeData());
onMount(() => {
if (shouldProcess && !hasProcessed) {
hasProcessed = true;
process();
}
});
</script>
```
## Risk Assessment
### High Priority Risks
- **SSR/Hydration mismatch**: If guards are inconsistent between server and client
- **Mitigation**: Use numeric constants; avoid conditional rendering based on `browser`
- **Testing**: Check for hydration warnings in console
### Medium Priority Risks
- **Regression in auto-processing**: Share page might not auto-process URLs
- **Mitigation**: Thorough testing of share target flow
- **Testing**: Test with Instagram share and manual URL input
- **Connection status flicker**: Status indicator might show wrong state briefly
- **Mitigation**: Initialize with sensible defaults
- **Testing**: Watch for visual flicker on page load
### Low Priority Risks
- **Performance**: Minimal, browser checks are fast
- **Breaking changes**: Unlikely, only fixing internal implementation
## Testing Strategy
### Unit Testing
- Not applicable - these are integration-level fixes
- Existing tests should continue to pass
### Integration Testing
**Manual testing required:**
1. **SSR Testing:**
```bash
npm run build
npm run preview
# Check server console for errors
# Navigate to all pages
```
2. **EventSource Connection:**
- Open queue dashboard
- Check browser DevTools → Network → EventSource
- Verify "Live updates" status indicator
- Add queue item and verify real-time update
3. **Share Page:**
- Navigate to `/share`
- Manually enter URL → should work
- Share from Instagram → should auto-process
- Check no duplicate processing
4. **LLM Health:**
- Check health indicator appears
- Verify polling happens (check Network tab)
- No SSR errors in console
### Edge Cases
- **Server restart** while client connected → Reconnection works
- **Network disconnection** → Graceful degradation
- **JavaScript disabled** → Progressive enhancement (no errors)
- **Multiple tabs** open → Each maintains own connection
### Hydration Testing
- Disable JavaScript after SSR
- Enable JavaScript and check hydration
- Look for console warnings:
- "Hydration failed"
- "The server response doesn't match the client content"
### Browser Compatibility
- Modern browsers with EventSource support
- Browsers without EventSource → Should show disconnected status (no crash)
## Success Metrics
### Must Have (Phase 1)
1. ✅ No `EventSource is not defined` errors
2. ✅ No `setInterval is not defined` errors
3. ✅ Production build succeeds
4. ✅ SSR renders without errors
5. ✅ Live updates work in browser
### Should Have (Phase 2)
6. ✅ No `$effect` anti-patterns
7. ✅ No hydration warnings
8. ✅ Share page auto-processing works
### Nice to Have (Phase 3)
9. ✅ SSR best practices documentation
10. ✅ Inline comments explaining patterns
11. ✅ All routes tested and verified
## Anti-Patterns to Avoid
### ❌ Don't Do This
```typescript
// 1. Browser API in $derived
let data = $derived(localStorage.getItem('key')); // SSR crash!
// 2. Side effects in $effect without guard
$effect(() => {
fetch('/api/data'); // Runs during SSR!
});
// 3. Static constants without guard
if (ws.readyState === WebSocket.OPEN) // SSR crash!
// 4. Initialization in $effect
$effect(() => {
// Use onMount instead
initialize();
});
```
### ✅ Do This Instead
```typescript
// 1. Load in onMount
let data = $state<string | null>(null);
onMount(() => {
data = localStorage.getItem('key');
});
// 2. Guard browser APIs in $effect
$effect(() => {
if (!browser) return;
fetch('/api/data');
});
// 3. Use numeric constants or guard
if (ws.readyState === 1) // WebSocket.OPEN = 1
// OR
if (browser && ws.readyState === WebSocket.OPEN)
// 4. Initialize in onMount
onMount(() => {
initialize();
});
```
## References
### Official Documentation
- [SvelteKit SSR](https://svelte.dev/llms-full.txt) - From llms-full.txt
- [Svelte Runes](https://svelte.dev/llms-full.txt) - $state, $derived, $effect
- [SvelteKit $app modules](https://svelte.dev/llms-full.txt) - $app/environment, $app/stores
### Our Codebase Examples
- **Good:** [PushNotificationManager.ts](src/lib/client/PushNotificationManager.ts) - Excellent SSR-safe patterns
- **Good:** [ServiceWorkerMessageHandler.ts](src/lib/client/ServiceWorkerMessageHandler.ts) - Client-only scope
- **Fix:** [+page.svelte](src/routes/+page.svelte) - EventSource needs guards
- **Fix:** [LlmHealthIndicator.svelte](src/routes/share/components/LlmHealthIndicator.svelte) - setInterval needs guard
- **Improve:** [share/+page.svelte](src/routes/share/+page.svelte) - $effect anti-pattern
### Web APIs
- [EventSource MDN](https://developer.mozilla.org/en-US/docs/Web/API/EventSource)
- [Server-Sent Events](https://developer.mozilla.org/en-US/docs/Web/API/Server-sent_events)
## Appendix: Complete Code Changes
### A. +page.svelte (Queue Dashboard)
**Before:**
```svelte
<script lang="ts">
import { page } from '$app/stores';
import { onMount, onDestroy } from 'svelte';
// ... state declarations
onMount(async () => {
await loadQueueItems();
startSSEConnection();
});
function startSSEConnection() {
try {
eventSource = new EventSource('/api/queue/stream');
// ...
eventSource.addEventListener('error', (event) => {
// ...
setTimeout(() => {
if (eventSource?.readyState === EventSource.CLOSED) {
startSSEConnection();
}
}, 5000);
});
}
}
</script>
<!-- Template -->
<div class="w-2 h-2 rounded-full {eventSource?.readyState === EventSource.OPEN ? 'bg-green-400' : 'bg-red-400'}"></div>
```
**After:**
```svelte
<script lang="ts">
import { page } from '$app/stores';
import { browser } from '$app/environment';
import { onMount, onDestroy } from 'svelte';
// ... state declarations
onMount(async () => {
await loadQueueItems();
if (browser) {
startSSEConnection();
}
});
function startSSEConnection() {
if (!browser) return;
try {
eventSource = new EventSource('/api/queue/stream');
// ...
eventSource.addEventListener('error', (event) => {
// ...
setTimeout(() => {
if (eventSource?.readyState === 2) { // CLOSED = 2
startSSEConnection();
}
}, 5000);
});
}
}
</script>
<!-- Template -->
<div class="w-2 h-2 rounded-full {browser && eventSource?.readyState === 1 ? 'bg-green-400' : 'bg-red-400'}"></div>
```
### B. LlmHealthIndicator.svelte
**Before:**
```svelte
<script lang="ts">
// ...
$effect(() => {
checkHealth();
const interval = setInterval(checkHealth, pollInterval);
return () => clearInterval(interval);
});
</script>
```
**After (Option 1 - Guard $effect):**
```svelte
<script lang="ts">
import { browser } from '$app/environment';
// ...
$effect(() => {
if (!browser) return;
checkHealth();
const interval = setInterval(checkHealth, pollInterval);
return () => clearInterval(interval);
});
</script>
```
**After (Option 2 - Use onMount - RECOMMENDED):**
```svelte
<script lang="ts">
import { onMount } from 'svelte';
// ...
onMount(() => {
checkHealth();
const interval = setInterval(checkHealth, pollInterval);
return () => clearInterval(interval);
});
</script>
```
### C. share/+page.svelte
**Before:**
```svelte
<script lang="ts">
// ...
let targetUrl = $derived(sharedUrl || extractUrl(sharedText));
$effect(() => {
if (targetUrl && status === 'idle') {
process();
}
});
</script>
```
**After:**
```svelte
<script lang="ts">
import { onMount } from 'svelte';
// ...
let targetUrl = $derived(sharedUrl || extractUrl(sharedText));
let hasAutoProcessed = $state(false);
onMount(() => {
if (targetUrl && status === 'idle' && !hasAutoProcessed) {
hasAutoProcessed = true;
process();
}
});
</script>
```
---
**Plan Complete - Ready for Implementation**

802
docs/plans/FixPushNotificationSSRAndSSL.md Normal file
View File

@@ -0,0 +1,802 @@
# Execution Plan: Fix Push Notification SSR Bug, Regenerate SSL, and Code Cleanup
## Context
The application is experiencing a critical SSR (Server-Side Rendering) bug where `PushNotificationManager` attempts to access `localStorage` during server-side rendering, causing the application to crash:
```
ReferenceError: localStorage is not defined
at PushNotificationManager.generateClientId (src/lib/client/PushNotificationManager.ts:256:20)
at new PushNotificationManager (src/lib/client/PushNotificationManager.ts:31:26)
```
Additionally:
- The SSL certificate expired on Dec 21, 2025 (yesterday)
- The codebase contains dead/unused code that should be deleted
- There are opportunities to consolidate duplicate code
**CRITICAL:** All work must be done in the **current branch** (`feat/async-in-memory-processing-queue`), not a new branch.
## Research Summary
### SvelteKit SSR & localStorage Best Practices
From SvelteKit documentation and community best practices:
1. **Browser API Detection:** Use `browser` from `$app/environment` to check if code is running in browser
2. **Lazy Initialization:** Don't access browser APIs at module level or in constructors
3. **onMount Lifecycle:** Use Svelte's `onMount` for browser-only initialization
4. **Guard Pattern:** Wrap all browser API access with browser checks
**Key Pattern:**
```typescript
import { browser } from '$app/environment';
if (browser) {
// Browser-only code here
localStorage.getItem('key');
}
```
### SSL Certificate Strategy
For local development with 10-year validity:
- Leverage the external Caddy container's CA (already trusted on the system)
- Extract Caddy's CA private key to sign a custom certificate with 10-year validity
- Use OpenSSL to generate and sign the certificate with Caddy's CA
- No manual trust steps needed - Caddy CA already trusted
- Alternative: Use Caddy's automatic generation if 10-year validity not strictly required (90-day certs)
## User Stories
### Story 0: Fix PushNotificationManager SSR Issue 🔴 CRITICAL
**As a** developer
**I want** the PushNotificationManager to work correctly in SSR context
**So that** the application doesn't crash when components are rendered on the server
**Acceptance Criteria:**
- ✅ PushNotificationManager constructor does not access `localStorage`
- ✅ `clientId` is generated lazily only in browser context
- ✅ All browser APIs (window, Notification, navigator) are guarded with browser checks
- ✅ Module-level singleton instantiation is safe for SSR
- ✅ NotificationSettings.svelte component works without errors
- ✅ No SSR-related errors in console
- ✅ Push notifications still work correctly in browser
**Technical Approach:**
1. **Lazy ClientId Generation:**
```typescript
import { browser } from '$app/environment';
class PushNotificationManager {
private _clientId: string | null = null;
private get clientId(): string {
if (!this._clientId && browser) {
this._clientId = this.generateClientId();
}
return this._clientId || 'ssr-fallback';
}
private generateClientId(): string {
if (!browser) return '';
const stored = localStorage.getItem('push-client-id');
if (stored) return stored;
const id = `client_${Date.now()}_${Math.random().toString(36).substr(2, 9)}`;
localStorage.setItem('push-client-id', id);
return id;
}
}
```
2. **Guard Browser API Checks:**
```typescript
private checkSupport(): void {
if (!browser) {
this.state.supported = false;
this.state.permission = 'denied';
return;
}
this.state.supported = (
'serviceWorker' in navigator &&
'PushManager' in window &&
'Notification' in window
);
this.state.permission = this.state.supported ? Notification.permission : 'denied';
}
```
3. **Safe Service Worker Initialization:**
```typescript
private async initializeServiceWorker(): Promise<void> {
if (!browser || !this.state.supported) return;
// Rest of initialization
}
```
**Files:**
- `src/lib/client/PushNotificationManager.ts` (update)
- `src/routes/components/NotificationSettings.svelte` (verify)
**Testing:**
- Test component renders without errors in SSR
- Test push notification subscribe/unsubscribe in browser
- Test that clientId persists across browser sessions
- Verify no localStorage access during SSR
---
### Story 1: Generate 10-Year SSL Certificate Using External Caddy CA
**As a** developer
**I want** a valid SSL certificate with 10-year validity signed by the external Caddy CA
**So that** I don't have to regenerate certificates frequently and they're automatically trusted
**Acceptance Criteria:**
- ✅ New SSL certificate valid for 10 years (3650 days)
- ✅ Certificate signed by existing Caddy CA (already trusted on system)
- ✅ Certificate files in `.ssl/` directory:
- `localhost.key` (private key)
- `localhost.crt` (certificate signed by Caddy CA)
- `root.crt` (Caddy CA certificate - copied from container)
- ✅ Certificate automatically trusted (no manual trust needed)
- ✅ `vite.config.ts` points to correct certificate files
- ✅ Certificate expiration date verified: ~2035
- ✅ Caddy container ID identified or documented
**Technical Approach:**
This approach leverages the external Caddy container's CA that's already trusted on the system, but generates a certificate with custom 10-year validity.
1. **Identify Caddy Container:**
```bash
# Find the Caddy container
docker ps | grep caddy
# Or use the known ID from previous work (might have changed)
CADDY_CONTAINER=$(docker ps --filter "ancestor=caddy" --format "{{.ID}}" | head -1)
echo "Caddy container: $CADDY_CONTAINER"
```
2. **Export Caddy's CA Certificate and Private Key:**
```bash
# Copy the CA certificate (already done, but verify it exists)
docker cp $CADDY_CONTAINER:/data/caddy/pki/authorities/local/root.crt .ssl/root.crt
# Copy the CA private key (needed to sign our custom certificate)
docker cp $CADDY_CONTAINER:/data/caddy/pki/authorities/local/root.key .ssl/caddy-ca.key
# Verify CA certificate
openssl x509 -in .ssl/root.crt -text -noout | grep "Subject:"
```
3. **Generate New Server Certificate with 10-Year Validity:**
```bash
# Generate server private key (2048-bit is sufficient)
openssl genrsa -out .ssl/localhost.key 2048
# Generate Certificate Signing Request (CSR)
openssl req -new \
-key .ssl/localhost.key \
-out .ssl/localhost.csr \
-subj "/O=Caddy Local Authority/CN=localhost"
# Create OpenSSL config for Subject Alternative Names (SAN)
cat > .ssl/localhost.ext << 'EOF'
authorityKeyIdentifier=keyid,issuer
basicConstraints=CA:FALSE
keyUsage = digitalSignature, nonRepudiation, keyEncipherment, dataEncipherment
extendedKeyUsage = serverAuth
subjectAltName = @alt_names
[alt_names]
DNS.1 = localhost
DNS.2 = *.localhost
IP.1 = 127.0.0.1
IP.2 = ::1
EOF
# Sign the certificate with Caddy's CA (10 years = 3650 days)
openssl x509 -req \
-in .ssl/localhost.csr \
-CA .ssl/root.crt \
-CAkey .ssl/caddy-ca.key \
-CAcreateserial \
-out .ssl/localhost.crt \
-days 3650 \
-sha256 \
-extfile .ssl/localhost.ext
# Cleanup temporary files and CA private key (security)
rm .ssl/localhost.csr .ssl/localhost.ext .ssl/caddy-ca.key
# Set restrictive permissions
chmod 600 .ssl/localhost.key
chmod 644 .ssl/localhost.crt .ssl/root.crt
```
4. **Verify Certificate:**
```bash
# Check expiration date (should be ~2035)
openssl x509 -enddate -noout -in .ssl/localhost.crt
# Verify certificate is signed by Caddy CA
openssl verify -CAfile .ssl/root.crt .ssl/localhost.crt
# Check certificate details
openssl x509 -in .ssl/localhost.crt -text -noout | grep -A 1 "Subject:"
openssl x509 -in .ssl/localhost.crt -text -noout | grep -A 3 "Subject Alternative Name"
```
5. **Verify Vite Configuration:**
```bash
# Ensure vite.config.ts already points to correct files
grep -A 3 "https:" vite.config.ts
```
**Alternative: If Caddy CA Private Key is Not Accessible**
If the CA private key is not accessible from the container, use Caddy's built-in certificate generation but with a workaround:
1. **Trigger Caddy Certificate Generation:**
```bash
# Run temporary Caddy reverse-proxy to trigger cert generation
docker exec -d $CADDY_CONTAINER caddy reverse-proxy \
--from localhost:8443 \
--to localhost:8080
# Wait for certificate generation (5-10 seconds)
sleep 10
# Stop the temporary process
docker exec $CADDY_CONTAINER pkill -f "caddy reverse-proxy"
```
2. **Copy Generated Certificates:**
```bash
# Copy Caddy-generated certificates
docker cp $CADDY_CONTAINER:/data/caddy/certificates/local/localhost/localhost.crt .ssl/
docker cp $CADDY_CONTAINER:/data/caddy/certificates/local/localhost/localhost.key .ssl/
docker cp $CADDY_CONTAINER:/data/caddy/pki/authorities/local/root.crt .ssl/
```
3. **Note on Validity:**
- Caddy-generated certificates typically have 90-day validity
- If 10-year validity is required, must use OpenSSL approach with CA key
- Document renewal process in README if using short-lived certs
**Files:**
- `.ssl/localhost.key` (create - server private key)
- `.ssl/localhost.crt` (create - server certificate signed by Caddy CA)
- `.ssl/root.crt` (copy from Caddy container - CA certificate)
- `README.md` (update with certificate info and renewal instructions)
- `.gitignore` (verify .ssl/ is ignored except for .gitkeep)
**Testing:**
- Verify certificate dates: `openssl x509 -enddate -noout -in .ssl/localhost.crt`
- Verify CA signature: `openssl verify -CAfile .ssl/root.crt .ssl/localhost.crt`
- Test HTTPS server starts: `npm run dev`
- Verify browser shows secure connection (should be automatic - CA already trusted)
- Test certificate valid until ~2035 (if using OpenSSL approach)
**Documentation Note:**
Since the Caddy CA is already trusted on the system, no manual trust steps are needed. Document in README:
- How to check certificate expiration
- How to regenerate using same process
- Caddy container identification steps
---
### Story 2: Audit and Delete Dead/Unused Code
**As a** developer
**I want** to remove all dead and unused code from the codebase
**So that** the codebase is cleaner and easier to maintain
**Acceptance Criteria:**
- ✅ All unused imports removed
- ✅ All unreferenced functions/types deleted
- ✅ All commented-out code blocks removed
- ✅ Unused test fixtures cleaned up
- ✅ No deprecation markers (code is deleted, not deprecated)
- ✅ All tests still passing
- ✅ No broken imports or references
**Audit Areas:**
1. **Check for Unused Imports:**
```bash
# Use TypeScript compiler to find unused imports
npx tsc --noEmit
# Or use eslint if configured
npm run lint
```
2. **Scan for Unreferenced Code:**
- Search for functions/classes that are never imported
- Check test files for unused fixtures
- Look for commented-out code blocks (`// `, `/* */`)
3. **Verify Deprecated Endpoints:**
- `/api/extract` returns 410 Gone ✅ KEEP (migration helper)
- `/api/extract-stream` already deleted ✅
- Check for any other deprecated routes
4. **Clean Up Test Files:**
- `src/tests/fixtures.ts` - review localStorage fixtures
- Remove any unused test helpers
- Delete obsolete test files
5. **Review Client Components:**
- `ServiceWorkerMessageHandler.ts` - verify usage
- Check for unused utility functions
**Files to Review:**
- `src/lib/client/*` - Client utilities
- `src/tests/*` - Test files and fixtures
- `src/routes/components/*` - UI components
- All import statements across codebase
**Deletion Checklist:**
- [ ] Unused imports removed
- [ ] Commented-out code deleted
- [ ] Unreferenced functions deleted
- [ ] Obsolete test fixtures removed
- [ ] Dead code paths eliminated
- [ ] Verify no broken imports with `npx tsc --noEmit`
**Testing:**
- Run full test suite: `npm test`
- Build project: `npm run build`
- Check for TypeScript errors: `npx tsc --noEmit`
- Verify dev server starts: `npm run dev`
---
### Story 3: Consolidate Duplicate Code
**As a** developer
**I want** to consolidate duplicate and similar code
**So that** the codebase has less redundancy and is easier to maintain
**Acceptance Criteria:**
- ✅ Duplicate type definitions merged
- ✅ Similar utility functions consolidated
- ✅ Repeated code blocks extracted to functions
- ✅ Common patterns extracted to shared utilities
- ✅ No functionality broken
- ✅ All tests still passing
**Consolidation Areas:**
1. **Type Definitions:**
- Check for duplicate interfaces/types across files
- Move shared types to appropriate locations:
- Domain types → `src/lib/server/queue/types.ts`
- Client types → `src/lib/client/types.ts` (create if needed)
- Shared types → `src/lib/types.ts` (create if needed)
2. **Utility Functions:**
- Look for similar string formatting functions
- Check for duplicate validation logic
- Identify common data transformation patterns
3. **Component Patterns:**
- Similar error handling across components
- Repeated state management patterns
- Common UI patterns
4. **API Response Handling:**
- Similar fetch patterns
- Duplicate error handling
- Common response transformations
**Investigation Steps:**
1. **Search for Duplicate Type Definitions:**
```bash
# Look for common type names
grep -r "interface.*State" src/
grep -r "type.*Config" src/
```
2. **Find Similar Function Signatures:**
```bash
# Look for validation functions
grep -r "function validate" src/
grep -r "async function.*fetch" src/
```
3. **Identify Repeated Patterns:**
- SSE connection setup
- Error handling blocks
- Loading state management
- Form validation
**Consolidation Strategy:**
For each duplicate found:
1. Determine the most complete/correct version
2. Extract to shared location if used in multiple places
3. Update all references to use shared version
4. Delete duplicate versions
5. Verify tests pass
**Files:**
- Potentially create: `src/lib/utils/` directory for shared utilities
- Potentially create: `src/lib/types.ts` for shared types
- Update all files with consolidated references
**Testing:**
- Run full test suite after each consolidation
- Verify no regression in functionality
- Check TypeScript compilation succeeds
---
### Story 4: Verify and Test Complete Solution
**As a** developer
**I want** to verify all changes work correctly together
**So that** the fixes are production-ready
**Acceptance Criteria:**
- ✅ All unit tests passing
- ✅ Integration tests passing
- ✅ No SSR errors in development
- ✅ No SSR errors in production build
- ✅ SSL certificate works correctly
- ✅ Push notifications work in browser
- ✅ No console warnings or errors
- ✅ Application builds successfully
- ✅ All TypeScript errors resolved
**Testing Checklist:**
1. **SSR Testing:**
```bash
# Test dev server (SSR enabled)
npm run dev
# Visit pages and check console for errors
# Test production build
npm run build
npm run preview
```
2. **Push Notification Testing:**
- Open NotificationSettings component
- Verify no SSR errors
- Test subscribe/unsubscribe in browser
- Verify clientId persists across refresh
3. **SSL Certificate Testing:**
- Verify HTTPS connection works
- Check certificate validity in browser
- Test across different browsers (Chrome, Firefox)
4. **Code Quality:**
```bash
# TypeScript check
npx tsc --noEmit
# Linting
npm run lint
# Unit tests
npm test
# Build
npm run build
```
5. **Manual Testing:**
- Test all queue operations
- Test extraction flow
- Verify push notifications
- Check HTTPS connection
- Test on mobile browsers (if applicable)
**Regression Testing:**
- Queue creation works
- SSE progress updates work
- Extraction completes successfully
- Tandoor integration works
- All existing features functional
**Performance Check:**
- Bundle size acceptable
- No memory leaks
- Reasonable load times
- No performance degradation
---
## Technical Specifications
### Browser API Guard Pattern
All browser API access must follow this pattern:
```typescript
import { browser } from '$app/environment';
// Module level - safe for SSR
class MyClass {
private browserOnlyState: SomeType | null = null;
// Constructor - safe for SSR
constructor() {
// NO browser API access here
}
// Methods can check browser context
someMethod() {
if (!browser) {
return; // or return safe default
}
// Browser APIs safe here
const data = localStorage.getItem('key');
}
// Lazy initialization pattern
private _clientId: string | null = null;
private get clientId(): string {
if (!this._clientId && browser) {
this._clientId = this.initializeClientId();
}
return this._clientId || 'fallback-value';
}
}
```
### SSL Certificate File Structure
```
.ssl/
├── localhost.key # Server private key (2048-bit RSA)
├── localhost.crt # Server certificate (signed by Caddy CA, 10 years)
├── root.crt # Caddy CA certificate (copied from container, already trusted)
└── .gitkeep # Track directory but ignore contents
```
### Code Deletion Guidelines
1. **Before Deleting:**
- Search entire codebase for references
- Check test files for usage
- Verify not used in comments or documentation
- Check git history for context
2. **Safe to Delete:**
- No references found
- Confirmed not used in any import
- Not referenced in documentation
- Clearly obsolete/deprecated
3. **Keep but Document:**
- Migration helper endpoints (like /api/extract)
- Fallback strategies (like legacy extraction)
- Backward compatibility shims
4. **Delete Immediately:**
- Commented-out code
- Unused imports
- Unreferenced functions
- Obsolete test fixtures
---
## Dependencies
### Story Dependencies
- Story 0 (SSR Fix) → No dependencies, can start immediately
- Story 1 (SSL) → No dependencies, can start immediately
- Story 2 (Dead Code) → Should wait for Story 0 completion
- Story 3 (Consolidation) → Should wait for Story 2 completion
- Story 4 (Verification) → Depends on all previous stories
### Execution Order
1. **Story 0** - Critical SSR fix (blocks development)
2. **Story 1** - SSL regeneration (parallel with Story 0)
3. **Story 2** - Dead code cleanup
4. **Story 3** - Code consolidation
5. **Story 4** - Final verification and testing
---
## Risk Assessment
### High Risk
**Risk:** Breaking push notification functionality
- **Impact:** Users lose real-time updates
- **Likelihood:** Medium
- **Mitigation:** Thorough testing in browser and SSR contexts
- **Rollback:** Revert PushNotificationManager changes, keep old version
**Risk:** SSL certificate not trusted by system
- **Impact:** Development blocked, HTTPS warnings
- **Likelihood:** Low (clear instructions provided)
- **Mitigation:** Detailed trust instructions for all platforms
- **Rollback:** Regenerate old certificate or disable HTTPS temporarily
### Medium Risk
**Risk:** Deleting code that's actually used
- **Impact:** Runtime errors, broken functionality
- **Likelihood:** Low (comprehensive search before delete)
- **Mitigation:** Thorough searching, test suite verification
- **Rollback:** Git revert specific deletions
**Risk:** Consolidation introducing subtle bugs
- **Impact:** Broken functionality in edge cases
- **Likelihood:** Low
- **Mitigation:** Incremental consolidation, test after each change
- **Rollback:** Git revert to pre-consolidation state
### Low Risk
**Risk:** TypeScript compilation errors after changes
- **Impact:** Development blocked temporarily
- **Likelihood:** Very Low
- **Mitigation:** Run tsc check frequently
- **Rollback:** Easy to fix type errors
---
## Testing Strategy
### Unit Tests
- Test PushNotificationManager in isolation
- Mock browser APIs for testing
- Test lazy initialization patterns
- Verify state management
### Integration Tests
- Test NotificationSettings component
- Verify SSE integration still works
- Test queue system end-to-end
- Verify extraction pipeline
### SSR Tests
- Render components server-side
- Verify no localStorage access
- Check no window/navigator access
- Ensure safe module initialization
### Manual Tests
- Browser push notifications
- SSL certificate trust
- HTTPS connection
- Cross-browser compatibility
---
## Documentation Updates
### README.md
Add/update sections:
- SSL Certificate Setup (detailed trust instructions)
- HTTPS Development Setup
- Browser Requirements
- Troubleshooting SSL issues
### Code Comments
- Document browser API guard patterns
- Explain lazy initialization approach
- Note SSR safety considerations
- Document clientId generation logic
---
## Success Metrics
1. **Zero SSR Errors:** No localStorage or browser API errors during SSR
2. **Push Notifications Working:** Subscribe/unsubscribe functional in browser
3. **SSL Valid:** Certificate valid until ~2035, trusted by browsers
4. **Clean Codebase:** No unused imports, no dead code, no duplicates
5. **All Tests Passing:** 100% test suite success rate
6. **TypeScript Clean:** Zero compilation errors
7. **No Console Errors:** Clean browser console in dev and prod
---
## Rollback Plan
If critical issues arise:
1. **SSR Fix Rollback:**
```bash
git revert <commit-hash-of-ssr-fix>
# Or restore old PushNotificationManager.ts
```
2. **SSL Rollback:**
```bash
# Generate quick temporary certificate
openssl req -x509 -newkey rsa:2048 -nodes \
-keyout .ssl/localhost.key \
-out .ssl/localhost.crt \
-days 365 -subj "/CN=localhost"
```
3. **Code Cleanup Rollback:**
```bash
git revert <cleanup-commit-hash>
# Or restore specific deleted files from git history
```
4. **Full Rollback:**
```bash
# Reset to before all changes
git reset --hard <commit-before-changes>
```
---
## Timeline Estimate
- **Story 0 (SSR Fix):** 2-3 hours
- **Story 1 (SSL):** 1-2 hours (can be parallel)
- **Story 2 (Dead Code):** 2-4 hours
- **Story 3 (Consolidation):** 3-5 hours
- **Story 4 (Verification):** 1-2 hours
**Total Estimated Time:** 9-16 hours
---
## Branch Strategy
⚠️ **IMPORTANT:** All work MUST be done in the current branch:
- Branch: `feat/async-in-memory-processing-queue`
- Do NOT create a new feature branch
- Commit incrementally with clear messages
- Keep all changes contained in this branch
---
## Completion Criteria
The plan is complete when:
1. ✅ PushNotificationManager works in both SSR and browser contexts
2. ✅ No localStorage errors in any context
3. ✅ SSL certificate valid for 10 years
4. ✅ HTTPS development server working
5. ✅ All dead code deleted (not deprecated)
6. ✅ All duplicate code consolidated
7. ✅ All tests passing
8. ✅ No TypeScript errors
9. ✅ No console warnings/errors
10. ✅ Application builds successfully
11. ✅ Documentation updated
12. ✅ All changes committed to current branch
---
## Notes
- SvelteKit documentation emphasizes avoiding browser APIs in SSR context
- The `browser` environment variable is the recommended pattern
- SSL certificates for local development typically don't need to be from a real CA
- 10-year validity is reasonable for local development certificates
- Code should be deleted, not deprecated, when truly unused
- Consolidation should focus on real duplicates, not just similar patterns
- Keep backward compatibility for migration helper endpoints

1709
docs/plans/FixQueueTypesMismatchAndEnhancements.md Normal file
View File

File diff suppressed because it is too large Load Diff