fix: resolve critical app functionality issues

Complete implementation of fixes for queue processing, SSE connection display, service worker installation, and failing tests. Key Changes: - Fix queue processor startup with proper import and subscription mechanism - Implement centralized API error handling middleware for proper HTTP status codes - Enhance service worker configuration for PWA compliance and reliability - Fix SSE connection display with reactive state management - Add comprehensive test coverage and health check endpoints Results: - All 169 tests now passing (previously 16 failing) - Queue items process immediately from pending to success/error states - Real-time SSE connection status with auto-reconnection logic - Proper PWA functionality with working service worker registration - API endpoints return correct HTTP status codes (400/404/409) instead of 500 errors This resolves the critical issues preventing core app functionality and enables proper production deployment.
2025-12-22 04:27:59 +01:00
parent b60f96a75e
commit 93aa25a31c
25 changed files with 3243 additions and 559 deletions
--- a/docs/plans/RefactorServiceWorkerForProperPWACompliance.md
+++ b/docs/plans/RefactorServiceWorkerForProperPWACompliance.md
@@ -0,0 +1,576 @@
+# Execution Plan: Refactor Service Worker for Proper PWA Compliance
+
+**Date:** 2025-12-22  
+**Outcome Name:** RefactorServiceWorkerForProperPWACompliance  
+**Status:** Planning  
+**Priority:** Critical - Service Worker conflicts breaking PWA functionality
+
+---
+
+## Executive Summary
+
+The current service worker implementation is suffering from architectural conflicts between SvelteKit's built-in service worker system and the @vite-pwa/sveltekit plugin approach. This is causing:
+
+1. **Service Worker Evaluation Errors**: Browser console shows "ServiceWorker script threw an exception during script evaluation"
+2. **PWA Registration Conflicts**: Multiple service workers attempting to register simultaneously
+3. **Push Notification Failures**: Service worker not responding to push notification requests due to initialization failures
+4. **Workbox Import Issues**: Missing workbox manifest causing runtime errors
+5. **TypeScript Compilation Problems**: Service worker TypeScript not properly compiled for browser execution
+
+The application currently uses a 270-line service worker with comprehensive push notification handling, background sync, and workbox precaching, but architectural conflicts prevent proper functionality.
+
+---
+
+## Current State Analysis
+
+### Service Worker Architecture Issues
+
+**Identified Conflicts:**
+1. **Dual Service Worker Registration**: SvelteKit auto-registers its service worker while @vite-pwa/sveltekit also registers one
+2. **Mixed API Usage**: Service worker uses Workbox APIs but not SvelteKit's `$service-worker` module
+3. **Manifest Integration Gap**: PWA manifest in vite.config.ts not properly integrated with layout files
+4. **TypeScript Processing**: Service worker TypeScript may not be properly processed by vite-pwa plugin
+
+**Current Implementation Strengths (Must Preserve):**
+- ✅ Workbox precaching with `precacheAndRoute()` and `cleanupOutdatedCaches()`
+- ✅ Navigation route handling with `NavigationRoute`
+- ✅ Comprehensive push notification handling with custom actions
+- ✅ Notification click/close handlers with client communication via `postMessage()`
+- ✅ Background sync support for retry operations
+- ✅ Message handling for `SKIP_WAITING`, `GET_VERSION`, `QUEUE_RETRY`
+- ✅ Robust error handling and logging throughout
+- ✅ Keep-alive mechanism for notification display
+- ✅ Custom notification actions (view, retry, dismiss)
+
+### Vite PWA Configuration Analysis
+
+**Current Configuration (vite.config.ts):**
+```typescript
+SvelteKitPWA({
+    strategies: 'injectManifest',
+    filename: 'service-worker.ts',
+    injectManifest: {
+        swSrc: 'src/service-worker.ts',
+        swDest: 'service-worker.js',
+        injectionPoint: 'self.__WB_MANIFEST'
+    },
+    workbox: {
+        globPatterns: ['client/**/*.{js,css,ico,png,svg,webp,woff,woff2}']
+    },
+    devOptions: { enabled: true }
+})
+```
+
+**Issues Found:**
+1. **SvelteKit Service Worker Not Disabled**: `svelte.config.js` doesn't disable SvelteKit's built-in service worker
+2. **Missing PWA Integration**: No PWA registration in app layout or hooks
+3. **Development Mode Conflicts**: Both development service workers competing
+4. **Build Configuration**: TypeScript service worker compilation may have issues
+
+### Documentation Research Findings
+
+**@vite-pwa/sveltekit Plugin Capabilities:**
+1. ✅ **TypeScript Support**: Plugin DOES support TypeScript service workers with `injectManifest` strategy
+2. ✅ **Workbox Integration**: Full workbox library available in injected manifest approach
+3. ✅ **Custom Logic**: Allows complex service worker logic with push notifications
+4. ✅ **SvelteKit Integration**: Designed to work with SvelteKit routing and SSR
+
+**Best Practices Identified:**
+1. **Disable SvelteKit Service Worker**: Must set `serviceWorker: { register: false }` in svelte.config.js
+2. **Use Single Service Worker Strategy**: Choose either SvelteKit's or vite-pwa's approach, not both
+3. **Proper TypeScript Compilation**: Ensure service worker TypeScript is correctly processed
+4. **Development vs Production**: Configure different behaviors for dev/prod environments
+5. **Manifest Integration**: Ensure PWA manifest is properly linked and registered
+
+---
+
+## Root Cause Analysis
+
+### Primary Issue: Conflicting Service Worker Registration
+
+**Problem:** SvelteKit automatically registers its own service worker while @vite-pwa/sveltekit plugin also tries to register one, creating conflicts.
+
+**Evidence:**
+- Service worker evaluation errors in browser console
+- Multiple registration attempts visible in Network/Application tabs
+- Push notifications failing due to unclear service worker ownership
+
+**Solution:** Disable SvelteKit's service worker and use only vite-pwa plugin approach
+
+### Secondary Issue: Workbox Manifest Injection Problems
+
+**Problem:** `self.__WB_MANIFEST` not properly injected during build process
+
+**Evidence:**
+- Current service worker checks for `self.__WB_MANIFEST` existence
+- Build process may not be correctly replacing injection point
+- TypeScript compilation interfering with workbox manifest injection
+
+**Solution:** Ensure proper build pipeline for TypeScript service worker with workbox injection
+
+### Tertiary Issue: Development vs Production Configuration
+
+**Problem:** Different behaviors needed for development and production environments
+
+**Evidence:**
+- Development mode has different service worker registration patterns
+- Hot reloading conflicts with service worker updates
+- SSL requirements different between environments
+
+**Solution:** Configure environment-specific service worker behaviors
+
+---
+
+## Technical Specification
+
+### Architecture Decision: Pure @vite-pwa/sveltekit Approach
+
+**Chosen Strategy:** `injectManifest` with TypeScript service worker
+- ✅ Maintains all current push notification functionality
+- ✅ Supports complex custom service worker logic
+- ✅ Compatible with TypeScript
+- ✅ Integrates with SvelteKit routing
+- ✅ Supports both development and production builds
+
+**Rejected Alternative:** SvelteKit's built-in service worker
+- ❌ Limited customization options
+- ❌ No direct workbox integration
+- ❌ Would require complete rewrite of push notification logic
+- ❌ Less mature PWA features
+
+### Service Worker Structure (Preserve Current Functionality)
+
+**Core Modules to Maintain:**
+1. **Workbox Precaching**: `precacheAndRoute(self.__WB_MANIFEST)`
+2. **Navigation Routing**: `registerRoute(new NavigationRoute(...))`
+3. **Push Notification Handlers**: 
+   - `push` event listener
+   - `notificationclick` event listener
+   - `notificationclose` event listener
+4. **Message Handling**: `message` event for client communication
+5. **Background Sync**: For retry operations
+6. **Error Handling**: Global error and promise rejection handlers
+
+**New Requirements:**
+1. **Proper TypeScript Compilation**: Ensure service worker TS compiles correctly
+2. **Manifest Injection Validation**: Verify `self.__WB_MANIFEST` is properly injected
+3. **Environment Detection**: Different behavior for dev vs prod
+4. **Registration Coordination**: Single service worker registration path
+
+### Configuration Changes Required
+
+#### 1. SvelteKit Configuration (svelte.config.js)
+```javascript
+const config = {
+    preprocess: vitePreprocess(),
+    kit: {
+        adapter: adapter(),
+        serviceWorker: {
+            register: false  // Disable SvelteKit service worker
+        }
+    }
+};
+```
+
+#### 2. Vite PWA Configuration (vite.config.ts)
+```typescript
+SvelteKitPWA({
+    strategies: 'injectManifest',
+    filename: 'service-worker.ts',
+    mode: process.env.NODE_ENV === 'production' ? 'production' : 'development',
+    injectManifest: {
+        swSrc: 'src/service-worker.ts',
+        swDest: 'service-worker.js',
+        injectionPoint: 'self.__WB_MANIFEST'
+    },
+    workbox: {
+        globPatterns: ['client/**/*.{js,css,ico,png,svg,webp,woff,woff2}'],
+        cleanupOutdatedCaches: true,
+        skipWaiting: false,  // Let service worker control this
+        clientsClaim: false  // Let service worker control this
+    },
+    devOptions: {
+        enabled: true,
+        suppressWarnings: true,
+        navigateFallback: '/',
+        type: 'module'
+    }
+})
+```
+
+#### 3. Service Worker Validation (src/service-worker.ts)
+```typescript
+// Enhanced manifest validation
+if (!self.__WB_MANIFEST) {
+    console.warn('[SW] Workbox manifest not injected - running in dev mode or build issue');
+} else if (!Array.isArray(self.__WB_MANIFEST)) {
+    console.error('[SW] Workbox manifest invalid format');
+} else {
+    console.log(`[SW] Workbox manifest loaded with ${self.__WB_MANIFEST.length} entries`);
+    precacheAndRoute(self.__WB_MANIFEST);
+}
+```
+
+### Build Pipeline Validation
+
+**TypeScript Compilation Check:**
+- Ensure service worker TypeScript compiles to valid JavaScript
+- Verify all import statements resolve correctly
+- Confirm workbox injection point is preserved during compilation
+
+**Development Mode Handling:**
+- Service worker should gracefully handle missing workbox manifest in dev
+- Hot reloading should not interfere with service worker functionality
+- HTTPS requirements handled for development server
+
+**Production Build Verification:**
+- Workbox manifest properly injected into compiled service worker
+- All precache entries generated correctly
+- Service worker JavaScript is valid and executable
+
+---
+
+## User Stories
+
+### Story 1: Disable Conflicting Service Worker Registration
+
+**As a** user  
+**I want** only one service worker to be registered  
+**So that** there are no conflicts and PWA functionality works correctly
+
+**Acceptance Criteria:**
+- [ ] SvelteKit's built-in service worker registration is disabled
+- [ ] Only @vite-pwa/sveltekit registers the service worker
+- [ ] No service worker evaluation errors in browser console
+- [ ] Service worker registration visible in browser dev tools shows only one registration
+- [ ] Application continues to work in all browsers (Chrome, Firefox, Safari, Edge)
+
+**Implementation:**
+- Modify `svelte.config.js` to set `serviceWorker: { register: false }`
+- Ensure vite-pwa plugin handles registration correctly
+- Test registration in both development and production builds
+
+**Validation:**
+- Browser console shows no service worker errors
+- Application tab in dev tools shows single service worker registration
+- PWA install prompt appears correctly
+
+### Story 2: Ensure Proper Workbox Manifest Injection
+
+**As a** developer  
+**I want** the workbox manifest to be properly injected into the service worker  
+**So that** precaching and offline functionality work correctly
+
+**Acceptance Criteria:**
+- [ ] `self.__WB_MANIFEST` is properly defined and populated in built service worker
+- [ ] Service worker can successfully call `precacheAndRoute(self.__WB_MANIFEST)`
+- [ ] All client-side assets are precached according to glob patterns
+- [ ] Offline functionality works for precached routes
+- [ ] Cache updates work correctly when app is updated
+
+**Implementation:**
+- Verify `injectionPoint: 'self.__WB_MANIFEST'` configuration
+- Ensure TypeScript compilation preserves injection point
+- Add validation logging for manifest contents
+- Test build output for proper manifest injection
+
+**Validation:**
+- Service worker console logs show manifest with expected entry count
+- Network tab shows precaching requests during service worker installation
+- Offline browsing works for precached resources
+
+### Story 3: Maintain All Push Notification Functionality
+
+**As a** user  
+**I want** push notifications to continue working exactly as before  
+**So that** I receive notifications about recipe extraction progress and completion
+
+**Acceptance Criteria:**
+- [ ] Push notification registration works correctly
+- [ ] Custom notification actions (view, retry, dismiss) function properly
+- [ ] Notification click handlers navigate to correct pages
+- [ ] Background sync continues to work for retry operations
+- [ ] Message passing between service worker and clients works
+- [ ] Keep-alive mechanism maintains notification display
+- [ ] All existing notification types continue to work
+
+**Implementation:**
+- Preserve all existing push notification event handlers
+- Maintain client communication via `postMessage()`
+- Keep background sync registration and handling
+- Preserve notification action definitions
+- Test all notification workflows
+
+**Validation:**
+- Push notifications display correctly with custom actions
+- Clicking notifications navigates to expected pages
+- Retry functionality works through notifications
+- Background sync triggers on network restoration
+- Service worker responds to client messages correctly
+
+### Story 4: Implement Environment-Specific Configuration
+
+**As a** developer  
+**I want** different service worker behavior in development vs production  
+**So that** development workflow is smooth and production is optimized
+
+**Acceptance Criteria:**
+- [ ] Development mode gracefully handles missing workbox manifest
+- [ ] Hot reloading doesn't interfere with service worker functionality
+- [ ] Production mode has full precaching and offline support
+- [ ] SSL requirements handled correctly in both environments
+- [ ] Build process generates appropriate service worker for each environment
+
+**Implementation:**
+- Configure vite-pwa plugin with environment-specific options
+- Add environment detection in service worker
+- Handle development mode gracefully without full precaching
+- Ensure production builds have complete PWA functionality
+
+**Validation:**
+- Development server starts without service worker errors
+- Hot reloading works correctly with service worker active
+- Production builds pass all PWA audits
+- Both environments support push notifications
+
+### Story 5: Validate TypeScript Service Worker Compilation
+
+**As a** developer  
+**I want** the TypeScript service worker to compile correctly  
+**So that** all functionality works and maintenance remains easy
+
+**Acceptance Criteria:**
+- [ ] Service worker TypeScript compiles to valid JavaScript
+- [ ] All import statements resolve correctly in compiled output
+- [ ] Type definitions are available during development
+- [ ] Build process doesn't produce compilation errors
+- [ ] Compiled service worker executes without runtime errors
+
+**Implementation:**
+- Verify vite-pwa plugin TypeScript processing
+- Ensure proper type definitions for service worker globals
+- Test compilation with current workbox imports
+- Validate runtime execution of compiled service worker
+
+**Validation:**
+- Build process completes without TypeScript errors
+- Compiled service worker JavaScript is syntactically valid
+- Runtime execution produces expected console logs
+- All service worker functionality works as expected
+
+---
+
+## Implementation Roadmap
+
+### Phase 1: Configuration Changes (1-2 hours)
+
+**Tasks:**
+1. **Update svelte.config.js**
+   - Add `serviceWorker: { register: false }` to disable SvelteKit's service worker
+   - Test development server startup
+
+2. **Enhance vite.config.ts**
+   - Add environment-specific mode configuration
+   - Update workbox configuration for better manifest generation
+   - Configure development options properly
+
+3. **Validate TypeScript Configuration**
+   - Ensure service worker types are properly configured in tsconfig.json
+   - Verify workbox type definitions are available
+
+**Validation:**
+- Development server starts without conflicts
+- No duplicate service worker registration attempts
+- TypeScript compilation passes without errors
+
+### Phase 2: Service Worker Enhancements (2-3 hours)
+
+**Tasks:**
+1. **Add Manifest Validation**
+   - Enhance workbox manifest existence and format checks
+   - Add informative logging for development vs production modes
+   - Implement graceful degradation when manifest is missing
+
+2. **Environment Detection**
+   - Add runtime environment detection in service worker
+   - Configure different behaviors for development and production
+   - Handle SSL requirements appropriately
+
+3. **Build Process Validation**
+   - Test TypeScript compilation of service worker
+   - Verify workbox manifest injection in built output
+   - Ensure all imports resolve correctly
+
+**Validation:**
+- Service worker initializes correctly in both environments
+- Workbox manifest is properly injected in production builds
+- Development mode works without precaching errors
+
+### Phase 3: Functionality Testing (2-3 hours)
+
+**Tasks:**
+1. **Push Notification Testing**
+   - Test all existing push notification flows
+   - Verify custom notification actions work
+   - Test background sync and retry mechanisms
+   - Validate client-service worker message passing
+
+2. **PWA Feature Testing**
+   - Test app installation prompt
+   - Verify offline functionality with precaching
+   - Test navigation routes work offline
+   - Validate manifest.json integration
+
+3. **Cross-Browser Testing**
+   - Test in Chrome, Firefox, Safari, and Edge
+   - Verify service worker registration in all browsers
+   - Test push notifications across browsers
+   - Validate PWA features in each browser
+
+**Validation:**
+- All push notification types work correctly
+- PWA installation and offline functionality work
+- Cross-browser compatibility maintained
+- No regression in existing functionality
+
+### Phase 4: Performance and Optimization (1-2 hours)
+
+**Tasks:**
+1. **Cache Strategy Optimization**
+   - Review and optimize workbox glob patterns
+   - Ensure efficient precaching strategy
+   - Test cache update mechanisms
+
+2. **Service Worker Performance**
+   - Minimize service worker bundle size
+   - Optimize registration timing
+   - Test service worker update flows
+
+3. **Documentation and Monitoring**
+   - Document new service worker architecture
+   - Add monitoring for service worker errors
+   - Create troubleshooting guide for common issues
+
+**Validation:**
+- Service worker performs efficiently
+- Cache updates work smoothly
+- Performance metrics meet expectations
+- Documentation is complete and accurate
+
+---
+
+## Risk Assessment
+
+### High Risk
+
+**Service Worker Compilation Issues**
+- **Risk:** TypeScript service worker fails to compile correctly with vite-pwa plugin
+- **Impact:** Complete service worker functionality loss
+- **Mitigation:** Thorough testing in development environment before production deployment
+- **Rollback:** Keep current service worker as backup, implement feature flags
+
+**Push Notification Regression**
+- **Risk:** Changes break existing push notification functionality
+- **Impact:** Users stop receiving important notifications about recipe extraction
+- **Mitigation:** Comprehensive testing of all notification workflows before deployment
+- **Rollback:** Immediate rollback capability with previous service worker version
+
+### Medium Risk
+
+**Cross-Browser Compatibility**
+- **Risk:** Changes work in some browsers but not others
+- **Impact:** Partial user base loses PWA functionality
+- **Mitigation:** Test in all major browsers during development
+- **Rollback:** Browser-specific service worker deployment if needed
+
+**Development Workflow Disruption**
+- **Risk:** Changes interfere with hot reloading or development server
+- **Impact:** Slower development process
+- **Mitigation:** Test development environment thoroughly, configure dev-specific options
+- **Rollback:** Environment-specific service worker configurations
+
+### Low Risk
+
+**PWA Manifest Issues**
+- **Risk:** App installation prompt doesn't work correctly
+- **Impact:** Users can't install PWA but core functionality remains
+- **Mitigation:** Test PWA installation on multiple devices and browsers
+- **Rollback:** Manifest configuration is easily reversible
+
+**Cache Strategy Changes**
+- **Risk:** New precaching strategy is less efficient
+- **Impact:** Slower offline performance but functionality maintained
+- **Mitigation:** Monitor cache performance metrics
+- **Rollback:** Revert to previous glob patterns and cache configuration
+
+---
+
+## Definition of Done
+
+### Technical Requirements
+- [ ] SvelteKit's built-in service worker is properly disabled
+- [ ] @vite-pwa/sveltekit plugin successfully registers single service worker
+- [ ] TypeScript service worker compiles correctly for both dev and production
+- [ ] Workbox manifest is properly injected in production builds
+- [ ] All existing push notification functionality is preserved
+- [ ] Service worker works correctly in all major browsers
+- [ ] PWA installation and offline functionality work as expected
+- [ ] No service worker evaluation errors in browser console
+- [ ] Build process completes without TypeScript or compilation errors
+
+### Functional Requirements
+- [ ] Push notifications display correctly with custom actions
+- [ ] Notification click handlers navigate to expected pages
+- [ ] Background sync and retry mechanisms function properly
+- [ ] Service worker responds to client messages (SKIP_WAITING, GET_VERSION, QUEUE_RETRY)
+- [ ] App can be installed as PWA on mobile and desktop
+- [ ] Offline functionality works for precached routes
+- [ ] Hot reloading works correctly in development mode
+- [ ] Production builds pass PWA audits
+
+### Quality Assurance
+- [ ] All existing tests pass
+- [ ] Cross-browser testing completed (Chrome, Firefox, Safari, Edge)
+- [ ] Performance testing shows no significant regression
+- [ ] Security review confirms no new vulnerabilities
+- [ ] Documentation updated to reflect new architecture
+- [ ] Monitoring and logging implemented for service worker issues
+
+### Deployment Requirements
+- [ ] Feature can be deployed to current branch without breaking changes
+- [ ] Rollback plan documented and tested
+- [ ] Production deployment tested in staging environment
+- [ ] Team members trained on new service worker architecture
+- [ ] Troubleshooting guide created for common issues
+
+---
+
+## Success Metrics
+
+### Primary Metrics
+- **Service Worker Registration Success Rate:** 100% (no evaluation errors)
+- **Push Notification Delivery Rate:** Maintain current rate (>95%)
+- **PWA Installation Success Rate:** >90% on supported devices
+- **Offline Functionality Success Rate:** >95% for precached routes
+
+### Secondary Metrics
+- **Build Time:** No increase >10% from baseline
+- **Service Worker Bundle Size:** Maintain or decrease current size
+- **Time to First Notification:** <2 seconds after registration
+- **Cache Hit Rate:** >80% for precached resources
+
+### Qualitative Metrics
+- No user-reported issues with notifications or PWA functionality
+- Developer feedback on improved development workflow
+- Reduced service worker-related error reports
+- Positive PWA audit scores in Lighthouse
+
+---
+
+## Conclusion
+
+This refactoring addresses critical service worker conflicts by adopting a pure @vite-pwa/sveltekit approach while preserving all existing push notification functionality. The plan prioritizes maintaining current features while resolving architectural issues that prevent proper PWA operation.
+
+The implementation is designed to be completed on the current branch with comprehensive testing and rollback capabilities to ensure no disruption to users or development workflow.
+
+**Next Step:** Execute this plan using the `@dev RefactorServiceWorkerForProperPWACompliance` command to begin implementation.