Compare commits

...

17 Commits

Author SHA1 Message Date
1072679360 fix(whisper): handle model warmup events
All checks were successful
Build & Push Docker Image / test (push) Successful in 12s
Build & Push Docker Image / build-and-push (push) Successful in 52s
- Ignore backend model lifecycle webhooks so model warmup does not
  mark jobs done early
- Parse batched SSE messages and relay model load states during
  submit retries

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-05-15 00:08:32 +02:00
f70cefc5e9 fix(progress): separate model warmup state
All checks were successful
Build & Push Docker Image / test (push) Successful in 11s
Build & Push Docker Image / build-and-push (push) Successful in 42s
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-05-12 00:52:33 +02:00
929c482497 refactor(transcript): drop Tonemark rewrite
All checks were successful
Build & Push Docker Image / test (push) Successful in 10s
Build & Push Docker Image / build-and-push (push) Successful in 50s
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-05-12 00:10:32 +02:00
df50e74939 test(vitest): serialize db-backed suites
All checks were successful
Build & Push Docker Image / test (push) Successful in 12s
Build & Push Docker Image / build-and-push (push) Successful in 40s
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-05-11 23:29:31 +02:00
34196b8110 test(push): relax flaky call count
Some checks failed
Build & Push Docker Image / test (push) Failing after 10s
Build & Push Docker Image / build-and-push (push) Has been skipped
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-05-11 23:25:53 +02:00
3a72bb815f fix(postprocess): trim adjacent word overlap
Some checks failed
Build & Push Docker Image / test (push) Failing after 11s
Build & Push Docker Image / build-and-push (push) Has been skipped
Remove residual one-word suffix-prefix carry-over between adjacent caption segments so reprocessed transcripts no longer repeat bridge words across lines.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-05-11 23:24:21 +02:00
6beb436687 fix(postprocess): drop tiny carry-over text
All checks were successful
Build & Push Docker Image / test (push) Successful in 11s
Build & Push Docker Image / build-and-push (push) Successful in 43s
Collapse one-word and very short caption carry-over fragments so reprocessed YouTube transcripts do not retain residual prefix chains.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-05-11 23:14:31 +02:00
672b161cda fix(transcript): collapse rolling segment echoes
All checks were successful
Build & Push Docker Image / test (push) Successful in 12s
Build & Push Docker Image / build-and-push (push) Successful in 45s
Normalize incremental backend hypothesis chains before persistence and ignore stale or replayed webhook callbacks so duplicate transcript text does not survive ingest.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-05-11 22:46:38 +02:00
Giancarmine Salucci
35a2d86dbb test: add beforeEach cleanup in push.test.ts to prevent flaky state leakage
All checks were successful
Build & Push Docker Image / test (push) Successful in 10s
Build & Push Docker Image / build-and-push (push) Successful in 42s
Adds a beforeEach hook that clears subscriptions and resets mocks before
each test, making the suite robust against any state left by a previous
test even if afterEach didn't run cleanly.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-05-10 15:56:05 +02:00
Giancarmine Salucci
470dd1642f fix: clear modelWarming notice when job completes via SSE done event
Some checks failed
Build & Push Docker Image / test (push) Failing after 10s
Build & Push Docker Image / build-and-push (push) Has been skipped
Previously, if a job completed while the model-warming notice was shown
(e.g. model loaded mid-job), the 'Warming up model' banner persisted on
the Done screen because the SSE 'done' handler didn't clear modelWarming.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-05-10 15:52:28 +02:00
Giancarmine Salucci
10a3669b42 fix: FormData stream exhausted on retry + undefined segments crash
All checks were successful
Build & Push Docker Image / test (push) Successful in 32s
Build & Push Docker Image / build-and-push (push) Successful in 46s
Two bugs triggered together when the model was unloaded during a job:

1. submitJob() created FormData/createReadStream once outside the retry loop.
   After a 503, the audio ReadStream was consumed and subsequent retries sent
   an empty body to whisper, causing it to return segments:undefined.

2. webhook handler cast whisperJob.segments as Segment[] without guarding
   against undefined, so deduplicateSegments(undefined) crashed with
   'Cannot read properties of undefined (reading 'map')' — stored as job.error.

Fixes:
- Move FormData + createReadStream inside the retry loop (fresh stream per attempt)
- Use (whisperJob.segments ?? []) in webhook handler
- Add Array.isArray guard at top of deduplicateSegments() as belt-and-suspenders

Tests:
- New: verifies createReadStream called once per attempt (3 attempts = 3 streams)
- New: webhook handles segments:undefined without throwing
- New: webhook handles segments:null without throwing
- 150/150 passing

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-05-10 15:37:07 +02:00
Giancarmine Salucci
a76625d378 ci: use npm install instead of npm ci to avoid lock file version mismatch
All checks were successful
Build & Push Docker Image / test (push) Successful in 10s
Build & Push Docker Image / build-and-push (push) Successful in 44s
Lock file was generated with npm 11 (Node 24), CI runs npm 10 (Node 22).
npm install avoids the strict sync check and matches the Dockerfile.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-05-09 15:54:32 +02:00
Giancarmine Salucci
76051e52dd ci: add test job before Docker build
Some checks failed
Build & Push Docker Image / test (push) Failing after 45s
Build & Push Docker Image / build-and-push (push) Has been skipped
Run vitest before building the image so a failing test blocks the push.
build-and-push now depends on the test job passing.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-05-09 15:51:24 +02:00
Giancarmine Salucci
53f874aec7 feat: proxy POST /model/unload endpoint
All checks were successful
Build & Push Docker Image / build-and-push (push) Successful in 43s
- Add unloadModel() to whisper.ts: POSTs to /model/unload with 10s
  timeout, returns parsed JSON body, throws on non-ok response
- Create src/routes/api/model/unload/+server.ts: thin POST proxy,
  passes whisper's response through, returns 502 if whisper unreachable
- Add 3 unloadModel tests (success, WHISPER_URL config, error propagation)
  — 147/147 passing

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-05-09 15:48:47 +02:00
Giancarmine Salucci
04142b17a8 feat: whisper-side cancellation + SSE-triggered retry
All checks were successful
Build & Push Docker Image / build-and-push (push) Successful in 48s
- Add cancelJob() to whisper.ts: sends DELETE /jobs/:id to the whisper
  server (best-effort, errors silently ignored)
- DELETE /api/jobs/[id] now calls cancelJob() when cancelling an active
  job that has a whisperJobId, stopping GPU use immediately
- Webhook handler guards against locally-cancelled jobs: returns ok early
  so whisper's late completion cannot overwrite cancelled status or send
  a phantom 'Transcript ready' notification
- Replace blind sleep(Retry-After + 1s) in submitJob() with
  waitForModelReady(): subscribes to /model/events SSE and proceeds as
  soon as state:ready arrives; falls back to the Retry-After timeout if
  SSE is unreachable or closes without model_ready
- Refactor retry tests to use URL-aware makeJobFetch() helper; add 7 new
  tests (3 SSE-triggered retry, 3 cancelJob, 1 webhook cancelled-guard)
  — 144/144 passing

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-05-09 00:40:40 +02:00
Giancarmine Salucci
01845bec25 test: comprehensive coverage for 503 retry loop and getModelStatus
submitJob — 503 retry behavior (10 new tests):
- calls onModelWaiting with correct state + retryAfterSecs on each 503
- retries until model ready and returns job_id
- tracks all three model states (unloaded, loading, waiting_for_gpu)
- uses retry_after_secs from response body
- falls back to Retry-After header when body field absent
- falls back to 15s when both body and header are absent
- throws after maxAttempts exhausted (fetch called exactly N times)
- does NOT call onModelWaiting for non-503 errors
- does NOT retry on non-503 errors (throws immediately, one fetch call)
- works correctly without an onModelWaiting callback

getModelStatus (6 new tests):
- returns parsed status for each model state tag
- includes optional fields (loaded_at, vram_*, retry_in_secs)
- calls the correct WHISPER_URL/model/status endpoint
- throws when server returns non-ok

Uses vi.useFakeTimers()/runAllTimersAsync() to eliminate real delays.
Rejection handler attached before timer advance to avoid unhandled-rejection
false positives from Vitest's detector.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-05-09 00:14:09 +02:00
Giancarmine Salucci
b90d57984c feat: model-on-demand lifecycle — retry on 503, live status pill, warming indicator
- whisper.ts: add getModelStatus(); fix submitJob() to retry on 503 using
  Retry-After header instead of throwing; optional onModelWaiting callback
  lets the pipeline surface model state to the UI during the wait
- pipeline.ts: pass onModelWaiting callback → emits model_warming SSE event
  so the job detail page can show 'Warming up model…' while waiting
- types.ts: add ModelStateTag union and ModelStatus interface
- api/model/status: GET route proxies whisper /model/status (falls back to
  {state:'unloaded'} if whisper unreachable)
- api/model/events: GET route relays whisper SSE stream to the browser;
  AbortController tied to request.signal cleans up on disconnect
- layout.svelte: status pill is now live — initial fetch + EventSource on
  /api/model/events; dot colour + label reflect real model state with a
  pulsing animation while loading or waiting_for_gpu
- jobs/[id]/+page.svelte: handle model_warming event type → show a yellow
  'Warming up model…' sub-label with spinner inside the progress card
- whisper.test.ts: update submitJob mocks to status:202 to match real API

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-05-09 00:08:21 +02:00
29 changed files with 1452 additions and 529 deletions

View File

@@ -15,8 +15,28 @@ env:
IMAGE_NAME: mozempk/tonemark
jobs:
test:
runs-on: ubuntu-latest
steps:
- name: Checkout repository
uses: actions/checkout@v4
- name: Set up Node.js
uses: actions/setup-node@v4
with:
node-version: '22'
cache: 'npm'
- name: Install dependencies
run: npm install
- name: Run tests
run: npm test
build-and-push:
runs-on: ubuntu-latest
needs: test
steps:
- name: Checkout repository

42
package-lock.json generated
View File

@@ -12,7 +12,8 @@
"better-sqlite3": "^12.9.0",
"form-data": "^4.0.5",
"node-fetch": "^3.3.2",
"web-push": "^3.6.7"
"web-push": "^3.6.7",
"youtube-transcript": "^1.3.1"
},
"devDependencies": {
"@sveltejs/adapter-auto": "^7.0.1",
@@ -89,6 +90,27 @@
"node": ">=18"
}
},
"node_modules/@emnapi/core": {
"version": "1.10.0",
"resolved": "https://registry.npmjs.org/@emnapi/core/-/core-1.10.0.tgz",
"integrity": "sha512-yq6OkJ4p82CAfPl0u9mQebQHKPJkY7WrIuk205cTYnYe+k2Z8YBh11FrbRG/H6ihirqcacOgl2BIO8oyMQLeXw==",
"license": "MIT",
"optional": true,
"dependencies": {
"@emnapi/wasi-threads": "1.2.1",
"tslib": "^2.4.0"
}
},
"node_modules/@emnapi/runtime": {
"version": "1.10.0",
"resolved": "https://registry.npmjs.org/@emnapi/runtime/-/runtime-1.10.0.tgz",
"integrity": "sha512-ewvYlk86xUoGI0zQRNq/mC+16R1QeDlKQy21Ki3oSYXNgLb45GV1P6A0M+/s6nyCuNDqe5VpaY84BzXGwVbwFA==",
"license": "MIT",
"optional": true,
"dependencies": {
"tslib": "^2.4.0"
}
},
"node_modules/@emnapi/wasi-threads": {
"version": "1.2.1",
"resolved": "https://registry.npmjs.org/@emnapi/wasi-threads/-/wasi-threads-1.2.1.tgz",
@@ -896,7 +918,6 @@
"resolved": "https://registry.npmjs.org/@sveltejs/kit/-/kit-2.59.1.tgz",
"integrity": "sha512-d8OON70AphLdDesuTIl//M2O6fRTIicX8aYv8vhCiYEhTTI2OboKqey0Hu1A4VFhqwgqtq0vKDmPFGkw8kKmgw==",
"license": "MIT",
"peer": true,
"dependencies": {
"@standard-schema/spec": "^1.0.0",
"@sveltejs/acorn-typescript": "^1.0.5",
@@ -938,7 +959,6 @@
"resolved": "https://registry.npmjs.org/@sveltejs/vite-plugin-svelte/-/vite-plugin-svelte-7.1.1.tgz",
"integrity": "sha512-FOJdbE5pxae68DoTBJ49t1dIA7TSmMHR6CsuJhX90cO/UfrEMHA7KJNUj3WdZuUDJPu4ujqpJ2Tgqd2gTWr6Xg==",
"license": "MIT",
"peer": true,
"dependencies": {
"deepmerge": "^4.3.1",
"magic-string": "^0.30.21",
@@ -1313,7 +1333,6 @@
"integrity": "sha512-38C0/Ddb7HcRG0Z4/DUem8x57d2p9jYgp18mkaYswEOQBGsI1CG4f/hjm0ZCeaJfWhSZ4k7jgs29V1Zom7Ki9A==",
"dev": true,
"license": "MIT",
"peer": true,
"dependencies": {
"@bcoe/v8-coverage": "^1.0.2",
"@vitest/utils": "4.1.5",
@@ -1467,7 +1486,6 @@
"resolved": "https://registry.npmjs.org/acorn/-/acorn-8.16.0.tgz",
"integrity": "sha512-UVJyE9MttOsBQIDKw1skb9nAwQuR5wuGD3+82K6JgJlm/Y+KI92oNsMNGZCYdDsVtRHSak0pcV5Dno5+4jh9sw==",
"license": "MIT",
"peer": true,
"bin": {
"acorn": "bin/acorn"
},
@@ -3021,7 +3039,6 @@
"resolved": "https://registry.npmjs.org/rollup/-/rollup-4.60.3.tgz",
"integrity": "sha512-pAQK9HalE84QSm4Po3EmWIZPd3FnjkShVkiMlz1iligWYkWQ7wHYd1PF/T7QZ5TVSD6uSTon5gBVMSM4JfBV+A==",
"license": "MIT",
"peer": true,
"dependencies": {
"@types/estree": "1.0.8"
},
@@ -3255,7 +3272,6 @@
"resolved": "https://registry.npmjs.org/svelte/-/svelte-5.55.5.tgz",
"integrity": "sha512-2uCs/LZ9us+AktdzYJM8OcxQ8qnPS1kpaO7syGT/MgO+6Qr1Ybl+TqPq+97u7PHqmmMlye5ZkoyXONy5mjjAbw==",
"license": "MIT",
"peer": true,
"dependencies": {
"@jridgewell/remapping": "^2.3.4",
"@jridgewell/sourcemap-codec": "^1.5.0",
@@ -3428,7 +3444,6 @@
"integrity": "sha512-y2TvuxSZPDyQakkFRPZHKFm+KKVqIisdg9/CZwm9ftvKXLP8NRWj38/ODjNbr43SsoXqNuAisEf1GdCxqWcdBw==",
"devOptional": true,
"license": "Apache-2.0",
"peer": true,
"bin": {
"tsc": "bin/tsc",
"tsserver": "bin/tsserver"
@@ -3455,7 +3470,6 @@
"resolved": "https://registry.npmjs.org/vite/-/vite-8.0.10.tgz",
"integrity": "sha512-rZuUu9j6J5uotLDs+cAA4O5H4K1SfPliUlQwqa6YEwSrWDZzP4rhm00oJR5snMewjxF5V/K3D4kctsUTsIU9Mw==",
"license": "MIT",
"peer": true,
"dependencies": {
"lightningcss": "^1.32.0",
"picomatch": "^4.0.4",
@@ -3553,7 +3567,6 @@
"integrity": "sha512-9Xx1v3/ih3m9hN+SbfkUyy0JAs72ap3r7joc87XL6jwF0jGg6mFBvQ1SrwaX+h8BlkX6Hz9shdd1uo6AF+ZGpg==",
"dev": true,
"license": "MIT",
"peer": true,
"dependencies": {
"@vitest/expect": "4.1.5",
"@vitest/mocker": "4.1.5",
@@ -3689,6 +3702,15 @@
"integrity": "sha512-l4Sp/DRseor9wL6EvV2+TuQn63dMkPjZ/sp9XkghTEbV9KlPS1xUsZ3u7/IQO4wxtcFB4bgpQPRcR3QCvezPcQ==",
"license": "ISC"
},
"node_modules/youtube-transcript": {
"version": "1.3.1",
"resolved": "https://registry.npmjs.org/youtube-transcript/-/youtube-transcript-1.3.1.tgz",
"integrity": "sha512-NDCjwad113TGybbYF51y9Z4tcwzBHUZWQdF9veULNca18L+FdDbHHtTHIr69WVa3bB90l67S8kN0HtL2JO9fhg==",
"license": "MIT",
"engines": {
"node": ">=18.0.0"
}
},
"node_modules/zimmerframe": {
"version": "1.1.4",
"resolved": "https://registry.npmjs.org/zimmerframe/-/zimmerframe-1.1.4.tgz",

View File

@@ -34,6 +34,7 @@
"better-sqlite3": "^12.9.0",
"form-data": "^4.0.5",
"node-fetch": "^3.3.2",
"web-push": "^3.6.7"
"web-push": "^3.6.7",
"youtube-transcript": "^1.3.1"
}
}

View File

@@ -16,10 +16,10 @@
type RecordState = 'idle' | 'requesting' | 'recording' | 'stopping';
let state = $state<RecordState>('idle');
let error = $state('');
let elapsed = $state(0); // seconds
let liveData = $state<Float32Array | null>(null);
let recordState: RecordState = $state('idle');
let error: string = $state('');
let elapsed: number = $state(0); // seconds
let liveData: Float32Array | null = $state(null);
let mediaRecorder: MediaRecorder | null = null;
let chunks: Blob[] = [];
@@ -60,12 +60,12 @@
async function startRecording() {
error = '';
state = 'requesting';
recordState = 'requesting';
try {
stream = await navigator.mediaDevices.getUserMedia({ audio: true });
} catch {
error = 'Microphone access denied';
state = 'idle';
recordState = 'idle';
return;
}
@@ -81,11 +81,11 @@
elapsed = 0;
timerInterval = setInterval(() => elapsed++, 1000);
state = 'recording';
recordState = 'recording';
}
function stopRecording() {
state = 'stopping';
recordState = 'stopping';
mediaRecorder?.stop();
if (timerInterval) clearInterval(timerInterval);
if (animFrame) cancelAnimationFrame(animFrame);
@@ -99,7 +99,7 @@
const ext = mime.includes('ogg') ? 'ogg' : mime.includes('mp4') ? 'mp4' : 'webm';
const blob = new Blob(chunks, { type: mime });
const filename = `recording-${new Date().toISOString().slice(0, 19).replace(/[T:]/g, '-')}.${ext}`;
state = 'idle';
recordState = 'idle';
ondone?.(blob, filename);
}
@@ -116,15 +116,18 @@
{ length: IDLE_BARS },
(_, i) => 3 + Math.abs(Math.sin(i * 0.7) + Math.cos(i * 0.31)) * 20
);
const liveBars = $derived.by<number[]>(() =>
liveData ? Array.from(liveData.slice(0, IDLE_BARS), (value) => Number(value)) : []
);
</script>
<div class="recorder">
<!-- Waveform display -->
<div class="waveform-area" aria-hidden="true">
{#if state === 'recording' && liveData}
{#if recordState === 'recording' && liveData}
<!-- Live waveform from AnalyserNode -->
<svg viewBox="0 0 {IDLE_BARS * 5} 28" preserveAspectRatio="none" class="waveform-svg">
{#each Array.from(liveData).slice(0, IDLE_BARS) as v, i}
{#each liveBars as v, i}
{@const h = 2 + v * 24}
<rect
x={i * 5}
@@ -147,8 +150,8 @@
width="3"
height={h}
rx="1.5"
fill={state === 'idle' ? 'rgba(255,255,255,0.15)' : accent}
opacity={state === 'idle' ? 1 : 0.3}
fill={recordState === 'idle' ? 'rgba(255,255,255,0.15)' : accent}
opacity={recordState === 'idle' ? 1 : 0.3}
/>
{/each}
</svg>
@@ -156,7 +159,7 @@
</div>
<!-- Timer (recording only) -->
{#if state === 'recording'}
{#if recordState === 'recording'}
<div class="timer" style="color: {accent}">
<span class="rec-dot" style="background: {accent}"></span>
{formatTime(elapsed)}
@@ -170,15 +173,15 @@
<!-- Buttons -->
<div class="btn-row">
{#if state === 'idle' || state === 'requesting'}
{#if recordState === 'idle' || recordState === 'requesting'}
<button
class="btn-record"
style="background: {accent}; color: #0c0d10;"
onclick={startRecording}
disabled={state === 'requesting'}
disabled={recordState === 'requesting'}
aria-label="Start recording"
>
{#if state === 'requesting'}
{#if recordState === 'requesting'}
<svg width="13" height="13" viewBox="0 0 13 13" style="animation: spin 1s linear infinite">
<circle cx="6.5" cy="6.5" r="5" stroke="currentColor" stroke-width="1.5" fill="none" stroke-dasharray="20 12"/>
</svg>
@@ -190,7 +193,7 @@
Record
{/if}
</button>
{:else if state === 'recording'}
{:else if recordState === 'recording'}
<button
class="btn-stop"
onclick={stopRecording}

52
src/lib/job-progress.ts Normal file
View File

@@ -0,0 +1,52 @@
import type { Job, JobStatus } from '$lib/types.js';
export const TERMINAL_JOB_STATUSES: readonly JobStatus[] = ['done', 'failed', 'cancelled'];
const STATUS_LABELS: Record<JobStatus, string> = {
pending: 'Pending',
downloading: 'Downloading',
preparing: 'Preparing',
warming_model: 'Loading model',
transcribing: 'Transcribing',
processing: 'Processing',
done: 'Done',
failed: 'Failed',
cancelled: 'Cancelled'
};
const STATUS_COLORS: Record<JobStatus, string> = {
done: '#cdf24e',
failed: '#ff6b6b',
cancelled: 'rgba(232,233,236,0.3)',
processing: '#76daa2',
transcribing: '#80c7f7',
warming_model: '#76daa2',
preparing: '#fbc94b',
downloading: '#a78bfa',
pending: 'rgba(232,233,236,0.4)'
};
export function isTerminalJobStatus(status: JobStatus): boolean {
return TERMINAL_JOB_STATUSES.includes(status);
}
export function getJobStatusLabel(status: JobStatus): string {
return STATUS_LABELS[status];
}
export function getJobStatusColor(status: JobStatus): string {
return STATUS_COLORS[status];
}
export function getDisplayJobProgress(
job: Pick<Job, 'status' | 'progress' | 'segmentsJson'>,
options: { hasTranscript?: boolean } = {}
): number {
const progress = Math.max(0, Math.min(100, Math.round(job.progress)));
if (job.status === 'warming_model') return Math.min(progress, 15);
if (!isTerminalJobStatus(job.status)) return Math.min(progress, 99);
if (job.status === 'done' && !options.hasTranscript) return Math.min(progress, 99);
return progress;
}

View File

@@ -1,8 +1,9 @@
import { execFile } from 'child_process';
import { promisify } from 'util';
import { existsSync } from 'fs';
import { mkdir, unlink, writeFile } from 'fs/promises';
import { mkdir, writeFile } from 'fs/promises';
import { join } from 'path';
import { fetchTranscript, type TranscriptResponse } from 'youtube-transcript';
const execFileAsync = promisify(execFile);
const TMP_DIR = join(process.env.DATA_DIR ?? '/tmp/.whisper-pwa', 'downloads');
@@ -26,43 +27,33 @@ export interface AudioResult {
export type DownloadResult = CaptionResult | AudioResult;
/** Try to get auto-generated captions from YouTube. Returns null if unavailable. */
async function tryGetCaptions(url: string, outDir: string): Promise<CaptionResult | null> {
const jsonPath = join(outDir, 'info.json');
async function tryGetCaptions(url: string, _outDir: string): Promise<CaptionResult | null> {
try {
await execFileAsync('yt-dlp', [
'--write-auto-subs',
'--sub-langs', 'en.*',
'--skip-download',
'--write-info-json',
'--no-playlist',
'-o', join(outDir, '%(title)s.%(ext)s'),
url
]);
// Find the VTT/SRT file
const { readdirSync } = await import('fs');
const files = readdirSync(outDir);
const vttFile = files.find((f) => f.endsWith('.vtt') || f.endsWith('.srt'));
if (!vttFile) return null;
let title = 'Untitled';
if (existsSync(jsonPath)) {
try {
const info = JSON.parse((await import('fs')).readFileSync(jsonPath, 'utf8'));
title = info.title ?? title;
} catch { /* ignore */ }
}
const content = (await import('fs')).readFileSync(join(outDir, vttFile), 'utf8');
const segments = parseVtt(content);
const transcript = await fetchTranscript(url, { lang: 'en' });
const segments = transcriptEntriesToSegments(transcript);
if (segments.length === 0) return null;
const title = await getYouTubeTitle(url);
return { type: 'captions', segments, title };
} catch {
return null;
}
}
async function getYouTubeTitle(url: string): Promise<string> {
try {
const { stdout } = await execFileAsync('yt-dlp', [
'--dump-single-json',
'--skip-download',
'--no-playlist',
url
]);
return JSON.parse(stdout).title ?? 'Untitled';
} catch {
return 'Untitled';
}
}
/** Download best audio from YouTube. Returns path to audio file. */
async function downloadAudio(url: string, outDir: string): Promise<{ audioPath: string; title: string }> {
await execFileAsync('yt-dlp', [
@@ -124,39 +115,22 @@ export async function cleanupJobTmp(jobId: string) {
} catch { /* ignore */ }
}
/** Parse a WebVTT string into segments. */
function parseVtt(
content: string
export function transcriptEntriesToSegments(
entries: TranscriptResponse[]
): Array<{ index: number; start: number; end: number; text: string; words: [] }> {
const segments: Array<{ index: number; start: number; end: number; text: string; words: [] }> = [];
const blocks = content.split(/\n\n+/);
let index = 0;
for (const block of blocks) {
const lines = block.trim().split('\n');
const timeLine = lines.find((l) => l.includes('-->'));
if (!timeLine) continue;
const [startStr, endStr] = timeLine.split('-->').map((s) => s.trim().split(' ')[0]);
const start = vttTimeToSec(startStr);
const end = vttTimeToSec(endStr);
const text = lines
.filter((l) => !l.includes('-->') && !/^\d+$/.test(l.trim()) && l.trim())
.join(' ')
.replace(/<[^>]+>/g, '')
.trim();
if (text) {
segments.push({ index: index++, start, end, text, words: [] });
}
}
return segments;
}
function vttTimeToSec(t: string): number {
const parts = t.split(':').map(Number);
if (parts.length === 3) return parts[0] * 3600 + parts[1] * 60 + parts[2];
if (parts.length === 2) return parts[0] * 60 + parts[1];
return parts[0];
const useMilliseconds = entries.some((entry) => entry.offset > 1000 || entry.duration > 1000);
return entries
.map((entry) => {
const start = useMilliseconds ? entry.offset / 1000 : entry.offset;
const duration = useMilliseconds ? entry.duration / 1000 : entry.duration;
return {
index: 0,
start,
end: start + duration,
text: entry.text.trim(),
words: [] as []
};
})
.filter((entry) => entry.text.length > 0)
.map((entry, index) => ({ ...entry, index }));
}

View File

@@ -96,15 +96,13 @@ async function runJob(
if (captionSegments) {
// Caption fast path — skip whisper
const { deduplicateSegments } = await import('./postprocess.js');
const { writeOutputs } = await import('./formatter.js');
const segments = deduplicateSegments(captionSegments);
const paths = await writeOutputs(segments, title, jobId);
const paths = await writeOutputs(captionSegments, title, jobId);
updateJob({
id: jobId,
status: 'done',
progress: 100,
segmentsJson: JSON.stringify(segments),
segmentsJson: JSON.stringify(captionSegments),
outputDir: paths.srt.replace(/\/[^/]+$/, '')
});
emitProgress(jobId, { type: 'done' });
@@ -126,11 +124,16 @@ async function runJob(
// ── 4. Submit to whisper with webhook ────────────────────────────────
setJobStatus(jobId, 'transcribing', 10);
emitProgress(jobId, { type: 'status', status: 'transcribing' });
emitProgress(jobId, { type: 'status', status: 'transcribing', progress: 10 });
const webhookUrl = `${WEBHOOK_BASE_URL}/api/webhook/${jobId}`;
const whisperJobId = await submitJob(wavPath, webhookUrl, language);
const whisperJobId = await submitJob(wavPath, webhookUrl, language, (state, retryAfterSecs) => {
setJobStatus(jobId, 'warming_model', 10);
emitProgress(jobId, { type: 'model_warming', status: 'warming_model', state, retryAfterSecs, progress: 10 });
});
updateJob({ id: jobId, whisperJobId });
setJobStatus(jobId, 'transcribing', 10);
emitProgress(jobId, { type: 'status', status: 'transcribing', progress: 10 });
// ── 5. Open SSE for live progress (non-blocking relay) ───────────────
streamJob(

View File

@@ -1,108 +0,0 @@
import type { Segment } from '$lib/types.js';
// ── Collapse consecutive repeated phrases within a segment's text ────────────
function collapseRepeats(text: string): string {
let prev = '';
// Keep applying until stable
while (true) {
const next = collapseOnce(text);
if (next === prev || next === text) return next;
prev = text;
text = next;
}
}
function collapseOnce(text: string): string {
// Match any repeated phrase (2+ words) appearing consecutively
return text.replace(/\b(.{10,}?)\s+\1\b/gi, '$1');
}
// ── Merge consecutive segments with identical (or near-identical) text ───────
function normalise(s: string) {
return s.toLowerCase().replace(/[^\w\s]/g, '').replace(/\s+/g, ' ').trim();
}
function mergeConsecutive(segments: Segment[]): Segment[] {
const out: Segment[] = [];
for (const seg of segments) {
const last = out[out.length - 1];
if (last && normalise(last.text) === normalise(seg.text)) {
last.end = seg.end;
} else {
out.push({ ...seg });
}
}
return out;
}
// ── N-gram deduplication ─────────────────────────────────────────────────────
const NGRAM_N = 6;
const LOOKBACK_CHARS = 500;
const SIMILARITY_THRESHOLD = 0.6;
function ngrams(text: string, n: number): string[] {
const words = text.toLowerCase().split(/\s+/);
const grams: string[] = [];
for (let i = 0; i <= words.length - n; i++) {
grams.push(words.slice(i, i + n).join(' '));
}
return grams;
}
function jaccardSimilarity(a: string, b: string): number {
const ga = new Set(ngrams(a, NGRAM_N));
const gb = new Set(ngrams(b, NGRAM_N));
// If neither text is long enough to produce n-grams they cannot be compared;
// treat as dissimilar so short segments are never incorrectly discarded.
if (ga.size === 0 && gb.size === 0) return 0;
const intersection = [...ga].filter((g) => gb.has(g)).length;
const union = new Set([...ga, ...gb]).size;
return union === 0 ? 0 : intersection / union;
}
function ngramDedup(segments: Segment[]): Segment[] {
const out: Segment[] = [];
for (const seg of segments) {
const windowText = out
.slice(-20)
.map((s) => s.text)
.join(' ')
.slice(-LOOKBACK_CHARS);
if (windowText.length > 0 && jaccardSimilarity(seg.text, windowText) >= SIMILARITY_THRESHOLD) {
continue; // duplicate — skip
}
out.push(seg);
}
return out;
}
// ── Full deduplication pipeline ──────────────────────────────────────────────
export function deduplicateSegments(segments: Segment[]): Segment[] {
// 1. Collapse repeats within each segment's text
let result = segments.map((s) => ({
...s,
text: collapseRepeats(s.text.trim())
}));
// 2. Remove empty segments
result = result.filter((s) => s.text.length > 0);
// 3. First merge pass
result = mergeConsecutive(result);
// 4. N-gram dedup
result = ngramDedup(result);
// 5. Second merge pass (catches new adjacencies after dedup)
result = mergeConsecutive(result);
// 6. Re-index
result.forEach((s, i) => (s.index = i));
return result;
}

View File

@@ -1,22 +1,131 @@
import { execFile } from 'child_process';
import { promisify } from 'util';
import type { ModelStateTag, ModelStatus } from '$lib/types.js';
const execFileAsync = promisify(execFile);
const MODEL_STATES = new Set<ModelStateTag>(['unloaded', 'loading', 'waiting_for_gpu', 'ready']);
function isModelStateTag(value: unknown): value is ModelStateTag {
return typeof value === 'string' && MODEL_STATES.has(value as ModelStateTag);
}
function extractSseMessages(buffer: string): { messages: { eventType: string; data: string }[]; rest: string } {
const normalized = buffer.replace(/\r/g, '');
const chunks = normalized.split('\n\n');
const rest = chunks.pop() ?? '';
const messages = chunks
.map((chunk) => {
let eventType = '';
const dataLines: string[] = [];
for (const line of chunk.split('\n')) {
if (line.startsWith('event:')) {
eventType = line.slice(6).trim();
} else if (line.startsWith('data:')) {
dataLines.push(line.slice(5).trim());
}
}
return { eventType, data: dataLines.join('\n') };
})
.filter((message) => message.data.length > 0);
return { messages, rest };
}
function whisperUrl() {
return process.env.WHISPER_URL ?? 'http://localhost:8080';
}
/** Submit an audio file to whisper-rtx2080. Returns the whisper job id. */
/** Get the current model state from whisper-rtx2080. */
export async function getModelStatus(): Promise<ModelStatus> {
const { default: fetch } = await import('node-fetch');
const res = await fetch(`${whisperUrl()}/model/status`, {
signal: AbortSignal.timeout(5000)
});
if (!res.ok) throw new Error(`/model/status returned ${res.status}`);
return res.json() as Promise<ModelStatus>;
}
/**
* Wait for the whisper model to become ready.
*
* Subscribes to /model/events SSE and resolves as soon as a payload with
* state:"ready" arrives. Falls back to a plain timeout (`timeoutMs`) if the
* SSE connection fails or closes without that event, so the retry loop can
* try again without hanging indefinitely.
*/
async function waitForModelReady(
timeoutMs: number,
onStateChange?: (state: ModelStateTag) => void
): Promise<void> {
const { default: fetch } = await import('node-fetch');
const ac = new AbortController();
return new Promise((resolve) => {
let done = false;
const finish = () => {
if (!done) {
done = true;
ac.abort();
resolve();
}
};
const timer = setTimeout(finish, timeoutMs);
fetch(`${whisperUrl()}/model/events`, { signal: ac.signal as AbortSignal })
.then(async (res) => {
if (!res.body) {
clearTimeout(timer);
return finish();
}
let buf = '';
for await (const chunk of res.body) {
if (ac.signal.aborted) break;
buf += chunk.toString();
const { messages, rest } = extractSseMessages(buf);
buf = rest;
for (const message of messages) {
try {
const payload = JSON.parse(message.data) as { state?: unknown };
if (!isModelStateTag(payload.state)) continue;
if (payload.state === 'ready') {
clearTimeout(timer);
finish();
return;
}
onStateChange?.(payload.state);
} catch { /* ignore parse errors */ }
}
}
// Stream ended without model_ready → proceed to retry immediately
clearTimeout(timer);
finish();
})
.catch(() => {
// SSE unreachable — fallback timer will fire eventually
});
});
}
/**
* Submit an audio file to whisper-rtx2080. Returns the whisper job id.
*
* Handles 503 (model not ready) transparently: retries after subscribing to
* /model/events SSE — proceeds as soon as state:"ready" arrives, or after the
* Retry-After timeout elapses (whichever comes first).
* Calls `onModelWaiting` on each 503 so the caller can surface the wait to the user.
*/
export async function submitJob(
wavPath: string,
webhookUrl: string,
language?: string
language?: string,
onModelWaiting?: (state: ModelStateTag, retryAfterSecs: number) => void,
maxAttempts = 20
): Promise<string> {
const FormData = (await import('form-data')).default;
const { createReadStream } = await import('fs');
const { default: fetch } = await import('node-fetch');
for (let attempt = 1; attempt <= maxAttempts; attempt++) {
// Recreate form with a fresh readable stream on every attempt.
// A consumed ReadStream cannot be rewound, so reusing it across retries
// would send an empty body to whisper after the first 503.
const form = new FormData();
form.append('audio', createReadStream(wavPath));
form.append('task', 'transcribe');
@@ -29,13 +138,58 @@ export async function submitJob(
headers: form.getHeaders()
});
if (!res.ok) {
if (res.status === 202) {
const json = (await res.json()) as { job_id: string };
return json.job_id;
}
if (res.status === 503) {
const body = (await res.json().catch(() => ({}))) as {
state?: string;
retry_after_secs?: number;
};
const state = isModelStateTag(body.state) ? body.state : 'unloaded';
const waitSecs = body.retry_after_secs ?? parseInt(res.headers.get('Retry-After') ?? '15');
onModelWaiting?.(state, waitSecs);
let lastState = state;
await waitForModelReady((waitSecs + 1) * 1000, (nextState) => {
if (nextState === lastState) return;
lastState = nextState;
onModelWaiting?.(nextState, waitSecs);
});
continue;
}
const text = await res.text();
throw new Error(`whisper /jobs returned ${res.status}: ${text}`);
}
const json = (await res.json()) as { job_id: string };
return json.job_id;
throw new Error(`Whisper model did not become ready after ${maxAttempts} attempts`);
}
/** Unload the model from VRAM. Throws if the whisper server returns non-ok. */
export async function unloadModel(): Promise<{ ok: boolean }> {
const { default: fetch } = await import('node-fetch');
const res = await fetch(`${whisperUrl()}/model/unload`, {
method: 'POST',
signal: AbortSignal.timeout(10000)
});
if (!res.ok) throw new Error(`/model/unload returned ${res.status}`);
return res.json() as Promise<{ ok: boolean }>;
}
/**
* Cancel a queued or running job on the whisper server (best-effort).
* Errors are silently ignored — local job status is already set to cancelled.
*/
export async function cancelJob(whisperJobId: string): Promise<void> {
try {
const { default: fetch } = await import('node-fetch');
await fetch(`${whisperUrl()}/jobs/${whisperJobId}`, {
method: 'DELETE',
signal: AbortSignal.timeout(5000)
});
} catch { /* best-effort */ }
}
/** Open an SSE stream from whisper and call onProgress/onDone callbacks. */
@@ -52,20 +206,12 @@ export async function streamJob(
let buf = '';
for await (const chunk of res.body) {
buf += chunk.toString();
const lines = buf.split('\n');
buf = lines.pop() ?? '';
let eventType = '';
let dataLine = '';
for (const line of lines) {
if (line.startsWith('event:')) eventType = line.slice(6).trim();
else if (line.startsWith('data:')) dataLine = line.slice(5).trim();
}
if (!dataLine) continue;
const { messages, rest } = extractSseMessages(buf);
buf = rest;
for (const message of messages) {
try {
const payload = JSON.parse(dataLine);
const payload = JSON.parse(message.data);
if (payload.type === 'progress') {
onProgress(payload.percent ?? 0, payload.chunk ?? 0, payload.total ?? 0);
} else if (payload.type === 'done') {
@@ -77,6 +223,7 @@ export async function streamJob(
}
} catch { /* ignore parse errors */ }
}
}
}
/** Check if the whisper server is healthy. */

View File

@@ -1,6 +1,27 @@
export type AudioMode = 'auto' | 'standard' | 'aggressive' | 'none';
export type JobStatus = 'pending' | 'downloading' | 'preparing' | 'transcribing' | 'processing' | 'done' | 'failed' | 'cancelled';
export type ModelStateTag = 'unloaded' | 'loading' | 'waiting_for_gpu' | 'ready';
export interface ModelStatus {
state: ModelStateTag;
loaded_at?: string;
vram_needed_mb?: number;
vram_free_mb?: number;
retry_in_secs?: number;
vram_used_mb?: number;
vram_total_mb?: number;
}
export type JobStatus =
| 'pending'
| 'downloading'
| 'preparing'
| 'warming_model'
| 'transcribing'
| 'processing'
| 'done'
| 'failed'
| 'cancelled';
export interface Segment {
index: number;

View File

@@ -1,9 +1,10 @@
<script lang="ts">
import '../app.css';
import { onMount } from 'svelte';
import { onMount, onDestroy } from 'svelte';
import { browser } from '$app/environment';
import { page } from '$app/stores';
import { accent } from '$lib/accent.js';
import type { ModelStatus } from '$lib/types.js';
let { children } = $props();
@@ -11,8 +12,43 @@
// The store subscriber handles everything; just subscribing here keeps it alive.
$effect(() => { void $accent; });
// ── Model status ───────────────────────────────────────
let modelStatus = $state<ModelStatus>({ state: 'unloaded' });
let modelEs: EventSource | null = null;
function refreshModelStatus() {
fetch('/api/model/status')
.then((r) => r.json())
.then((s) => (modelStatus = s as ModelStatus))
.catch(() => {});
}
function subscribeModelEvents() {
modelEs?.close();
modelEs = new EventSource('/api/model/events');
modelEs.addEventListener('model_loading', () => refreshModelStatus());
modelEs.addEventListener('model_ready', () => refreshModelStatus());
modelEs.addEventListener('model_unloaded', () => refreshModelStatus());
modelEs.addEventListener('model_waiting_for_gpu',() => refreshModelStatus());
modelEs.onerror = () => { /* browser reconnects automatically */ };
}
const modelStateMeta: Record<string, { dot: string; label: string; pulse: boolean }> = {
unloaded: { dot: 'var(--text-dim)', label: 'model unloaded', pulse: false },
loading: { dot: '#f0b429', label: 'model loading…', pulse: true },
waiting_for_gpu: { dot: '#f97316', label: 'waiting for GPU', pulse: true },
ready: { dot: '#5dd47a', label: 'whisper-large-v3',pulse: false }
};
const modelMeta = $derived(
modelStateMeta[modelStatus.state] ?? modelStateMeta.unloaded
);
// Push notification setup
onMount(async () => {
refreshModelStatus();
subscribeModelEvents();
if (!browser || !('serviceWorker' in navigator) || !('PushManager' in window)) return;
try {
const reg = await navigator.serviceWorker.ready;
@@ -42,6 +78,8 @@
}
});
onDestroy(() => modelEs?.close());
function urlBase64ToUint8Array(base64: string): Uint8Array {
const pad = '='.repeat((4 - (base64.length % 4)) % 4);
const b64 = (base64 + pad).replace(/-/g, '+').replace(/_/g, '/');
@@ -135,8 +173,12 @@
<!-- Status dot -->
<div class="status-pill">
<div class="status-dot"></div>
<span>whisper-large-v3</span>
<div
class="status-dot"
class:pulse={modelMeta.pulse}
style="background: {modelMeta.dot}"
></div>
<span>{modelMeta.label}</span>
</div>
</nav>
@@ -268,8 +310,15 @@
width: 6px;
height: 6px;
border-radius: 3px;
background: #5dd47a;
flex-shrink: 0;
transition: background 0.4s;
}
.status-dot.pulse {
animation: dot-pulse 1.4s ease-in-out infinite;
}
@keyframes dot-pulse {
0%, 100% { opacity: 1; }
50% { opacity: 0.3; }
}
/* ── Main content ─────────────────────────────────────── */

View File

@@ -1,6 +1,7 @@
<script lang="ts">
import { onMount } from 'svelte';
import type { Job, AudioMode } from '$lib/types.js';
import { getDisplayJobProgress, getJobStatusLabel } from '$lib/job-progress.js';
import SourceIcon from '$lib/components/SourceIcon.svelte';
import Waveform from '$lib/components/Waveform.svelte';
import RecordButton from '$lib/components/RecordButton.svelte';
@@ -95,8 +96,7 @@
const parts: string[] = [];
if (job.source && !job.source.startsWith('http')) parts.push(job.source.split('/').pop() ?? '');
if (job.audioMode) parts.push(job.audioMode);
if (job.status === 'done') parts.push('done');
else parts.push(job.status);
parts.push(getJobStatusLabel(job.status).toLowerCase());
return parts.join(' · ');
}
@@ -144,7 +144,7 @@
<!-- Decorative waveform -->
<div class="dropzone-wave">
<Waveform bars={DROPZONE_BARS} progress={0} {ACCENT} height={38} />
<Waveform bars={DROPZONE_BARS} progress={0} accent={ACCENT} height={38} />
</div>
<input
@@ -271,7 +271,9 @@
<div class="recent-meta mono">{jobMeta(job)}</div>
</div>
{#if job.status !== 'done' && job.status !== 'failed' && job.status !== 'cancelled'}
<div class="recent-progress mono" style="color: {ACCENT}">{job.progress}%</div>
<div class="recent-progress mono" style="color: {ACCENT}">
{getDisplayJobProgress(job, { hasTranscript: Boolean(job.segmentsJson) })}%
</div>
{/if}
<svg width="14" height="14" viewBox="0 0 14 14" style="color: var(--text-dim); flex-shrink:0">
<path d="M5 3l4 4-4 4" stroke="currentColor" stroke-width="1.4" fill="none" stroke-linecap="round" stroke-linejoin="round"/>
@@ -586,4 +588,3 @@
}
}
</style>

View File

@@ -1,5 +1,6 @@
import { json, error } from '@sveltejs/kit';
import { getJob, setJobStatus, deleteJob } from '$lib/server/db.js';
import { cancelJob } from '$lib/server/whisper.js';
import { rm } from 'fs/promises';
export async function GET({ params }) {
@@ -8,7 +9,7 @@ export async function GET({ params }) {
return json(job);
}
const ACTIVE = new Set(['pending', 'downloading', 'preparing', 'transcribing', 'processing']);
const ACTIVE = new Set(['pending', 'downloading', 'preparing', 'warming_model', 'transcribing', 'processing']);
export async function DELETE({ params }) {
const job = getJob(params.id);
@@ -17,6 +18,8 @@ export async function DELETE({ params }) {
if (ACTIVE.has(job.status)) {
// Cancel active job (keeps DB record)
setJobStatus(params.id, 'cancelled', 0);
// Best-effort: tell whisper to drop the queued job so it stops using GPU
if (job.whisperJobId) cancelJob(job.whisperJobId).catch(() => {});
} else {
// Hard-delete terminal job + clean up output files
deleteJob(params.id);

View File

@@ -1,10 +1,9 @@
import { json, error } from '@sveltejs/kit';
import { getJob, updateJob } from '$lib/server/db.js';
import { deduplicateSegments } from '$lib/server/postprocess.js';
import { writeOutputs } from '$lib/server/formatter.js';
import type { Segment } from '$lib/types.js';
/** POST /api/jobs/[id]/reprocess — re-run post-processing and regenerate all output files. */
/** POST /api/jobs/[id]/reprocess — regenerate output files from stored canonical segments. */
export async function POST({ params }) {
const job = getJob(params.id);
if (!job) throw error(404, 'Job not found');
@@ -14,8 +13,7 @@ export async function POST({ params }) {
}
try {
const rawSegments = JSON.parse(job.segmentsJson) as Segment[];
const segments = deduplicateSegments(rawSegments);
const segments = JSON.parse(job.segmentsJson) as Segment[];
const paths = await writeOutputs(segments, job.title, job.id);
const outputDir = paths.srt.replace(/\/[^/]+$/, '');

View File

@@ -0,0 +1,43 @@
const WHISPER_URL = process.env.WHISPER_URL ?? 'http://localhost:8080';
/** Relay the whisper /model/events SSE stream to the browser. */
export async function GET({ request }) {
const { default: fetch } = await import('node-fetch');
const ac = new AbortController();
request.signal.addEventListener('abort', () => ac.abort());
const stream = new ReadableStream({
async start(controller) {
try {
const upstream = await fetch(`${WHISPER_URL}/model/events`, {
signal: ac.signal as AbortSignal
});
if (!upstream.body) {
controller.close();
return;
}
for await (const chunk of upstream.body) {
if (ac.signal.aborted) break;
controller.enqueue(chunk instanceof Buffer ? chunk : Buffer.from(String(chunk)));
}
} catch {
// upstream closed, client disconnected, or whisper unreachable — all fine
} finally {
controller.close();
}
},
cancel() {
ac.abort();
}
});
return new Response(stream, {
headers: {
'Content-Type': 'text/event-stream',
'Cache-Control': 'no-cache',
Connection: 'keep-alive',
'X-Accel-Buffering': 'no'
}
});
}

View File

@@ -0,0 +1,14 @@
import { getModelStatus } from '$lib/server/whisper.js';
export async function GET() {
try {
const status = await getModelStatus();
return new Response(JSON.stringify(status), {
headers: { 'Content-Type': 'application/json' }
});
} catch {
return new Response(JSON.stringify({ state: 'unloaded' }), {
headers: { 'Content-Type': 'application/json' }
});
}
}

View File

@@ -0,0 +1,13 @@
import { json } from '@sveltejs/kit';
import { unloadModel } from '$lib/server/whisper.js';
/** Proxy for POST /model/unload on the whisper backend. */
export async function POST() {
try {
const body = await unloadModel();
return json(body);
} catch (err: unknown) {
const message = err instanceof Error ? err.message : String(err);
return json({ ok: false, error: message }, { status: 502 });
}
}

View File

@@ -1,54 +1,90 @@
import { json, error } from '@sveltejs/kit';
import { getJob, updateJob, setJobStatus } from '$lib/server/db.js';
import { deduplicateSegments } from '$lib/server/postprocess.js';
import { writeOutputs } from '$lib/server/formatter.js';
import { sendNotification } from '$lib/server/push.js';
import { cleanupJobTmp } from '$lib/server/downloader.js';
import { emitProgress } from '$lib/server/pipeline.js';
import type { Segment, WhisperJob } from '$lib/types.js';
const WHISPER_JOB_STATUSES = new Set<WhisperJob['status']>([
'queued',
'running',
'done',
'failed',
'cancelled'
]);
function isWhisperJobWebhook(payload: unknown): payload is WhisperJob {
if (!payload || typeof payload !== 'object') return false;
const candidate = payload as Record<string, unknown>;
return (
typeof candidate.id === 'string' &&
typeof candidate.status === 'string' &&
WHISPER_JOB_STATUSES.has(candidate.status as WhisperJob['status'])
);
}
export async function POST({ params, request }) {
const jobId = params.jobId;
const job = getJob(jobId);
if (!job) throw error(404, 'Job not found');
const jobId = params.jobId;
const job = getJob(jobId);
if (!job) throw error(404, 'Job not found');
const whisperJob = (await request.json()) as WhisperJob;
const payload = (await request.json()) as unknown;
if (!isWhisperJobWebhook(payload)) {
// whisper-rtx2080 also fires model lifecycle events to registered job webhooks.
return json({ ok: true, ignored: 'not_a_job_event' });
}
const whisperJob = payload;
if (whisperJob.status === 'failed' || whisperJob.status === 'cancelled') {
const msg = whisperJob.error ?? `Whisper job ${whisperJob.status}`;
updateJob({ id: jobId, status: 'failed', error: msg });
emitProgress(jobId, { type: 'error', message: msg });
return json({ ok: true });
}
try {
setJobStatus(jobId, 'processing', 90);
emitProgress(jobId, { type: 'status', status: 'processing', progress: 90 });
const rawSegments = whisperJob.segments as Segment[];
const segments = deduplicateSegments(rawSegments);
const paths = await writeOutputs(segments, job.title, jobId);
const outputDir = paths.srt.replace(/\/[^/]+$/, '');
updateJob({
id: jobId,
status: 'done',
progress: 100,
segmentsJson: JSON.stringify(segments),
outputDir
});
emitProgress(jobId, { type: 'done', status: 'done' });
await sendNotification(jobId, '✅ Transcript ready', job.title);
await cleanupJobTmp(jobId);
return json({ ok: true });
} catch (err: unknown) {
const message = err instanceof Error ? err.message : String(err);
updateJob({ id: jobId, status: 'failed', error: message });
emitProgress(jobId, { type: 'error', message });
return json({ ok: false, error: message }, { status: 500 });
}
// Discard the result if the job was cancelled locally while whisper was running
if (job.status === 'cancelled') {
return json({ ok: true });
}
// Ignore stale callbacks from a previous whisper job after a local retry/reset.
if (job.whisperJobId && whisperJob.id !== job.whisperJobId) {
return json({ ok: true, ignored: 'stale_whisper_job' });
}
// Ignore replayed success callbacks after the transcript is already persisted.
if (job.status === 'done' && job.segmentsJson) {
return json({ ok: true, ignored: 'duplicate_webhook' });
}
if (whisperJob.status === 'failed' || whisperJob.status === 'cancelled') {
const msg = whisperJob.error ?? `Whisper job ${whisperJob.status}`;
updateJob({ id: jobId, status: 'failed', error: msg });
emitProgress(jobId, { type: 'error', message: msg });
return json({ ok: true });
}
try {
setJobStatus(jobId, 'processing', 90);
emitProgress(jobId, { type: 'status', status: 'processing', progress: 90 });
const segments = (whisperJob.segments ?? []) as Segment[];
const paths = await writeOutputs(segments, job.title, jobId);
const outputDir = paths.srt.replace(/\/[^/]+$/, '');
updateJob({
id: jobId,
status: 'done',
progress: 100,
segmentsJson: JSON.stringify(segments),
outputDir
});
emitProgress(jobId, { type: 'done', status: 'done' });
await sendNotification(jobId, '✅ Transcript ready', job.title);
await cleanupJobTmp(jobId);
return json({ ok: true });
} catch (err: unknown) {
const message = err instanceof Error ? err.message : String(err);
updateJob({ id: jobId, status: 'failed', error: message });
emitProgress(jobId, { type: 'error', message });
return json({ ok: false, error: message }, { status: 500 });
}
}

View File

@@ -1,6 +1,7 @@
<script lang="ts">
import { onMount } from 'svelte';
import type { Job } from '$lib/types.js';
import { getDisplayJobProgress, getJobStatusColor, getJobStatusLabel, isTerminalJobStatus } from '$lib/job-progress.js';
import SourceIcon from '$lib/components/SourceIcon.svelte';
import Waveform from '$lib/components/Waveform.svelte';
import { accent } from '$lib/accent.js';
@@ -10,27 +11,6 @@
let jobs = $state<Job[]>([]);
let loading = $state(true);
const statusColor: Record<string, string> = {
done: '#cdf24e',
failed: '#ff6b6b',
cancelled: 'rgba(232,233,236,0.3)',
transcribing: '#80c7f7',
preparing: '#fbc94b',
downloading: '#a78bfa',
pending: 'rgba(232,233,236,0.4)'
};
const statusLabel: Record<string, string> = {
pending: 'Pending',
downloading: 'Downloading',
preparing: 'Preparing',
transcribing: 'Transcribing',
processing: 'Processing',
done: 'Done',
failed: 'Failed',
cancelled: 'Cancelled'
};
function jobKind(job: Job): 'youtube' | 'audio' | 'video' | 'file' {
const s = job.source ?? '';
if (s.includes('youtube') || s.includes('youtu.be')) return 'youtube';
@@ -108,8 +88,8 @@
<div class="job-info">
<div class="job-name">{job.title || job.source}</div>
<div class="job-meta mono">
<span style="color: {statusColor[job.status] ?? 'rgba(232,233,236,0.5)'}">
{statusLabel[job.status] ?? job.status}
<span style="color: {getJobStatusColor(job.status)}">
{getJobStatusLabel(job.status)}
</span>
{#if job.createdAt}
<span>·</span>
@@ -122,14 +102,24 @@
</div>
</div>
{#if !['done', 'failed', 'cancelled'].includes(job.status)}
{#if !isTerminalJobStatus(job.status)}
<div class="job-wave">
<Waveform bars={40} progress={job.progress} accent={ACCENT} height={28} pattern="medium" />
<Waveform
bars={40}
progress={getDisplayJobProgress(job, { hasTranscript: Boolean(job.segmentsJson) })}
accent={ACCENT}
height={28}
pattern="medium"
/>
</div>
{:else if job.status === 'done'}
<div class="job-pct mono" style="color: {ACCENT}">{job.progress}%</div>
<div class="job-pct mono" style="color: {ACCENT}">
{getDisplayJobProgress(job, { hasTranscript: Boolean(job.segmentsJson) })}%
</div>
{:else}
<div class="job-pct mono" style="color: {statusColor[job.status]}">{job.status}</div>
<div class="job-pct mono" style="color: {getJobStatusColor(job.status)}">
{getJobStatusLabel(job.status)}
</div>
{/if}
<!-- Row actions -->
@@ -142,7 +132,7 @@
title="Retry"
></button>
{/if}
{#if ['done', 'failed', 'cancelled'].includes(job.status)}
{#if isTerminalJobStatus(job.status)}
<button
class="row-btn danger"
onclick={(e) => deleteJob(e, job)}

View File

@@ -2,6 +2,7 @@
import { onMount, onDestroy } from 'svelte';
import { page } from '$app/stores';
import type { Job, Segment } from '$lib/types.js';
import { getDisplayJobProgress, getJobStatusLabel, isTerminalJobStatus } from '$lib/job-progress.js';
import SourceIcon from '$lib/components/SourceIcon.svelte';
import Waveform from '$lib/components/Waveform.svelte';
import { accent } from '$lib/accent.js';
@@ -13,19 +14,9 @@
let segments = $state<Segment[]>([]);
let error = $state('');
let chunkInfo = $state({ chunk: 0, total: 0 });
let modelWarming = $state<{ state: string; retryAfterSecs: number } | null>(null);
let eventSource: EventSource | null = null;
const statusLabel: Record<string, string> = {
pending: 'Pending',
downloading: 'Downloading…',
preparing: 'Preparing audio…',
transcribing: 'Transcribing…',
processing: 'Post-processing…',
done: 'Done',
failed: 'Failed',
cancelled: 'Cancelled'
};
// Pipeline stages derived from job status
const pipelineStages = $derived.by(() => {
const status = job?.status ?? 'pending';
@@ -33,16 +24,26 @@
{ k: 'fetch', label: 'Fetch source' },
{ k: 'extract', label: 'Extract audio track' },
{ k: 'process', label: `Audio processing · ${job?.audioMode ?? 'auto'}` },
{ k: 'transcribe', label: 'Transcribing' },
{ k: 'transcribe', label: status === 'warming_model' ? 'Loading model' : 'Transcribing' },
{ k: 'finalize', label: 'Format &amp; save' }
];
const order = ['pending', 'downloading', 'preparing', 'transcribing', 'processing', 'done'];
const idx = order.indexOf(status);
const stageIndex = {
pending: 0,
downloading: 1,
preparing: 2,
warming_model: 3,
transcribing: 3,
processing: 4,
done: 5,
failed: -1,
cancelled: -1
}[status];
return stages.map((s, i) => ({
...s,
done: i < idx - 1 || status === 'done',
active: i === idx - 1 && status !== 'done' && status !== 'failed',
pending: i > idx - 1 && status !== 'done'
done: status === 'done' || i + 1 < stageIndex,
active: i + 1 === stageIndex && !isTerminalJobStatus(status),
pending: status !== 'done' && i + 1 > stageIndex
}));
});
@@ -56,7 +57,7 @@
onMount(async () => {
await loadJob();
if (job && !['done', 'failed', 'cancelled'].includes(job.status)) {
if (job && !isTerminalJobStatus(job.status)) {
openStream();
}
});
@@ -70,6 +71,7 @@
return;
}
job = await res.json();
segments = [];
if (job?.segmentsJson) {
try {
segments = JSON.parse(job.segmentsJson);
@@ -83,11 +85,19 @@
try {
const data = JSON.parse(e.data);
if (data.type === 'progress') {
modelWarming = null;
chunkInfo = { chunk: data.chunk ?? 0, total: data.total ?? 0 };
if (job) job = { ...job, progress: data.progress ?? job.progress, status: 'transcribing' };
} else if (data.type === 'model_warming') {
modelWarming = { state: data.state ?? 'loading', retryAfterSecs: data.retryAfterSecs ?? 30 };
chunkInfo = { chunk: 0, total: 0 };
if (job) job = { ...job, status: 'warming_model', progress: data.progress ?? job.progress };
} else if (data.type === 'status') {
if (data.status !== 'warming_model') modelWarming = null;
if (data.status !== 'transcribing') chunkInfo = { chunk: 0, total: 0 };
if (job) job = { ...job, status: data.status, progress: data.progress ?? job.progress };
} else if (data.type === 'done') {
modelWarming = null;
eventSource?.close();
loadJob();
} else if (data.type === 'error') {
@@ -108,8 +118,21 @@
}
const formats = ['srt', 'txt', 'md', 'json'] as const;
const isActive = $derived(!job || !['done', 'failed', 'cancelled'].includes(job.status));
const isTerminal = $derived(job !== null && ['done', 'failed', 'cancelled'].includes(job.status));
const hasTranscript = $derived((job?.segmentsJson ? true : false) || segments.length > 0);
const displayProgress = $derived(job ? getDisplayJobProgress(job, { hasTranscript }) : 0);
const progressStatusLabel = $derived.by(() => {
if (!job) return 'Pending';
if (job.status === 'warming_model') {
const state = modelWarming?.state?.replace(/_/g, ' ');
return state ? `Loading model (${state})…` : 'Loading model…';
}
if (job.status === 'preparing') return 'Preparing audio…';
if (job.status === 'processing') return 'Saving transcript…';
if (job.status === 'transcribing') return 'Transcribing…';
return job.status === 'done' ? 'Done' : `${getJobStatusLabel(job.status)}…`;
});
const isActive = $derived(!job || !isTerminalJobStatus(job.status));
const isTerminal = $derived(job !== null && isTerminalJobStatus(job.status));
const canRetry = $derived(
job !== null &&
['failed', 'cancelled'].includes(job.status) &&
@@ -194,7 +217,7 @@
<div class="progress-wave">
<Waveform
bars={140}
progress={job.progress}
progress={displayProgress}
accent={ACCENT}
height={80}
pattern="default"
@@ -204,9 +227,9 @@
<div class="progress-footer">
<div class="progress-left">
<span class="progress-pct mono">
{job.progress}<span style="color: var(--text-dim); font-weight: 400">%</span>
{displayProgress}<span style="color: var(--text-dim); font-weight: 400">%</span>
</span>
<span class="progress-status">{statusLabel[job.status] ?? job.status}</span>
<span class="progress-status">{progressStatusLabel}</span>
</div>
{#if chunkInfo.total > 0}
<span class="progress-chunks mono">
@@ -215,11 +238,20 @@
{/if}
</div>
{#if modelWarming}
<div class="warming-notice mono">
<svg width="12" height="12" viewBox="0 0 12 12" fill="none" style="flex-shrink:0; animation: spin 1.5s linear infinite">
<circle cx="6" cy="6" r="4.5" stroke="currentColor" stroke-width="1.4" fill="none" stroke-dasharray="14 8"/>
</svg>
Warming up model ({modelWarming.state.replace(/_/g, ' ')}) — retrying in {modelWarming.retryAfterSecs}s…
</div>
{/if}
<!-- Progress bar -->
<div class="progress-bar-track">
<div
class="progress-bar-fill"
style="width: {job.progress}%; background: {ACCENT}; box-shadow: 0 0 12px {ACCENT}80;"
style="width: {displayProgress}%; background: {ACCENT}; box-shadow: 0 0 12px {ACCENT}80;"
></div>
</div>
</div>
@@ -261,7 +293,7 @@
{@html stage.label}
</span>
{#if stage.active}
<span class="mono" style="font-size: 11.5px; color: {ACCENT}">{job.progress}%</span>
<span class="mono" style="font-size: 11.5px; color: {ACCENT}">{displayProgress}%</span>
{/if}
</div>
{/each}
@@ -484,6 +516,16 @@
color: var(--text-muted);
}
.warming-notice {
display: flex;
align-items: center;
gap: 8px;
margin-top: 10px;
font-size: 11.5px;
color: #f0b429;
opacity: 0.9;
}
.progress-bar-track {
height: 4px;
border-radius: 2px;
@@ -666,4 +708,3 @@
}
}
</style>

View File

@@ -7,17 +7,18 @@ import { describe, it, expect, vi, afterEach } from 'vitest';
const execFileMock = vi.hoisted(() => {
const fn = vi.fn();
type ExecFilePromisifyArgs = [string, string[]];
type ExecFileCallback = (err: Error | null, stdout: string, stderr: string) => void;
type ExecFileMock = (...args: [...ExecFilePromisifyArgs, ExecFileCallback]) => void;
const invoke = fn as unknown as ExecFileMock;
Object.defineProperty(fn, Symbol.for('nodejs.util.promisify.custom'), {
configurable: true,
value: (...args: unknown[]) =>
value: (...args: ExecFilePromisifyArgs) =>
new Promise<{ stdout: string; stderr: string }>((resolve, reject) => {
(fn as ReturnType<typeof vi.fn>)(
...args,
(err: Error | null, stdout: string, stderr: string) => {
invoke(...args, (err: Error | null, stdout: string, stderr: string) => {
if (err) reject(err);
else resolve({ stdout, stderr });
}
);
});
})
});
return fn;

View File

@@ -123,7 +123,7 @@ describe('setJobStatus', () => {
it('transitions through all valid statuses', () => {
const job = createJob('src', 'title', 'auto');
const statuses = ['downloading', 'preparing', 'transcribing', 'processing', 'done'] as const;
const statuses = ['downloading', 'preparing', 'warming_model', 'transcribing', 'processing', 'done'] as const;
for (const status of statuses) {
setJobStatus(job.id, status, 50);
expect(getJob(job.id)!.status).toBe(status);

View File

@@ -0,0 +1,80 @@
import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest';
import { rm } from 'fs/promises';
import type { TranscriptResponse } from 'youtube-transcript';
const { mockExecFile, mockFetchTranscript } = vi.hoisted(() => ({
mockExecFile: vi.fn(),
mockFetchTranscript: vi.fn()
}));
const TEST_DATA_DIR = `/tmp/tonemark-downloader-test-${Date.now()}`;
vi.stubEnv('DATA_DIR', TEST_DATA_DIR);
vi.mock('child_process', () => ({
execFile: mockExecFile
}));
vi.mock('youtube-transcript', () => ({
fetchTranscript: mockFetchTranscript
}));
import { downloadYouTube, transcriptEntriesToSegments } from '$lib/server/downloader.js';
beforeEach(() => {
vi.clearAllMocks();
mockExecFile.mockImplementation((...args: unknown[]) => {
const cb = args.at(-1) as (...callbackArgs: unknown[]) => void;
cb(null, JSON.stringify({ title: 'Fetched Title' }), '');
});
});
afterEach(async () => {
await rm(TEST_DATA_DIR, { recursive: true, force: true }).catch(() => {});
});
describe('transcriptEntriesToSegments', () => {
it('converts millisecond transcript offsets into second-based segments', () => {
const entries: TranscriptResponse[] = [
{ text: 'Hello everyone.', offset: 15240, duration: 4240, lang: 'en' },
{ text: 'Um, welcome to this talk.', offset: 16600, duration: 5080, lang: 'en' }
];
expect(transcriptEntriesToSegments(entries)).toEqual([
{ index: 0, start: 15.24, end: 19.48, text: 'Hello everyone.', words: [] },
{ index: 1, start: 16.6, end: 21.68, text: 'Um, welcome to this talk.', words: [] }
]);
});
it('preserves second-based transcript offsets and drops empty text', () => {
const entries: TranscriptResponse[] = [
{ text: ' ', offset: 0, duration: 1.5, lang: 'en' },
{ text: 'Clean caption cue', offset: 91.08, duration: 3.72, lang: 'en' }
];
expect(transcriptEntriesToSegments(entries)).toEqual([
{ index: 0, start: 91.08, end: 94.8, text: 'Clean caption cue', words: [] }
]);
});
});
describe('downloadYouTube', () => {
it('uses fetched transcript entries directly for caption jobs', async () => {
mockFetchTranscript.mockResolvedValue([
{ text: 'Hello everyone.', offset: 15240, duration: 4240, lang: 'en' },
{ text: 'Um, welcome to this talk.', offset: 16600, duration: 5080, lang: 'en' }
] satisfies TranscriptResponse[]);
const result = await downloadYouTube('https://youtube.com/watch?v=qdh_x-uRs9g', 'job-1');
expect(mockFetchTranscript).toHaveBeenCalledWith('https://youtube.com/watch?v=qdh_x-uRs9g', {
lang: 'en'
});
expect(result).toMatchObject({
type: 'captions',
segments: [
{ index: 0, start: 15.24, end: 19.48, text: 'Hello everyone.', words: [] },
{ index: 1, start: 16.6, end: 21.68, text: 'Um, welcome to this talk.', words: [] }
]
});
});
});

View File

@@ -0,0 +1,49 @@
import { describe, expect, it } from 'vitest';
import { getDisplayJobProgress, getJobStatusLabel, isTerminalJobStatus } from '$lib/job-progress.js';
import type { Job } from '$lib/types.js';
function makeJob(overrides: Partial<Job> = {}): Job {
return {
id: 'job-1',
status: 'transcribing',
title: 'Job',
source: 'https://example.com/audio.mp3',
audioMode: 'auto',
meanVolume: null,
whisperJobId: null,
progress: 42,
outputDir: null,
segmentsJson: null,
error: null,
createdAt: new Date().toISOString(),
updatedAt: new Date().toISOString(),
...overrides
};
}
describe('job progress helpers', () => {
it('keeps active jobs below 100 percent', () => {
expect(getDisplayJobProgress(makeJob({ status: 'transcribing', progress: 100 }))).toBe(99);
});
it('keeps model loading in an early progress band', () => {
expect(getDisplayJobProgress(makeJob({ status: 'warming_model', progress: 80 }))).toBe(15);
});
it('allows 100 percent once finished job has transcript payload', () => {
expect(
getDisplayJobProgress(makeJob({ status: 'done', progress: 100, segmentsJson: JSON.stringify([]) }), {
hasTranscript: true
})
).toBe(100);
});
it('holds done jobs below 100 percent until transcript data exists', () => {
expect(getDisplayJobProgress(makeJob({ status: 'done', progress: 100 }))).toBe(99);
});
it('exposes model-loading label as active state', () => {
expect(getJobStatusLabel('warming_model')).toBe('Loading model');
expect(isTerminalJobStatus('warming_model')).toBe(false);
});
});

View File

@@ -1,127 +0,0 @@
import { describe, it, expect } from 'vitest';
import {
deduplicateSegments
} from '$lib/server/postprocess.js';
import type { Segment } from '$lib/types.js';
// ── helpers ──────────────────────────────────────────────────────────────────
function seg(index: number, start: number, end: number, text: string): Segment {
return { index, start, end, text, words: [] };
}
// ── collapseRepeats (tested indirectly via deduplicateSegments) ───────────────
describe('deduplicateSegments — collapseRepeats', () => {
it('leaves text without repetition unchanged', () => {
const input = [seg(0, 0, 5, ' Hello world, this is a sentence.')];
const [out] = deduplicateSegments(input);
expect(out.text).toBe('Hello world, this is a sentence.');
});
it('collapses a consecutive repeated phrase inside a segment', () => {
const input = [seg(0, 0, 5, ' the quick brown fox the quick brown fox')];
const [out] = deduplicateSegments(input);
expect(out.text).not.toMatch(/the quick brown fox.*the quick brown fox/i);
});
it('handles multiple repetitions recursively', () => {
// "welcome everyone" = 16 chars — qualifies for the ≥10-char collapse regex
const input = [seg(0, 0, 5, ' welcome everyone welcome everyone welcome everyone')];
const result = deduplicateSegments(input);
const text = result[0]?.text ?? '';
expect((text.match(/welcome everyone/gi) ?? []).length).toBeLessThan(3);
});
});
// ── mergeConsecutive ──────────────────────────────────────────────────────────
describe('deduplicateSegments — mergeConsecutive', () => {
it('merges adjacent segments with identical text', () => {
const input = [
seg(0, 0, 2, ' Hello world.'),
seg(1, 2, 4, ' Hello world.')
];
const result = deduplicateSegments(input);
expect(result).toHaveLength(1);
expect(result[0].end).toBe(4);
});
it('keeps adjacent segments with different text', () => {
const input = [
seg(0, 0, 2, ' First sentence.'),
seg(1, 2, 4, ' Second sentence.')
];
const result = deduplicateSegments(input);
expect(result).toHaveLength(2);
});
it('normalises punctuation and case for merge comparison', () => {
const input = [
seg(0, 0, 2, ' Hello, World!'),
seg(1, 2, 4, ' hello world')
];
const result = deduplicateSegments(input);
expect(result).toHaveLength(1);
});
});
// ── ngramDedup ────────────────────────────────────────────────────────────────
describe('deduplicateSegments — ngramDedup', () => {
it('passes through completely unique segments', () => {
const input = [
seg(0, 0, 5, ' The cat sat on the mat quite happily today.'),
seg(1, 5, 10, ' Later the dog ran across the yard chasing a ball.')
];
expect(deduplicateSegments(input)).toHaveLength(2);
});
it('removes a segment that is highly similar to recent context', () => {
// Repeat a long sentence verbatim — should be caught as duplicate
const longText =
' This is a very specific and unique sentence about transcription quality matters greatly.';
const input = [seg(0, 0, 5, longText), seg(1, 5, 10, longText)];
// After mergeConsecutive the second one is already merged, so result is 1
expect(deduplicateSegments(input)).toHaveLength(1);
});
});
// ── deduplicateSegments — full pipeline ──────────────────────────────────────
describe('deduplicateSegments — full pipeline', () => {
it('returns empty array for empty input', () => {
expect(deduplicateSegments([])).toEqual([]);
});
it('removes segments whose text is empty after trimming', () => {
const input = [seg(0, 0, 1, ' '), seg(1, 1, 2, ' Hello.')];
const result = deduplicateSegments(input);
expect(result).toHaveLength(1);
expect(result[0].text).toBe('Hello.');
});
it('re-indexes output segments starting from 0', () => {
const input = [
seg(5, 0, 2, ' First unique sentence here.'),
seg(8, 2, 4, ' Second different sentence there.')
];
const result = deduplicateSegments(input);
result.forEach((s, i) => expect(s.index).toBe(i));
});
it('runs the full pipeline: trim → remove empty → merge → ngram → merge → reindex', () => {
const input = [
seg(0, 0, 2, ' Good morning everyone.'),
seg(1, 2, 3, ' '), // empty — removed
seg(2, 3, 5, ' Good morning everyone.'), // duplicate — merged
seg(3, 5, 7, ' Welcome to our presentation today.')
];
const result = deduplicateSegments(input);
expect(result).toHaveLength(2);
expect(result[0].text).toBe('Good morning everyone.');
expect(result[1].text).toBe('Welcome to our presentation today.');
expect(result[0].index).toBe(0);
expect(result[1].index).toBe(1);
});
});

View File

@@ -1,4 +1,4 @@
import { describe, it, expect, vi, afterEach } from 'vitest';
import { describe, it, expect, vi, beforeEach, afterEach } from 'vitest';
// ── Hoist mock functions so they're available inside vi.mock() factories ───────
const { mockSetVapidDetails, mockWebPushSend } = vi.hoisted(() => ({
@@ -24,12 +24,16 @@ import { sendNotification, getVapidPublicKey } from '$lib/server/push.js';
import { savePushSubscription, deletePushSubscription, getAllSubscriptions } from '$lib/server/db.js';
import { rm } from 'fs/promises';
afterEach(async () => {
mockSetVapidDetails.mockReset();
beforeEach(() => {
// Ensure a clean subscription table before each test
for (const s of getAllSubscriptions()) deletePushSubscription(s.endpoint);
mockWebPushSend.mockReset();
mockSetVapidDetails.mockReset();
});
afterEach(async () => {
// Remove all test subscriptions between tests
const subs = getAllSubscriptions();
for (const s of subs) deletePushSubscription(s.endpoint);
for (const s of getAllSubscriptions()) deletePushSubscription(s.endpoint);
await rm(TEST_DATA_DIR, { recursive: true, force: true }).catch(() => {});
});
@@ -134,6 +138,15 @@ describe('sendNotification', () => {
.mockResolvedValueOnce({});
await sendNotification('job-8', 'title', 'body');
expect(mockWebPushSend).toHaveBeenCalledTimes(3);
const calledEndpoints = mockWebPushSend.mock.calls.map(
([sub]) => (sub as { endpoint: string }).endpoint
);
expect(calledEndpoints).toEqual(
expect.arrayContaining([
'https://fcm.example.com/push/ok1',
'https://fcm.example.com/push/fail',
'https://fcm.example.com/push/ok2'
])
);
});
});

View File

@@ -7,7 +7,6 @@ const {
mockGetJob,
mockUpdateJob,
mockSetJobStatus,
mockDeduplicateSegments,
mockWriteOutputs,
mockSendNotification,
mockCleanupJobTmp,
@@ -16,7 +15,6 @@ const {
mockGetJob: vi.fn(),
mockUpdateJob: vi.fn(),
mockSetJobStatus: vi.fn(),
mockDeduplicateSegments: vi.fn((segs: Segment[]) => segs),
mockWriteOutputs: vi.fn(),
mockSendNotification: vi.fn(),
mockCleanupJobTmp: vi.fn(),
@@ -29,10 +27,6 @@ vi.mock('$lib/server/db.js', () => ({
setJobStatus: mockSetJobStatus
}));
vi.mock('$lib/server/postprocess.js', () => ({
deduplicateSegments: mockDeduplicateSegments
}));
vi.mock('$lib/server/formatter.js', () => ({
writeOutputs: mockWriteOutputs
}));
@@ -91,7 +85,6 @@ function makeSeg(index: number, text: string): Segment {
beforeEach(() => {
vi.clearAllMocks();
mockDeduplicateSegments.mockImplementation((segs: Segment[]) => segs);
mockWriteOutputs.mockResolvedValue({
srt: '/out/dir/title.srt',
txt: '/out/dir/title.txt',
@@ -113,6 +106,120 @@ describe('POST /api/webhook/[jobId] — job not found', () => {
});
});
// ── Ignore backend model lifecycle webhooks ─────────────────────────────────────
describe('POST /api/webhook/[jobId] — non-job webhook payloads', () => {
it('ignores model_ready events sent to job webhooks', async () => {
mockGetJob.mockReturnValue(makeJob('job-model-ready'));
const res = await POST(
makeEvent('job-model-ready', {
type: 'model_ready',
loaded_at: new Date().toISOString()
}) as any
);
expect(res.status).toBe(200);
expect(await res.json()).toEqual({ ok: true, ignored: 'not_a_job_event' });
expect(mockSetJobStatus).not.toHaveBeenCalled();
expect(mockUpdateJob).not.toHaveBeenCalled();
expect(mockWriteOutputs).not.toHaveBeenCalled();
expect(mockSendNotification).not.toHaveBeenCalled();
});
it('ignores model_unloaded events sent to job webhooks', async () => {
mockGetJob.mockReturnValue(makeJob('job-model-unloaded'));
const res = await POST(
makeEvent('job-model-unloaded', {
type: 'model_unloaded',
unloaded_at: new Date().toISOString()
}) as any
);
expect(res.status).toBe(200);
expect(await res.json()).toEqual({ ok: true, ignored: 'not_a_job_event' });
expect(mockSetJobStatus).not.toHaveBeenCalled();
expect(mockUpdateJob).not.toHaveBeenCalled();
expect(mockWriteOutputs).not.toHaveBeenCalled();
expect(mockSendNotification).not.toHaveBeenCalled();
});
it('ignores payloads with invalid status values', async () => {
mockGetJob.mockReturnValue(makeJob('job-invalid-status'));
const res = await POST(
makeEvent('job-invalid-status', {
id: 'bogus-whisper-id',
status: 'model_ready',
segments: []
}) as any
);
expect(res.status).toBe(200);
expect(await res.json()).toEqual({ ok: true, ignored: 'not_a_job_event' });
expect(mockSetJobStatus).not.toHaveBeenCalled();
expect(mockUpdateJob).not.toHaveBeenCalled();
expect(mockWriteOutputs).not.toHaveBeenCalled();
});
});
// ── Local cancellation guard ──────────────────────────────────────────────────
describe('POST /api/webhook/[jobId] — locally cancelled job', () => {
it('returns ok without processing when the local job is already cancelled', async () => {
mockGetJob.mockReturnValue({ ...makeJob('job-lc'), status: 'cancelled' });
const payload = makeWhisperJob({ status: 'done' });
const res = await POST(makeEvent('job-lc', payload) as any);
expect(res.status).toBe(200);
expect(await res.json()).toEqual({ ok: true });
// Must not touch outputs, status, or notifications
expect(mockSetJobStatus).not.toHaveBeenCalled();
expect(mockUpdateJob).not.toHaveBeenCalled();
expect(mockWriteOutputs).not.toHaveBeenCalled();
expect(mockSendNotification).not.toHaveBeenCalled();
});
});
// ── Duplicate / stale callback guards ──────────────────────────────────────────
describe('POST /api/webhook/[jobId] — duplicate and stale callbacks', () => {
it('ignores replayed success callbacks after the transcript is already done', async () => {
mockGetJob.mockReturnValue({
...makeJob('job-done'),
status: 'done',
segmentsJson: JSON.stringify([makeSeg(0, 'Already saved.')]),
whisperJobId: 'whisper-id'
});
const res = await POST(makeEvent('job-done', makeWhisperJob()) as any);
expect(res.status).toBe(200);
expect(await res.json()).toEqual({ ok: true, ignored: 'duplicate_webhook' });
expect(mockSetJobStatus).not.toHaveBeenCalled();
expect(mockUpdateJob).not.toHaveBeenCalled();
expect(mockWriteOutputs).not.toHaveBeenCalled();
});
it('ignores stale callbacks from an older whisper job after retry', async () => {
mockGetJob.mockReturnValue({
...makeJob('job-stale'),
status: 'transcribing',
whisperJobId: 'current-whisper-job'
});
const res = await POST(
makeEvent('job-stale', makeWhisperJob({ id: 'old-whisper-job', segments: [makeSeg(0, 'stale')] })) as any
);
expect(res.status).toBe(200);
expect(await res.json()).toEqual({ ok: true, ignored: 'stale_whisper_job' });
expect(mockSetJobStatus).not.toHaveBeenCalled();
expect(mockUpdateJob).not.toHaveBeenCalled();
expect(mockWriteOutputs).not.toHaveBeenCalled();
});
});
// ── Whisper job failed / cancelled ───────────────────────────────────────────
describe('POST /api/webhook/[jobId] — whisper failure', () => {
@@ -162,25 +269,21 @@ describe('POST /api/webhook/[jobId] — whisper failure', () => {
describe('POST /api/webhook/[jobId] — success with segments', () => {
const segments = [makeSeg(0, 'Hello world.'), makeSeg(1, 'This is a test.')];
it('runs deduplication on received segments', async () => {
it('passes received segments through unchanged', async () => {
mockGetJob.mockReturnValue(makeJob('job-3'));
await POST(makeEvent('job-3', makeWhisperJob({ segments })) as any);
expect(mockDeduplicateSegments).toHaveBeenCalledWith(segments);
expect(mockWriteOutputs).toHaveBeenCalledWith(segments, 'Test Video', 'job-3');
});
it('calls writeOutputs with the deduplicated segments and job title', async () => {
it('calls writeOutputs with the received segments and job title', async () => {
mockGetJob.mockReturnValue(makeJob('job-4', 'My Lecture'));
const deduped = [makeSeg(0, 'Hello world.')];
mockDeduplicateSegments.mockReturnValue(deduped);
await POST(makeEvent('job-4', makeWhisperJob({ segments })) as any);
expect(mockWriteOutputs).toHaveBeenCalledWith(deduped, 'My Lecture', 'job-4');
expect(mockWriteOutputs).toHaveBeenCalledWith(segments, 'My Lecture', 'job-4');
});
it('stores serialised segments_json in the database', async () => {
mockGetJob.mockReturnValue(makeJob('job-5'));
const deduped = [makeSeg(0, 'Result text.')];
mockDeduplicateSegments.mockReturnValue(deduped);
await POST(makeEvent('job-5', makeWhisperJob({ segments })) as any);
@@ -188,7 +291,7 @@ describe('POST /api/webhook/[jobId] — success with segments', () => {
expect.objectContaining({
id: 'job-5',
status: 'done',
segmentsJson: JSON.stringify(deduped)
segmentsJson: JSON.stringify(segments)
})
);
});
@@ -270,6 +373,34 @@ describe('POST /api/webhook/[jobId] — empty segments', () => {
});
});
// ── Undefined / missing segments (model returned no segments field) ───────────
describe('POST /api/webhook/[jobId] — undefined segments', () => {
it('completes the job as done when segments field is absent from whisper payload', async () => {
mockGetJob.mockReturnValue(makeJob('job-noseg'));
// Simulate whisper returning a result without a segments field
const payload = { ...makeWhisperJob(), segments: undefined as unknown as never[] };
const res = await POST(makeEvent('job-noseg', payload) as any);
expect(res.status).toBe(200);
expect(await res.json()).toEqual({ ok: true });
expect(mockUpdateJob).toHaveBeenCalledWith(
expect.objectContaining({ status: 'done', id: 'job-noseg' })
);
});
it('does not throw "cannot read properties of undefined" when segments is null', async () => {
mockGetJob.mockReturnValue(makeJob('job-nullseg'));
const payload = { ...makeWhisperJob(), segments: null as unknown as never[] };
// Must NOT throw — previously crashed with "Cannot read properties of undefined (reading 'map')"
await expect(POST(makeEvent('job-nullseg', payload) as any)).resolves.toBeDefined();
expect(mockUpdateJob).toHaveBeenCalledWith(
expect.objectContaining({ status: 'done' })
);
});
});
// ── Internal error handling ───────────────────────────────────────────────────
describe('POST /api/webhook/[jobId] — internal errors', () => {

View File

@@ -1,4 +1,4 @@
import { describe, it, expect, vi, afterEach } from 'vitest';
import { describe, it, expect, vi, afterEach, beforeEach } from 'vitest';
import { Readable } from 'stream';
// ── Hoist mocks so they're available inside vi.mock() factories ───────────────
@@ -6,7 +6,8 @@ import { Readable } from 'stream';
const mocks = vi.hoisted(() => ({
fetch: vi.fn(),
append: vi.fn(),
getHeaders: vi.fn(() => ({ 'content-type': 'multipart/form-data; boundary=test' }))
getHeaders: vi.fn(() => ({ 'content-type': 'multipart/form-data; boundary=test' })),
createReadStream: vi.fn(() => 'STREAM_PLACEHOLDER')
}));
vi.mock('node-fetch', () => ({ default: mocks.fetch }));
@@ -19,9 +20,9 @@ vi.mock('form-data', () => ({
})
}));
vi.mock('fs', () => ({ createReadStream: vi.fn(() => 'STREAM_PLACEHOLDER') }));
vi.mock('fs', () => ({ createReadStream: mocks.createReadStream }));
import { submitJob, streamJob } from '$lib/server/whisper.js';
import { submitJob, streamJob, getModelStatus, cancelJob, unloadModel } from '$lib/server/whisper.js';
afterEach(() => vi.clearAllMocks());
@@ -31,6 +32,7 @@ describe('submitJob', () => {
it('POSTs to /jobs and returns job_id', async () => {
mocks.fetch.mockResolvedValue({
ok: true,
status: 202,
json: () => Promise.resolve({ job_id: 'whisper-job-abc' })
});
const id = await submitJob('/tmp/audio.wav', 'http://host/api/webhook/job-1');
@@ -41,6 +43,7 @@ describe('submitJob', () => {
vi.stubEnv('WHISPER_URL', 'http://localhost:8091');
mocks.fetch.mockResolvedValue({
ok: true,
status: 202,
json: () => Promise.resolve({ job_id: 'x' })
});
await submitJob('/tmp/audio.wav', 'http://host/api/webhook/job-1');
@@ -54,6 +57,7 @@ describe('submitJob', () => {
it('includes task=transcribe in the form', async () => {
mocks.fetch.mockResolvedValue({
ok: true,
status: 202,
json: () => Promise.resolve({ job_id: 'x' })
});
await submitJob('/tmp/audio.wav', 'http://host/webhook');
@@ -63,6 +67,7 @@ describe('submitJob', () => {
it('includes webhook_url in the form', async () => {
mocks.fetch.mockResolvedValue({
ok: true,
status: 202,
json: () => Promise.resolve({ job_id: 'x' })
});
await submitJob('/tmp/audio.wav', 'http://192.168.1.10:3000/api/webhook/job-99');
@@ -75,6 +80,7 @@ describe('submitJob', () => {
it('includes language when provided', async () => {
mocks.fetch.mockResolvedValue({
ok: true,
status: 202,
json: () => Promise.resolve({ job_id: 'x' })
});
await submitJob('/tmp/audio.wav', 'http://host/webhook', 'en');
@@ -84,6 +90,7 @@ describe('submitJob', () => {
it('omits language field when not provided', async () => {
mocks.fetch.mockResolvedValue({
ok: true,
status: 202,
json: () => Promise.resolve({ job_id: 'x' })
});
await submitJob('/tmp/audio.wav', 'http://host/webhook');
@@ -101,6 +108,426 @@ describe('submitJob', () => {
});
});
// ── submitJob — 503 retry & model-warming behavior ───────────────────────────
/** Minimal 503 response the whisper server returns when model not ready. */
function make503(state: string, retry_after_secs: number, headerRetryAfter?: string) {
return {
status: 503,
json: () => Promise.resolve({ error: 'model_not_ready', state, retry_after_secs }),
headers: {
get: (h: string) =>
h.toLowerCase() === 'retry-after' ? (headerRetryAfter ?? String(retry_after_secs)) : null
}
};
}
function make202(job_id: string) {
return { status: 202, json: () => Promise.resolve({ job_id }) };
}
/**
* URL-aware fetch mock: /model/events calls resolve immediately with no body
* (causing waitForModelReady to call finish() right away, so the retry loop
* proceeds without any real timer delay), and /jobs calls consume the
* provided response queue in order.
*/
function makeJobFetch(...responses: object[]) {
let idx = 0;
return (url: string) => {
if (String(url).includes('/model/events')) {
return Promise.resolve({ ok: true, status: 200, body: null });
}
return Promise.resolve(responses[idx++]);
};
}
describe('submitJob — 503 retry behavior', () => {
beforeEach(() => vi.useFakeTimers());
afterEach(() => vi.useRealTimers());
it('calls onModelWaiting with state and retryAfterSecs on first 503', async () => {
mocks.fetch.mockImplementation(makeJobFetch(make503('unloaded', 30), make202('job-1')));
const onModelWaiting = vi.fn();
const id = await submitJob('/tmp/audio.wav', 'http://host/webhook', undefined, onModelWaiting);
expect(id).toBe('job-1');
expect(onModelWaiting).toHaveBeenCalledOnce();
expect(onModelWaiting).toHaveBeenCalledWith('unloaded', 30);
});
it('retries and returns job_id once model becomes ready', async () => {
mocks.fetch.mockImplementation(makeJobFetch(make503('loading', 10), make202('ready-id')));
const id = await submitJob('/tmp/audio.wav', 'http://host/webhook');
expect(id).toBe('ready-id');
const jobCalls = mocks.fetch.mock.calls.filter(([url]) => String(url).endsWith('/jobs'));
expect(jobCalls).toHaveLength(2);
});
it('calls onModelWaiting once per 503, not on success', async () => {
mocks.fetch.mockImplementation(
makeJobFetch(make503('loading', 0), make503('loading', 0), make202('final-id'))
);
const onModelWaiting = vi.fn();
const id = await submitJob('/tmp/audio.wav', 'http://host/webhook', undefined, onModelWaiting, 10);
expect(id).toBe('final-id');
expect(onModelWaiting).toHaveBeenCalledTimes(2);
});
it('passes the correct state for each 503 response', async () => {
mocks.fetch.mockImplementation(
makeJobFetch(
make503('unloaded', 0),
make503('loading', 0),
make503('waiting_for_gpu', 0),
make202('job-x')
)
);
const onModelWaiting = vi.fn();
const id = await submitJob('/tmp/audio.wav', 'http://host/webhook', undefined, onModelWaiting, 10);
expect(id).toBe('job-x');
expect(onModelWaiting).toHaveBeenNthCalledWith(1, 'unloaded', 0);
expect(onModelWaiting).toHaveBeenNthCalledWith(2, 'loading', 0);
expect(onModelWaiting).toHaveBeenNthCalledWith(3, 'waiting_for_gpu', 0);
});
it('falls back to Retry-After header when body lacks retry_after_secs', async () => {
mocks.fetch.mockImplementation(
makeJobFetch(
{
status: 503,
json: () => Promise.resolve({ state: 'loading' }),
headers: { get: (h: string) => (h.toLowerCase() === 'retry-after' ? '7' : null) }
},
make202('fallback-id')
)
);
const onModelWaiting = vi.fn();
const id = await submitJob('/tmp/audio.wav', 'http://host/webhook', undefined, onModelWaiting);
expect(id).toBe('fallback-id');
expect(onModelWaiting).toHaveBeenCalledWith('loading', 7);
});
it('falls back to 15s when both body and header are absent', async () => {
mocks.fetch.mockImplementation(
makeJobFetch(
{
status: 503,
json: () => Promise.resolve({ state: 'unloaded' }),
headers: { get: () => null }
},
make202('default-wait-id')
)
);
const onModelWaiting = vi.fn();
const id = await submitJob('/tmp/audio.wav', 'http://host/webhook', undefined, onModelWaiting);
expect(id).toBe('default-wait-id');
expect(onModelWaiting).toHaveBeenCalledWith('unloaded', 15);
});
it('throws after maxAttempts 503 responses', async () => {
mocks.fetch.mockImplementation(
makeJobFetch(make503('loading', 0), make503('loading', 0), make503('loading', 0))
);
await expect(
submitJob('/tmp/audio.wav', 'http://host/webhook', undefined, undefined, 3)
).rejects.toThrow(/did not become ready after 3 attempts/i);
const jobCalls = mocks.fetch.mock.calls.filter(([url]) => String(url).endsWith('/jobs'));
expect(jobCalls).toHaveLength(3);
});
it('does NOT call onModelWaiting for non-503 errors', async () => {
mocks.fetch.mockResolvedValue({
status: 500,
text: () => Promise.resolve('internal error')
});
const onModelWaiting = vi.fn();
await expect(
submitJob('/tmp/audio.wav', 'http://host/webhook', undefined, onModelWaiting)
).rejects.toThrow('500');
expect(onModelWaiting).not.toHaveBeenCalled();
});
it('creates a fresh ReadStream for each attempt (stream not reused across retries)', async () => {
mocks.fetch.mockImplementation(
makeJobFetch(make503('loading', 0), make503('loading', 0), make202('fresh-stream-id'))
);
const id = await submitJob('/tmp/audio.wav', 'http://host/webhook', undefined, undefined, 10);
expect(id).toBe('fresh-stream-id');
// 3 attempts → 3 separate createReadStream calls, one fresh stream per form
expect(mocks.createReadStream).toHaveBeenCalledTimes(3);
expect(mocks.createReadStream).toHaveBeenNthCalledWith(1, '/tmp/audio.wav');
expect(mocks.createReadStream).toHaveBeenNthCalledWith(2, '/tmp/audio.wav');
expect(mocks.createReadStream).toHaveBeenNthCalledWith(3, '/tmp/audio.wav');
});
it('does NOT retry on non-503 errors (throws immediately)', async () => {
mocks.fetch.mockResolvedValue({
status: 400,
text: () => Promise.resolve("missing 'audio' field")
});
await expect(
submitJob('/tmp/audio.wav', 'http://host/webhook', undefined, undefined, 10)
).rejects.toThrow('400');
expect(mocks.fetch).toHaveBeenCalledTimes(1);
});
it('works correctly without an onModelWaiting callback', async () => {
mocks.fetch.mockImplementation(makeJobFetch(make503('unloaded', 0), make202('no-cb-id')));
const id = await submitJob('/tmp/audio.wav', 'http://host/webhook');
expect(id).toBe('no-cb-id');
});
});
// ── submitJob — SSE-triggered retry ──────────────────────────────────────────
describe('submitJob — SSE-triggered retry', () => {
beforeEach(() => vi.useFakeTimers());
afterEach(() => vi.useRealTimers());
it('retries immediately when /model/events fires state:ready before Retry-After timeout', async () => {
let jobCallIdx = 0;
mocks.fetch.mockImplementation((url: string) => {
if (String(url).includes('/model/events')) {
// SSE stream that immediately emits model_ready
return Promise.resolve({
ok: true,
status: 200,
body: Readable.from(['data: {"state":"ready"}\n\n'])
});
}
jobCallIdx++;
if (jobCallIdx === 1) return Promise.resolve(make503('loading', 30));
return Promise.resolve(make202('sse-triggered-id'));
});
// SSE fires model_ready before the 31s timeout — no timer advancement needed
const id = await submitJob('/tmp/audio.wav', 'http://host/webhook');
expect(id).toBe('sse-triggered-id');
const sseCalls = mocks.fetch.mock.calls.filter(([url]) =>
String(url).includes('/model/events')
);
expect(sseCalls).toHaveLength(1);
});
it('falls back to Retry-After sleep when /model/events is unreachable', async () => {
let jobCallIdx = 0;
mocks.fetch.mockImplementation((url: string) => {
if (String(url).includes('/model/events')) {
return Promise.reject(new Error('Connection refused'));
}
jobCallIdx++;
if (jobCallIdx === 1) return Promise.resolve(make503('loading', 5));
return Promise.resolve(make202('fallback-sleep-id'));
});
// SSE failed → must wait for the Retry-After timer
const p = submitJob('/tmp/audio.wav', 'http://host/webhook');
await vi.runAllTimersAsync();
await expect(p).resolves.toBe('fallback-sleep-id');
});
it('proceeds immediately when SSE stream closes without model_ready', async () => {
let jobCallIdx = 0;
mocks.fetch.mockImplementation((url: string) => {
if (String(url).includes('/model/events')) {
// Empty stream — closes without emitting anything
return Promise.resolve({ ok: true, status: 200, body: Readable.from([]) });
}
jobCallIdx++;
if (jobCallIdx === 1) return Promise.resolve(make503('loading', 30));
return Promise.resolve(make202('stream-closed-id'));
});
// Stream closed without model_ready → should not wait the full 31s timeout
const id = await submitJob('/tmp/audio.wav', 'http://host/webhook');
expect(id).toBe('stream-closed-id');
});
it('relays intermediate model states from /model/events while waiting to retry', async () => {
let jobCallIdx = 0;
mocks.fetch.mockImplementation((url: string) => {
if (String(url).includes('/model/events')) {
return Promise.resolve({
ok: true,
status: 200,
body: Readable.from([
'data: {"state":"loading"}\n\ndata: {"state":"waiting_for_gpu"}\n\ndata: {"state":"ready"}\n\n'
])
});
}
jobCallIdx++;
if (jobCallIdx === 1) return Promise.resolve(make503('unloaded', 30));
return Promise.resolve(make202('state-relay-id'));
});
const onModelWaiting = vi.fn();
const id = await submitJob('/tmp/audio.wav', 'http://host/webhook', undefined, onModelWaiting);
expect(id).toBe('state-relay-id');
expect(onModelWaiting).toHaveBeenNthCalledWith(1, 'unloaded', 30);
expect(onModelWaiting).toHaveBeenNthCalledWith(2, 'loading', 30);
expect(onModelWaiting).toHaveBeenNthCalledWith(3, 'waiting_for_gpu', 30);
expect(onModelWaiting).toHaveBeenCalledTimes(3);
});
});
// ── unloadModel ───────────────────────────────────────────────────────────────
describe('unloadModel', () => {
it('POSTs to /model/unload and returns parsed body', async () => {
mocks.fetch.mockResolvedValue({
ok: true,
json: () => Promise.resolve({ ok: true })
});
const result = await unloadModel();
expect(result).toEqual({ ok: true });
expect(mocks.fetch).toHaveBeenCalledWith(
expect.stringContaining('/model/unload'),
expect.objectContaining({ method: 'POST' })
);
});
it('uses the configured WHISPER_URL', async () => {
vi.stubEnv('WHISPER_URL', 'http://gpu-box:9090');
mocks.fetch.mockResolvedValue({
ok: true,
json: () => Promise.resolve({ ok: true })
});
await unloadModel();
expect(mocks.fetch).toHaveBeenCalledWith(
'http://gpu-box:9090/model/unload',
expect.anything()
);
vi.unstubAllEnvs();
});
it('throws when whisper returns a non-ok response', async () => {
mocks.fetch.mockResolvedValue({ ok: false, status: 409 });
await expect(unloadModel()).rejects.toThrow('/model/unload');
});
});
// ── cancelJob ─────────────────────────────────────────────────────────────────
describe('cancelJob', () => {
it('sends DELETE to the correct whisper job URL', async () => {
mocks.fetch.mockResolvedValue({ ok: true, status: 200 });
await cancelJob('whisper-job-abc');
expect(mocks.fetch).toHaveBeenCalledWith(
expect.stringContaining('/jobs/whisper-job-abc'),
expect.objectContaining({ method: 'DELETE' })
);
});
it('uses the configured WHISPER_URL', async () => {
vi.stubEnv('WHISPER_URL', 'http://gpu-box:9090');
mocks.fetch.mockResolvedValue({ ok: true, status: 200 });
await cancelJob('job-xyz');
expect(mocks.fetch).toHaveBeenCalledWith(
'http://gpu-box:9090/jobs/job-xyz',
expect.objectContaining({ method: 'DELETE' })
);
vi.unstubAllEnvs();
});
it('swallows errors silently (best-effort)', async () => {
mocks.fetch.mockRejectedValue(new Error('Connection refused'));
await expect(cancelJob('dead-job')).resolves.not.toThrow();
});
});
// ── getModelStatus ────────────────────────────────────────────────────────────
describe('getModelStatus', () => {
it('returns parsed status when model is ready', async () => {
const readyStatus = {
state: 'ready',
loaded_at: '2026-05-09T00:00:00.000Z',
vram_used_mb: 4096,
vram_total_mb: 8192
};
mocks.fetch.mockResolvedValue({
ok: true,
json: () => Promise.resolve(readyStatus)
});
const status = await getModelStatus();
expect(status.state).toBe('ready');
expect(status.loaded_at).toBe('2026-05-09T00:00:00.000Z');
expect(status.vram_used_mb).toBe(4096);
});
it('returns parsed status when model is unloaded', async () => {
mocks.fetch.mockResolvedValue({
ok: true,
json: () => Promise.resolve({ state: 'unloaded' })
});
const status = await getModelStatus();
expect(status.state).toBe('unloaded');
});
it('returns parsed status when model is loading', async () => {
mocks.fetch.mockResolvedValue({
ok: true,
json: () => Promise.resolve({ state: 'loading' })
});
const status = await getModelStatus();
expect(status.state).toBe('loading');
});
it('returns parsed status when waiting_for_gpu with VRAM fields', async () => {
const waitingStatus = {
state: 'waiting_for_gpu',
vram_needed_mb: 3951,
vram_free_mb: 512,
retry_in_secs: 30
};
mocks.fetch.mockResolvedValue({
ok: true,
json: () => Promise.resolve(waitingStatus)
});
const status = await getModelStatus();
expect(status.state).toBe('waiting_for_gpu');
expect(status.vram_needed_mb).toBe(3951);
expect(status.vram_free_mb).toBe(512);
});
it('calls the correct WHISPER_URL endpoint', async () => {
vi.stubEnv('WHISPER_URL', 'http://gpu-box:9090');
mocks.fetch.mockResolvedValue({
ok: true,
json: () => Promise.resolve({ state: 'ready' })
});
await getModelStatus();
expect(mocks.fetch).toHaveBeenCalledWith(
'http://gpu-box:9090/model/status',
expect.objectContaining({ signal: expect.anything() })
);
vi.unstubAllEnvs();
});
it('throws when the server returns a non-ok response', async () => {
mocks.fetch.mockResolvedValue({ ok: false, status: 503 });
await expect(getModelStatus()).rejects.toThrow('/model/status');
});
});
// ── streamJob SSE parsing ─────────────────────────────────────────────────────
function makeSSEResponse(lines: string[]) {
@@ -108,6 +535,10 @@ function makeSSEResponse(lines: string[]) {
return { ok: true, body };
}
function makeSSEChunkResponse(chunks: string[]) {
return { ok: true, body: Readable.from(chunks) };
}
describe('streamJob — SSE event parsing', () => {
it('calls onProgress for progress events with percent, chunk, total', async () => {
const onProgress = vi.fn();
@@ -201,6 +632,27 @@ describe('streamJob — SSE event parsing', () => {
expect(onProgress).toHaveBeenNthCalledWith(3, 75, 3, 4);
});
it('handles multiple SSE events delivered in a single chunk', async () => {
const onProgress = vi.fn();
const onDone = vi.fn();
const onError = vi.fn();
mocks.fetch.mockResolvedValue(
makeSSEChunkResponse([
'data: {"type":"progress","percent":25,"chunk":1,"total":2}\n\n' +
'data: {"type":"progress","percent":50,"chunk":2,"total":2}\n\n' +
'data: {"type":"done","job":{}}\n\n'
])
);
await streamJob('whisper-id', onProgress, onDone, onError);
expect(onProgress).toHaveBeenCalledTimes(2);
expect(onProgress).toHaveBeenNthCalledWith(1, 25, 1, 2);
expect(onProgress).toHaveBeenNthCalledWith(2, 50, 2, 2);
expect(onDone).toHaveBeenCalledOnce();
expect(onError).not.toHaveBeenCalled();
});
it('defaults chunk and total to 0 when missing from progress event', async () => {
const onProgress = vi.fn();
const onDone = vi.fn();

View File

@@ -5,6 +5,7 @@ export default defineConfig({
test: {
environment: 'node',
globals: true,
fileParallelism: false,
include: ['src/tests/**/*.test.ts'],
coverage: {
provider: 'v8',