Commit Graph

13 Commits

Author SHA1 Message Date
Giancarmine Salucci
ec6140e3bb TRUEREF-0022 fix, more tests 2026-03-30 19:11:09 +02:00
Giancarmine Salucci
7630740403 feat(TRUEREF-0022): complete iteration 0 — worker-thread indexing, parallel jobs, SSE progress
- Move IndexingPipeline.run() into Worker Threads via WorkerPool
- Add dedicated embedding worker thread with single model instance
- Add stage/stageDetail columns to indexing_jobs schema
- Create ProgressBroadcaster for SSE channel management
- Add SSE endpoints: GET /api/v1/jobs/:id/stream, GET /api/v1/jobs/stream
- Replace UI polling with EventSource on repo detail and admin pages
- Add concurrency settings UI and API endpoint
- Build worker entries separately via esbuild
2026-03-30 17:08:23 +02:00
U811073
6f3f4db19b fix(TRUEREF-0021): reduce event loop blocking, add busy_timeout, and add TRUEREF-0022 PRD 2026-03-30 15:46:15 +02:00
Giancarmine Salucci
0bf01e3057 last fix 2026-03-29 12:44:06 +02:00
Giancarmine Salucci
666ec7d55f feat(MULTIVERSION-0001): wire trueref.json into pipeline + per-version rules
- Add migration 0003: recreate repository_configs with nullable version_id
  column and two partial unique indexes (repo-wide: version_id IS NULL,
  per-version: (repository_id, version_id) WHERE version_id IS NOT NULL)
- Update schema.ts to reflect the new composite structure with uniqueIndex
  partial constraints via drizzle-orm sql helper
- IndexingPipeline: parse trueref.json / context7.json after crawl, apply
  excludeFiles filter before diff computation, update totalFiles accordingly
- IndexingPipeline: persist repo-wide rules (version_id=null) and
  version-specific rules (when versionId set) via upsertRepoConfig helper
- Add matchesExcludePattern static helper supporting plain filename,
  glob prefix (docs/legacy*), and exact path patterns
- context endpoint: split getRules into repo-wide + version-specific lookup
  with dedup merge; pass versionId at call site
- Update test DB loaders to include migration 0003
- Add pipeline tests for excludeFiles, repo-wide rules persistence, and
  per-version rules persistence
- Add integration tests for merged rules, repo-only rules, and dedup logic

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-28 10:44:30 +01:00
Giancarmine Salucci
5a3c27224d chore(FEEDBACK-0001): linting 2026-03-27 02:23:01 +01:00
Giancarmine Salucci
e7a2a83cdb feat(TRUEREF-0020): add job status page with pause/resume/cancel controls
- Extend indexing_jobs schema to support 'paused' and 'cancelled' status
- Add JobQueue methods: pauseJob(), resumeJob(), cancelJob()
- Create POST /api/v1/jobs/[id]/{pause,resume,cancel} endpoints
- Implement /admin/jobs page with auto-refresh (3s polling)
- Add JobStatusBadge component with color-coded status display
- Action buttons appear contextually based on job status
- Optimistic UI updates with error handling
- All 477 existing tests pass, no regressions
2026-03-25 20:38:14 +01:00
Giancarmine Salucci
9519a66cef test(embeddings): fix 6 remaining test failures
- Fix schema.test.ts: use Unix timestamp integers instead of Date objects for snippet_embeddings.createdAt
- Fix embedding.service.test.ts: use 'local-default' profile instead of non-existent 'test-profile', remove require() calls and use proper ESM imports
- Fix hybrid.search.service.test.ts: update VectorSearch.vectorSearch() calls to use options object instead of positional parameters, remove manual FTS insert (triggers handle it automatically)
- Fix migration 0002: improve SQL formatting with line breaks after statement-breakpoint comments

All 459 tests now passing (18 skipped).
2026-03-25 19:41:24 +01:00
Giancarmine Salucci
169df4d984 feat(TRUEREF-0020): add embedding profiles, default local embeddings, and version-scoped semantic retrieval
- Add embedding_profiles table with provider registry pattern
- Install @xenova/transformers as runtime dependency
- Update snippet_embeddings with composite PK (snippet_id, profile_id)
- Seed default local profile using Xenova/all-MiniLM-L6-v2
- Add provider registry (local-transformers, openai-compatible)
- Update EmbeddingService to persist and retrieve by profileId
- Add version-scoped VectorSearch with optional versionId filtering
- Add searchMode (auto|keyword|semantic|hybrid) to HybridSearchService
- Update API /context route to load active profile, support searchMode/alpha params
- Extend MCP query-docs tool with searchMode and alpha parameters
- Update settings API to work with embedding_profiles table
- Add comprehensive test coverage for profiles, registry, version scoping

Status: 445/451 tests passing, core feature complete
2026-03-25 19:16:37 +01:00
Giancarmine Salucci
fef6f66930 wip(TRUEREF-0018): commit version-scoped indexing work 2026-03-25 19:03:22 +01:00
Giancarmine Salucci
3d1bef5003 feat(TRUEREF-0002): implement repository management service and REST API
Add RepositoryService with full CRUD, ID resolution helpers, input
validation, six SvelteKit API routes (GET/POST /api/v1/libs,
GET/PATCH/DELETE /api/v1/libs/:id, POST /api/v1/libs/:id/index), and
37 unit tests covering all service operations.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-22 17:43:06 +01:00
Giancarmine Salucci
f57b622505 feat(TRUEREF-0001): implement complete database schema and core data models
Define all SQLite tables via Drizzle ORM (repositories, repository_versions,
documents, snippets, snippet_embeddings, indexing_jobs, repository_configs,
settings), generate the initial migration, create FTS5 virtual table and
sync triggers in fts.sql, add shared TypeScript types in src/lib/types.ts,
and write 21 unit tests covering insertions, cascade deletes, FK constraints,
blob storage, JSON fields, and FTS5 trigger behaviour.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-22 17:18:01 +01:00
Giancarmine Salucci
18437dfa7c chore: initial project scaffold 2026-03-22 17:08:15 +01:00