Commit Graph

17 Commits

Author SHA1 Message Date
Giancarmine Salucci
c1e1a9d05e residual fixes 2026-03-24 18:17:42 +01:00
Giancarmine Salucci
391eb7f411 fix(ui): resolve state_referenced_locally warnings and add folder picker
- Fix Svelte state_referenced_locally warning in +page.svelte and repos/[id]/+page.svelte
  by initializing $state with empty defaults and syncing via $effect
- Add FolderPicker component with server-side filesystem browser
  (single-click to navigate, double-click or "Select This Folder" to confirm)
- Git repos highlighted with orange folder icon and "git" badge
- Add GET /api/v1/fs/browse endpoint listing subdirectories
- Wire FolderPicker into AddRepositoryModal for local source type
- Auto-fills title from the selected folder name

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-23 09:17:44 +01:00
Giancarmine Salucci
9e3f62e329 feat(TRUEREF-0017): implement incremental re-indexing with checksum diff
- computeDiff classifies files into added/modified/deleted/unchanged buckets
- Only changed and new files are parsed and re-embedded on re-runs
- Deleted files removed atomically from DB
- Progress counts all files including unchanged for accurate reporting
- ~20x speedup for re-indexing large repositories with few changes

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-23 09:07:20 +01:00
Giancarmine Salucci
22bf4c1014 feat(TRUEREF-0016): implement web UI search explorer
- Two-step search workflow: resolve library then query documentation
- URL state sync (/search?lib=...&q=...)
- LibraryResult, SnippetCard, SearchInput components
- Code/info snippet display with breadcrumbs and token counts
- Copy-as-markdown button for full response

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-23 09:07:13 +01:00
Giancarmine Salucci
90d93786a8 feat(TRUEREF-0015): implement web UI repository dashboard
- Repository list with state badges, stats, and action buttons
- Add repository modal for GitHub URLs and local paths
- Live indexing progress bar polling every 2s
- Confirm dialog for destructive actions
- Repository detail page with versions and recent jobs
- Settings page placeholder

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-23 09:07:06 +01:00
Giancarmine Salucci
542f4ce66c feat(TRUEREF-0014): implement repository version management
- VersionService with list, add, remove, getByTag, registerFromConfig
- GitHub tag discovery helper for validating tags before indexing
- Version ID format: /owner/repo/tag (e.g. /facebook/react/v18.3.0)
- GET/POST /api/v1/libs/:id/versions
- DELETE /api/v1/libs/:id/versions/:tag
- POST /api/v1/libs/:id/versions/:tag/index

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-23 09:06:59 +01:00
Giancarmine Salucci
f31db2db2c feat(TRUEREF-0013): implement trueref.json config file support
- Lenient parser for trueref.json and context7.json (trueref.json takes precedence)
- Validates folders, excludeFolders, excludeFiles, rules, previousVersions
- Stores config in repository_configs table
- JSON Schema served at GET /api/v1/schema/trueref-config.json for IDE validation
- Rules injected at top of every query-docs response

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-23 09:06:50 +01:00
Giancarmine Salucci
21f6acbfa3 feat(TRUEREF-0009-0010): implement indexing pipeline job queue and public REST API
- SQLite-backed job queue with sequential processing and startup recovery
- Atomic snippet replacement in single transaction
- context7-compatible GET /api/v1/libs/search and GET /api/v1/context
- Token budget limiting and JSON/txt response format support
- CORS headers on all API routes via SvelteKit handle hook
- Library ID parser supporting /owner/repo and /owner/repo/version

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-23 09:06:35 +01:00
Giancarmine Salucci
d3d577a2e2 feat(TRUEREF-0008): implement hybrid semantic search with RRF
- Cosine similarity vector search over stored embeddings
- Reciprocal Rank Fusion (K=60) combining FTS5 + vector rankings
- Configurable alpha weight between keyword and semantic search
- Graceful degradation to FTS5-only when no embedding provider configured

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-23 09:06:25 +01:00
Giancarmine Salucci
33bdf30709 feat(TRUEREF-0006): implement SQLite FTS5 full-text search engine
- BM25 ranking via SQLite FTS5 bm25() function
- Query preprocessor with wildcard expansion and special char escaping
- Library search with composite scoring (name match, trust score, snippet count)
- Trust score computation from stars, coverage, and source type
- Response formatters for library and snippet results

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-23 09:06:18 +01:00
Giancarmine Salucci
f6be3cfd47 feat(TRUEREF-0005): implement document parser and chunker
- Markdown parser with heading-based section splitting and code block extraction
- Code file parser with regex boundary detection for 10+ languages
- Sliding window chunker with configurable token limits and overlap
- Language detection from file extensions

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-23 09:06:12 +01:00
Giancarmine Salucci
1c15d6c474 feat(TRUEREF-0003-0004): implement GitHub and local filesystem crawlers
- GitHub crawler with rate limiting, semaphore concurrency, retry logic
- File filtering by extension, size, and trueref.json rules
- Local filesystem crawler with SHA-256 checksums and progress callbacks
- Shared types and file filter logic between both crawlers

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-23 09:06:07 +01:00
Giancarmine Salucci
956b2a3a62 feat(TRUEREF-0009): implement indexing pipeline and job queue
Implements the end-to-end indexing pipeline with a SQLite-backed job
queue, startup recovery, and REST API endpoints for job status.

- IndexingPipeline: orchestrates crawl → parse → atomic replace → embed
  → repo stats update with progress tracking at each stage
- JobQueue: sequential SQLite-backed queue (no external broker), deduplicates
  active jobs per repository, drains queued jobs on startup
- startup.ts: stale job recovery (running→failed), repo state reset, singleton
  initialization wired from hooks.server.ts
- GET /api/v1/jobs with repositoryId/status/limit filtering
- GET /api/v1/jobs/[id] single job lookup
- hooks.server.ts: initializes DB and pipeline on server start
- 18 unit tests covering queue, pipeline stages, recovery, and atomicity

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-22 18:22:20 +01:00
Giancarmine Salucci
bf4caf5e3b feat(TRUEREF-0007): implement pluggable embedding generation and vector storage
Add EmbeddingProvider interface with OpenAI-compatible, local (optional
@xenova/transformers via dynamic import), and Noop (FTS5-only fallback)
implementations. EmbeddingService batches requests and persists Float32Array
blobs to snippet_embeddings. GET/PUT /api/v1/settings/embedding endpoints
read and write embedding config from the settings table.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-22 18:07:26 +01:00
Giancarmine Salucci
3d1bef5003 feat(TRUEREF-0002): implement repository management service and REST API
Add RepositoryService with full CRUD, ID resolution helpers, input
validation, six SvelteKit API routes (GET/POST /api/v1/libs,
GET/PATCH/DELETE /api/v1/libs/:id, POST /api/v1/libs/:id/index), and
37 unit tests covering all service operations.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-22 17:43:06 +01:00
Giancarmine Salucci
f57b622505 feat(TRUEREF-0001): implement complete database schema and core data models
Define all SQLite tables via Drizzle ORM (repositories, repository_versions,
documents, snippets, snippet_embeddings, indexing_jobs, repository_configs,
settings), generate the initial migration, create FTS5 virtual table and
sync triggers in fts.sql, add shared TypeScript types in src/lib/types.ts,
and write 21 unit tests covering insertions, cascade deletes, FK constraints,
blob storage, JSON fields, and FTS5 trigger behaviour.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-22 17:18:01 +01:00
Giancarmine Salucci
18437dfa7c chore: initial project scaffold 2026-03-22 17:08:15 +01:00