trueref/FINDINGS.md

# trueref — Findings

Research notes backing the choices in [ARCHITECTURE.md](ARCHITECTURE.md). Each section ends with a verdict and follow-up questions if any.

---

## F1. Context7 ingestion behavior (what we replicate functionally)

- Context7 ingests git repositories and crawls associated docs sites driven by a `context7.json` manifest at the repo root, plus an optional `llms.txt` index.
- It produces snippets shaped roughly as `{ title, description, source, code, language }` and serves them via two MCP tools: `resolve-library-id` and `get-library-docs`.
- The `get-library-docs` API accepts `topic` and `tokens` parameters; topic biases retrieval, tokens caps the response size (defaults observed in client docs: ~5000).
- Source: upstash/context7 GitHub repo & MCP docs.

**Verdict:** functional parity is achievable without copying the manifest schema. Our chunk model captures the same fields under different names (`symbol`/`content`/`filePath`/`language`). MCP tool signatures are kept **byte-identical** for LLM compatibility.

---

## F2. Embedded vector store choice — Lucene 9 over Qdrant

- Qdrant is a Rust binary; embedding it in a fat JAR requires extracting & spawning a child process, contradicting the "single JAR, embedded everything" goal.
- **Apache Lucene ≥9.0** ships HNSW kNN (`KnnFloatVectorField`) alongside BM25 in a single index segment. Pure JVM, no native deps.
- Lucene supports **filtered kNN** (`KnnFloatVectorQuery` with a `BooleanQuery` filter), which we need for `(repoId, versionId)` scoping.
- Trade-off: Lucene HNSW lacks Qdrant's payload-rich filtering tricks (e.g. quantization presets, named vectors). Acceptable for our scale; we get BM25 in the same store for free.

**Verdict:** Lucene 9 (we'll target the latest 9.x). One `IndexWriter`, refresh-on-search via `SearcherManager`.

---

## F3. Embedding model — bge-m3

- BAAI/bge-m3: 568M params, 8192 ctx, multilingual (100+ langs), trained on multi-functionality (dense + sparse + colbert).
- ONNX export available (BAAI provides it; community variants on HuggingFace).
- License: MIT-style (model weights), works for self-hosted commercial use.
- Vector dim: 1024 (dense). Sparse vocab compatible with Lucene if we want SPLADE-like sparse — out of scope for v1.

**Verdict:** bge-m3 (dense only for v1). Sparse channel deferred.

---

## F4. Reranker — bge-reranker-v2-m3

- Cross-encoder, scores (query, passage) pairs.
- Same family as embedder: balanced quality/cost, ONNX-exportable.
- Apache 2.0 license.

**Verdict:** bge-reranker-v2-m3. Top-K candidates from RRF fed in, top-N (default 20) returned.

---

## F5. ML runtime — ONNX Runtime (Java bindings)

- ONNX Runtime has **official Java bindings** (`com.microsoft.onnxruntime:onnxruntime` + `onnxruntime_gpu`).
- Execution providers we will support:
  - **CUDA** (`onnxruntime_gpu`): Linux + Windows with NVIDIA driver ≥ matching CUDA 12.x.
  - **DirectML** (`onnxruntime-directml`): Windows, any DX12 GPU.
  - **CPU**: always-on fallback.
- ONNX Runtime has **no Vulkan execution provider**. Our earlier "Vulkan fallback" wish is not satisfiable in this stack — we drop it.
- Generative LLMs in ONNX (e.g. Phi-3.5-mini) are possible but awkward (KV cache management, tokenizer differences). Since we picked **retrieval-only**, no generative model is needed.

**Verdict:** ONNX Runtime, providers tried in order: cuda → directml → cpu. Vulkan dropped (documented).

---

## F6. Java version — 21 LTS, not 25

- Spring Boot 3.5.x officially supports Java 17–23.
- Spring AI 1.0.x targets the same range.
- Java 25 is supported by neither at time of writing; risking obscure reflection/MR-JAR issues with downstream libs (JGit, Lucene, ONNX bindings).
- Java 21 is LTS and has stable virtual threads + structured concurrency (`StructuredTaskScope` was preview through 23, finalizing soon — we'll guard usage behind a thin wrapper to ease later upgrade).

**Verdict:** Java 21 LTS. Re-evaluate to 25 once Spring Boot certifies it.

---

## F7. Differential indexing scheme

- We chose **dedupe-by-content-hash** AND **git-diff-driven file skipping**.
- The hash dedupe alone gives constant-cost embeddings for unchanged code across tags.
- The git-diff path additionally avoids parsing/chunking unchanged files, which dominates ingest CPU on large repos.
- Storage model:
  - `chunks`: one row per unique `content_hash`. Vector lives in Lucene keyed by `chunkId`.
  - `chunk_versions`: many-to-many; one row per `(chunk, version, file, line range)`.
  - Search: `BooleanQuery(filter=chunk_versions.version_id IN scope)` joined to vector field.
- The chunk dedupe ratio is reported as a UI metric — it's the most intuitive measure of "differential" effectiveness.

**Verdict:** confirmed; both mechanisms compose without conflict.

---

## F8. MCP transport — Streamable HTTP

- The current MCP spec (revision 2025-03-26) defines **Streamable HTTP**: a single `POST /mcp` endpoint that may upgrade to SSE for long-lived/streamed responses; replaces the deprecated 2024-11-05 SSE transport.
- Spring AI 1.0 ships an MCP server module that supports Streamable HTTP via Spring MVC.
- We expose **only** Streamable HTTP, no SSE-only legacy endpoint (per user spec).

**Verdict:** Streamable HTTP only at `/mcp`.

---

## F9. Embedded SQL store — H2 (MVCC)

- H2 in MVCC mode supports concurrent readers and a single writer with row-level locking. Good enough for our metadata write rates (jobs, versions, chunk_versions).
- File-based, single JAR dependency, JDBC.
- Considered & rejected:
  - **DuckDB**: column-store, slower OLTP, no good Flyway story.
  - **SQLite**: poor concurrency under write load.
  - **Embedded Postgres (zonky)**: pulls a 100+ MB native binary per OS — fights the fat JAR goal.

**Verdict:** H2 file-based, MVCC=true, with Flyway migrations.

---

## F10. Job orchestration — custom virtual-thread orchestrator

- Spring Batch is feature-rich but requires a JobRepository (typically Postgres or H2) and adds startup cost we don't need.
- Our jobs are **per-tag**, **simple linear stage sequences**, with persistence-of-status as the only durability requirement.
- Custom orchestrator: each `IngestionJob` runs on a virtual thread; stages execute sequentially; stage transitions are durably written to H2 in a transaction; `JobEventBus` emits events for SSE.
- Crash recovery: on startup, scan jobs in `RUNNING` status, mark them `FAILED` (or resume specific resumable stages — v2).

**Verdict:** custom orchestrator. Spring Batch deferred unless we hit a ceiling.

---

## F11. Code parser — pure-Java heuristic for v1, tree-sitter pluggable for v2

The Java tree-sitter ecosystem in 2026 is fragmented:

- **`io.github.tree-sitter:jtreesitter`** uses Project Panama FFI → requires **Java 22+**. We target Java 21 LTS, so this is out.
- **`io.github.bonede:tree-sitter`** is JNI-based and works on Java 21, but bundling per-OS (linux/windows/mac × x64/arm64) native grammar binaries for many languages bloats the fat JAR significantly and creates a packaging matrix we don't want to maintain in v1.
- **`ai.serenade.treesitter:java-tree-sitter`** is unmaintained.

**Decision (v1):** ship a pure-Java heuristic `CodeParser` adapter. Strategies, tried in order per file:

1. **Markdown / `.txt` / `.rst`**: split by ATX/Setext headings; large sections further split by paragraph.
2. **Brace-balanced languages** (java, c, c++, c#, go, rust, js, ts, kotlin, scala, swift): walk the file tracking brace depth + line-based heuristics (function signatures, top-level declarations) to extract chunks of complete top-level constructs. Symbol name extracted via a tiny regex per language.
3. **Indent-based languages** (python, yaml, ruby): split on top-level `def`/`class`/`module` boundaries; symbol name from the declaration line.
4. **Fallback** (any text file): sliding-window of N lines (default 80) with M lines overlap (default 10).

The `CodeParser` port is unchanged. A future tree-sitter implementation (when JDK upgrade or upstream packaging matures) can be swapped in by providing an alternate `@Component` and toggling a config flag — that's exactly what hexagonal architecture buys us.

**Verdict:** pure-Java heuristic parser for v1; tree-sitter remains a documented future enhancement.

---

## F12. Concurrency caps & GPU contention

- User chose **unbounded virtual threads**. This is safe for I/O-bound stages.
- ONNX inference is GPU-bound; calling the same `OrtSession` from many threads concurrently is unsupported. Two mitigations:
  1. A **session pool** of size N (config `embedding.session-count`, default 2).
  2. A **`Semaphore(N)`** acquired by any caller before invoking inference. Pool & semaphore sizes match.
- This means tag-level parallelism is naturally throttled by GPU capacity without explicit per-tag limits.

**Verdict:** session pool + semaphore. Document the knob clearly in `application.yml`.

---

## F13. Frontend in fat JAR

- SvelteKit `@sveltejs/adapter-static` produces a fully static bundle (HTML/CSS/JS). We build it as a Maven sub-step (frontend-maven-plugin) and copy `frontend/build/` to `bootstrap/src/main/resources/static/`. Spring serves it by default.
- SPA fallback: a `WebMvcConfigurer` maps all unmatched non-API paths to `index.html` so client-side routing works.

**Verdict:** static adapter + Spring static-resource serving. Single artifact preserved.

---

## F14. Open questions / future work

1. **Sparse channel** (bge-m3 sparse / SPLADE) for stronger lexical recall — deferred to v2.
2. **Per-language reranker fine-tuning** — out of scope (no fine-tuning, per spec).
3. **Compaction job** to truly delete orphan chunks (currently soft-delete on versions). Schedule TBD.
4. **Watched-folder** auto-discovery semantics: how often do we rescan `./data/watched/`? Default proposal: every 5 min + on filesystem watch event (Java NIO `WatchService`).
5. **Repo size cap**: do we need a maximum total cloned size to prevent runaway disk use? Currently unlimited; could add per-repo and global caps in v2.
6. **GPU memory introspection**: Linux NVML via JNI (`jnvml`) for GPU mem gauges; on Windows + DirectML we surface only "available/in-use" booleans.

---

## F15. References (for re-checking when libraries bump)

- Context7 repo & MCP tool surface — to sanity-check schema fidelity on releases.
- Spring AI 1.0.x release notes — verify MCP server Streamable HTTP module name & API.
- Spring Boot 3.5.x release notes — confirm Java version compatibility window.
- Lucene 9.x kNN docs — confirm filtered vector query API surface.
- ONNX Runtime Java release notes — confirm CUDA/DirectML EP availability per version.
- BAAI/bge-m3 model card — confirm ONNX export availability/format.
- MCP spec 2025-03-26 — Streamable HTTP transport requirements.

> Use the Context7 MCP lookup skill before bumping any of the above to fetch fresh, version-specific docs.

---

## F16. Smoke-test log (2026-04-21)

End-to-end smoke after first assembly:
- `mvn -pl trueref-bootstrap -am package` → BUILD SUCCESS, fat JAR ~582 MB.
- `mvn test` → **16 tests pass** (parser 6, pooling 5, disk cache 5), **0 failures**.
- `java -jar trueref-bootstrap/target/trueref.jar --trueref.embedding.session-count=0` — started in 3.6 s.
- `GET /actuator/health` → `UP` (db H2, disk, ping, ssl).
- `POST /api/repos` + `GET /api/repos` — round-trips a repo.
- `GET /swagger-ui.html` → 302 redirect (to `/swagger-ui/index.html`), `GET /v3/api-docs` → 200.
- `GET /` → 200 (SvelteKit SPA served from Spring static resources).
- Historical note: at this point the server still used the legacy WebMVC SSE transport, so `POST /mcp` without an established `GET /sse` session returned HTTP 500. This was later replaced by the Streamable HTTP transport on `GET`/`POST /mcp`.

Fixes landed during smoke:
- `V1__init_schema.sql`: H2 in PostgreSQL mode rejects `AUTO_INCREMENT`. Switched `job_log_events.id` to `BIGINT GENERATED BY DEFAULT AS IDENTITY` and removed the explicit `NULL` constraint.
- `OnnxProperties.sessionCount` can now be 0 (disables the ONNX stack, for environments where models aren't available); `GpuSemaphore` accepts 0 permits by internally using 1 (never acquired in disabled mode).
- `OnnxEmbeddingService` / `OnnxRerankerService` short-circuit in disabled mode; reranker pass-through preserves input order.
- `ApplicationBeans` exposes only concrete beans (not both the class and its interface) to avoid ambiguous autowiring.