mozempk/trueref
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
Files
c5f950c2c08ca953cc579bb34fc1a43cbeca6035
trueref/FINDINGS.md
moze c5f950c2c0
Some checks failed
Build and publish Docker image / Build and push (push) Failing after 1m27s
Details
Initial commit: trueref v0.1.0-SNAPSHOT 
Java 21 / Spring Boot 3.5.3 multi-module Maven project.
Hybrid BM25+HNSW search with RRF, cross-encoder reranker,
ONNX Runtime 1.22.0 (CPU + CUDA 12 GPU variants).
2026-05-06 00:49:16 +02:00

12 KiB
Raw Blame History

trueref — Findings

Research notes backing the choices in ARCHITECTURE.md. Each section ends with a verdict and follow-up questions if any.

F1. Context7 ingestion behavior (what we replicate functionally)

  • Context7 ingests git repositories and crawls associated docs sites driven by a context7.json manifest at the repo root, plus an optional llms.txt index.
  • It produces snippets shaped roughly as { title, description, source, code, language } and serves them via two MCP tools: resolve-library-id and get-library-docs.
  • The get-library-docs API accepts topic and tokens parameters; topic biases retrieval, tokens caps the response size (defaults observed in client docs: ~5000).
  • Source: upstash/context7 GitHub repo & MCP docs.

Verdict: functional parity is achievable without copying the manifest schema. Our chunk model captures the same fields under different names (symbol/content/filePath/language). MCP tool signatures are kept byte-identical for LLM compatibility.

F2. Embedded vector store choice — Lucene 9 over Qdrant

  • Qdrant is a Rust binary; embedding it in a fat JAR requires extracting & spawning a child process, contradicting the "single JAR, embedded everything" goal.
  • Apache Lucene ≥9.0 ships HNSW kNN (KnnFloatVectorField) alongside BM25 in a single index segment. Pure JVM, no native deps.
  • Lucene supports filtered kNN (KnnFloatVectorQuery with a BooleanQuery filter), which we need for (repoId, versionId) scoping.
  • Trade-off: Lucene HNSW lacks Qdrant's payload-rich filtering tricks (e.g. quantization presets, named vectors). Acceptable for our scale; we get BM25 in the same store for free.

Verdict: Lucene 9 (we'll target the latest 9.x). One IndexWriter, refresh-on-search via SearcherManager.

F3. Embedding model — bge-m3

  • BAAI/bge-m3: 568M params, 8192 ctx, multilingual (100+ langs), trained on multi-functionality (dense + sparse + colbert).
  • ONNX export available (BAAI provides it; community variants on HuggingFace).
  • License: MIT-style (model weights), works for self-hosted commercial use.
  • Vector dim: 1024 (dense). Sparse vocab compatible with Lucene if we want SPLADE-like sparse — out of scope for v1.

Verdict: bge-m3 (dense only for v1). Sparse channel deferred.

F4. Reranker — bge-reranker-v2-m3

  • Cross-encoder, scores (query, passage) pairs.
  • Same family as embedder: balanced quality/cost, ONNX-exportable.
  • Apache 2.0 license.

Verdict: bge-reranker-v2-m3. Top-K candidates from RRF fed in, top-N (default 20) returned.

F5. ML runtime — ONNX Runtime (Java bindings)

  • ONNX Runtime has official Java bindings (com.microsoft.onnxruntime:onnxruntime + onnxruntime_gpu).
  • Execution providers we will support:
    • CUDA (onnxruntime_gpu): Linux + Windows with NVIDIA driver ≥ matching CUDA 12.x.
    • DirectML (onnxruntime-directml): Windows, any DX12 GPU.
    • CPU: always-on fallback.
  • ONNX Runtime has no Vulkan execution provider. Our earlier "Vulkan fallback" wish is not satisfiable in this stack — we drop it.
  • Generative LLMs in ONNX (e.g. Phi-3.5-mini) are possible but awkward (KV cache management, tokenizer differences). Since we picked retrieval-only, no generative model is needed.

Verdict: ONNX Runtime, providers tried in order: cuda → directml → cpu. Vulkan dropped (documented).

F6. Java version — 21 LTS, not 25

  • Spring Boot 3.5.x officially supports Java 17–23.
  • Spring AI 1.0.x targets the same range.
  • Java 25 is supported by neither at time of writing; risking obscure reflection/MR-JAR issues with downstream libs (JGit, Lucene, ONNX bindings).
  • Java 21 is LTS and has stable virtual threads + structured concurrency (StructuredTaskScope was preview through 23, finalizing soon — we'll guard usage behind a thin wrapper to ease later upgrade).

Verdict: Java 21 LTS. Re-evaluate to 25 once Spring Boot certifies it.

F7. Differential indexing scheme

  • We chose dedupe-by-content-hash AND git-diff-driven file skipping.
  • The hash dedupe alone gives constant-cost embeddings for unchanged code across tags.
  • The git-diff path additionally avoids parsing/chunking unchanged files, which dominates ingest CPU on large repos.
  • Storage model:
    • chunks: one row per unique content_hash. Vector lives in Lucene keyed by chunkId.
    • chunk_versions: many-to-many; one row per (chunk, version, file, line range).
    • Search: BooleanQuery(filter=chunk_versions.version_id IN scope) joined to vector field.
  • The chunk dedupe ratio is reported as a UI metric — it's the most intuitive measure of "differential" effectiveness.

Verdict: confirmed; both mechanisms compose without conflict.

F8. MCP transport — Streamable HTTP

  • The current MCP spec (revision 2025-03-26) defines Streamable HTTP: a single POST /mcp endpoint that may upgrade to SSE for long-lived/streamed responses; replaces the deprecated 2024-11-05 SSE transport.
  • Spring AI 1.0 ships an MCP server module that supports Streamable HTTP via Spring MVC.
  • We expose only Streamable HTTP, no SSE-only legacy endpoint (per user spec).

Verdict: Streamable HTTP only at /mcp.

F9. Embedded SQL store — H2 (MVCC)

  • H2 in MVCC mode supports concurrent readers and a single writer with row-level locking. Good enough for our metadata write rates (jobs, versions, chunk_versions).
  • File-based, single JAR dependency, JDBC.
  • Considered & rejected:
    • DuckDB: column-store, slower OLTP, no good Flyway story.
    • SQLite: poor concurrency under write load.
    • Embedded Postgres (zonky): pulls a 100+ MB native binary per OS — fights the fat JAR goal.

Verdict: H2 file-based, MVCC=true, with Flyway migrations.

F10. Job orchestration — custom virtual-thread orchestrator

  • Spring Batch is feature-rich but requires a JobRepository (typically Postgres or H2) and adds startup cost we don't need.
  • Our jobs are per-tag, simple linear stage sequences, with persistence-of-status as the only durability requirement.
  • Custom orchestrator: each IngestionJob runs on a virtual thread; stages execute sequentially; stage transitions are durably written to H2 in a transaction; JobEventBus emits events for SSE.
  • Crash recovery: on startup, scan jobs in RUNNING status, mark them FAILED (or resume specific resumable stages — v2).

Verdict: custom orchestrator. Spring Batch deferred unless we hit a ceiling.

F11. Code parser — pure-Java heuristic for v1, tree-sitter pluggable for v2

The Java tree-sitter ecosystem in 2026 is fragmented:

  • io.github.tree-sitter:jtreesitter uses Project Panama FFI → requires Java 22+. We target Java 21 LTS, so this is out.
  • io.github.bonede:tree-sitter is JNI-based and works on Java 21, but bundling per-OS (linux/windows/mac × x64/arm64) native grammar binaries for many languages bloats the fat JAR significantly and creates a packaging matrix we don't want to maintain in v1.
  • ai.serenade.treesitter:java-tree-sitter is unmaintained.

Decision (v1): ship a pure-Java heuristic CodeParser adapter. Strategies, tried in order per file:

  1. Markdown / .txt / .rst: split by ATX/Setext headings; large sections further split by paragraph.
  2. Brace-balanced languages (java, c, c++, c#, go, rust, js, ts, kotlin, scala, swift): walk the file tracking brace depth + line-based heuristics (function signatures, top-level declarations) to extract chunks of complete top-level constructs. Symbol name extracted via a tiny regex per language.
  3. Indent-based languages (python, yaml, ruby): split on top-level def/class/module boundaries; symbol name from the declaration line.
  4. Fallback (any text file): sliding-window of N lines (default 80) with M lines overlap (default 10).

The CodeParser port is unchanged. A future tree-sitter implementation (when JDK upgrade or upstream packaging matures) can be swapped in by providing an alternate @Component and toggling a config flag — that's exactly what hexagonal architecture buys us.

Verdict: pure-Java heuristic parser for v1; tree-sitter remains a documented future enhancement.

F12. Concurrency caps & GPU contention

  • User chose unbounded virtual threads. This is safe for I/O-bound stages.
  • ONNX inference is GPU-bound; calling the same OrtSession from many threads concurrently is unsupported. Two mitigations:
    1. A session pool of size N (config embedding.session-count, default 2).
    2. A Semaphore(N) acquired by any caller before invoking inference. Pool & semaphore sizes match.
  • This means tag-level parallelism is naturally throttled by GPU capacity without explicit per-tag limits.

Verdict: session pool + semaphore. Document the knob clearly in application.yml.

F13. Frontend in fat JAR

  • SvelteKit @sveltejs/adapter-static produces a fully static bundle (HTML/CSS/JS). We build it as a Maven sub-step (frontend-maven-plugin) and copy frontend/build/ to bootstrap/src/main/resources/static/. Spring serves it by default.
  • SPA fallback: a WebMvcConfigurer maps all unmatched non-API paths to index.html so client-side routing works.

Verdict: static adapter + Spring static-resource serving. Single artifact preserved.

F14. Open questions / future work

  1. Sparse channel (bge-m3 sparse / SPLADE) for stronger lexical recall — deferred to v2.
  2. Per-language reranker fine-tuning — out of scope (no fine-tuning, per spec).
  3. Compaction job to truly delete orphan chunks (currently soft-delete on versions). Schedule TBD.
  4. Watched-folder auto-discovery semantics: how often do we rescan ./data/watched/? Default proposal: every 5 min + on filesystem watch event (Java NIO WatchService).
  5. Repo size cap: do we need a maximum total cloned size to prevent runaway disk use? Currently unlimited; could add per-repo and global caps in v2.
  6. GPU memory introspection: Linux NVML via JNI (jnvml) for GPU mem gauges; on Windows + DirectML we surface only "available/in-use" booleans.

F15. References (for re-checking when libraries bump)

  • Context7 repo & MCP tool surface — to sanity-check schema fidelity on releases.
  • Spring AI 1.0.x release notes — verify MCP server Streamable HTTP module name & API.
  • Spring Boot 3.5.x release notes — confirm Java version compatibility window.
  • Lucene 9.x kNN docs — confirm filtered vector query API surface.
  • ONNX Runtime Java release notes — confirm CUDA/DirectML EP availability per version.
  • BAAI/bge-m3 model card — confirm ONNX export availability/format.
  • MCP spec 2025-03-26 — Streamable HTTP transport requirements.

Use the Context7 MCP lookup skill before bumping any of the above to fetch fresh, version-specific docs.

F16. Smoke-test log (2026-04-21)

End-to-end smoke after first assembly:

  • mvn -pl trueref-bootstrap -am package → BUILD SUCCESS, fat JAR ~582 MB.
  • mvn test → 16 tests pass (parser 6, pooling 5, disk cache 5), 0 failures.
  • java -jar trueref-bootstrap/target/trueref.jar --trueref.embedding.session-count=0 — started in 3.6 s.
  • GET /actuator/health → UP (db H2, disk, ping, ssl).
  • POST /api/repos + GET /api/repos — round-trips a repo.
  • GET /swagger-ui.html → 302 redirect (to /swagger-ui/index.html), GET /v3/api-docs → 200.
  • GET / → 200 (SvelteKit SPA served from Spring static resources).
  • POST /mcp one-shot JSON-RPC returns HTTP 500 — expected, the WebMVC MCP transport requires an SSE session established by GET /sse first; MCP clients that implement the Streamable-HTTP spec do this automatically. Verified MCP tools register: tools/list handler is reached (error thrown is transport-level session lookup, not bean wiring).

Fixes landed during smoke:

  • V1__init_schema.sql: H2 in PostgreSQL mode rejects AUTO_INCREMENT. Switched job_log_events.id to BIGINT GENERATED BY DEFAULT AS IDENTITY and removed the explicit NULL constraint.
  • OnnxProperties.sessionCount can now be 0 (disables the ONNX stack, for environments where models aren't available); GpuSemaphore accepts 0 permits by internally using 1 (never acquired in disabled mode).
  • OnnxEmbeddingService / OnnxRerankerService short-circuit in disabled mode; reranker pass-through preserves input order.
  • ApplicationBeans exposes only concrete beans (not both the class and its interface) to avoid ambiguous autowiring.
Reference in New Issue View Git Blame Copy Permalink