Recovered from crashed session (Node OOM). Repo contains full P0-P6 scaffold: plugin.json/marketplace.json, AGENTS.md, ADRs 0001-0006, lh CLI (init/index/graph/lane/run/memory/host/report/doctor), 10 .github/agents, 12 CLI skills, instructions, context7 mcp.json, and unit/e2e test suite. Fixed: run.mjs read --in-tokens/--out-tokens but tests and CLI docs use --input-tokens/--output-tokens, so telemetry totals were always 0. Now accepts both forms. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2.1 KiB
ADR 0002 — web-tree-sitter (WASM) for the code index, with a regex fallback
- Status: Accepted
- Date: 2026-09-09
Context
The harness must work on brownfield repos in any language, be OS-agnostic, and install with no build step. It needs a token-budgeted repo map so agents stop reading whole files.
Candidates evaluated:
| Option | Verdict |
|---|---|
tree-sitter (native bindings) |
Requires a native compile toolchain — fails "easy install" |
web-tree-sitter (WASM) |
Pure npm, no compiler, runs anywhere Node runs |
universal-ctags |
External binary the user must install; 200+ languages but not npm-installable |
ast-grep |
Prebuilt binary via npm, good, but heavier and rule-oriented rather than map-oriented |
| SCIP / LSIF | Per-language indexers — far too much install surface |
| Zoekt | Requires Docker |
Host /lsp |
Excellent fidelity but CLI-only and not available in every host/language |
Decision
Use web-tree-sitter, the sole runtime dependency.
Grammars are not vendored and not fetched at import time. They are cached lazily under
.agents/.cache/wasm/ and only downloaded when the user passes --fetch. When a grammar is
absent, the indexer silently degrades to a per-language regex extractor and reports the
degradation in lh index --stats.
The repo map is ranked with a hand-rolled PageRank over the reference graph and truncated to
config.index.budget tokens, degrading full signatures → names → file-level counts.
Consequences
lh indexworks fully offline on first run, at lower fidelity. This is a hard requirement.- No compiler, no Docker, no external binary, no service.
npm installis the whole setup. - Fidelity varies by language and by whether a grammar has been fetched.
--statsmakes this visible. - We own a small glob matcher and a small PageRank implementation rather than taking dependencies.
Alternatives considered
Rejected --fetch-by-default: it would make the first run fail on an air-gapped machine and
would surprise users with network traffic. Opt-in is the safer default.