Files
redsen-lean-harness/docs/adr/0002-web-tree-sitter-index.md
T
mozempkandCopilot 383129f571 feat: scaffold redsen-lean-harness v0.1.0
Recovered from crashed session (Node OOM). Repo contains full P0-P6
scaffold: plugin.json/marketplace.json, AGENTS.md, ADRs 0001-0006,
lh CLI (init/index/graph/lane/run/memory/host/report/doctor), 10
.github/agents, 12 CLI skills, instructions, context7 mcp.json, and
unit/e2e test suite.

Fixed: run.mjs read --in-tokens/--out-tokens but tests and CLI docs
use --input-tokens/--output-tokens, so telemetry totals were always 0.
Now accepts both forms.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-09-09 22:44:15 +02:00

2.1 KiB

ADR 0002 — web-tree-sitter (WASM) for the code index, with a regex fallback

  • Status: Accepted
  • Date: 2026-09-09

Context

The harness must work on brownfield repos in any language, be OS-agnostic, and install with no build step. It needs a token-budgeted repo map so agents stop reading whole files.

Candidates evaluated:

Option Verdict
tree-sitter (native bindings) Requires a native compile toolchain — fails "easy install"
web-tree-sitter (WASM) Pure npm, no compiler, runs anywhere Node runs
universal-ctags External binary the user must install; 200+ languages but not npm-installable
ast-grep Prebuilt binary via npm, good, but heavier and rule-oriented rather than map-oriented
SCIP / LSIF Per-language indexers — far too much install surface
Zoekt Requires Docker
Host /lsp Excellent fidelity but CLI-only and not available in every host/language

Decision

Use web-tree-sitter, the sole runtime dependency.

Grammars are not vendored and not fetched at import time. They are cached lazily under .agents/.cache/wasm/ and only downloaded when the user passes --fetch. When a grammar is absent, the indexer silently degrades to a per-language regex extractor and reports the degradation in lh index --stats.

The repo map is ranked with a hand-rolled PageRank over the reference graph and truncated to config.index.budget tokens, degrading full signatures → names → file-level counts.

Consequences

  • lh index works fully offline on first run, at lower fidelity. This is a hard requirement.
  • No compiler, no Docker, no external binary, no service. npm install is the whole setup.
  • Fidelity varies by language and by whether a grammar has been fetched. --stats makes this visible.
  • We own a small glob matcher and a small PageRank implementation rather than taking dependencies.

Alternatives considered

Rejected --fetch-by-default: it would make the first run fail on an air-gapped machine and would surprise users with network traffic. Opt-in is the safer default.