2 Commits

Include renames

Author	SHA1	Message	Date
mozempkandClaude Opus 4.8	ea852087c4	feat(slot-proxy): live load/prefill/decode progress on :48090 ... Add a slot-cache-proxy service (profile stack) in front of llama-swap-stack. Injects return_progress into streaming chat, taps llama.cpp prompt_progress SSE, serves GET /progress {loading|prefill|generating|idle} with pct/tok_s/ ETA. The pi-load-progress extension polls it; point pi llmruntime baseUrl at http://localhost:48090/v1. host net + pid:host (load tracker reads the containerised llama-server /proc/io). Transparent superset of llama-swap (no endpoint lost; /progress is net-new). Script mirrors llmruntime-x570. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-07-21 13:22:33 +02:00
mozempk	4ad296608b	Initial commit: tuned multi-model llama.cpp stack ... - 5 models: SmolLM3-3B, Gemma4-E2B/E4B, Qwen3-4B, Qwen3.5-9B - TurboQuant image (FORCE_MMQ): +6-11% free speed on Turing GPUs - Bigctx profiles (-nkvo KV in RAM): 2-16x context gain - turbo2 KV: 2x smaller, benchmarked against PPL quality gate - Per-model env files with justified parameters - kv_quant_test.sh + cpu_ctx_test.sh benchmark scripts - docs/FINDINGS.md: surprises, pitfalls, recommendations - docs/ARCHITECTURE.md: compose + test script design	2026-05-06 15:56:40 +02:00