swap-stack: slot KV save/restore for duo prompt caches (x570 S5 pattern)

--slot-save-path + --cache-reuse 256 on both duo members, ./kv-cache
mounted at /cache. Measured on ornith (3K-token system prompt): warm
prefix reuse 296ms vs 106.7s cold prefill (361x); slot save 84MB in
59ms, GDN recurrent state serializes. scripts/duo-warmboot.sh wraps
save/restore/status via llama-swap /upstream passthrough. Restore
after restart: run 'duo-warmboot.sh restore' post stack boot.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
2026-07-10 16:13:20 +02:00
co-authored by Claude Fable 5
parent 85e27a1a47
commit 597b882743
4 changed files with 45 additions and 0 deletions
+2
View File
@@ -71,6 +71,7 @@ models:
--ctx-size 131072
--batch-size 1024 --ubatch-size 512
--cont-batching --parallel 1
--slot-save-path /cache/ornith-duo --cache-reuse 256
ttl: 0
"qwen3-4b-duo":
@@ -86,6 +87,7 @@ models:
--ctx-size 24576
--batch-size 512 --ubatch-size 256
--cont-batching --parallel 1
--slot-save-path /cache/qwen-duo --cache-reuse 256
ttl: 0
# ── MoE offload models (2026-07-10) ────────────────────────────────────────