[Sync] LLM clients → Qwen3.6-35B-A3B-Thinking default, Ornith preset, fl GPU monitor abbrs

Re-export of private HEAD 73b4a65: default-model flip + 8k thinking budget
in the aichat/opencode/pi client configs, the Ornith-1.5-35B-A3B preset and
its gate bench in fl/.config/llamacpp, and the fl fish abbrs `vram` /
`vramx` with the new gpumon.sh (sysfs VRAM/GTT monitor).

fl/.config/fish/host.fish

@@ -1,2 +1,11 @@
-# fl runs the llama.cpp router as a systemd service (llama.service) — no shell abbr needed.
+# fl runs the llama.cpp router as a systemd service (llama.service) — no launch abbr needed.
 # Presets: ~/.config/llamacpp/config.ini; see ~/.config/llamacpp/README.md.
+
+## GPU monitors — the RX 7600 XT also drives the display, so VRAM headroom is a safety matter
+## (config.ini header: keep ≥ 2.5 GB free; GTT growth = VRAM spilling to RAM = the freeze precursor).
+# vram : quick rocm-smi % view + who holds the render node. fuser prints its table on STDERR,
+#        so it must be 2>&1 — with 2>/dev/null the "LLM GPU usage" section is always empty.
+abbr vram 'watch -c -d -n 1 \'echo "=== VRAM ==="; rocm-smi --showmemuse --showuse 2>/dev/null | grep -E "GPU use|Allocated|Read/Write"; echo ""; echo "=== LLM GPU usage ==="; fuser -v /dev/dri/renderD128 2>&1 | grep -iE "COMMAND|llama"\''
+# vramx: absolute GiB used/free + GTT + temp straight from amdgpu sysfs (no ROCm), per-process VRAM
+#        via fdinfo, and which preset the router has resident. Body = gpumon.sh (runs bare too).
+abbr vramx 'watch -c -d -n 1 ~/.config/llamacpp/gpumon.sh'

fl/.config/llamacpp/README.md

@@ -4,7 +4,7 @@ llama.cpp in **router mode** serving ~10 model presets from one 16 GB GPU.
 Last full tune: **2026-07-16** (decode numbers measured then via `bench.py`, unless a preset comment in `config.ini` says otherwise).
 Last full clean sweep: **2026-08-08 03:22** — all 11 presets loaded and ran green (baseline 2.0 GB) after the week's coder-race/KV-fix/gemma-sidecar round.
 
-- **Box**: AMD RX 7600 XT 16 GB (~288 GB/s; backend = Arch `llama.cpp-vulkan` / RADV, **not** ROCm —
+- **Box**: AMD RX 7600 XT 16 GB (~288 GB/s; backend = Arch `llama-cpp` 0.5.0 + `ggml-vulkan` / RADV = llama.cpp **b11146** since 2026-10-02, **not** ROCm —
   `rocm-smi` is only the VRAM monitor) + Ryzen 5600X (6 cores) + 48 GB RAM.
   ⚠ The GPU **also drives the display** — see [VRAM safety](#vram-safety-the-golden-rules).
 - **Endpoints**: LAN `http://$SERVER:11343/v1` · remote `https://<own-domain>/api/v1`
@@ -35,6 +35,14 @@ symlinks, so on the server `dotsync` (pull; re-stows only if files were added/re
 `sudo systemctl restart llama.service` — the router reads presets once at startup.
 After ANY preset change: restart + `./bench.py -m <changed-ids>` and check the guards.
 
+**Upgrading llama.cpp itself** (2026-10-02, b10615 → b11146): Arch replaced `llama.cpp-vulkan`
+with a unified `llama-cpp` package on top of the system `ggml`, whose backends are split out.
+`sudo pacman -Syu ggml llama-cpp ggml-vulkan` — name **`ggml`** explicitly (it conflicts with
+the old `ggml-cpu` without replacing it, and the removal prompt defaults to N), and **`ggml-vulkan`**
+explicitly (only an optdepend — without it llama-server silently runs on the CPU backend).
+Then restart, and `llama-server --list-devices` must show `Vulkan0`. `whisper-cpp` moves to the
+same `ggml` (CPU backends are inside the base package) — restart its unit too.
+
 ## VRAM safety (the golden rules)
 
 The amdgpu driver does not OOM cleanly — an overcommitted model spills into GTT (system RAM),
@@ -62,7 +70,7 @@ starves the desktop and freezes the PC (reboot). Hard-won rules:
 | `GLM-4.7-Flash-UD-Q4_K_XL` | **21.5** @moe22 | 24k | reasoning | quality coder (opencode subagents); KV fix + moe22 ✓ verified clean 08-08 |
 | `GLM-4.7-Flash-UD-Q3_K_XL` | **26.0** @moe12 (2.0 daytime baseline; ~28 clean) | 24k | reasoning | **fast coder** — re-added 09-21 because Q4's 21.5 dragged in agent loops; 3-bit, evening-safe at moe12 (13.4 used / 3.8 free @2.0); Q4 stays for hard reviews |
 | `Qwen3-Coder-Next-UD-IQ3_XXS` | 16.0 | **128k** | 80B-A3B | long-session coder (128k ctx) |
-| `Qwen3.8-27B-UD-IQ3_XXS` | 14.5 (10.9 w/o MTP) | 24k | vision, MTP, hybrid reasoner | ⚠ **clean-desktop specialty** (kept 09-10 over the ≥22 gate) — dense-27B quality, slow: its Gated-DeltaNet layers are Vulkan-bound (prefill ~30). 2.5 GB free @1.6 baseline @24k (✓ 09-10); **never on a busy evening desktop** (~1 GB free = GTT freeze) |
+| `Qwen3.8-27B-UD-IQ3_XXS` | **15.5** (10.9 w/o MTP) | 24k | vision, MTP, hybrid reasoner @ **xhigh** | ⚠ **clean-desktop specialty** (kept 09-10 over the ≥22 gate) — dense-27B quality, slow: its Gated-DeltaNet layers are Vulkan-bound (prefill ~35). 2.5 GB free @1.7 baseline @24k (✓ 10-02 on b11146, `-n 512`, acceptance 0.84). The *felt* slowness is the **xhigh thinking preamble** (17.5 s before the first answer token on a trivial prompt; `reasoning_effort=low` measured 6.45 s at the same decode on 10-02 — siblings benched then removed by decision, the kwarg is one line if ever wanted); **never on a busy evening desktop** (~1 GB free = GTT freeze) |
 | `Qwen3-Embedding-0.6B` | 33 (CPU) | 8k | CPU-only, `/v1/embeddings` | RAG/search embedder (not a chat model) |
 | `Ornith-1.5-35B-A3B-IQ3_XXS` | **32.1** @moe16 | 24k | MTP, vision, thinking | **agentic coder** (added 09-21) — Qwen3.5-MoE derivative claiming +15 Terminal-Bench / +5.6 SWE-bench over Qwen3.6-35B; the only coder with MTP; 13.8 used / 3.3 free @2.0 |
 
@@ -144,6 +152,7 @@ the router, but chats/presets saved against the old id need re-picking.
 2026-09-21: `GLM-4.7-Flash-UD-Q3_K_XL` (⚡ fast coder) added to all six client files next to the Q4.
 2026-09-21: `Ornith-1.5-35B-A3B-IQ3_XXS` (agentic coder, thinking + vision) added to all six client files after passing its gate (32.1 t/s @moe16).
 2026-09-21: default model → `Qwen3.6-35B-A3B-Thinking` everywhere (opencode global + plan/build/auto, pi defaultModel + thinking level medium, aichat common/lw/wm) — gemma-4-26B refused routine sysadmin work. Thinking preset gets `reasoning-budget 8192`. wm aichat caught up (GLM Q3, Ornith).
+2026-10-12: Ornith in pi: "Response was truncated before completion." = `finish_reason: length` — the `<think>` block counts against pi's 4096 `max_tokens` and the preset has no reasoning-budget. Fixed client-side: pi sends `thinking_budget_tokens` per thinking level on Ornith (`compat.thinkingTokenBudgetField` + `settings.json > thinkingBudgets`, common + lw). The preset deliberately keeps NO CLI reasoning-budget — it would override the per-request field (see the preset comment).
 
 ## Tuning cheat-sheet
 
@@ -183,8 +192,17 @@ verifies a clean card with rocm-smi before every load, records a ⚠ row and ski
 generation when a model lands too tight (free < 1.5 GB or GTT ballooning).
 Results: `bench-results.md` (latest) + `bench-history.md` (append-only; diff runs there).
 
+**Reading TTFT**: bench.py stamps it on the first non-empty `content` delta, so for thinking
+models TTFT = prompt eval **plus the whole reasoning preamble**, and `0.00` means the answer never
+started within `-n` tokens (thinking ate the budget). It is the column that compares
+reasoning-effort siblings; for stable MTP decode numbers use `-n 512`.
+
 ## Watch list
 
 - **GLM MTP in llama.cpp** — the single biggest pending win (primary coder ~22 → ~30+).
+- **DFlash2 for Qwen3.8-27B** — `z-lab/Qwen3.8-27B-DFlash2-GGUF` drafter, upstream 1.81× vs the
+  MTP head's 1.33× here (b11146 has the support). Blockers: #27408 (spec + mmproj → image requests
+  500, so only as a text-only sibling preset) and #28158 (Vulkan OOB draft token, open). Trial =
+  `spec-type draft-dflash`, `spec-draft-n-max 3`, no mmproj; n-max 7 does not fit the VRAM envelope.
 - **Coder-Next 256k**: if the load log's `n_ctx_train` says 262144, ctx can likely double
   again for ~1 GB (KV measured nearly flat).

fl/.config/llamacpp/config.ini

@@ -299,6 +299,7 @@ sleep-idle-seconds = 60
 #     (157/195, mean len 2.6) — so MTP IS engaging; at 2.6 tok/step that is ~175 ms per step ≈
 #     3–4× a plain forward pass. NOTE the box runs the Arch llama.cpp-vulkan package (RADV,
 #     device Vulkan0) — NOT HIP/ROCm (this header said "ROCm" until 09-10); rocm-smi = monitoring only.
+#     [10-02: that package is gone — now Arch `llama-cpp` 0.5.0 (b11146) + `ggml-vulkan`, see RE-PROBE below]
 #     Suspects: the 48 GatedDeltaNet layers on the Vulkan backend (upstream #20354: missing/slow
 #     GDN shader → CPU fallback) and/or recurrent-state checkpoint/restore per speculative step
 #     (~150 MiB for this model; reported net-negative on hybrid 27Bs). The hybrid 35B-A3B does
@@ -327,6 +328,57 @@ sleep-idle-seconds = 60
 #     buffer buys nothing), MTP stays (+33% for 1.5 GB). ✓ CONFIRMED @24576 2026-09-10 03:51
 #     (1.6 baseline): 15.1 t/s / 31 pp / 14.6 used / 2.5 free — footprint 13.0, exactly as
 #     predicted; free = the 2.5 target at a normal idle, ~2.8 pristine, ~0.8 on a 3.3 evening.
+#   ✦ RE-PROBE 2026-10-02 03:11 on llama.cpp b11146 = Arch `llama-cpp 0.5.0` + `ggml-vulkan 0.25.3`
+#     (the `llama.cpp-vulkan b10615` package is gone; `ggml` conflicts with the old `ggml-cpu` and
+#     does not replace it → name it: `sudo pacman -Syu ggml llama-cpp ggml-vulkan`, answer y twice;
+#     `ggml-vulkan` is only an optdepend — without it the server silently runs on CPU).
+#     4.0 DIRTY baseline: 14.8 t/s / 18 pp / 14.4 used / 2.8 free. Decode IDENTICAL to 09-10
+#     (14.5–15.1) → the build moves nothing on speed, as predicted. VRAM columns INVALID this run:
+#     14.4 used at a 4.0 baseline = model "added" 10.4 GB < its own 10.93 GB file — amdgpu evicted
+#     the desktop's allocations to GTT to make room (the 08-16 masking effect, 3 GB this time);
+#     footprint stays 13.0 from the clean 09-10 runs. TTFT 0.00 — decoded in the 03:34 entry
+#     below (NOT a bench quirk, as this line first claimed).
+#     Re-run 03:16 @1.8 normal-idle baseline: 13.8 t/s / 33 pp / 14.7 used / 2.4 free → footprint
+#     12.9, VRAM half exactly as on 09-10 (2.5 @1.6); 13.8 is inside the ±15% MTP acceptance
+#     swing (14.5–15.1 before). b11146 confirmed NEUTRAL on both speed and VRAM for this preset.
+#     Third run 03:19 @1.7: 12.5 t/s / 38 pp / TTFT 17.1 / 14.7 used / 2.4 free. Three back-to-back
+#     runs → decode BAND 12.5–14.8 with VRAM identical each time; prefill 18→33→38 = Vulkan
+#     pipeline warm-up on the first load after the restart, decode drift = acceptance at temp 1.0
+#     over 256 tokens (±15% documented) and possibly GPU heat after 3 consecutive loads.
+#   ✦ DEFINITIVE 2026-10-02 03:34, -n 512 @1.7 baseline: 15.5 t/s / 38 pp / TTFT 20.6 / 14.7 used /
+#     2.5 free. Server log: eval 442 tok @ 15.50 t/s (64.5 ms/tok), draft acceptance 0.842
+#     (278/330), mean len 2.68, graphs reused 163 → MTP = 1.42× over the 10.9 base. The 256-token
+#     runs under-read (short window + acceptance noise). REFERENCE: 15.5 (10.9 w/o MTP).
+#     TTFT DECODED (bench.py stamps ttft on the first non-empty `content` delta; reasoning deltas
+#     do not count): TTFT = prompt eval + the whole THINKING preamble. 17.1 s / 20.6 s ≈ 230-300
+#     tokens of xhigh thinking on the bench's trivial prompt; "0.00" on the 256-token runs =
+#     thinking consumed the entire budget and no answer was ever produced. The GGUF's embedded
+#     template (header read 10-02) defaults reasoning_effort to XHIGH — heaviest of
+#     xhigh/medium/low — and this preset passes nothing, so every answer pays it. That, not the
+#     15 t/s, is why it feels slow. → sibling presets benched 03:49 (next entry), then removed.
+#   ✦ SIBLING BENCH 2026-10-02 03:49, -n 512 @1.9: xhigh 16.2 t/s / TTFT 17.5 s / 14.9 used / 2.2
+#     free; -low 15.9 t/s / TTFT 6.45 s / 15.0 / 2.1 → reasoning_effort=low cuts the preamble 63%
+#     at identical decode. -medium SKIPPED by the guard (15.7 used / 1.4 free at load) = VRAM
+#     drain lag after two back-to-back loads, not a medium cost (same weights). → USER DECISION
+#     10-02: both siblings REMOVED from the config. The lever remains one line on THIS preset —
+#     chat-template-kwargs = {"reasoning_effort":"low"} (or "medium") — or per request via
+#     chat_template_kwargs; xhigh stays the default here.
+#     What b11146 DOES carry for this preset: #27812 (merged 08-28) fixes the Vulkan graph
+#     optimizer reordering nodes that alias Qwen3.8's recurrent state — on b10615 that meant
+#     possible wrong tokens at temp 0 and acceptance numbers "that mean nothing"; everything from
+#     10-02 on is from a correct build. Also DFlash2 support (#27342, 08-27). NOT in b11146:
+#     RDNA3 int8 coopmat prefill kernels (#27952, 09-24 — K-quant/q8 tensors only, not IQ3_XXS
+#     blocks anyway) and the GDN tune (#29476, 09-29 — an Intel fix, AMD measured flat).
+#     CORRECTION to the 09-10 suspects: the Vulkan GDN shader has existed since 2026-03
+#     (#20334 → #20662 → #24581); there is no CPU fallback — 10.9 base IS the RDNA3 kernel
+#     ceiling. The remaining lever is speculative acceptance LENGTH, not the kernel:
+#     z-lab/Qwen3.8-27B-DFlash2-GGUF (Q4_K_M 1.14 GB drafter, mrope-fixed 08-24) — upstream
+#     10.4 → 18.9 t/s (1.81×, AL ≈ 5) on this model vs MTP's 1.33× here (AL 2.6). Costs: the
+#     drafter + ~150 MiB recurrent rollback plane PER draft token → spec-draft-n-max 3–4 fits
+#     the MTP envelope, the default 7 does not. DFlash + mmproj = open #27408 (image requests
+#     stall/500) → any trial is a separate TEXT-ONLY preset; this one keeps MTP + vision.
+#     Open on Vulkan for MTP and DFlash alike: #28158 — intermittent draft token id 248320
+#     (= n_vocab) → HTTP 500 "Invalid input batch", mostly after very long prompts. If seen: that.
 #   ⚠ Needs a CURRENT llama.cpp build: first "qwen35" file HERE with ssm.*/hybrid-attn keys
 #   (the 9B is regular-attention qwen35). If the load fails with unknown-architecture /
 #   missing-tensor errors, update llama.cpp on fl before touching this preset.
@@ -493,6 +545,13 @@ sleep-idle-seconds = 30
 #     Role: agentic coder with reasoning — the only coder here with MTP (Coder-30B 30, GLM Q3 26).
 #     Instruct alias (reasoning-budget = 0, same file) still optional — add if agent loops want
 #     the ~4 s thinking TTFT gone.
+#   ⚠ NO reasoning-budget LINE IN THIS PRESET, ON PURPOSE (2026-10-12): pi caps the think block
+#     PER REQUEST (`thinking_budget_tokens`, from pi settings.json > thinkingBudgets, tested on
+#     b11429: budget 40 → trace cut at ~40 tokens, answer still arrives, finish_reason stop) and
+#     llama.cpp IGNORES that field whenever the preset sets a CLI --reasoning-budget. Adding one
+#     here "like the Thinking preset" silently disables the pi knob and brings back pi's
+#     "Response was truncated before completion." (= finish_reason length: think block + answer
+#     > the client's 4096 max_tokens). An instruct alias (budget 0) in a SEPARATE section is fine.
 # ⬇ DOWNLOAD from bartowski/Ornith-1.5-35B-A3B-GGUF (imatrix; the official repo has only Q4_K_M+
 #   ≥ 21.7 GB, unsloth has no 1.5 quants yet):
 #     Ornith-1.5-35B-A3B-IQ3_XXS.gguf     15,340,447,392 B · sha256 8918ccb9ee29abe3875efec0c3e86f0e35ef0b8a03e3f0e1c422869859a518d0

fl/.config/llamacpp/gpumon.sh

@@ -0,0 +1,70 @@
+#!/bin/sh
+# gpumon.sh — one screen of GPU state for fl (RX 7600 XT · amdgpu · Vulkan backend).
+#
+#   watch -c -d -n 1 ~/.config/llamacpp/gpumon.sh    (fish abbr `vramx`, fl/.config/fish/host.fish)
+#   ~/.config/llamacpp/gpumon.sh                      (one shot)
+#
+# Why not rocm-smi: `--showmemuse` gives percentages and hides GTT. The freeze mode on this box is
+# VRAM overcommit spilling into GTT (system RAM) — see the config.ini header — so GTT and the
+# absolute free-VRAM figure (the ≥ 2.5 GB gate) are the numbers that matter. amdgpu exposes both
+# in sysfs: instant, no ROCm needed. Per-process VRAM comes from /proc/<pid>/fdinfo (kernel
+# drm-usage-stats), readable only for your own processes unless root. llama.service runs as the
+# login user (anon), so its children show up; if they ever don't:
+#   systemctl show -p User llama.service      and run this with sudo.
+# No absolute GTT alarm on purpose: bench.py judges GTT as a delta over a per-model baseline
+# (a browser-heavy desktop legitimately sits on a few GB) — `watch -d` shows the movement.
+#
+# GPUMON_HOST overrides the router host (default 127.0.0.1) — for a dry run from another box.
+
+dev=/sys/class/drm/renderD128/device
+host=${GPUMON_HOST:-127.0.0.1}
+port=11343
+
+echo "=== GPU (amdgpu sysfs) ==="
+if [ -r "$dev/mem_info_vram_used" ]; then
+  temp=$(cat "$dev"/hwmon/hwmon*/temp1_input 2>/dev/null | head -n 1)
+  awk -v u="$(cat "$dev/mem_info_vram_used")" -v t="$(cat "$dev/mem_info_vram_total")" \
+      -v g="$(cat "$dev/mem_info_gtt_used" 2>/dev/null)" -v gt="$(cat "$dev/mem_info_gtt_total" 2>/dev/null)" \
+      -v b="$(cat "$dev/gpu_busy_percent" 2>/dev/null)" -v m="$(cat "$dev/mem_busy_percent" 2>/dev/null)" \
+      -v c="${temp:-0}" 'BEGIN {
+    G = 1073741824
+    free = (t - u) / G
+    flag = free < 1.5 ? "   <<< HARD FLOOR (bench.py guard)" : free < 2.5 ? "   <-- below the 2.5 GB target" : ""
+    printf "VRAM  %5.2f / %5.2f GiB   free %5.2f GiB%s\n", u / G, t / G, free, flag
+    printf "GTT   %5.2f / %5.2f GiB   (VRAM spilled to RAM — must not grow while a model is resident)\n", g / G, gt / G
+    printf "busy  gpu %3d%%   mem %3d%%   edge %d°C\n", b, m, c / 1000
+  }'
+else
+  echo "no amdgpu sysfs under $dev — not the fl box, or the GPU is not renderD128"
+fi
+
+echo
+echo "=== processes holding the GPU  (VRAM from fdinfo · RSS · uptime; own-user only unless root) ==="
+# fuser puts the PID list on stdout (the pretty table of -v goes to stderr, hence no -v here).
+pids=$(fuser /dev/dri/renderD128 /dev/dri/card* 2>/dev/null | tr ' ' '\n' | grep -E '^[0-9]+$' | sort -un)
+if [ -z "$pids" ]; then
+  echo "(nothing visible — no /dev/dri here, or the GPU users belong to another user)"
+else
+  for p in $pids; do
+    [ -r "/proc/$p/fdinfo" ] || continue
+    # one drm client can be reachable through several fds — dedupe by drm-client-id, then sum
+    vram=$(awk 'FNR == 1 { cid = "" }
+                /^drm-client-id:/ { cid = $2 }
+                /^drm-(total|memory)-vram:/ { v[cid] = $2 }
+                END { for (k in v) s += v[k]; printf "%.2f", s / 1048576 }' /proc/"$p"/fdinfo/* 2>/dev/null)
+    name=$(cat /proc/"$p"/comm 2>/dev/null)
+    alias=$(tr '\0' '\n' < /proc/"$p"/cmdline 2>/dev/null | awk '$0 == "--alias" { getline; print; exit }')
+    set -- $(ps -o rss=,etime= -p "$p" 2>/dev/null | awk '{ printf "%.1f %s", $1 / 1048576, $2 }')
+    printf '%7s GiB  %7s  %-16s  rss %5s GiB  up %-11s %s\n' "$vram" "$p" "$name" "${1:-?}" "${2:-?}" "$alias"
+  done | sort -rn | head -n 15
+fi
+
+echo
+echo "=== router $host:$port — preset status (anything not unloaded; the sysfs line above is the truth for VRAM) ==="
+if out=$(curl -sf -m 1 "http://$host:$port/models" 2>/dev/null); then
+  printf '%s' "$out" \
+    | jq -r '.data[] | select(.status.value != "unloaded") | "\(.status.value)\t\(.id)"' 2>/dev/null \
+    | { grep . || echo "(nothing resident)"; }
+else
+  echo "(router not answering — systemctl status llama.service)"
+fi