[Sync] LLM clients → Qwen3.6-35B-A3B-Thinking default, Ornith preset, fl GPU monitor abbrs
This commit is contained in:
@@ -52,7 +52,7 @@ starves the desktop and freezes the PC (reboot). Hard-won rules:
|
||||
| model id | decode t/s | ctx | extras | role |
|
||||
|---|---|---|---|---|
|
||||
| `gemma-4-E4B-it-UD-Q8_K_XL` | **57.3** | **96k** | vision, MTP, prefill 185 t/s | fast small generalist, long docs, images (ctx 96k ✓ 08-08, n_ctx_train 131072) |
|
||||
| `Qwen3.6-35B-A3B-Thinking` | 38.8 | 24k | MTP, vision, reasoning | hard problems, slow-but-smart answers |
|
||||
| `Qwen3.6-35B-A3B-Thinking` | 38.8 | 24k | MTP, vision, reasoning (budget 8k = "medium") | **default model** in opencode/pi/aichat since 09-21; hard problems, slow-but-smart answers |
|
||||
| `gpt-oss-20b` | 38.4 | **64k** | reasoning, tools | fast reasoning + tool use · fast long-context (37 t/s vs Coder-Next 16; 131k ruled out) |
|
||||
| `gpt-oss-20b-low` | 37.2 | **64k** | reasoning LOW, TTFT 0.8 s | same model, snappy answers (no long preamble) |
|
||||
| `Qwen3.6-35B-A3B-MTP-UD-IQ3_XXS` | 35.9 | 24k | MTP, vision, thinking OFF | **daily driver** — compact instant answers |
|
||||
@@ -129,6 +129,9 @@ The same catalog is mirrored in every client — when adding/removing a preset,
|
||||
- `common/.config/opencode/opencode.json` (both providers + agent model overrides)
|
||||
- `common/.pi/agent/models.json` (both providers; `contextWindow` = server `ctx-size`)
|
||||
- `common/.config/aichat/config.yaml` (both clients)
|
||||
- `wm/.config/aichat/config.yaml` — full-file overlay for the off-LAN host (default client `duskadiy`);
|
||||
every aichat change must be mirrored here too (missed for GLM Q3 + Ornith until 09-21). With the
|
||||
lw/ overlays that makes **seven** client files.
|
||||
|
||||
Rule: client model **id = config.ini section name**, client context ≤ server `ctx-size`.
|
||||
The embedder is deliberately absent from chat clients.
|
||||
@@ -140,6 +143,7 @@ the router, but chats/presets saved against the old id need re-picking.
|
||||
(14.5 t/s MTP / 10.9 base, 2.8 GB free @1.5); server ctx restored to 24576 = client ctx, no client edits.
|
||||
2026-09-21: `GLM-4.7-Flash-UD-Q3_K_XL` (⚡ fast coder) added to all six client files next to the Q4.
|
||||
2026-09-21: `Ornith-1.5-35B-A3B-IQ3_XXS` (agentic coder, thinking + vision) added to all six client files after passing its gate (32.1 t/s @moe16).
|
||||
2026-09-21: default model → `Qwen3.6-35B-A3B-Thinking` everywhere (opencode global + plan/build/auto, pi defaultModel + thinking level medium, aichat common/lw/wm) — gemma-4-26B refused routine sysadmin work. Thinking preset gets `reasoning-budget 8192`. wm aichat caught up (GLM Q3, Ornith).
|
||||
|
||||
## Tuning cheat-sheet
|
||||
|
||||
|
||||
@@ -165,6 +165,13 @@ mmproj = /home/anon/software/models/mmproj-Qwen3.6-35B-A3B-F16.gguf
|
||||
# 13.9 used / 3.3 free (2.8 dirty baseline), decode unchanged at 36-37 t/s.
|
||||
spec-type = draft-mtp
|
||||
spec-draft-n-max = 2
|
||||
reasoning-budget = 8192 # "MEDIUM" thinking (added 2026-09-21 when this became the default model in
|
||||
# opencode/pi/aichat): caps runaway 15-20k-token think loops, leaves 16k of
|
||||
# the 24k ctx for prompt+answer; normal traces (0.5-5k) are untouched. The
|
||||
# Qwen3.6 template knows only enable_thinking on/off (no reasoning_effort
|
||||
# levels — verified in the gguf header), so this budget IS the depth knob.
|
||||
# -1 = unlimited (old behaviour), 0 = the MTP alias. b10615's server README
|
||||
# documents N>0; if startup rejects it → set -1 and update llama.cpp.
|
||||
ctx-size = 24576 # thinking eats ctx — same footprint as the non-thinking preset
|
||||
n-gpu-layers = 99
|
||||
n-cpu-moe = 16
|
||||
|
||||
Reference in New Issue
Block a user