feat(gallery): add vllm.cpp text-generation models

[localai-bot]

Adds eight curated vllm-cpp entries to the model gallery. Until now the backend had gallery coverage only for MiniMax-H3 video, so serving text on it meant hand-writing engine_args.

Entries

Entry	Model	Size	Needs
qwen3.6-27b-nvfp4-vllm-cpp	Qwen3.6-27B, NVFP4	25 GB	Blackwell GPU
qwen3.6-27b-nvfp4-mtp-vllm-cpp	the same, MTP speculative decoding	25 GB	Blackwell GPU
qwen3.6-27b-nvfp4-dflash-vllm-cpp	the same, DFlash speculative decoding	28 GB	Blackwell GPU
qwen3.6-35b-a3b-nvfp4-vllm-cpp	Qwen3.6-35B-A3B MoE, NVFP4	23 GB	Blackwell GPU
qwen3.6-35b-a3b-nvfp4-mtp-vllm-cpp	the same, MTP speculative decoding	23 GB	Blackwell GPU
qwen3-coder-30b-a3b-vllm-cpp	Qwen3-Coder-30B-A3B, bf16	57 GB	Blackwell GPU, or CPU
qwen3-4b-vllm-cpp	Qwen3-4B, bf16	8 GB	CPU, Metal, Vulkan, Blackwell GPU
qwen3-0.6b-vllm-cpp	Qwen3-0.6B, bf16	1.4 GB	CPU, Metal, Vulkan, Blackwell GPU

The flagship tier is what vllm.cpp gates its own correctness and speed claims on. No backend code is needed: master already carries speculative_config, the parser knobs and resolveDraftModelPath.

Three details that are load-bearing

The 27B revision pin. unsloth/Qwen3.6-27B-NVFP4 was re-quantized in place, under the same repository name, from NVFP4 to FP8 W8A8. Its HEAD is now ccdaab7e. The entries pin 890bdef7…, which is the NVFP4 arm that beats vLLM 1.007-1.045x at every concurrency. Without the pin the entry resolves to different weights, with different numerics, and nothing reports that anything changed.

tool_parser: qwen3_coder is named explicitly. That dialect is byte-identical on the wire to step3p5's, so chat-template sniffing cannot separate the two and auto-detection picks the wrong parser. hermes and think_auto do auto-detect correctly and are named only for stability.

enable_prefix_caching is deliberately never set. It defaults on for dense models and off for the GDN hybrids, and that per-model default is the right answer.

num_blocks is sized per model from its real KV footprint rather than copied between entries: 20 KiB/token on the 35B (10 of 40 layers are full-attention), 64 KiB/token on the 27B (16 of 64), but 144 KiB/token on the dense 4B.

Docs

Adds features/vllm-cpp.md: installation, the model table, the pinning rationale, and how to choose between the speculative variants. Cross-linked from the existing engine_args reference in text-generation.md, which stays the configuration reference rather than being duplicated.

It also records that the CUDA images are built for Blackwell only, which is narrower than vllm.cpp's own ten-architecture release. That is what turns an otherwise cryptic no kernel image is available for execution on the device into something a user can act on. Widening the build is a separate PR.

Verification

• Gallery suite green (go test ./core/gallery/...).
• All eight entries decode and validate as a ModelConfig; both DFlash file hashes computed from the completed download.
• qwen3-0.6b-vllm-cpp confirmed end to end on a real cluster: chat, plus tool calling returning engine-parsed tool_calls with finish_reason: tool_calls.
• Backend logs confirm the settings reach the engine as authored, including the managed snapshot as load target and prefixCaching="model-default".
• The five NVFP4 flagship entries are not runtime-verified. No available node has kernels for them: the one Blackwell box was unavailable and the other GPU node is sm_110, which LocalAI does not build. They are verified by construction only.

🤖 Generated with Claude Code