feat(windows): add Vulkan GPU detection for Intel Arc and other non-CUDA GPUs

[darthcav]

Summary

Fixes #925 — Docker Model Runner does not use Intel Arc GPU via Vulkan on Windows.

Root cause

CanUseGPU in gpuinfo_windows.go only detects NVIDIA CUDA (amd64) and Qualcomm Adreno OpenCL (arm64). There is no Vulkan path, so Intel Arc and other Vulkan-capable GPUs (AMD without CUDA, etc.) are silently ignored and fall back to the cpu llama.cpp variant.

Changes

gpuinfo_windows.go

• hasVulkanCapableGPU() — uses PCI enumeration (via ghw) to find GPUs that are neither NVIDIA nor Adreno; Intel Arc, AMD GPUs, and similar fall into this category.
• hasVulkan() — probes vulkan-1.dll via syscall.LoadLibrary, mirroring the existing hasOpenCL / OpenCL.dll pattern. Returns true only when a Vulkan-capable GPU is present and the Vulkan runtime is loadable.
• CanUseGPU() — on amd64, after the CUDA check, now also calls hasVulkan().
• hasNVIDIAGPU(), hasSupportedAdrenoGPU(), hasVulkanCapableGPU() — added nil guards for DeviceInfo, Vendor, and Product fields before dereferencing .Name, preventing panics when ghw fails to retrieve full PCI details for a card.

download_windows.go

• Adds canUseVulkan detection in ensureLatestLlamaCpp.
• Priority order: CUDA > Vulkan > CPU (OpenCL stays arm64-only).
• Selects desiredVariant = "vulkan" when Vulkan is detected.

Known limitation / follow-up required

No vulkan image variant of docker/docker-model-backend-llamacpp currently exists on Docker Hub (tags available: cpu, cuda, opencl, rocm, metal, generic). When the vulkan tag is not found, downloadLatestLlamaCpp logs a warning and falls back to the vendored Docker Desktop binary — which already ships ggml-vulkan.dll. A TODO comment in download_windows.go marks this gap.

A follow-up task to publish docker/docker-model-backend-llamacpp:latest-vulkan (a Windows build with the Vulkan backend compiled in) is needed to complete the end-to-end fix.

Test plan

• Build compiles cleanly for GOOS=windows GOARCH=amd64 ✓
• Existing unit tests pass (go test ./pkg/inference/backends/llamacpp/...) ✓
• Manual verification on a Windows machine with Intel Arc GPU: docker model run ai/smollm2 "Test" should show Vulkan backend in logs once the vulkan Docker Hub image variant is published.

🤖 Generated with Claude Code

[sourcery-ai]

Hey - I've found 1 issue, and left some high level feedback:

• In hasVulkanCapableGPU, accessing gpu.DeviceInfo.Vendor.Name and gpu.DeviceInfo.Product.Name assumes these nested fields are always non-nil; consider adding nil checks to avoid panics on unexpected ghw data.
• The Adreno/Qualcomm detection in hasVulkanCapableGPU is case-sensitive while the vendor comparison is lowercased; normalizing product (and possibly using vendor IDs) would make the heuristic more robust and avoid misclassifying GPUs as Vulkan-capable.

Prompt for AI Agents

Please address the comments from this code review:

## Overall Comments
- In `hasVulkanCapableGPU`, accessing `gpu.DeviceInfo.Vendor.Name` and `gpu.DeviceInfo.Product.Name` assumes these nested fields are always non-nil; consider adding nil checks to avoid panics on unexpected `ghw` data.
- The Adreno/Qualcomm detection in `hasVulkanCapableGPU` is case-sensitive while the vendor comparison is lowercased; normalizing `product` (and possibly using vendor IDs) would make the heuristic more robust and avoid misclassifying GPUs as Vulkan-capable.

## Individual Comments

### Comment 1
<location path="pkg/inference/backends/llamacpp/gpuinfo_windows.go" line_range="111-113" />
<code_context>
+	if err != nil {
+		return false, err
+	}
+	for _, gpu := range gpus.GraphicsCards {
+		vendor := strings.ToLower(gpu.DeviceInfo.Vendor.Name)
+		product := gpu.DeviceInfo.Product.Name
+		isNVIDIA := vendor == "nvidia"
+		isAdreno := strings.Contains(product, "Adreno") || strings.Contains(product, "Qualcomm")
</code_context>
<issue_to_address>
**issue (bug_risk):** Guard against nil DeviceInfo/Product to avoid potential panics from ghw.GPU()

ghw may return GraphicsCards where `DeviceInfo`, `Vendor`, or `Product` is nil. Directly accessing `gpu.DeviceInfo.Vendor.Name` and `gpu.DeviceInfo.Product.Name` can therefore panic. Please add nil checks (e.g., skip entries with missing DeviceInfo/Product) before dereferencing these fields.
</issue_to_address>

---

Sourcery is free for open source - if you like our reviews please consider sharing them ✨

• X
• Mastodon
• LinkedIn
• Facebook

_{Help me be more useful! Please click 👍 or 👎 on each comment and I'll use the feedback to improve your reviews.}

[gemini-code-assist]

Code Review

This pull request introduces Vulkan GPU detection for Windows amd64 systems as a fallback when CUDA is not available. It includes logic to identify Vulkan-capable hardware and verify the presence of the Vulkan runtime library. A critical review comment identifies potential nil pointer dereferences in the hardware discovery logic, recommending defensive checks to ensure stability.

[ericcurtin]

@darthcav go may not be your coding language but this PR is perfectly clean :) The important thing is that you tested and confirmed this worked. Did you test it?

[darthcav]

@ericcurtin The problem is that I cannot do a full compilation in my laptop and check it against the local GPU. I would need someone to compile that...

[ericcurtin]

@ericcurtin The problem is that I cannot do a full compilation in my laptop and check it against the local GPU. I would need someone to compile that...

Well you could also use the coding agent you used for this to figure out how to build it.

I don't have an Intel Arc machine, so even if I wanted to, I can't test this.

[YasharthPanwar-2003]

Hi @ericcurtin and @darthcav !

I have a Windows machine running an Intel i7 processor with Intel Xe Graphics . Can I help you test it out! . Let me know if my hardware is applicable and what the best way is for me to run or compile it to help you verify the fix.

[ericcurtin]

@YasharthPanwar-2003 sure that would be helpful, thanks

[ericcurtin]

@darthcav could you rebase in the meantime to get this green?

[darthcav]

@ericcurtin I will look into that tomorrow.

[darthcav]

@ericcurtin Rebase done!

[darthcav]

@ericcurtin I managed to compile the dmr in Windows, and it detects correctly the Intel GPU.

I also created a small snippet to compile and test Vulkan-compatible GPUs (see below).

It would be nice if this can be accepted and integrated into the next version of Docker Desktop.

---

Testing GPU availability

The following snippet should verify that the PR works locally in Windows. Just compile:

// Command gpu-probe reports GPU detection results for the model-runner backend.
// Run on Windows to verify CUDA, Vulkan, or OpenCL detection before deploying dmr.
package main

import (
"context"
"flag"
"fmt"
"os"

	"github.com/docker/model-runner/pkg/inference/backends/llamacpp"
	"github.com/jaypipes/ghw"
)

func main() {
nvBin := flag.String("nv-gpu-info", "", "path to com.docker.nv-gpu-info.exe (only needed for NVIDIA GPUs)")
flag.Parse()

	fmt.Println("=== GPU probe ===")

	gpus, err := ghw.GPU()
	if err != nil {
		fmt.Fprintf(os.Stderr, "warning: could not enumerate GPUs: %v\n", err)
	} else {
		fmt.Printf("Graphics cards found: %d\n", len(gpus.GraphicsCards))
		for i, card := range gpus.GraphicsCards {
			if card.DeviceInfo == nil {
				fmt.Printf("  [%d] (no device info)\n", i)
				continue
			}
			vendor := "(unknown vendor)"
			if card.DeviceInfo.Vendor != nil {
				vendor = card.DeviceInfo.Vendor.Name
			}
			product := "(unknown product)"
			if card.DeviceInfo.Product != nil {
				product = card.DeviceInfo.Product.Name
			}
			fmt.Printf("  [%d] %s – %s\n", i, vendor, product)
		}
	}

	fmt.Println()
	ok, err := llamacpp.CanUseGPU(context.Background(), *nvBin)
	if err != nil {
		fmt.Fprintf(os.Stderr, "CanUseGPU error: %v\n", err)
		os.Exit(1)
	}
	if ok {
		fmt.Println("Result: GPU acceleration available")
	} else {
		fmt.Println("Result: no supported GPU found — will use CPU")
	}
}

And then run:

.\gpu-probe.exe

You should get something like:

=== GPU probe ===
Graphics cards found: 1
[0] Intel Corporation – Intel(R) Arc(TM) Graphics

Result: GPU acceleration available

[darthcav]

Hi @ericcurtin and @darthcav !

I have a Windows machine running an Intel i7 processor with Intel Xe Graphics . Can I help you test it out! . Let me know if my hardware is applicable and what the best way is for me to run or compile it to help you verify the fix.

See my comment above.