After an inspiring podcast mention, I set up Start9 OS v0.4.0 on the NVIDIA DGX Spark (Gigabyte AI TOP ATOM GB10, ARM64, Blackwell GPU, 128 GB RAM) and want to share my experience as a heads-up for others considering this route.
Setup:
Hardware: Gigabyte AI TOP ATOM DGX Spark (GB10, ARM64, sm_121)
OS: Start9 OS v0.4.0 (fresh install)
Services: Ollama (Start9 package), Open WebUI
Observation:
Ollama does not detect the integrated Blackwell GPU at all. Instead, inference runs entirely on the ARM CPU.
An 8B model (Hermes 3) takes minutes instead of seconds for simple prompts.
Larger models (Qwen 3.6 35B) are practically unusable for chat.
CPU runs constantly at 95% while the GPU sits idle.
Assessment:
The LXC container architecture of StartOS v0.4.0 currently seems to lack GPU passthrough support for the NVIDIA Blackwell architecture (sm_121) on the GB10. This is not a criticism — Start9 OS v0.4.0 is a fantastic release and the ARM64 support itself runs stably. But for anyone buying the DGX Spark primarily as a dedicated AI inference machine for Start9: you are currently running a very powerful, very power-hungry ARM64 server with 128 GB RAM whose GPU is not addressable for LLMs.
Question for Start9 / Community:
Is there a roadmap or timeline for CUDA/Vulkan GPU acceleration within LXC services for ARM64/Blackwell? Or a preferred workaround?
Thanks for the great project — I am staying tuned and looking forward to updates!
The diagnosis isn’t quite right, in a way that works in your favor. StartOS 0.4.0’s LXC services do support NVIDIA GPU acceleration. What’s missing is on the host side: the standard aarch64 image is the FOSS-only build and contains no NVIDIA driver at all, so there are no /dev/nvidia* devices to hand to the container — which is exactly why Ollama logs total_vram="0 B" and falls back to the CPU backend.
StartOS publishes three flavors per architecture that you probably missed, and the one you want is aarch64-nvidia. It bundles the official NVIDIA driver (580.159.03, the same 580 branch DGX OS ships for the Spark) plus the NVIDIA container toolkit. On that flavor, StartOS injects the driver userspace into any service container that requests it and passes through the GPU device nodes — and the current Ollama package already requests both on its aarch64 build, so no package-side change is needed. Upstream Ollama has supported the GB10 (sm_121, CUDA 13) since launch day. If Secure Boot is enabled, the setup wizard will walk you through enrolling the StartOS signing key (a blue MokManager screen on reboot) so the signed NVIDIA modules can load.
Suggested path: take a full backup, flash the -nvidia ISO, and keep your existing data drive when the installer offers. After boot, nvidia-smi over SSH on the host should show the GB10; then Ollama’s startup log should report a CUDA library instead of library=cpu.
It’s also not just Ollama anymore. A llama.cpp package shipped a few weeks ago with dedicated CPU/CUDA/ROCm/Vulkan variants — including a native ARM64 CUDA build — and StartOS automatically installs the variant matching your hardware, so on an aarch64-nvidia install your Spark would get the CUDA build without you choosing anything (Open WebUI can point at its OpenAI-compatible API too). And a vLLM package is in release-candidate stage that explicitly targets Blackwell (sm_120/121) with NVFP4 quantization on ARM64. So far from being unsupported, hardware like yours is exactly where things are headed.
BUT: I don’t know that anyone has tested StartOS on a DGX Spark yet — the GB10 needs the open kernel modules (the 580 installer’s default) and a recent kernel, and you may well be the first to exercise this combination. If the GPU still doesn’t show up on the nvidia image, that’s a genuinely valuable bug report: please file it at Issues · Start9Labs/start-technologies · GitHub with the host nvidia-smi output and Ollama log, and link this thread.
If you get it working, please report back with details like tokens/sec — a 128 GB unified-memory Spark running local inference on StartOS would be a great data point for the community.
Actually, the aarch64-nvidia ISO apart, and as currently shipped it won’t fix your GB10 after all. The NVIDIA kernel module inside is the proprietary flavor (modinfo license “NVIDIA”), and per NVIDIA’s own docs, Blackwell GPUs are only supported by the open kernel modules. So on a DGX Spark the driver won’t bind and you’d land right back at CPU-only, just with a bigger ISO. You could try and confirm this though.
I think everything else checks out — the bundled driver version (580.159.03) explicitly supports the GB10, the kernel is new enough, and the container GPU plumbing is in place. The root cause is a quirk of NVIDIA’s installer when run in a GPU-less build environment (it silently defaults to proprietary), and the fix is a single flag in the image build. I guess we can create an issue.
thank you very much for the detailed technical analysis and the clarification regarding the driver flavors. That was incredibly helpful.
I can confirm that I was indeed running the correct image: I flashed startos-0.4.0-beta.9-d1f153d_aarch64-nvidia.iso (see attached screenshot). So the behavior I reported — Ollama falling back to library=cpu, total_vram="0 B", and the GPU not being detected — occurred with the official NVIDIA-flavored build.
Your explanation regarding the proprietary vs. open kernel modules makes perfect sense and completely explains why the driver won’t bind on the DGX Spark / GB10.
Definitive answer, confirmed by Start9’s CTO who runs StartOS on this exact hardware: StartOS beta.9 aarch64-nvidia on a DGX Spark works, GPU acceleration functional. So:
Take a full backup.
Flash the aarch64-nvidia ISO (published alongside the standard image for every release) and keep your existing data drive when the installer offers.
Don’t skip the MOK step. If Secure Boot is enabled, the setup wizard walks you through enrolling the StartOS signing key — a blue MokManager screen appears on reboot and you have about 10 seconds to choose “Enroll MOK” (it reappears on every boot until enrolled). Without it the NVIDIA module can’t load and you’d see exactly the CPU-only behavior you reported.
Verify with nvidia-smi over SSH on the host; Ollama’s startup log should then report a CUDA backend instead of library=cpu.
Would love to see tokens/sec once it’s running — and let us know how it goes.
Thanks for your message yesterday — you were absolutely right!
It was indeed the missing MOK enrollment. I reflashed the aarch64-nvidia ISO, paid close attention to the blue MokManager screen this time, and enrolled the key properly.
GPU is now fully recognized and functional:
NVRM version: NVIDIA UNIX aarch64 Kernel Module 580.159.03 Fri Apr 24 06:47:43 UTC 2026
GCC version: gcc version 14.2.0 (Debian 14.2.0-19)
nvidia-smi shows the GB10 with Driver 580.159.03 and CUDA 13.0.
Thank you and the team for the excellent support and for building a system that actually works with Blackwell hardware!
