Hi, StuPleb,
I’d like to answer your last question (better late than never):
“Can you tell us anything about performance?”.
The NVIDIA GB10 GPU is detected but not utilized. The logs show:
skipping CUDA device — compute capability not in compiled architectures
device=“NVIDIA GB10” cc=1210
System Details
library=CUDA compute=12.1 name=CUDA0
description=“NVIDIA GB10”
total=“121.7 GiB” available=“118.3 GiB”
- GPU: NVIDIA GB10 (Blackwell, compute capability 1210)
- VRAM: 121.7 GiB total, 118.3 GiB available
- CUDA driver: 13.0
- OLLAMA_VULKAN: true (no CUDA backend present)
Performance Notes
GPU utilization: Not available (compute capability 1210 not in compiled architectures)
- ollama-logs.html
Baseline: CPU-only inference (estimated 20 tokens/sec)
Context size: Default 262,144 tokens based on VRAM
Model Info (Llama)
- Architecture: llama
- Embedding dimension: 4096
- Layers: 32
- Heads: 32
- Key-value heads: 8
- Rotary embedding size: 128
- Training context size: 131072
- Token cache: special tokens = 256; token to piece = 0.7999 MB
Links:
Original thread with GPU detection issue:
(closed)
Note:
This appears to be an Ollama-specific limitation. Start9OS currently offers Ollama, llama.cpp, vLLM, and Open WebUI.
On this NVIDIA GB10 hardware with CUDA driver 13.0 and compute capability 1210, the following applications can utilize GPU acceleration:
CUDA-based AI/ML Applications
PyTorch with CUDA for deep learning and model execution
TensorFlow with CUDA for ML workloads and models
CUDA-based compiler nvcc for CUDA programs (when available)
cuDNN and other CUDA tools (if installed via Start9 or as ARM64 ports)
Vulkan-based Applications
Game Engines: Unity (with Vulkan), Godot, Unreal (via Vulkan plugins)
Vulkan-based rendering/compute applications:
vulkan-tools and vulkan-validation-layers
Vulkan-based AI tools such as GPU computer software
once again:
the logs confirm that Ollama skips the GPU due to compute capability 1210 not being listed in the compiled architectures, while the system correctly detects the NVIDIA GB10 with driver 13.0 and CUDA support.