Dev agency tests on-prem AI assistant on Framework Desktop (Ollama + Vulkan)
Kalmia, a software company, runs LLMs on a Framework Desktop (Ryzen AI Max+ 395, 128GB) and found Ollama with the Vulkan backend the most stable option; ROCm and vLLM were not yet reliable on this APU.
2-10 people11-50 peopleDocument search / chat with filesCustomer support$1k-3kSensitivity: High (client/confidential)
Hardware
| Item | Approx. price (as of 2026-10-16) |
|---|---|
| On sale −20% Minisforum MS-S1 MAX (Ryzen AI Max+ 395, 128GB, 2TB) (Minisforum Store, related current offer) | $3,879.00 |
| On sale −6% MSI GeForce RTX 5060 Ti 16GB Ventus 3X OC (Newegg, related current offer) | $579.99 |
| Framework Desktop, AMD Ryzen AI Max+ 395, Radeon 8060S, 128GB shared memory | ~€2,500 price range (as stated for Ryzen AI Max+ 395 hardware) |
| Framework Desktop DIY (Ryzen AI Max+ 395, 128GB) (Framework, related current price) | $3,449.00 |
Source prices are as stated by the source. Retailer offers: prices checked 2026-10-16, may change. Some links may be affiliate links; we may earn a commission at no extra cost to you. This never changes what we list.
Software stack
- Ollama (Vulkan backend)
- vLLM (tested, not viable on Vulkan)
- ROCm 7/7.1 (tested)
Models used
- Not stated in source
Good at
- Ollama + Vulkan: 'consistent inference without frequent breakage'
- Company says its AI assistant could realistically be deployed on this class of hardware for SMEs
Weak at
- ROCm instabilities; some Linux kernel updates broke ROCm
- vLLM has no Vulkan backend, so multi-user serving options are limited
Rough total cost
Source cites the ~€2,500 price range for Ryzen AI Max+ 395 hardware.
Getting started
- Allocate GPU memory in firmware (up to 50% of system memory per source).
- Install Ollama and use the Vulkan backend.
- Avoid relying on ROCm until kernel/ROCm fixes land.
Source & credit
Running LLMs on AMD hardware: Kalmia's hands-on experience with Framework Desktop, Vulkan and ROCm — Kalmia blog
Summarized from the source; quotes are the source author's words. Check the original before buying.