Gadget Upgrade (Rebuild Continued)
gadget eGPU Expansion — Dual-GPU AI Node
Update: The second GPU has been chosen and purchased. I picked up an NVIDIA Quadro RTX 6000 (24GB) for $1,425, and it’s due to arrive Wednesday. Below is why I chose this card, what it adds, and how it changes the way my home AI stack is organized.
Overview
gadget is moving from a single-GPU inference node to a dual-GPU AI node, with an external workstation GPU connected over OCuLink. Everything except the new card was already on hand, so the RTX 6000 is the only new purchase in this build.
Build
| Component | Detail |
|---|---|
| Host | Minisforum G1 Pro — Ryzen 9 8945HX (16C/32T), 32GB DDR5, 1TB NVMe |
| GPU 1 (internal) | NVIDIA RTX 5060 8GB GDDR7 |
| GPU 2 (external) | NVIDIA Quadro RTX 6000 24GB GDDR6 (purchased, $1,425) |
| eGPU dock | Minisforum DEG2 (OCuLink PCIe 4.0 x4 / TB5) |
| Link | NFHK M.2 2280 PCIe adapter in the spare M.2 slot → OCuLink to the DEG2 |
| eGPU power | ASRock Pro-750G 80+ Gold 750W, dedicated to the dock and GPU |
| OS | Windows 11 for validation; Ubuntu planned for production |
Why the Quadro RTX 6000?
Running AI models locally comes down to VRAM. A model has to fit in GPU memory to run at a usable speed, and memory is the scarce resource. I looked at several options before settling on this card:
- RTX 3090 (24GB): the usual budget favorite, but used prices have climbed past what it’s worth to me.
- Tesla V100 (32GB): very cheap for the memory, but it’s a passive server card that needs a cooling mod and an unusual power adapter. Its driver support is also ending, which would lock the whole machine to an aging driver branch.
- RTX 5060 Ti (16GB): a good card, but currently selling well above MSRP, and it has less memory.
The Quadro RTX 6000 was the best fit for this build:
- 24GB of VRAM. Three times the 5060’s 8GB, and enough to run 24B-class models entirely on one card.
- Built for 24/7 work. It’s a workstation card with a blower-style fan that pushes heat out the back, so it suits an always-on machine and needs no cooling mods.
- Standard power. It uses ordinary PCIe 8-pin + 6-pin connectors at about 295W, well within the 750W supply.
- Current driver support. It’s a Turing-generation card, so it stays on NVIDIA’s current drivers and CUDA releases, unlike older Tesla cards that have been moved to legacy support.
- Solid memory bandwidth. About 672 GB/s, roughly the same as the RTX 5070 in my main AI node, so generation speed holds up.
What It Brings: Bigger Models
The RTX 6000 combines with the 5060 for 32GB of total VRAM on one machine. Software like LM Studio and llama.cpp can split a model’s layers across both cards, so they work as one memory pool.
| Configuration | Usable VRAM | Practical model size (Q4) |
|---|---|---|
| RTX 5060 alone (before) | 8GB | ~7–8B |
| Quadro RTX 6000 alone | 24GB | ~24B, or 20B with long context |
| Both GPUs combined | 32GB | ~32B |
Moving from 7–8B models to the 24–32B range makes a real difference. Larger models reason better, follow complex instructions more reliably, write with more nuance and handle long documents with fewer mistakes.
The tradeoff: a model split across both cards runs closer to the speed of the slower card. So I’ll split only when a model is too large for the RTX 6000 alone, and run separate models on each card the rest of the time.
How This Changes My AI Stack
Before this upgrade, my main AI node did nearly all the work, and model size was capped by its 12GB card. With gadget upgraded, the stack splits into two nodes with distinct jobs:
| Node | Role | Strength |
|---|---|---|
| dexter (RTX 5070 12GB) | The fast node | Quick, everyday questions, chat, and my phone-based assistant. Optimized for speed. |
| gadget (RTX 6000 + RTX 5060, 32GB) | The heavy node | Larger models for deep analysis, research reports, long documents and creative writing, with vision and smaller helper models on the 5060. |
Both nodes sit behind a single AI router, so my tools and apps connect the same way they always have. I choose the model by task: fast for quick answers, deep when quality matters most.
Everything still runs on hardware in my lab. Larger, more capable local models mean even fewer tasks need a cloud AI service, so my data stays private and nothing is sent to a third party.
Cost
- New spend: $1,425 for the Quadro RTX 6000. The host, eGPU dock, M.2 adapter and power supply were already on hand.
- Avoided spend: A comparable dual-GPU tower would have meant buying a new CPU, motherboard, RAM, case, storage and power supply. Reusing the mini PC skips all of that.
- Ongoing savings: Work that would otherwise need paid cloud AI runs locally for the cost of electricity.
- Future upgrades: The external GPU can be swapped later without rebuilding anything else.
What’s Next
- Confirm the M.2 adapter’s connector type, install it and route the OCuLink cable.
- When the card arrives Wednesday, install it in the DEG2 and validate that both GPUs are detected and stable under load.
- Test both modes: one large model split across both cards, and separate models on each card.
- Add gadget’s new models to the AI router alongside dexter.
- Move gadget to Ubuntu once the hardware is proven stable.
I’ll post benchmarks and real-world results once it’s up and running.