Gadget Re Build
gadget eGPU Expansion — Dual-GPU AI Node
Overview
gadget is being expanded from a single-GPU inference node into a dual-GPU AI node by attaching an external desktop GPU over OCuLink. The build reuses hardware already on hand. The only remaining purchase is the second GPU.
Build
| Component | Detail |
|---|---|
| Host | Minisforum G1 Pro — Ryzen 9 8945HX (16C/32T), 32GB DDR5, 1TB NVMe |
| GPU 1 (internal) | NVIDIA RTX 5060 8GB GDDR7, PCIe 4.0 |
| GPU 2 (external) | TBD (target: RTX 3090 24GB or RTX 5060 Ti 16GB) |
| eGPU dock | Minisforum DEG2 (OCuLink PCIe 4.0 x4 / TB5) |
| Link | NFHK M.2 2280 PCIe adapter in the spare M.2 slot → OCuLink to DEG2 |
| eGPU power | ASRock Pro-750G 80+ Gold 750W, dedicated to the dock and GPU |
| OS | Windows 11 for validation; Ubuntu planned for production |
Benefits
- Larger models. VRAM pools across both GPUs through layer splitting (LM Studio / llama.cpp). The node goes from an 8GB ceiling to 24–32GB, enough for 24B–32B class models that can’t run on either card alone.
- Parallel roles. Each GPU can run its own model at full speed at the same time. For example, a main reasoning or writing model runs on the eGPU while vision or embeddings run on the 5060, with no contention.
- Fits the existing stack. Both endpoints register in the LiteLLM router as separate models. This expands the local AI “team” without changing how clients connect.
- Data stays local. Larger, more capable models run on-prem, so fewer tasks need cloud AI. That means less exposure of personal data and no third-party data handling.
- Modular and upgradable. The eGPU card can be swapped or upgraded independently, and the dock can move to another host if roles change.
Cost Savings
- Reuses existing hardware. The host, dock, adapter and PSU are already owned. The only new spend is one GPU.
- Avoids a new workstation. A comparable dual-GPU tower would mean buying a new CPU, motherboard, RAM, case, storage and PSU. This build skips all of that.
- Lower cloud spend. Work that would otherwise need paid API calls runs locally at the cost of electricity.
- Efficient power use. The mini PC platform idles low, and the eGPU can be powered down when it isn’t needed.
- Cheaper upgrades. Future capacity comes from a GPU swap, not a platform rebuild.
How Dual-GPU Enables Bigger Models
Local LLM size is limited by VRAM, not compute. A model has to fit in GPU memory to run at usable speed, and anything that spills into system RAM slows down sharply.
| Configuration | Usable VRAM | Practical model class (Q4) |
|---|---|---|
| RTX 5060 alone | 8GB | ~7–8B |
| 5060 + 16GB card | 24GB | ~14–24B |
| 5060 + 24GB card | 32GB | ~24–32B, or smaller models with long context |
Layer splitting places part of the model on each GPU, so both cards’ memory acts as one pool. Only small activation data crosses the OCuLink link per token, so the x4 connection has minimal impact on generation speed. The tradeoff is that a split model runs at roughly the pace of the slower card. The best practice is to split only when a model is too large for the bigger GPU, and to run separate models per GPU the rest of the time.
Status / Next Steps
- Confirm the NFHK adapter connector (SFF-8612 OCuLink vs SFF-8643).
- Install the adapter, connect the DEG2 and validate the link with a spare GPU.
- Select and purchase GPU 2.
- Test split mode and per-GPU mode; register endpoints in LiteLLM.
- Migrate gadget to Ubuntu once the hardware is proven stable.