The GPU Landscape: From Data Centers to Desktop AI

AMD kicked off Computex season with a bang, announcing the Ryzen AI Max 400 Gorgon Halo refresh alongside the Ryzen AI Halo workstation starting at $3,999. The headline number is 192 GB of unified memory — enough to run a 300B+ parameter LLM entirely on-device. The flagship Ryzen AI Max+ Pro 495 packs 16 Zen 5 cores, 40 RDNA 3.5 compute units (branded Radeon 8065S), and a 5.2 GHz boost clock. AMD frames this around the 'token economy,' claiming that at six million tokens per day, the Halo box pays for itself in six months versus cloud API usage. The key architectural advantage is unified memory — the CPU and GPU share the same pool of LPDDR5X memory, eliminating the VRAM bottleneck that forces traditional workstations to use expensive HBM memory. This makes the Halo uniquely suited for large-context-window agentic AI workloads where memory capacity matters more than raw compute throughput.

Nvidia, meanwhile, is shifting up the stack. Its CES 2026 keynote conspicuously skipped consumer GPUs, instead detailing the Vera Rubin platform and the new CPX chip, a bandwidth-optimized companion to HBM-equipped Rubin GPUs. The CPX uses cheaper GDDR7 memory for the memory-bandwidth-bound token generation phase of inference, while Rubin handles the compute-heavy prefill stage — a disaggregated architecture that directly tackles the two-phase nature of LLM inference. On the desktop side, Nvidia's DGX Spark, built around the GB10 chip and priced at $4,700 with 128 GB of unified memory, remains the primary competitor to AMD's Halo box. Independent benchmarks show the DGX Spark consistently ahead in prompt processing and image generation (roughly 4x faster on Flux.2 via ComfyUI), while the Ryzen AI Max+ 395 fights back in memory-bandwidth-sensitive token generation at short context lengths. The Spark is locked to Linux; AMD offers Windows and Linux support.

GPU benchmarking for AI has matured significantly this year. Reputable outlets now test two distinct phases of LLM inference: prompt processing (prefill) and token generation (decoding), across multiple context lengths. Dense models like Llama 3.1 8B and Gemma 3 27B are tested alongside mixture-of-experts architectures like GPT-OSS 120B and Qwen3-30B-A3B, since MoE models activate only a fraction of their parameters per token but still demand enormous memory pools. Key metrics include tokens per second, prompt processing latency, and power efficiency. On the image and video side, ComfyUI workflows with models like Flux.2 and LTX-2 are becoming standard benchmarks. MLPerf Client 1.0, released in August 2025, is also standardizing local AI benchmarking with a GUI and broader hardware support. When reading a review today, look for tests that separate prefill from decode, vary context length, and test on both Linux and Windows.

The PCIe slot is having a renaissance. AMD's MI350P delivers 144 GB of HBM3E in a standard PCIe form factor with roughly 40% higher FP16 and FP8 theoretical throughput than Nvidia's H200 NVL. Intel's Crescent Island inference GPU uses Xe3P architecture with 160 GB of LPDDR5X memory. Meanwhile, a budding secondhand market for FPGA-based PCIe cards is opening up as hyperscalers refresh their inference infrastructure. Older Xilinx Alveo and Intel Stratix cards, originally deployed for video transcoding and network processing, are being repurposed for low-latency AI inference workloads. A modded Tesla V100 SMX, hacked onto a custom PCIe PCB with 3D-printed cooling, recently demonstrated better inference efficiency than midrange modern GPUs for just $200. Startups like MemryX are shipping $149 M.2 AI accelerator modules delivering 24 TOPS, and Phison's aiDAPTIV+ stack enables consumer PCs to run MoE models three times larger than their VRAM would otherwise allow. If you are willing to tinker, the PCIe ecosystem in 2026 offers an unprecedented range of cost-performance tradeoffs.

Match your hardware to your bottleneck. If you are doing heavy prompt processing or image generation (compute-bound), Nvidia's DGX Spark or a used RTX 4090 still leads on raw CUDA throughput. If you need large context windows for agentic workflows (memory-bandwidth-bound), AMD's unified memory architecture is uniquely compelling — 128 GB to 192 GB of VRAM-addressable memory without the cost of HBM. For Mixture-of-Experts models, capacity matters more than peak FLOPs, making the Ryzen AI Halo or a multi-GPU PCIe setup the better choice. Consider software maturity: AMD's ROCm has improved dramatically but still trails CUDA in stability. If you are on a budget, watch the secondhand FPGA market and the modded Tesla V100 route. Never buy without checking real inference benchmarks at your target context length; synthetic gaming scores are irrelevant for LLM workloads. One local Halo box at $3,999 can save $750 per month in cloud API costs — the breakeven math for going local has never been better.