A comprehensive test of NVLink bridges using dual RTX 3090 GPUs reveals the interconnect technology's value depends heavily on your underlying hardware platform.

The testing, conducted on a server-grade AMD Epyc 7532 system with full PCIe Gen4 x16 bandwidth per GPU slot, found NVLink provided only marginal improvements for most AI workloads. Tensor parallel inference saw 30% faster prompt processing, while token generation speeds remained identical. FSDP training showed dramatic 3x improvements, but only when compared against shared host memory transfers.

The picture changed dramatically when the author discovered Nvidia artificially disables peer-to-peer PCIe communication on consumer GPUs through driver restrictions. Installing patched open-source drivers to enable proper P2P transfers nearly eliminated NVLink's advantages on the server platform.

"Enabling P2P almost entirely makes up the difference between SHM and NVLink," the author noted. Prompt processing speeds became virtually identical, and FSDP training reached over 90% of NVLink performance.

Consumer Hardware Changes the Equation

The value proposition shifts significantly on consumer platforms. Even premium motherboards like the $1,300 MSI MEG X870E GODLIKE and $1,000 Gigabyte Z890 AORUS XTREME AI TOP only support one full x16 PCIe slot. Installing a second GPU forces both cards into x8/x8 bandwidth sharing.

Testing at reduced PCIe bandwidths revealed NVLink's true benefit. At x8 configuration, P2P performance dropped 5% for tensor parallel inference and 15% at x4. NVLink maintained consistent performance across all configurations, completely bypassing PCIe limitations.

For FSDP training workloads, the bandwidth restrictions proved more severe. Performance degraded substantially at x8 and became nearly unusable at x4 bandwidth, while NVLink maintained full speed.

The findings suggest NVLink bridges, currently priced at $200-400, offer minimal value for users with server-grade hardware and full PCIe bandwidth. However, they become essential for consumer setups where PCIe lane limitations create bottlenecks for multi-GPU AI workloads.

The testing used models including Llama 3.3 70B, Gemma 4 31B, TinyLlama 1.1B, and Qwen2.5 3B across various parallelization strategies to measure real-world AI performance impacts.