AI network fabrics are becoming part of the compute engine

Distributed AI has always depended on communication, but the dependency is changing in kind. Earlier clusters could often treat networking as a fast transport service underneath compute nodes. Frontier training, large mixture-of-experts inference and disaggregated serving make the fabric part of the algorithmic critical path. Tokens, activations, gradients and cache state move because the model graph requires them to move, and a delayed exchange can leave many expensive accelerators idle at once.

That is why modern AI-network designs are adding functions that look less like conventional packet forwarding and more like runtime machinery: collective offload, memory-aware transport, adaptive path selection, rapid link recovery, congestion control tuned for synchronized traffic and software interfaces that expose fabric state to schedulers.

The practical consequence is that a switch upgrade can change model throughput, batch economics and failure behavior even when the GPUs and model weights remain identical. Infrastructure teams therefore need to qualify the network with the workload in the same way they qualify kernels, quantization or serving policy.

The phrase “AI network” hides several distinct domains. Scale-up connects accelerators that must behave like one large compute resource. Tensor parallelism and expert parallelism can require fine-grained, latency-sensitive exchanges inside this domain, so memory semantics, ordering and all-to-all behavior matter as much as headline bandwidth.

Scale-out connects racks or pods into a larger cluster. Here the design must sustain large east-west flows, collective synchronization and many concurrent jobs while preventing congestion in one tenant or rail from creating long tails elsewhere. Ultra Ethernet and Spectrum-X are examples of efforts focused on this data-center-scale problem, although their implementation models differ.

Scale-across extends work between facilities. Propagation delay, routing domains, failure isolation and operational ownership become dominant. A fabric that is excellent inside one rack is not automatically appropriate between campuses. The emerging architecture is therefore hierarchical rather than one universal interconnect used everywhere.

UALink is a scale-up interconnect for accelerators and switches. Its 200G 1.0 specification targets up to 1,024 accelerators in an AI computing pod, while the newer Common 2.0 specification adds In-Network Compute and separates physical-layer evolution from higher-level semantics. Its manageability specification also treats the fabric as a centrally controlled system rather than a collection of independent links.

OCP ESUN focuses on using Ethernet for the scale-up network itself. The first 1.0 operator specification specifies lossless behavior, link-level retry, congestion management and a compact four-byte header intended to reduce overhead for the small messages common in tightly coupled accelerator traffic. ESUN deliberately separates switch/network requirements from endpoint transport work.

Ultra Ethernet addresses a broader high-performance Ethernet communication stack for AI and HPC, including modern RDMA, transport, congestion control, interoperability and operations. Version 1.0.3 became the current published specification on July 16, 2026. These projects can complement one another, but collapsing them into one “open Ethernet standard” obscures where interoperability actually exists.

Collective operations are synchronization points. In all-reduce, for example, many accelerators contribute partial values and each eventually needs the reduced result. If every byte must traverse endpoints repeatedly, accelerators and links spend time moving and combining data that has a known communication pattern.

In-network compute moves some of that work into the fabric. UALink Common 2.0 explicitly introduces the capability. NVIDIA’s NVLink 6 switch path exposes SHARP-based collective offload and the company reports 130 TFLOPS of aggregate in-network compute inside a Vera Rubin NVL72 rack. The exact operations and numerical semantics vary by implementation, but the systems idea is common: combine data while it is already crossing the network so fewer bytes and fewer endpoint cycles are needed.

This is not free acceleration. Offloaded reductions must preserve required precision, ordering and failure semantics; collective libraries need to know when the operation is safe; and debugging becomes harder when computation is distributed across accelerators and switch silicon. The performance benefit therefore needs to be evaluated together with portability and incident-response cost.

MoE models expose why scale-up bandwidth cannot be judged by a single peak number. A router selects experts for tokens, those tokens must be dispatched to accelerators holding the selected experts, the experts compute their outputs, and results must be gathered and reordered. At large batch sizes this creates sustained all-to-all traffic rather than a simple bulk transfer between two endpoints.

A slow or imbalanced path can erase the arithmetic efficiency that sparse activation was supposed to provide. NVIDIA’s July NVLink analysis uses this workload to argue for tightly coupled scale-up fabrics and reports up to 2.3 times higher simulated decode throughput than an off-the-shelf Ethernet comparison in selected 72-accelerator scenarios. That figure is vendor simulation, not an independent universal benchmark, but the underlying communication pattern is real and can be measured directly.

The engineering lesson is to benchmark the actual expert topology, token distribution, batch regime and collective implementation. A network that wins an all-reduce microbenchmark can still lose on MoE all-to-all under realistic skew.

Synchronized AI traffic creates incast, hot paths and correlated bursts that ordinary enterprise traffic rarely produces at the same scale. Static equal-cost routing can leave capacity idle while one path queues, and one congested collective can lengthen the step time of an entire distributed job.

New AI fabrics react faster. The Spectrum-X research paper describes a multiplane architecture and hardware-accelerated load balancing intended to respond to changing network conditions at microsecond timescales. AMD emphasizes programmable NIC and DPU logic so transport and optimization policy can evolve in software. UEC defines congestion-management behavior as part of the communication architecture rather than an implementation afterthought.

This creates a new release surface. Routing policy, NIC firmware, congestion thresholds and collective scheduling can materially alter model performance. Operators should version those settings with the serving or training stack and reproduce benchmark results after changes.

A 102.4-terabit switch is impressive only if the cluster converts that capacity into useful accelerator progress. Tail latency, retransmissions, failed links, topology imbalance, collective serialization and maintenance events can all reduce effective output while leaving many physical counters healthy.

This is why vendors increasingly frame networking in tokens per watt, job-completion time and sustained utilization. Those claims need independent reproduction, but the metric shift is correct. A fabric that is five percent slower in a clean benchmark but recovers gracefully from link faults may produce more useful work over a long training run than a faster fabric with brittle failure behavior.

Capacity planning should therefore include the cost of stranded accelerator time. When thousands of GPUs wait on one communication phase, a few milliseconds of tail latency can represent far more money than the switch power saved by a cheaper design.

Electrical signaling becomes harder as lane rates and reach rise. Retimers, pluggable optical modules and high-speed SerDes consume power and board area, while every additional component becomes another failure point. The network power budget therefore grows at the same time operators want to allocate more facility power to accelerators.

NVIDIA Spectrum-6 supports both pluggable and co-packaged optics, and Broadcom has moved 102.4-terabit switching into production while also pursuing co-packaged optical variants. The attraction is not merely higher port speed. Bringing optics closer to the switch ASIC can reduce electrical reach and improve bandwidth density and power efficiency.

The trade-off is serviceability and supply-chain complexity. A failed pluggable module is simple to replace; a tightly integrated optical engine changes the maintenance model. Production adoption will depend on whether packaging yield, field replacement, thermal design and optical component reliability improve quickly enough to offset those operational costs.

Open scale-up efforts are motivated partly by procurement and interoperability. A common fabric can let operators combine accelerators, switches and NICs from different suppliers, reduce dependency on one platform roadmap and reuse operational tooling. UALink, ESUN and UEC all make interoperability an explicit goal.

The competing argument is that AI performance depends on details across silicon, collective libraries, topology, firmware and schedulers, so vertically integrated systems can optimize the whole path more aggressively. NVIDIA presents NVLink and Spectrum-X in exactly these terms, with hardware and software developed as one platform.

Neither model wins automatically. Open standards need mature implementations, compliance testing and predictable semantics before theoretical vendor choice becomes practical portability. Proprietary co-design must justify lock-in with measurable workload goodput and lifecycle reliability. Buyers should demand workload-level evidence from both.

The standards are now moving quickly enough that implementation quality becomes the harder question. UEC 1.0.3 is current, UALink has added in-network compute and manageability, ESUN has a first operator specification, and multiple 102.4-terabit switch families are shipping or entering production systems. The next stage is not specification publication; it is multi-vendor deployment evidence.

Watch for public interoperability events, independent collective benchmarks, long-duration failure tests and software support that makes fabric features visible to common training and inference runtimes. In-network compute needs especially careful validation because performance gains are meaningful only when numerical behavior and recovery semantics remain predictable.

If those pieces mature, the network will be treated less like data-center plumbing and more like an accelerator subsystem with its own compute, memory semantics and runtime APIs. If interoperability lags, operators may continue buying vertically integrated fabrics even while open specifications look attractive on paper.