Al's Impact on Data Center Networks

The rapid growth of AI is impacting large data centers as high volumes of traffic and intensive processing requirements are pushing the limits of hyperscale data centers.

To successfully support AI’s rapid growth, data center architectures and the high-speed networks they rely on, must be re-evaluated. AI application model complexity and size dictate the level of compute, memory and network type needed to connect AI accelerators (like GPUs) used for training and inferencing. At the same time, AI workloads are driving an unprecedented demand for low latency and high bandwidth connectivity between servers, storage, and accelerators.

The scale required for support doesn’t come from simply adding racks to a data center. Handling large AI training and inference workloads requires a separate, scalable, routable backend network infrastructure to connect distributed GPU nodes. AI apps have less impact on the frontend Ethernet networks that use general purpose servers to provide AI data ingestion for the training process.

The requirements for new backend network differ considerably from traditional data center frontend access networks. In addition to higher traffic and increased network bandwidth per accelerator, the backend network needs to support thousands of synchronized parallel jobs, as well as data and compute-intensive workloads. The network must be scalable, and provide low latency and high bandwidth connectivity between servers, storage, and the GPUs essential for AI training and inferencing.

Data center networks must transform to support new AI workloads