Direct-to-Chip Liquid Cooling AI Cluster Architectures Inspired by OCP Principles and Technologies

Rich Lappenbusch, Senior Principal - Super Micro Computer, Inc In this session, we will be revealing the blueprints of modern data center infrastructure supporting the training of Large Language Models. First, we will provide an overview of designing cluster-level architecture. Then, we will showcase an onsite deployment of one of the largest AI training clusters in the world. We explore why many modern data centers are starting to opt for liquid-cooling infrastructure. We will briefly cover liquid-cooling key components and provide an overview of liquid-cooling deployments. We highlight a few examples of AI power costs, ranging from real-time inferencing workloads to training LLM foundational models. Then, we will present a cost-benefit analysis of liquid versus air-cooled AI infrastructure.