EXECUTIVE SUMMARY:

• Legacy data centres were built for yesterday’s workloads – and retrofitting an existing data centre for AI is not an easy task. To unlock the full value of AI, the AI data centre infrastructure needs to evolve
• When committing to a new data centre build, AI-centric design must be considered, alongside high-density power and cooling, optimized architecture, and sustainability
• AI data centres are not just bigger or merely scaled up, but different by design – and hyperscalers are mapping these new blueprints out of necessity

 

AI infrastructure demands

AI is quickly becoming the dominant consumer of compute, and traditional infrastructure just can’t keep up. The industry is shifting to fundamentally new architectures and data centres that don’t adapt will be left behind as the industry transitions to radically new designs.

It is hard to overlook the infrastructure constraints of traditional data centres as AI workloads grow in complexity and scale. We’re entering a new era in data infrastructure, one that legacy data centres weren’t built to handle. To address the specific requirements of large-scale artificial intelligence, top hyperscalers such as AWS, Google, and Microsoft are spearheading the evolution by building a new class of data centre from the ground up: AI-native infrastructure.

Why can’t legacy facilities handle AI’s demands?

Legacy data centres were designed for general-purpose computing. They were built to account for predictable workloads, with moderate power usage and flexible hardware.

AI has different needs and many legacy data centres are unsuitable for the task due to the scope and intricacy of the criteria.

AI workloads are vastly more power intensive than traditional workloads. They require three to ten times as much electricity per rack, therefore merely adding extra GPUs to the same old racks is not an option. The extreme heat produced by GPUs and TPUs cannot be controlled using conventional air-cooling methods, so in meeting the requirements of contemporary AI, liquid cooling infrastructure such as direct-to-chip becomes necessary.

Unpredictable bursts of energy required for ultra-fast connections between thousands of nodes are necessary for AI training, and the requirement for densely populated, high-performance clusters conflicts with the sprawl of traditional data halls. Long cable runs and low-density racks can increase latency, reducing performance for large AI jobs.

Legacy data centres built for yesterday

Legacy data centres were built for yesterday’s workloads. AI isn’t just demanding more, it’s demanding different. Hyperscalers know it, and they’re not waiting around. The future of digital infrastructure is being redefined by the emergence of the purpose-built AI data centre era.

Retrofitting an existing data centre for AI isn’t easy. Typically, data centres will need to leverage their existing investments in air cooling while selectively deploying liquid cooling where needed. Although infrastructure can be reworked and redesigned, concessions will always need to be made. These compromises could come at the expense of performance and efficiency. Power availability (typically capped at the site level) and cooling capacity (particularly in raised-floor environments), while rack weight and floor loading, together with ceiling height restrictions that reduce airflow design, are physical constraints on most older sites. Add in the layout obstructions and interconnect distances that cause latency bottlenecks, and this could be a compromise too far.

What makes AI workloads different

Traditional data centres tend to average 5–20 kW per rack, whereas the power draw per rack due to AI workloads can be significantly higher than traditional compute, pushing 30–100 kW per rack or higher. Infrastructure needs to be approached very differently to support this degree of power density, as on-site substations, busways, and high-capacity PDUs are increasingly the standard rather than the exception.

AI workloads are inherently unforgiving of infrastructure failures. While traditional workloads can often handle transient faults or recover from minor slowdowns, AI training that is running for days or weeks require near-perfect uptime, clean compute environments, and dependable performance. Even small variations or inconsistencies in hardware, firmware, or thermal performance can be catastrophic. Not because it can’t recover, but because the cost of failure is so high, with every crash potentially hours (or days) of lost compute, wasted energy, and missed opportunity.

Designing for AI

To unlock the full value of AI, the AI data centre infrastructure must evolve. AI demands infrastructure that’s not just fault-tolerant, but fault-predictive and self-healing.

When embarking on a new data centre build, you must consider:

• High-density power and cooling: Custom power paths must be able to handle 80–100kW racks or more, while air cooling, the mainstay of legacy facilities is not enough. Advanced thermal strategies such as liquid cooling and direct-to-chip solutions must be integrated into the infrastructure of the AI data centre.
• Architecture: Physical GPU/TPU cluster layouts need to be optimized to ensure latency is minimized and training throughput maximized. Consolidated floorplans and thermal awareness allows for increased efficiency, faster deployment, and future expansion.
• AI-centric design: Real-time predictive failure monitoring and telemetry should be used on every component from temperature to power draw. Machine learning-based fault prediction isn’t optional anymore. It’s how downtime can be pre-empted, and uptime can be optimized.
• Sustainability: Carbon-neutral power resources, energy storage, recycling and reusing waste heat output and using alternative building materials can all assist with sustainability and environmentally friendly policies and strategies. Adherence to green strategies can not only improve the facility’s efficiency but can provide competitive advantage.

 

Legacy data centres were designed for flexible, general-purpose compute. However, AI clusters depend on ultra-low-latency interconnects between accelerators. That changes everything from the physical layout to how cable trays are built. New facilities need to be dense, compact, and often modular designed to reduce data movement friction.

Bigger and better

AI data centres aren’t just bigger — they’re different by design. Larger footprints are not a luxury but rather a need to accommodate the density and specialized layout, thermal management and performance characteristics AI environments.

Hyperscalers are starting to plan and construct data centres in pod-based, modular designs that are tailored for AI workloads and optimized for independent cooling, powering, and scaling. Workloads are not distributed equally throughout the data centre by AI clusters. Rather, concentrated compute pods (hundreds to thousands of GPUs in a tightly integrated fabric) are needed, calling for larger real estate to accommodate the GPU/TPU cages or liquid-cooled racks, zoning to isolate various workloads and effectively manage thermal loads, and a larger whitespace per cluster to accommodate power, cooling, and cabling routes.

Space is needed to manage the significant heat produced by high-density AI workloads, for heat exchange devices, immersion tanks, liquid cooling loops, and greater hot/cold aisle separations, often with isolated or enclosed cooling corridors.

Each pod needs short, direct power paths, larger substations, and dedicated power rooms. They also require extra room for redundant switchgear, transformers, and UPS systems, as well as increased floor loads and reinforced infrastructure to support denser, heavier racks.

A fundamental rethink

Hyperscalers aren’t building these new AI driven data centres because it’s trendy or because they want the biggest facility. It’s because it’s necessary. AI is not an experimental upgrade cycle. It’s fundamental infrastructure. And with any foundational shift in computing, it demands a matching evolution in physical and digital architecture.

Companies that continue trying to run next-generation AI on last-generation infrastructure will find themselves bottlenecked in performance, efficiency and ultimately competitiveness. The AI-native future is rapidly overshadowing the computing era for which legacy data centres were constructed.

For this reason, hyperscalers are designing data centres that embrace and give priority to specially designed AI infrastructure. They are not merely scaled up facilities. They are precision engineered, and offer the performance, resilience, and AI acceleration that will define the next decade.

Piece from Gordon Johnson, Senior CFD Manager at Subzero Engineering