Executive Summary
- A single AI rack now demands 20 times more power than that of a traditional server and demands for energy are set to increase.
- Data centre customers need to look at how an infrastructure ‘under tremendous strain’ can be used more efficiently.
- Here, Sarah Beechey, Practice Leader for Data Centre & AI at SHI explains and explores Software Defined Data Centre (SDDC).
Data centres were built for predictable workloads but the demands of artificial intelligence (AI) are now putting them under tremendous strain. A single AI rack now demands 20 times more power than that of a traditional server and demands for energy are set to increase.
AI training and inference model use currently accounts for 25% of data energy consumption and is predicted to rise to 60% over the next three to five years, according to a recent report from Capgemini. That means data centre customers, be they enterprises, datacentre-as-a-service or cloud service providers, will need to look at how that infrastructure can be used more efficiently.
One way of doing that is virtualise server, storage and network utilisation to create a Software Defined Data Centre (SDDC). An SDDC uses software to manage configuration and dynamically shape infrastructure to match demand, providing greater agility and, as they can utilise a multi-tenant environment, they can allow the organisation to share costs.
SDDC and AI
From an AI perspective, an SDDC allows changes to be made to improve performance by optimising the computing resources, storage and networking without the need to make physical changes. Each of those data centre layers is treated as software, which streamlines resource use and allows adjustments to be made in near realtime. So that equates to greater flexibility but also less infrastructure needlessly consuming power or standing idle.
In fact, alongside demand for data centre virtualisation and the increased adoption of hyper-converged infrastructure, its AI and automation that are seeing renewed interest in SDDC.
It represents the ideal way to reduce the data centre footprint as well as improving energy efficiency and providing scalability, enabling organisations to modernise their data centres, improve resource utilisation and ensure business continuity, according to the SDDC Global Strategic Business Report. The same report claims demand is so strong that the market is tipped to grow by over 20% CAGR from a $10.2bn dollar industry in 2024 to $312.1bn in 2030.
However, moving to a software-defined infrastructure requires significant planning. That’s because building an SDDC involves considerable expertise. The virutalised resources typically come from different vendors and so require integration and configuration to make them interoperable and able to be centrally managed.
Switching to the new environment can also result in some application downtime and will require operations teams to adjust their ways of working. To minimise this, it’s wise to develop the SDDC in phases and to test virtualised layers before committing to go-live. So, what are the key considerations when building an SDDC?
Building blocks for SDDC
To start with, these advanced systems require power densities exceeding 150kW per rack, so it’s necessary to consider power and cooling investment but at the same time that investment needs to allow for those regulatory compliance initiatives that on the horizon.
The Climate Neutral Data Centre Pact will require data centres across Europe to only use renewable energy by 2030, so power planning must make provision for renewable energy sourcing and carbon footprint management.
Another area to consider is the software foundation itself for the management of the infrastructure. Today, most data centre switching vendors are aligned around VxLAN Fabric aka EVPN-VxLAN as their primary architecture.
This provides scalable, resilient Layer 2 and 3 connectivity over distributed environments so creates a flexible underlay that supports dynamic policy management, microsegmentation and the east-west traffic volumes to meet AI demands.
A central management platform unifies the software-defined compute, software-defined storage and software-defined network elements of the stack, allowing all three to be monitored and managed but its orchestration and automation platforms that then facilitate automated processes.
These infrastructure-as-code solutions streamline the provisioning of virtual capacity, policy enforcement and lifecycle management and modern versions can even predict failures and trigger the environment to self-heal.
Cost as a key consideration
Of course, the SDDC concept is not new – it was first coined back in 2012 by VMWAre’s Steve Herrod – but what we are now seeing is the technology in real demand as a way to cope with the unpredictability and costs associated with AI resource consumption.
Organisations are now realising they simply cannot afford dedicated AI infrastructure but they can afford an SSDC because of this is able to offered via a multi-tenant architecture.
Not only does that allow the organisation to share AI hardware but it also ensures security through the logical separation of those using the service. And it allows that capacity to be scaled to accommodate the variations that occur between training phases and inference workloads without the need for manual reconfiguration.
Building SDDC infrastructure is challenging because it encompasses so many elements, from power system design for AI workloads to cooling architecture, multi-tenant security frameworks and orchestration platform selection.
But those that do commit to building these AI-ready facilities will have the competitive advantage. In contrast, those that try to wait it out in the hope that infrastructure costs will drop could find themselves priced out of the market.
