Executive Summary
- As AI adoption accelerates, enterprises face growing challenges around infrastructure, data sovereignty, and sole dependence on individual providers.
- Should we be holding all data in one place, taking the data to the AI or the AI to the data? And what about location restrictions and regulations?
- Colin Fernandes, EMEA Enablement Director, Acceldata looks at the ‘messy reality’ of enterprise data and offers an approach that can balance change, scale, and control.
Companies everywhere are investing in AI, as leadership teams want competitive advantages. In Accenture’s Pulse of Change Report, 78 percent of business leaders surveyed saw AI as more beneficial to revenue growth than cost reduction. ISG found that 31 percent of prioritised AI programmes are now in production, while 50 percent of projects were delivering on expected efficiency gains.
At the same time, companies and governments have to look at control and resilience around how they use AI. When Anthropic temporarily suspended its latest model Fable from the market due to US Government concerns, companies realised how dependent they are on these models, the applications and infrastructure and their providers.
Balancing speed and sovereignty
Rather than relying on one provider, enterprises will look at hybrid infrastructure options instead. Hybrid IT provides more flexibility around changing business or regulatory requirements, without asking companies to rearchitect their infrastructure completely. According to Gartner’s How to Use Distributed Hybrid Infrastructure as a Path to Sovereignty, those in charge of data want to use hybrid architectures to meet their companies’ sovereignty goals, while maintaining control over infrastructure and data.
According to ISG, too, sovereignty is now one of the top three considerations for infrastructure provider decisions. Rather than wanting to run everything in the cloud with one provider, companies have to work with what is in place, whether they are running on-premise data warehouses, in private or hybrid clouds, or moving to public cloud infrastructure. Most enterprises will have a mix of different existing technologies.
Managing data in reality
Getting everything in one place for AI is a great approach in theory. However, this ignores the messy reality for enterprise data – 80 percent of global enterprises run hybrid data architectures, and 75 percent operate four or more data platforms in production, according to GLG Research.
Migrating all that data to AI is a huge, potentially expensive undertaking and can collide with sovereignty rules. Most enterprises already run Extract, Transfer and Load (ETL) pipelines to get data where it is needed. However, each ETL operation is built for a single source, target and format, so portability is costly and interoperability is assumed rather than engineered in as standard.
Any component update can break the downstream job, and pipeline lineage rarely flags the issue before it becomes a data quality problem. To remove this problem, hybrid AI gateways can act as the connective layer between open data architectures and AI models.
Rather than multiple pipelines, there is one policy and routing point across models and environments, so workloads move without rewriting and runtime governance holds wherever they land.
For those managing these substantial data projects, the work to get data into AI systems is significant. AI budgets have the scope to improve those deployments, putting in better rules around data quality. At the same time, teams should consider whether they need to move that data centrally or rely on brittle ETL pipelines at all.
Instead, rather than migrating data centrally away from current locations, AI systems can and should be deployed to where that data exists today. This allows companies to work with data where it is, not where cloud vendors would like it to be. When data exists in multiple places, taking a federated approach is necessary.
Federated data and the hybrid stack
Federated architecture leaves data where it is stored, while presenting it to the system as one logical data set. In practice, a query is translated into the right form for each underlying system, executed locally, and the results returned in a single format.
Say that your customer records sit in on-premise databases for one region and long-term analytics live in another location in a data warehouse. A query on customer marketing results would need data from both, so it is decomposed, each system runs its own portion, and then one answer comes back to the requester. The important element is that processing happens next to the data, rather than through repeated round trips and duplicate copies.
From a user perspective, this is simpler, but it only works if the system knows what it is querying. Federated data depends on a metadata layer underneath that contains all the catalogue holding schemas, ownership of the data, classification and lineage across every registered source. Queries use that metadata for semantic discovery that finds the right data set by meaning rather than by table name. The catalogue then has to act as a control point for access policy, residency rules and data contracts to be enforced once and applied everywhere.
This is where residency and sovereignty are not the same. Residency is based on geography, while sovereignty is about jurisdiction and who can lawfully compel access to that data, wherever it sits. For example, under the US CLOUD Act, a US-headquartered provider can be ordered to produce data held on European soil.
Other regulations like GDPR, NIS2, India’s DPDP Act and Gulf localisation rules all apply to the same scenario in different ways. Federating your data ensures that it is easier to meet both sovereignty and residency needs by keeping compute processing next to the data and moving only the results back.
Federated data and AI
The same principle extends to AI infrastructure too. AI Inference and compute should run where the data already sits, inside the trust perimeter, and only the output leaves. This removes the long pipeline that would otherwise be built to feed an external model, and it also removes the need for data copies, latency lag and residency exposure.
It also changes the economics. When external large language models are used, routing becomes a commercial decision. You can match model capability to the value of the task, so a commodity model will handle high-volume, low-stakes work such as classification, extraction, or summarisation.
This preserves your AI token budget when you need it for those jobs that justify a frontier model. That decision is made per workload, not per contract, and it can be made again next quarter as prices and capabilities develop over time.
In practice, a hybrid ecosystem involves a working set of parts from different suppliers: two or three model providers, a hyperscaler or two alongside existing on-premise capacity. From the data side, this relies on open table formats, that established data catalogue and the accuracy of the metadata involved.
For the vast majority of large enterprises, their IT estate is already hybrid. The question is whether the parts are connected through open interfaces, and therefore substitutable, or through proprietary ones, and therefore permanent.
‘Be positive around change while staying in control’
Two conditions hold a system like this together. First, data stays inside the established trust perimeter, so the AI programme inherits the governance already in place rather than opening another weaker path around it. Secondly no layer — model, engine, storage format or catalogue — becomes an element that cannot be replaced.
This approach ensures that you can be positive around change while staying in control. Taking an open, federated approach means you can design the infrastructure to suit your organisation, rather than being dictated to by suppliers with their own agendas. With so much emphasis on implementing AI at scale across businesses, teams can build on their existing hybrid and on-premise data.
