The real bottleneck of AI infrastructure: data transfer rather than GPU computing power.

When GPUs Wait for Data, Compute Investment Is Being Wasted

Over the past 18 months, companies have frantically purchased GPUs, trained large models, and deployed AI tools. Yet as they shift from experimentation to production, ROI pressure intensifies. An IDC 2025 report notes that organizations are moving from one-time AI deployments to repeatable, scalable production architectures, where performance, security, reliability, and operational consistency become as important as model innovation.

But many enterprises are discovering that the root cause of poor AI project performance is not insufficient compute power, but inefficient data delivery. Expensive GPUs spin idle while waiting for data, with each second of idleness costing several dollars. Uptime Institute's 2025 annual outage analysis shows that over half of enterprises incur outage costs exceeding $100,000, and one in five exceed $1 million — and outages often stem from data path congestion.

The Iceberg Model: The Overlooked 90% of Infrastructure

Nirav Shah, Senior Vice President of Product Marketing at F5, uses an iceberg model to describe AI infrastructure: above the surface are LLMs, AI applications, orchestration frameworks, and expensive GPU clusters — the 10% that attracts most investment; below the surface are storage, networking, traffic management, security controls, and the systems responsible for data delivery — the 90% that determines whether the investment can truly generate value.

Modern AI systems rely on massive amounts of unstructured data stored in S3-compatible object storage. Training, fine-tuning, RAG, and inference workloads all require continuous data flow between storage and GPU environments. When the pipeline is constrained, GPU utilization drops sharply. Shah points out: "The symptoms look like a compute problem, but the root cause is often data starvation."

Unlike traditional enterprise applications, AI workloads amplify even minor infrastructure weaknesses. A latency spike, throughput bottleneck, or traffic surge that goes unnoticed in a conventional environment has a disproportionate impact in AI scenarios.

From Tightly Coupled to Loosely Coupled: Architectural Change Drives Flexibility

The current bottleneck stems from architectural decisions made before the AI era. Historically, enterprises connected applications directly to storage environments — simple and efficient. But at AI scale, that simplicity becomes a liability. Storage systems are forced to handle tasks that are not their own: terminating encrypted connections, managing network traffic, enforcing security policies, and processing large volumes of distributed AI requests. Each encrypted transaction consumes CPU, each connection incurs overhead, and each traffic spike adds pressure to the storage system.

The solution is to shift to a loosely coupled architecture: inserting an application delivery controller (ADC) between compute and storage as an intelligent control plane. The ADC becomes the front door to storage, handling TLS termination, certificate management, traffic optimization, policy enforcement, and protocol-aware S3 processing. By moving networking and encryption functions to specialized infrastructure, storage systems can focus on serving data. At the same time, upgrading, scaling, or migrating the storage environment does not force application changes, bringing operational flexibility.

The Middle Layer Is No Longer Just an "Extra Hop": ADC Improves Rather than Degrades Performance

In the past, infrastructure teams were skeptical of any additional layer in the data path, believing that every component adds latency.In the past, infrastructure teams were skeptical of any additional layers in the data path, believing that each component added latency. AI infrastructure is challenging this assumption. Independent testing by SecureIQLab evaluated the impact of placing an ADC in front of enterprise object storage. The results showed negligible throughput loss—and under real network conditions, traffic managed by a control layer often achieved better throughput. The reason is that an ADC does not simply pass traffic; it optimizes connections, manages protocols, offloads encryption processing, and intelligently directs requests, helping the data path run more efficiently.

Case Study: Data Delivery Optimization Delivers 5x Performance Improvement

A large global financial services institution was preparing to scale its AI infrastructure, using Kubernetes workloads and S3 object storage. The existing environment relied on a shared virtual load balancer, which led to performance and reliability issues as data volumes increased. Instead of adding compute capacity, the institution focused on the storage-to-compute boundary: it deployed dedicated physical ADC infrastructure in front of the object storage, creating a centralized control point for traffic management and S3 optimization. The result: object creation, read, and delete operations improved by at least 5x, delete latency improved by more than an order of magnitude, with no performance regression.

Three Dimensions of Resilience: Reachability, Policy, and Delivery

F5 Solutions Architect Mark Menger proposed three dimensions for evaluating AI data delivery architectures:

• Reachability: Ensure AI workloads always have access to healthy storage resources. When storage clusters degrade or become unavailable, traffic is automatically redirected without disrupting AI applications.
• Policy: Prevent self-inflicted disruptions. AI workloads can generate abnormal traffic patterns such as thundering herd effects and retry storms. An intelligent control layer can shape traffic and enforce policies to maintain a consistent security posture without sacrificing performance.
• Delivery: Focus on continuity. Storage node failures, hardware upgrades, software patches—a resilient architecture isolates AI clients from these changes, ensuring data flow remains uninterrupted even as the backend infrastructure changes.

Together, these three ensure that GPUs remain efficient even under stress from the underlying environment.

Evolution from ADC to ADSP: A Unified Delivery and Security Platform

As AI environments grow larger and more distributed, traffic management alone is no longer sufficient. Organizations need observability, security, policy enforcement, and operational consistency. This is driving interest in Application Delivery and Security Platforms (ADSP), which combine application delivery, traffic engineering, security controls, and visibility.

“AI breaks the pattern of treating delivery and security as separate problems,” said Shah. “When data is constantly moving between storage, compute, and applications across hybrid multi-cloud environments, you need a unified platform that both delivers and protects traffic.” This mirrors the evolution of web infrastructure 20 years ago: from simple load balancing to complex application delivery controllers.

CloudTechDaily InsightThe bottleneck of AI infrastructure is shifting from computation to data transfer. If enterprises focus solely on GPU procurement while neglecting the pipeline efficiency from storage to computation, they will face severe investment waste and unreliable performance. Loosely coupled architectures and intelligent control layers (such as ADC/ADSP) can not only improve GPU utilization but also reduce operational complexity and enhance resilience. For CIOs and cloud architects, the data transfer layer should become a core component of AI infrastructure strategy. In the future, as inference and edge AI demand grows, optimizing data paths will determine the success or failure of AI deployment. Enterprises need to reassess their current architectures: have they decoupled networking, security, and traffic management functions from storage? Have they introduced a unified delivery control plane? The answers will directly impact the ROI and scalability of AI projects.