Businesses are increasingly investing tens of millions of dollars in training advanced AI applications, particularly large language models (LLMs) and generative AI, to improve operations and create new products. Yet, the costs continue to climb as companies move these models between data centers and cloud environments for testing and validation. As AI becomes more mainstream, a pressing question emerges: How should we manage the resource-intensive training requirements of these models?
Our CTO, Sven Oehme, recently wrote an article on Forbes discussing deployment choices and his thoughts are summarized here.
Cloud Turbulence: The AI Deployment Dilemma
Many enterprises have adopted a “cloud first” or “cloud only” strategy, attracted by the simplicity, scalability, and rapid deployment of public clouds. However, those looking to train LLMs or generative AI often encounter significant constraints with certain cloud implementations. Regulatory requirements, performance and latency issues, and soaring costs can make some cloud deployments less appealing or even impractical. Traditional services from the well-known hyperscalers are generally not a fit for these workloads, and while these providers are building out specific offerings, there are other deployment options that might be a better fit for resource-intensive requirements.
Cosmic Complexity: Navigating the Exponential Demands of AI Infrastructure
Today’s LLMs and generative models are vastly more complex than their predecessors. They are up to 10 times larger and require up to 100 times more compute resources. These models handle billions or even trillions of parameters to generate sophisticated and accurate responses and predictions in real time. This exponential growth in complexity drives the need for a new type of infrastructure capable of delivering the necessary speed, scale, and agility.
Major cloud providers like AWS, Azure, and Google Cloud Platform (GCP) are revising their infrastructure strategies to meet these demands but are often over allocated due to demand. Regardless of location, AI projects now require infrastructure that is faster, more scalable, and more secure, with highly efficient power footprints at any scale.
Mission-Critical or Ad Hoc?: Reserved and On-Demand AI Resources
Public cloud providers with unique approaches have emerged to challenge the hyperscalers. Vendors like Lambda, Scaleway, Bitdeer offer different models to accommodate various needs in their specialized GPU clouds:
- Reserved Models: These provide dedicated, full-stack resources with guaranteed Service Level Agreements (SLAs) for the most critical AI deployments. These typically carry a price premium, but also mean that customers won’t encounter issues when GPU resources are fully allocated.
- On-Demand Models: GPUs and powerful storage resources supplied on an as-needed basis, which is ideal for handling fast growth or fluctuating usage. While more cost effective, if GPUs are not available when needed, there could be project delays.
These AI-specialized clouds cater to applications requiring intense computational power and parallel processing capabilities, promising faster performance, lower latency, and reduced costs compared to traditional hyperscale clouds. They often also feature specialized workflows to help simplify the deployment of AI training workloads.
Re-entry Phase: The Return to On-Premises and Colocation
Despite the flexibility offered by cloud deployments, many IT departments are re-evaluating their cloud strategies. According to IDC, 70 to 80 percent of organizations plan to repatriate some of their compute and storage resources to private cloud or non-cloud environments. A recent Citrix survey cited by InfoWorld found that 93% of IT leaders have been involved in cloud repatriation projects over the past three years.
Houston, We Have a Problem: Skyrocketing Costs and Lagging Performance
The return to on-premises or colocation facilities is partly driven by rising public cloud costs, which became more evident as the global pandemic receded. The increasing volume of data that organizations must store and process contributes to these costs. Moreover, moving data in and out of hybrid cloud environments can incur substantial egress charges. According to the Uptime Institute, 42% of organizations that recently migrated their production apps from the cloud cited cost as the primary reason.
Performance is another critical factor. While public clouds can deliver good performance, data transfer speeds between the cloud and on-premises or colocation centers can be slower. This latency can be unacceptable for applications that require real-time processing, such as those in finance, aerospace, autonomous driving, and life sciences.
Gravity’s Pull: The Case for On-Premises AI
Organizations with high sensitivity to data latency and security often find on-premises solutions more predictable and effective. With advancements in GPU-based servers, parallel file systems, and fiber optic networks capable of delivering speeds in petabits per second, on-premises infrastructure can now match or even exceed the scale and speed of cloud environments. Additionally, on-premises deployments can more easily meet compliance and data sovereignty requirements, often at a lower cost.
Mission Support: DDN’s Role in AI Infrastructure
DDN plays a crucial role in both AI cloud deployments and on-premises infrastructure solutions. As the world’s leading data intelligence platform, DDN specializes in AI storage solutions that are essential for handling the massive datasets and high throughput requirements of LLM training and deployment. Our goal is to supercharge every data workload in the world to reliably accelerate massive data sets for actionable real-time insights. Many of the specialist GPU service providers leverage DDN as their backend storage to accelerate innovation, deliver cost savings and operational efficiency, and enhance security and reliability at scale.
Orbital Operations: Solutions for AI Cloud Suppliers
- Scalable Storage Solutions: DDN provides massively scalable storage solutions that integrate seamlessly with major cloud providers. This ensures that AI cloud suppliers can offer reliable, high-speed storage to their customers, supporting the extensive data requirements of LLMs.
- High Throughput: DDN’s storage systems are designed for high throughput, which is critical for the rapid training and inference processes in AI. This capability helps cloud providers meet the performance needs of their most demanding AI workloads.
- Data Management: DDN offers advanced data management features that allow cloud providers to efficiently handle large datasets, ensuring data is accessible and secure.
Earth-Based Operations: Optimizing On-Premises AI Deployments
- Tightly Integrated Solutions: DDN works with NVIDIA to ensure that AI infrastructure is completely optimized. Customers can utilize reference architectures to quickly deploy the exact same blueprint NVIDIA uses on their internal Selene and Eos supercomputers.
- Data Sovereignty and Compliance: For organizations that require on-premises control to meet compliance and data sovereignty requirements, DDN offers solutions that ensure data remains within the organization’s control while providing the necessary performance, efficiency and scalability.
- Enhanced Performance: DDN’s storage systems are supercharged for AI workloads, offering the low latency and high throughput needed for real-time data processing and training of LLMs. Using DDN allows customers a 30% increase in GPU processing time with the same infrastructure, by reducing wait times brought on by data intensive operations.
Mission Debrief: Navigating the AI Deployment Frontier
Training large language models and other AI systems presents significant challenges and costs. As businesses navigate these complexities, the decision between cloud, on-premises, and hybrid solutions will depend on a variety of factors, including cost, performance, regulatory requirements, and the specific needs of their AI applications.
Companies like DDN provide valuable support by offering high-performance storage solutions that enhance both cloud and on-premises AI infrastructure. By carefully considering these factors and leveraging the right technology partners, companies can make informed decisions that balance the benefits of cloud flexibility with the control and performance of on-premises infrastructure, ensuring the successful integration of AI into their operations.
