An advanced AI strategy needs a flexible data architecture that scales intelligently and cost-effectively to meet a wide range of emerging demands. All-flash data platforms are ideally suited for the rigorous demands of high-performance AI and HPC workloads running on GPUs. However, you can realize important technical and economic benefits by integrating your all-flash performance pools and your HDD-based capacity pools, representing “hot” and “cold” data respectively, within a single data platform. DDN EXAScaler with Hot Pools uses DDN’s well-established File Level Redundancy (FLR) technology, which lets you simultaneously take advantage of multiple storage pools to automatically balance performance and capacity. As a result, you can utilize and manage your AI storage and data more efficiently than ever.
Automated, policy-based data movement
The DDN AI400X appliance, an all-flash system, lets you operate your entire AI workflow from a single high-performance pool, from which all deep learning operations can take place concurrently and continuously. Using Hot Pools you can augment the all-flash AI400X system with a DDN HDD-based capacity pool, which provides a denser, lower cost repository to store your valuable data until you’re ready to actively process it. Hot Pools eliminates tedious performance and capacity management and operational disruption with automated, policy-based data movement between pools. It also further simplifies operations by maintaining both performance and capacity pools within a single namespace.
No operational disruption or performance impact
Hot Pools lets you simultaneously maximize the advantages of performance and capacity pools. For example, AI data ingested in the performance pool can be automatically and transparently migrated to the capacity pool for storage and processing once the performance pool reaches a predetermined capacity threshold. During data migration to the capacity pool the data ingest operation continues uninterrupted and full performance is maintained on the performance pool. An application can be launched at any time to read the freshly ingested data from the capacity pool.
Seamlessly engage data for other Deep Learning phases
As a complementary capability, Hot Pools can automatically promote file data from the capacity pool to the performance pool if the associated files are regularly accessed. You can also initiate the migration of capacity pool data to the performance pool and seamlessly engage other phases of the deep learning workflow such as training and simulation.
Data is always accessible, no matter where it resides
All of the aforementioned operations – data ingest to the performance pool, data movement between performance and capacity pools, and data read from the capacity pool – occur simultaneously without compromising performance, as Hot Pools keep GPUs fully saturated while data movement occurs automatically in the background.
Hot Pools is more than just a storage tiering solution that moves data between pools. Its sophisticated FLR technology more accurately provides an automated caching process that creates file level copies which reside in both pools. As a result, your applications can always write to the performance pool and can transparently access data whether it resides in either the performance or capacity pool. With Hot Pools, you get faster performance, better data protection and greater effective capacity compared to storage tiering. The system dashboard provides a view in which to evaluate and validate Hot Pools operations, with the real-time flow of data through the pools.
The best performance and economics for your AI workflows
Simplify data management for your AI and HPC workflows with powerful, intelligent automation at scale. With EXAScaler Hot Pools, you can make the most of your all-flash and large-capacity pools for the best performance and economics.
Summary
- Simplify data management for AI and HPC workflows with powerful, intelligent automation at scale.
- Realize several technical and economic benefits by integrating all-flash performance pools and HDD-based capacity pools, representing “hot” and “cold” data within a single data platform.
- Augmenting the all-flash system with an HDD-based capacity pool provides a denser, lower-cost repository to store valuable data until it’s ready for processing.
- DDN Hot Pools lets organizations simultaneously maximize the advantages of performance and capacity pools. Achieve faster performance, better data protection and greater effective capacity than storage tiering.
