Cracking AI’s storage bottleneck and supercharging inference at the edge

As artificial intelligence (AI) becomes integral to various sectors, from healthcare innovations like advanced medical imaging to sophisticated fraud detection and wildlife conservation efforts, a significant challenge arises: data storage limitations. At Venture Beat's Transform 2025 event, industry leaders Greg Matson, head of products and marketing at Solidigm, and Roger Cummings, CEO of PEAK:AIO, engaged in a discussion with Michael Stewart, managing partner at M12, about how cutting-edge storage technologies are enhancing AI applications across enterprises. The MONAI framework, a pivotal development in medical imaging, is revolutionizing the way data is processed, making it faster, safer, and more secure. Innovations in storage capabilities empower researchers to build upon this framework, enabling rapid iteration and innovation. By collaborating with Solidigm, PEAK:AIO has integrated energy-efficient, high-performance storage solutions, allowing MONAI to manage over two million full-body CT scans on a single node within their IT architecture. Matson emphasized the necessity for storage hardware to be tailored to specific use cases as enterprise AI infrastructure evolves. For instance, the unique demands of MONAI's edge applications and the requirements of training clusters highlight the need for high-capacity solid-state storage solutions. However, inference and model training require a different approach, necessitating high-performance SSDs with exceptional input/output operations per second. To achieve optimal performance at the edge, it is essential to scale storage down to a single node, bringing inference capabilities closer to the data. This process involves eliminating memory bottlenecks by incorporating memory into the AI infrastructure, aligning it with both data and metadata. The closer the data is to computational resources, the quicker insights can be derived. Matson pointed out the trend of large-scale deployments in AI, where data centers are designed with specialized hardware to minimize the distance between data and GPUs. These facilities are outfitted with high-capacity solid-state storage to ensure rapid access to petabyte-scale data. This technology is now being adapted for use at the edge and within enterprise settings. For organizations investing in AI systems, it is crucial to maximize performance by utilizing all-solid-state solutions. This approach allows for the handling of vast data volumes while delivering remarkable processing capabilities in compact edge systems. Cummings noted the importance of providing open, scalable solutions that operate at memory speed, leveraging the latest technology to achieve these goals. Looking ahead, the demands for hardware in the training and inference data pipeline will continue to rise, whether through high-speed SSDs or energy-efficient, high-capacity alternatives. Matson predicts a future where one-petabyte SSDs running on low power may replace multiple hard drives, or where high-performance products approach memory speeds. Major GPU manufacturers are now exploring new storage architectures to complement high-bandwidth memory within systems. The evolution from general-purpose SSDs in cloud computing is leading to a clearer distinction between capacity and performance, a trend that is expected to accelerate in the coming years.