Getting infrastructure right for generative AI [Video]

Ensuring a cost-effective approach for delivering the massive storage, bandwidth, and computing resources necessary for genAI is no easy task. Here’s how innovative IT leaders are coping.

Credit: Gorodenkoff / Shutterstock

Facts, it has been said, are stubborn things. For generative AI, a stubborn fact is that it  consumes very large quantities of compute cycles, data storage, network bandwidth, electrical power, and air conditioning. As CIOs respond to corporate mandates to “just do something” with genAI, many are launching cloud-based or on-premises initiatives. But while the payback promised by many genAI projects is nebulous, the costs of the infrastructure to run them is finite, and too often, unacceptably high.

Infrastructure-intensive or not, generative AI is on the march. According to IDC, genAI workloads are increasing from 7.8% of the overall AI server market in 2022 to 36% in 2027. In storage, the curve is similar, with growth from 5.7% of AI storage in 2022 …