I will pull one story from my AI bucket to focus on. From the Wall Street Journal it’s titled “The AI Boom Is Here. The Cloud May Not Be Ready.” The core premise – is that cloud systems were not built at scale to use the GPUs required for much of AI. Quote:
But much of the infrastructure wasn’t built for running such large and complex systems. Cloud sold itself as a convenient replacement for on-premise servers that could easily scale up and down capacity with a pay-as-you-go pricing model. Much of today’s cloud footprint consists of servers designed to run multiple workloads at the same time that leverage general-purpose CPU chips.
A minority of it, according to analysts, runs on chips optimized for AI, such as GPUs and servers designed to function in collaborative clusters to support bigger workloads, including large AI models. GPUs are better for AI since they can handle many computations at once, whereas CPUs handle fewer computations simultaneously.
And this
Microsoft Azure and Google Cloud Platform said they are similarly working to make AI infrastructure a greater part of their overall fleets. However, Microsoft’s Chappell said that that doesn’t mean the company is necessarily moving away from the shared server—general purpose computing—which is still valuable for companies.
Other hardware providers have an opportunity to make a play here, said Lee Sustar, principal analyst at tech research and advisory firm Forrester, covering public cloud computing for the enterprise.
Dell Technologies expects that high cloud costs, linked to heavy use—including training models—could push some companies to consider on-premises deployments. The computer maker has a server designed for that use.
It would be a hot take to say, “On-Prem will make a comeback.” That’s not the story. The hardware play is “to the data center,” not “to your office.”
AI cloud services are added to the portfolio. Sure, the cloud isn’t ready. And NVIDIA will kill it by selling many GPU-enabled systems for those data center buildouts. Microsoft is correct; it’s not moving away; it’s expanding. Dell may be right. I’d question just how many organizations will be training models.
This is a good read.