Amazon Web Services (AWS), Microsoft Azure, Google Cloud and Oracle account for almost two-thirds of the cloud market, but they are not the only option for IT buyers, with neoclouds grabbing attention and pulling in lots of investment, suggesting there is an alternative – at least for now.
So, what do enterprises need to understand about neoclouds, and where might these upstarts fit into the cloud and artificial intelligence (AI) strategies of general businesses?
“They are a group of organisations that have come out of nowhere over the past few years,” says Spencer Lamb, chief commercial officer at colocation provider Kao Data, with the aim of providing large-scale graphics processing unit (GPU) as a service compute.
There’s no single neocloud model. Some neoclouds operate their own datacentres, while others install their precious GPU capacity in colocation facilities such as those offered by Kao. Their growth has often been bankrolled, in part, by the GPU providers.
Many have clear roots in the cryptocurrency world, which pioneered the operation of large amounts of GPUs, including novel approaches to liquid cooling. But their original customers were organisations looking to train large language models (LLMs) and other foundational AI platforms, which means, ironically, one major customer base for neoclouds is the hyperscalers’ themselves.
ABI principal analyst Leo Gergs says that, right now “it’s the technology sector that’s consuming this, and it’s a highly speculative market”. Although arguably that’s true of the datacentre market as a whole, as players grapple for very real-world resources such as land, power and silicon.
European Datacentre Association secretary general Michael Winterson adds: “They’re filling an absolute, real need, they’re able to speculate and build, and they are getting the support of their key suppliers and customers.”
Of course, the traditional hyperscalers have the technical nous to scale up and operate their own GPU fleets. The problem is that they also have processes that are long and complicated.
“The hyperscale procurement function is like a small country in its own right,” says Lamb. This is unsurprising – a major AWS region going offline can scupper numerous actual small countries. Neoclouds, by comparison, are “small, agile companies”.
“When we deal with them, we’re dealing with a handful of people who are able to make decisions over the negotiation table,” adds Lamb.
This means that while hyperscalers’ decisions on new GPU clusters are measured in years, a neocloud will be ready to move in six months – or even less.
For example, Schneider’s vice-president for secure power and datacentres in the UK and Ireland, Matthew Baynes, says many of its discussions with neoclouds centre on reference designs and standardised processes and modules aimed at speed deployments further, meaning it’s “12 weeks from discussion to landing on site”.
Running hot?
So, what can enterprises learn from neoclouds about operating GPUs at scale – never mind niceties such as cooling or resilience?
The answer is “plenty”, but that doesn’t mean enterprises should start building out their own GPU infrastructure, says Lamb. And the neoclouds themselves need to learn more about the needs of enterprise customers.
Most enterprises will end up having their own training model, which will sit on a GPU somewhere and evolve, develop and become very bespoke to them over time Dan Chester, Vast Data
Dan Chester, EMEA sales director for cloud service providers at Vast Data, says it’s important to grasp that many neoclouds haven’t focused on the sort of elastic compute services offered by traditional cloud firms. Rather, they were focused on the demands of those LLM builders with a need for “extreme” volumes of dedicated GPUs.
Even when it comes to the model training jobs neoclouds first soaked up, Chester says: “They only ran a subset of the workload in the neocloud. The rest of their workload was happening in the classical hyperscalers. These people are running single, large jobs that are potentially going to run for weeks at a time, and need to be optimised to the absolute max.”
This is more reminiscent of the high-performance computing (HPC) world, where scientists or engineers running massive simulations need a dedicated, highly optimised cluster.
“There aren’t many enterprises that need thousands upon thousands of GPUs on a dedicated basis,” says Chester, adding that another class of player had emerged – companies such as Nebius and NexGen Cloud, which “are building what I would consider to be much more of a true cloud”.
What is common across the neoclouds is a laser focus on deploying GPUs at scale and at blistering speed. And that will become more important, Lamb predicts, as enterprises take more control of their AI stack.
“Most enterprises will end up having their own training model that they will acquire, which will sit on a GPU somewhere, and then it will evolve, change and develop, and become very bespoke to them over time,” he says, predicting that the balance of corporate compute capacity could move to GPU over time.
Winterson says: “It does not make sense for most corporations to build their own AI inference systems unless they can guarantee they’re going to use it 24/7, because it’s just too expensive. So, in a sense, the neocloud market today is less risky for an enterprise than the colocation market in 1999.”
The issue for enterprises is understanding where neoclouds fit into their processes.
As Scheider’s Baynes points out: “They’re focused solely on delivering GPU as a service for AI infrastructure, but not the enterprise applications, HR, Oracle, SAP, whatever. You can spin up and lease GPU capacity with these guys very quickly to run applications, production models and algorithms, to build an application for your enterprise.”
Channelling the future
If low latency and speed are key, neoclouds could be a compelling option for enterprises, suggests Baynes, and they may be more cost-effective than hyperscalers. At the same time, he says, a lot will depend on an enterprise’s due diligence procedures.
But neclouds have some learning to do. ABI’s Gergs says hyperscalers and LLM builders are pouring money into neoclouds, but the neocloud operators as a whole, “don’t have a clue what enterprise AI use case will look like, and what they need to do to be able to crack the enterprise AI market”.
The specialist AI compute companies will grow to become major AI computer providers alongside today’s hyperscalers Youlian Tzanev, NexGen
As they move from foundational training and inference, they will need to learn about enterprise use cases and buying models, he predicts, adding: “How would enterprises consume these services and applications so that I can go to them, and then I can go to market with [these services and applications]?”
Right now, they’re not partnering with independent software providers and other channel partners, he added, while enterprises want to embark on “almost a consultative journey”.
That’s been taken on board by at least some neocloud players. NexGen’s founder and chief strategy officer, Youlian Tzanev, says its focus so far has been on AI users in the financial sector, healthcare and general markets, adding: “The challenges for most commercial users of AI are the management of infrastructure and support of fine-tuning. Inference on open source models, with the need to provide extensive data privacy, is challenging and requires teams with deep data experience. Many customers want a more supported AI service.” This is in contrast with the more generic services hyperscalers offer.
That’s just as well, as Gergs says: “Enterprises will not change their buying behaviour and will not change their behaviour of consuming services to suit the technology industry’s needs and requirements. It has to be the other way around.”
While neoclouds have attracted investment and major customers, things will continue to change fast. Longer term, Winterson predicts, there will be “a culling”, just as what happened with earlier generations of internet infrastructure, not least when the dot com boom crashed.
Some operators will be absorbed by the hyperscalers, which are their key customers, while others may establish ongoing business models by developing new revenue streams. Others, still, will have a more painful landing.
But Winterson adds: “The assets won’t go away. They’ll be swallowed up and used … They would be bought as ongoing concerns and incorporated into someone else’s business model.”
For his part, Tzanev predicts that in five years’ time, the successful AI compute companies will be those that focus on delivering a full-stack AI cloud: “The specialist AI compute companies, like NexGen, will grow to become major AI computer providers alongside today’s hyperscalers.” At which point, he predicts, the term neocloud will disappear.
Does that mean the grip of the cloud giants will have finally been broken? At the very least, it suggests that speed and agility can win out against sheer weight and dominance – for a while, at least.
