\n<\/p>\n
GPUs\u2019 ability to perform many computations in parallel make them well-suited to running today\u2019s most capable AI. But GPUs are becoming tougher to procure, as companies of all sizes increase their investments in AI-powered products.<\/p>\n
Nvidia\u2019s best-performing AI cards sold out<\/a> last year, and the CEO of chipmaker TSMC suggested<\/a> that general supply could be constrained into 2025. The problem\u2019s so acute, in fact, that it has the U.S. Federal Trade Commission\u2019s attention \u2014 the agency recently announced<\/a> it\u2019s investigating several partnerships between AI startups and cloud giants like Google and AWS over whether the startups might have anti-competitive, privileged access to GPU compute.<\/p>\n What\u2019s the solution? It depends on your resources, really. Tech giants like Meta, Google, Amazon and Microsoft are buying up what GPUs they can<\/a> and developing<\/a> their own<\/a> custom chips<\/a>. Ventures with fewer resources are at the mercy of the market \u2014 but it doesn\u2019t have to be that way forever, say John Yue and Michael Yu.<\/p>\n Yue and Yu are the co-founders of Inference.ai<\/a>, a platform that provides infrastructure-as-a-service cloud GPU compute through partnerships with third-party data centers. Inference uses algorithms to match companies\u2019 workloads with GPU resources, Yue says \u2014 aiming to take the guesswork out of choosing and acquiring infrastructure.<\/p>\n \u201cInference brings clarity to the confusing hardware landscape for founders and developers with new chips coming from Nvidia, Intel, AMD, Groq [and so on] \u2014 allowing higher throughput, lower latency and lower cost,\u201d Yue said. \u201cOur tools and team allow for decision-makers to filter out a lot of the noise and quickly find the right fit for their project.\u201d<\/p>\n Inference essentially provides customers a GPU instance in the cloud, along with 5TB of object storage. The company claims that \u2014 thanks to its algorithmic matching tech and deals with data center operators \u2014 it can offer dramatically cheaper GPU compute with better availability than major public cloud providers.<\/p>\n \u201cThe hosted GPU market is confusing and changes daily,\u201d Yue said. \u201cPlus, we\u2019ve seen pricing vary up to 1000% for the same configuration. Our tools and team allow for decision makers to filter out a lot of the noise and quickly find the right fit for their project.\u201d<\/p>\n Now, TechCrunch wasn\u2019t able to put those claims to the test. But regardless of whether they\u2019re true, Inference has competition \u2014 and lots of it.<\/p>\n See: CoreWeave, a crypto mining operation-turned-GPU provider, which is reportedly<\/a> expected to rake in around $1.5 billion in revenue by 2024. Its close competitor, Lambda Labs, secured<\/a> $300 million in venture capital last October. There\u2019s also Together<\/a> \u2014 a GPU cloud \u2014 not to mention startups like Run.ai<\/a> and Exafunction<\/a>, which aim to reduce AI dev costs by abstracting away the underlying hardware.<\/p>\n Inference\u2019s investors seem to think there\u2019s room for another player, though. The startup recently closed a $4 million round from Cherubic Ventures, Maple VC and Fusion Fund, which Yue says is being put toward build out Inference\u2019s deployment infrastructure.<\/p>\n In an emailed statement, Cherubic\u2019s Matt Cheng added:<\/p>\n \u201cThe requirements for processing capacity will keep on increasing as AI is the foundation of so many of today\u2019s products and systems. We\u2019re confident that the Inference team, with their past knowledge in hardware and cloud infrastructure, has what it takes to succeed. We decided to invest because accelerated computing and storage services are driving the AI revolution, and Inference product will fuel the next wave of AI growth.\u201d<\/p>\n<\/p><\/div>\n