\n<\/p>\n
AI is all the rage \u2014 particularly text-generating AI, also known as large language models (think models along the lines of ChatGPT<\/a>). In one recent survey<\/a> of ~1,000 enterprise organizations, 67.2% say that they see adopting large language models (LLMs) as a top priority by early 2024.<\/p>\n But barriers stand in the way. According to the same survey, a lack of customization and flexibility, paired with the inability to preserve company knowledge and IP, were \u2014 and are \u2014 preventing many businesses from deploying LLMs into production.<\/p>\n That got Varun Vummadi and Esha Manideep Dinne thinking: what might a solution to the enterprise LLM adoption challenge look like? In search of one, they founded Giga ML<\/a>, a startup building a platform that lets companies deploy LLMs on-premise \u2014 ostensibly cutting costs and preserving privacy in the process.<\/p>\n \u201cData privacy and customizing LLMs are some of the biggest challenges faced by enterprises when adopting LLMs to solve problems,\u201d Vummadi told TechCrunch in an email interview. \u201cGiga ML addresses both of these challenges.\u201d<\/p>\n Giga ML offers its own set of LLMs, the \u201cX1 series,\u201d for tasks like generating code and answering common customer questions (e.g. \u201cWhen can I expect my order to arrive?\u201d). The startup claims the models, built atop Meta\u2019s Llama 2<\/a>, outperform popular LLMs on certain benchmarks, particularly the MT-Bench<\/a> test set for dialogs. But it\u2019s tough to say how X1 compares qualitatively; this reporter tried Giga ML\u2019s online demo<\/a> but ran into technical issues. (The app timed out no matter what prompt I typed.)<\/p>\n Even if Giga ML\u2019s models\u00a0are\u00a0<\/em>superior in some aspects, though, can they really make a splash in the ocean<\/a> of open source<\/a>, offline<\/a> LLMs<\/a>?<\/p>\n In talking to Vummadi, I got the sense that Giga ML isn\u2019t so much trying to create the best-performing LLMs out there but instead building tools to allow businesses to fine-tune LLMs locally without having to rely on third-party resources and platforms.<\/p>\n \u201cGiga ML\u2019s mission is to help enterprises safely and efficiently deploy LLMs on their own on-premises infrastructure or virtual private cloud,\u201d Vummadi said. \u201cGiga ML simplifies the process of training, fine-tuning and running LLMs by taking care of it through an easy-to-use API, eliminating any associated hassle.\u201d<\/span><\/p>\n Vummadi emphasized the privacy advantages of running models offline \u2014 advantages likely to be persuasive for some businesses.<\/span><\/p>\n Predibase, the low-code AI dev platform, found that less than a quarter of enterprises are comfortable using commercial LLMs because of concerns over sharing sensitive or proprietary data with vendors. Nearly 77% of respondents to the survey said that they either don\u2019t use or don\u2019t plan to use commercial LLMs beyond prototypes in production \u2014\u00a0 citing issues relating to privacy, cost, and lack of customization.<\/span><\/p>\n \u201cIT managers at the C-suite level find Giga ML\u2019s offerings valuable because of the secure on-premise deployment of LLMs, customizable models tailored to their specific use case and fast inference, which ensures data compliance and maximum efficiency,\u201d Vummadi said.\u00a0<\/span><\/p>\n Giga ML, which has raised ~$3.74 million in VC funding to date from Nexus Venture Partners, Y Combinator, Liquid 2 Ventures, 8vdx and several others, plans in the near term to grow its two-person team and ramp up product R&D. A portion of the capital is going toward supporting Giga ML\u2019s customer base, as well, Vummadi said, which currently includes unnamed \u201centerprise\u201d companies in finance and healthcare.<\/span><\/p>\n<\/p><\/div>\n