\n<\/p>\n
If AI lab PrismML<\/a> isn\u2019t on your radar yet, it should be \u2014 not because it\u2019s raised gobs of money (it hasn\u2019t yet, just a $22.25 million seed round), but because of the technical minds involved and the potentially industry-changing tech it\u2019s developing.<\/p>\n PrismML is betting that capable, high-performing, reasoning large language models don\u2019t, in fact, have to be large.<\/p>\n It is making reasoning models so small they can fit on PCs and smartphones. (It\u2019s even rumored to be in talks with Apple<\/a>, though CEO Babak Hassibi declined to comment on that to TechCrunch.)<\/p>\n On Thursday, PrismML released Bonsai 2 27B<\/a>, its latest in a family of models, which compresses Qwen3.8 27B, a widely used open-source model from Alibaba, down to 5.9 GB. That\u2019s small enough to fit on a PC and, possibly, a high-end smartphone. It\u2019s a 9x to 10x reduction in memory versus the original.<\/p>\n PrismML was founded by a group of Caltech researchers and is led by Hassibi, a Caltech professor and an expert in compression technologies. The startup also counts Ion Stoica as an advisor. Stoica is a co-founder of Databricks (and other companies) and the director of Berkeley\u2019s famed Sky Computing Lab, which has birthed many technologies and startups, from Letta<\/a> to SGLang<\/a>. <\/p>\n PrismML is also backed by investors Khosla Ventures, Cerberus Capital, and Caltech.<\/p>\n This startup is certainly not the only company working on LLM compression tech. Multiverse Computing, founded by a well-known professor from Spain\u2019s Donostia International Physics Center, is another. (And Multiverse Computing has raised gobs of cash<\/a>.)<\/p>\n But Hassibi says that PrismML\u2019s compression tech is unique because its LLMs have lost virtually no performance compared with the originals. Bonsai 2 matches 98% of Qwen\u2019s aggregate benchmark scores. That\u2019s up from the first Bonsai, released a couple of months ago in March, that matched 95%. That original model has already been downloaded over 11 million times, and PrismML\u2019s even smaller models have been downloaded another 2.6 million times, the company says.<\/p>\n So this shows that PrismML\u2019s compression results have improved from one release to the next. Whether it could ever get to 100% benchmark performance parity is a question that remains to be seen. Compression will likely always have some<\/em> impact, Hassibi says.<\/p>\n Still, perfect benchmark parity is fairly academic anyway. LLMs are not so accurate in their uncompressed form, and benchmarks not so perfectly reflective of actual tasks, that a 2% degradation would likely meaningfully affect how a model performs in actual use. (Plus, the surrounding software \u2014 the harness a model runs inside of \u2014 matters a lot when it comes to accuracy<\/a>, too.)<\/p>\n PrismML says it achieves this by shrinking the \u201cweights\u201d that make up a model \u2014 weights are, essentially, the information a model learns and stores during training. Normally, each weight requires 16 bits. PrismML\u2019s approach, called \u201cternary\u201d weights, simplifies that down to three: +1, \u22121, or 0. With far smaller values to store for each weight, the model takes up dramatically less space. (For a deeper dive on the compression technique, here\u2019s the project\u2019s GitHub page<\/a>.)<\/p>\n The startup\u2019s next goal is to apply this compression technique to even bigger models. \u201cThe next models that we will release, hopefully in the next couple of months, will be in the several-hundred-billion-parameter range, and I expect it will be easier to retain the intelligence there,\u201d Hassibi told TechCrunch.<\/p>\n As model size grows, he added, \u201cThere is more room to be able to compress them without losing the intelligence. So I would just say, as a general trend, for larger models, it\u2019s easier to get to 100%.\u201d<\/p>\n Stoica tells us that he\u2019s excited for this tech because it\u2019s making it possible for advanced models to run on users\u2019 devices. \u201cYou are going to have intelligence at your fingertips, and it\u2019s going to be free because it\u2019s going to run on the device you already bought. It\u2019s also going to be private, because you\u2019re not going to send it to the cloud.\u201d<\/p>\n<\/div>\n When you purchase through links in our articles, we may earn a small commission<\/a>. This doesn\u2019t affect our editorial independence.<\/em><\/p>\n