\n<\/p>\n
AI startup Hugging Face offers a wide range of data science hosting and development tools, including a GitHub-like portal for AI code repositories, models and data sets, as well as web dashboards to demo AI-powered applications.<\/p>\n
But some of Hugging Face\u2019s most impressive \u2014 and capable \u2014\u00a0 tools these days come from a two-person team that was formed just in January.<\/p>\n
H4, as it\u2019s called \u2014 \u201cH4\u201d being short for \u201chelpful, honest, harmless and huggy\u201d \u2014 aims to develop tools and \u201crecipes\u201d to enable the AI community to build AI-powered chatbots along the lines of ChatGPT<\/a>. ChatGPT\u2019s release was the catalyst for H4\u2019s formation, in fact, according to Lewis Tunstall, a machine learning engineer at Hugging Face and one of H4\u2019s two members.<\/p>\n \u201cWhen ChatGPT was released by OpenAI in late 2022, we started brainstorming on what it might take to replicate its capabilities with open source libraries and models,\u201d Tunstall told TechCrunch in an email interview. \u201cH4\u2019s primary research focus is around alignment, which broadly involves teaching LLMs how to behave according to feedback from humans (or even other AIs).\u201d<\/p>\n H4 is behind a growing number of open source large language models, including Zephyr-7B-\u03b1, a fine-tuned, chat-centric version of the eponymous Mistral 7B model recently released by French AI startup Mistral<\/a>. H4 also forked Falcon-40B, a model from the Technology Innovation Institute in Abu Dhabi \u2014 modifying the model to respond more helpfully to requests in natural language.<\/p>\n To train its models, H4 \u2014 like other research teams at Hugging Face \u2014 relies on a dedicated cluster of more than 1,000 Nvidia A100 GPUs. Tunstall and his other H4 co-worker, Ed Beeching, are based remotely in Europe, but receive support from several internal Hugging Face teams, among them the model testing and evaluation team.<\/p>\n \u201cThe small size of H4 is a deliberate choice, as it allows us to be more nimble and adapt to an ever-changing research landscape,\u201d Beeching told TechCrunch via email. \u201cWe also have several external collaborations with groups such as LMSYS<\/a> and LlamaIndex<\/a>, who we collaborate with on joint releases.\u201d<\/p>\n Lately, H4 has been investigating different alignment techniques and building tools to test how well techniques proposed by the community and industry really work. The team this month released a handbook containing all the source code and data sets they used to build Zephyr, and H4 plans to update the handbook with code from its future AI models as they\u2019re released.<\/p>\n I asked whether H4 had any pressure from Hugging Face higher-ups to commercialize their work. The company, after all, has raised hundreds of millions of dollars from a pedigreed cohort of investors that includes Salesforce, IBM, AMD, Google, Amazon Intel and Nvidia. Hugging Face\u2019s last funding round<\/a> valued it at $4.5 billion \u2014 reportedly more than 100 times the company\u2019s annualized revenue.<\/p>\n Tunstall said that H4 doesn\u2019t directly monetize its tools. But he acknowledged that the tools do<\/em> feed into Hugging Face\u2019s Expert Acceleration Program, Hugging Face\u2019s enterprise-focused offering that provides guidance from Hugging Face teams to build custom AI solutions.<\/p>\n Asked if he sees H4 in competition with other open source AI initiatives, like EleutherAI<\/a> and LAION<\/a>, Beeching said that it isn\u2019t H4\u2019s objective. Rather, he said, the intention is to \u201cempower\u201d the open AI community by releasing the training code and data sets associated with H4\u2019s chat models.<\/p>\n \u201cOur work would not be possible without the many contributions from the community,\u201d Beeching said.<\/p>\n<\/p><\/div>\n