\n<\/p>\n
Giskard<\/a> is a French startup working on an open-source testing framework for large language models. It can alert developers of risks of biases, security holes and a model\u2019s ability to generate harmful or toxic content.<\/p>\n While there\u2019s a lot of hype around AI models, ML testing systems will also quickly become a hot topic as regulation is about to be enforced in the EU with the AI Act, and in other countries. Companies that develop AI models will have to prove that they comply with a set of rules and mitigate risks so that they don\u2019t have to pay hefty fines.<\/p>\n Giskard is an AI startup that embraces regulation and one of the first examples of a developer tool that specifically focuses on testing in a more efficient manner.<\/p>\n \u201cI worked at Dataiku before, particularly on NLP model integration. And I could see that, when I was in charge of testing, there were both things that didn\u2019t work well when you wanted to apply them to practical cases, and it was very difficult to compare the performance of suppliers between each other,\u201d Giskard co-founder and CEO Alex Combessie told me.<\/p>\n There are three components behind Giskard\u2019s testing framework. First, the company has released an open-source Python library<\/a> that can be integrated in an LLM project \u2014 and more specifically retrieval-augmented generation (RAG) projects. It is quite popular on GitHub already and it is compatible with other tools in the ML ecosystems, such as Hugging Face, MLFlow, Weights & Biases, PyTorch, Tensorflow and Langchain.<\/p>\n After the initial setup, Giskard helps you generate a test suite that will be regularly used on your model. Those tests cover a wide range of issues, such as performance, hallucinations, misinformation, non-factual output, biases, data leakage, harmful content generation and prompt injections.<\/p>\n \u201cAnd there are several aspects: you\u2019ll have the performance aspect, which will be the first thing on a data scientist\u2019s mind. But more and more, you have the ethical aspect, both from a brand image point of view and now from a regulatory point of view,\u201d Combessie said.<\/p>\n Developers can then integrate the tests in the continuous integration and continuous delivery (CI\/CD) pipeline so that tests are run every time there\u2019s a new iteration on the code base. If there\u2019s something wrong, developers receive a scan report in their GitHub repository, for instance.<\/p>\n Tests are customized based on the end use case of the model. Companies working on RAG can give access to vector databases and knowledge repositories to Giskard so that the test suite is as relevant as possible. For instance, if you\u2019re building a chatbot that can give you information on climate change based on the most recent report from the IPCC and using a LLM from OpenAI, Giskard tests will check whether the model can generate misinformation about climate change, contradicts itself, etc.<\/p>\n Image Credits:<\/strong> Giskard<\/p>\n<\/div>\n