garak (Generative AI Red-teaming & Assessment Kit) is an open-source vulnerability scanner for language models, intended to do for LLMs roughly what nmap or Metasploit do for networks. The tool checks whether a model can be made to fail in unwanted ways, such as hallucination, data leakage, prompt injection, misinformation, toxic output and jailbreaks, using static, dynamic and adaptive probes. Supported targets include Hugging Face, OpenAI, Cohere, Groq, AWS Bedrock, NVIDIA NIM, local GGUF models and any REST endpoint. You install garak with pip and run it yourself, for example from the command line.
garak was created by Leon Derczynski and is now developed and supported by NVIDIA, a US company; the code is published under NVIDIA's GitHub organisation. The licence is Apache 2.0, so you can run the tool entirely under your own control and keep data within your own environment. No hosted service is required. The project is, however, steered by a single vendor, which is relevant for long-term dependency.