vLLM is an open-source engine for running and serving large language models. It provides an OpenAI-compatible API and supports the CUDA, ROCm, XPU and CPU compute backends. Developers and organisations use it to run models on their own hardware or in their own cloud accounts. It is distributed as a Python package and as a Docker image under the Apache 2.0 licence, and it was originally developed in the Sky Computing Lab at the University of California, Berkeley.
vLLM is a community project within the PyTorch Foundation, which is hosted by the Linux Foundation, a non-profit organisation under US tax law (501(c)(6)). The project website describes installation as a Python package or Docker image rather than as a hosted service. An organisation that runs vLLM itself keeps prompts and outputs within its own infrastructure. This makes it relevant for European organisations looking to replace hosted inference APIs from US providers. Governance sits with a US-based organisation, although the code may be used and modified freely.