← Discover
GitHub · vllm-project/vllm↑ +128 today

High-throughput and memory-efficient inference and serving engine for AI models

vllm-project/vllm is an inference and serving engine that provides high throughput and memory efficiency for large language models (LLMs). Developed in Python, this tool aims to accelerate the execution processes of artificial intelligence models.

Updates

  • October 21, 2026: Stars 94,616 → 94,625, latest release v0.32.0 (October 18, 2026).

What you get

  • offers PagedAttention management providing memory efficiency
  • works compatibly with popular Hugging Face models
  • includes OpenAI-compatible API server support

Installation

vllm installation
uv pip install vllm
vllm download-kernels  # precompiled kernels for faster startup

If you don't write code

Paste this into your AI agent (Claude Code · Codex · Antigravity)

I want to install the vllm engine on my computer. Use the following commands to perform the installation: uv pip install vllm vllm download-kernels # precompiled kernels for faster startup

Repository: https://github.com/vllm-project/vllm. Follow the official instructions in the README of this repository for setup and execution.

Before starting, check which OS we are on and if the necessary tools (e.g., Python, Node.js, Git, Docker) are installed. Install any missing ones step-by-step according to this system.

Create a new, empty folder to work in. Do not touch any files outside this folder and do not delete anything. Tell me what you are going to do in one sentence before running each command.

Tell me what I should see on the screen at the end of each step. If an error occurs, I will paste the error message to you exactly as it is, and we will fix it together.

Related dictionary terms

Who it is forIt is intended for developers who want to run large language models quickly and efficiently on their own server.
LicenseApache-2.0

Links

TreScout did not build this tool · we found it in GitHub trends and wrote it up. This page describes the repository as of 2026-10-21: The star count and our text belong to that day, the repository may have changed since. Check the repository link for the current state. The introductory text was prepared with AI from the repository's own documentation, and this page was machine-translated from the Turkish original · the Turkish version prevails.