Run giant AI models with 4GB VRAM
AirLLM allows large language models with 70 billion parameters to be run on graphics processing units with only 4 GB of video memory (VRAM). This library uses memory optimization techniques to enable the use of high-capacity models with low hardware requirements.
Updates
- September 6, 2026: Stars 33,307 → 33,755, latest release v4.0.0 (September 5, 2026).
- August 31, 2026: Stars 31,598 → 33,307, latest release v3.3.0 (August 28, 2026).
- August 19, 2026: Stars 30,796 → 31,598, latest release v3.2.0 (August 18, 2026).
- August 12, 2026: Stars 29,265 → 30,796, latest release v3.1.0 (July 29, 2026).
What you get
- Possibility to run models with 70B parameters with 4GB VRAM.
- Ability to use 405B Llama3.1 models with 8GB VRAM.
- Up to 3x speed increase with block-based compression.
Installation
pip install airllmIf you don't write code
I want to run a model with 70B parameters using the AirLLM library on my graphics card with low VRAM capacity. I used the pip install airllm command for installation. How can I create the Python code structure needed to load my model and output it with a simple text input, using the AutoModel class? I know I need to make sure I have enough disk space during the process, can you explain the basic steps I need to follow to start the process?
Related dictionary terms
Links
TreScout did not build this tool · we found it in GitHub trends and wrote it up. This page describes the repository as of 2026-06-04: The star count and our text belong to that day, the repository may have changed since. Check the repository link for the current state. This page was machine-translated from the Turkish original · the Turkish version prevails.