Heterogeneous acceleration for AI models
Ktransformers offers a flexible framework that supports heterogeneous hardware optimizations in the inference and fine-tuning processes of large language models. This structure aims to increase model performance by using different hardware resources efficiently.
Updates
- September 15, 2026: Stars 19,255 → 19,517, latest release v0.7.1 (September 15, 2026).
- August 18, 2026: Stars 19,145 → 19,255, latest release v0.7.0 (August 17, 2026).
- August 2, 2026: Stars 18,491 → 19,145, latest release v0.6.4 (July 23, 2026).
What you get
- Efficient work by using processor and graphics card resources together
- Low hardware requirements in large-scale MoE models
- Integrated inference and fine-tuning support with popular frameworks
Installation
cd kt-kernel
pip install .If you don't write code
I run large language models heterogeneously on CPU and GPU using ktransformers. How can I integrate the kt-kernel library into my system to use my hardware resources efficiently and optimize model inference performance, and what are the optimal configuration settings for MoE models?
Related dictionary terms
Links
TreScout did not build this tool · we found it in GitHub trends and wrote it up. This page describes the repository as of 2026-07-20: The star count and our text belong to that day, the repository may have changed since. Check the repository link for the current state. This page was machine-translated from the Turkish original · the Turkish version prevails.