热门 GitHub

A high-throughput and memory-efficient inference and serving engine for LLMs

Fully open reproduction of DeepSeek-R1

Cost-efficient and pluggable Infrastructure components for GenAI inference

A framework for efficient model inference with omni-modality models

System Level Intelligent Router for Mixture-of-Models at Cloud, Data Center and Edge

Community maintained hardware plugin for vLLM on Ascend