Pinned Loading

1. vllm-project/vllm vllm-project/vllm Public

   A high-throughput and memory-efficient inference and serving engine for LLMs

   Python 84.9k 18.7k

2. ai-dynamo/dynamo ai-dynamo/dynamo Public

   A Datacenter Scale Distributed Inference Serving Framework

   Rust 7.4k 1.3k

3. llm-d/llm-d-kv-cache llm-d/llm-d-kv-cache Public

   Distributed KV cache scheduling & offloading libraries

   Go 157 143