DEV Community

#vllm

Posts

đź‘‹ Sign in for the ability to sort posts by relevant, latest, or top.
Building a vLLM Container Image

Building a vLLM Container Image

Comments
4 min read
SGLang vs vLLM: what is really different inside, and how to compare them fairly

SGLang vs vLLM: what is really different inside, and how to compare them fairly

Comments
8 min read
vLLM vs Ollama: which should you use to serve an LLM?

vLLM vs Ollama: which should you use to serve an LLM?

Comments
10 min read
vLLM 0.34 Stops Reserving 8% of Your GPU: Extensible KV Cache, the Block Size 64 Default, and What to Delete From Your Launch Flags

vLLM 0.34 Stops Reserving 8% of Your GPU: Extensible KV Cache, the Block Size 64 Default, and What to Delete From Your Launch Flags

Comments
14 min read
Five Serving Engines on a Lambda GPU: Gemma 4 E2B Under vLLM, SGLang, llama.cpp, PyTorch and JAX

Five Serving Engines on a Lambda GPU: Gemma 4 E2B Under vLLM, SGLang, llama.cpp, PyTorch and JAX

4
Comments 1
11 min read
How do you host your own LLM behind an OpenAI-style API?

How do you host your own LLM behind an OpenAI-style API?

Comments 1
9 min read
Inside vLLM's Scheduler: Continuous Batching, Chunked Prefill, and Who Waits

Inside vLLM's Scheduler: Continuous Batching, Chunked Prefill, and Who Waits

Comments
10 min read
Tuning vLLM on NVIDIA vGPUs

Tuning vLLM on NVIDIA vGPUs

Comments
8 min read
llm-d v1.0 on Kubernetes: KV-Cache Routing, Disaggregation, and Production Readiness

llm-d v1.0 on Kubernetes: KV-Cache Routing, Disaggregation, and Production Readiness

Comments
10 min read
Serving 753B MoE Across Nodes: vLLM & RDMA Limits

Serving 753B MoE Across Nodes: vLLM & RDMA Limits

Comments
1 min read
Day 5: Local AI Runtimes — Deploying Ollama and vLLM for Bare-Metal Execution

Day 5: Local AI Runtimes — Deploying Ollama and vLLM for Bare-Metal Execution

Comments
3 min read
Prefill vs Decode: Why Splitting Them Wins

Prefill vs Decode: Why Splitting Them Wins

1
Comments 1
10 min read
vLLM 0.33, Plainly: CUDA 13, Model Runner V2, and What Breaks on Upgrade

vLLM 0.33, Plainly: CUDA 13, Model Runner V2, and What Breaks on Upgrade

Comments
9 min read
llm-compressor for vLLM: The FP8/AWQ Deployment Tradeoff

llm-compressor for vLLM: The FP8/AWQ Deployment Tradeoff

Comments
9 min read
Day-0 Model Support Is a Fork: What 26 NVIDIA Dynamo Snapshot Builds Say About Pinning Your LLM Serving Stack

Day-0 Model Support Is a Fork: What 26 NVIDIA Dynamo Snapshot Builds Say About Pinning Your LLM Serving Stack

1
Comments 1
15 min read
đź‘‹ Sign in for the ability to sort posts by relevant, latest, or top.