Skip to content
Navigation menu
Search
Powered by Algolia
Search
Log in
Create account
DEV Community
Close
#
vllm
Follow
Hide
Posts
Left menu
đź‘‹
Sign in
for the ability to sort posts by
relevant
,
latest
, or
top
.
Right menu
Building a vLLM Container Image
Sanskriti Harmukh
Sanskriti Harmukh
Sanskriti Harmukh
Follow
for
Vultr
Nov 20
Building a vLLM Container Image
#
docker
#
vllm
#
llm
#
containers
Comments
Add Comment
4 min read
Moderation
SGLang vs vLLM: what is really different inside, and how to compare them fairly
RONI DAS
RONI DAS
RONI DAS
Follow
Nov 20
SGLang vs vLLM: what is really different inside, and how to compare them fairly
#
vllm
#
llm
#
machinelearning
#
ai
Comments
Add Comment
8 min read
Moderation
vLLM vs Ollama: which should you use to serve an LLM?
Viswaretas Kotra
Viswaretas Kotra
Viswaretas Kotra
Follow
Nov 19
vLLM vs Ollama: which should you use to serve an LLM?
#
llm
#
vllm
#
ollama
#
ai
Comments
Add Comment
10 min read
Moderation
vLLM 0.34 Stops Reserving 8% of Your GPU: Extensible KV Cache, the Block Size 64 Default, and What to Delete From Your Launch Flags
AI Tech News
AI Tech News
AI Tech News
Follow
Nov 18
vLLM 0.34 Stops Reserving 8% of Your GPU: Extensible KV Cache, the Block Size 64 Default, and What to Delete From Your Launch Flags
#
ai
#
opensource
#
llm
#
vllm
Comments
Add Comment
14 min read
Moderation
Five Serving Engines on a Lambda GPU: Gemma 4 E2B Under vLLM, SGLang, llama.cpp, PyTorch and JAX
xbill
xbill
xbill
Follow
for
AWS Community Builders
Nov 19
Five Serving Engines on a Lambda GPU: Gemma 4 E2B Under vLLM, SGLang, llama.cpp, PyTorch and JAX
#
aws
#
serverless
#
gemma
#
vllm
4
 reactions
Comments
1
 comment
11 min read
Moderation
How do you host your own LLM behind an OpenAI-style API?
Viswaretas Kotra
Viswaretas Kotra
Viswaretas Kotra
Follow
Nov 18
How do you host your own LLM behind an OpenAI-style API?
#
api
#
llm
#
vllm
#
selfhosted
Comments
1
 comment
9 min read
Moderation
Inside vLLM's Scheduler: Continuous Batching, Chunked Prefill, and Who Waits
Daniel Sam Pete Thiyagu
Daniel Sam Pete Thiyagu
Daniel Sam Pete Thiyagu
Follow
Nov 17
Inside vLLM's Scheduler: Continuous Batching, Chunked Prefill, and Who Waits
#
vllm
#
llm
#
googlecloud
#
agents
Comments
Add Comment
10 min read
Moderation
Tuning vLLM on NVIDIA vGPUs
Florian Lutz
Florian Lutz
Florian Lutz
Follow
Nov 17
Tuning vLLM on NVIDIA vGPUs
#
ai
#
vllm
#
kubernetes
#
nvidia
Comments
Add Comment
8 min read
Moderation
llm-d v1.0 on Kubernetes: KV-Cache Routing, Disaggregation, and Production Readiness
Jangwook Kim
Jangwook Kim
Jangwook Kim
Follow
Nov 17
llm-d v1.0 on Kubernetes: KV-Cache Routing, Disaggregation, and Production Readiness
#
llmd
#
kubernetes
#
vllm
#
inferenceserving
Comments
Add Comment
10 min read
Moderation
Serving 753B MoE Across Nodes: vLLM & RDMA Limits
[email protected]
[email protected]
[email protected]
Follow
Nov 16
Serving 753B MoE Across Nodes: vLLM & RDMA Limits
#
vllm
#
moe
#
rdma
#
dgx
Comments
Add Comment
1 min read
Moderation
Day 5: Local AI Runtimes — Deploying Ollama and vLLM for Bare-Metal Execution
Suresh Kumar Pallapothu
Suresh Kumar Pallapothu
Suresh Kumar Pallapothu
Follow
Nov 12
Day 5: Local AI Runtimes — Deploying Ollama and vLLM for Bare-Metal Execution
#
ai
#
ollama
#
vllm
#
aiintermediateseries
Comments
Add Comment
3 min read
Moderation
Prefill vs Decode: Why Splitting Them Wins
Daniel Sam Pete Thiyagu
Daniel Sam Pete Thiyagu
Daniel Sam Pete Thiyagu
Follow
Nov 13
Prefill vs Decode: Why Splitting Them Wins
#
vllm
#
llm
#
googlecloud
#
machinelearning
1
 reaction
Comments
1
 comment
10 min read
Moderation
vLLM 0.33, Plainly: CUDA 13, Model Runner V2, and What Breaks on Upgrade
Daniel Sam Pete Thiyagu
Daniel Sam Pete Thiyagu
Daniel Sam Pete Thiyagu
Follow
Nov 9
vLLM 0.33, Plainly: CUDA 13, Model Runner V2, and What Breaks on Upgrade
#
vllm
#
llm
#
googlecloud
#
machinelearning
Comments
Add Comment
9 min read
Moderation
llm-compressor for vLLM: The FP8/AWQ Deployment Tradeoff
Jangwook Kim
Jangwook Kim
Jangwook Kim
Follow
Nov 10
llm-compressor for vLLM: The FP8/AWQ Deployment Tradeoff
#
quantization
#
vllm
#
fp8
#
awq
Comments
Add Comment
9 min read
Moderation
Day-0 Model Support Is a Fork: What 26 NVIDIA Dynamo Snapshot Builds Say About Pinning Your LLM Serving Stack
AI Tech News
AI Tech News
AI Tech News
Follow
Nov 9
Day-0 Model Support Is a Fork: What 26 NVIDIA Dynamo Snapshot Builds Say About Pinning Your LLM Serving Stack
#
ai
#
opensource
#
llm
#
vllm
1
 reaction
Comments
1
 comment
15 min read
Moderation
đź‘‹
Sign in
for the ability to sort posts by
relevant
,
latest
, or
top
.
We're a place where coders share, stay up-to-date and grow their careers.
Log in
Create account