Skip to content
Navigation menu
Search
Powered by Algolia
Search
Log in
Create account
DEV Community
Close
#
quantization
Follow
Hide
Posts
Left menu
👋
Sign in
for the ability to sort posts by
relevant
,
latest
, or
top
.
Right menu
Unsloth Dynamic V3.1: Qwen3.8-27B Local Agent Benchmarks
[email protected]
[email protected]
[email protected]
Follow
Nov 17
Unsloth Dynamic V3.1: Qwen3.8-27B Local Agent Benchmarks
#
quantization
#
llmserving
#
localai
#
qwen
Comments
Add Comment
1 min read
Moderation
How SteerDistill Shrinks 30‑Billion‑Parameter LLMs to 3 GB Without Losing Accuracy
amrit
amrit
amrit
Follow
Nov 12
How SteerDistill Shrinks 30‑Billion‑Parameter LLMs to 3 GB Without Losing Accuracy
#
llm
#
quantization
#
aiagents
#
ai
Comments
Add Comment
6 min read
Moderation
Cutting 64 of 256 Experts from a GGUF by Byte Copy: What the Headers Say About GLM-5.3-SLIM on One B200
GINIGEN AI
GINIGEN AI
GINIGEN AI
Follow
Nov 11
Cutting 64 of 256 Experts from a GGUF by Byte Copy: What the Headers Say About GLM-5.3-SLIM on One B200
#
ai
#
machinelearning
#
edgeai
#
quantization
Comments
Add Comment
13 min read
Moderation
Day 4: Optimizing Model Weights — Quantization Basics (GGUF, AWQ) for Efficient Inference
Suresh Kumar Pallapothu
Suresh Kumar Pallapothu
Suresh Kumar Pallapothu
Follow
Nov 10
Day 4: Optimizing Model Weights — Quantization Basics (GGUF, AWQ) for Efficient Inference
#
ai
#
quantization
#
aiintermediateseries
Comments
Add Comment
4 min read
Moderation
llm-compressor for vLLM: The FP8/AWQ Deployment Tradeoff
Jangwook Kim
Jangwook Kim
Jangwook Kim
Follow
Nov 10
llm-compressor for vLLM: The FP8/AWQ Deployment Tradeoff
#
quantization
#
vllm
#
fp8
#
awq
Comments
Add Comment
9 min read
Moderation
2비트라는 이름의 GGUF가 3비트 이름의 파일보다 컸습니다: 공개 Darwin-180B 양자화 파일 6개를 헤더까지 열어 보니
Vidraft_lab
Vidraft_lab
Vidraft_lab
Follow
Nov 8
2비트라는 이름의 GGUF가 3비트 이름의 파일보다 컸습니다: 공개 Darwin-180B 양자화 파일 6개를 헤더까지 열어 보니
#
ai
#
machinelearning
#
llm
#
quantization
Comments
Add Comment
4 min read
Moderation
One Omni Model, Two GGUF Files: What LFM3-Omni-2B Really Costs in RAM, from the Tensor Headers Up
GINIGEN AI
GINIGEN AI
GINIGEN AI
Follow
Nov 8
One Omni Model, Two GGUF Files: What LFM3-Omni-2B Really Costs in RAM, from the Tensor Headers Up
#
ai
#
machinelearning
#
edgeai
#
quantization
Comments
Add Comment
12 min read
Moderation
Quantization, Explained: From FP32 to INT4, and What Each Precision Costs You
Aleksei Romanov
Aleksei Romanov
Aleksei Romanov
Follow
for
g factor
Nov 5
Quantization, Explained: From FP32 to INT4, and What Each Precision Costs You
#
quantization
#
machinelearning
#
llm
#
ai
Comments
Add Comment
11 min read
Moderation
A 763B MoE at "2 Bits": Where the Bytes Really Go in the GSQ-RCO GGUFs, and Why 100% Recovery Sits Inside the Noise
GINIGEN AI
GINIGEN AI
GINIGEN AI
Follow
Nov 2
A 763B MoE at "2 Bits": Where the Bytes Really Go in the GSQ-RCO GGUFs, and Why 100% Recovery Sits Inside the Noise
#
ai
#
machinelearning
#
edgeai
#
quantization
Comments
Add Comment
15 min read
Moderation
FP8 Cut Our TPOT by 25%. At High Concurrency, It Also Changed the Answers.
Huilin Ye
Huilin Ye
Huilin Ye
Follow
Oct 30
FP8 Cut Our TPOT by 25%. At High Concurrency, It Also Changed the Answers.
#
llm
#
vllm
#
quantization
#
benchmarking
Comments
Add Comment
3 min read
Moderation
Why T Quant is the Secret Weapon for Mobile AI in 2026
amrit
amrit
amrit
Follow
Oct 26
Why T Quant is the Secret Weapon for Mobile AI in 2026
#
diffusionllm
#
quantization
#
mobileai
#
ai
Comments
Add Comment
6 min read
Moderation
LFM2.5-24B-A2B on Your Own Hardware: The RAM, KV Cache and Bandwidth Budget
GINIGEN AI
GINIGEN AI
GINIGEN AI
Follow
Oct 19
LFM2.5-24B-A2B on Your Own Hardware: The RAM, KV Cache and Bandwidth Budget
#
ai
#
machinelearning
#
edgeai
#
quantization
Comments
Add Comment
11 min read
Moderation
How One Startup Cut Diffusion‑LLM Inference Costs by 45 % – No Calibration Needed
amrit
amrit
amrit
Follow
Oct 12
How One Startup Cut Diffusion‑LLM Inference Costs by 45 % – No Calibration Needed
#
ai
#
quantization
#
diffusionllm
Comments
Add Comment
6 min read
Moderation
Model Distillation & Quantization: Running 8B Parameter Models at a Fraction of Cloud Cost
Muhammad Tahir
Muhammad Tahir
Muhammad Tahir
Follow
Oct 12
Model Distillation & Quantization: Running 8B Parameter Models at a Fraction of Cloud Cost
#
modeldistillation
#
quantization
#
vllm
#
awq
Comments
Add Comment
10 min read
Moderation
Four GPUs, One Quantization Ladder: Gemma 4 E2B From a 4 GB Laptop to an NVIDIA L4
xbill
xbill
xbill
Follow
for
Google Developer Experts
Oct 30
Four GPUs, One Quantization Ladder: Gemma 4 E2B From a 4 GB Laptop to an NVIDIA L4
#
gemma
#
llamacpp
#
quantization
#
nvidia
13
reactions
Comments
2
comments
11 min read
Moderation
👋
Sign in
for the ability to sort posts by
relevant
,
latest
, or
top
.
We're a place where coders share, stay up-to-date and grow their careers.
Log in
Create account