DEV Community

InferenceKey

Organization Settings Admin

Open models on your own GPUs: experiments, measurements and the exact commands behind them. Full write-ups at inferencekey.com

inferencekey.com Joined Joined on 
Sizing a looped LLM: SenseNova-Loop-9B stores 32 layers, runs 56, and fits a 20 GB card

Sizing a looped LLM: SenseNova-Loop-9B stores 32 layers, runs 56, and fits a 20 GB card

Comments
4 min read
Running a dense 65B coder on one GPU: what memory bandwidth and 2-bit files cost you

Running a dense 65B coder on one GPU: what memory bandwidth and 2-bit files cost you

Comments
4 min read
Building on LFM3-Omni-2B: a silent-video trap, a text-only fast server and what to measure

Building on LFM3-Omni-2B: a silent-video trap, a text-only fast server and what to measure

Comments
4 min read
Running a dense 33B agent model on one 24 GB GPU: the offload trap and the empty reply

Running a dense 33B agent model on one 24 GB GPU: the offload trap and the empty reply

Comments
5 min read
Clover-35B-A3B on llama.cpp: what "AR mode only" means, and a GGUF that answers in Gs

Clover-35B-A3B on llama.cpp: what "AR mode only" means, and a GGUF that answers in Gs

Comments
4 min read
Serving Hy-SimulMT live on one 48 GB GPU: the startup spike, the FP8 trap, what to measure

Serving Hy-SimulMT live on one 48 GB GPU: the startup spike, the FP8 trap, what to measure

1
Comments 4
5 min read
Serving Groot-19B-A3B at 256K on one RTX 5090: the memory budget and the thinking trap

Serving Groot-19B-A3B at 256K on one RTX 5090: the memory budget and the thinking trap

Comments
5 min read
Parsing a whole PDF in one request with Unlimited OCR 2: the crop_mode setting and a silent cut

Parsing a whole PDF in one request with Unlimited OCR 2: the crop_mode setting and a silent cut

Comments
4 min read
loading...