inference-optimisation-skill
inference-optimisation is an MIT-licensed agent skill that guides Claude Code or another agent through tuning a local LLM server's speed (single-user and concurrent decode, speculative decoding, prefill, KV cache size and startup time) with benchmarks, controls and a long-context stability gate.
Why it matters
For people self-hosting open models on their own GPUs who want more tokens per second without guessing at flags: it makes the agent compute a theoretical ceiling first, change one thing at a time, and prove results against controls.
Highlights
- Workflow: compute the theoretical ceiling, change one thing at a time, measure against interleaved controls, log every result including failures, gate on long-context retrieval, then qualify a clean pinned build.
- Ships scripts for ceilings (ceiling.py), benchmark runs and a results ledger (bench.py), interleaved A/B queues (queue.sh) and real memory footprint on unified-memory GPUs (mem.sh).
- The makers report results such as Intel Arc B390 laptop decode going from 43.6 to 53 tok/s and 256K prefill from 297 to 628 tok/s on a 35B model; their linked repos hold the methods and raw data.unverified
- Works with Claude Code (copy the skill folder into ~/.claude/skills) and any agent that loads SKILL.md-style skills; other models can follow PROMPT.md.
- The README stresses that single-run gains are often noise (±5–10%) and asks for medians and full-context stability checks before calling a setup best.
Taken from the project's own README, docs or launch page. Claims about speed or accuracy are the maker's unless stated otherwise.
Worth knowing: The speedup numbers are the makers' own measurements on specific hardware and models; your results will differ. Tuning runs can take many hours of benchmarking.
Quick facts
| Pricing | Free |
|---|---|
| Free or paid | Free (open source) |
| License | MIT |
| Language | Python and shell scripts (Python 3) |
| Platform | Any hardware running a local LLM server (examples cover Intel Arc, RTX 3090 and H200 with vLLM, SGLang and similar engines) |
| GitHub stars | ★ 602 (as of 2026-11-17) |
| Maker | sybil-solutions (GitHub) |
Links
Related ranking
See how established tools compare in Best AI Coding Tools for Developers. New releases aren't scored until we review them against the rubric.
What we couldn't verify
- highlights[2] (benchmark numbers are the makers' claims)
Checked 2026-11-17 from the tool's own pages. Prices and star counts change, so confirm on the official site. Listing here is not a ranking or an endorsement.
One email a week on AI that saves business owners time
Every Friday: one tool pick with the business use case, what changed in tools you already use, and the week's AI news in plain English. No spam.
Free. Unsubscribe any time. See our privacy policy.