Home / New releases / inference-optimisation-skill
New tool2026-11-16via GitHubAI Coding Tools for Developers

inference-optimisation-skill

inference-optimisation is an MIT-licensed agent skill that guides Claude Code or another agent through tuning a local LLM server's speed (single-user and concurrent decode, speculative decoding, prefill, KV cache size and startup time) with benchmarks, controls and a long-context stability gate.

Free (open source)MITPython and shell scripts (Python 3)★ 602 (as of 2026-11-17)by sybil-solutions (GitHub)

Why it matters

For people self-hosting open models on their own GPUs who want more tokens per second without guessing at flags: it makes the agent compute a theoretical ceiling first, change one thing at a time, and prove results against controls.

Highlights

Taken from the project's own README, docs or launch page. Claims about speed or accuracy are the maker's unless stated otherwise.

Try it if: You run a local model with vLLM, SGLang or llama.cpp and want your agent to squeeze out more speed with measured, reproducible changes.

Worth knowing: The speedup numbers are the makers' own measurements on specific hardware and models; your results will differ. Tuning runs can take many hours of benchmarking.

Quick facts

PricingFree
Free or paidFree (open source)
LicenseMIT
LanguagePython and shell scripts (Python 3)
PlatformAny hardware running a local LLM server (examples cover Intel Arc, RTX 3090 and H200 with vLLM, SGLang and similar engines)
GitHub stars★ 602 (as of 2026-11-17)
Makersybil-solutions (GitHub)

Links

Related ranking

See how established tools compare in Best AI Coding Tools for Developers. New releases aren't scored until we review them against the rubric.

What we couldn't verify

Checked 2026-11-17 from the tool's own pages. Prices and star counts change, so confirm on the official site. Listing here is not a ranking or an endorsement.