llmfit: Check Which LLMs Run on Your Hardware

Picking a local large language model often comes down to guesswork about whether a given model will fit in available memory and run at a usable speed. llmfit is a terminal tool that aims to remove that uncertainty. According to its repository at https://github.com/AlexsJones/llmfit, it detects the host hardware and then scores each model across quality, speed, fit, and context dimensions to show which ones will actually run well on that machine.

What the tool does

llmfit right-sizes LLM models to a system’s RAM, CPU, and GPU. It ships with an interactive terminal UI (TUI) as the default mode and also offers a classic CLI mode. The project describes support for multi-GPU setups, Mixture-of-Experts (MoE) architectures, dynamic quantization selection, speed estimation, and local runtime providers including Ollama, llama.cpp, MLX, Docker Model Runner, and LM Studio.

In the default TUI, the system’s specs (CPU, RAM, GPU name, VRAM, and backend) are shown at the top of the screen. Models appear in a scrollable table sorted by a composite score. Each row reports the model’s score, estimated tokens per second, the best quantization for the detected hardware, the run mode, memory usage, and a use-case category. The interface uses Vim-inspired modes, with navigation, search, and filtering driven by keyboard shortcuts.

llmfit is written in Rust and is distributed under the MIT license. The README documents installation through several package managers, including Scoop on Windows, Homebrew and MacPorts on macOS and Linux, and uv or pip as a Python package. It can also be run via Docker, where the container prints JSON output from its recommend command that can be queried further with tools such as jq.

Planning and simulating hardware

Beyond reporting what fits the current machine, llmfit includes a Plan mode that inverts the usual question. Instead of asking what fits the existing hardware, Plan mode estimates what hardware a given model configuration would require, showing minimum and recommended VRAM, RAM, and CPU cores, along with feasible run paths such as GPU, CPU offload, or CPU-only.

A separate Hardware Simulation feature lets users override RAM, VRAM, and CPU core count to see which models would fit on different target hardware. When simulation is active, all model scores, fit levels, and speed estimates are recalculated against the simulated specifications, and a badge indicates that the displayed results no longer reflect the real detected hardware.

Real-world benchmark data

The tool also includes a Community Leaderboard view. Rather than relying only on llmfit’s theoretical speed estimates, this view surfaces measured performance data from other users running similar hardware, including reported tokens per second, time to first token, and peak VRAM usage. The README states this data is sourced from a community benchmark database and that public benchmarks are available without authentication, while fuller access uses an API key.

For anyone weighing which open-source model to download, or comparing what a potential hardware upgrade might allow, llmfit consolidates these estimates and measurements into a single terminal interface instead of a series of trial-and-error downloads.