Ubuntu Inference Snaps: Packaged Local AI Models

Canonical maintains inference snaps, packaged generative AI models built to run on local hardware. According to the official documentation at https://documentation.ubuntu.com/inference-snaps/, each inference snap automatically detects the host machine’s hardware and installs the runtime and model weight optimizations that best match its capabilities. The project is described as a member of the Ubuntu family, is open source, and welcomes community contributions.

How The Snaps Work

A snap is Canonical’s packaging format for delivering software as a self-contained unit separate from the underlying operating system. Inference snaps apply that model to generative AI workloads. Rather than asking a user to choose a runtime and download matching model weights, the snap inspects the machine and selects what fits. The documentation notes that inference snaps make use of available CPU, GPU, or NPU hardware, and that the system supports multiple engines with documented steps for switching between them.

Canonical’s reference material includes an “Available snaps” page listing the models that ship in this format, along with an engine manifest and guidance for installing drivers. The documentation also covers running inference snaps under Windows Subsystem for Linux (WSL), not only on native Ubuntu.

Installation follows the standard snap workflow. In a walkthrough published on the Ubuntu blog covering Ubuntu Core 26 (https://ubuntu.com/blog/ubuntu-core-26-ai-box), Canonical demonstrates installing an inference snap with sudo snap install and checking it with a status subcommand. Configuration is applied through snap options rather than by manually editing system files. In that example the running engine reported was CPU.

The Local API

After installation, an inference snap exposes a local, OpenAI-compatible API. The documentation frames this as a standardized and reliable local API that lets developers add AI features without complex hardware-specific tuning. Because the interface follows the OpenAI convention, the same client code that targets that API style can point at a local snap instead.

The Ubuntu Core 26 walkthrough shows the API served on a localhost port with an /v1 base path and a /chat/completions endpoint that accepts a JSON messages payload, the same shape used by many hosted chat services. A separate web UI port is also available. Settings such as the host binding are changed through snap options, for example exposing the service beyond localhost when needed.

Tooling And Management

Canonical lists several integrations for inference snaps, including Open WebUI, OpenCode, OpenShell, Visual Studio Code, and JetBrains IDEs. A command line interface is documented for direct interaction, alongside guides for service management, viewing logs, setting environment variables, and general troubleshooting.

Because inference snaps are delivered as snaps, they inherit that format’s update behavior. The Ubuntu Core 26 article describes transactional updates with rollback and fleet management through Landscape, which is relevant for deploying the same model configuration across multiple machines.

Why Local Packaging Matters

The appeal of this approach is that the packaging absorbs the hardware-matching work that local inference usually requires. A developer installs a snap, and the snap decides which runtime and weights suit the device. Keeping inference local also keeps prompts and data on the machine rather than sending them to an external service.

The documentation is the authoritative place for the current list of supported models, engines, and configuration options, since those details change as the project evolves. For anyone evaluating local AI on Ubuntu, the inference snaps documentation and the Ubuntu Core 26 blog post together show both the concept and a concrete end-to-end setup.