Reducing CUDA Binary Bloat via Kernel Consolidation

Template-based CUDA libraries can generate enormous numbers of kernel variants, and compiling all of them produces large binaries and long build times. NVIDIA’s CUTLASS, a header-only template library for GEMM and related operations, documents several ways to control how many kernels get built so that binary size stays manageable.

Why Template Instantiation Causes Bloat

CUTLASS expresses GPU operations as C++ templates parameterized over data types, math instructions, and layouts. Each combination compiles to a distinct kernel. The project documentation at https://github.com/NVIDIA/cutlass warns that building every available kernel produces “tens of thousands” of kernels with “long build times,” and that doing so “would also result in a large binary size and on some platforms linker to fail on building the library.”

Setting the build option CUTLASS_LIBRARY_KERNELS=all triggers this full instantiation. Because the combinatorial space is so large, the library does not default to that behavior.

Filtering Which Kernels Get Built

Rather than instantiating everything, CUTLASS defaults to a limited set. By default “only one tile size is instantiated for each data type, math instruction, and layout,” which keeps the standard build from exploding in size.

For builds that need specific kernels, the documentation recommends generating only a subset. It states it is “highly recommended to generate only a subset of kernels,” and supports filtering through a comma-delimited list of kernel names with wildcard characters. For example, a build can request a family of half-precision tensor-op GEMM kernels:

-DCUTLASS_LIBRARY_KERNELS=cutlass_tensorop_s*gemm_f16_*_nt_align8

Or narrow all the way down to a single SGEMM kernel:

-DCUTLASS_LIBRARY_KERNELS=cutlass_simt_sgemm_128x128_8x2_nn_align1

Selecting only the kernels an application actually uses is the primary mechanism for keeping the compiled library small.

Limiting Target Architectures

A second lever controls how many GPU architectures the kernels are compiled for. By default CUTLASS builds for several compute capabilities, listed in the documentation as “5.0, 6.0, 6.1, 7.0, 7.5, 8.0, 8.6, 8.9, and 9.0.” Each additional architecture multiplies the amount of generated machine code.

The build can restrict this with CUTLASS_NVCC_ARCHS. Passing a single architecture, such as -DCUTLASS_NVCC_ARCHS=80, is described as a way “to reduce compile time,” and compiling for fewer targets also cuts down the volume of emitted code.

Avoiding the Build Entirely

For projects that consume CUTLASS rather than ship its precompiled library, the documentation notes that CUTLASS is “a header-only template library and does not need to be built to be used by other projects.” In that model, only the specific templates a project instantiates end up in its own binary, which avoids carrying unused kernel variants altogether.

The documentation also points to CUTLASS 4 Python DSLs as offering “orders of magnitude faster compile times” compared with the C++ template approach, an alternative path for teams where long compilation is a bottleneck.

Choosing an Approach

The common thread across these techniques is selectivity. Kernel name filtering, architecture targeting, and header-only consumption all reduce binary size by avoiding the instantiation of kernels that an application will never call. For teams shipping GPU code to size-constrained environments, narrowing the build to the kernels and architectures actually in use is the documented way to keep CUDA binaries lean.