CompileIQ matmul POC

An inference-only Triton FP16 matmul with GPU/shape-specific CompileIQ ACFs. matmul(a, b) accepts contiguous matrices on the same CUDA device, with A[M,K] @ B[K,N] and positive M/N/K divisible by 32/64/32.

Usage

For the original tested environment, install CUDA-enabled PyTorch 2.10.0, Triton 3.6.0, kernels>=0.17.1, and nvidia-cuda-nvcc==13.3.73. The exact PTXAS patch is not required; 13.3.33 also passed. On ARM64, install torch==2.10.0+cu130 from https://download.pytorch.org/whl/cu130; the plain PyPI wheel is CPU-only. Point both environment variables below at your installed PTXAS before starting Python:

export TRITON_PTXAS_PATH=/absolute/path/to/ptxas-13.3.73
export TRITON_PTXAS_BLACKWELL_PATH="$TRITON_PTXAS_PATH"

import torch
from kernels import get_kernel

kernel = get_kernel("sayakpaul/matmul-compileiq", version=0, trust_remote_code=True)
a = torch.rand((4096, 4096), device="cuda", dtype=torch.float16) - 0.5
b = torch.rand_like(a) - 0.5
c = kernel.matmul(a, b)
torch.testing.assert_close(c, a @ b, atol=1e-2, rtol=0)

For the pip-installed compiler, the binary is usually at <venv>/lib/python3.12/site-packages/nvidia/cu13/bin/ptxas (adjust the Python version to match the environment). No CompileIQ installation is needed to load this kernel.

Selection and portability

Four independently tuned ACFs are indexed by GPU model + (M,N,K): NVIDIA GB10 and NVIDIA A100-SXM4-80GB, each at 2048³ and 4096³. Compute capability is not a second lookup key. Tiles remain 32×64×32, four warps, three stages. Version 0 is an experimental POC.

Set both TRITON_PTXAS_PATH and TRITON_PTXAS_BLACKWELL_PATH to PTXAS 13.3 before importing Triton. CompileIQ is only used offline and is not a runtime dependency. The ACF is applied at JIT compilation, with normal Triton caching.

Each ACF records validated Torch/Triton/PTXAS major.minor tuples separately from its lookup key. Full tested versions are provenance, not exact patch restrictions. A100 ACFs also passed with Torch 2.11 / Triton 3.6 and the matched Torch 2.12.1 / Triton 3.7.1 pair. PTXAS 13.3.33 and 13.3.73 worked on both GPUs; 13.4.92 rejected the 13.3 ACFs. This is measured coverage, not a guarantee for every untested patch.

Unknown GPU names, shapes or release tuples warn and run the same kernel without controls, rather than raising a version mismatch error. COMPILEIQ_DISABLE_ACF=1 forces this baseline path. Checksums still detect corrupt packaged ACFs. CompileIQ is not needed at runtime.

See the source repository's portability report for the complete experiments and compatibility policy.

Measured ACF speedup

For the original GB10 4096³ configuration, the winning ACF reduced measured kernel latency by 0.765%, equivalent to a 1.0077× speedup or about 32 microseconds per 4096×4096 matmul.

The expanded matrix uses each GPU/shape's own winner:

Latency reduction is (baseline - ACF) / baseline; speedup is baseline / ACF. The comparison uses the same kernel, input shape, dtype, launch configuration and compiler. The baseline is this illustrative Triton matmul compiled without controls, rather than PyTorch/cuBLAS or another optimized GEMM implementation.

CompileIQ searched five generations and evaluated 136 candidates. We then remeasured the saved winner independently on September 22, 2026, using ten fresh input seeds and alternating baseline/ACF measurement order. Each round used triton.testing.do_bench with 100 ms warmup, 1000 ms measurement and mean latency reporting; the table gives the median of those ten round means. The ACF was faster in nine of ten rounds. Every tuned output matched the PyTorch reference exactly for these validation inputs.

These timings cover direct launches of the compiled kernel with preallocated output, excluding JIT compilation, output allocation and the packaged Torch operator's dispatch overhead. GPU clocks were unlocked and latency varied between rounds. The small measured gain is provisional and specific to the validated configuration; it does not establish an end-to-end speedup for kernel.matmul or a benefit on other hardware or software versions.

Licenses

The Python kernel code follows the repository's Apache-2.0 license. The binary ACF is a CompileIQ Software Output under the included NVIDIA license at controls/compileiq-license.txt. ACF output copyright: © NVIDIA Corporation, 2026.