PyTorch & CUDASep 24, 2026(Updated Nov 5, 2026)

How to Fix 'fatal error: cuda.h: No such file or directory' in Triton and vLLM on Linux

By B. Cho · Systems & AI Infrastructure Engineer

fatal error: cuda.h: No such file or directory means a particular C/C++ compilation step could not find the CUDA driver development header. It does not prove that the NVIDIA driver is absent or that every PyTorch/Triton installation needs a complete system CUDA Toolkit. A missing nvcc executable, a missing cuda_runtime.h, a PyTorch extension compiled with the wrong ABI, and a JIT cache issue are related but different failures.

The older edition of this article installed several CUDA packages unconditionally, constructed CUDA_HOME by assuming a particular Python site-packages layout, appended exports to ~/.bashrc, and deleted the entire Triton cache before verifying the error. Those changes may be unnecessary or damage a working environment. Use read-only diagnosis first, then make one scoped correction only if a development header is actually missing.

This guide is for a Linux machine with a compatible CUDA-enabled NVIDIA device when running a real GPU test. Our published helper is intentionally CPU-only and cannot prove GPU kernel compilation success.

1. Find out what is actually missing

Collect the active Python, driver and toolkit facts without installing or changing anything:

python3 --version
python3 -m pip show torch triton
nvidia-smi
command -v nvcc || true
printf 'CUDA_HOME=%s\n' "${CUDA_HOME:-<unset>}"
printf 'CPATH=%s\n' "${CPATH:-<unset>}"

The presence of nvidia-smi shows the driver management interface is available, not that CUDA development headers exist. The presence of torch or triton as Python modules proves imports only, not a successful JIT build.

Test common CUDA header paths using a read-only tool

Download and inspect the CUDA header path checker (Python source). It looks in common known CUDA Toolkit and Python NVIDIA package directories. It does not run nvcc, download dependencies, modify your shell profile, or delete any cache.

curl -fsS https://agentic-pulse.dev/tools/cuda_headers_check.py -o cuda_headers_check.py
python3 cuda_headers_check.py --self-test
python3 cuda_headers_check.py

The tool’s four synthetic classifications were tested for this article. It distinguishes a discovered header/compiler pair, header-only, compiler-only and neither. A negative result does not mean cuda.h is absent from every path: it may be installed under a custom mount, sysroot, or platform-specific directory.

For the actual compiler failure, also note the failing build command and the include paths it passed. The compiler’s -I list matters more than a generic claim that one package “must be missing.”

2. Match the error to the right prerequisite

Observed error	Evidence to collect	What not to assume
`fatal error: cuda.h`	Exact failing C/C++ compile command, header paths and CUDA Toolkit layout	Installing a PyTorch wheel automatically provides developer headers
`cuda_runtime.h` missing	Runtime development include path and compiler options	`cuda.h` and `cuda_runtime.h` are the same file
`CUDA_HOME environment variable is not set`	`CUDA_HOME`, `nvcc` path and the extension build’s expected toolkit root	Every Triton workload needs `nvcc` installed globally
`undefined symbol` when importing an extension	PyTorch, CUDA, C++ ABI and extension version	A header installation can fix binary ABI compatibility
JIT kernel fails after an otherwise successful compile	Complete traceback, cache path, driver and GPU compute capability	It was definitely a corrupt cache
`torch.cuda.is_available() == False`	Driver, runtime, actual GPU access	Changing `CPATH` can repair a driver/GPU problem

PyTorch’s C++/CUDA extension documentation notes that compiler discovery uses heuristics and that CUDA_HOME can specify the toolkit root when those heuristics fail.

3. Correct the verified missing header or path

A. Existing system CUDA Toolkit but incorrect include path

Before changing anything, locate the actual header and compiler. These examples examine common locations without attempting to repair them:

for root in "${CUDA_HOME:-}" /usr/local/cuda /usr/local/cuda-12 /usr/local/cuda-13; do
  if [[ -z "$root" ]]; then continue; fi
  for relative in include/cuda.h targets/x86_64-linux/include/cuda.h targets/aarch64-linux/include/cuda.h; do
    if [[ -f "$root/$relative" ]]; then
      printf 'Found %s\n' "$root/$relative"
    fi
  done
done

If your existing toolkit actually resides at /usr/local/cuda and the header is found there, a temporary command-local environment may help an extension build discover it:

# Only after confirming the directory and its headers on your own host.
CUDA_HOME=/usr/local/cuda \
CPATH="/usr/local/cuda/include:${CPATH:-}" \
python3 your_existing_authorized_build_command.py

Replace the final filename with the command that originally failed. Do not blindly add it to ~/.bashrc, /etc/profile.d or a global systemd service. For recent Toolkit layouts, the header can be under a targets/<architecture>/include subtree, so use the directory that actually exists.

B. Development headers are genuinely missing

Use the NVIDIA CUDA installation guide to select a compatible Toolkit and development package for the installed OS, GPU and environment. NVIDIA documents that CUDA Python wheels focus on runtime use and that developer tools may need to be installed separately. Therefore the historical advice “install nvidia-cuda-nvcc-cu12 nvidia-cuda-runtime-cu12 and the problem always disappears” is unsupported.

Do not upgrade the host display driver, change the system-wide CUDA symlink, or mix unrelated CUDA major versions just because a tutorial has a hard-coded 12.6 package list. Those changes require a maintenance plan and compatibility checks.

If your deployment is containerized, select an image that provides development headers only if the affected workload actually compiles code in that container. NVIDIA’s devel image family differs from the slimmer runtime family. Keep images pinned to a tested tag/digest rather than always using latest. Verify that the relevant header is visible inside the same container running the failing build, not merely on the host.

4. Rule out a stale cache without deleting working artifacts

The old guide deleted ~/.triton/cache unconditionally. Avoid that as a first response. If the source error has been fixed and you specifically suspect stale compiled artifacts, use a new isolated cache directory for one authorized test run:

TEST_CACHE=$(mktemp -d)
printf 'Temporary test cache: %s\n' "$TEST_CACHE"
# Run only your existing reviewed Triton test command using this temporary cache.
TRITON_CACHE_DIR="$TEST_CACHE" python3 your_existing_triton_test.py

The directory is created only for this trial, not permanently configured. After reviewing the result, remove the test directory only by an explicit approved cleanup step. Do not delete an existing shared Triton cache with rm -rf or mistake an import check for a compilation check.

Be aware that the exact JIT cache behavior and compilation backend are version-specific. Some Triton code paths do not require the exact host headers that another extension requires. The error-producing compiler invocation is the deciding evidence.

5. A real verification sequence on a supported GPU

After making a single verified correction, validate each level separately:

python -c "import torch, triton" succeeds. This only tests imports.
The needed cuda.h and other headers can be opened along the actual include path of the failed compiler.
Use the official Triton vector-addition tutorial or your minimal affected kernel to trigger a JIT compile on an approved, compatible GPU.
Compare the kernel output against a reference implementation using torch.allclose and record the result, device, driver, CUDA version and package versions.
Only after that, retry the original vLLM/PyTorch workload in a suitable maintenance window.

The official Triton tutorial includes both a minimal JIT kernel and a reference comparison. A passing vector-add test is stronger evidence than an import, but it still does not guarantee every FlashAttention, custom extension or vLLM model will compile.

Our actual testing for this updated article was limited to Python syntax and four synthetic path-status classifications for the linked helper. We did not use an NVIDIA GPU, compile a real Triton kernel, install CUDA packages or mutate an existing cache. The tool is provided to let readers reproduce the relevant prerequisite check locally.

6. When the header exists but the failure persists

Do not repeat installations. Inspect the exact compiler invocation and whether it is running in a Python virtual environment, container, isolated wheel-build environment or different user namespace. Compare sys.prefix, CUDA_HOME, the failing build’s compiler, and the actual include paths. If the failure is an extension ABI mismatch, use PyTorch’s CUDA extension notes rather than changing only environment variables.

For vLLM and PyTorch deployments, also distinguish insufficient GPU memory from missing headers. The vLLM memory troubleshooting guide covers a different failure family.

Bottom line: an exact error traceback plus the location of cuda.h tells you which environment to fix. Repeated broad installations and cache deletion before diagnosis make that task harder.

Primary sources

Corrected and reviewed 2026-11-05; original article URL and first publication date preserved. No actual GPU runtime benchmark claimed.

Written by B. Cho

Systems & AI Infrastructure Engineer

Specializing in GPU inference engines (vLLM, SGLang, Ollama), CUDA memory allocation, and Model Context Protocol (MCP) infrastructure on Linux workstations.