Back DockerFit
Common Error

Fix: PyTorch CUDA Version Mismatch

Resolve the "CUDA version mismatch" or "PyTorch was compiled with CUDA X.X but CUDA Y.Y is installed" error

Error Message

RuntimeError: The NVIDIA driver on your system is too old

or

RuntimeError: CUDA error: no kernel image is available for execution on the device

or

PyTorch was compiled with CUDA 12.1 but CUDA 11.8 is installed

Root Cause

This error occurs when there's a mismatch between:

  • PyTorch CUDA version - The CUDA version PyTorch was compiled against
  • Container CUDA version - The CUDA runtime in your Docker base image
  • Host NVIDIA driver - The driver version on your host machine

Key insight: PyTorch binaries are compiled for specific CUDA versions. Installing PyTorch with pip's default will likely give you a version that doesn't match your setup.

Solution

Use DockerFit to generate a Dockerfile with verified compatible versions:

  1. Check your host driver version: nvidia-smi
  2. Select the matching CUDA version based on your driver (see table below)
  3. Generate a Dockerfile with compatible PyTorch + CUDA combination

CUDA & Driver Compatibility

CUDA Version Min Driver (Linux) Recommended GPU
CUDA 12.4 >=550 H100, A100, L4
CUDA 12.1 >=530 A100, A10G, T4
CUDA 11.8 >=450 T4, V100, RTX 30xx

Generate Fixed Dockerfile

Select your target configuration below to generate a verified Dockerfile:

Configuration

Local GPU or CPU environment

稳定版本，广泛兼容

Requires NVIDIA Driver >=530.30.02
Dockerfile
1# syntax=docker/dockerfile:1
2# ^ Required for BuildKit cache mounts and advanced features
3 
4# Generated by DockerFit (https://tools.eastondev.com/docker)
5# PYTORCH 2.4.1 + CUDA 12.1 | Python 3.11
6# Multi-stage build for optimized image size
7 
8# ==============================================================================
9# Stage 1: Builder - Install dependencies and compile
10# ==============================================================================
11FROM nvidia/cuda:12.1.1-cudnn8-devel-ubuntu22.04 AS builder
12 
13# Build arguments
14ARG DEBIAN_FRONTEND=noninteractive
15 
16# Environment variables
17ENV PYTHONUNBUFFERED=1
18ENV PYTHONDONTWRITEBYTECODE=1
19ENV TORCH_CUDA_ARCH_LIST="8.0;8.6;9.0"
20 
21# Install Python 3.11 from deadsnakes PPA (Ubuntu 22.04)
22RUN apt-get update && apt-get install -y --no-install-recommends \
23    software-properties-common \
24    && add-apt-repository -y ppa:deadsnakes/ppa \
25    && apt-get update && apt-get install -y --no-install-recommends \
26    python3.11 \
27    python3.11-venv \
28    python3.11-dev \
29    build-essential \
30    git
31    && rm -rf /var/lib/apt/lists/*
32 
33# Create virtual environment
34ENV VIRTUAL_ENV=/opt/venv
35RUN python3.11 -m venv $VIRTUAL_ENV
36ENV PATH="$VIRTUAL_ENV/bin:$PATH"
37 
38# Upgrade pip
39RUN pip install --no-cache-dir --upgrade pip setuptools wheel
40 
41# Install PyTorch with BuildKit cache
42RUN --mount=type=cache,target=/root/.cache/pip \
43    pip install torch torchvision torchaudio \
44    --index-url https://download.pytorch.org/whl/cu121
45 
46# Install project dependencies
47COPY requirements.txt .
48RUN --mount=type=cache,target=/root/.cache/pip \
49    pip install -r requirements.txt
50 
51# ==============================================================================
52# Stage 2: Runtime - Minimal production image
53# ==============================================================================
54FROM nvidia/cuda:12.1.1-cudnn8-runtime-ubuntu22.04 AS runtime
55 
56# Labels
57LABEL maintainer="Generated by DockerFit"
58LABEL version="2.4.1"
59LABEL description="PYTORCH 2.4.1 + CUDA 12.1"
60 
61# Environment variables
62ENV PYTHONUNBUFFERED=1
63ENV PYTHONDONTWRITEBYTECODE=1
64ENV NVIDIA_VISIBLE_DEVICES=all
65ENV NVIDIA_DRIVER_CAPABILITIES=compute,utility
66 
67# Install Python 3.11 runtime from deadsnakes PPA (Ubuntu 22.04)
68RUN apt-get update && apt-get install -y --no-install-recommends \
69    software-properties-common \
70    && add-apt-repository -y ppa:deadsnakes/ppa \
71    && apt-get update && apt-get install -y --no-install-recommends \
72    python3.11 \
73    libgomp1
74    && apt-get remove -y software-properties-common \
75    && apt-get autoremove -y \
76    && rm -rf /var/lib/apt/lists/*
77 
78# Create non-root user for security
79ARG USERNAME=appuser
80ARG USER_UID=1000
81ARG USER_GID=$USER_UID
82RUN groupadd --gid $USER_GID $USERNAME \
83    && useradd --uid $USER_UID --gid $USER_GID -m $USERNAME
84 
85# Copy virtual environment from builder
86COPY --from=builder --chown=$USERNAME:$USERNAME /opt/venv /opt/venv
87ENV VIRTUAL_ENV=/opt/venv
88ENV PATH="$VIRTUAL_ENV/bin:$PATH"
89 
90# Set working directory
91WORKDIR /app
92 
93# Copy application code
94COPY --chown=$USERNAME:$USERNAME . .
95 
96# Switch to non-root user
97USER $USERNAME
98 
99# Expose port
100EXPOSE 8000
101 
102# Default command
103CMD ["python", "main.py"]
🚀 Recommended

High-Performance GPU Cloud

Deploy your Docker containers with powerful NVIDIA GPUs. A100/H100 available, 32+ global locations.

  • NVIDIA A100/H100 GPU instances
  • Hourly billing, starting at $0.004/h
  • 32+ global data centers
  • One-click container & bare metal deployment
🎁 Deploy Now

Related Issues

Docker GPU Not Found

NVIDIA Container Toolkit not installed or configured

CUDA Out of Memory

Model or batch size too large for available VRAM