[DOCS] whisper-rs: add build instructions for CUDA support on Arch Linux (GCC 15 incompatibility)

[guidovicino]

Thank you for your project! It’s exactly what I was looking for. While installing it on my Arch Linux setup with Hyprland and an NVIDIA card, I ran into a few issues. I’ve explained below how I solved them, in case you’d like to add support for CUDA users.

Feature Description

Document how to build waystt with CUDA for the local Whisper models on Arch/rolling distros, including the required change in Cargo.toml (feature name is cuda).

Problem Statement

For those with an NVIDIA card, there's an option to enable CUDA to use the GPU with Whisper.

Proposed Solution

Cargo.toml – enable the correct feature:

# Local transcription
#whisper-rs = "0.15"
whisper-rs = { version = "0.15", features = ["cuda"] }

Build instructions for CUDA users

# Set CUDA paths (Arch defaults to /opt/cuda)
export CUDAToolkit_ROOT=/opt/cuda
export CUDA_HOME=/opt/cuda
export PATH="$CUDA_HOME/bin:$PATH"
export LD_LIBRARY_PATH="$CUDA_HOME/lib64:$CUDA_HOME/lib64/stubs:${LD_LIBRARY_PATH}"

# Force nvcc to use GCC 14 (needed on Arch with CUDA 12+)
# Install if missing: sudo pacman -S --needed gcc14
export CMAKE_CUDA_HOST_COMPILER=/usr/bin/g++-14
export CUDAHOSTCXX=/usr/bin/g++-14
export CMAKE_CUDA_FLAGS="--compiler-bindir=/usr/bin/g++-14 -Xcompiler=-fPIC"

# Clean and build with CUDA
cargo clean
cargo build --release --features cuda    # or: --features gpu

[llevering]

Nice addition! However if I try to apply the change and build I get the following error:
error: the package 'waystt' does not contain this feature: cuda

I guess I need another change that I am somehow missing.

[farnoy]

@llevering You can do that with cargo run --release --features whisper-rs/cuda - it's a feature flag of the whisper-rs library, not waystt.

[llevering]

@farnoy Thanks, I do end up with all kind of build errors. I will wait for when it is packaged up :)

[tomjo]

Thanks for these instructions, used them to build and use successfully on Fedora 43 (small change: different cuda path, install packages)

I did have one issue with cuda13-1 on fedora 43, I patched my local cuda until there is a fix. See https://forums.developer.nvidia.com/t/fedora-43-and-nvcc-cuda13-1-error-exception-specification-is-incompatible-rsqrt-rsqrtf/354510

Fix:

FILE=/usr/local/cuda/targets/x86_64-linux/include/crt/math_functions.h
sudo cp -a "$FILE" "$FILE.bak.$(date +%F-%H%M%S)"
sudo perl -pi -e '
  if (/^\s*extern\s+__DEVICE_FUNCTIONS_DECL__.*\bdouble\s+rsqrt\s*\(\s*double\s+x\s*\)\s*;/ && !/noexcept/) {
    s/\)\s*;/) noexcept (true);/;
  }
  if (/^\s*extern\s+__DEVICE_FUNCTIONS_DECL__.*\bfloat\s+rsqrtf\s*\(\s*float\s+x\s*\)\s*;/ && !/noexcept/) {
    s/\)\s*;/) noexcept (true);/;
  }
' "$FILE"

sudo dnf config-manager addrepo --from-repofile https://developer.download.nvidia.com/compute/cuda/repos/fedora42/x86_64/cuda-fedora42.repo
sudo dnf -y install cuda-toolkit-13-1

sudo dnf -y install gcc14 gcc14-c++ alsa-lib-devel pkgconf-pkg-config cmake

export CUDA_HOME=/usr/local/cuda
export PATH=$CUDA_HOME/bin${PATH:+:${PATH}}
export LD_LIBRARY_PATH=$CUDA_HOME/lib64${LD_LIBRARY_PATH:+:${LD_LIBRARY_PATH}}

export CMAKE_CUDA_HOST_COMPILER=/usr/bin/g++-14
export CUDAHOSTCXX=/usr/bin/g++-14
export CMAKE_CUDA_FLAGS="--compiler-bindir=/usr/bin/g++-14 -Xcompiler=-fPIC"

cargo build --release --features whisper-rs/cuda