Cuquantum appliance: Can´t install and run own python package package in Cuquantum appliance when using apptainer

[TomasBezdek-Research]

Cuquantum appliance: Can´t install and run own python package package in Cuquantum appliance when using apptainer

I am a user of Cuquantum appliance 24.08 with apptainer container on HPC cluster, and I need to install my own local package VHA which has dependency on Qiskit and Qiskit Aer. After installation of this package by pip install --no-deps ./VHA, if I use pip show, the package is successfully found, but after running python script or interactive python console when trying to import VHA, I get module not found error, see below for more details.

Setting up the container

To ensure the container works properly with apptainer, and that correct conda environment is activated on every use, i build a container from following .def file.

Bootstrap: docker
From: nvcr.io/nvidia/cuquantum-appliance:24.08-x86_64

%post
    # Ensure conda is initialized
    conda init
    # Ensure environment activation on every start
    echo ". /opt/conda/etc/profile.d/conda.sh" >> ~/.bashrc
    echo "conda activate cuquantum-24.08" >> ~/.bashrc

%environment
    source /opt/conda/etc/profile.d/conda.sh
    conda activate cuquantum-24.08

And after that, I run

apptainer shell cuquantum-appliance.sif

and in the interactive shell, i use

pip install --no-deps ./vha
pip install --no-deps pyscf>=2.6 qiskit_algorithms>=0.3 qiskit_nature>=0.7.2 qiskit_ibm_runtime>=0.24.1 nlopt~=2.7 cma~=3.3.0 IPython traitlets stack_data executing asttokens pure_eval pygments decorator prompt_toolkit wcwidth click h5py

I also tried to install the local package directly to .sif image, but get the same result, i build this image from following .def file.

Bootstrap: docker
From: nvcr.io/nvidia/cuquantum-appliance:24.08-x86_64

%files
    /home/tom/vha

%post
    # Activate the conda environment and install the local package
    # Ensure conda is initialized
    conda init
    # Ensure environment activation on every start
    echo ". /opt/conda/etc/profile.d/conda.sh" >> ~/.bashrc
    echo "conda activate cuquantum-24.08" >> ~/.bashrc

    /bin/bash -c "
    source /opt/conda/etc/profile.d/conda.sh
    conda activate cuquantum-24.08
    pip install --no-deps /vha
    pip install --no-deps pyscf>=2.6 qiskit_algorithms>=0.3 qiskit_nature>=0.7.2 qiskit_ibm_runtime>=0.24.1 nlopt~=2.7 cma~=3.3.0 IPython traitlets stack_data executing asttokens pure_eval pygments decorator prompt_toolkit wcwidth click h5py
    "


%environment
    source /opt/conda/etc/profile.d/conda.sh
    conda activate cuquantum-24.08

Trying to use package

After setting the container, in both cases I get a same result.
I tried just to import the VHA package with following steps.

1. Activate container

apptainer shell ./cuquantum-appliance.sif

2. Use pip show to ensure VHA is installed, and get positive result.
   This for installing the vha during building container.

(cuquantum-24.08) pip show vha
Name: vha
Version: 0.1.dev21+g451e767.d20241102
Summary: 
Home-page: 
Author: 
Author-email: 
License: 
Location: /opt/conda/envs/cuquantum-24.08/lib/python3.11/site-packages
Requires: cma, nlopt, pyscf, qiskit, qiskit-aer, qiskit-algorithms, qiskit-ibm-runtime, qiskit-nature
Required-by: 

This for installing the VHA after build

cuquantum-24.08) pip show vha
Name: vha
Version: 0.1.dev21+g451e767
Summary: 
Home-page: 
Author: 
Author-email: 
License: 
Location: /home/tom/.local/lib/python3.11/site-packages
Requires: cma, nlopt, numpy, pyscf, qiskit, qiskit-aer, qiskit-algorithms, qiskit-ibm-runtime, qiskit-nature
Required-by: 

3. Trying to import VHA

cuquantum-24.08) python3 
Python 3.11.9 | packaged by conda-forge | (main, Apr 19 2024, 18:36:13) [GCC 12.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
import vha
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ModuleNotFoundError: No module named 'vha'

I also get the same error when trying to run a python script that import vha.

[ssarikurt]

There is a mistake in /home/cuquantum/entrypoint.sh, that prevents the proper Conda environment from being loaded. Please try to fix it as below and try again. It worked for me when I added some extra pip packages to the relevant conda environment. I hope it works for your case.

add below line to the /home/cuquantum/entrypoint.sh

CONDA_ENV_NAME=cuquantum-24.08

[TomasBezdek-Research]

Thank you @ssarikurt for advise, i tried it, I add it to the entypoint.sh above the section where conda is initialize, but it didn't activate the correct env on startup.

My current state of /home/cuquantum/entrypoint.sh is

#!/usr/bin/env bash

nvidia_dev_present () {
  num_nvidia_devs=$(find /dev -name *nvidia* -type f | wc --lines)
  if [[ "${num_nvidia_devs}" > 0 ]]
  then
    echo "true"
    return 0
  fi
  echo "false"
  return 1
}

nvidia_driver_present=$(nvidia_dev_present)
if [[ "${nvidia_driver_present}" != "true" ]]
then
IFS='' read -r -d '' nvidia_driver_status <<- EOM
WARNING: no nvidia devices were detected
WARNING: gpu functionality will not be available
EOM
else
IFS='' read -r -d '' nvidia_driver_status <<- EOM
INFO: nvidia devices detected
INFO: gpu functionality will be available
EOM
fi

cuquantum_license=$(</home/cuquantum/LICENSE)
IFS='' read -r -d '' cuquantum_appliance_version <<- EOM

==========================================================================
===                 NVIDIA CUQUANTUM APPLIANCE v24.08                  ===
==========================================================================
=== COPYRIGHT © NVIDIA CORPORATION & AFFILIATES.  All rights reserved. ===
==========================================================================

EOM

echo "${cuquantum_license}"
echo "${cuquantum_appliance_version}"
echo "${nvidia_driver_status}"
CONDA_ENV_NAME=cuquantum-24.08

if [[ $# == 0 ]]
then
  conda activate ${CONDA_ENV_NAME} && \
    bash -i
else
  conda activate ${CONDA_ENV_NAME} && \
    "$@"
fi

Could you provide more details, how did you add it to the entrypoint.sh?

Do you @mtjrider have advice how to set cuquantum appliance to work correctly with apptainer on HPC cluster, and what could cause my problem?