Cannot call neighbors: Failed in nopython mode pipeline (step: nopython frontend) #173
Description
Describe the bug
I tried several permutations of specifying specific version numbers, installing dependencies manually etc, but there was always some issue. I just load muon, load a mudataset with per-module neighbors pre-computed, then try to call mu.pp.neighbors, but get this issue:
output
---------------------------------------------------------------------------
TypingError Traceback (most recent call last)
Cell In[38], line 3
1 for name, adata in objs.items():
2 # adata = adata.copy()
----> 3 mu.pp.neighbors(adata, n_multineighbors=100, low_memory=True)
File [/fh/fast/_IRC/FHIL/grp/analyses/bm04/.venv/lib/python3.11/site-packages/muon/_core/preproc.py:442](http://gizmok3.fhcrc.org:45309/lab/tree/analyses/bm04/03_analysis/03_notebooks/01_primary_analyses/analyses/bm04/.venv/lib/python3.11/site-packages/muon/_core/preproc.py#line=441), in neighbors(mdata, n_neighbors, n_bandwidth_neighbors, n_multineighbors, neighbor_keys, metric, low_memory, key_added, weight_key, add_weights_to_modalities, eps, copy, random_state)
432 metric_kwds = dict(
433 X=X,
434 neighbors_indices=neighbordistances.indices,
(...) 438 bbox_norm=bbox_norm,
439 )
441 logging.info(f"Calculating kernel bandwidth for '{mod1}' modality...")
--> 442 nn_indices, _, _ = nearest_neighbors(
443 np.arange(N)[:, np.newaxis],
444 n_neighbors=n_bandwidth_neighbors,
445 metric=cmetric,
446 metric_kwds=metric_kwds,
447 random_state=randomstate,
448 angular=False,
449 low_memory=lmemory,
450 )
452 csigmas = np.empty((N,), dtype=neighbordistances.dtype)
453 if issparse(X):
File [/fh/fast/_IRC/FHIL/grp/analyses/bm04/.venv/lib/python3.11/site-packages/umap/umap_.py:330](http://gizmok3.fhcrc.org:45309/lab/tree/analyses/bm04/03_analysis/03_notebooks/01_primary_analyses/analyses/bm04/.venv/lib/python3.11/site-packages/umap/umap_.py#line=329), in nearest_neighbors(X, n_neighbors, metric, metric_kwds, angular, random_state, low_memory, use_pynndescent, n_jobs, verbose)
327 n_trees = min(64, 5 + int(round((X.shape[0]) ** 0.5 [/](http://gizmok3.fhcrc.org:45309/) 20.0)))
328 n_iters = max(5, int(round(np.log2(X.shape[0]))))
--> 330 knn_search_index = NNDescent(
331 X,
332 n_neighbors=n_neighbors,
333 metric=metric,
334 metric_kwds=metric_kwds,
335 random_state=random_state,
336 n_trees=n_trees,
337 n_iters=n_iters,
338 max_candidates=60,
339 low_memory=low_memory,
340 n_jobs=n_jobs,
341 verbose=verbose,
342 compressed=False,
343 )
344 knn_indices, knn_dists = knn_search_index.neighbor_graph
346 if verbose:
File [/fh/fast/_IRC/FHIL/grp/analyses/bm04/.venv/lib/python3.11/site-packages/pynndescent/pynndescent_.py:923](http://gizmok3.fhcrc.org:45309/lab/tree/analyses/bm04/03_analysis/03_notebooks/01_primary_analyses/analyses/bm04/.venv/lib/python3.11/site-packages/pynndescent/pynndescent_.py#line=922), in NNDescent.__init__(self, data, metric, metric_kwds, n_neighbors, n_trees, leaf_size, pruning_degree_multiplier, diversify_prob, n_search_trees, tree_init, init_graph, init_dist, random_state, low_memory, max_candidates, max_rptree_depth, n_iters, delta, n_jobs, compressed, parallel_batch_queries, verbose)
920 if verbose:
921 print(ts(), "NN descent for", str(n_iters), "iterations")
--> 923 self._neighbor_graph = nn_descent(
924 self._raw_data,
925 self.n_neighbors,
926 self.rng_state,
927 effective_max_candidates,
928 self._distance_func,
929 self.n_iters,
930 self.delta,
931 low_memory=self.low_memory,
932 rp_tree_init=True,
933 init_graph=_init_graph,
934 leaf_array=leaf_array,
935 verbose=verbose,
936 )
938 if np.any(self._neighbor_graph[0] < 0):
939 warn(
940 "Failed to correctly find n_neighbors for some samples."
941 " Results may be less than ideal. Try re-running with"
942 " different parameters."
943 )
File [/fh/fast/_IRC/FHIL/grp/analyses/bm04/.venv/lib/python3.11/site-packages/numba/core/dispatcher.py:424](http://gizmok3.fhcrc.org:45309/lab/tree/analyses/bm04/03_analysis/03_notebooks/01_primary_analyses/analyses/bm04/.venv/lib/python3.11/site-packages/numba/core/dispatcher.py#line=423), in _DispatcherBase._compile_for_args(self, *args, **kws)
420 msg = (f"{str(e).rstrip()} \n\nThis error may have been caused "
421 f"by the following argument(s):\n{args_str}\n")
422 e.patch_message(msg)
--> 424 error_rewrite(e, 'typing')
425 except errors.UnsupportedError as e:
426 # Something unsupported is present in the user code, add help info
427 error_rewrite(e, 'unsupported_error')
File [/fh/fast/_IRC/FHIL/grp/analyses/bm04/.venv/lib/python3.11/site-packages/numba/core/dispatcher.py:365](http://gizmok3.fhcrc.org:45309/lab/tree/analyses/bm04/03_analysis/03_notebooks/01_primary_analyses/analyses/bm04/.venv/lib/python3.11/site-packages/numba/core/dispatcher.py#line=364), in _DispatcherBase._compile_for_args.<locals>.error_rewrite(e, issue_type)
363 raise e
364 else:
--> 365 raise e.with_traceback(None)
TypingError: Failed in nopython mode pipeline (step: nopython frontend)
Failed in nopython mode pipeline (step: nopython frontend)
Failed in nopython mode pipeline (step: nopython frontend)
Failed in nopython mode pipeline (step: nopython frontend)
Failed in nopython mode pipeline (step: nopython frontend)
Failed in nopython mode pipeline (step: nopython frontend)
Failed in nopython mode pipeline (step: nopython frontend)
No implementation of function Function(<built-in function setitem>) found for signature:
>>> setitem(readonly array(int32, 1d, C), int64, int32)
There are 16 candidate implementations:
- Of which 14 did not match due to:
Overload of function 'setitem': File: <numerous>: Line N[/A.](http://gizmok3.fhcrc.org:45309/A.)
With argument(s): '(readonly array(int32, 1d, C), int64, int32)':
No match.
- Of which 2 did not match due to:
Overload in function 'SetItemBuffer.generic': File: numba[/core/typing/arraydecl.py](http://gizmok3.fhcrc.org:45309/core/typing/arraydecl.py): Line 221.
With argument(s): '(readonly array(int32, 1d, C), int64, int32)':
Rejected as the implementation raised a specific error:
NumbaTypeError: Cannot modify readonly array of type: readonly array(int32, 1d, C)
raised from [/fh/fast/_IRC/FHIL/grp/analyses/bm04/.venv/lib/python3.11/site-packages/numba/core/typing/arraydecl.py:228](http://gizmok3.fhcrc.org:45309/lab/tree/analyses/bm04/03_analysis/03_notebooks/01_primary_analyses/analyses/bm04/.venv/lib/python3.11/site-packages/numba/core/typing/arraydecl.py#line=227)
During: typing of setitem at [/fh/fast/_IRC/FHIL/grp/analyses/bm04/.venv/lib/python3.11/site-packages/umap/sparse.py](http://gizmok3.fhcrc.org:45309/lab/tree/analyses/bm04/03_analysis/03_notebooks/01_primary_analyses/analyses/bm04/.venv/lib/python3.11/site-packages/umap/sparse.py) (62)
File "../../../.venv/lib/python3.11/site-packages/umap/sparse.py", line 62:
def sparse_sum(ind1, data1, ind2, data2):
<source elided>
if val != 0:
result_ind[nnz] = j1
^
During: Pass nopython_type_inference
During: resolving callee type: type(CPUDispatcher(<function sparse_sum at 0x14fbd59fd940>))
During: typing of call at [/fh/fast/_IRC/FHIL/grp/analyses/bm04/.venv/lib/python3.11/site-packages/umap/sparse.py](http://gizmok3.fhcrc.org:45309/lab/tree/analyses/bm04/03_analysis/03_notebooks/01_primary_analyses/analyses/bm04/.venv/lib/python3.11/site-packages/umap/sparse.py) (108)
File "../../../.venv/lib/python3.11/site-packages/umap/sparse.py", line 108:
def sparse_diff(ind1, data1, ind2, data2):
return sparse_sum(ind1, data1, ind2, -data2)
^
During: Pass nopython_type_inference
During: resolving callee type: type(CPUDispatcher(<function sparse_diff at 0x14fbd59fdc60>))
During: typing of call at [/fh/fast/_IRC/FHIL/grp/analyses/bm04/.venv/lib/python3.11/site-packages/umap/sparse.py](http://gizmok3.fhcrc.org:45309/lab/tree/analyses/bm04/03_analysis/03_notebooks/01_primary_analyses/analyses/bm04/.venv/lib/python3.11/site-packages/umap/sparse.py) (236)
File "../../../.venv/lib/python3.11/site-packages/umap/sparse.py", line 236:
def sparse_euclidean(ind1, data1, ind2, data2):
aux_inds, aux_data = sparse_diff(ind1, data1, ind2, data2)
^
During: Pass nopython_type_inference
During: resolving callee type: type(CPUDispatcher(<function _jaccard_sparse_euclidean_metric at 0x14fbd59719e0>))
During: typing of call at [/fh/fast/_IRC/FHIL/grp/analyses/bm04/.venv/lib/python3.11/site-packages/pynndescent/pynndescent_.py](http://gizmok3.fhcrc.org:45309/lab/tree/analyses/bm04/03_analysis/03_notebooks/01_primary_analyses/analyses/bm04/.venv/lib/python3.11/site-packages/pynndescent/pynndescent_.py) (967)
File "../../../.venv/lib/python3.11/site-packages/pynndescent/pynndescent_.py", line 967:
def _partial_dist_func(x, y):
return _distance_func(x, y, *dist_args)
^
During: Pass nopython_type_inference
During: resolving callee type: type(CPUDispatcher(<function NNDescent._set_distance_func.<locals>._partial_dist_func at 0x14fbc6e66de0>))
During: typing of call at [/fh/fast/_IRC/FHIL/grp/analyses/bm04/.venv/lib/python3.11/site-packages/pynndescent/pynndescent_.py](http://gizmok3.fhcrc.org:45309/lab/tree/analyses/bm04/03_analysis/03_notebooks/01_primary_analyses/analyses/bm04/.venv/lib/python3.11/site-packages/pynndescent/pynndescent_.py) (90)
File "../../../.venv/lib/python3.11/site-packages/pynndescent/pynndescent_.py", line 90:
def generate_leaf_updates(leaf_block, dist_thresholds, data, dist):
<source elided>
d = dist(data[p], data[q])
^
During: Pass nopython_type_inference
During: resolving callee type: type(CPUDispatcher(<function generate_leaf_updates at 0x14fbd52dbd80>))
During: typing of call at [/fh/fast/_IRC/FHIL/grp/analyses/bm04/.venv/lib/python3.11/site-packages/pynndescent/pynndescent_.py](http://gizmok3.fhcrc.org:45309/lab/tree/analyses/bm04/03_analysis/03_notebooks/01_primary_analyses/analyses/bm04/.venv/lib/python3.11/site-packages/pynndescent/pynndescent_.py) (111)
File "../../../.venv/lib/python3.11/site-packages/pynndescent/pynndescent_.py", line 111:
def init_rp_tree(data, dist, current_graph, leaf_array):
<source elided>
updates = generate_leaf_updates(leaf_block, dist_thresholds, data, dist)
^
During: Pass nopython_type_inference
During: resolving callee type: type(CPUDispatcher(<function init_rp_tree at 0x14fbd52db880>))
During: typing of call at [/fh/fast/_IRC/FHIL/grp/analyses/bm04/.venv/lib/python3.11/site-packages/pynndescent/pynndescent_.py](http://gizmok3.fhcrc.org:45309/lab/tree/analyses/bm04/03_analysis/03_notebooks/01_primary_analyses/analyses/bm04/.venv/lib/python3.11/site-packages/pynndescent/pynndescent_.py) (347)
File "../../../.venv/lib/python3.11/site-packages/pynndescent/pynndescent_.py", line 347:
def nn_descent(
<source elided>
if rp_tree_init:
init_rp_tree(data, dist, current_graph, leaf_array)
^
During: Pass nopython_type_inference
To Reproduce
See below the minimal docker/apptainer image used for testing.
Python env apptainer def file
Bootstrap: docker
From: python:3.12
%post
apt-get update && apt-get install -y --no-install-recommends \
build-essential libhdf5-dev libxml2-dev libxslt1-dev \
&& rm -rf /var/lib/apt/lists/*
pip install --upgrade pip
# Pin scientific stack to avoid NumPy/SciPy ABI errors
# pip install "numpy<2" "scipy==1.10" "pandas<2.2"
pip install \
numpy \
scipy \
pandas \
pandas \
matplotlib \
seaborn \
h5py \
anndata \
scanpy \
"scikit-learn" \
"umap-learn" \
numba \
protobuf \
tqdm \
mudata
# pip install "anndata<0.10" "scanpy<1.10" "mudata==0.2.2"
pip install muon
%runscript
exec python "$@"
Expected behaviour
Neighbors should run on a mudata object or communicate what dependencies aren't satisfied to run the command. If specific versions of packages are needed, that should be communicated or enforced.
System
- OS: [Debian Linux]
- Python version [>=3.10]
- Versions of libraries involved:
>>> print(sys.version)
3.12.12 (main, Nov 18 2025, 11:56:31) [GCC 14.2.0]
>>> print(np.__version__)
2.3.5
>>> print(numba.__version__)
0.62.1
>>> print(mu.__version__)
0.1.7
>>> print(scipy.__version__)
1.16.3
Additional context
Add any other context about the problem here.