Parallel computation of Jaccard index uses up server resources. Proposed solution. #11
Description
When I run the algorithm and set n_jobs I find that there are more processes than expected. I believe that this is why this happens:
File: PhenoGraph/phenograph/core.py
135: with closing(Pool()) as pool: # replace with: with closing(Pool(n_jobs)) as pool:
136: jaccard_values = pool.starmap(calc_jaccard, zip(range(n), repeat(idx)))
I can correct this in my clone of the library. Is there a reason why you can't set the maximum number of jobs for this process pool?