Unable to run nextflow on mox

[yaaminiv]

I installed nextflow in my /gscratch/srlab/yaaminiv/programs directory using these Bioconda instructions. My goal is to use nextflow to identify SNPs and create a relatedness matrix from WGBS data with the EpiDiverse/snp workflow.

To check the installation, I ran nextflow run hello and got the following error:

I believe the CAPSULE EXCEPTION: Could not parse version: 11.0.9.1-internal while processing attribute Min-Java-Version: 1.8.0 (for stack trace, run with -Dcapsule.log=verbose) refers to my java version:

nextflow requires java version > 8, and I already have that! What do I need to do to ensure I can initialize the nextflow environment?

[yaaminiv]

Tried running the command on the build node just in case my login node was the issue. Got the same error.

[kubu4]

Your prompt suggests you haven't activated Anaconda. Usually, to run conda installs, you have to activate a conda environment first, then run the desired commands.

P. S. Could you also please paste error messages instead of solely screenshots? That would allow us to then copy/paste error messages for searching, instead of having to type them.

[yaaminiv]

Usually, to run conda installs, you have to activate a conda environment first, then run the desired commands.

Gotcha! I ran conda init, then nextflow run hello. I got the following error about insufficient memory, but that's probably because I'm in my user directory:

(base) [yaaminiv@mox1 ~]$ nextflow run hello
N E X T F L O W  ~  version 21.04.0
Pulling nextflow-io/hello ...
#
# There is insufficient memory for the Java Runtime Environment to continue.
# Native memory allocation (malloc) failed to allocate 32744 bytes for ChunkPool::allocate
# An error report file with more information is saved as:
# /gscratch/home/yaaminiv/hs_err_pid11813.log
[thread 12036 also had an error]
#
# Can't open file to dump replay data. Error: Not enough space

I moved to /gscratch/scrubbed/yaaminiv/Hawes/analyses/epidiverse-snp/, ran the code, and a similar error:

N E X T F L O W  ~  version 21.04.0
Launching `nextflow-io/hello` [focused_murdock] - revision: e6d9427e5b [master]
[3.889s][warning][os,thread] Failed to start thread - pthread_create failed (EAGAIN) for attributes: stacksize: 1024k, guardsize: 0k, detached.
#
# There is insufficient memory for the Java Runtime Environment to continue.
# Native memory allocation (malloc) failed to allocate 32 bytes for AllocateHeap
# An error report file with more information is saved as:
# /gscratch/scrubbed/yaaminiv/Hawes/analyses/epidiverse-snp/hs_err_pid13319.log

The error log is pretty detailed, but this seems to be the gist:

# There is insufficient memory for the Java Runtime Environment to continue.
# Native memory allocation (malloc) failed to allocate 32 bytes for AllocateHeap
# Possible reasons:
#   The system is out of physical RAM or swap space
#   The process is running with CompressedOops enabled, and the Java Heap may be blocking the growth of the native heap
# Possible solutions:
#   Reduce memory load on the system
#   Increase physical memory or swap space
#   Check if swap backing store is full
#   Decrease Java heap size (-Xmx/-Xms)
#   Decrease number of Java threads
#   Decrease Java thread stack sizes (-Xss)
#   Set larger code cache with -XX:ReservedCodeCacheSize=
#   JVM is running with Unscaled Compressed Oops mode in which the Java heap is
#     placed in the first 4GB address space. The Java Heap base address is the
#     maximum limit for the native heap growth. Please use -XX:HeapBaseMinAddress
#     to set the Java Heap base and to place the Java Heap above 4GB virtual address.
# This output file may be truncated or incomplete.
#
#  Out of Memory Error (allocation.cpp:46), pid=13319, tid=13428
#
# JRE version: OpenJDK Runtime Environment (11.0.9.1) (build 11.0.9.1-internal+0-adhoc..src)
# Java VM: OpenJDK 64-Bit Server VM (11.0.9.1-internal+0-adhoc..src, mixed mode, tiered, compressed oops, g1 gc, linux-amd64)
# No core dump will be written. Core dumps have been disabled. To enable core dumping, try "ulimit -c unlimited" before starting Java again
#

I'll work on clearing up space in my scrubbed directory and seeing if that will allow me to execute nextflow run hello

[kubu4]

You don't need to clean anything; it's not a storage space issue. It seems to be a RAM issue. Try doing these steps again on a build node, which will have more than sufficient RAM for stuff like this.

[yaaminiv]

Running on a build node worked; thanks! I always forget about those...

N E X T F L O W  ~  version 21.04.0
Launching `nextflow-io/hello` [deadly_tuckerman] - revision: e6d9427e5b [master]
executor >  local (4)
[60/c490c7] process > sayHello (4) [100%] 4 of 4 ✔
Ciao world!

Bonjour world!

Hola world!

Hello world!

[kubu4]

Build node should really be the default node you do anything on Mox with. The login node is specifically designed solely for access to Mox and data transfer.