undefined reference to symbol 'MPI_Rsend' and error adding symbols: DSO missing from command line

[sun51027]

Hi
I have problems with "make" files.
I follow the guide in Stellarator-Tools, and cmake runs well.
While when compiling ("make"), error occurred:

/usr/bin/ld: /home/linshih/Library/scalapack/build/lib/libscalapack.a(BI_Rsend.c.o): undefined reference to symbol 'MPI_Rsend'
/home/linshih/Library/openmpi-4.1.5/lib/libmpi.so: error adding symbols: DSO missing from command line
collect2: error: ld returned 1 exit status
_deps/parvmec-build/CMakeFiles/xvmec.dir/build.make:106: recipe for target '_deps/parvmec-build/xvmec' failed
make[2]: *** [_deps/parvmec-build/xvmec] Error 1
CMakeFiles/Makefile2:929: recipe for target '_deps/parvmec-build/CMakeFiles/xvmec.dir/all' failed
make[1]: *** [_deps/parvmec-build/CMakeFiles/xvmec.dir/all] Error 2
Makefile:145: recipe for target 'all' failed
make: *** [all] Error 2.   

Does it matter with my environment setting?
I've downloaded latest version SCALAPACK and OpenMPI and point them into correct paths, so cmake runs well. But I still don't understand what I should do to deal with this.
Thanks!

[cianciosa]

To help debug this, you can run the make file with make VERBOSE=1 to see the command that is sent to the compiler. But it looks like it’s missing the mpi library when linking the final executables. Get Outlook for iOS<https://aka.ms/o0ukef>

…

________________________________ From: sun51027 ***@***.***> Sent: Friday, June 30, 2023 2:49:13 AM To: ORNL-Fusion/PARVMEC ***@***.***> Cc: Subscribed ***@***.***> Subject: [EXTERNAL] [ORNL-Fusion/PARVMEC] undefined reference to symbol 'MPI_Rsend' and error adding symbols: DSO missing from command line (Issue #34) Hi I have problems with "make" files. I follow the guide in Stellarator-Tools, and cmake runs well. While when compiling ("make"), error occurred: /usr/bin/ld: /home/linshih/Library/scalapack/build/lib/libscalapack.a(BI_Rsend.c.o): undefined reference to symbol 'MPI_Rsend' /home/linshih/Library/openmpi-4.1.5/lib/libmpi.so: error adding symbols: DSO missing from command line collect2: error: ld returned 1 exit status _deps/parvmec-build/CMakeFiles/xvmec.dir/build.make:106: recipe for target '_deps/parvmec-build/xvmec' failed make[2]: *** [_deps/parvmec-build/xvmec] Error 1 CMakeFiles/Makefile2:929: recipe for target '_deps/parvmec-build/CMakeFiles/xvmec.dir/all' failed make[1]: *** [_deps/parvmec-build/CMakeFiles/xvmec.dir/all] Error 2 Makefile:145: recipe for target 'all' failed make: *** [all] Error 2. Does it matter with my environment setting? I've downloaded latest version SCALAPACK and OpenMPI and point them into correct paths, so cmake runs well. But I still don't understand what I should do to deal with this. Thanks! — Reply to this email directly, view it on GitHub<https://urldefense.us/v2/url?u=https-3A__github.com_ORNL-2DFusion_PARVMEC_issues_34&d=DwMCaQ&c=v4IIwRuZAmwupIjowmMWUmLasxPEgYsgNI-O7C4ViYc&r=jHzqd7h8AUyesPAQHKZPYGMox50RMWt0Xdplk3LbqH8&m=sU9qeKDNGWlIW5Rl4uD8EjaXXvX9_J79BYAxeiimL1pyAreYaydTAtvjrl-TfSN5&s=6QKf-irONVGb3vJyREtyUt7EvkFmgeix1FSBrtxSdkU&e=>, or unsubscribe<https://urldefense.us/v2/url?u=https-3A__github.com_notifications_unsubscribe-2Dauth_ADAPYD7ZIBSZNPIJD26CGMLXNZZGTANCNFSM6AAAAAAZZN3NKI&d=DwMCaQ&c=v4IIwRuZAmwupIjowmMWUmLasxPEgYsgNI-O7C4ViYc&r=jHzqd7h8AUyesPAQHKZPYGMox50RMWt0Xdplk3LbqH8&m=sU9qeKDNGWlIW5Rl4uD8EjaXXvX9_J79BYAxeiimL1pyAreYaydTAtvjrl-TfSN5&s=tv2cx_wrpGpfYTkXFblcCEOTeJcDnudlL5FlA58pd9E&e=>. You are receiving this because you are subscribed to this thread.Message ID: ***@***.***>

[sun51027]

Hi I tried make VERBOSE=1
I still don't know what exactly the problems are.

[ 98%] Linking Fortran executable xvmec
cd /home/linshih/testspace/Stellarator-Tools/build/_deps/parvmec-build && /home/linshih/cmake-3.26.4-linux-x86_64/bin/cmake -E cmake_link_script CMakeFiles/xvmec.dir/link.txt --verbose=1
/usr/bin/gfortran -Wl,-rpath -Wl,/usr/lib/openmpi/lib -Wl,--enable-new-dtags -L/usr/lib/openmpi/lib -pthread CMakeFiles/xvmec.dir/Sources/TimeStep/vmec.f.o -o xvmec  -Wl,-rpath,/home/linshih/Library/openmpi-4.1.5/lib: libvmec.a ../libstell-build/libstell.a /home/linshih/Library/openmpi-4.1.5/lib/libmpi_usempif08.so /home/linshih/Library/openmpi-4.1.5/lib/libmpi_usempi_ignore_tkr.so /home/linshih/Library/openmpi-4.1.5/lib/libmpi_mpifh.so /home/linshih/Library/openmpi-4.1.5/lib/libmpi.so /usr/lib/x86_64-linux-gnu/libnetcdff.so /usr/lib/x86_64-linux-gnu/libnetcdf.so /home/linshih/Library/scalapack/build/lib/libscalapack.a /usr/lib/libblas.so /usr/lib/liblapack.so
/usr/bin/ld: /home/linshih/Library/scalapack/build/lib/libscalapack.a(BI_Rsend.c.o): undefined reference to symbol 'MPI_Rsend'
/home/linshih/Library/openmpi-4.1.5/lib/libmpi.so: error adding symbols: DSO missing from command line
collect2: error: ld returned 1 exit status
_deps/parvmec-build/CMakeFiles/xvmec.dir/build.make:106: recipe for target '_deps/parvmec-build/xvmec' failed

Is it possible that the gfortran or other lib like mpi too old so it fails?
I guessed "openmpi" in my /usr/lib/ could be obsolete, so I installed the latest version openmpi-4.5.1 in my home directory lest affect other users on the server.
(I've done this cmake -DCMAKE_PREFIX_PATH=/home/linshih/Library/openmpi-4.1.5/lib)
Thank you!

[cianciosa]

I think the problem is the order that the MPI and Scalapack libraries are being linked. In the LIBSTELL/CMakeLists.txt file change

target_link_libraries (stell

                       PUBLIC

                       $<$<BOOL:${MPI_Fortran_FOUND}>:MPI::MPI_Fortran>
                       $<$<BOOL:${NetCDF_FOUND}>:NetCDF::NetCDF>
                       $<$<BOOL:${SCALAPACK_FOUND}>:SCALAPACK::SCALAPACK>
                       $<$<AND:$<BOOL:${BLAS_FOUND}>,$<VERSION_GREATER_EQUAL:${CMAKE_VERSION},3.18>>:BLAS::BLAS>
                       $<$<AND:$<BOOL:${LAPACK_FOUND}>,$<VERSION_GREATER_EQUAL:${CMAKE_VERSION},3.18>>:LAPACK::LAPACK>
                       $<$<AND:$<BOOL:${BLAS_FOUND}>,$<VERSION_LESS:${CMAKE_VERSION},3.18>>:${BLAS_LIBRARIES}>
                       $<$<AND:$<BOOL:${LAPACK_FOUND}>,$<VERSION_LESS:${CMAKE_VERSION},3.18>>:${LAPACK_LIBRARIES}>
)

to

target_link_libraries (stell

                       PUBLIC

                       $<$<BOOL:${NetCDF_FOUND}>:NetCDF::NetCDF>
                       $<$<BOOL:${SCALAPACK_FOUND}>:SCALAPACK::SCALAPACK>
                       $<$<BOOL:${MPI_Fortran_FOUND}>:MPI::MPI_Fortran>
                       $<$<AND:$<BOOL:${BLAS_FOUND}>,$<VERSION_GREATER_EQUAL:${CMAKE_VERSION},3.18>>:BLAS::BLAS>
                       $<$<AND:$<BOOL:${LAPACK_FOUND}>,$<VERSION_GREATER_EQUAL:${CMAKE_VERSION},3.18>>:LAPACK::LAPACK>
                       $<$<AND:$<BOOL:${BLAS_FOUND}>,$<VERSION_LESS:${CMAKE_VERSION},3.18>>:${BLAS_LIBRARIES}>
                       $<$<AND:$<BOOL:${LAPACK_FOUND}>,$<VERSION_LESS:${CMAKE_VERSION},3.18>>:${LAPACK_LIBRARIES}>
)

and see if that fixes things.

[sun51027]

Thank you so much! This fixed DSO missing problem, and passed "Linking Fortran executable xvmec" .
But another problem occurred:

[ 98%] Linking Fortran executable xvmec
[ 98%] Built target xvmec
[ 98%] Building CXX object _deps/parvmec-build/CMakeFiles/xwout_diff.dir/Testing/test_utilities/wout_diff.cpp.o
[ 98%] Building CXX object _deps/parvmec-build/CMakeFiles/xwout_diff.dir/Testing/test_utilities/vmec_test_commandline_parser.cpp.o
/home/linshih/testspace/Stellarator-Tools/PARVMEC/Testing/test_utilities/vmec_test_commandline_parser.cpp:15:74: error: specialization of ‘template<class TYPE> TYPE vmec_test::commandline_parser::get(const string&) const’ in different namespace [-fpermissive]
 vmec_test::commandline_parser::get<std::string> (const std::string &key) const {
                                                                          ^
In file included from /home/linshih/testspace/Stellarator-Tools/PARVMEC/Testing/test_utilities/vmec_test_commandline_parser.cpp:6:0:
/home/linshih/testspace/Stellarator-Tools/PARVMEC/Testing/test_utilities/vmec_test_commandline_parser.hpp:99:14: error:   from definition of ‘template<class TYPE> TYPE vmec_test::commandline_parser::get(const string&) const’ [-fpermissive]
         TYPE get(const std::string &key) const {
              ^
_deps/parvmec-build/CMakeFiles/xwout_diff.dir/build.make:89: recipe for target '_deps/parvmec-build/CMakeFiles/xwout_diff.dir/Testing/test_utilities/vmec_test_commandline_parser.cpp.o' failed

Does this relate to my environment setting?
Thank you again!

[cianciosa]

No that's just a coding error on my part. What compiler, version of that compiler, and platform are you trying to build on?

[sun51027]

Compiler:
The Fortran compiler identification is GNU 5.4.0

I use the server of our instituite, it is ubuntu:

No LSB modules are available.
Distributor ID:	Ubuntu
Description:	Ubuntu 16.04.7 LTS
Release:	16.04

Other information
Found OpenMP_C: -fopenmp (found version "4.0")
Found OpenMP_CXX: -fopenmp (found version "4.0")
Found OpenMP_Fortran: -fopenmp (found version "4.0")
Found OpenMP: TRUE (found version "4.0")
Found NetCDF: /usr/lib/x86_64-linux-gnu/libnetcdf.so (found version "4.4.0")
Found SCALAPACK: /home/linshih/Library/scalapack/build/lib/libscalapack.a
Found MPI_C: /home/linshih/Library/openmpi-4.1.5/lib/libmpi.so (found version "3.1")
Found MPI_CXX: /home/linshih/Library/openmpi-4.1.5/lib/libmpi.so (found version "3.1")
Found MPI_Fortran: /home/linshih/Library/openmpi-4.1.5/lib/libmpi_usempif08.so (found version "3.1")
Found MPI: TRUE (found version "3.1")

Thank you!

[cianciosa]

The error that you’re seeing is only in one of the test utilities and is not needed for compiling just VMEC if you run make help You can get a list of targets you can build make xvmec will build just the VMEC tool and it’s associated dependencies and should avoid the problem until I can up load a fix. Mark On Jul 2, 2023, at 4:09 AM, sun51027 ***@***.******@***.***>> wrote: Compiler: The Fortran compiler identification is GNU 5.4.0 I use the server of our instituite, it is ubuntu: No LSB modules are available. Distributor ID: Ubuntu Description: Ubuntu 16.04.7 LTS Release: 16.04 Other information Found OpenMP_C: -fopenmp (found version "4.0") Found OpenMP_CXX: -fopenmp (found version "4.0") Found OpenMP_Fortran: -fopenmp (found version "4.0") Found OpenMP: TRUE (found version "4.0") Found NetCDF: /usr/lib/x86_64-linux-gnu/libnetcdf.so (found version "4.4.0") Found SCALAPACK: /home/linshih/Library/scalapack/build/lib/libscalapack.a Found MPI_C: /home/linshih/Library/openmpi-4.1.5/lib/libmpi.so (found version "3.1") Found MPI_CXX: /home/linshih/Library/openmpi-4.1.5/lib/libmpi.so (found version "3.1") Found MPI_Fortran: /home/linshih/Library/openmpi-4.1.5/lib/libmpi_usempif08.so (found version "3.1") Found MPI: TRUE (found version "3.1") Thank you! — Reply to this email directly, view it on GitHub<https://urldefense.us/v2/url?u=https-3A__github.com_ORNL-2DFusion_PARVMEC_issues_34-23issuecomment-2D1616435214&d=DwMCaQ&c=v4IIwRuZAmwupIjowmMWUmLasxPEgYsgNI-O7C4ViYc&r=jHzqd7h8AUyesPAQHKZPYGMox50RMWt0Xdplk3LbqH8&m=IkI8i7wPRpYH1MFhjTL8s0J8SOCUynxNZNuwZ3WWidiI1a2oyWb-u-UhVE77lW4O&s=xMYqQeviEynYo2IKnasNDLpaLdxvysoF5qld1IM9gXQ&e=>, or unsubscribe<https://urldefense.us/v2/url?u=https-3A__github.com_notifications_unsubscribe-2Dauth_ADAPYD3OWAWCANQC2FGTNCTXOEUEFANCNFSM6AAAAAAZZN3NKI&d=DwMCaQ&c=v4IIwRuZAmwupIjowmMWUmLasxPEgYsgNI-O7C4ViYc&r=jHzqd7h8AUyesPAQHKZPYGMox50RMWt0Xdplk3LbqH8&m=IkI8i7wPRpYH1MFhjTL8s0J8SOCUynxNZNuwZ3WWidiI1a2oyWb-u-UhVE77lW4O&s=GUdnWDVvNXT20mimnevRvym_46-s5vYeZyOiAr0W8XA&e=>. You are receiving this because you commented.Message ID: ***@***.***>

[sun51027]

I would also like to have a test run, so if test utilities are able to run, this may help me a lot.
Thank you so much!

[cianciosa]

The test utilities are only for the ci_test. I did some digging and this looks like a bug in the gnu compiler that was fixed in a later version. If you want to run some test cases, there are example vmec cases in PARVMEC/Testing/tests On Jul 12, 2023, at 3:18 AM, sun51027 ***@***.******@***.***>> wrote: I would also like to have a test run, so if test utilities are able to run, this may help me a lot. Thank you so much! — Reply to this email directly, view it on GitHub<https://urldefense.us/v2/url?u=https-3A__github.com_ORNL-2DFusion_PARVMEC_issues_34-23issuecomment-2D1631979030&d=DwMCaQ&c=v4IIwRuZAmwupIjowmMWUmLasxPEgYsgNI-O7C4ViYc&r=jHzqd7h8AUyesPAQHKZPYGMox50RMWt0Xdplk3LbqH8&m=i8gpKCatLe9uTyOgD-EQxLT2se8Kmnxqd7M37HCL1Rmkz7i2aWXR1u5Le5crA0-z&s=Xdmx18zbLYZyVmOU52tfJ39GAgnHqZ2CgDOxO5ZZhow&e=>, or unsubscribe<https://urldefense.us/v2/url?u=https-3A__github.com_notifications_unsubscribe-2Dauth_ADAPYD6CNYGVYDQAU7NBQNLXPZFTJANCNFSM6AAAAAAZZN3NKI&d=DwMCaQ&c=v4IIwRuZAmwupIjowmMWUmLasxPEgYsgNI-O7C4ViYc&r=jHzqd7h8AUyesPAQHKZPYGMox50RMWt0Xdplk3LbqH8&m=i8gpKCatLe9uTyOgD-EQxLT2se8Kmnxqd7M37HCL1Rmkz7i2aWXR1u5Le5crA0-z&s=XrJ0X762MVBaUEh3CXRT-mwzmOCPucyqUPhD3D7QmYc&e=>. You are receiving this because you commented.Message ID: ***@***.***>