2.4 release fails to build on Amazon Ubuntu linux - 2.3.3 works cool.

[PeterAronZentai]

I am using DLAMI - pytorch 2.8 for Ubuntu.

Here is the compiler output by following AWS docs on having EFS on Ubuntu linux.
It scares me this apparent lack of basic QA check and processes.

Why do you publish the main branch to be used if that is apparently just a dev branch? Also, releasing on a Friday, late evening - who does that? I consider AWS documentation and included tools to be part of the paid offering for which we are paying grands per month - I don't think that the open source "release quick and fix later" sentiment is very fair toward paying customers.

Caused by:
  process didn't exit successfully: `/home/ubuntu/efs-utils/src/proxy/target/release/build/aws-lc-fips-sys-f50767e163597313/build-script-main` (exit status: 101)
  --- stdout
  cargo:rerun-if-env-changed=AWS_LC_FIPS_SYS_NO_PREFIX
  cargo:rerun-if-env-changed=AWS_LC_FIPS_SYS_PREGENERATING_BINDINGS
  cargo:rerun-if-env-changed=AWS_LC_FIPS_SYS_EXTERNAL_BINDGEN
  cargo:rerun-if-env-changed=AWS_LC_FIPS_SYS_NO_ASM
  cargo:rerun-if-env-changed=AWS_LC_FIPS_SYS_CPU_JITTER_ENTROPY
  cargo:rustc-cfg=x86_64_unknown_linux_gnu
  cargo:rerun-if-env-changed=AWS_LC_FIPS_SYS_STATIC
  cargo:warning=Building with: CMake
  cargo:warning=Symbol Prefix: Some("aws_lc_fips_0_13_9")
  cargo:rerun-if-env-changed=CMAKE

  --- stderr
  Missing dependency: Go is required for FIPS.

  thread 'main' (5277) panicked at /home/ubuntu/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/aws-lc-fips-sys-0.13.9/builder/main.rs:518:34:
  called `Result::unwrap()` on an `Err` value: "Required build dependency is missing. Halting build."
  note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
warning: build failed, waiting for other jobs to finish...
Reading package lists... Done

[ericrichtert]

Same on RHEL 8.10. See also #293

[YangjinanHu]

efs-utils 2.4.0 were updated to use AWS-LC. Go is now a required build dependency. More information on build instruction: https://github.com/aws/efs-utils/blob/master/INSTALL.md

[ericrichtert]

Two things to consider:

• Create a Changelog file in the repo
• Let AWS build the RPM and distribute it just like the cw-agent

[PeterAronZentai]

@YangjinanHu - but you realise this is a breaking change introduced Friday late evening - breaking all our weekend automation.

[YangjinanHu]

We will introduce checks that prevent Friday release in our release steps for breaking changes. Thanks for the information.

[PeterAronZentai]

Thanks - that should help avoiding this. Also, if you intend to do breaking changes, then do anticipate automations upstream. One easy to implement practice is not targeting the main branch in your EFS documentation but toward a release branch that can be pinned. (Remember a change from 2.3 to 2.4 should not be breaking according to semver)

[lisenet]

Same issue when building 2.4.1 on Rocky 9. No issues when building 2.3.3.

[YangjinanHu]

Hi @lisenet, could you share your installed dependencies in your build environment and the build log that contains the failure? Thanks!

See https://github.com/aws/efs-utils/blob/master/INSTALL.md for dependencies requirement.

[jsf9k]

I'm seeing the same behavior with 2.4.1 on Debian Trixie. If I install the cmake and golang system packages then I get:

...
[ 97%] Generating curve25519/curve25519_x25519base_alt.S.S
[ 97%] Generating bcm-delocated.S"
--- stderr
CMake Deprecation Warning at CMakeLists.txt:10 (cmake_policy):
The OLD behavior for policy CMP0091 will be removed from a future version
of CMake.
The cmake-policies(7) manual explains that the OLD behaviors of all
policies are deprecated and that a policy should be set to OLD only under
specific short-term circumstances.  Projects should be ported to the NEW
behavior and not rely on setting a policy to OLD.
CMake Deprecation Warning at aws-lc/CMakeLists.txt:1 (cmake_minimum_required):
Compatibility with CMake < 3.10 will be removed from a future version of
CMake.
Update the VERSION argument <min> value.  Or, use the <min>...<max> syntax
to tell CMake that the project requires at least <min> but has been updated
to work with policies introduced by <max> or earlier.
error while processing \"\\t.section\\t.data.rel.ro.local,\\\"aw\\\"\\n\" on line 233995: \".data section found in module\"
gmake[2]: *** [aws-lc/crypto/fipsmodule/CMakeFiles/bcm_hashunset.dir/build.make:135: aws-lc/crypto/fipsmodule/bcm-delocated.S] Error 1
gmake[2]: *** Deleting file 'aws-lc/crypto/fipsmodule/bcm-delocated.S'
gmake[1]: *** [CMakeFiles/Makefile2:346: aws-lc/crypto/fipsmodule/CMakeFiles/bcm_hashunset.dir/all] Error 2
gmake: *** [Makefile:136: all] Error 2
thread 'main' panicked at /root/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/cmake-0.1.54/src/lib.rs:1119:5: command did not execute successfully, got: exit status: 2 
build script failed, must exit now
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

[YangjinanHu]

Hi @jsf9k, there is GCC version requirement for building the AWS-LC FIPS branch, consider using gcc-13 and g++-13 to build efs-utils. You can follow the steps in GCC Version Requirements Section in the install instruction to resolve your build failure.

More information on the GCC version requirement and related fixes: aws/aws-lc#2010

[jsf9k]

@YangjinanHu - I installed gcc-13 and g++-13 and exported the CC and CXX environment variables but I still get an error:

...
note: lto1: fatal error: bytecode stream in file '/tmp/efs-utils/src/proxy/target/release/deps/libs2n_tls_sys-b912f8a9535161b8.rlib' generated with LTO version 13.1 instead of the expected 14.0
compilation terminated.
lto-wrapper: fatal error: cc returned 1 exit status
compilation terminated.
/usr/bin/ld: error: lto-wrapper failed
collect2: error: ld returned 1 exit status
error: could not compile `efs-proxy` (bin "efs-proxy") due to 1 previous error

[YangjinanHu]

Hi @jsf9k, from the error message "generated with LTO version 13.1 instead of the expected 14.0", in your case GCC 13 was used to compile the C components while the system used GCC 14 for the final link. This mismatch caused the build failure.

Try cleaning the build cache by running cargo clean in efs-utils/src/proxy folder and retry building efs-utils in efs-utils folder. If you have set RUSTFLAGS, unset RUSTFLAGS.