Publish the latest certified version of AWS-LC as a separate package

[conradludgate]

Problem:

In order to use rustls with older aws-lc-rs (for FIPS certification), we have to use an older version of rustls (0.23.20). In theory we can build rustls without the aws-lc-rs dependency and we can manage it ourselves, but in practice some indirect dependency will keep the default feature. As time goes on, it's more and more difficult to keep using this older version of rustls.

Solution:

Republish aws-lc-rs 1.11.1 under a new name (eg aws-lc-rs-certified), so that we can have both aws-lc-rs 1.14.X+ and aws-lc-rs-certified in the same dependency tree. rustls can still depend on the older aws-lc-rs, but it will be unused as we can install a custom CryptoProvider.

Requirements / Acceptance Criteria:

The latest rustls 0.23 with default features enabled must be usable alongside this new crate. This new crate must be targeting AWS-LC 2.0 for FIPS compliance.

Potential extension

Publish a rustls crypto provider targeting this crate.

[conradludgate]

I have been planning to publish these myself, see

• https://github.com/conradludgate/aws-lc-rs
• https://github.com/conradludgate/rustls-aws-lc-rs

(the cl in the package name stands for conradludgate, for a lack of crate name-spacing)

My colleagues are not super comfortable with us maintaining core cryptographic libraries (personally I am comfortable, but collectively they are not), which is why I feel it would be beneficial for AWS to publish these.

[jlizen]

It seems like the core issue is, the library goes from FIPS certified to not across a non-breaking version, so cargo is resolving to uncertified?

Can you explain why just taking a direct dependency on aws-lc-rs pinning to <=1.11.1 doesn't work? Is it because many other libraries point to a newer version explicitly, so cargo finds no solution?

[conradludgate]

Correct. For now we are managing to keep aws-lc-rs at =1.11.1 but I'm suspecting it's not going to be maintainable for much longer.

Through indirect dependencies, we cannot reasonably disable the aws-lc-rs feature of rustls unless we submit patches upstream to all of them or maintain forks. Too many libraries depend on rustls with default features (had this pain already when I tried and failed to remove ring). rustls 0.23.21 and onwards depends on aws-lc-rs 1.12+ when the feature is enabled. Not being able to update rustls is a security risk.

The latest version of reqwest (0.13) now depends on rustls-platform-verifier which depends on rustls 0.23.27. Given how popular reqwest is, this is going to cause the ecosystem to move over to reqwest 0.13 and we will have a large amount of tech debt since we cannot update.

[justsmth]

Hi, thanks for raising visibility of this issue. We will need to have some conversations internally before any significant steps are taken like, e.g., creating/maintaining a new crate.

I'm a little confused about the use-case here:

... so that we can have both aws-lc-rs 1.14.X+ and aws-lc-rs-certified in the same dependency tree.

My understanding is that in a strict FIPS-compliant environment, an application can not link to any cryptographic implementation (for an in-scope operation) that is not contained within an approved FIPS module.

• Wouldn't any consumer with "aws-lc-rs >= 1.12.+" in its dependency tree already be (potentially) violating strict compliance rules.
• By creating a separate aws-lc-rs-certified crate, would we just be masking consumers' non-compliance issues?

(I'm certainly not an expert on FIPS compliance rules, so feel free to correct my misunderstanding.)

I'd be curious to hear thoughts from rustls maintainers (@ctz or @djc?) on this.

[conradludgate]

The intention is that the non-FIPS certified implementations that are linked will actually go completely unused, as I would override the default rustls CryptoProvider with the certified one. I'll have to review internally whether having unused cryptographic modules causes issues for compliance.

[justsmth]

The intention is that the non-FIPS certified implementations that are linked will actually go completely unused, ...

Might we be able to resolve this problem by implementing an updated (rustls) CryptoProvider that only uses the aws-lc-rs v1.11.1 APIs?

[jlizen]

as I would override the default rustls CryptoProvider with the certified one.

This would still involve whackamole with dependencies that are implicitly configuring their own providers right?

Sounds like shaky footing from a compliance angle, visibility into non certified usage by dependencies wouldn't be great. Presumably any runtime validation of which APIs are used would need to itself be FIPS certified anyway.

A new, certified provider pinned to 1.11 still is fragile in the way that any mutually exclusive cargo feature is, but at least it gives a clear ask for dependencies that are bringing in the wrong crypto provider (make it configurable to use the certified one).

(I know rustls plans to remove the providers in core anyway, but libraries will anyway have their own feature flags with providers, probably)

The alternative would be something like, validating which symbols show up in your binary, but that seems tricky to get correct in a way that satisfies compliance requirements. And also much more machinery to enforce compared to just failing the build at the cargo dependency graph level.

[justsmth]

I'll have to review internally whether having unused cryptographic modules causes issues for compliance.

I'm not familiar with tooling around this. How would one prevent a cryptographic module in the dependency tree from being used by (transitive) dependencies on its path? How would one confirm that a specific cryptographic module is not being used?

(To confirm, I suppose one could list the symbols in the binary and check that none originate from a non-approved cryptographic module?)

[conradludgate]

One thing we can do is patch the newer aws-lc-rs versions to not link to anything and all functions error. I think our teams would be happy to maintain this as there's no logic. As long as the public functions are there and it compiles then it's ok.

[justsmth]

One thing we can do is patch the newer aws-lc-rs versions to not link to anything and all functions error. I think our teams would be happy to maintain this as there's no logic. As long as the public functions are there and it compiles then it's ok.

Here's an idea related to this that might work, and be easy for us to maintain: #987

[jlizen]

#987

This seems promising! But, will this simply panic on usage, or also avoid linking non-FIPS-compliant symbols in the first place? If symbols are still present, would probably need new FIPS cert to validate that the runtime panic-on-usage is sufficient to allow keeping the symbols linked right?

It seems like what we would really need is:

• aws-lc-rs-certified and aws-lc-sys-certified that are always on 1.11 (and bump up to aws-lc-3 next certification)

And then the optional but highly desirable additional improvements would be:

• rustls provider using *-certified
  • leaf consumer can do this manually, but central is nice to have a clear ask for dependencies relying on non-certified implicitly
• cfg flag in aws-lc-sys/rs to avoid linking non-certified FIPS symbols but still expose the same public API
  • leaf consumer could manually patch this in via cargo patches, though it is unpleasant

Technically if you have a cfg flag to avoid linking non-fips symbols, you don't actually need the new *-certified crate (though you do still want the new rustls provider). But, it strikes me as much more complicated to maintain a -sys crate build that selectively includes certain symbols based on cfg flag. Compared to just having the sys crate build respect a cfg flag that wipes out ALL symbols.

[justsmth]

The way I see it -- the goal is to ensure that consumers needing FIPS-compliance have the following:

1. Ability to use the latest version of rustls (and/or others crates?)
2. Ability to detect whether (and prevent) an undesired version of aws-lc-rs is actually being used (and not just in the dependency tree).

... are there other requirements? (I mention rustls in (1), but there might be other crates to include.)

This seems promising! But, will this simply panic on usage, or also avoid linking non-FIPS-compliant symbols in the first place?

Right. This is about (2) -- detection/confirmation. The goal should be to remove it from the dependency tree, but this can be difficult or impossible when the dependency comes from a transitive dependency. When use of an undesired aws-lc-rs version is detected, the consumer would need to investigate further to identify how/where it's being used.

It seems like what we would really need is:

• aws-lc-rs-certified and aws-lc-sys-certified that are always on 1.11 (and bump up to aws-lc-3 next certification)

For solving (1), I think a rustls v0.23.x (or later) CryptoProvider that only uses the aws-lc-rs v1.11.1 APIs would unblock consumers. (?)

[jlizen]

Ability to detect whether an undesired version of aws-lc-rs is actually being used (and not just in the dependency tree).

It's more than just about usage though, right? For compliance reasons, presence of the non-FIPS symbols at all in the binary matters?

Saying, "we have a system to detect and prevent usage of non-FIPS APIs therefore it is FIPS-compliant to link in non-FIPS symbols", seems like it is is a claim that itself would require FIPS review? (Insert: standard "i am not a lawyer" disclaimer, this seems like it would need legal consult)

But yes, if detecting usage alone is sufficient, than that does indeed seem far preferable, and having a provider using the certified APIs only + the panic mechanism would suffice since the cargo graph can continue to resolve to the newest aws-lc-rs version. And then, consumers that are sensitive to this can pin to >= whatever version of aws-lc-rs that adds the "panic on non-fips usage" cfg flag support.

[justsmth]

It's more than just about usage though, right? For compliance reasons, presence of the non-FIPS symbols at all in the binary matters?

Right, it's not perfect. It actually would do more than just detect, but should also prevent usage of undesired versions of aws-lc-rs. Whether that is sufficient for compliance is not something I'm at all qualified to discuss.