feat: add decryption capability in userdata with Jinja template

[acolombier]

This aims to fix #4417 by implementing the plan proposed in #6655. It uses GPG to pass sensitive information in an encrypted way.

• I have signed the CLA: https://ubuntu.com/legal/contributors
• I have included a comprehensive commit message using the guide below
• I have added unit tests to cover the new behavior under tests/unittests/
  • Test files should map to source files i.e. a source file cloudinit/example.py should be tested by tests/unittests/test_example.py
  • Run unit tests with tox -e py3
• I have kept the change small, avoiding unnecessary whitespace or non-functional changes.
• I have added a reference to issues that this PR relates to in the PR message
• I have updated the documentation with the changed behavior.
  • If the change doesn't change the user interface and is trivial, this step may be skipped.
  • Cloud-config documentation is generated from the jsonschema.
  • Generate docs with tox -e doc.

Proposed Commit Message

feat(userdata): add decryption capability in userdata

When using Jinja template, this commit add a helper function 
which can be used to decrypt an encrypted message at 
runtime, using a GPG private key stored in the cloudinit's 
user keyring, or the keyring pointed by the envvar 
`GNUPGHOME`

Fixes GH-4417

Additional Context

1. This currently requires to add a dependencies to python-gnupg, which might turn out to be controversial. Note that due to the simple use of GPG (--decrypt), it should be possible to also directly wrap GPG invocation by a subprocess Now use the command directly via std lib.
2. I have added documentation, by I expect some user manual text might need updating too. Since I couldn't find where this would be the most relevant, I am keeping this as a TODO, and will follow your feedback as of where to document this.

Test Steps

Unit test can be use to confirm it works. To test on a real setup, make sure to install the GPG key with "encrypt" usage on the cloudinit's user keyring (root by default) or to any keyring pointed by the GNUPGHOME envvar (e.g using systemd).
Encrypt sensitive value and pass it to the userdata in base64 format (see doc/userdata.txt for further details)

Merge type

• Squash merge using "Proposed Commit Message"
• Rebase and merge unique commits. Requires commit messages per-commit each referencing the pull request number (#<PR_NUM>)

[holmanb]

Thanks for the proposal.

Test failures aside, this isn't packaged in Ubuntu main - I expect other distros would need packaging work too - so this approach is a non-starter.

On another note, this only addresses one security concern (secrets in meta-data), while other significant ones remain: platform trust, platform confidentiality, etc. I don't think that this is the path that we would prefer to take - a more comprehensive TPM-based solution would be preferred.

[acolombier]

Thank you for your initial input.

this isn't packaged in Ubuntu main

That's a fair point. Would it make sense to explore this as an optional feature, perhaps enabled conditionally, whether GPG is available or not? (note that I have suggested dropping the added Python deps and use subprocess instead)

this only addresses one security concern (secrets in meta-data), while other significant ones remain

I understand your perspective. To clarify, I actually submitted a separate feature request (#6655) to address userdata sensitive info separately to platform trust, as I see these as distinct but complementary security layers. For context, TPM-based solutions might not always be practical in PaaS environments, where TPMs are often virtualised and don’t provide the level of trust as in bare-metal/IaaS scenarios.
In those cases, the security guarantees the current PR offers - such as verifying the VM image contains the correct GPG key to allow decryption - can still be valuable for protecting sensitive userdata, and is sufficient in most case for PaaS on public cloud.
That said, I’m open to discussion if there are other attack vectors I might be missing. Would it be possible to revisit #6655 and consider this PR as a focused solution for userdata encryption in public cloud contexts? My immediate need is specifically around securing sensitive information in userdata, and it seems that TPM support is a separate effort which I am not equip to implement. (No IaaS available for development) .

[acolombier]

Hi @holmanb, I have pushed a fixup that brings the two mentioned changes above:

• Drop the introduced python-gnupg dependencies: no need to change packages or distribution requirement, everything now relies on std lib (except for testing, tho I could change that too)
• Make this feature optional. If gpg is not installed on the machine, this function will just skip process. This way, no need for gpg to be a dependency.

This certainly would require more tests and likely more explicit documentation, but keen to hear if this is now aiming to something that could be accepted, before I spent more time on it.