x.crypto.mldsa: add ML-DSA signature algorithm

[cestef]

This adds the ML-DSA signature algorithm, as per FIPS 204. The code added was ported from golang's own implementation of the algorithm, thus why I added their BSD license to the code.

They are also in the process of discussing whether to add this impl to the public go crypto API: golang/go#77626

ML-DSA is NIST's postquantum digital signature standard, which is believed to be resistant against quantum computers. It leverages the shortest vector problem as the one-way function, more specifically the short integer problem.

We implement the three parameters sets: ML-DSA-44, ML-DSA-65 and ML-DSA-87 (specified in FIPS 204, sec. 4).

Even though optional, Pre-Hash ML-DSA is also implemented in prehash.v (we basically sign the hash of the message instead of the message itself), and is disabled by default (configurable on .sign(...) via prehash: <hash-alg>).

The code lives at vlib/x/crypto/mldsa, and XOF was added to the vlib/crypto/sha3 module in xof.v. I can split this into a separate PR if you think this is too much.

import crypto.sha3

// one-shot (existing)
digest := sha3.shake256(message, 32)

// streaming (new)
mut xof := sha3.new_shake256()
xof.write(rho)        
xof.write([u8(row)])   
xof.write([u8(col)])   
coefs := xof.read(256) // squeeze out 256 bytes

// we can read again from the same state
more := xof.read(64)

// reset and reuse
xof.reset()
xof.write(seed)
hash := xof.read(64)

XOF was needed because ML-DSA makes extensive use of it: we already exposed sha3.shakeXXX() oneshot functions, but here we need to stream input in multiple writes before reading output. See FIPS 202 if you're curious about it, I also have a short post with visual explainations of XOFs: https://blog.cstef.dev/posts/sponge

Code is mainly tested by using NIST's ACVP test vectors (615 total):

• ML-DSA-keyGen-FIPS204: nist_keygen_test.v (75 vecs)
• ML-DSA-sigVer-FIPS204: nist_sigver_test.v (180 vecs)
• ML-DSA-sigGen-FIPS204: nist_siggen_test.v (360 vecs)

Example

import x.crypto.mldsa

fn main() {
        // generate a new ML-DSA-65 key pair
        sk := mldsa.PrivateKey.generate(.ml_dsa_65)!
        pk := sk.public_key()

        // sign a message (with an optional context string)
        msg := 'Hello ML-DSA'.bytes()
        sig := sk.sign(msg, context: 'not-a-drill')!

        // verify the signature with the same context
        verified := pk.verify(msg, sig, context: 'not-a-drill')!
        assert verified // true

        // deterministic signing is also available
        sig2 := sk.sign(msg, context: 'not-a-drill', deterministic: true)!
        verified2 := pk.verify(msg, sig2, context: 'not-a-drill')!
        assert verified2 // true
}

Benchmarks (Intel Core Ultra 5 135U)

mldsa_bench_test.v vs BenchmarkMLDSA @ crypto/internal/fips140test

Operation	Param	Go	V	V/Go
KeyGen	ML-DSA-44	165 µs	120 µs	1.38
	ML-DSA-65	250 µs	194 µs	1.29
	ML-DSA-87	328 µs	316 µs	1.04
Sign	ML-DSA-44	275 µs	261 µs	1.06
	ML-DSA-65	453 µs	434 µs	1.04
	ML-DSA-87	498 µs	469 µs	1.06
Verify	ML-DSA-44	127 µs	35 µs	3.66
	ML-DSA-65	186 µs	47 µs	3.94
	ML-DSA-87	273 µs	67 µs	4.10

Closes #26683

---

This replaces #26708, which was accidentally closed during a history rewrite.

[JalonSolov]

Tsk. The Markdown stuff is from @medvednikov changelog edits. Don't worry about those.

[JalonSolov]

Closing/re-opening to re-run CI with latest V.

[JalonSolov]

You have one example in the readme... how about some tests, to guard against regressions?

[blackshirt]

Is this should be marked with @[noinit] tag ? there are constructor for this and i think the outside user should be prohibited to create this opaque directly with sha3.Shake{}

[cestef]

done in 664dde3

[blackshirt]

I think its should follow the io.Writer and hash.Hash.write semantics, so aligns with the rest

[cestef]

the Shake XOF API intentionally differs from io.Writer/io.Reader (I initially implemented it like you mentioned), absorption never fails (so !int is kindof misleading) and squeezing produces exact byte counts (so read(n) []u8 makes more sense than preallocating a buffer imo?). Go's ShakeHash also never returns errors from write/read, the signatures exist only for interface conformance. No harm in implementing a io.Reader/Writer wrapper though, if that's you meant!

[blackshirt]

Even we have no interface XOF, i think this also should follow the io.Reader semantic

[cestef]

You have one example in the readme... how about some tests, to guard against regressions?

@JalonSolov

I was told they should be moved to vlang/slower_tests, see vlang/slower_tests#5

I will be adding a few basic test vectors back here as well, maybe validating against the go impl as @tankf33der mentioned

[tankf33der]

You have one example in the readme... how about some tests, to guard against regressions?

@JalonSolov

I was told they should be moved to vlang/slower_tests, see vlang/slower_tests#5

I will be adding a few basic test vectors back here as well, maybe validating against the go impl as @tankf33der mentioned

@cestef - I remind you that they should have as much coverage as possible, whatever that means — only you can do this, it is your responsibility. Thank you.

[cestef]

I added a gen.vsh script that downloads the golang source at master and adds gen.go into it. we need to do this kind of hack because go hasn't yet put their mldsa in their public crypto api.

we currently test against 12 test vecs (4 per kind, 44/65/87), to regenerate test vectors:

v run vlib/x/crypto/mldsa/testdata/gen.vsh

this will pull https://github.com/golang/go on master to /tmp/go, build the toolchain, add our gen.go and run it.
test vecs will be at vlib/x/crypto/mldsa/testdata/vectors.json, to be then used in mldsa_test.v

is this what you had in mind? @tankf33der

[JalonSolov]

It doesn't need to be tested against the Go code, it just needs to be tested that it gets the "correct" output.

In other words, hash something, then check that against the known hash value. Things like that.

Simple unit tests that make sure the code isn't completely broken.

Small tests like that can go in the main repo.

Larger tests that take longer should go to the slow tests repo.

[cestef]

It doesn't need to be tested against the Go code, it just needs to be tested that it gets the "correct" output.

@JalonSolov
right, we are simply using go's implementation (supposed to be a correct one) to generate those values (once in a while, not in the CI) and compare against them. we also have few roundtrip tests (pk roundtrip, sk roundtrip) and that's it for the unit tests in the main repo.

do you think this is too overkill?

[JalonSolov]

Not as long as they don't slow things down too much. And generating the hashes with Go is completely disconnected.

Technically, Go shouldn't be involved at all, unless we find a problem with the hashes already generated - those should be hard-coded in the tests.

[cestef]

CI having a rough day huh

[tankf33der]

@cestef I played around with the code and got a lot of emotions out of it. I managed to make the code hang on the signature. If you make the private key fields public and modifiable and simply zero out the first value, the code will hang. This is not a problem, right?

import x.crypto.mldsa

fn main() {
	mut pr := mldsa.PrivateKey.generate(mldsa.Kind.ml_dsa_44)!
	pr.t0[0][0] = 0
	_ := pr.sign([]u8{len: 10_000, init: index})!
}

[cestef]

@tankf33der yeah that's expected, ML-DSA signing uses rejection sampling. so if you corrupt the key internals it can loop forever. I'll add a max iteration count as a safeguard though, good catch

[tankf33der]

@cestef - add tests from wycheproof project for mldsa to slower_tests too.

[cestef]

@cestef - add tests from wycheproof project for mldsa to slower_tests too.

see a0f6035