feat: add test 6.2.20

[aschierl-xitaso]

No description provided.

[aschierl-xitaso]

@tschmidtb51
What is the expected behavior for additional properties appearing in cvss_v4 metrics?
Do we want to get both an error from 6.1.8 (CVSS schema validation using cvss-v4.0.rev.json which has set "unevaluatedProperties": false), and a warning from 6.2.20 (checking for additional properties)?
Currently, the test cases look like that (oasis_csaf_tc-csaf_2_1-2024-6-2-20-02.json and oasis_csaf_tc-csaf_2_1-2024-6-1-08-05.json).
Or should 6.1.8 only complain about the properties that were erroneously present in 4.0 and have been removed in Revision 1?

[tschmidtb51]

@tschmidtb51 What is the expected behavior for additional properties appearing in cvss_v4 metrics? Do we want to get both an error from 6.1.8 (CVSS schema validation using cvss-v4.0.rev.json which has set "unevaluatedProperties": false), and a warning from 6.2.20 (checking for additional properties)? Currently, the test cases look like that (oasis_csaf_tc-csaf_2_1-2024-6-2-20-02.json and oasis_csaf_tc-csaf_2_1-2024-6-1-08-05.json).

Currently, yes. If the CVSS group accepts the changes suggested in cvss-v4.0.rev.json.
However, the TC might change that behavior with switching to Revision 2. I opened an issue to track it: oasis-tcs/csaf#1287

Or should 6.1.8 only complain about the properties that were erroneously present in 4.0 and have been removed in Revision 1?

No. 6.1.8 complains about anything not compliant with the respective schema.

[aschierl-xitaso]

So, in the current draft of CSAF 2.1, the test cases

• oasis_csaf_tc-csaf_2_1-2024-6-1-08-05.json - Mandatory Test: Invalid CVSS (failing example 5)
• oasis_csaf_tc-csaf_2_1-2024-6-1-08-06.json - Mandatory Test: Invalid CVSS (failing example 6)

should be non-failing w.r.t. test 6.1.8 (because cvss-v4.0.1.json allows additional properties), and fail test 6.2.20.

Once the draft switches to cvss-v4.0.rev.json, they should fail test 6.1.8 (but additional properties in cvss_v2 or cvss_v3 remain allowed), and also fail test 6.2.20.

And maybe later, test 6.2.20 is changed so that it no longer complains about additional properties in cvss_v4, so that oasis_csaf_tc-csaf_2_1-2024-6-2-20-02.json is non-faling w.r.t. test 6.2.20.

[tschmidtb51]

We hang in the limbo here: We pointed out to the CVSS folks that the change to the new JSON schema is crucial for us - also as the old values floated around and some implementations might still be carrying those. Therefore, we expect the change to come but it is not there yet.
However, if it does not, we will still keep the test files you mentioned and extent the test description about those cvss_v4 properties...

And maybe later, test 6.2.20 is changed so that it no longer complains about additional properties in cvss_v4, so that oasis_csaf_tc-csaf_2_1-2024-6-2-20-02.json is non-faling w.r.t. test 6.2.20.

This would probably something that would be resolved by filtering those out that are reported through a schema check. Let's see what happens in oasis-tcs/csaf#1288 🫣