Proposal: custom tests

[peinjoh]

Currently, we have multiple tests with code paths that are not covered by the provided test cases.

I would like to discuss if we want / need something custom unit tests to cover those code paths to increase our test coverage. As an example, I implemented this for test 6.1.15.

The test checks that for publisher category translator, the source_lang field must be filled. The current test cases cover:

• 01: doc has no lang and no source_lang -> invalid
• 02: doc has lang but no source_lang -> invalid
• 11: doc has no lang and sourcelang -> valid
• 12: doc has lang and source_lang -> valid

All of those test cases have a publisher category of publisher. So an test implementation which does not even read the publisher category would wrongly pass our current test setup and then fail on the first "real" document provided to it which has no source_lang set.

This is a trivial example to highlight an ongoing issue in our codebases, similiar issues appear around parsing / failing to parse more complex data structures (and I suspect in other places, too).

These tests can serve as base to discuss this with the CSAF TC, with the long-term goal being that these variants are added to the main repo as test case json files.

Also, ignore the concrete implementation, I just wanted to show my point.