have `generator` support schema validation

[rfuwa14]

/pkgs/generators/k8s/k8s.nix barely supports the JsonSchema validation (only required field is parsed). While the k8s native schema doesn't add that much of validation in their schema (as of v1.34.0), some CRDs define these validation values. It is nice nixidy support these and integrated into the module.

NOTE

I wonder if /pkgs/generators/k8s/k8s.nix and /pkgs/generators/crd/jsonschema.nix can be merged into a single generator. Major of their logic is shared. And at this point, I think it is easier to manage the generator code in another language like python.

[arnarg]

I prototyped some support for validators in integer, number and string on this branch: https://github.com/arnarg/nixidy/tree/feat/number-validators

Nice diff: https://github.com/arnarg/nixidy/compare/fdbbfbb..1c690a5

The error message is however not super readable 😛

error: A definition for option `applications.miniflux.resources."cilium.io".v2.CiliumNetworkPolicy.allow-postgres-egress.spec.egress."[definition 1-entry 1]".toNodes."[definition 1-entry 1]".matchLabels.blabla' is not of type `string, with maximum length of 63, matching pattern "^(([A-Za-z0-9][-A-Za-z0-9_.]*)?[A-Za-z0-9])?$"'. Definition values:
       - In `/nix/store/9lw2b05k3lrwwk8smdzkchaqs963c8bv-source/modules/cilium/generated.nix': "abcd?"

Regarding your note, I'm undecided what I want to do long term, I kinda like the idea of having it in nix but they can probably be merged. I think I may have put too much pre-processing though in crd2jsonschema.py.

[rfuwa14]

I think I may have put too much pre-processing though in crd2jsonschema.py.

I agree with this statement. As long as we generate the same schema.json from crd manifesst, k8s generator can handle the conversion to nix module. Then we only maintain and improve a signle generator implementaiton.

I think it is ok to keep the generator in nix, but I'd start with refactoring crd2jsonschema.py to produce the same schema.json as ones from k8s.

[arnarg]

The way the k8s generator works is by reading swagger.json from kubernetes' repository. The resource "roots" are found by going through the list of paths the api supports and finding any path that supports POST.

There's no good way (that I could find) to get which resources are namespaced in swagger.json so I started parsing api__v1.json for apiVersion v1 and aggregated_v2.json for the rest.

Additionally, the core k8s resources don't use any jsonschema validators anywhere (that I have found) and just rely on x-kubernetes-* fields.

For these reasons I think it would be better to pre-process the k8s data to be more in-line with the structure that comes out of crd2jsonschema.py. With that I don't mean to mangle the jsonschema so much but to find the roots and output them in the same format as is done for CRDs.

[arnarg]

I had a bit of free time on a train ride and I managed to do the pre-processing I talked about in the previous comment 😄