Revise, refactor and robust(ize) function normalization tools

[thorwhalen]

Background

Yet one more instance of the "from many-to-many to many-to-one-to-many" trick: Normalizing functions.

When you build a framework that is parametrized by functions provided by the user, you'd need to assume, at some level, something about how these functions can be used within the framework. It's a normal thing to do: Impose a user interface -- an API. The more restrictive the API, the cleaner the framework's code can be -- but also the more annoying it can be for the user to fit their case within these imposed constraints.

Therefore the need for "normalizing" tools -- that is, tools that to help the user get from where they are, what they have, to what they need to talk to the frame work. These tools can be used explicitly (the user uses the tools to prepare their specification) or implicitly (offer a friendly and flexible Postel-principled interface that does a lot of this preparation automatically). Where we should be on the implicit-explicit spectrum depends on the context: Anywhere we are in that spectrum, we need these normalizing tools to implement this separate concern.

Function normalization is one (frequent) instance of this. Even more particular: The function's parameter's kinds.

The kind of a parameter can be a great means to tune the interface of a function, but can become an annoyance when you plug the function in a framework that is meant to do something with that function. What if that framework calls the function with keywords, but you had position-only kinds? Or visa versa. What if your framework doesn't know how to deal with variadics?

For example, being able to go from a function that has the (a, *args, b, **kwargs) (a position only, a variadic positional, a position-or-keyword (the only "nice kind"), and a variadic keyword) to a function that can do the same with only the "nice kind" position-or-keyword (a, args: tuple, b, kwargs: dict) can go a long way.

Tasks

A bunch of (kind-) normalizing functions have popped up organically, following the needs of concrete use cases. These need to be:

• reviewed
• merged (i.e. refactored for more reuse)
• tested for pickability
• probably moved to i2.wrapper
• covered with validation to help the user know what's up before a problem happens
• covered more carefully with tests
• better documented

Here are a list of functions to start with (but there's probably more to hunt):

• In i2.signatures: all_pk_signature (aliased as ch_signature_to_all_pk), normalized_func, ch_variadics_to_non_variadic_kind, tuple_the_args (a partial of ch_variadics_to_non_variadic_kind).
• In i2.wrapper: nice_kinds

Things to consider

┆Issue is synchronized with this Asana task by Unito

[thorwhalen]

@sylvainbonnot says (elsewhere):

the jungle of functions changing kinds:

• ch_kinds: in i2.signatures (copied verbatim in dol/signatures), used in i2.wrapper (as sig.ch_kinds)
• ch_kinds_to_position_or_keyword: in i2.signatures, equal to

def ch_kinds_to_position_or_keyword(self):
        return all_pk_signature(self)

used in i2.wrapper, meshed.dag , copied in dol.signatures

• all_pk_signature: in i2.signatures used by ch_kinds_to_position_or_keyword (in i2.signatures), copied in dol.signatures
• nice_kinds: in i2.wrapper, uses sig.ch_kinds, is used: nowhere.
• ch_func_to_all_pk: depends on all_pk_signature, is defined in i2.signatures definition,
  copied in dol.signatures link, used in i2.multi_object (i2.multi_object), i2.meshed.dag, py2http.decorators
• ch_signature_to_all_pk: defined in i2.signatures (deprecated to all_pk_signature), copied in dol.signatures, used in i2.deco, py2http.decorators
• ch_variadics_to_non_variadic_kind: in i2.signatures, used in meshed.base, copied in dol.signatures. Only treats VP variadics.

[thorwhalen]

What do you propose as a course of action to minimize how many versions we have?

I'd propose:

• Having a general (theorem) function that covers all the needs (according to some parametrization)
• Replace the code contents of all other functions/methods (let's call them corollaries) to use that general function
• Run the doctests of the corollaries (and dependents -- i.e. those objects that use the corollary functions) to make sure those changed corollaries still work.
• Replace any usage of these corollaries with a call to the general function if there's only one call, or partialize and used partialized if several calls.
• At this point, none of the corollaries are used any more, so we can remove them, but let's keep the doctest coverage by adding them to the doctests of the general function, or to tests.

One problem you might encounter in the above plan is that some functions require wrapping, and others don't. I'd propose therefore to have one general function in i2.signatures that takes care of pure signature stuff, and one in i2.wrapper that uses i2.signature, but adds the wrapper stuff (e.g., to carry out the variadics processing).

Another aspect that's missing in that plan is "what about the methods"? Do we really want to eliminate the methods?
I'd say probably not if we already use them (for example Sig.ch_func_to_all_pk).
If there's no (significant) usages though, maybe yes, we can remove the "corollary" method.

Also, we should consider issuing a warning (or raising error) if the use tries to use things like ch_kinds_to_position_or_keyword when there's variadics, unless (a flag) we tell it to ignore the error.
(This might not be applicable to ch_kinds if we want it's interface to resemble it's siblings' interface (ch_defaults, ch_names, ch_annotations).)

_Note: Ignore anything in dol.signatures: It is supposed to be a manually maintained copy of i2.signatures because I didn't want to have a dependency for just a few things I needed. Perhaps we can consider a refactor one day where we include only what dol needs. _