Read (parse) and write (compile) parameter information from doc strings

[thorwhalen]

We would like to

• detect what argument specification format a doc string uses to describe it's arguments (e.g. rst, google...)
• parse out this information (e.g. a dict that gives us the name and description of function arguments (and other info if available, e.g. type)
• given this parsed information, be able to generate a string (of a given arg description format -- example rst, google) that can be inserted into doc strings

See resolution of issue #35 to see how one can parse doc strings to separate text and doctests.
This current issue should be aware of the tools used there and work with them (or change them to work well in this feature extension).

In general, what we want is be able to both parse and compile doc strings, meaning be able to go from string to semantic structure and vice versa. Be aware of issue #36 in this matter.

Possible applications:

• Reuse python docs in wrapped versions, semi-automatically transforming the base docs to fit the context. For example:
  • In a GUI dispatch, use the argument's short descriptions in info hover-overs.
  • In an http-webservice dispatch, reuse the argument's short descriptions in the OpenAPI specification
  • In a http-webservice dispatch of a function we may want to remove doctests (or one day, transform them to web requests?) from docs
• Write tools to diagnose name and description consistency over a scope as well as take semi-automatic action on these, such as:
  • Use argument descriptions of one function in another containing the same name (and/or type and default): Either in dispatches, or on the function "itself".
  • Create and maintain a glossary, reusing it to "inject" in the (wrapped) functions

Note: i2.scrap.simple_pymint has a pretty ugly doc parser that handles params. Take with a huge grain of salt. It's not well designed.

Appendix: The reusing argument descriptions discomfort

The idea of reusing argument descriptions often makes developers feel uncomfortable and wary.
The arguments they give are usually valid, but only within a limited scope,
the major argument being the obvious one of conflict/collision.

Here are a few thoughts to counter the worries.

We have the same problem with any variable name and it's a real problem -- but we still write code, supported by scope, namespaces, and tools (like IDEs) that mitigate the problem.

Name-based matching is only one method, but we should be able to contextualize with other aspects of arguments such as annotations, defaults, kinds, containing function (and all it's properties, such as name, module, return annotation). We've developed quite a few of these "routing" tools to make contextualizing easier.

I make an effort to say "semi-automatic" to point to the fact that we should explore the spectrum between totally manual and completely automatic. Different contexts warrant different approaches. For example, we can take advantage of these "object (arguments etc.) properties (decriptions etc.) centralization" tools by simply doing diagnosis and possibly action suggestions, but taking no action itself. We can take automatic action but only after user validation. Full automation is only one way to do things, and should always be scoped carefully.

It aligned with separation of concerns and reuse. There is the concern of functionality, and then there's the concern of it's description. Solutions like Bob's "no docs" are ideal to keep them aligned, of course, but clean code is only documentation to someone who reads the code. If there is to be separate natural language documentation, centralizing this documentation will help, not hurt, in keeping documentation and functionality consistent and aligned.

┆Issue is synchronized with this Asana task by Unito