GPT Implementation?

[HimBono]

Description

(Describe in detail what you want to work on)

Additional Comments (if any)

Have you read the Contributing Guidelines on Pull Requests on mkdocs?

(Write your answer here.)
Adding GPT API feature to this could make it much more intelligent and broader
at least in the case where an answer is not found it would resolve to it.

[george43g]

it seems to me that AI assistants either make this tool obsolete, or that this tool would need a fairly extensive rewrite to utilise LLMs in order to stay useful - it doesn't seem to me that the accuracy or quality of the responses could be matched in any other way if avoiding utilising AI. I suggest that the tool could be developed further to provide an efficient and free alternative to using paid commercial LLMs such as ChatGPT etc...
By taking advantage of the fact that the questions and answers of this tool:

• always begin with "how do I"
• will always usually be fairly short
• are likely to be technical questions, sent via the terminal or other integrations
• many questions are likely to be repeated, both by users and between users

There area already other tools such as built in AI LLMs/agents that run in terminals (Warp, AmazonQ, etcc) that provide a faster and better experience, however these are mostly commercial products off the top of my head.

That we can use these assumptions in order to extract some value and optimisations from this tool in order to find a niche for it to fulfil its role and usefulness.

For example, the limiteed query size and response size would mean that the token / context length can be limited (although depending on how the tool is used, and ignoring privacy concerns, automatically including context could greatly improve the accuracy of the tool) meaning that the cost and speed of the LLM answer can be greatly optimised, as well as this, given the tool is often used in a CLI and given the short expected length of input and output, the one-shot-prompt used to configure the LLM can also be configured and optimised to expect and respond quick/short one-line answers.
Given the above, the cost of the AI queries could be therefore greatly reduced.

The second optimisation that comes to mind would be using a cache, that groups/batches similar queries. For example, if some kind of DB or memcache/redis could sit in between the client and AI endpoint, any user that fires off a request that has already been answered in the last month could simply receive the cached response, saving even more cost.
If there isn't a direct cache hit, however a fuzzy find / string limilarity algorithm detects that the request is similar to one or more existing question/answer pairs in the cache, then another smaller/faster LLM/AI could be used to then determine if this query is semantically similar enough to an existing one (imagine if same words used in a query but in a different order, essentially having the same meaning) in order to utilise the cache anyway. Then providing a resposne from the cache for that query could in itself be cached, by adding the alternate wording of the query into the cache as well, thereyby bypassing the need to check for question similarity via AI the next time this this query is sent to the server.
All these optimisaitons could lead to a tool being created where short questions and answers recieve a very fast, effiecient and accurate response from the server, whilst keeping the cost of the AI/LLM quite low.
In order to fund the project, sponsorships or advertisers could pay to appear alongside certain queries.