Add DeepHermes to list of supported models

[K-Mistele]

DeepHermes (NousResearch/DeepHermes-3-Llama-3-8B-Preview) is now a supported model and can be specified with VLLM_MODELS in .env.

I tested it using the second cluster where it is now being served. It is important to note that vLLM does not automatically extract reasoning traces into choices[0].message.reasoning_content. This can be enabled with a CLI flag; however currently it only works on deepseek due to the model's tokenizer and the assumption that the reasoning parser makes that each of <think> and </think> will be a single token in the model's tokenizer. However while I was testing it, I found that choices[0].message.reasoning_content was being automatically set, and the reasoning trace generated by DeepHermes was still extracted into it despite the aforementioned constraints.

After a little more debugging, it appears that LiteLLM is automatically extracting the reasoning trace based on the presence of <think> and </think> into the reasoning_content field; it is unclear if this behavior can be disabled.
As a result, it is not necessary to use the vLLM reasoning parser.

Example request:

curl -X POST http://marketagents-cluster-2:4000/v1/chat/completions \
-H "Content-Type: application/json" \
-H "Authorization: Bearer [REDACTED]" \
-d '{
  "model": "NousResearch/DeepHermes-3-Llama-3-8B-Preview",
  "messages": [
    {
      "role": "system",
      "content": "You are a deep thinking AI, you may use extremely long chains of thought to deeply consider the problem and deliberate with yourself via systematic reasoning processes to help come to a correct solution prior to answering. You should enclose your thoughts and internal monologue inside <think> </think> tags, and then provide your solution or response to the problem."
    },
    {
      "role": "user",
      "content": "What is the capital of France?"
    }
  ]
}'

example response:

{
  "id": "chatcmpl-8762dc89423f44228e535a1af062cfca",
  "created": 1740265510,
  "model": "NousResearch/DeepHermes-3-Llama-3-8B-Preview",
  "object": "chat.completion",
  "system_fingerprint": null,
  "choices": [
    {
      "finish_reason": "stop",
      "index": 0,
      "message": {
        "content": "\n\n\nThe capital of France is **Paris**. \n\nParis is not only the administrative, cultural, and political center of France but also a global symbol of art, fashion, and gastronomy. It is home to renowned landmarks such as the Eiffel Tower, the Louvre Museum, and the Champs-Élysées, as well as significant institutions like the French Parliament and the Élysée Palace (residence of the President of France). Established as the capital in the 14th century, Paris has remained the nation’s heart throughout history.",
        "role": "assistant",
        "tool_calls": null,
        "function_call": null,
        "refusal": null,
        "reasoning_content": "Okay, so I need to figure out the capital of France. Hmm, let me start by recalling what I know. France is a country in Europe, right? I think their capital is a well-known city. Maybe Paris? I've heard that Paris is not just the capital but also a major cultural hub with the Eiffel Tower, the Louvre, and all that.\n\nWait, but just to make sure I'm not mixing things up. Are there any other cities that could be the capital of France? I can't think of any right now. Maybe some smaller towns? I don't think so. Isn't France's government based in Paris? Yeah, that makes sense. Also, the French president's official residence is the Élysée Palace, which is in Paris. Plus, Paris is the city with the most iconic landmarks, like the Notre-Dame Cathedral, the Arc de Triomphe, and the Champs-Élysées. So all these points point to Paris being the capital.\n\nBut let me double-check in case I'm missing something. Sometimes there can be cases where a country's capital changes, but I'm pretty sure France's capital has been Paris for a long time. For instance, I remember that during World War II, Paris was a key area under German occupation, and after the war, it remained the capital. Before that, in the 18th and 19th centuries, Paris was the center of many political and cultural movements. So its status as the capital is well-established.\n\nAnother thing to consider is the official language. France's official language is French, and Paris is the main city where the language is most commonly spoken and taught internationally. Also, the Louvre Museum is the most visited museum in the world, and it's in Paris. The city's universities, like the Sorbonne, are also located there. All these cultural and educational institutions reinforce the idea that Paris is the center of French life.\n\nWait, could it be a trick question? Like maybe the capital was moved recently? But from what I know, major geopolitical shifts affecting capital cities are rare unless there's a significant change in government. France hasn't undergone a major political upheaval that would require moving the capital. Also, the country's administrative heart is in Paris. For example, the French Parliament, the Assemblée Nationale and the Senate, are both located in Paris.\n\nJust to cross-verify, I'll think of other countries' capitals. Like the UK's capital is London, Italy's is Rome, Germany's is Berlin, Spain's is Madrid. So following that pattern, France's capital would logically be a major city. The only other contender I can think of is Strasbourg, which is known for the European Parliament but that's not France's own capital. Marseille is a big port city but not the capital. Lyon is a cultural center but not the administrative capital. So yes, Paris is the only candidate.\n\nAnother angle: historical capitals. Before Paris, France had other cities as capitals, like Orléans and Troyes, but those were during medieval times. The transfer to Paris happened under Charles V in the 14th century, I think. Since then, Paris has been the capital without major interruptions. So no, it's definitely Paris now.\n\nI guess there's no other answer here. Unless the question is a joke, but the user asked it seriously. So the answer should be Paris."
      }
    }
  ],
  "usage": {
    "completion_tokens": 818,
    "prompt_tokens": 91,
    "total_tokens": 909,
    "completion_tokens_details": null,
    "prompt_tokens_details": null
  },
  "service_tier": null,
  "prompt_logprobs": null
}

[K-Mistele]

fwiw the model's native tool calling ability is poor since it's not based on the other hermes 3 / llama 3 models but on unsloth AI's llama model, so function calling is not baked into the tokenizer.

[K-Mistele]

FYI, also added:

• fixes to docs for docker build etc.
• patched ./docker-ctl.sh down so that it doesn't destroy the RMQ and postgres data volumes when bringing the node down, since doing so will wipe all of your data.

When stopping the currently-running nodes at node.naptha.ai etc. to apply this update; do not use the docker-ctl.sh down command unless you wish to wipe the postgres databases.

[moarshy]

@K-Mistele @richardblythman - all looks good. Just want to check on this, we are now setting docker-compose to pull node image from the hub instead of building it locally. Quick note about the deploy sequence: Since we have separate GitHub Actions for DockerHub push and deployment, we need to make sure the deploy action only runs after the DockerHub push is complete. Otherwise we might try to deploy before the new image is available on DockerHub.
Would it make sense to combine these into a single workflow file? Here is the PR to deploy #230 . @K-Mistele could you take a look and see whats best?

@richardblythman - Also curious if there were specific reasons you were building Docker images locally in the compose file instead of pulling from DockerHub? Want to make sure we're not missing any important use cases.

[richardblythman]

@K-Mistele PR looks good but testing locally and running the hello world agent doesn't work.

Probably because of using napthaai/node:latest instead of the local Dockerfile. I agree we should pull the image instead of building (unless doing dev work on node). Maybe the latest image is out of date?

[K-Mistele]

@K-Mistele PR looks good but testing locally and running the hello world agent doesn't work.

Probably because of using napthaai/node:latest instead of the local Dockerfile. I agree we should pull the image instead of building (unless doing dev work on node). Maybe the latest image is out of date?

oops did this get resolved @richardblythman ?

[richardblythman]

@K-Mistele yep I think it's all good now. I tested it before merging.