o1 fails?

[mitchcapper]

First amazing work the proxy does great and easy to setup / follow. Appreciate you building this bridge and giving such detailed instructions for using it with aider and the like.

For the issue, it seems I may be doing something wrong with o1:

If I dump in vscode the models it seems it is avail and listed just as "o1". I tried o1-ga although not sure why it said no o1 language model not no o1-ga language model.

ERROR: No language model available for model: o1
Request received. Model: o1-ga.
ERROR: No language model available for model: o1
Request received. Model: o1.

Trying to use gpt-4o also gave an error but different:

ERROR: Error in streaming mode: {"name":"Error","message":"Invalid request: the last message must be a User message."}
Request received. Model: gpt-4o. 

I did add these things to the model file directly even to get the aider proxy errors above:

- name: o1
  extra_params:
    api_key: n/a
    model: openai/o1
    api_base: http://localhost:14823/v1

- name: GPT-4o
  extra_params:
    api_key: n/a
    api_base: http://localhost:14823/v1   

OK nevermind I was able to get around that GPT-4o error message fixing the case, oddly it showed it in lowercase in VSCode but per above the aider config was upper (even though in aider I did switch to it using /model gpt-4o).

- name: gpt-4o
  extra_params:
    api_key: n/a
    api_base: http://localhost:14823/v1    

vs dump of Available Copilot Models:

[
  {
    "id": "gpt-3.5-turbo",
    "vendor": "copilot",
    "family": "gpt-3.5-turbo",
    "version": "gpt-3.5-turbo-0613",
    "name": "GPT 3.5 Turbo",
    "maxInputTokens": 12114
  },
  {
    "id": "gpt-4",
    "vendor": "copilot",
    "family": "gpt-4",
    "version": "gpt-4-0613",
    "name": "GPT 4",
    "maxInputTokens": 28501
  },
  {
    "id": "gpt-4o",
    "vendor": "copilot",
    "family": "gpt-4o",
    "version": "gpt-4o-2024-05-13",
    "name": "GPT 4o",
    "maxInputTokens": 63827
  },
  {
    "id": "gpt-4o-mini",
    "vendor": "copilot",
    "family": "gpt-4o-mini",
    "version": "gpt-4o-mini-2024-07-18",
    "name": "GPT 4o Mini",
    "maxInputTokens": 12115
  },
  {
    "id": "o1",
    "vendor": "copilot",
    "family": "o1-ga",
    "version": "o1-2024-12-17",
    "name": "o1 (Preview)",
    "maxInputTokens": 19827
  },
  {
    "id": "o3-mini",
    "vendor": "copilot",
    "family": "o3-mini",
    "version": "o3-mini-2025-01-31",
    "name": "o3-mini (Preview)",
    "maxInputTokens": 19827
  },
  {
    "id": "claude-3.5-sonnet",
    "vendor": "copilot",
    "family": "claude-3.5-sonnet",
    "version": "claude-3.5-sonnet",
    "name": "Claude 3.5 Sonnet (Preview)",
    "maxInputTokens": 85734
  },
  {
    "id": "gemini-2.0-flash-001",
    "vendor": "copilot",
    "family": "gemini-2.0-flash",
    "version": "gemini-2.0-flash-001",
    "name": "Gemini 2.0 Flash (Preview)",
    "maxInputTokens": 127827
  }
]

[lutzleonhardt]

Hi Mitch,

Glad you like it :)
I think you need a payed subscription to get access to the more advanced models:

o3-mini is only icluded as a preview without having a sub.

[lutzleonhardt]

@mitchcapper Feel free to reopen the issue if this is not the problem

[mitchcapper]

I can't reopen but yeah not the problem:) Claude, GPT 4o, o3-mini all just fine. I assume Gemini wouldn't as not an openai style endpoint I believe. It at least believes in itself:

[lutzleonhardt]

OK, so it means you have a payed Sub for Copilot?

[mitchcapper]

Sorry yes their maintainer gifted sub which I believe is direct equivalent of their copilot pro. I can use O1 as my model in VS code without a problem.

[lutzleonhardt]

Gemini 2 Flash works for me with this config:

- name: gemini-2.0-flash-001
  extra_params:
    model: openai/gemini-2.0-flash
    api_key: n/a
    api_base: http://localhost:3000/v1

This line (the prefix openai) is telling aider/liteLLM that the endpoint is an openai-like endpoint
model: openai/gemini-2.0-flash

[lutzleonhardt]

@mitchcapper

Can you try this for o1 (I have no sub):

- name: o1
  straming: false
  extra_params:
    model: openai/o1-ga
    api_key: n/a
    api_base: http://localhost:3000/v1

I am not sure about streaming on or off...

[mitchcapper]

Gemini 2 Flash does work.

That may have worked for o1? Oddly in non streaming mode it gives:

Request received. Model: o1-ga. Preview: what is the using command in c
ERROR: Error in non-streaming mode: {"name":"Error","message":"rate limit exceeded\n"}

Although I haven't been using o1 at all (minus to test commands here). In the github chat selecting o1 and sending a message now says the same of wait 35 min and try again. I will be in bed by then but will test tomorrow and see if that does fix.

[mitchcapper]

This worked both with and without streaming mode. I would guess o1 must have some heavy rate limiting by VS for extensions so using it with aider even in diff mode soon seems to go to useless pretty fast.

[mitchcapper]

Given the limited number of VS code copilot options right now would it make sense to manually add them all? I can do a PR if desired.