Does Prompt Caching Work?

[yennor]

Issue

I'm using openrouter do access the models. when i start up with:
aider-ce --cache-prompts --no-stream

It shows me for some models (not all. Eg. for older sonnet models, it shows the prompt cache, but not for the newer 4.5 or 4.6).
"with diff edit format, prompt cache, infinite output"

But when I look at the openrouter infos (export the log), then no cache costs are listed at all.

also from:
https://cecli.dev/docs/usage/caching.html

in the screeen shot, it displays a "cache hit" entry after each prompt. Which aren't shown in cecli.

Version and model info

cecli 0.97.0

[dwash96]

So this definitely depends on LiteLLM being updated to support reading however the model provider returns cached_tokens. With settings:

With configuration:

model: openrouter/anthropic/claude-sonnet-4.6:cache
model-overrides: 
  openrouter/anthropic/claude-sonnet-4.6:
    cache:
      model_settings:
        cache_control: true  

We are sending cache control messages parts:

        {
            "role": "system",
            "content": [
                {
                    "type": "text",
                    "text": "Act as an expert software developer.\nAlways use best practices when coding.\nRespect and use existing conventions, libraries, etc that are already present in the code base.\nThink step-by-step. Plan your changes carefully. Explain your plan first.\n{final_reminders}\nTake requests for changes to the supplied code.\nIf the request is ambiguous, ask questions.\nConsider potential edge cases.\nThink about how to verify your changes are correct.\nOnce you understand the request and have a plan, you MUST:\n1. Decide if you need to propose *SEARCH/REPLACE* edits to any files that haven't been added to the chat. You can create new files without asking!\nBut if you need to propose edits to existing files not already added to the chat, you *MUST* tell the user their full path names and ask them to *add the files to the chat*.\nEnd your reply and wait for their approval.\nYou can keep asking if you then decide you need to edit more files.\n2. Think step-by-step and explain the needed changes in a few short sentences.\n3. Describe each change with a *SEARCH/REPLACE block* per the examples below.\nAll changes to files must use this *SEARCH/REPLACE block* format.\nONLY EVER RETURN CODE IN A *SEARCH/REPLACE BLOCK*!\n{shell_cmd_prompt}\n",
                    "cache_control": {
                        "type": "ephemeral"
                    }
                }
            ]
        }

And receiving:

ModelResponse(id='gen-1771475170-Sv4FPgyXOad5XuEIVyDZ', created=1771475170, model='anthropic/claude-sonnet-4.6', object='chat.completion',system_fingerprint=None, choices=[Choices(finish_reason='stop', index=0, message=Message(content='Hello! How can I help you today?', role='assistant',tool_calls=None, function_call=None, provider_specific_fields={'refusal': None, 'reasoning': None}), provider_specific_fields={'native_finish_reason':'stop'})], usage=Usage(completion_tokens=12, prompt_tokens=2804, total_tokens=2816,completion_tokens_details=CompletionTokensDetailsWrapper(accepted_prediction_tokens=None, audio_tokens=None, reasoning_tokens=0,rejected_prediction_tokens=None, text_tokens=None, image_tokens=None), prompt_tokens_details=PromptTokensDetailsWrapper(audio_tokens=None,cached_tokens=0, text_tokens=None, image_tokens=None, cache_write_tokens=0), cost=0.008592, is_byok=False, cost_details={'upstream_inference_cost':0.008592, 'upstream_inference_prompt_cost': 0.008412, 'upstream_inference_completions_cost': 0.00018}), provider='Google')

With an empty cache_tokens field. It is possible that the vertex API router to claude doesn't report them or the response format is different and LiteLLM can't handle it yet

[yennor]

But the same thing happens, when using claude sonnet 4 or 3.5 (3.5 is the one used in the documentation). So also if it doesn't work (anymore) for the older models? that has to do with litellm?