KV Cache with RoPE scaling. #7
Description
Currently RoPE the keys, and cache these. From a 1-minute think, this feels like it won't work with RoPE scaling.
To read:
https://arxiv.org/abs/2309.00071
ggml-org/llama.cpp#2060
{{ message }}
Currently RoPE the keys, and cache these. From a 1-minute think, this feels like it won't work with RoPE scaling.
To read:
https://arxiv.org/abs/2309.00071
ggml-org/llama.cpp#2060