Pythonista Lab on Testflight: Problems with default LLM

[oscampo]

Thanks for this opportunity to access to Pythonista Lab!!
Today I downloaded ad give an eye. I was amazed to realize that it's possible interact with an LLM, so I run your script and downloaded llama, but whe process finished I get this error:

I have an iPad Pro (model 2024)

[omz]

@oscampo Did the download finish successfully? Could you run this in the console, to see if the model file is where it's expected?

!stat Llama-3.2-3B-Instruct-Q4_K_M.gguf

[oscampo]

The model was downloaded successfully. I get this when run those line of code:

[omz]

Hmm, not sure why this doesn't work for you.

Can you load the model like this?

import llama_cpp
llm = llama_cpp.Llama('Llama-3.2-3B-Instruct-Q4_K_M.gguf')
llm

[oscampo]

Whe I try to load the model I get this error:

[omz]

Okay, thanks! It looks like the llama_cpp library can't be loaded, not quite sure why right now...

Does this work for anyone else?

[Tomkirn]

Unfortunately got the same result.

[omz]

Okay, now that I've installed it directly from TestFlight, I can reproduce the issue. Looking into it.

[omz]

This should be fixed in build 7, which is on TestFlight now, could you try that?

[oscampo]

Hi,
I confirm that invoking %%llama is now working, but lines as "%%llm -c 42" in the example file do not work. I changed it to "%%llama -c 42" and it worked.