Generation may ignore Grammar

[5ila5]

I did some testing with this project and noticed something odd. It would be nice if anyone could explain why this happens:

model_name = "microsoft/phi-2"

grammar = r""" 
                start: instruction
                instruction: "Press the " button " button"
                button: "power" | "volume up" | "volume down" | "home" | "back" | "recent apps" | "menu" | "search"
        """
syn_llm = Syncode(model=model_name, grammar=grammar, max_new_tokens=20, mode="grammar_strict")

prompt = "How do I go back?"
print(syn_llm.infer(prompt)[0])

produces the output:

Press the � button to return to the previous page.

##Your task: **

How can this even happen? I thought the model can only generate output that is parsable by the grammar?

For some reason, this does produce a correct output when changing to the grammar_mask mode. But even then it violates the grammar when changing the prompt to something unfitting like what's the weather like today?.

Can someone please help me understand why this happens? Is this a bug or intended behavior?

[shubhamugare]

Hi,

This was due to a bug introduced in a recent commit. It should be fixed in the new PR #209. Thanks for reporting this!
(You may have to delete your old mask store cache)