[onert] Add Attention operator in circle_schema.fbs

[glistening]

It introduces Attention operator in circle_schema.

ONE-DCO-1.0-Signed-off-by: Sanggyu Lee sg5.lee@samsung.com

With #16055 and #16056, I confirmed attention op works using GGMA. GGMA generates the exactly same sequence of tokens that HuggingFace transformer (python) generated.

[hseok-oh]

LGTM

[glistening]

@Samsung/nncc_committers

It adds ATTENTION operator, which corresponds to LLamaAttention in modeling_llama.py.

• It is for cpu backend, not supposed to be used in npu backend.
• It does not use the frontends in this repo. Attention op is generated using TICO (newly introduced frontend), especially using [test+operators] Fuse llama attention to circle attention TICO#217.

[glistening]

Attention

Description of Input and Output

input

• 0: state
• 1: wq
• 2: wk
• 3: wv
• 4: wo
• 5: cos
• 6: sin
• 7: mask
• 8: kcache
• 9: vcache
• 10: pos

output

• 0: state

For details, please see #16055.

As of this writing, it supports LLaMA style Attention.
For standard attention (which, injects position embeddings out of decode layers), it will not get cos and sin, (that is, null pointer).

[glistening]

It does not use the frontends in this repo. Attention op is generated using TICO (newly introduced frontend), especially using Samsung/TICO#217.

Again, it does not require any change in luci-based frontends. It is luci-free solution like recently merged RUN_MODEL ¹.
torch.export()-based frontend (TICO) will generate Attention operator, and onert supports Attention kernel.

Footnotes

1. https://github.com/Samsung/ONE/issues/15676#issuecomment-3204541259 ↩

[jinevening]

LGTM Thank you