Skip to content

[test] Add LlamaDecoderLayerWithKVCache using captured input#215

Closed
glistening wants to merge 3 commits intoSamsung:mainfrom
glistening:decoderlayer
Closed

[test] Add LlamaDecoderLayerWithKVCache using captured input#215
glistening wants to merge 3 commits intoSamsung:mainfrom
glistening:decoderlayer

Conversation

@glistening
Copy link
Contributor

@glistening glistening commented Jul 17, 2025 •
edited
Loading

It adds LlamaDecoderLayerWithKVCache test, which uses captured input.

TICO-DCO-1.0-Signed-off-by: Sanggyu Lee sg5.lee@samsung.com

./ccex test -m LlamaDecoderLayerWithKVCache
open llama.decoderlayer.circle

[test] Add LlamaDecoderLayer using captured input
55fa695 
It adds LlamaDecoderLayer test, which uses captured input.

TICO-DCO-1.0-Signed-off-by: Sanggyu Lee <sg5.lee@samsung.com>
@glistening
Copy link
Contributor Author

glistening commented Jul 17, 2025

It is a few line modified version from #208.
It generates circle model for LlamaDecoderLayer while #208 generates the whole LlamaModel for prefill pass with padded user tokens.

@glistening
Copy link
Contributor Author

glistening commented Jul 17, 2025 •
edited
Loading

Hmm... ./ccex format on my local does not show any error. But CI complains. I will fix it manually.

(ADD) Ah, by default ./ccex format fixed the format error. I added the change.

glistening
glistening commented Jul 17, 2025
View reviewed changes
test/modules/model/LlamaDecoderLayerWithCache/model.py Outdated
Comment on lines 7 to 39
Copy link
Contributor Author

@glistening glistening Jul 17, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It would be good to put this code in somewhere like TICO/util.

test/modules/model/LlamaDecoderLayerWithCache/model.py Outdated
Comment on lines 42 to 55
Copy link
Contributor Author

@glistening glistening Jul 17, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this code can be generated simply using gpt for new models.

@glistening glistening mentioned this pull request Jul 17, 2025
Open
@glistening glistening changed the title [test] Add LlamaDecoderLayer using captured input [test] Add LlamaDecoderLayerWithKVCache using captured input Jul 17, 2025
@glistening glistening force-pushed the decoderlayer branch 4 times, most recently from 99e8496 to 040cca8 Compare July 17, 2025 06:06
Rename
fe498e2 
- forward_old → forward_org
- output filename : llama → tinyllama
- LlamaDecoderLayerWithCache → LlamaDecoderLayerWithKVCache
@glistening
Copy link
Contributor Author

glistening commented Nov 4, 2025

See #217.

@glistening glistening closed this Nov 4, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@glistening

Comments