Why are models divided into smaller chunks?

[satabios]

Why is the model divided into smaller chunks, and how does this benefit inference? Does it assist with shared memory or reduce RAM usage? Any insights or suggestions would be appreciated. Additionally, is there a guide for best practices on this topic?

[chengzhe-xu]

Hi @satabios , thank you for your interest in this repo
Are we talking about some specific sub-project / model?

[satabios]

For example, in the case of

VAD:

• extract_img_feat
• pts_bbox_head

PETR:

• extract_img_feat
• pts_head_memory

are broken down into smaller parts.

1. What is the criteria for breaking the models in such fashion?
2. How does it help system/kernel wise?

I hope this clears the question.

[Jian-danai]

cc @jin-yc10