Support CUDA graphs #14
Description
CUDA graphs would be a great optimization to have, but we need to also support dynamic batching, so we will have to cache them.
{{ message }}
CUDA graphs would be a great optimization to have, but we need to also support dynamic batching, so we will have to cache them.