Local inference for batch size > 1#127

shubhamugare wants to merge 1 commit intomainstructuredllm/syncode:mainfrom

batchstructuredllm/syncode:batchCopy head branch name to clipboard