split model into gpus

[saintvrai]

how can i split the model into several gpu using the provided config.yaml? for example i want to split the devices into 3 id and 4 id.

# --- settings

id: "collider"
host: 0.0.0.0
port: 8080
log: collider.log
deadline: 180
swap: "/home/sessions"
debug:

# --- pods
pods:

  -
    model: default
    threads: 12
    gpus: [37]
    batchsize: 512

# --- models

models:

  -
    id: default
    name: ministral
    path: /home/ubuntu/Downloads/Ministral-8B-Instruct-2410-Q8_0.gguf
    locale: ru_RU

    preamble: ""
    prefix: "\nUSER: "
    suffix: "\nASSISTANT:"

    contextsize: 60000
    predict: 2048

    janus: 1
    depth: 200
    scale: 0.97
    hi: 0.99
    lo: 0.96

    temperature: 0.1
    top_k: 8
    top_p: 0.4
    typicalp: 1.0
    repetition_penalty: 1.1
    penaltylastn: -1

    mirostat: 2
    mirostatent: 0.1
    mirostatlr: 0.1