Skip to content

新的v3-0324 正确的运行格式是什么 #14

@113514089

Description

@113514089

PYTHONPATH="$PYTHONPATH" TORCH_CUDA_ARCH_LIST="8.9"
ktransformers --model_path deepseek-ai/DeepSeek-V3-0324 \ 这个应该填什么
--gguf_path ./R1Q4GGUF \ 这个是gguf文件的路径,应该用不到了吧?
--port 10002
--optimize_config_path ./ktransformers/optimize/optimize_rules/DeepSeek-V3-Chat.yaml \
--use_cuda_graph
--temperature 0.5
--cpu_infer 120
--port 10002
--max_new_tokens 8192

现在R1Q4KM正常运行, 这个V3-0324 应该怎么运行?
Or you want to start server with transformers, the model_path should include safetensors 这是KT官网的介绍
ktransformers --type transformers --model_path /mnt/data/model/Qwen2-0.5B-Instruct --port 10002 --web True 参数就不详细填了, 就指定model_path 就可以运行吗?

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions