-
Notifications
You must be signed in to change notification settings - Fork 34
Open
Description
raphy@raphy:/2HardDisk$ mkdir DeepseekCpp
raphy@raphy:/2HardDisk$ cd DeepseekCpp/
raphy@raphy:/2HardDisk/DeepseekCpp$ curl -s https://packagecloud.io/install/repositories/github/git-lfs/script.deb.sh | sudo bash
raphy@raphy:/2HardDisk/DeepseekCpp$ sudo apt-get -y install git-lfs python3-dev build-essential
raphy@raphy:/2HardDisk/DeepseekCpp$ git clone https://huggingface.co/deepseek-ai/deepseek-vl2-tiny
raphy@raphy:/2HardDisk/DeepseekCpp$ git clone https://github.com/andrewkchan/deepseek.cpp.git
raphy@raphy:/2HardDisk/DeepseekCpp/deepseek.cpp$ python3 -m venv .deepseekcpp
raphy@raphy:/2HardDisk/DeepseekCpp/deepseek.cpp$ source .deepseekcpp/bin/activate
(.deepseekcpp) raphy@raphy:/2HardDisk/DeepseekCpp/deepseek.cpp$ pip install .
Installing collected packages: nvidia-cusparselt-cu12, mpmath, typing-extensions, triton, sympy, setuptools, safetensors, nvidia-nvtx-cu12, nvidia-nvshmem-cu12, nvidia-nvjitlink-cu12, nvidia-nccl-cu12, nvidia-curand-cu12, nvidia-cufile-cu12, nvidia-cuda-runtime-cu12, nvidia-cuda-nvrtc-cu12, nvidia-cuda-cupti-cu12, nvidia-cublas-cu12, numpy, ninja, networkx, MarkupSafe, fsspec, filelock, cuda-pathfinder, nvidia-cusparse-cu12, nvidia-cufft-cu12, nvidia-cudnn-cu12, jinja2, cuda-bindings, nvidia-cusolver-cu12, torch, deepseek-cpp
Successfully installed MarkupSafe-3.0.3 cuda-bindings-12.9.4 cuda-pathfinder-1.4.0 deepseek-cpp-0.1.0 filelock-3.25.0 fsspec-2026.2.0 jinja2-3.1.6 mpmath-1.3.0 networkx-3.6.1 ninja-1.13.0 numpy-2.4.2 nvidia-cublas-cu12-12.8.4.1 nvidia-cuda-cupti-cu12-12.8.90 nvidia-cuda-nvrtc-cu12-12.8.93 nvidia-cuda-runtime-cu12-12.8.90 nvidia-cudnn-cu12-9.10.2.21 nvidia-cufft-cu12-11.3.3.83 nvidia-cufile-cu12-1.13.1.3 nvidia-curand-cu12-10.3.9.90 nvidia-cusolver-cu12-11.7.3.90 nvidia-cusparse-cu12-12.5.8.93 nvidia-cusparselt-cu12-0.7.1 nvidia-nccl-cu12-2.27.5 nvidia-nvjitlink-cu12-12.8.93 nvidia-nvshmem-cu12-3.4.5 nvidia-nvtx-cu12-12.8.90 safetensors-0.7.0 setuptools-82.0.0 sympy-1.14.0 torch-2.10.0 triton-3.6.0 typing-extensions-4.15.0
(.deepseekcpp) raphy@raphy:/2HardDisk/DeepseekCpp/deepseek.cpp$ pip install packaging
(.deepseekcpp) raphy@raphy:/2HardDisk/DeepseekCpp/deepseek.cpp$ python convert.py --quant fp16 vl2-tiny ../deepseek-vl2-tiny/
Traceback (most recent call last):
File "/2HardDisk/DeepseekCpp/deepseek.cpp/convert.py", line 577, in <module>
metadata = Metadata(config, tokenizer_config,args.quant, args.n_layers, args.mla, args.bsize)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/2HardDisk/DeepseekCpp/deepseek.cpp/convert.py", line 48, in __init__
arch = config["architectures"][0]
~~~~~~^^^^^^^^^^^^^^^^^
KeyError: 'architectures'
With DeepSeek-V2-Lite it works fine :
(.deepseekcpp) raphy@raphy:/2HardDisk/DeepseekCpp/deepseek.cpp$ python convert.py --quant fp16 v2-lite-f16 ../DeepSeek-V2-Lite/
Converting tensor 78: torch.Size([2816, 2048])48])
Saved shard 0
Converting tensor 158: torch.Size([2816, 2048])48])
Saved shard 1
Converting tensor 238: torch.Size([2816, 2048])48])
Saved shard 2
Converting tensor 269: torch.Size([102400, 2048])])
Saved shard 3
OS: Ubuntu 24.04
Python 3.12.3
gcc 14.2.0
Why with deepseek-vl2-tiny it gives " KeyError: 'architectures'" error?
How to make it work?
Reactions are currently unavailable
Metadata
Metadata
Assignees
Labels
No labels