mirrors/ollama-for-amd
1
0
Fork 0
mirror of https://github.com/likelovewant/ollama-for-amd.git synced 2025-12-23 23:18:26 +00:00
Code Issues Packages Projects Releases Wiki Activity

convert: qwen2 from safetensors (#8408)

Browse Source 
Add native support for converting Qwen2 family models (including Qwen2.5)
from safetensors to gguf format so we can run it.
This commit is contained in:
Bruce MacDonald
2025-01-14 10:34:37 -08:00
committed by GitHub
parent a30f347201
commit f6f3713001
5 changed files with 397 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

2
convert/tokenizer.go
View File

@@ -100,6 +100,8 @@ func parseTokenizer(fsys fs.FS, specialTokenTypes []string) (*Tokenizer, error)
t.Pre = "deepseek-llm"
case "21cde974d587f0d54dc8d56b183cc1e6239600172035c68fbd6d4b9f8da0576e":
t.Pre = "deepseek-coder"
case "1ff7f41064896984db5d1bb6ff64fa4bc29007d08c1b439e505b7392777a319e":
t.Pre = "qwen2"
case "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855":
// noop, empty pretokenizer
default: