mirrors/ollama-for-amd
1
0
Fork 0
mirror of https://github.com/likelovewant/ollama-for-amd.git synced 2025-12-22 23:03:55 +00:00
Code Issues Packages Projects Releases Wiki Activity
Files
d3450dd52efcbca0348e9e8ac42fad6263ec9f91
ollama-for-amd/model/bytepairencoding.go
Devon Rifkin fc5fb09f51 model: fix boundary in bpe 
0x007e is a tilde and was getting adjusted (+0x00a2) to 0x0120 in the
encode, but then in the decode it was getting adjusted down (-0x0100) to
0x0020. The boundary for the +0x00a2 case has been adjusted to fix this

Fixes: #11966
2025-08-19 18:34:49 -07:00

5.3 KiB
Raw Blame History

View Raw
Reference in New Issue View Git Blame Copy Permalink