mirrors/ollama-for-amd
1
0
Fork 0
mirror of https://github.com/likelovewant/ollama-for-amd.git synced 2025-12-21 22:33:56 +00:00
Code Issues Packages Projects Releases Wiki Activity
Files
fc5fb09f514758ae9f59b11632c3d8f32e951d49
ollama-for-amd/model/bytepairencoding.go
Devon Rifkin fc5fb09f51 model: fix boundary in bpe 
0x007e is a tilde and was getting adjusted (+0x00a2) to 0x0120 in the
encode, but then in the decode it was getting adjusted down (-0x0100) to
0x0020. The boundary for the +0x00a2 case has been adjusted to fix this

Fixes: #11966
2025-08-19 18:34:49 -07:00

5.3 KiB
Raw Blame History

View Raw
Reference in New Issue View Git Blame Copy Permalink