mirrors/ollama-for-amd
1
0
Fork 0
mirror of https://github.com/likelovewant/ollama-for-amd.git synced 2025-12-21 14:26:30 +00:00
Code Issues Packages Projects Releases Wiki Activity

model: force rope factor 1.0 for Gemma 3 (#13445)

Browse Source
This commit is contained in:
Jeffrey Morgan
2025-12-12 13:27:08 -08:00
committed by GitHub
parent 7730895158
commit 3af5d3b738
1 changed files with 3 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

3
model/models/gemma3/model_text.go
View File

@@ -90,12 +90,15 @@ func newTextModel(c fs.Config) *TextModel {
// Google's Gemma 3 release with sliding window attention does // Google's Gemma 3 release with sliding window attention does
// not use final logit softcapping, and so force it to 0.0 // not use final logit softcapping, and so force it to 0.0
// The QAT weights for Gemma 3 also included an incorrect
// value for the rope scale, so we need to set it to 1.0 here.
// TODO (jmorganca): this should ideally be set to 0.0 in the // TODO (jmorganca): this should ideally be set to 0.0 in the
// model configuration instead of here, as future versions of // model configuration instead of here, as future versions of
// models may include both sliding window attention and final // models may include both sliding window attention and final
// logit softcapping. // logit softcapping.
if slices.Contains(m.TextConfig.slidingWindowPattern, true) { if slices.Contains(m.TextConfig.slidingWindowPattern, true) {
m.TextConfig.finalLogitSoftcap = 0.0 m.TextConfig.finalLogitSoftcap = 0.0
m.TextConfig.ropeScale = 1.0
} }
if numBlocks == gemma27BLayerCount { if numBlocks == gemma27BLayerCount {