mirrors/ollama-for-amd
1
0
Fork 0
mirror of https://github.com/likelovewant/ollama-for-amd.git synced 2025-12-21 14:26:30 +00:00
Code Issues Packages Projects Releases Wiki Activity
Files
0682dae0275af6ab376cfa346ef27562b574684d
ollama-for-amd/sample
History
Parth Sareen 0682dae027 sample: improve ollama engine sampler performance (#9374) 
This change bring in various interface cleanups along with greatly improving the performance of the sampler.

Tested with llama3.2 on local machine.
Improves performance from ~ 70 tokens/s -> 135 tokens/s with topK(40) enabled.
Without topK performance is ~ 110 tokens/s
2025-03-07 12:37:48 -08:00
..
samplers_benchmark_test.go
sample: improve ollama engine sampler performance (#9374)
2025-03-07 12:37:48 -08:00
samplers_test.go
sample: improve ollama engine sampler performance (#9374)
2025-03-07 12:37:48 -08:00
samplers.go
sample: improve ollama engine sampler performance (#9374)
2025-03-07 12:37:48 -08:00
transforms_test.go
sample: improve ollama engine sampler performance (#9374)
2025-03-07 12:37:48 -08:00
transforms.go
sample: improve ollama engine sampler performance (#9374)
2025-03-07 12:37:48 -08:00