Jesse Gross a8d9c2648e llamarunner: Record the time for all batches during prompt processing ...

Currently, we only record the time for the last batch when processing
the prompt. This results in unrealistically high numbers for the
old llama runner.

Before:
total duration:       31.273112939s
load duration:        4.97054657s
prompt eval count:    32768 token(s)
prompt eval duration: 235.137439ms
prompt eval rate:     139356.80 tokens/s
eval count:           1873 token(s)
eval duration:        18.173182374s
eval rate:            103.06 tokens/s

After:
total duration:       30.024798033s
load duration:        4.758588663s
prompt eval count:    32768 token(s)
prompt eval duration: 7.779621548s
prompt eval rate:     4212.03 tokens/s
eval count:           1769 token(s)
eval duration:        17.148014223s
eval rate:            103.16 tokens/s

2025-10-22 13:52:58 -07:00

..

cache_test.go

Runner for Ollama engine

2025-02-13 17:09:26 -08:00

cache.go

refactor: use the built-in max/min to simplify the code (#12280)

2025-09-16 17:14:21 -07:00

image_test.go

Llama cpp bump (df1b612): granite docling / mamba2 optimizations / multimodal encoding fixes (#12552)

2025-10-13 15:26:18 -07:00

image.go

Llama cpp bump (df1b612): granite docling / mamba2 optimizations / multimodal encoding fixes (#12552)

2025-10-13 15:26:18 -07:00

runner.go

llamarunner: Record the time for all batches during prompt processing

2025-10-22 13:52:58 -07:00