mirror of
https://github.com/likelovewant/ollama-for-amd.git
synced 2025-12-21 14:26:30 +00:00
ggml: Support closing backends
In order to iteratively find the best memory allocation, we need to be able to free backend memory so we can try again.
This commit is contained in:
@@ -15,6 +15,9 @@ import (
|
||||
)
|
||||
|
||||
type Backend interface {
|
||||
// Close frees all memory associated with this backend
|
||||
Close()
|
||||
|
||||
Load(ctx context.Context, progress func(float32)) error
|
||||
|
||||
// BackendMemory returns the memory allocations that were made for this model
|
||||
|
||||
Reference in New Issue
Block a user