Daniel Hiltgen bd6c1d6b49 flash attn: add auto mode for llama engine (#13052) ...

* flash attn: add auto mode for llama engine

If the user does not specify fa in the environment, use auto-mode.

* review comments

* ensure kv cache quantized types have FA explicitly enabled

additional review comments

2025-12-12 13:27:19 -08:00

..

llm_darwin.go

Optimize container images for startup (#6547)

2024-09-12 12:10:30 -07:00

llm_linux.go

Optimize container images for startup (#6547)

2024-09-12 12:10:30 -07:00

llm_windows.go

win: lint fix (#10571)

2025-05-05 11:08:12 -07:00

server_test.go

llm: Don't always evict models on CPU-only systems

2025-12-02 10:58:08 -08:00

server.go

flash attn: add auto mode for llama engine (#13052)

2025-12-12 13:27:19 -08:00

status.go

logs: catch rocm errors (#12888)

2025-10-31 09:54:25 -07:00