API/CLI context enhancements (#11331)

* API: expose context size of loaded models

* CLI: add context UX

This adds a column in the ps output to show the models context size.
This commit is contained in:
Daniel Hiltgen
2025-07-08 11:59:06 -07:00
committed by GitHub
parent 43107b15b9
commit 34088dbcfb
3 changed files with 14 additions and 9 deletions

View File

@@ -1404,6 +1404,9 @@ func (s *Server) PsHandler(c *gin.Context) {
Details: modelDetails,
ExpiresAt: v.expiresAt,
}
if v.Options != nil {
mr.ContextLength = v.Options.NumCtx / v.numParallel
}
// The scheduler waits to set expiresAt, so if a model is loading it's
// possible that it will be set to the unix epoch. For those cases, just
// calculate the time w/ the sessionDuration instead.