Jesse Gross 8a35bb926e runner.go: Increase survivability of main processing loop ...

Currently, if an error occurs during the prep stages (such as
tokenizing) of a single request, it will only affect that request.
However, if an error happens during decoding, it can take down the
entire runner.

Instead, it's better to drop the tokens that triggered the error and try to
keep going. However, we also need to stop when we run out of tokens,
otherwise, this just causes an infinite loop. This is likely the cause
of at least some of the hanging issues that have been reported.

Bug #7573

2024-11-14 17:18:41 -08:00

25 KiB

Raw Blame History

View Raw