Michael Yang 333e360422 model: handle multiple eos tokens (#10577) ...

* get eos_token_id from generation_config.json

* refactor

* include both ids and strings in trace

* comments

* remove special case for gemma3 special vocab (#10743)

2025-05-16 13:40:23 -07:00

5.9 KiB

Raw Blame History

View Raw