Include token count in api.Chunk
And calculate the tokens/chunk for gemini responses, fixing the tok/s meter for gemini models. Further, only consider the first candidate of streamed gemini responses.
This commit is contained in:
@@ -9,7 +9,8 @@ import (
|
||||
type ReplyCallback func(model.Message)
|
||||
|
||||
type Chunk struct {
|
||||
Content string
|
||||
Content string
|
||||
TokenCount uint
|
||||
}
|
||||
|
||||
type ChatCompletionClient interface {
|
||||
|
||||
Reference in New Issue
Block a user