Private
Public Access
1
0

Include token count in api.Chunk

And calculate the tokens/chunk for gemini responses, fixing the tok/s
meter for gemini models.

Further, only consider the first candidate of streamed gemini responses.
This commit is contained in:
2024-06-09 20:45:18 +00:00
parent 42c3297e54
commit dfe43179c0
6 changed files with 26 additions and 16 deletions

View File

@@ -111,7 +111,7 @@ func (m Model) Update(msg tea.Msg) (Model, tea.Cmd) {
m.replyCursor.Blink = false
cmds = append(cmds, m.replyCursor.BlinkCmd())
m.tokenCount++
m.tokenCount += msg.TokenCount
m.elapsed = time.Now().Sub(m.startTime)
case msgResponse:
cmds = append(cmds, m.waitForResponse()) // wait for the next response