Handle syntax highlight for streamed response #5

New Issue

mlow · 2023-11-18T22:44:26-07:00

mlow commented

2023-11-18 22:44:26 -07:00

It's easy to highlight a message when we have the entire contents of it, but it would also be nice to have syntax highlighting on a message from the LLM as it's being streamed in.

Syntax highlighting with Chroma only works on text blobs (i.e., it can't handle a stream of text), so one possible solution is to update HandleDelayedResponse so that it re-draws the whole received text using every so often. It could be changed to accept a callback which does post-processing on the received text, which gets called numerous times with the up-till-now response as it is being received.

It's easy to highlight a message when we have the entire contents of it, but it would also be nice to have syntax highlighting on a message from the LLM as it's being streamed in. Syntax highlighting with Chroma **only** works on text blobs (i.e., it can't handle a stream of text), so one possible solution is to update `HandleDelayedResponse` so that it re-draws the whole received text using every so often. It could be changed to accept a callback which does post-processing on the received text, which gets called numerous times with the up-till-now response as it is being received.

mlow commented

2024-05-23 16:56:26 -06:00

This is handled fairly well in the TUI by re-rendering the entire incoming message for each chunk that is received.

Closing as completed, may revisit for the CLI later

This is handled fairly well in the TUI by re-rendering the entire incoming message for each chunk that is received. Closing as completed, may revisit for the CLI later

mlow closed this issue

2024-05-23 16:56:26 -06:00

Sign in to join this conversation.

No Label

No Milestone

No project

No Assignees

1 Participants

Notifications

Due Date

The due date is invalid or out of range. Please use the format 'yyyy-mm-dd'.

No due date set.

Dependencies

No dependencies set.

Reference: mlow/lmcli#5