Skip to content

🔧 #195 #196: Set router ctx in stream chunks & handle end of stream in case of some errors#203

Merged
roma-glushko merged 4 commits into
developfrom
195-set-router-ctx-in-stream-chunks
Apr 16, 2024
Merged

🔧 #195 #196: Set router ctx in stream chunks & handle end of stream in case of some errors#203
roma-glushko merged 4 commits into
developfrom
195-set-router-ctx-in-stream-chunks

Conversation

@roma-glushko

@roma-glushko roma-glushko commented Apr 15, 2024

Copy link
Copy Markdown
Member

Description

Changes

  • Passed RouterID and ModelID information in the chat stream messages
  • Introduced a new ChatStreamMessage type that joins both chunk and error messages. Removed unneeded context from provider chatStream structs
  • defined a set of possible error codes during chat streaming
  • started simplifying logging by using context-based loggers
  • Introduced finish_reason on the error schema

Testing

glide-chat-stream-demo.mov

…nd error message types & removed all unneeded context from stream objects & defined a set of possible error codes
@codecov

codecov Bot commented Apr 15, 2024

Copy link
Copy Markdown

Codecov Report

Attention: Patch coverage is 73.27586% with 31 lines in your changes are missing coverage. Please review.

Project coverage is 67.19%. Comparing base (9a0ee6d) to head (017b181).
Report is 1 commits behind head on develop.

Files Patch % Lines
pkg/routers/router.go 66.66% 17 Missing ⚠️
pkg/api/http/handlers.go 0.00% 5 Missing ⚠️
pkg/providers/openai/chat.go 25.00% 3 Missing ⚠️
pkg/telemetry/telemetry.go 0.00% 3 Missing ⚠️
pkg/providers/cohere/chat.go 0.00% 1 Missing ⚠️
pkg/providers/cohere/finish_reason.go 0.00% 1 Missing ⚠️
pkg/providers/openai/finish_reasons.go 0.00% 1 Missing ⚠️
Additional details and impacted files
@@             Coverage Diff             @@
##           develop     #203      +/-   ##
===========================================
- Coverage    67.34%   67.19%   -0.15%     
===========================================
  Files           78       78              
  Lines         3586     3573      -13     
===========================================
- Hits          2415     2401      -14     
- Misses        1043     1045       +2     
+ Partials       128      127       -1     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@roma-glushko roma-glushko requested a review from mkrueger12 April 15, 2024 14:08
@roma-glushko roma-glushko merged commit 4a9735c into develop Apr 16, 2024
@roma-glushko roma-glushko deleted the 195-set-router-ctx-in-stream-chunks branch April 16, 2024 19:03
roma-glushko added a commit to EinStack/glide-py that referenced this pull request Apr 16, 2024
- adjusted all stream schemas to reflect changes in EinStack/glide#203
- raise client error if stream was interrupted by error
- introduced finish_reason filed on the error schema to indicate fatal/terminal errors
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

🔧 Signal end of chat stream in case of error 🔧 Set router context for stream response chunks

1 participant