You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
In a LLM chat scenario, tokens only need to be streamed from server to client, not the other way around, so SSE might work better than WebSockets, especially in a serverless environment where long-running sessions are difficult to keep alive.
Hi all. WS support requires more invasive changes than I expected. Nevertheless, passing the memory with the request to maintain the state is possible. I made these changes available at #10
To support chat-like langchain applications we need state.
Fast API sessions don't seem very straightforward, so websockets may be the way to go: https://fastapi.tiangolo.com/advanced/websockets/
See also https://github.com/hwchase17/chat-langchain for a reference on how it could be done.
The text was updated successfully, but these errors were encountered: