[Feat]: Optimize real-time sidebar fetching

[Spiral-Memory]

As an EmbeddedChat developer,

I need to:
Find better ways to filter messages in sidebars
So That:
There must be a smooth experience while chatting even when the sidebar is opened.
Acceptance Criteria

• Optimize sidebar for "Thread Messages, Pinned Messages, Starred Messages, and Mentioned Messages"

[Spiral-Memory]

Working on this!
Thinking about better approach and also, looking for awesome ideas and discussions from others.

[Suvendu-UI]

can i contribute in this issue ?

[Spiral-Memory]

Yes sure, would love to see if you can find a way to optimise it

[Spiral-Memory]

Let me tell you the current problem. Currently, we use messagesStore to filter out the messages and display them on the sidebar. This process can be a little slow when there are tons of messages in the array.

Additionally, when the sidebar is opened and you send a message, it reiterates to filter the messages, which results in computation for every message.

We want to find a way to filter only when the message matches the filter criteria. This information can be obtained through real-time WebSocket communication. This can be one approach, would love to see a optimised approach

[devanshkansagra]

Hey @Spiral-Memory, I think the message filtering is only required in Thread messages as there are no search boxes othere than Search Messages, File gallery and Thread Messages, and I have observed that only thread message is using messageStore, else others are directly filtered from RCInstance and EmbeddedChatAPI

[Spiral-Memory]

Nope, you need to revisit the code and understand, how exactly are these working 🥹

[devanshkansagra]

Hey, @Spiral-Memory thank you for your response I will take your follow up. I have got one thing to ask do we have to implement filtering in Pinned Messages, Starred Messages, and Mentioned Messages also like the Thread one?

[Spiral-Memory]

See, there are two kinds of filtering.

In any sidebar option, the first filtering step is to retrieve the required messages. For example, it first determines whether a message is pinned or starred. That's one level of filtering (except in the case of files, where we fetch from the API, and the result remains unchanged even if the message changes. To refresh, one has to close and reopen the sidebar).

Let's refer to this as layer 1 filtering and call it "X."

The second filtering is based on the text in the input. So, from "X," the messages matching the search keywords will be filtered and displayed with some debounce mechanism.

Here, we are talking about optimizing the first filtering. The problem with the current approach is that every time a new message arrives, layer 1 filtering happens again, regardless of whether the message contains a particular tag.

We need to modify it so that filtering only occurs if a message with a particular tag arrives, ensuring that we are not reiterating over the messages unnecessarily.

It's hard to explain, what i meant to say, this can be understood only when you explore in the code and how it functions.