Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

WIP: Handling 409 response from websocket server #2099

Draft
wants to merge 3 commits into
base: main
Choose a base branch
from
Draft

Conversation

filmaj
Copy link
Contributor

@filmaj filmaj commented Nov 22, 2024

⚠️ Work In Progress!

Related to #2094

TODO

  • Integration test depicting the issue.
  • what should this library do if the websocket server returns a 409 HTTP response during handshake?

@filmaj filmaj added bug M-T: A confirmed bug report. Issues are confirmed when the reproduction steps are documented discussion M-T: An issue where more input is needed to reach a decision server-side-issue draft pkg:socket-mode applies to `@slack/socket-mode` backend Issue related to the Slack API labels Nov 22, 2024
Copy link

codecov bot commented Nov 22, 2024

Codecov Report

Attention: Patch coverage is 0% with 14 lines in your changes missing coverage. Please review.

Project coverage is 91.63%. Comparing base (278514f) to head (6f918b0).
Report is 3 commits behind head on main.

Additional details and impacted files
@@            Coverage Diff             @@
##             main    #2099      +/-   ##
==========================================
- Coverage   91.65%   91.63%   -0.03%     
==========================================
  Files          38       38              
  Lines       10311    10320       +9     
  Branches      647      647              
==========================================
+ Hits         9451     9457       +6     
- Misses        848      851       +3     
  Partials       12       12              
Flag Coverage Δ
cli-hooks 95.24% <ø> (+0.01%) ⬆️
cli-test 94.48% <ø> (+0.01%) ⬆️
oauth 77.39% <ø> (ø)
socket-mode 57.97% <0.00%> (-0.25%) ⬇️
web-api 96.88% <ø> (ø)
webhook 96.65% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

---- 🚨 Try these New Features:

this.logger.debug('WebSocket open event received (connection established)!');
this.websocket = ws;
Copy link
Contributor Author

@filmaj filmaj Nov 22, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not tracking the current websocket instance was a problem in the case the web socket server returns an error. Since only once the open event was emitted would we keep track of the websocket (this line), any errors during the websocket handshake would call into disconnect which, crucially, only terminates the underlying network resources for the tracked this.websocket.

That was the underlying issue that caused the "doubling up" of reconnection attempts (see "Issue 1" described by @gdhardy1 in this comment): each time an error was encountered during websocket handshake, there would be an 'untracked' ws instance still executing reconnect attempts in addition to a new one being spawned. Essentially fork-bombing ourselves - or websocket-instance-bombing ourselves, I guess.

this.logger.debug('Continuing with reconnect...');
this.emit(State.Reconnecting);
cb.apply(this).then(res);
if (this.shuttingDown) {
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

While not a problem brought up by users, I noticed a problem here when writing integration tests:

  • if you turn on the socket mode client and it is seeing errors from the web socket server, and
  • in between reconnect attempts you call disconnect() to stop the client,

.. then the client will keep trying to reconnect forever via this part of the code. Adding this guard here just in case.

@@ -61,10 +61,7 @@ function errorWithCode(error: Error, code: ErrorCode): CodedError {
* A factory to create SMWebsocketError objects.
*/
export function websocketErrorWithOriginal(original: Error): SMWebsocketError {
const error = errorWithCode(
new Error(`Failed to send message on websocket: ${original.message}`),
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Removed "Failed to send message" as part of this error, as this error can be raised unrelated to message-sending. In this PR, I noticed that websocket handshake errors also used this code path, which made the error message misleading.

@@ -168,6 +169,47 @@ describe('Integration tests with a WebSocket server', () => {
assert.isTrue(debugLoggerSpy.calledWith(sinon.match('Unable to parse an incoming WebSocket message')));
await client.disconnect();
});
it('should maintain one serial reconnection attempt if WSS server sends unexpected HTTP response during handshake, like a 409', async () => {
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is the test testing for the exponential reconnection doubling spiral bug.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
backend Issue related to the Slack API bug M-T: A confirmed bug report. Issues are confirmed when the reproduction steps are documented discussion M-T: An issue where more input is needed to reach a decision draft pkg:socket-mode applies to `@slack/socket-mode` server-side-issue
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant