Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

DHE: Envoy Support #163

Open
bmingles opened this issue Oct 28, 2024 · 0 comments
Open

DHE: Envoy Support #163

bmingles opened this issue Oct 28, 2024 · 0 comments
Assignees
Labels
enhancement New feature or request
Milestone

Comments

@bmingles
Copy link
Collaborator

bmingles commented Oct 28, 2024

We need to add envoy support to the vscode extension:

  • In a non-envoy configuration, the grpcUrl reflects the worker url + port (currently supported, but we are currently using this to determine where the jsapi is rooted from)
  • In an envoy configuration, the grpcUrl will be the same as the server, but we use an "envoy header" to make things work. The jsapi for a worker will be at /worker/[someenvoyprefix]/jsapi/dh-core.js (this is provided via designated.jsApiUrl prop on the query info for the worker)

Things to implement:

  • The DhcService needs to be refactored to account for the 2 urls
  • Can probably use the jsApiUrl as the source of truth for where to dowload the jsapi. Will need to account for the fact that it is the full URL since current logic appends jsapi/dh-core.js to the grpcUrl
  • Creating the Core+ client will need to include the an envoy prefix header in the options:
    {
       headers: { 'envoy-prefix': envoyPrefix },
    }
    

Findings

Enable http2 in native fetch (via undici config)

http2 can be enabled by setting a flag on the underlying undici implementation

process.env.NODE_TLS_REJECT_UNAUTHORIZED = '0'

setGlobalDispatcher(
  new Agent({
    allowH2: true,
  }),
)

Diagnostics for undici can also be enabled by subscribing to diagnostics channels:
https://github.com/nodejs/undici/blob/main/docs/docs/api/DiagnosticsChannel.md

This works in a basic nodejs application, but we were unable to get things fully working inside of vscode. There seems to be something different in the header objects used (possibly vscode swapping out Buffer or some other data structure) for something more compatible with its env). Regardless of the cause, there is an error with calling push on an undefined object that we couldn't work around.

Minimal example of test here:
https://github.com/bmingles/deephaven-repl/blob/5a9beb190410d942f2a6073b0deb597b43902d56/src/fetchtest.mts

Incorporating DH apis here:
https://github.com/bmingles/deephaven-repl/blob/5a9beb190410d942f2a6073b0deb597b43902d56/src/community.mts

node:http2

I was able to successfully make a connection (and see expected header responses) using node:http2. Colin and I are planning to implement a gRPC transport backed by node:http2 instead of relying on fetch. He will expose a way to provide this to the jsapi (deephaven/deephaven-core#6404), and I will implement the transport on the node side of things.

@bmingles bmingles added enhancement New feature or request triage labels Oct 28, 2024
@bmingles bmingles self-assigned this Oct 28, 2024
@mofojed mofojed removed the triage label Oct 29, 2024
@mofojed mofojed added this to the November 2024 milestone Oct 29, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

2 participants