Skip to content
This repository has been archived by the owner on Jan 24, 2024. It is now read-only.

[FEATURE] Add a default protocol configuration for the unknown host #1930

Open
KamenRiderKuuga opened this issue Jun 28, 2023 · 3 comments
Open
Labels
type/feature Indicates new functionality

Comments

@KamenRiderKuuga
Copy link

Is your feature request related to a problem? Please describe.
In our company's internal environment, we use LAN IP to connect to KoP, sometimes we need to proxy this address to the external network address, so that external developers can connect to our KoP.

Due to the external address must be set in kafkaAdvertisedListeners as first, so every time we adjust the address of the proxy server, we need to adjust the KoP configuration at the same time.

Describe the solution you'd like

After configuring the proxy, the external environment can actually access the KoP server, but the protocol corresponding to the address it uses cannot be found.

Because the configuration of Pulsar cannot be hot-updated, and in this case, there is actually no need to update the configuration frequently. It is hoped that there will be a default protocol configuration for unknown hosts to use.

kafkaDefaultProtocol=PLAINTEXT

In this way, no matter what address is used, as long as it can successfully connect to the server, the default protocol will be used

Describe alternatives you've considered
Maybe we can use 0.0.0.0 to represent all other IP addresses?

kafkaAdvertisedListeners=PLAINTEXT://0.0.0.0:9092
@KamenRiderKuuga KamenRiderKuuga added the type/feature Indicates new functionality label Jun 28, 2023
@eolivelli
Copy link
Contributor

eolivelli commented Jun 28, 2023

In the sister project/fork https://github.com/datastax/starlight-for-kafka I am maintaining a Proxy component that allows you to let external clients to connect to KOP.
It is 100% compatible with KOP (so you can keep KOP on the brokers) is a proxy extension

@BewareMyPower
Copy link
Collaborator

@eolivelli Recently there is an internal discussion in SN that whether should introduce your proxy solution again. As you know, the Pulsar proxy based solution was rejected long days ago. Would you mind us porting the proxy back to KoP?

Though at the moment we didn't reach the consensus internally, @ericsyh was investigating the proxy handler based proxy solution recently.

@KamenRiderKuuga
Copy link
Author

@eolivelli Thanks, in the Starlight for Kafka proxy extension I find this:

kafkaAdvertisedListeners must contain the public address that clients will use to connect to the proxy. In the example above we are using pulsar-proxy:9092, but this address is available only inside the Kubernetes cluster. If you are exposing your service outside of the Kubernetes cluster, you must use the public name.

I'm not sure if this can achieve what I mentioned above, let me try.

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
type/feature Indicates new functionality
Projects
None yet
Development

No branches or pull requests

3 participants