Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Calculate topology based on registered data nodes #324

Closed
wants to merge 9 commits into from

Conversation

lujiajing1126
Copy link
Contributor

@lujiajing1126 lujiajing1126 changed the title setup selector Calculate topology based on registered data nodes Sep 6, 2023
@lujiajing1126 lujiajing1126 added the enhancement New feature or request label Sep 6, 2023
@lujiajing1126 lujiajing1126 self-assigned this Sep 6, 2023
@lujiajing1126 lujiajing1126 added this to the 0.5.0 milestone Sep 6, 2023
@codecov-commenter
Copy link

codecov-commenter commented Sep 6, 2023

Codecov Report

Merging #324 (93c1b56) into main (8996af5) will decrease coverage by 0.59%.
Report is 1 commits behind head on main.
The diff coverage is 12.45%.

@@            Coverage Diff             @@
##             main     #324      +/-   ##
==========================================
- Coverage   40.57%   39.99%   -0.59%     
==========================================
  Files         110      113       +3     
  Lines       11775    12021     +246     
==========================================
+ Hits         4778     4808      +30     
- Misses       6528     6741     +213     
- Partials      469      472       +3     
Files Changed Coverage Δ
banyand/liaison/grpc/measure.go 1.76% <0.00%> (-0.09%) ⬇️
banyand/liaison/grpc/property.go 0.00% <0.00%> (ø)
banyand/liaison/grpc/stream.go 2.02% <0.00%> (-0.14%) ⬇️
banyand/metadata/schema/property.go 0.00% <0.00%> (ø)
banyand/metadata/schema/schema.go 58.82% <ø> (ø)
bydbctl/internal/cmd/rest.go 52.31% <ø> (ø)
pkg/node/interface.go 5.40% <5.40%> (ø)
banyand/liaison/grpc/node.go 10.00% <10.00%> (ø)
banyand/liaison/grpc/discovery.go 45.58% <40.00%> (-0.40%) ⬇️
pkg/node/maglev.go 64.28% <64.28%> (ø)
... and 3 more

... and 2 files with indirect coverage changes

📣 We’re building smart automated test selection to slash your CI/CD build times. Learn more

@lujiajing1126
Copy link
Contributor Author

Generally be ready to be reviewed. Test cases still need to need polished.

@lujiajing1126 lujiajing1126 marked this pull request as ready for review September 18, 2023 15:22
}, flags.EventuallyTimeout).Should(HaveLen(1))
}

// TODO: refactor the following startup logic
Copy link
Contributor Author

@lujiajing1126 lujiajing1126 Sep 18, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Any idea how we can better organize such (fine-grained) services setup @hanahmily

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would you be able to use a mock to test the behavior of the selector?

I have introduced a cluster integration in #333 . We can use it to test the cluster selector.

}, flags.EventuallyTimeout).Should(HaveLen(1))
}

// TODO: refactor the following startup logic
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would you be able to use a mock to test the behavior of the selector?

I have introduced a cluster integration in #333 . We can use it to test the cluster selector.

}

// NewClusterNodeRegistry creates a cluster-aware node registry.
func NewClusterNodeRegistry(metaRepo metadata.Repo, selector node.Selector) NodeRegistry {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

#333 will use queue.Client to listen to node events, reducing the need for the liaison node to set up two etcd watchers. Please merge this once PR is merged.

@lujiajing1126
Copy link
Contributor Author

I would raise another PR since there are too many conflicts.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[Feature] Implement Shard Topology Calculation and Caching in BanyanDB Liaison Nodes
3 participants