Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add historic storage for keeping track of flagsets #242

Merged
merged 4 commits into from
Oct 19, 2023

Conversation

mredolatti
Copy link
Contributor

Add an optimized historic storage to support flagsets when serving /splitChanges in the split-proxy app.

  • The new component is expected to return all the flags that must be returned given a change number & a list of flagsets. together with it's associations & traffic-type (this data is the minimum required to return a "fake" payload for splits that the user has to remove. This frees the proxy from keeping track of all split definitions it has seen since it started.
  • Based on previous stress-test results , and the reduced number of flagsets & splits, performance tends to be better with contiguos allocated memory chunks using binary search vs constructing maps to check for uniqueness & membership checks. The code gets a little uglier at some point, but reducing allocations & GC pressure proved to improve CPU usage quite a bit in the past. A benchmark was written to validate the approach. it can be run with the command cd splitio/proxy/storage/optimized && go test -bench=. -count=5 -benchmem.

Sample results here for reference:

goos: darwin
goarch: amd64
pkg: github.com/splitio/split-synchronizer/v5/splitio/proxy/storage/optimized
cpu: Intel(R) Core(TM) i9-9980HK CPU @ 2.40GHz
BenchmarkFlagSetProcessing/sorted-slice-16                  6470            194996 ns/op          119632 B/op        190 allocs/op
BenchmarkFlagSetProcessing/sorted-slice-16                  7689            184180 ns/op          120015 B/op        191 allocs/op
BenchmarkFlagSetProcessing/sorted-slice-16                  7058            179562 ns/op          120483 B/op        192 allocs/op
BenchmarkFlagSetProcessing/sorted-slice-16                  7476            183769 ns/op          119587 B/op        190 allocs/op
BenchmarkFlagSetProcessing/sorted-slice-16                  7656            178133 ns/op          119867 B/op        191 allocs/op
BenchmarkFlagSetProcessing/maps-16                          1940            605115 ns/op         1535541 B/op       2380 allocs/op
BenchmarkFlagSetProcessing/maps-16                          1942            615861 ns/op         1534754 B/op       2379 allocs/op
BenchmarkFlagSetProcessing/maps-16                          1915            615651 ns/op         1535475 B/op       2380 allocs/op
BenchmarkFlagSetProcessing/maps-16                          2038            635693 ns/op         1534103 B/op       2378 allocs/op
BenchmarkFlagSetProcessing/maps-16                          1953            606300 ns/op         1535843 B/op       2380 allocs/op
PASS

@mredolatti mredolatti requested a review from a team as a code owner October 17, 2023 22:44
@mredolatti mredolatti changed the base branch from master to project/flagsets October 19, 2023 14:31
@sonarqube-pull-requests
Copy link

@mredolatti mredolatti merged commit 353237e into project/flagsets Oct 19, 2023
6 checks passed
@mredolatti mredolatti deleted the feat/proxy_flagset_optimized_storage branch October 19, 2023 18:40
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants