Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

StreamBuilder causing data races #82

Open
nickzelei opened this issue Jul 19, 2024 · 0 comments
Open

StreamBuilder causing data races #82

nickzelei opened this issue Jul 19, 2024 · 0 comments

Comments

@nickzelei
Copy link

nickzelei commented Jul 19, 2024

We are using Bento as a part of our Go process and we spawn multiple bento syncs at once.
We instantiate a new bento environment and stream builder for each one.

This seems to be causing data races when running go tests with -race flag.

Here is a really simple repo where this can be easily reproduced: https://github.com/nickzelei/benthos-stream-race

Here is the full stack trace:

WARNING: DATA RACE
Read at 0x00c0000f7808 by goroutine 17:
  github.com/warpstreamlabs/bento/internal/docs.FieldSpecs.SetDefault()
      /Users/nick/go/pkg/mod/github.com/warpstreamlabs/bento@v1.1.0/internal/docs/field.go:655 +0xb8
  github.com/warpstreamlabs/bento/internal/docs.FieldSpecs.SetDefault()
      /Users/nick/go/pkg/mod/github.com/warpstreamlabs/bento@v1.1.0/internal/docs/field.go:660 +0x15c
  github.com/warpstreamlabs/bento/public/service.NewStreamBuilder()
      /Users/nick/go/pkg/mod/github.com/warpstreamlabs/bento@v1.1.0/public/service/stream_builder.go:80 +0x164
  github.com/warpstreamlabs/bento/public/service.(*Environment).NewStreamBuilder()
      /Users/nick/go/pkg/mod/github.com/warpstreamlabs/bento@v1.1.0/public/service/environment.go:106 +0x4c
  main.sync()
      /Users/nick/code/playground/benthos-stream-race/main.go:51 +0x48
  main.main.func1()
      /Users/nick/code/playground/benthos-stream-race/main.go:38 +0x30
  golang.org/x/sync/errgroup.(*Group).Go.func1()
      /Users/nick/go/pkg/mod/golang.org/x/sync@v0.6.0/errgroup/errgroup.go:78 +0x7c

Previous write at 0x00c0000f7808 by goroutine 18:
  github.com/warpstreamlabs/bento/internal/docs.FieldSpecs.SetDefault()
      /Users/nick/go/pkg/mod/github.com/warpstreamlabs/bento@v1.1.0/internal/docs/field.go:662 +0x214
  github.com/warpstreamlabs/bento/internal/docs.FieldSpecs.SetDefault()
      /Users/nick/go/pkg/mod/github.com/warpstreamlabs/bento@v1.1.0/internal/docs/field.go:660 +0x15c
  github.com/warpstreamlabs/bento/public/service.NewStreamBuilder()
      /Users/nick/go/pkg/mod/github.com/warpstreamlabs/bento@v1.1.0/public/service/stream_builder.go:80 +0x164
  github.com/warpstreamlabs/bento/public/service.(*Environment).NewStreamBuilder()
      /Users/nick/go/pkg/mod/github.com/warpstreamlabs/bento@v1.1.0/public/service/environment.go:106 +0x4c
  main.sync()
      /Users/nick/code/playground/benthos-stream-race/main.go:51 +0x48
  main.main.func2()
      /Users/nick/code/playground/benthos-stream-race/main.go:41 +0x30
  golang.org/x/sync/errgroup.(*Group).Go.func1()
      /Users/nick/go/pkg/mod/golang.org/x/sync@v0.6.0/errgroup/errgroup.go:78 +0x7c

Goroutine 17 (running) created at:
  golang.org/x/sync/errgroup.(*Group).Go()
      /Users/nick/go/pkg/mod/golang.org/x/sync@v0.6.0/errgroup/errgroup.go:75 +0x10c
  main.main()
      /Users/nick/code/playground/benthos-stream-race/main.go:37 +0x70

Goroutine 18 (running) created at:
  golang.org/x/sync/errgroup.(*Group).Go()
      /Users/nick/go/pkg/mod/golang.org/x/sync@v0.6.0/errgroup/errgroup.go:75 +0x10c
  main.main()
      /Users/nick/code/playground/benthos-stream-race/main.go:40 +0x80

The fix I've found is to simply wrap the builder code in a global mutex. It's safe to use after the streambldr.Build call.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant