Twister2 Configuration Standards

Every configuration property should start with the prefix

twister2

Then, it should have the name of the component.

twister2.network

Then it should have the property name. For example here is a property with a name shuffle.memory.limit in the network related configurations.

twister2.network.shuffle.memory.limit

If we specify this property, it is common to both streaming and batch environments. If we want to have specific property for streaming or batch environment we can add stream or batch add the end. For example the following property only applies to streaming environment.

twister2.network.shuffle.memory.limit.stream

Now if we want this to be applied to a specific operation, we need to add the operation name at the end. The following property only applies to streaming keyed-gather operation.

twister2.network.shuffle.memory.limit.stream.keyed-gather.

Now in the configuration files all three properties can be present. In that case, the specific property takes precedent.

twister2.network.shuffle.memory.limit: 1000 twister2.network.shuffle.memory.limit.stream: 10 twister2.network.shuffle.memory.limit.stream.keyed-gather: 100

Also every property in the files should have a clear block of description.