Release of IBM Stream Kafka Toolkit v3.2.5

Release of IBM Stream Kafka Toolkit v3.2.5

What's new in this release and what has changed

This release is a bug-fix release to fix the log4j issue.

#230

For all changes, please see the CHANGELOG

Kafka Toolkit v3.2.4

Release of IBM Stream Kafka Toolkit v3.2.4

What's new in this release and what has changed

This release is a bugfix release to fix the log4j issue.

#230

For all changes, please see the CHANGELOG

Kafka Toolkit v3.2.3

What's new in this release and what has changed

This release is a bugfix release to fix the log4j issue.

#230

For all changes, please see the CHANGELOG

Kafka Toolkit v3.2.2

What's new in this release and what has changed

This release is a bugfix release.

• #228 KafkaConsumer: support submission time parameter for multiple topics

This improvements allows to consume multiple topics from a submission time parameter, like this:

stream <MessageType.StringMessage> ConsumedRecords = KafkaConsumer() {
    param
        propertiesFile: "etc/consumer.properties";
        topic: getSubmissionTimeValue("topics");
}

To subscribe to the three topics "topic1", "topic2", and "topic3", the application can be submitted with the command

streamtool submitjob <sab file> -P topics=topic1,topic2,topic3

For all changes, please see the CHANGELOG

You find the SPL documentation at https://ibmstreams.github.io/streamsx.kafka/docs/user/SPLDoc/.

Kafka Toolkit v3.2.1

What's new in this release and what has changed

This release is a bugfix release.

• #227 Expansion of {applicationDir} in vmArg does not work
• new krb5Debug operator parameter

For all changes, please see the CHANGELOG

You find the SPL documentation at https://ibmstreams.github.io/streamsx.kafka/docs/user/SPLDoc/.

Kafka Toolkit v3.2.0

What's new in this release and what has changed

The most important change in this release is the upgrade of the Kafka client library from version 2.3.1 to 2.5.1. This client version includes bugfixes and supports incremental cooperative rebalancing for Kafka consumers (KIP-429), which has advantages for large consumer groups.

For details, how to activate this feature, please see the KafkaConsumer in the SPL documentation.

For producers, the default partitioning strategy for messages without a key has been changed from round-robin to sticky: A new partition is chosen by the producer client when a batch has been completed for being sent to the broker (the partition is "sticky" as long as a batch is not complete). This feature is part of the new Kafka client. It improves producer throughput for unkeyed messages.

For all changes, please see the CHANGELOG.
You find the SPL documentation at https://ibmstreams.github.io/streamsx.kafka/docs/user/SPLDoc/.

Kafka Toolkit v3.1.3

What's new in this release and what has changed

This release is a bugfix release.

• #223 I18n: update message translation

For all changes, please see the CHANGELOG

You find the SPL documentation at https://ibmstreams.github.io/streamsx.kafka/docs/user/SPLDoc/.

Kafka Toolkit v3.1.2

What's new in this release and what has changed

This release is a bugfix release.

• #224 Application directory cannot be used as file location for files specified via system property and vmArg parameter

  This fix allows a user to bundle a file, for example a configuration file for a security provider, with the application and reference the file in a system property. For example when you place the Kerberos configuration krb5.conf in the etc directory of the application, you can pass the file as vmArg: "-Djava.security.krb5.conf={applicationDir}/etc/krb5.conf"; to the security provider of the JVM. For use of Kerberos authentication see also this article in the toolkit documentation.

• #222 KafkaConsumer: Reset of CR may fail when partitions are revoked before the reset actually happens

For all changes, please see the CHANGELOG

You find the SPL documentation at https://ibmstreams.github.io/streamsx.kafka/docs/user/SPLDoc/.

Kafka Toolkit v3.1.1

What's new in this release and what has changed

• This toolkit release does not deliver the vulnerable version of log4j.jar (#216) anymore.
• The downloadable archive now includes the KafkaConsumerVariableStartPositionSample (#214)

Compared with the previous release v3.1.0, the function of the toolkit is unchanged.

For all changes, please see the CHANGELOG

You find the SPL documentation at https://ibmstreams.github.io/streamsx.kafka/docs/user/SPLDoc/.

Kafka Toolkit v3.1.0

What's new and what has changed

New features in this release:

• #213 KafkaConsumer: dynamic startPosition parameter

This adds a new operator parameter startPositionStr to the KafkaConsumer operator. Its semantic is equivalent to the startPosition parameter, but it takes an expression of type rstring, so that a submission time parameter can be used for the start position of the operator.

Note for the new feature:

When the new startPositionStr parameter is set at submission time, a JobControlPlane operator must be added to the application graph, when the start position can be different from "Default" (what is why you would use this new feature). When time is also an option for the start position, the startTime parameter must be used. startTime will be ignored when the start position is not "Time". The JobControlPlane will not be used by the KafkaConsumer operator when the start position is the default start position. A sample can be found in GitHub here.

For all changes, please see the CHANGELOG

You find the SPL documentation at https://ibmstreams.github.io/streamsx.kafka/docs/user/SPLDoc/.