Skip to content

Releases: apache/flink-cdc

Release 3.2.1

27 Nov 08:12
@leonardBang leonardBang
release-3.2.1
c703d18
Compare
Choose a tag to compare
Release 3.2.1 Latest
Latest

Release Announcement

https://flink.apache.org/2024/11/27/apache-flink-cdc-3.2.1-release-announcement/

Release Notes

https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12315522&version=12355099

Contributors

We would like to express gratitude to all the contributors working on this release:

yuxiqian, Xin Gong, Hang Ruan, wudi, qg-lin, Timi988, lvyanquan, ConradJam, Runkang He, Junbo wang, MOBIN, Leonard Xu, Sergei Morozov, liuzeshan

Downloads

We highly recommend all users to upgrade to Flink CDC 3.2.1.

Apache Flink CDC 3.2.1 tar

Apache Flink CDC 3.2.1 (asc, sha512)

Apache Flink CDC 3.2.1 Source Release (asc, sha512)

Pipeline Connectors Jars

  • MySQL Pipeline Connector 3.2.1 (jar, asc, sha1)
  • Doris Pipeline Connector 3.2.1 (jar, asc, sha1)
  • StarRocks Pipeline Connector 3.2.1 (jar, asc, sha1)
  • Kafka Pipeline Connector 3.2.1 (jar, asc, sha1)
  • Paimon Pipeline Connector 3.2.1 (jar, asc, sha1)
  • Elasticsearch Pipeline Connector 3.2.1 (jar, asc, sha1)

Source Connector Jars

For more details, check the updated documentation and the release notes. We encourage you to download the release and share your feedback with the community through the Flink mailing lists or JIRA.

Assets 2
Loading

Release 3.2.0

20 Sep 16:31
@leonardBang leonardBang
release-3.2.0
This tag was signed with the committer’s verified signature.
PatrickRen Qingsheng Ren
GPG key ID: E7AF737F59524D6C
Learn about vigilant mode.
6b9dda3
Compare
Choose a tag to compare
Release 3.2.0

Release Announcement

https://flink.apache.org/2024/09/05/apache-flink-cdc-3.2.0-release-announcement/

Release Notes

https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12315522&version=12354594

Contributors

We would like to express gratitude to all the contributors working on this release:

ChengJie1053, ConradJam, FangXiangmin, GOODBOY008, Hang Ruan, He Wang, Hongshun Wang, Jiabao Sun, Joao Boto, Junbo wang, Kunni, Laffery, Leonard Xu, MOBIN, Muhammet Orazov, North Lin, PONYLEE, Paul Lin, Qingsheng Ren, SeungMin, Shawn Huang, Thorne, Wink, Xie Yi, Xin Gong, Zhongmin Qiao, Zmm, gong, gongzhongqiang, hk__lrzy, joyCurry30, lipl, lvyanquan, ouyangwulin, skylines, wuzexian, yanghuaiGit, yux, yuxiqian, ιΌŽζ˜•

Downloads

Apache Flink CDC 3.2.0 tar

Apache Flink CDC 3.2.0 (asc, sha512)

Apache Flink CDC 3.2.0 Source Release (asc, sha512)

Pipeline Connectors Jars

  • MySQL Pipeline Connector 3.2.0 (jar, asc, sha1)
  • Doris Pipeline Connector 3.2.0 (jar, asc, sha1)
  • StarRocks Pipeline Connector 3.2.0 (jar, asc, sha1)
  • Kafka Pipeline Connector 3.2.0 (jar, asc, sha1)
  • Paimon Pipeline Connector 3.2.0 (jar, asc, sha1)
  • Elasticsearch Pipeline Connector 3.2.0 (jar, asc, sha1)

Source Connector Jars

Assets 2
Loading

Release 3.1.1

20 Jun 06:18
@leonardBang leonardBang
release-3.1.1
b163b8e
Compare
Choose a tag to compare
Release 3.1.1

Release Announcement

https://flink.apache.org/2024/06/18/apache-flink-cdc-3.1.1-release-announcement/

Release Notes

https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12315522&version=12354763

Downloads

Apache Flink CDC 3.1.1 tar

Apache Flink CDC 3.1.1 (asc, sha512)

Apache Flink CDC 3.1.1 Source Release (asc, sha512)

Pipeline Connectors Jars

Source Connector Jars

Assets 2
Loading

Release 3.1.0

24 May 11:05
@leonardBang leonardBang
release-3.1.0
5452f30
Compare
Choose a tag to compare
Release 3.1.0

Release Announcement

https://flink.apache.org/2024/05/17/apache-flink-cdc-3.1.0-release-announcement/

Release Notes

https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12315522&version=12354387

Downloads

Apache Flink CDC 3.1.0 tar

Apache Flink CDC 3.1.0 (asc, sha512)

Apache Flink CDC 3.1.0 Source Release (asc, sha512)

Pipeline Connectors Jars

Source Connector Jars

Assets 2
Loading

Release 3.0.1

22 Jan 02:16
@lvyanquan lvyanquan
release-3.0.1
bb3a4fb
Compare
Choose a tag to compare
Release 3.0.1

Release Notes

Documentation: https://ververica.github.io/flink-cdc-connectors/release-3.0

Downloads

Flink CDC

Flink CDC 3.0.1 (tar, asc, sha512)

The binary release can be verified by public key with fingerprint F8822354BF424A5CFF9AFD57BAF388953EF5327A.

Flink CDC Pipeline Connectors

All connectors are release in JAR and available in Maven central repository.

  • Apache Doris pipeline connector 3.0.1 (jar, asc, sha1)
  • MySQL pipeline connector 3.0.1 (jar, asc, sha1)
  • StarRocks pipeline connector 3.0.1 (jar, asc, sha1)
  • Values pipeline connector 3.0.1 (for testing purpose, jar, asc, sha1)

Flink CDC Source Connectors

All connectors are release in JAR and available in Maven central repository.

Improvements and Bug fixes

1.[cdc-base][hotfix] Fix SourceSplitSerializer to get totalFinishedSplitSize from recovery in 4th version
2.[hotfix][cdc-common] Fix NoSuchMethodError of CollectionUtil.newHashMapWithExpectedSize
3.[cdc][e2e] Unify logic to clean docker image when connector e2e test finished (#2978)
4.[pipeline-connector][mysql] fix timestamp with timezone format (#2952)
5.[doc] fix invalid page in website and update yaml in tutorial.
6.[cdc-connector][oracledb] Fix oracle read exceed high_watermark in backfill phase
7.[pipeline-connector][starrocks] Optimize char type mapping for starrocks pipeline connector (#2895)
8.[pipeline-connector][doris][starrocks] Remove SimpleDateFormat for thread safe (#2954)
9.[build] Add github labeler for pipeline connector. (#2851)
10.[docs][cdc-connector][oceanbase] update docker image and os compatibility in quickstart (#2897)
11.[docs] Add Supported Connectors and Supported Flink Versions to Overview. (#2920)
12.[cdc-common][pipeline-doris] Fix typo and wrong value for 'precision' of time type. (#2886)
13.[cdc-connector][mongodb] Avoid mongodb source to read data after high_watermark in backfill phase (#2893)
14.[cdc-connector][oceanbase] Fallback to current timestamp when query gts failure (#2868)
15.[cdc-connector][sqlserver][tests] Fix UT errors by correcting right output (#2864)
16.[cdc] Add Deprecated annotation to source function which connector implemented incremental
17.[hotfix] Remove mysqlToDoris.yml
18.[hotfix] Fix BinaryRecordData deserialize
19.[postgres] Add commits of PostgresConnection back to distinguish with debezium's origin class (#2839)
20.[hotfix][cdc-rumtime][minor] Clean up useless code in BinaryRecordDataSerializer (#2845)
21.[cdc-connector][sqlserver] Fix SqlServerTableSource params order error (#2847)
22.[docs] Fix overview docs to remove prefix of pipeline options (#2833)
23.[3.0][docs] fix mysql pipeline connector option in readme (#2834)

Contributor List (sorted alphabetically)

  1. GOODBOY008
  2. He Wang
  3. Hongshun Wang
  4. joyCurry30
  5. Kunni
  6. Shawn Huang
  7. TorinJie
  8. Xin Gong
  9. gongzhongqiang
  10. xuzifu666
  11. εΌ η”°
Assets 5
Loading

Release 3.0.0

07 Dec 03:02
@leonardBang leonardBang
release-3.0.0
ba0d8ba
Compare
Choose a tag to compare
Release 3.0.0

Release Notes

Documentation: https://ververica.github.io/flink-cdc-connectors/release-3.0

Downloads

Flink CDC

Flink CDC 3.0.0 (tar, asc, sha512)

The binary release can be verified by public key with fingerprint C460A307EAF2701FC9D774EF96719B746D3D0C8A.

Flink CDC Pipeline Connectors

All connectors are release in JAR and available in Maven central repository.

  • Apache Doris pipeline connector 3.0.0 (jar, asc, sha1)
  • MySQL pipeline connector 3.0.0 (jar, asc, sha1)
  • Starrocks pipeline connector 3.0.0 (jar, asc, sha1)
  • Values pipeline connector 3.0.0 (for testing purpose, jar, asc, sha1)

Flink CDC Source Connectors

All connectors are release in JAR and available in Maven central repository.

Features and Improvements

  1. [pipeline-connector][starrocks] Fix char/varchar length inconsistency between cdc and starrocks (#2830)
  2. [cdc-cli][cdc-composer] Use "source-table" and "sink-table" as keywords of route (#2825)
  3. [build][hotfix] Remove duplicate dependency (#2826)
  4. [cdc-cli][cdc-dist] Support loading global config from FLINK_CDC_HOME (#2822)
  5. [cdc-common][hotfix] Remove redundant prefix of pipeline options (#2821)
  6. [cdc-source-connector][oceanbase][e2e] Add OceanBase e2e test case (#2521)
  7. [build] Use flink-shaded-force-shading to force all submodules to generate dependency-reduced-pom.xml
  8. [build] Use ${project.version} instead of ${revision} for dependency version
  9. [build] Skip maven-deploy-plugin for test modules and flink-cdc-dist
  10. [cdc-pipeline-connector][mysql] Fix precision problem of BIT type conversion (#2820)
  11. [cdc-source-connector][mysql] fix deserialization issue on table metadata binlog event (#2682)
  12. [pipeline-connector][mysql] Remove unnecessary serverTimeZone in DebeziumEventDeserializationSchema (#2816)
  13. [build][hotfix] Remove useless suppressions. (#2817)
  14. [cdc-source-connector][mysql] Fix attempted load of isSuspended in MySqlSplitSerializer::deserializeSplit when cdc version <= 2.2.0 (#2550)
  15. [cdc-pipeline-connector][mysql] Ensure the inference of MEDIUMINT_UNSIGNED type matches INT type in table schema (#2811)
  16. [cdc-connector][oceanbase] Add option for obcdc extra configs (#2543)
  17. [pipeline-connector][mysql] Enable send schema change by default (#2815)
  18. [3.0][cdc-runtime] Remove waitForFlushSuccess field in request handler (#2812)
  19. [pipeline-connector][starrocks] Add starrocks pipeline connector (#2765)
  20. [pipeline-connector][doris] add doris pipeline connector. (#2810)
  21. [cdc-pipeline-connector][mysql] Ensure the precision inference of DECIMAL type matches table schema
  22. [3.0][cdc-common & runtime] Remove useless flink imports
  23. [cdc-pipeline-connector][mysql] Add tests for mysql pipeline connector
  24. [cdc-pipeline-connector][mysql] Support more mysql types
  25. [cdc-pipeleine-connecotr][mysql] Send CreateTableEvent in the mysql pipeline connector
  26. [cdc-pipeline-connector][mysql] Parse the alter statement to generate SchemaChangeEvent
  27. [cdc-pipeline-connector][mysql] Introduce mysql cdc pipeline DataSource
  28. [cdc-cli][cdc-composer][cdc-dist] Support submitting job to general Flink environments (#2807)
  29. [cdc-runtime][hotfix] Setup waitFlushSuccess before responding operator to flush. (#2805)
  30. [cdc-common] Introduce "pipeline.local-time-zone" config option which help handle time zone well (#2797)
  31. [cdc-common][hotfix] Remove useless flink imports (#2803)
  32. [build] Let surefire plugin to keep the full stacktrace on test failure (#2804)
  33. [pipeline-common] Use column name to check whether the schema contains specific column or not. (#2801)
  34. [cdc-composer][route][sink] Add name for route and improve name for sink
  35. [cdc-composer][hotfix] Always chain operator by default
  36. [cdc-composer][sink] Use correct name for sink operator
  37. [cdc-runtime] SchemaRegistry should complete the future after making the checkpoint (#2800)
  38. [cdc-connector][sqlserver][test...
Read more
Assets 5
Loading

Release 2.4.2

31 Oct 15:36
@leonardBang leonardBang
release-2.4.2
2fa81c1
Compare
Choose a tag to compare
Release 2.4.2

Release Notes

Documentation: https://ververica.github.io/flink-cdc-connectors/release-2.4

Improvements and Bug fixes

  1. [sqlserver] Fix get latest lsn (#2551)
  2. [sqlserver] Add support bigint type as table split column (#2416)
  3. [hotfix] Fix vitess docker image tag
  4. [e2e] Remove db docker image after test cases are executed. (#2508)
  5. [postgres] Not drop replication slot for stream split (#2436)
  6. [postgres] Close jdbc connection after creating replication slot for stream split (#2425)
  7. [cdc-base] Fix TM hangs caused by uncaught exception (#2511)
  8. [postgres] Close idle readers when snapshot finished (#2400)
  9. [hotfix][mysql] Keep assigned splits in order to fix wrong meta group calculation
  10. [mysql] Add finished unack splits to state for the MysqlSourceReader (#2399)
  11. [cdc-base] Fix parsing error when serializing and deserializing the table name with dot (#2443)
  12. [oracle] Correct the naming error (#2405)
  13. [sqlserver] Add table filter to speed up SqlServerSchema read (#2369)
  14. [hotfix][sqlserver] Fix backfill stream task hang (#2374)
  15. [hotfix][mysql] Trim default value for BIGINT and SMALLINT types when parsing MySQL DDL (#2418)
  16. [Doc] Use redo log instead of binlog in Oracle document (#2408)
  17. [mysql] Filter databases that do not need to be read when discovering tables (#2160)
  18. [test] Fix TiDBE2eITCase by updating the container version to avoid tikv disk full failure (#2402)
  19. [test][postgres] Close postgres containers after tests and fix container reuse bug (#2394)
  20. [mysql] quota the field names when executing mysql query (#2381) (#2388)
  21. [test][mysql] Provide the server-time-zone setting in MySqlSourceExampleTest#testConsumingAllEvents (#2364)
  22. [hotfix][mysql] Catch underlying throwable in snapshot reading.
  23. [hotfix][mysql] Fix failed test LegacyMySqlSourceTest#testSnapshotOnFailedSource (#2384)
  24. [db2] [tests] Fix unstable Db2ConnectorITCase due to cleanup failure
  25. [mongodb][hotfix] Fix SourceRecord range check (#2343)
  26. [polish] Add type parameter for MySqlRecords(#2300)
  27. [doc] fix errors in PostgresParallelSourceExample (#2324)
  28. [docs][oracle] Add an example of Incremental Snapshot based DataStream for Oracle CDC Connector
  29. [mongodb] [tests] Loosen E2E timeout limitations
  30. [build] Add vcs.xml for IDEA (#2064)
  31. [hotfix] update dbz option name 'xx. whitelist' to 'xx.include.list'(xx = database, schema or table) (#2212)

Contributor List (sorted alphabetically)

  1. BIN
  2. FocusComputing
  3. GOODBOY008
  4. Hang Ruan
  5. Hongshun Wang
  6. Jiabao Sun
  7. Kunni
  8. Leonard Xu
  9. Malcolmjian
  10. North.Lin
  11. Paddy Gu
  12. Tyrantlucifer
  13. e-mhui
  14. gongzhongqiang
  15. wuzhenhua
  16. yuxiqian
  17. zhaomin
Assets 2
Loading

Release-2.4.1

21 Jul 09:34
@ruanhang1993 ruanhang1993
release-2.4.1
355aa59
Compare
Choose a tag to compare
Release-2.4.1

Release Notes

Documentation: https://ververica.github.io/flink-cdc-connectors/release-2.4

Features and Improvements

  1. [hotfix] Fix sqlserver monitor same table in other database (#2335)
  2. [debezium] Fix DebeizumSourceFunction can not do savepoint after close (#2259)
  3. [mysql] skip closing reader when the reader received the binlog split (#2261)
  4. [MySQL] Optimize how to construct config table.include.list (#2274)
  5. [oracle] Fix class conflict when using multi cdc sql connector
  6. [MySQL] Fix the startup mode exception message (#2264)
  7. [hotfix][mysql] remove unused code (#2231)
  8. [mysql] Fix assigning duplicate snapshot splits when enable scan newly added tables (#2326)
  9. [docs] Add connector features table to help users quickly understand the supported features of each connector
  10. [mysql] Remove the finished snapshot splits for the binlog split when restoring from the checkpoint (#2292) (#2318)
  11. [hotfix][sqlserver] Fix sqlserver close idle and chunk key column missing (#2310)
  12. [docs] Update Postgres CDC document for slot.name option (#2280)
  13. [postgres] Fix the slot name conflict bug (#2251)
  14. [docs] add docs for the mysql tables without primary keys (#2238)
  15. [oracle] Use oracle connection in context for each reader subtask (#2254)
  16. [hotfix] Update support version (#2256)

Contributor List (sorted alphabetically)

  1. Hang Ruan
  2. Leonard Xu
  3. SileiWu
  4. empcl
  5. gongzhongqiang
  6. skylines
  7. syyfffy
Assets 2
Loading

Release-2.4.0

25 Jun 08:02
@ruanhang1993 ruanhang1993
release-2.4.0
f3d26a1
Compare
Choose a tag to compare
Release-2.4.0

Release Notes

Documentation: https://ververica.github.io/flink-cdc-connectors/release-2.4

Features and Improvements

  1. [hotfix] Add vitess connector to the release profile
  2. [docs][hotfix] Update debezium reference links to 1.9 version
  3. [build] Update the copyright year to 2023 (#2205)
  4. [postgres] Fix postgres e2e test
  5. [postgres] scan.incremental.snapshot.enabled is closed by default
  6. [postgres] Backfill task will be able to end when there is not new change data but read the ending lsn
  7. [postgres] Create slot for backfill task before snapshot reading
  8. [postgres] Prepare a slot for the unique global stream split
  9. [mysql] Fix GTID issues to recover from checkpoint normally in specifying startup mode (#2220)
  10. [doc][hotfix] Fix driver version and correct vitess doc path (#2224)
  11. [vitess] Add e2e case for Vitess connector
  12. [vitess] Add Vitess tests in Azure pipeline
  13. [vitess] Add Vitess CDC connector (#456)
  14. [debezium] Bump debezium version to 1.9.7.Final (#2156)
  15. [oracle] Fix the backfill task not running bug in oracle cdc connector (#2218)
  16. [hotfix][sqlserver] Merge high and low watermark data during snapshot reading process (#2179)
  17. [Postgres] Use Incremental Snapshot Framework for Postgres CDC Connector (#2216)
  18. [oracle] Fix the incremental phase not skip the data that has been read in the snapshot full phase (#2215)
  19. [mongodb] Support specific timestamp startup mode. (#2207)
  20. [tests][mysql] Shutdown mysql container timely for legacy mysql tests
  21. [mongodb][hotfix] Fix regex inference of database or collection only contains dash. (#2209)
  22. [hotfix][build] Use different name for each job
  23. [hotfix][Mysql] MySqlBinlogSplitAssigner should support method isStreamSplitAssigned
  24. [MySQL] Remove deleted tables when using a new table filter and restarting from checkpoint
  25. [build][mysql][e2e] Increase the azure test job timeout to 90 minutes
  26. [mysql] Fix MySql CDC Connector outputs duplicated binlog when enable newly-added-table
  27. [mysql][minor] Print clear log information for key path
  28. [mongodb][hotfix] Fix drop or rename record cause documentKey being empty. (#2210)
  29. [ci][oceanbase] Only compile necessary modules for self hosted pipeline
  30. [doc] Add docs for close idle readers feature. (#2208)
  31. [cdc-base] Close idle readers when snapshot finished (#2202)
  32. [mongodb] Add support for mongodb+srv connection protocol. (#2203)
  33. [oceanbase] Always cleanup resources after reading before exit (#2189)
  34. [mongodb][hotfix] Fix NullPointerException when executing ddl operation in MongoDB (#2107)
  35. [mysql-cdc] Supports MYSQL_TYPE_TYPED_ARRAY column type when parsing the table map event
  36. [mysql-cdc][hotfix] Optimize the error msg when binlog expire in source
  37. [mysql-cdc] Optimize the error msg when binlog expire or server id conflict This closes #2010.
  38. [oceanbase] add jdbc options and support oracle mode (#1854)
  39. [mysql-cdc] Add handler for catching async exceptions in snapshot reading executor This closes #2016.
  40. [hotfix][mysql-cdc] Use session timezone instead of local time zone as the default server time zone
  41. [mysql] Fix NullPointerException when database name or table name contains dot
  42. [mysql] Fix NullPointerException caused by mysql ignores the capitalization when splitting chunks
  43. [sqlserver] Fix old change data that will be captured when the latest mode starts (#2176)
  44. [core] support print configuration options for connectors (#2099)
  45. [oceanbase] support libobcdc 4.x and fix restore timestamp config (#2161)
  46. [sqlserver] Sqlserver incremental source. (#1732)
  47. [OceanBase][test] Update oceanbase and oblogproxy docker image (#2060)
  48. [oceanbase] use real charset to convert change record (#2159)
  49. [mysql] Support tables which do not contain a primary key (#2150)
  50. [common] Bump flink to 1.17.0 (#2057)
  51. [doc] add mongodb cdc chinese doc (#2042)
  52. [hotfix][test] ignore OracleConnectorITCase.testConsumingAllEvents util issue 1875 fix (#2111)
  53. [docs][db2] Correct CHAR(n) to CHARACTER(n) (#2052)
  54. [postgres] Change config option slot.name to be required (#1996)
  55. [docs] Update the outdated DingTalk QR code
  56. [mongodb][hotfix] Fix pollAwaitTimeMillis does not take effect (#1995)
  57. [tests][mysql] Skip unnecessary IT Cases which uses legacy debezium source implementation
  58. [mysql] Add note for MySQL TINYINT(1) type to Flink SQL Boolean type
  59. [github] Minor typo in issue bug-report (#2020)
  60. [mysql-cdc] Fix race condition between split and coordinator thread that async exception might not be caught
  61. [mysql/oracle] Fix the wrong comments in ChunkSplitter
  62. [hotfix][docs] Correct the flink file path in MySQL docs
  63. [hotfix][mongodb] Fix resume token not found #1879 (#1938)
  64. [mongodb][hotfix] Fix authentication failed when using a non admin auth source and duplicated usernames #1934 (#1935)
  65. [mysql] Add various test cases to cover newly added table process from read binlog blocking to read binlog un-blocking
  66. [hotfix][base] Fix typo in java docs
  67. [mysql] Optimize the newly added table process from read binlog blocking to read binlog un-blocking
  68. [minor][mysql] Rename SplitsSize to SplitsNumber
  69. [hotfix][mysql] Correct meta group id as it is 0 based
  70. [hotfix][base][tidb][mysql] Fix wrong condition for execution timeout
  71. [mongodb] Improve code structure (#1900)
  72. [build] Use issue form templates (#1830)
  73. [ci] Fix for dubious repo ownership issue (#1843)
  74. [docs] Update mysql-cdc.md to recommend latest version (#1842)
  75. [ci] Fix for dubious repo ownership issue (#1840)
  76. [docs][sqlserver] Update data type mapping (#1802)
  77. [ci][tidb] Fix dns cache works incorrect in latest jdk 8. (#1809)
  78. [postgresql] Upgrade postgresql driver from 42.2.26 to 42.2.27 to avoid CVE-2022-41946 (#1773)
  79. [minor][common] Improve the changelogMode expression (#1793)
  80. [hotfix][debezium] Forward root cause in DebeziumSourceFunction (#1791)
  81. [docs][oracle] Add new introduced config options from incremental snapshot framework (#1719)
  82. [mysql] Fix backward compatibility on deserializing binlog offset from old versions (#1758)
  83. [hotfix][docs] Correct the startup option's data type in Mysql docs and duplicate batch.size in MongoDB docs (#1751)
  84. [hotfix][docs] Correct the mysql driver version in MySQL docs
  85. [docs][site] update the supported cdc connectors picture
  86. [oracle] Add missing supported config options for oracle table source (#1727)

Contributor List (sorted alphabetically)

  1. Hang Ruan
  2. He Wang
  3. JasonLee
  4. Jiabao Sun
  5. Jingsong Lee
  6. Xiao Meng
  7. Leonard Xu
  8. Qingsheng Ren
  9. Robert Metzger
  10. Sergey Nuyanzin
  11. Simonas Gelazevicius
  12. Yaroslav Tkachenko
  13. Tigran Manasyan
  14. Tyrantlucifer
  15. ZhongLinLeo
  16. bb chen
  17. ehui
  18. emhui
  19. gintarasm
  20. gongzhongqiang
  21. liangyaohui97
  22. lin.zl
  23. molsionmo
  24. skylines
  25. tison
  26. wallkop
  27. wangxiaojing
  28. wuzhenhua
  29. xixingya
  30. zhaomin
  31. zhuyuan03
  32. 含风
Assets 2
Loading

Release-2.3.0

10 Nov 01:56
@leonardBang leonardBang
release-2.3.0
c65f9e0
Compare
Choose a tag to compare
Release-2.3.0

Release Notes

Documentation: https://ververica.github.io/flink-cdc-connectors/release-2.3

Features and Improvements

  1. [mysql] Improve error message under the case that start reading from earliest but schema change happened before (#1724)
  2. [mysql] Use event filter to drop events earlier than the specified timestamp in timestamp startup mode (#1724)
  3. [docs][oracle] Add new introduced config options from incremental snapshot framework (#1718)
  4. [docs][oceanbase][zh] Update outdated sections and translate them to Chinese
  5. [hotfix][docs] Add link for Oceanbase ZH docs
  6. [docs][oceanbase] Add Chinese documents for oceanbase-cdc connector(#1611)
  7. [db2] Fix stream api example in db2 doc. (#1725)
  8. [oracle] Add missing supported config options for oracle table source (#1720)
  9. [mysql] Fix MySqlBinlogSplitAssigner may reassign split after exception restart (#1713)
  10. [docs] Repair Connector Data Type links
  11. [docs] Update the connector download link in the document
  12. [docs] Update the flink cdc picture with supported database vendors.
  13. [tidb] Fix unstable TiDB region changed test. (#1702)
  14. [docs][mongodb] Add docs for MongoDB incremental source
  15. [oracle][mysql] Improve the Oracle all data types test and clean up debug logs
  16. [oracle] Properly support TIMESTAMP_LTZ type for oracle cdc connector
  17. [oracle]support use user given column as split column
  18. [oracle][minor] Clean up useless methods and classes
  19. [oracle] Parallel read supports startup of startupOptions
  20. [oracle] Add full type field test
  21. [oracle]OracleTableSource support splitSize, fetchSize ,connectMaxRetries, connectionPoolSize and add test
  22. [hotfix][oracle][db2] Update the license header
  23. [cdc-base][oralce] Remove useless method of JdbcSourceFetchTaskContext
  24. [cdc-base][oracle] Improve the base methods base on Oracle specific requirements
  25. [oracle] Support custom url for incremental snapshot source
  26. [oracle] Introduce SCAN_INCREMENTAL_SNAPSHOT_ENABLED option for OracleTableSourceFactory
  27. [minor][oracle] Improve tests and code format
  28. [oracle] Ignore testCheckpointAndRestore util DBZ-5245 and DBZ-4936 fix
  29. [oracle][test] Add tests for Oracle incremental snapshot source
  30. [oracle] Use ROWID as split column and replace limit syntax with ROWNUM
  31. [oracle] Let LogMinerStreamingChangeEventSource do not rely on oracle table LOG_MINING_FLUSH
  32. [oracle] Use Incremental Snapshot Framework for Oracle CDC Connector (#1079)
  33. [docs] Bump Flink version to 1.16.0
  34. [common] Bump Flink version to 1.16.0
  35. [docs][db2] Add db2 to README.md (#1699)
  36. [tidb] Checkpoint is not updated long after a task has been running (#1686)
  37. [hotfix] Add method getMaxResolvedTs back to class CDCClient. (#1695)
  38. [docs] Bump connector version to flink 1.15.2 in docs (#1684)
  39. [tidb] Fix data lost when region changed (#1632)
  40. [hotfix][docs] Correct reference link for DB2 docs (#1683)
  41. [mysql] Update docs of specifying starting offset feature of MySQL CDC source
  42. [hotfix][mysql] Remove unused constructor in MySqlTableSource
  43. [mysql] Support starting reading binlog from earliest offset / specific offset / timestamp (#1661)
  44. [mongodb] Support incremental snapshot for MongoDB CDC Connector
  45. [cdc-base] Extract a higher level IncrementalSource independent of JDBC dialects
  46. [hotfix][docs] Fix db2 doc download link (#1681)
  47. [docs][db2] Add Db2 CDC connector doc link
  48. [docs][db2] Add Db2 cdc connector docs
  49. [build][db2] Add Db2 pipeline script
  50. [e2e][db2] Add Db2 e2e test
  51. [db2] Add sql connector for Db2 CDC (#1626)
  52. [db2] Introduce Db2 cdc connector (#450)
  53. [tidb] Fix data loss in tidb-cdc when switch from snapshot read to change event read (#1207)
  54. [hotfix][cdc-base] Fix OffsetFactory is not initialized when restored from checkpoint (#1657)
  55. [common] Bump Flink version to 1.15.2 (#1504)
  56. [cdc-base] Abstract naming for variables of incremental framework (#1646)
  57. [mysql] ChunkSplitter supports to snapshot when splitting chunks for a table (#1463)
  58. [cdc-base] Fix StreamSplit's endingOffset of StreamSplitAssigner (#1659)
  59. [cdc-base] JdbcSourceConfig can be template type of DataSourceDialect (#1639)
  60. [mysql] Use the latest received meta of binlog split when request the next meta (#1479)
  61. [minor] Use Collections.singletonList instead of Arrays.asList if possible (#1569)
  62. [oceanbase] Add new deserialization schema with runtime converter (#1356)
  63. [mysql] Avoid duplicate split requests when add new table (#1156)
  64. [postgres] Upgrade org.postgresql:postgresql from 42.2.14 to 42.2.26 for CVE-2022-31197 (#1616)
  65. [test][e2e] Upgrade org.postgresql:postgresql from 42.3.1 to 42.2.26 for CVE-2022-31197(#1616)
  66. [hotfix][typo] Fix typo from schemaLess to schemaless
  67. [cdc-base] Reduce the memory usage of JM by sharing table schemas between splits
  68. [cdc-base] Release the scan fetcher thread resource after reading finished (#1619)
  69. [cdc-base] Optimize pure binlog phase check logic to improve performance (#1620)
  70. [cdc-base] Let all records of one snapshot split don't cross checkpoints (#1622)
  71. [postgres] Introduce changelog mode configuration with upsert/all modes
  72. [common] Introduce changelog mode configuration with upsert/all modes
  73. [hotfix][docs] Fix docs for scan.incremental.snapshot.chunk.size (#1617)
  74. [cdc-base] Adopt streaming merge during snapshotSplit scan to avoid OOM
  75. [docs][oceanbase][zh] Add video tutorials that synchronizes data from OceanBase to Elasticsearch (#1604)
  76. [docs][oceanbase] Add video tutorials that synchronizes data from OceanBase to Elasticsearch (#1605)
  77. [mysql] Fix parse error when ALTER SQL statement contains default string value and collate. (#1544)
  78. [docs][mysql] Add regex examples for multiple tables (#1564)
  79. [oceanbase] Shade apache commons dependency to avoid class conflict (#1532)
  80. [postgres] add geometry type support. (#1554)
  81. [build] Polish CI scripts and remove useless code (#1540)
  82. [mysql] fix parsing ddl error when use string default values for numeric types (#1522)
  83. [mysql] Enumerate captured tables lazily (#1507)
  84. [mysql] Reduce the memory usage of JM by sharing table schemas between splits (#1523)
  85. [hotfix][docs] Fix comment typo in MysqlSnapshotSplit (#1513)
  86. [mysql] Let all records of one snapshot split don't cross checkpoints (#1505)
  87. [tests] Remove usage of deprecated EnvironmentSettings.Builder#useBlinkPlanner (#1493)
  88. [cdc-base][mysql] Fix split key comparison when may contain different types (#1499)
  89. [docs][mysql] Update the description of mysql server-time-zone option (#1497)
  90. [mysql] Use local timezone as the default value of 'server-time-zone' option (#1407)
  91. [docs][postgres] Add two frequently debezium options in Postgres connector document (#1142)
  92. [mongodb] Allow mongo ARRAY to be converted to string type in Flink (#1475)
  93. [hotfix][docs] Fix the page links in MySQL Chinese document (#1466)
  94. [oceanbase][test] Use memory mode of obcdc and double startup timeout duration (#1474)
  95. [mysql][polardbx] Stop container timely in tests (#1468)
  96. [mysql][ploardbx] Support all charsets in column level (#1166)
  97. [mongodb] Expose updateLookup option for stream source. (#1461)
  98. [oceanbase] Use utc offset as 'server-time-zone' option's default value (#1443)
  99. [mysql][minor] Skip capture newly added tables according snapshotassigner's status (#1449)
  100. [polardbx][test] Bump the polardb-x image version from 2.0.1 to 2.10 (#1423)
  101. [build][test] Specify surefire provider to surefire-junit47 (#1423)
  102. [hotfix][docs] Fix typo in bug report doc (#1460)
  103. [minor][oracle] Rename JdbcConfigurationUtil to OracleJdbcUrlUtils
  104. [oracle] Support custom JDBC URL for Oracle (#1258)
  105. [docs] Correct the data type of 'server-id' option to String (#1444)
  106. [hotfix][test] Fix MySqlSourceITCase failure due to StreamExecutionEnvironment#getConfiguration returned unmodifiable object (#1446)
  107. [test] Fix tests missed 'update_before' records if the primary keys of source and sink are different (#1451)
  108. [hotfix][oracle] Fix the shell script interpreter assignment should be at first line (#1452)
  109. [hotfix][tidb-cdc] TiDB CDC dealUnsignedColumnValue throw NullPointerException. (#1448)
  110. [polardbx][test] Rename PolardbxSourceTCase to PolardbxSourceITCase (#1447)
  111. [mysql] Enable pass custom jdbc properties for debezium mysql connection (#1217)
  112. [mysql] Release the debezium reader thread resources after reading finished (#1358)
  113. [oceanbase] Introduce 'table-list' option to support capture list of tables (#1369)
  114. [cdc-base] Flink CDC base registers the identical history engine on multiple tasks (#1340)
  115. [hotfix][mysql] Fix compile error due to merge conflict
  116. [mysql] Generates multiple chunks when approximate row count is bigger than chunk size (#1193)
  117. [cdc-base] Fix NPE during snpashot scan phase (#1339)
  118. [docs] Improve MySQL ZH document and update the outdated links (#1435)
  119. [docs] Translate mysql cdc doc to Chinese (#1391)
  120. [mysql] Introduce 'scan.incremental.snapshot.chunk.key-column' option to support splitting table by custom column (#1415)
  121. [mysql] Optimize pure binlog phase check logic to improve performance (#1392)
  122. [oracle] Fix mining session stopped due to 'No more data to read from socket'
  123. [common] Ignore retriableException to make the debezium engine retry works
  124. [test][oracle] Fix unstable tests OracleE2eITCase due to reuse container
  125. [tidb] Fix NPE when the specified table does not exist (#1411)
  126. [docs] Add step up SQLServer Database cdc document in sqlserver-cdc.md.
  127. [mongodb] Expose batch.size config option for table source. (#1387)
  128. [mongodb] Example tests for MongoDBSource (#138...
Read more
Assets 2
Loading