v3.1.2

What's Changed

• SNOW-1964010: fixing infinity support in avro messages by @sfc-gh-rzukowski in #1068
• NO-SNOW Update snowflake-jdbc to 3.23.1 by @sfc-gh-mbobowski in #1070
• Bump confluent.version from 7.8.0 to 7.9.0 by @dependabot in #1062
• NO-SNOW Update snowflake-ingest-sdk to 3.1.2 / Release 3.1.2 by @sfc-gh-mbobowski in #1067

New Contributors

• @sfc-gh-rzukowski made their first contribution in #1068

Full Changelog: v3.1.1...v3.1.2

v3.1.1

Notable changes

• Updated vulnerable dependencies (snowflake-jdbc, bpckix-fips)
• Changed the default value of max_client_lag in Snowpipe Streaming from 120s to 30s

What's Changed

• NO-SNOW: Additional logging by @sfc-gh-akowalczyk in #1046
• [CONFLUENT] Add configuration to disable reprocess files cleanup by v1Cleaner by @sfc-gh-mbobowski in #1045
• NO-SNOW Set default max_client_lag to 30s by @sfc-gh-mbobowski in #1047
• [Snyk] Security upgrade net.snowflake:snowflake-jdbc from 3.20.0 to 3.22.0 by @sfc-gh-snowflakedb-snyk-sa in #1048
• NO-SNOW Bump guava to 33.4.0-jre by @sfc-gh-mbobowski in #1052
• CASEC-9813 Bump protobuf in test-data to 3.25.5 by @sfc-gh-mbobowski in #1054
• Release 3.1.1 by @sfc-gh-mbobowski in #1053
• SNOW-1904571 Add option to configure v2 cleaner interval by @sfc-gh-mbobowski in #1055
• SNOW-1899609 Bump bpckix-fips by @sfc-gh-rsawicki in #1049
• NO-SNOW Use Iceberg String instead of Snowflake VARCHAR in schema evolution by @sfc-gh-mbobowski in #1058
• Bump org.slf4j:slf4j-api from 1.7.25 to 2.0.16 by @dependabot in #1024
• SNOW-1933073 Add retry policy for channel migration by @sfc-gh-mbobowski in #1057
• Bump org.apache.maven.plugins:maven-surefire-plugin from 2.22.0 to 3.5.2 by @dependabot in #984
• Bump org.apache.maven.plugins:maven-javadoc-plugin from 3.4.1 to 3.11.2 by @dependabot in #1025
• NO_SNOW - Fix 3.1.1 Release version by @sfc-gh-xhuang in #1060
• Upgrade ingest sdk to 3.1.1 by @sfc-gh-xhuang in #1061

Full Changelog: v3.1.0...v3.1.1

v3.1.0

Warning

If the snowflake.topic2table.map parameter is configured, Snowflake recommends using this version.
We strongly recommend upgrading the connector, if you are on earlier versions 2.x, 1.9.x, and 1.8.x.

Notable Changes

• For the Snowpipe ingestion method, when the snowflake.topic2table.map parameter is configured to map multiple topics to a single table, the connector adds the topic’s salt hashCode to the stage file prefixes to avoid file collision and load data from all specified topics.
• The connector no longer throws the IndexOutOfBoundException when offsets are not continuous during schema evolution.

What's Changed

• NO-SNOW Improve offset logging for Snowpipe by @sfc-gh-mbobowski in #1027
• NO-SNOW speed up Iceberg schema evolution test by @sfc-gh-bzabek in #1028
• SNOW-1826209 Update Confluent to 7.8.0 by @sfc-gh-mbobowski in #1029
• SNOW-1859651 Salt prefix for files mapped from different topics by @sfc-gh-dseweryn in #1035
• SNOW-1875173: Support external OAuth by @sfc-gh-akowalczyk in #1040
• [Snyk] Fix for 1 vulnerabilities by @sfc-gh-snowflakedb-snyk-sa in #1031
• SNOW-1880066: Update profile for azure and gcp by @sfc-gh-akowalczyk in #1042
• Fix IndexOutOfBoundException thrown when offsets are not continous during schema-evolution by @sfc-gh-mbobowski in #1043
• NO-SNOW: Release 3.1.0 by @sfc-gh-akowalczyk in #1044

New Contributors

• @sfc-gh-dseweryn made their first contribution in #1035

Full Changelog: v3.0.0...v3.1.0

v3.0.0

Notable Changes

• Kafka Connector can ingest data into an Iceberg table with snowflake.streaming.iceberg.enabled set to true
  • Please refer to documentation for detailed information on setup and known limitations
• Changes in default parameter values
  • snowflake.streaming.enable.single.buffer is enabled by default for Snowpipe Streaming ingestion
  • snowflake.streaming.max.client.lag default value is set to 120 seconds to match the default value of buffer.flush.time.
• Fixed dependency vulnerabilities

What's Changed

• NO-SNOW Change kafka-connect-avro-converter scope to provided by @sfc-gh-mbobowski in #985
• NO-SNOW Remove FIPS test by @sfc-gh-mbobowski in #989
• No snow use com.fasterxml json by @sfc-gh-bzabek in #994
• SNOW-1728002 get rid of reflection when enabling iceberg streaming by @sfc-gh-bzabek in #995
• SNOW-1665420 add logic to parse Iceberg schema by @sfc-gh-bzabek in #996
• NO-SNOW Enable IcebergInitServiceIT by @sfc-gh-mbobowski in #1001
• SNOW-1728000 Iceberg e2e tests by @sfc-gh-mbobowski in #1002
• SNOW-1761519 Fix empty arrays in Iceberg ingestion by @sfc-gh-mbobowski in #1003
• [Snyk] Fix for 1 vulnerabilities by @sfc-gh-snowflakedb-snyk-sa in #1004
• NO_SNOW Update Snowflake JDBC to 3.20.0 by @sfc-gh-xhuang in #1000
• SNOW-1805174 Enable single buffer by default by @sfc-gh-mbobowski in #1005
• Update ConnectorConfigDefinition.java by @sfc-gh-achyzy in #1009
• SNOW-1831140 Iceberg schema evolution e2e test setup by @sfc-gh-mbobowski in #1010
• NO-SNOW Fix missing ubuntu packages error in CI by @sfc-gh-mbobowski in #1011
• SNOW-1729292 modify iceberg tree based on record data by @sfc-gh-bzabek in #1007
• SNOW-1831140 E2e tests of Iceberg JSON ingestion by @sfc-gh-mbobowski in #1013
• SNOW-1842220 Add comments to columns created by schema evolution by @sfc-gh-bzabek in #1014
• SNOW-1831140 Complete Iceberg e2e tests by @sfc-gh-mbobowski in #1015
• NO-SNOW Update Confluent dependencies to 7.7.2 by @sfc-gh-mbobowski in #1019
• SNOW-1820302 Integrate with ingest sdk 3.0.1 by @sfc-gh-mbobowski in #1016
• NO-SNOW Update jackson and commons-compress by @sfc-gh-mbobowski in #1021
• SNOW-1731255: Put null or empty json nodes to DLQ for schema evolution by @sfc-gh-wtrefon in #1017
• NO-SNOW Release 3.0.0 by @sfc-gh-mbobowski in #1022

Full Changelog: v2.5.0...asdasd

v2.5.0

Notable Changes

• Updated Streaming Ingest SDK to the 2.3.0 version
• Closing channels in parallel is now enabled by default. It reduces the time needed for the connector restart.

What's Changed

• SNOW-1658905 Limit SnowflakeSinkConnectorConfig responsibilities(#925)
• NO-SNOW Enable closing channels in parallel by default(#928)
• SNOW-1680410 adjust validations and create metadata column if not exists(#946)
• SNOW-1278872 Added jdk version and vendor to metrics(#972)
• Fix logging for closing partition channels - partition and topic misplaced(#977)

v2.4.1

Notable Changes

• Updated Streaming Ingest SDK to the 2.2.2 version

What Changed

SNOW-1649161 Fix NaN value handling in schematization (#920)
SNOW-1623269 Fail sink task on authorization exception from Snowflake (#916)
SNOW-1514185 Assign new channel when no offset is present in Snowflake (#913)
SNOW-1571459 parse jbdc properties (#909)

v2.4.0

Notable changes

• Upgrade of Snowflake Ingest SDK to the latest 2.2.0 version - containing a critical fix for potential corruption when change_tracking is enabled. - see https://github.com/snowflakedb/snowflake-ingest-java/releases/tag/v2.2.0 for more details
We strongly recommend upgrading Kafka connector to this 2.4.0+ version if you are using Snowpipe Streaming functionality
• Upgrade of Snowflake JDBC driver from version 3.14.5 to version 3.18.0 - see https://docs.snowflake.com/en/release-notes/clients-drivers/jdbc-2024 for more details,
• Fixed issue with connector loosing data when reopening a channel fails before fully initializing channel: Fix to SNOW-1514185: Do assign the reopened channel unless Kafka offsets are fully reset (#875),
• Improved logging experience in various components for improved troubleshooting experience,
• Updated dependencies with known vulnerabilities.

What's Changed

• SNOW-1618257 - upgrade to 2.2.0 ingest-sdk by @sfc-gh-xhuang in (#910)
• NO_SNOW Update dependencies by @sfc-gh-xhuang in (#898)
• SNOW-1229108 Add warning about buffer config when single buffer enabled by @sfc-gh-mbobowski in (#896)
• NO-SNOW Ignore OAuth test by @sfc-gh-mbobowski in (#906)
• Bump org.sonatype.plugins:nexus-staging-maven-plugin from 1.6.7 to 1.7.0 in (#857)
• Bump io.dropwizard.metrics:metrics-core from 4.2.3 to 4.2.26 in (#863)
• Bump org.assertj:assertj-core from 3.25.3 to 3.26.3 (#881)
• Bump dev.failsafe:failsafe from 3.2.1 to 3.3.2 (#859)
• SNOW-947731 Remove deprecated avro-python3 package by @sfc-gh-mbobowski in (#889) (
• SNOW-1541942 Extend Snowpipe initialization logging at INFO level by @sfc-gh-mbobowski in (#885)
• SNOW-1229110 Add single buffer usage data to telemetry by @sfc-gh-mbobowski in (#893)
• Upgrade com.google.protobuf to 3.24.4 by @sfc-gh-wtrefon in (#892)
• NO-SNOW Upgrade nexus-staging-maven-plugin version by @sfc-gh-xhuang in (#887)
• Remove insertRows DEBUG log (called once per row) by @sfc-gh-mbobowski in (#886)
• Adding logs to track schematization by @sfc-gh-mbobowski in (#884)
• NOSNOW: improve troubleshooting experience for scenario with skipped offsets by @sfc-gh-gjachimko in (#883)
• SNOW-1061855 Fix the E2E test for SchemaEvolutionDropTable with a single buffer by @sfc-gh-gjachimko in (#882)
• SNOW-1061851: Run SnowflakeSinkServiceV2IT for a single buffer by @sfc-gh-akowalczyk in (#876)
• SNOW-1514185: Do assign the reopened channel unless kafka offsets are fully reset by @sfc-gh-akowalczyk in (#875)
• NOSNOW: added utility to upload connector to our nexus for simplified k8s deployment by @sfc-gh-gjachimko in (#874)

v2.3.0

Notable changes

• (Snowpipe Streaming) Timestamp of pushing data into Snowflake is added to record_metadata. Note according to the configured kafka buffers, the timestamp value may be have a slight delay between when it is pushed to the Snowpipe Streaming channel buffer and when it is flushed and available on the table.
  • SNOW-1056407 Add SnowflakeConnectorPushTime to record_metadata by @sfc-gh-lshcharbaty in #833
• (Snowpipe Streaming) Optional parameter, snowflake.streaming.closeChannelsInParallel.enabled, for Kafka connector to close channels in parallel which significantly speeds up rebalance time
  • SNOW-1369280 Close channels asynchronously by @sfc-gh-lshcharbaty in #841
• Updated dependencies with known vulnerabilities
  • SGRC-4405 Update Kafka to 3.7.0 and Confluent to 7.6.0 by @sfc-gh-mbobowski in #854

What's Changed

• SNOW-1269024 Installing multiple Java versions for test purposes by @sfc-gh-krosinski in #824
• SNOW-1269025 Testing Confluent kafka connector using multiple JRE versions by @sfc-gh-krosinski in #826
• SNOW-1269038 Stress testing Confluent kafka connector using multiple JRE versions by @sfc-gh-krosinski in #827
• SNOW-1348656 Migrate RecordContentTest to JUnit5 by @sfc-gh-mbobowski in #828
• SNOW-1056407 Cleanup metadata properties class. Migrate records tests to junit5 by @sfc-gh-lshcharbaty in #831
• NO-SNOW Fix build warnings by declaring maven-plugin versions by @sfc-gh-xhuang in #837
• NO-SNOW Fix idempotent multiple column schema evolution by @sfc-gh-xhuang with contribution from @cmonty-paypal in #836
• SNOW-1348656 Send records that could not be parsed to DLQ by @sfc-gh-mbobowski in #830
• SNOW-1056407 Add tests MetaColumnTest for snowpipe streaming by @sfc-gh-lshcharbaty in #832
• SNOW-1369271 Ignore SFException when closing a channel by @sfc-gh-lshcharbaty in #839
• NO-SNOW Remove unused code path from RecordService by @sfc-gh-mbobowski in #838
• NO-SNOW Ugrade to ingest-sdk 2.1.1 by @sfc-gh-xhuang in #843
• SNOW-1167007: Migrate schema when receiving null values for not null cols by @sfc-gh-wtrefon in #849
• SNOW-1061848: Introduce TopicPartitionChannel interface by @sfc-gh-akowalczyk in #851
• SNOW-1061822: Add new properties for streaming client by @sfc-gh-wtrefon in #822
• SNOW-1061848: DirectTopicPartitionChannel - copy of the old implementation with removed buffer by @sfc-gh-akowalczyk in #852
• SNOW-1061848: Fix unit tests by @sfc-gh-akowalczyk in #855
• NO-SNOW Add dependabot by @sfc-gh-mbobowski in #856
• SNOW-1061848: Run TopicPartitionChannelIT against a non-buffered path by @sfc-gh-akowalczyk in #862
• SNOW-1475664: Move additional error handling from BufferedTopicPartitionChannel to fix E2E test by @sfc-gh-akowalczyk in #865
• NO_SNOW Add column comments from schemaregistry by @sfc-gh-akowalczyk with contribution from @wornjs in #870
• SNOW-1061854 Run e2e tests with single buffer by @sfc-gh-lshcharbaty in #867
• SNOW-1019628 Enable V2 file cleaner for Snowpipe ingestion by default by @sfc-gh-gjachimko in #868
• Release v2.3.0 by @sfc-gh-mbobowski in #873

New Contributors

• @sfc-gh-krosinski made their first contribution in #824
• @sfc-gh-nsienkiewicz-1 made their first contribution in #842

Full Changelog: v2.2.2...v.2.3.0

v2.2.2

Notable changes

• Fixed bug that caused stage files being not deleted in Snowpipe ingestion
  • SNOW-1019628 update file cleaner for snowpipe ingest by @sfc-gh-gjachimko in #807

What's Changed

• SNOW-1055524 E2E test with nullable SMT by @sfc-gh-lshcharbaty in #808
• SNOW-1057165 - Test-doubles for snowflake-ingest-java by @sfc-gh-achyzy in #809
• SNOW-1055524 E2E SMT without schema evolution by @sfc-gh-lshcharbaty in #811
• SNOW-1055524 Check table schema in e2e smt by @sfc-gh-lshcharbaty in #813
• SNOW-1055524 E2E SMT for a Snowpipe based connector by @sfc-gh-lshcharbaty in #812
• NO-SNOW Remove performance tests by @sfc-gh-mbobowski in #817
• SNOW-1055561: Check whether SMT returning null values no longer stops a data ingestion. by @sfc-gh-akowalczyk in #816
• Update to release 2.2.2 by @sfc-gh-rcheng in #819

New Contributors

• @sfc-gh-lshcharbaty made their first contribution in #808
• @sfc-gh-achyzy made their first contribution in #809
• @sfc-gh-akowalczyk made their first contribution in #816
• @sfc-gh-krosinski made their first contribution in #824

Full Changelog: v2.2.1...v2.2.2

v2.2.1

Notable Changes

• SNOW-979849: Add offset verification logic by @sfc-gh-tzhang in #795
• SNOW-1117446 Snowpipe Streaming client provider map by @sfc-gh-japatel in #794
• SNOW-1049869 Cleanup streaming ingest threads when SinkTask stop() is called by @sfc-gh-mbobowski in #800

What's Changed

• SNOW-922795 Added third-party-licenses to .zip distribution by @sfc-gh-mbobowski in #784
• PROD-39429 Add PR template by @sfc-gh-japatel in #786
• Run subset of e2e tests by @sfc-gh-mbobowski in #788
• NO-SNOW: upgrade JDBC to 3.14.5 by @sfc-gh-tzhang in #799
• [external contributor] Upgrade dependency versions by @bansallovish merged by @sfc-gh-xhuang in #802
• Upgrade connect-api to 3.7.0 by @sfc-gh-mbobowski in #803
• SNOW-1057151: Add poc test setup for EmbeddedConnectCluster by @sfc-gh-wtrefon in #801
• SNOW-1049869 Rewrite client optimization unit tests by @sfc-gh-mbobowski in #804
• Release v2.2.1 by @sfc-gh-rcheng in #805

New Contributors

• @sfc-gh-mbobowski made their first contribution in #784
• @sfc-gh-wtrefon made their first contribution in #801

Full Changelog: v2.2.0...v2.2.1