Release Notes 23.0

1. RELEASE INFO

2. CONTENT OF THE RELEASE

2.1 Highlights

• New Runtime Sizing (Medium default, Large, X-Large, XX-Large) in preview in SaaS Editions, ask for details!

• VPN Connectivity in preview in SaaS Editions

• New Deployment Management User Experience! (Deployments UI replaces the Jenkins UI)

2.2 User Experience Improvements

• New Deployments UI with many improvements

• Including: Compare between promoted and already installed version of the package

• Select all / Clear in package listing in Designer

• SQL Load Steps can be compared line-by-line in YAML mode between Package Commits

• Load step code editor resize to full screen

• New Entities only mode in Data Lineage

• And many more...

2.3. Loads

• Two new load options to automate scanning the latest source data by key

• New typing of source entity mapping: Source, Source (for Run ID), Dependency

• Default value changed to absolute for dag_weighting_method in Workflow Orchestration

2.4 Automatic schema evolution

• Shorter downtime of dependency VIEWs, when underlying schema changes

• Fixes in dependency VIEWs’ management in general

• Physical options to enable partitioning of tables in Google BigQuery, Azure SQL DB and Synapse

• Improvements in Multi-DBMS support (Google BigQuery, Azure Synapse SQL, Amazon Redshift Serverless)

2.5 Other

• New channel of AWS SNS in Notification Events in limited preview (SaaS Enterprise Edition)

• Security updates and improvements

• Apache Airflow version update

3. NOTIFICATIONS

3.1 New Deployments UI replaces Jenkins UI

User experience will improve. There are few simple videos to look through:

• Navigate to Deployments UI

• Promote and Demote a package

• Deploy a package

• Deployment logs

3.2 Possible breaking change in Entity Import in Designer

• Entity import will no longer support whitespaces in the attribute names.

• Importing entity attributes, load steps or load mappings will now have strict ordinal position check during import. Previously any position has been accepted and that has lead to inconsistent end result.

3.3 Decreased downtime of dependency VIEWs

With Snowflake and Google BigQuery the dependency views will be dropped only for the necessary time when entity re-create happens and for Azure SQL Database and Azure Synapse SQL the dependency VIEWs will be re-created only after the entity has been changed. These changes will decrease the downtime of the dependency VIEWs. No changes with Amazon Redshift.

3.4 Fixes and improvements to the dependency VIEWs' management

The dependency VIEWs are managed differently in three different cases:

a) Dependency VIEW is within the deployed package without changes

b) Dependency VIEW is within the deployed package with changes

c) Dependency VIEW is outside the deployed package

Case a): Failures in these dependency VIEWs will result package deployment as FAILED. Deployment UI will list the dependency VIEWs which failed to be created back.

Case b): Failures in these dependency VIEWs will be skipped for the VIEWs, which are to-be-changed successfully as part of the package deployment nevertheless.

Case c): Failures in these dependency VIEWs will result package deployment as SUCCESS_WITH_WARNINGS. This is because breaking change in the underlying entity can be fixed only by fixing the dependency VIEW and deploying the package of it also. Deployments UI will list the dependency VIEWs, which failed to be created back.

3.5 Layout version v1 is not supported anymore in Code preview / SQL export

Only the layout version v2 will be supported from here onwards. Might affect some Designer Only use cases and possible hybrid architecture customers.

3.6 Upgrade downtime will be longer

This is major release and lots of things are upgraded. Expect at least double downtime compared to the regular update releases.

3.7 Known Issues

3.7.1 Deployments UI's date filtering does not work correctly in a certain case

When clearing the date filter, there is a known minor issue that the search does not work correctly when moving off from the Deployments UI and coming back. For the time being it is advised not to remove the date filter in the new Deployments UI.

3.7.2 After upgrade there is one-time error in SQL Export

After upgrade there will come one-time error related to key already exists when using SQL Export first time. From there on it should work just ok.