Release Notes 23.0
1. RELEASE INFO
Product | AGILE DATA ENGINE |
Release | 23.0 |
Release date | 15th of February 2023 |
Release status | RELEASED |
Release type | MAJOR.MINOR.UPDATE.PATCH |
SaaS Availability | AVAILABLE | RESTRICTED (by request) |
AWS Availability | AVAILABLE | RESTRICTED (by request) |
Azure Availability | AVAILABLE | RESTRICTED (by request) |
Patch release scope | Not applicable | Design | Runtime | SaaS |
2. CONTENT OF THE RELEASE
2.1 Highlights
New Runtime Sizing (Medium default, Large, X-Large, XX-Large) in preview in SaaS Editions, ask for details!
VPN Connectivity in preview in SaaS Editions
New Deployment Management User Experience! (Deployments UI replaces the Jenkins UI)
2.2 User Experience Improvements
New Deployments UI with many improvements
Including: Compare between promoted and already installed version of the package
Select all / Clear in package listing in Designer
SQL Load Steps can be compared line-by-line in YAML mode between Package Commits
Load step code editor resize to full screen
New Entities only mode in Data Lineage
And many more...
2.3. Loads
Two new load options to automate scanning the latest source data by key
New typing of source entity mapping: Source, Source (for Run ID), Dependency
Default value changed to absolute for dag_weighting_method in Workflow Orchestration
2.4 Automatic schema evolution
Shorter downtime of dependency VIEWs, when underlying schema changes
Fixes in dependency VIEWs’ management in general
Physical options to enable partitioning of tables in Google BigQuery, Azure SQL DB and Synapse
Improvements in Multi-DBMS support (Google BigQuery, Azure Synapse SQL, Amazon Redshift Serverless)
2.5 Other
New channel of AWS SNS in Notification Events in limited preview (SaaS Enterprise Edition)
Security updates and improvements
Apache Airflow version update
3. NOTIFICATIONS
3.1 New Deployments UI replaces Jenkins UI
User experience will improve. There are few simple videos to look through:
3.2 Possible breaking change in Entity Import in Designer
Entity import will no longer support whitespaces in the attribute names.
Importing entity attributes, load steps or load mappings will now have strict ordinal position check during import. Previously any position has been accepted and that has lead to inconsistent end result.
3.3 Decreased downtime of dependency VIEWs
With Snowflake and Google BigQuery the dependency views will be dropped only for the necessary time when entity re-create happens and for Azure SQL Database and Azure Synapse SQL the dependency VIEWs will be re-created only after the entity has been changed. These changes will decrease the downtime of the dependency VIEWs. No changes with Amazon Redshift.
3.4 Fixes and improvements to the dependency VIEWs' management
The dependency VIEWs are managed differently in three different cases:
a) Dependency VIEW is within the deployed package without changes
b) Dependency VIEW is within the deployed package with changes
c) Dependency VIEW is outside the deployed package
Case a): Failures in these dependency VIEWs will result package deployment as FAILED. Deployment UI will list the dependency VIEWs which failed to be created back.
Case b): Failures in these dependency VIEWs will be skipped for the VIEWs, which are to-be-changed successfully as part of the package deployment nevertheless.
Case c): Failures in these dependency VIEWs will result package deployment as SUCCESS_WITH_WARNINGS. This is because breaking change in the underlying entity can be fixed only by fixing the dependency VIEW and deploying the package of it also. Deployments UI will list the dependency VIEWs, which failed to be created back.
3.5 Layout version v1 is not supported anymore in Code preview / SQL export
Only the layout version v2 will be supported from here onwards. Might affect some Designer Only use cases and possible hybrid architecture customers.
3.6 Upgrade downtime will be longer
This is major release and lots of things are upgraded. Expect at least double downtime compared to the regular update releases.
3.7 Known Issues
3.7.1 Deployments UI's date filtering does not work correctly in a certain case
When clearing the date filter, there is a known minor issue that the search does not work correctly when moving off from the Deployments UI and coming back. For the time being it is advised not to remove the date filter in the new Deployments UI.
3.7.2 After upgrade there is one-time error in SQL Export
After upgrade there will come one-time error related to key already exists when using SQL Export first time. From there on it should work just ok.