Release Notes 25.0

1. RELEASE INFO

2. CONTENT OF THE RELEASE

2.1 Highlights

• New modernized UI and unified styles together with UX improvements

• AI based hash key suggestion in Public Preview

• SQL support additions in Databricks SQL, Snowflake, Microsoft Fabric, Azure SQL Database etc.

• Template Framework improvements

• Insights (multi-tenant add-on application) in Public Preview, see Insights

• User Management v2 (multi-tenant user management) in Public Preview

• New End-User Admin UI (multi-tenant application) in Private Preview

• New Ask ADE (multi-tenant add-on application) in Private Preview

• Many platform performance, robustness and security improvements

2.2 User Experience Improvements

• New UI styles to improve user experience.

• Unified styles between different applications.

• New side menu bar with items to Insights, External API documentation and product documentation etc.

• Designer: New style and position (topmost) of the maintenance window banner

• Load name / Entity name / Package name is added to the beginning of a page header depending on the object level where user is. This is to help navigation between multiple of browser tabs.

2.3 Designer

• Infer Entity Metadata from a source file (.csv, .csv.gz, .json, .json.gz, .parquet) in Public Preview.

  • Data won’t be processed at ADE SaaS instances but at the user’s browser memory. Only the inferred entity metadata is imported to ADE metadata repository.

  • Fixed: Decimal scale disappears if zero when editing attribute

• AI based hash key suggestion in Public Preview

• Template Framework Improvements

  • Less maintenance overhead of the load logic templates is achieved with the new setting to choose that whether the Load Logic Template is generated on deployment / code preview time or when a new load is created (as before). The setting name is generatedOnDeployand currently it is to-be-set/changed in Config Load Templates package. The default value is false. The UI addition in Load Logic section when saving the template will be in forthcoming update releases. See Load logic template generation support on deployment time

  • When the new setting generateOnDeploy = true the load steps are shown in Designer UI as non-editable

  • New dynamic variables

    • source_entity_attribute_by_{attribute_type}

    • source_entity_name_{N}

    • source_entity_schema_{N}

    • target_entity_attribute_by_{attribute_type}

    • target_entity_transformation_by_{attribute_type}

• Fixed: Cannot edit DEFAULT ATTRIBUTE & TRANSFORMATION TYPE in CONFIG_ENTITY_DEFAULTS user interface

• Fixed: Prevent tab chars in entity names

• Fixed: Delete package does not do anything in certain cases

• Fixed: Saving an attribute (in a long list) doesn't retain the scrollbar position

2.4 Deployment Management

• New role-based authorization support for Deployment actions with roles promoter, deployer, admin in SaaS editions. Requires migration to the user management v2 as a prerequisite. See in detail: User Management - Authorization (RBAC) and Deployment Management - Access and Roles

• Robustness fixes of some corner cases in the UI.

• Support to handle multiple warehouse and workspace names for one environment added.

2.5 Automatic schema management

• Cluster by syntax support with SORT_KEY in Databricks SQL added. See SORT_KEY

• Support for Row Level Security in Microsoft Fabric added

• Support for HYBRID TABLE in Snowflake added. See Hybrid table support in Snowflake

• Support for Covering Index ie. INCLUDE syntax of SECONDARY_INDEX in Azure SQL Database added. See SECONDARY_INDEX

• SECONDARY_INDEX support in PostgreSQL added. See SECONDARY_INDEX

• Fixed: Altering entity name and adding description in same commit fails in Azure SQL Database

2.6 Loads and Workflows

• Airflow version updated.

• <target_entity_attribute_list*> variables for Snowflake are improved and couple of bugs are fixed

• New feature/behavior to start so-called Run ID logic sequence in middle of a Workflow when defining OPT_NUM_RUN_IDS_PER_TRANSACTION = -1 for a load. This is to optimize the downstream loads.

• Support for TRUNCATE TABLE added in Microsoft Fabric.

• Add Azure SQL Database Transaction ID to the Workflow Orchestration logs.

• Fixed: New Run ID feature with num transactions = -1 does not resolve to correct transformation formula when <target_entity_attribute_list_with_transform_cast_and_positions> used

• Fixed: Load/DAG is not showing up in Airflow UI as failed, but the error can be seen in Runtime Insights and by the Notification Events

• Fixed: PostgreSQL not generating MERGE clause, instead uses UPDATE&INSERT in transaction

2.7 APIs

• Retry mechanism added to Notify API to improve robustness in certain cases

2.8 Insights

• Add-on application with DevOps KPIs (DORA adapted to DataOps) and other dashboards bringing visibility to the overall solution with tag and user based groups and Data Product Catalog with Lineage etc. for different stakeholders.

• Requires migration to the User Management v2.

• Contact Agile Data Engine sales for Public Preview.

2.9 End-User Admin UI

• New user and user roles management.

• No downtime changes to certain configurations.

• Private Preview program is full. Wait for announcements.

2.10 Ask ADE

• Chat with ADE documentation.

• Private Preview program is full. Wait for announcements.

2.11 Private Editions

• AKS updated to version 1.29 in Private Azure Edition.

• Separate manual steps required in certain cases when upgrading to 25.0 release in Private AWS Edition.

3. NOTIFICATIONS

3.1 Cancelled features

3.1.1 SCD2 load option in Private Preview cancelled

Private Preview program with couple of cases showed that instead of trying to generalize the SCD2, we will be opening the template framework so, that users can implement their own SQL generation templates for their SCD2 use cases plus other use cases like SCD6 or customized 3NF / Data Vault templates, and have them shareable as Config Package exports in ADE Community or between projects. This feature is under development and included in the 25.0 product program.

3.2 Private AWS Edition Upgrade

Contact ADE support for details.