Skip to main content
Skip table of contents

Release Notes 20.2

1. RELEASE INFO

Product

AGILE DATA ENGINE

Release

20.2 (20.2.394)

Release date

16th of December 2020

Release status

RELEASED

Release type

MAJOR.MINOR.UPDATE.PATCH

SaaS Availability

AVAILABLE | RESTRICTED (by request)

AWS Availability

AVAILABLE | RESTRICTED (by request)

Azure Availability

AVAILABLE | RESTRICTED (by request)

Patch release scope

Not Applicable | Design | Runtime | SaaS

2. CONTENT OF THE RELEASE

2.1 Highlights

  • Metadata Repository DB's PostgreSQL upgrade to version 11

  • In Private AWS and SaaS Editions as part of ADE upgrade.

  • In Private Azure manually after ADE upgrade.

  • Tooling and manual procedure provided for Private Azure due Azure limitations. In Azure the current 9.5 will deprecate in Feb 2021.

  • Read the Notifications and the Manual upgrade instructions in these release notes.

  • New Deployment API v1 replaces Builder API and Version Control API.

    • In the next release Deployment API v2 will replace lots of logic implemented currently into Jenkins.

  • Improvements to Private Azure Infrastructure

    • PgBouncer as its own versioned component

    • Tag support

    • AKS version update

    • Etc.

  • Lots of improvements in Designer

    • E.g. new commit message format validation framework with Config Validations.

  • SaaS related

    • New SaaS Designer Only Option

    • Automatic SaaS Installer v1

    • Cloud foundation improved

2.2 Security

Security fixes

ID

Description

ADE‑3068

Upgrade Spring Boot and cleanup dependency handling of Process API

ADE‑3069

Upgrade Spring Boot and cleanup dependency handling of Notify API

2.3 Infrastructure

Improvements

ID

Description

ADE‑2168

PostgreSQL db version upgrades in SaaS, AWS and Azure editions

ADE‑3035

New dh-azure-pgbouncer component to ensure tested version in a release

ADE‑3009

Replace builder-api and versioncontrol-api with new deployment-api

ADE‑3065

Update AKS version in Azure edition

ADE‑3076

Tag support in Terraform resources in Azure edition

ADE‑3094

Jenkins viewable build amount in stage view configurable

ADE‑3083

Possible to skip TF apply in ade_update script run

ADE‑3108

Azure add Microsoft.Storage private endpoint

ADE‑3123

AWS create manual snapshot of RDS PostgreSQL in ADE update

ADE‑2969

Idea of instructing tee command usage during updates

Fixes

ID

Description

ADE‑3075

Remove obsolete NatGw from AWS network

2.4 Designer

New Features

ID

Description

ADE‑3024

New Config Package: CONFIG_VALIDATIONS and first implementation of commit message validation.

ADE‑2914

Config package dependency list in Designer

ADE‑2989

Add section of Triggering Schedules (non-editable) to Schedule details view

ADE‑3060

Template support for default attribute's name

ADE‑2960

Support searching schedules with app search

User Experience Improvements

ID

Description

ADE‑2939

Creating a view with GENERIC entity type overwrites the schema when name is changed

ADE‑2950

Unnecessary scroll bars in load step code view

ADE‑3049

Reference create/edit: minor issue when switching parent entities

ADE‑3059

User defined schema is overwritten with default, when entity name is touched

ADE‑3098

Using copy by ref should be available for hub dv_id references in links also

ADE‑2893

References: Schema aware list of parent entities

ADE‑2991

When deleting schedule, there should be validation of existing loads referring to the schedule

ADE‑3084

When new entity is created in Entity listing, the view should change to Entity summary

ADE‑3145

Longer Description field for Load Transformations

Fixes

ID

Description

ADE‑3056

Order problem in SQL Export for MS SQL SQL in Designer

ADE‑3140

Change CREATE PROCEDURE into CREATE OR ALTER PROCEDURE in SQL Export for MS SQL

ADE‑3080

CREATE PROCEDURE is generated for each load step alone and even with the same name in SQL Export e.g. in case of Status Satellites

ADE‑3034

Rename 'Schedule loads' to 'Scheduled loads'

ADE‑3079

Entity export in CSV does not include attribute_is_soft_deleted field

ADE‑3141

Commit history is not returned for deleted package

ADE‑3161

Entity deep clone sets cloned loads as disabled

ADE‑3171

Entity/Satellite deletion hangs and won't go through with couple of hundred attributes

2.5 Configurability

New Features

ID

Description

ADE‑3024

New Config Package: CONFIG_VALIDATIONS and first implementation of commit message validation

2.6 Deployment Management

New Features

ID

Description

ADE‑3009

Replace builder-api and versioncontrol-api components with new deployment-api

ADE‑3094

Jenkins viewable build amount in stage view configurable

Fixes

ID

Description

ADE‑3077

Cleared masking policy or collation will fail in entity recreation of the entity

ADE‑3078

Entity re-create loses table definitions e.g. columnstore definition

ADE‑3170

Rollback problem with Snowflake in entity change in certain case

2.7 Loads and Workflows

Improvements

ID

Description

ADE‑2942

Add Batch ID and executed SQL query to failed smoke test result messages

ADE‑3073

Alias name for DV loads source entities

ADE‑2992

Prevention of SELECTs from load steps to ensure customer data won't be transferred to ADE runtime in SaaS editions

2.8 DBMS Support

Fixes

ID

Description

ADE‑3077

Cleared masking policy or collation will fail in entity recreation of the entity

ADE‑3078

Entity re-create loses table definitions e.g. columnstore definition

ADE‑3170

Rollback problem with Snowflake in entity change in certain case

2.9 SaaS Editions

APIs

  • Development of new External API implementation has progressed and limited preview status is expected in the 21.0 release for Enterprise Edition.

Improvements

  • Lots of improvements in SaaS foundation

  • Lots of development done for upcoming External API

ID

Description

ADE‑2992

Prevention of SELECTs from load steps to ensure customer data won't be transferred to ADE runtime

ADE‑3081

Designer Only SaaS Option ie. Possiblility to disable Deployment Management from AWS installation

3. NOTIFICATIONS

3.1 Supported Upgrade Path

  • From 20.0.365 to 20.1

  • From 20.1 to 20.2.

  • Upgrading from earlier releases

    • First to 20.1.380 and then directly to 20.2.

    • E.g. from 19.3.334.patch4 to 20.0.354.patch6 and then directly 20.1.380 (Or any latest of 20.1) and then to 20.2

  • See earlier release notes also if upgrading from older releases.

3.2 Automatic Metadata Repository Database snapshots in Private AWS edition

  • When ADE update detects release change or PostgreSQL version change in main configuration, database snapshots will be created from all ADE RDS PostgreSQL instances of that environment.

  • This happens by default as a post step in update and the retention time of 62 days is applied for these snapshots. This feature can be configured in main configuration if needed.

3.3 Metadata Repository Database's PostgreSQL version upgrade

  • In AWS edition the PostgreSQL version will be upgraded from current 9.6 to 11. This will take about 25 minutes per environment.

  • In Azure edition, the upgrade is not done by default. It must be done by you manually after release upgrade at some point before the next release update.

3.4 Obsolete NAT gateway removed if exists in Private AWS edition

  • In ADE created network having multiple public subnets, there are also multiple Nat GW's. Those are reduced to one.

  • This also results reducing multiple outbound public IP's to one. So would be good to remove excess public IP's from any whitelistings.

  • Outbound public IP's will be also logged out in runtime ade update post step

    • e.g: "infrarun.infrarun.rel.post_steps - INFO - Outbound public IPs: 52.30.120.126"

3.5 Additional options to use with update scripts

For speeding up the update process:

  • packages-only: This will download ADE package and sync docker images to local machine and to target repository, but not run the installation.

    • Consider loading docker images to target ADE environment by this option beforehand. This update operation will not change anything in that environment except for adding new images to repo's.

  • skip-packages: Will skip the docker image downloading and syncing to target repo. Useful if needed to iterate ADE configuration changes etc.

    • E.g: ./update_ade_design.sh the-main-conf.yaml DESIGN skip-packages

JavaScript errors detected

Please note, these errors can depend on your browser setup.

If this problem persists, please contact our support.