Breadcrumbs

Release Notes 20.1

1. RELEASE INFO



Product

AGILE DATA ENGINE

Release

20.1

Release date

24th of September 2020

Release status

RELEASED

Release type

MAJOR.MINOR.UPDATE.PATCH

SaaS Availability

AVAILABLE | RESTRICTED (by request)

AWS Availability

AVAILABLE | RESTRICTED (by request)

Azure Availability

AVAILABLE | RESTRICTED (by request)

Patch release scope

Not applicable | Design | Runtime | SaaS

2. CONTENT OF THE RELEASE

2.1 Highlights

  • Critical security issues fixed in 3rd party software

  • Config Load Schedules Improvements

    • New UI to view/create/edit/delete load schedules in preview

    • Renaming of load schedules is supported

    • Improved tracking of recent changes within Config Load Schedules package

  • New Search in Designer in preview

  • New DAG start and status APIs introduced into Notify API Service in SaaS edition (to be changed into External API Service in 21.0 release)

  • New DAG generation mode OPTIMIZED_ENTITY_ORIENTED to reduce unnecessary overhead of loads into same table within a DAG

  • Dagger component's Apache Airflow upgraded

  • Terraform and Kubernetes provider version upgrades in Private Azure edition

  • Scheduled stop and start for ECS Cluster in Private AWS edition

  • Automated installation for the whole AWS environment in SaaS edition

2.2 Security

Critical security fixes

ID

Description

ADE‑2656

Update Apache Airflow to 1.10.11

ADE‑2923

Update openjdk base image

ADE‑2924

Update Grafana base image

ADE‑2933

Update spring framework version

ADE‑2870

Update 3rd party dependencies for Designer

Security fixes

ID

Description

ADE‑2629

It is possible to change config packages' name and any packages' created/modified user names and times and statuses with package export/import

2.3 Infrastructure

Improvements

ID

Description

ADE‑2323

Update Terraform to 1.12 (in Private Azure edition)

ADE‑2416

Kubernetes resource format to be altered to support 1.37.0 azurerm

ADE‑2511

Authorized IP ranges should be defined on Kubernetes Services (for Runtime)

ADE‑2946

Azure AKS outbound ip reconfiguration

ADE‑2918

Installation scripts are emptying the whole /target/ folder

ADE‑2920

Stop and start functionality of container cluster in Private AWS edition

ADE‑2932

Scheduled stop and start of ADE environment in Private AWS edition

ADE‑2922

Enable AWS CLI 2 usage in ADE infrastructure

ADE‑2276

JSON logs: dh-logger

Fixes

ID

Description

ADE‑2931

PgBouncer prevents initial installation of azure runtime

ADE‑2938

bastion_update_users fails if some users not created by it

2.4 Designer

New Features

ID

Description

ADE‑2855

New Search in Designer in Preview

ADE‑2882

CONFIG_LOAD_SCHEDULES improvement: New UI to view/modify and renaming support of schedule names

User Experience Improvements

ID

Description

ADE‑2825

Description for Transformation Formula types

ADE‑2891

Change logical name and entity name places in Create new entity

ADE‑2892

References: Parent and Child attribute lists should be ordered by attributes' ordinal position

ADE‑2858

Link entity's red color hard to read on dark background

ADE‑2915

AGILE_DATA_ENGINE as default value and always visible for Entity CSV Export

ADE‑2928

Package import spinner comes up only after import evaluation

ADE‑2934

Entity import is case sensitive for technical_attribute_type

ADE‑2837

Entity import and Mapping import to support CSVs in UTF-8, UTF-8 with BOM

Fixes

ID

Description

ADE‑2871

Entity edit form does not allow submitting unique schema + entityname + dv source combination

ADE‑2961

Entity attribute datatype change will lead to mismatch between ADE and DBMS

2.5 Configurability

New Features

ID

Description

ADE‑2882

CONFIG_LOAD_SCHEDULES improvement: New UI to view/modify and renaming support of schedule names

2.6 Loads and Workflows

Improvements

ID

Description

ADE‑2418

New DAG orientation mode: OPTIMIZED_ENTITY_ORIENTED

Fixes

ID

Description

ADE‑2929

Process api memory consumption keeps on raising

ADE‑2949

Dagger returns authentication error (http error 500)

2.7 DBMS Support

Fixes

ID

Description

ADE‑2944

BigQuery information schema query fix

APIs

ID

Description

ADE‑2945

DAG start and status APIs into Notify API Service

Improvements

ID

Description

ADE‑2635

Automate creation of hosted zones to the ADE accounts with given DNS

ADE‑2636

Automate creation of KMS keys to accounts

ADE‑2637

Automate creation of Authenticator certs

ADE‑2910

ADE bastion updater to use bastion ssh key as parameter

ADE‑2943

Allow defining additional ssh options to bastion updater

ADE‑2922

Enable AWS CLI 2 usage in ADE infrastructure

ADE‑2956

Support bootstrap script location to make it easier to use ade-installer by saas-installer generated scripts

ADE‑2964

Cloudwatch alarms for client/server errors colleted by loadbalancer metrics

3. NOTIFICATIONS

3.1 Supported Upgrade Path

  • From 20.0.365 to 20.1

  • See earlier release notes also if upgrading from older releases.

3.2 Additional options use with ade update scripts

For speeding up the update process:

  • packages-only: This will download ADE package and sync docker images to local machine and to target repository, but not run the installation.

    • Consider loading docker images to target ADE environment by this option beforehand. This update operation will not change anything in that environment except for adding new images to repo's.

  • skip-packages: Will skip the docker image downloading and syncing to target repo. Useful if needed to iterate ADE configuration changes etc.

    • E.g: ./update_ade_design.sh the-main-conf.yaml DESIGN skip-packages

3.3 Entity delete behavior change

  • When entity is deleted in Designer, it is renamed to <entity_name><entity_uuid> in metadata. At target database deleted table is renamed to Z<entity_uuid>BACKUP<entity_version_id> during the deploy.

3.4 Terraform upgrade in Private Azure Edition

  • Terraform needs to be upgraded from 0.11.11 to 0.12.29.

  • This will need the AKS cluster to be re-created. Estimate per cluster 17-23 minutes (deleting c. 7 minutes and creating new via ADE update stacks 10-15 minutes).

3.5 Possible to shutdown and startup ECS cluster by scheduled events in Private AWS Edition

  • The ECS cluster's services can be shutdown and startup by scheduled commands added to main configuration