Release Notes 20.1
1. RELEASE INFO
Product | AGILE DATA ENGINE |
Release | 20.1 |
Release date | 24th of September 2020 |
Release status | RELEASED |
Release type | MAJOR.MINOR.UPDATE.PATCH |
SaaS Availability | AVAILABLE | RESTRICTED (by request) |
AWS Availability | AVAILABLE | RESTRICTED (by request) |
Azure Availability | AVAILABLE | RESTRICTED (by request) |
Patch release scope | Not applicable | Design | Runtime | SaaS |
2. CONTENT OF THE RELEASE
2.1 Highlights
Critical security issues fixed in 3rd party software
Config Load Schedules Improvements
New UI to view/create/edit/delete load schedules in preview
Renaming of load schedules is supported
Improved tracking of recent changes within Config Load Schedules package
New Search in Designer in preview
New DAG start and status APIs introduced into Notify API Service in SaaS edition (to be changed into External API Service in 21.0 release)
New DAG generation mode OPTIMIZED_ENTITY_ORIENTED to reduce unnecessary overhead of loads into same table within a DAG
Dagger component's Apache Airflow upgraded
Terraform and Kubernetes provider version upgrades in Private Azure edition
Scheduled stop and start for ECS Cluster in Private AWS edition
Automated installation for the whole AWS environment in SaaS edition
2.2 Security
Critical security fixes
ID | Description |
|---|---|
ADE‑2656 | Update Apache Airflow to 1.10.11 |
ADE‑2923 | Update openjdk base image |
ADE‑2924 | Update Grafana base image |
ADE‑2933 | Update spring framework version |
ADE‑2870 | Update 3rd party dependencies for Designer |
Security fixes
ID | Description |
|---|---|
ADE‑2629 | It is possible to change config packages' name and any packages' created/modified user names and times and statuses with package export/import |
2.3 Infrastructure
Improvements
ID | Description |
|---|---|
ADE‑2323 | Update Terraform to 1.12 (in Private Azure edition) |
ADE‑2416 | Kubernetes resource format to be altered to support 1.37.0 azurerm |
ADE‑2511 | Authorized IP ranges should be defined on Kubernetes Services (for Runtime) |
ADE‑2946 | Azure AKS outbound ip reconfiguration |
ADE‑2918 | Installation scripts are emptying the whole /target/ folder |
ADE‑2920 | Stop and start functionality of container cluster in Private AWS edition |
ADE‑2932 | Scheduled stop and start of ADE environment in Private AWS edition |
ADE‑2922 | Enable AWS CLI 2 usage in ADE infrastructure |
ADE‑2276 | JSON logs: dh-logger |
Fixes
ID | Description |
|---|---|
ADE‑2931 | PgBouncer prevents initial installation of azure runtime |
ADE‑2938 | bastion_update_users fails if some users not created by it |
2.4 Designer
New Features
ID | Description |
|---|---|
ADE‑2855 | New Search in Designer in Preview |
ADE‑2882 | CONFIG_LOAD_SCHEDULES improvement: New UI to view/modify and renaming support of schedule names |
User Experience Improvements
ID | Description |
|---|---|
ADE‑2825 | Description for Transformation Formula types |
ADE‑2891 | Change logical name and entity name places in Create new entity |
ADE‑2892 | References: Parent and Child attribute lists should be ordered by attributes' ordinal position |
ADE‑2858 | Link entity's red color hard to read on dark background |
ADE‑2915 | AGILE_DATA_ENGINE as default value and always visible for Entity CSV Export |
ADE‑2928 | Package import spinner comes up only after import evaluation |
ADE‑2934 | Entity import is case sensitive for technical_attribute_type |
ADE‑2837 | Entity import and Mapping import to support CSVs in UTF-8, UTF-8 with BOM |
Fixes
ID | Description |
|---|---|
ADE‑2871 | Entity edit form does not allow submitting unique schema + entityname + dv source combination |
ADE‑2961 | Entity attribute datatype change will lead to mismatch between ADE and DBMS |
2.5 Configurability
New Features
ID | Description |
|---|---|
ADE‑2882 | CONFIG_LOAD_SCHEDULES improvement: New UI to view/modify and renaming support of schedule names |
2.6 Loads and Workflows
Improvements
ID | Description |
|---|---|
ADE‑2418 | New DAG orientation mode: OPTIMIZED_ENTITY_ORIENTED |
Fixes
ID | Description |
|---|---|
ADE‑2929 | Process api memory consumption keeps on raising |
ADE‑2949 | Dagger returns authentication error (http error 500) |
2.7 DBMS Support
Fixes
ID | Description |
|---|---|
ADE‑2944 | BigQuery information schema query fix |
2.8 SaaS Related
APIs
ID | Description |
|---|---|
ADE‑2945 | DAG start and status APIs into Notify API Service |
Improvements
ID | Description |
|---|---|
ADE‑2635 | Automate creation of hosted zones to the ADE accounts with given DNS |
ADE‑2636 | Automate creation of KMS keys to accounts |
ADE‑2637 | Automate creation of Authenticator certs |
ADE‑2910 | ADE bastion updater to use bastion ssh key as parameter |
ADE‑2943 | Allow defining additional ssh options to bastion updater |
ADE‑2922 | Enable AWS CLI 2 usage in ADE infrastructure |
ADE‑2956 | Support bootstrap script location to make it easier to use ade-installer by saas-installer generated scripts |
ADE‑2964 | Cloudwatch alarms for client/server errors colleted by loadbalancer metrics |
3. NOTIFICATIONS
3.1 Supported Upgrade Path
From 20.0.365 to 20.1
See earlier release notes also if upgrading from older releases.
3.2 Additional options use with ade update scripts
For speeding up the update process:
packages-only: This will download ADE package and sync docker images to local machine and to target repository, but not run the installation.
Consider loading docker images to target ADE environment by this option beforehand. This update operation will not change anything in that environment except for adding new images to repo's.
skip-packages: Will skip the docker image downloading and syncing to target repo. Useful if needed to iterate ADE configuration changes etc.
E.g: ./update_ade_design.sh the-main-conf.yaml DESIGN skip-packages
3.3 Entity delete behavior change
When entity is deleted in Designer, it is renamed to <entity_name><entity_uuid> in metadata. At target database deleted table is renamed to Z<entity_uuid>BACKUP<entity_version_id> during the deploy.
3.4 Terraform upgrade in Private Azure Edition
Terraform needs to be upgraded from 0.11.11 to 0.12.29.
This will need the AKS cluster to be re-created. Estimate per cluster 17-23 minutes (deleting c. 7 minutes and creating new via ADE update stacks 10-15 minutes).
3.5 Possible to shutdown and startup ECS cluster by scheduled events in Private AWS Edition
The ECS cluster's services can be shutdown and startup by scheduled commands added to main configuration