Release Notes 20.0
1. RELEASE INFO
Product | AGILE DATA ENGINE |
Release | 20.0 (20.0.349) |
Release date | 4th of June 2020 |
Release status | RELEASED |
Release type | MAJOR.MINOR.UPDATE.PATCH |
SaaS Availability | AVAILABLE | RESTRICTED (by request) |
AWS Availability | AVAILABLE | RESTRICTED (by request) |
Azure Availability | AVAILABLE | RESTRICTED (by request) |
Patch release scope | Not applicable | Design | Runtime | SaaS |
2. CONTENT OF THE RELEASE
2.1 Highlights
Announcement of Private Preview Availability of Agile Data Engine as a Service (SaaS) deployment model.
Contact Solita Sales about Private Preview Program.
Includes
Important security fixes
OPT_SF_STORAGE_INTEGRATION changed to OPT_STORAGE_INTEGRATION (supports Azure SQL DB's and Azure SQL DW's DATA SOURCEs in addition to Snowflake STORAGE INTEGRATION)
Package and attribute listing performance improvements
Improved Grafana dashboard for Runtime
Multi-active satellite support v1 also for Google BigQuery, Amazon Redshift, Azure SQL D* with DV_RUNNING_ID named attribute
Entity import fix for entity name based on the entity type naming convention
Fixes to the SQL templates of Azure SQL DB and Azure SQL DW (and other DBMSes) with features not that widely used yet
AWS Lambda trigger fix with Python Load Steps
Plus many other fixes and improvements
2.2 Security
Includes critical security fixes
ID | Description |
|---|---|
ADE‑2671 | Upgrade Grafana to 6.7.3 version |
ADE‑2600 | Manager should not return whole main conf, only the relevant parts |
ADE‑2627 | Dh-modeler outputs debug-level logging in production |
ADE‑2684 | Limit the verbose and detail logging from Airflow |
ADE‑2755 | Improve metarepo api related audit info handling |
ADE‑2737 | Update Java dependencies |
2.3 Infrastructure
Updates
ID | Description |
|---|---|
ADE‑2815 | Jenkins update to 2.222.4 |
Fixes
ID | Description |
|---|---|
ADE‑2585 | Update airflow db creation to handle grants in every update |
ADE‑2277 | JSON logs: dh-modeler |
ADE‑2780 | AWS Errors dashboard does not show anything |
ADE‑2777 | Include the improved Runtime Dashboards |
ADE‑2808 | ERROR: azure-cli-core 2.0.69 has requirement knack~=0.6.2, but you'll have knack 0.7.1 which is incompatible |
2.4 Designer
Improvements
ID | Description |
|---|---|
ADE‑2690 | Package import functionality should be conditional in GUI |
ADE‑2663 | AttributeType is made optional in CONFIG_ENTITY_DEFAULTS |
ADE‑2682 | Change of load option OPT_SF_STORAGE_INTEGRATION to generic OPT_STORAGE_INTEGRATION |
ADE‑2658 | Multi-active satellite support to other DBMSes (Google BigQuery, Amazon Redshift, Azure SQL DB, Azure SQL DW) with DV_RUNNING_ID attribute in Satellite |
Fixes
ID | Description |
|---|---|
ADE‑2612 | Entity import fix for entity name based on the entity type naming convention |
ADE‑2677 | Imported entities and packages should have user as creator or modifier |
ADE‑2631 | When modifying a package, its created timestamp is updated also |
ADE‑2680 | Package import fails in case there is missing scheduling name in the environment |
ADE‑2732 | Red bar keeps old package name after package name is updated |
ADE‑2741 | Reference creation throws null pointer exception |
ADE‑2754 | Reference creation form persists previous data after form closure |
ADE‑2757 | Package modified info doesn't get updated if making change e.g. for entity and then click package details link |
ADE‑2689 | Problems with created/modified user when cloning package/entity |
ADE‑2660 | Implement override to make sure that anonymous case we can map preferred_username to something else than user's own email |
ADE‑2704 | Modified user/timestamp not up to date in Package Dependencies view |
ADE‑2742 | Entity modified info does not update at Summary tab when changes done in other tabs |
ADE‑2665 | Export SQL fails for VIEW when no View SQL is defined |
ADE‑2666 | <loadname> token is not resolved for TRANSFORM_PERSIST type of load in SQL Export |
ADE‑2783 | Metarepository history captures satellite attribute changes with faulty item type |
ADE‑2806 | Improve attribute listing performance |
ADE‑2768 | Performance improvement for package list to query entity counts only instead of all entity names etc. |
Usability
ID | Description |
|---|---|
ADE‑2625 | Move into another package should not have config packages in the list |
ADE‑2657 | Small improvements in Designer UI |
ADE‑2699 | Designer to ask for confirmation on entity import when entity logical name contains entity name affixes |
ADE‑2739 | Inform user when creating reference without attributes |
ADE‑2748 | Physical option STORAGE.COLUMN_STORE's description needs Valid in MS SQL and MS SQL DW. |
ADE‑2749 | Reordering of Physical Options |
ADE‑2758 | Entity import does not indicate the reason for error on line within the file |
ADE‑2769 | Red highlight line to be placed as part of textarea field |
2.5 Deployment Management
Fixes
ID | Description |
|---|---|
ADE‑2661 | When PRIMARY KEY and UNIQUE KEY(s) defined, entity re-create fails with Snowflake |
ADE‑2679 | BigQuery loads with entity mapping fails if source is a view |
ADE‑2743 | Entity re-create fails since no enclosing in PK object name with Azure SQL DB and several other incoherent SQL syntax problems |
ADE‑2804 | System job in Jenkins fails on each run |
2.6 Loads and Workflows
Fixes
ID | Description |
|---|---|
ADE‑2756 | Fill the <from_file_list> also in the single file cases, not only manifest file case |
ADE‑2785 | Investigate and implement avoiding retries if needed for Lambda calls |
ADE‑2788 | Metarepository api fails to connect to database, halts everything |
ADE‑2790 | Delta logic does not work with a column name having special characters with Dockerized Crawler |
ADE‑2814 | Manager Dag Testing fails in case DAG is not existing in Airflow |
2.7 DBMS Support
Improvements
ID | Description |
|---|---|
ADE‑2682 | Change of OPT_SF_STORAGE_INTEGRATION to generic OPT_STORAGE_INTEGRATION |
ADE‑2658 | Multi-active satellite support to other DBMSes (Google BigQuery, Amazon Redshift, Azure SQL DB, Azure SQL DW) with DV_RUNNING_ID attribute in Satellite |
Fixes
ID | Description |
|---|---|
ADE‑2700 | Other than CSV files raise exception when loading to SQL DB |
2.9 SaaS Related (in Private Preview)
ID | Description |
|---|---|
ADE‑2621 | Limit load step languages only to SQL when feature flag loadStepScriptingAllowed = False |
ADE‑2609 | SaaS root url returns 503 |
ADE‑2638 | Automate creation of CNAME records to design and runtimes |
ADE‑2670 | 3rd party software opened into new windows since Navigation iframe blocks b2clogin.com domain |
ADE‑2694 | Coding bugs related to auth token refresh |
ADE‑2697 | Add missing audit event types to stored event types |
ADE‑2640 | Security improvement |
ADE‑2688 | In Dagger file copy operation given access token expires |
ADE‑2787 | Custom parameter targetDvDbmsproduct does not accept empty value |
ADE‑2683 | Security improvement |
ADE‑2682 | Change of load option OPT_SF_STORAGE_INTEGRATION to generic OPT_STORAGE_INTEGRATION |