What is Agile Data Engine
DataOps platform for cloud data warehouses
Agile Data Engine (ADE) is an integrated DataOps development and operations environment for designing, deploying and running cloud-based data warehouse solutions. Agile Data Engine is a combination of interconnected cloud native software modules that are used together with public cloud IaaS & PaaS services (AWS, Azure, GCP) to create customer specific data warehousing and platform solutions.
Agile Data Engine has capabilities for data modeling, ELT (transformations, loads) and load orchestration combined in the same software stack and exposes them to data professionals in the same development UI.
Agile Data Engine supports multiple of DBMS services:
Agile Data Engine solves certain major challenges in a modern data platform development
1. Creating cloud based data capability quickly and in a managed way
You should not be forced to compromise whether you do things fast or in a sustainable way. We enable for example fast data solution prototyping without quick and dirty approaches.
2. Enabling true agility in data platform development
Agile project management methods and mindset are not enough. Tools need to support and guide data professionals to work in a certain way, to be able to create business value continuously. As we know, business requirements and source systems are changing all the time, and it should be possible to do changes within hours and days, not weeks and months.
3. Addressing the requirements of operational analytics
Quite often traditional data warehouses do not support well enough operational needs of the business, if data is not refreshed often enough. We wanted to ensure that the data warehouse also supports these operational, business real time, requirements.
4. Unlocking person, technology and ecosystem lock-ins
Traditional approaches typically create a heavy dependency on individual people, on certain skilled data engineers for example. We call this person lock and it is one of the biggest pitfalls with data platforms - maintenance of the solution quickly becomes a nightmare. We are tackling this problem by standardizing the work process and deliverables. We also want to mitigate the risk for organisations to lock-in to only one cloud vendors and databases.
See introduction video about Agile Data Engine with demo:
Agile Data Engine introduction video
What is DataOps
DataOps is an important concept and mindset when talking about developing and operating data solutions in cloud. In the quickly changing business environment, and with large and complex data environments, it is very difficult to succeed without a new kind of mindset and methods.
DataOps is a similar concept to DevOps. Like DevOps, it standardizes and automates the process for developing, publishing and maintaining software products or systems. DataOps makes the same in context of analytical and data intensive solutions. With DataOps, you can significantly reduce the workload for recurring tasks such as testing and deployment, reducing costs and delays due to human errors. DataOps, along with cloud services, can make the development of new services and businesses smoother and remove bottlenecks and inefficiencies.
Agile Data Engine helps you to take DataOps practices into use for data warehousing and accelerates the business value created by your data team by improving the quality of solutions.