Skip to main content
Skip table of contents

DAG generation mode

DAG generation mode controls how workflows are generated. It can be set on Runtime environment level with CONFIG_ENVIRONMENT_VARIABLES and on schedule level with CONFIG_LOAD_SCHEDULES.


See also:


List of DAG generation modes

Mode

Description

ENTITY_ORIENTED

Generated DAGs show entity operators making them easier to read. This used to be the default mode. However, it is not recommended due to the overhead from unnecessary operators.

LOAD_ORIENTED

Generated DAGs contain only load tasks reducing the unnecessary scheduling/running overhead related to entity operators.

OPTIMIZED_ENTITY_ORIENTED

Loads are executed in entity operators in a sequential manner. I.e. the DAGs contain entity operators and loads into those entities are executed within the entity operators. Only entities that have incoming loads are included.

This mode prevents loads from blocking each other as multiple loads into the same target cannot be run parallel. This optimizes execution wait times caused by entity locks which consume pool slots when loads are queuing without actually doing anything.

As a possible downside, loads are not visually separate in the DAGs. When opening a failed entity node you need to search for the actual failure in the log in case there are multiple loads into that entity.

OPTIMIZED_LOAD_ORIENTED

Similar to LOAD_ORIENTED, but further excludes unnecessary tasks, e.g. disabled loads or loads that do not have any commands.

SEQUENTIAL_TASK_ORIENTED

A special mode for getting rid of overhead in simple DAGs meant to run in a fast cycle (near realtime). Generated DAGs only contain one task which runs all loads in sequence. The running order of loads which are not dependent on each other is random but the original order of dependent loads is followed.

This mode disables all parallelism inside the DAG, therefore it is advised to carefully test it case by case before enabling it. Do not set the mode with an environment variable, instead set it on schedule level with CONFIG_LOAD_SCHEDULES.


Notes

OPTIMIZED_ENTITY_ORIENTED or OPTIMIZED_LOAD_ORIENTED is recommended as the default setting.

Schedule level DAG generation mode settings override environment level settings.

JavaScript errors detected

Please note, these errors can depend on your browser setup.

If this problem persists, please contact our support.