Pools
Workflow Orchestration runs tasks in pools. Pool size sets the degree of parallelism i.e. the maximum number of concurrent tasks that can be run at the same time.
Pool settings can be accessed in the Workflow Orchestration of each Runtime environment. Select Pools from the Admin menu.
Note that the overall degree of parallelism is limited by:
Target database scaling
Agile Data Engine Runtime size (see table below)
See also:
Default pools
Workflow orchestration has two pools by default:
Pool | Default size | Description |
|---|---|---|
default_pool | 128 | Tasks that are disabled or have no loads are run in this pool, e.g. entity tasks, start tasks and end tasks. |
loading_default_pool | 2 | Load tasks that have actual operations and that are not disabled are run in this pool. The pool size limits concurrency against the target database. |
Increase the loading_default_pool size to allow more concurrency if the target database can handle it.
User defined pools
Users can define new pools and control in which pool loads or workflows are run to isolate tasks into separate pools. Operations running in separate pools do not affect each other from the Workflow Orchestration point of view. Obviously, the operations could still be running on the same resources within the target database.
Define a new pool from the Admin - Pools view in Workflow Orchestration by clicking Create. Define a pool name, pool size (number of slots) and a description, press Save.
Place schedules (workflows) into user defined pools with CONFIG_LOAD_SCHEDULES.
Place loads into user defined pools with OPT_LOAD_POOL. This overrides any schedule level pool definition.
Runtime size
Runtime size limits the overall degree of parallelism for active tasks per Agile Data Engine environment.
Runtime size | Maximum degree of parallelism |
|---|---|
Small | 2 |
Medium (default) | 5 |
Large | 10 |
X-Large | 20 |
XX-Large | 40 |
The sum of pool sizes (excluding default_pool) should not exceed the maximum degree of parallelism of the Runtime environment. Also ensure that there is sufficient available capacity in the target database accordingly.
Notes
If custom pools are defined for schedules or loads, those pools also have to be created in Workflow Orchestration in different Runtime environments and pool sizes have to be configured. Otherwise the schedules and loads will not be run at all.
You can check which pool a task is assigned to opening the task in Workflow Orchestration and viewing the Task Instance Details.