Loads

Loads are defined into entities to:

• Load data from source files into a target table (LOAD_FILE)

• Load data from source entities into a target table (TRANSFORM_PERSIST)

• Define a view or a materialized view (TRANSFORM_SQL_VIEW)

See also:

• Entities

• Entity mapping

• Load options

Tutorials

See the below videos for quick tutorials on how to create an incoming load and use a load template.

Creating loads

Loads are created in the Loads tab in the entity view with Create incoming load or Create outgoing load.

• Create incoming load creates a load where the target entity is the entity where the load was created in. Once the load is created, source entities can be mapped in the load view with Create entity mapping.

• Create outgoing load creates a load where the target entity is chosen when the load is created and the wizard navigates directly into a new entity mapping in the target entity.

Should I create an incoming or an outgoing load?
The end result is the same regardless of whether a load is created as an outgoing load from a source entity or as an incoming load into a target entity.

Note that the automatically generated load name will be more descriptive when creating an outgoing load because the source entity and the target entity are known.

Load properties:

Custom schedules

Custom schedules are not recommended because they cannot be edited in bulk. If a custom schedule defined for multiple loads has to be changed, the changes have to be done separately for all loads. Use CONFIG_LOAD_SCHEDULES instead.

Managing loads

Loads can be edited or deleted in the Loads tab in the entity view. Only incoming loads can be edited or deleted as they are a part of the entity that is being viewed. Outgoing loads can be edited or deleted in the entity view of the target entity.

Load view:

• A load can be disabled by unchecking the ENABLED box in the Edit incoming load view.

• Entity mappings can be added with Create entity mapping.

• Entity load logic can be saved as template with Save as a template. This saves the load steps and load options.

• Entity load logic can be added from a template with Add from template.

• In views and materialized views view logic can be added with Add view SQL.

• Load steps can be added with Add load step.

• Load options can be added with Add load option.

DBMS specific loads and load step

Agile Data Engine load handling has been extended with a support for specifying DBMS product specific loads or load steps

If DBMS product has been set on Load level:

• Behavior

  • Setting DBMS Product on load step -level will be disabled

  • Entire load will be provided only when the target DBMS is the one specified for the load

If DBMS product has been set on Load Step level:

• Behavior

  • Setting DBMS Product on load-level will be disabled

  • Load will be generated with all GENERIC (no DBMS Product defined) load steps and target DBMS specific load steps

  • For TRANSFORM_SQL_VIEW OVERRIDE steps the DBMS specific OVERRIDE will be chosen before the generic OVERRIDE

Notes

Views and materialized views can also have GATEKEEPER, POST, SMOKE_GREY and SMOKE_BLACK load steps. The selected load schedule will determine when these steps are executed in Runtime environments.

Load templates can be saved in the load view and edited or deleted with CONFIG_LOAD_TEMPLATES. Load templates can also be set as default for defined entity types.

Use of CUSTOM schedules is not recommended. Use CONFIG_LOAD_SCHEDULES instead.

OVERRIDE and OVERRIDE_FILE_LOAD can be used to override an automatically generated load logic. Some changes can also be done with load options without needing to customize the entire load.

Use Export SQL in the entity Summary tab to check what kind of Entity Loads SQL would be generated based on the defined loads.