Skip to main content
Skip table of contents

FTL Template Reference

Load steps support customizable FTL (FreeMarker Template Language) templates through the following approaches:

  • Product-defined transformations – Built-in logic provided by Agile Data Engine to handle common transformation patterns.

  • Product-defined templates – Preconfigured FTL templates included with the product for reuse across typical use cases.

  • User-defined templates – Custom templates created by users to support platform-specific logic and dynamic SQL generation.

Product-defined transformations

Product-defined transformations can be invoked using the standard <variable> syntax.

Product-defined FTL templates

Predefined templates can be directly used without modification, see Product-defined FTL templates for a list of available templates and instructions.

User-defined FTL templates

Any FTL template defined in the CONFIG_LOAD_TEMPLATES package can be called as a user-defined FTL template in an entity load step using the following format:

CODE
<ftltemplate:MY_TEMPLATE:param1=value:param2=value>

Escape control characters with a backslash (\) when needed.

Multi-line parameter values can be given with the triple-quote (""") syntax:

CODE
<ftltemplate:MY_TEMPLATE:param1=value:param2=
"""
SELECT
  *
FROM
  \<target_schema\>.\<target_entity_name\>
"""
>

The transformation engine uses the Freemarker Template Engine. The version currently in use is FreeMarker 2.3.33.

Helper functions and macros

There are multiple built-in functions and macros that can be used directly within FTL templates. See Supported FTL functions and macros.

Template model

The template model includes various predefined objects that can be referenced within FTL templates. Refer to the diagram below for an overview of the available objects and their structure.

ftl_model.png

Below are examples of how to reference commonly used objects within the FTL template model:

  • Target database name: model.targetDbName

  • Target load name: model.load.name

  • Target entity: model.load.target

  • Target entity attributes: model.load.target.attributes

  • All source entities: model.load.sources

These objects can be used within FTL templates to dynamically generate logic based on the load context.

Examples

Syntax examples

Basic usage

Refer to the above diagram for details about the template model structure. For a complete list of available predefined functions and macros, see Supported FTL functions and macros.

Note that model, function and macro references can be directly used in FTL templates. This example covers some basic use cases and syntax:

CODE
<#-- Assigning a variable -->
<#assign targetEntity = model.load.target>

<#-- Calling a function -->
<#assign targetRunId = loads.getAttributeForType(model.load.target.attributes, "RUN_ID")>

<#-- Printing a variable value -->
${targetEntity.name}

<#-- Calling a macro -->
<@sql.generateEntityAttributeList
    attributes=model.load.target.attributes
    prefix="src"
    indentation=2
/>

<#-- Conditional logic -->
<#if model.load.target.type == "HUB">
    -- Do something with HUB
<#elseif model.load.target.type == "SAT">
    -- Do something with SAT
<#else>
    -- Do something with ${model.load.target.type}
</#if>

<#-- Iterating over a list -->
<#list attributesFiltered as attr>
    - ${attr.name}
</#list>

<#-- Defining a reusable sequence -->
<#assign excludeTypes = ["DV_HASHKEY", "DV_LOAD_TIME", "RUN_ID"]>

<#-- List filtering with the defined sequence -->
<#assign attributesFiltered = model.load.target.attributes?filter(
    attr -> !(excludeTypes?seq_contains(attr.attributeType))
)>

Testing variable output

The ${var} syntax is useful for printing variable values during development or debugging.

Defining and calling a function

This example defines a FreeMarker function to return a fully qualified entity name. This function builds the name in the format [database.]schema.table, depending on whether a database name is provided.

Function definition:

CODE
<#-- 
  Function: entityFullyQualifiedName
  Description: Constructs a fully qualified entity name in the format [database.]schema.table.
  Parameters:
    - entityObject (REQUIRED): an object with `schemaName` and `name` fields.
    - databaseName (OPTIONAL): a string representing the database name. If provided, it will be prepended followed by a dot.
  Returns:
    - A string like "database.schema.table" or "schema.table" if no database name is given.
-->
<#function entityFullyQualifiedName entityObject databaseName="">
    <#-- Construct the database prefix if a database name is provided -->
    <#local targetDbUsed>
        <#compress>
            <#if databaseName?has_content>
                ${databaseName}.
            </#if>
        </#compress>
    </#local>

    <#-- Return the fully qualified name -->
    <#return "${targetDbUsed}${entityObject.schemaName}.${entityObject.name}">
</#function>

Function call:

CODE
<#-- Context: Target entity is staging.STG_CUSTOMER -->
<#assign targetEntity = model.load.target>

<#assign fullNameWithDb = entityFullyQualifiedName(sampleEntity, "dw_dev")>
<#assign fullNameWithoutDb = entityFullyQualifiedName(sampleEntity)>

${fullNameWithDb}
${fullNameWithoutDb}

Output:

CODE
dw_dev.staging.STG_CUSTOMER
staging.STG_CUSTOMER

Importing from another FTL template

You can import and use content from other FTL templates to promote code reuse and maintainability.

In this example:

  • The function from the previous example is saved into a separate FTL template called generic_functions.

  • It is then imported into another FTL template, and the function is called from there.

CODE
<#import "generic_functions" as utils>

<#assign targetEntity = model.load.target>
<#assign fullNameWithDb = utils.entityFullyQualifiedName(targetEntity, "dw_dev")>

Generate SELECT statement for mapped source entities

This FTL template can be used to accelerate e.g. publish fact or dimension load logic development, where multiple entities from the data warehouse need to be joined together. The goal is not to produce complete load logic, but rather to provide a helpful starting point for writing custom, manually-maintained logic tailored to the requirements.

FTL template definition:

CODE
<#-- 
  This template dynamically generates a SQL SELECT statement that:
  
  - Iterates over all source entities defined in the load model.
  - Assigns simple aliases (t1, t2, ...) to each source entity for use in the SQL.
  - Outputs a flat SELECT clause listing all attributes from all sources, 
    with optional inline comments for attribute descriptions.
  - Starts the FROM clause using the first source entity.
  - Appends JOIN clauses for the remaining source entities with placeholder ON conditions.
  - Includes inline comments listing available reference-based join suggestions
    for each JOINed entity (based on entity-level references).
-->

<#-- Step 1: Assign the list of source entities from the load -->
<#assign sources = model.load.sources>

<#-- Step 2: Generate aliases (t1, t2, ...) for each source entity -->
<#assign aliases = []>
<#list sources as source>
    <#assign alias = "t${source_index + 1}">
    <#assign aliases += [alias]>
</#list>

-- SELECT clause with entity-qualified attributes and inline descriptions
SELECT
<#list sources as source>
    <#assign alias = aliases[source_index]>
    -- Attributes from entity: ${source.schemaName}.${source.name}
    <#list source.attributes as attr>
    <#assign isLast = source?is_last && attr?is_last>
    ${alias}.${attr.name}<#if !isLast>,</#if><#if attr.description??> -- ${attr.description}</#if>
    </#list>

</#list>
-- FROM clause with JOINs based on mapped entities and references
FROM ${sources[0].schemaName}.${sources[0].name} ${aliases[0]}

<#-- Step 3: Loop over all sources (except the first) to emit JOINs -->
<#list sources[1..] as source>
    <#assign alias = aliases[source_index + 1]>
    JOIN ${source.schemaName}.${source.name} ${alias}
    ON ${alias}.<join_column> = ${aliases[0]}.<join_column>
    <#-- Step 4: Print reference-based join suggestions as comments -->
    <#list source.references?default([]) as ref>
        <#assign parent = ref.parentEntity>
        <#list ref.referenceAttributes?sort_by("position") as refAttr>
            <@utils.indent
                level=1
                content="-- Reference: ${alias}.${refAttr.attribute.name} = ${parent.name}.${refAttr.parentAttribute.name}"
            />
        </#list>
    </#list>

</#list>


Template used on an example load which has multiple mapped source entities:

Generated SQL
SQL
-- SELECT clause with entity-qualified attributes and inline descriptions
SELECT
    -- Attributes from entity: rdv.S_CUSTOMER_C
    t1.dv_id, -- Data Vault Referencing Hash Key
    t1.dv_load_time, -- Data Vault Load Time
    t1.dv_run_id, -- Data Vault Run Identifier
    t1.dv_source_system, -- Data Vault Source System
    t1.dv_source_entity, -- Data Vault Source Entity
    t1.dv_package_version, -- Data Vault Package Version
    t1.dv_load_name, -- Data Vault Load Name
    t1.dv_datahash, -- Data Vault Comparison Hash
    t1.customerid,
    t1.personid,
    t1.storeid,
    t1.territoryid,
    t1.accountnumber,
    t1.rowguid,
    t1.modifieddate,

    -- Attributes from entity: rdv.L_SALES_ORDER_HEADER_CUSTOMER
    t2.dv_id, -- Data Vault Hash Key
    t2.dv_load_time, -- Data Vault Load Time
    t2.dv_run_id, -- Data Vault Run Identifier
    t2.dv_source_system, -- Data Vault Source System
    t2.dv_source_entity, -- Data Vault Source Entity
    t2.dv_package_version, -- Data Vault Package Version
    t2.dv_load_name, -- Data Vault Load Name
    t2.customer_dv_id,
    t2.sales_order_header_dv_id,

    -- Attributes from entity: rdv.S_SALES_ORDER_HEADER_C
    t3.dv_id, -- Data Vault Referencing Hash Key
    t3.dv_load_time, -- Data Vault Load Time
    t3.dv_run_id, -- Data Vault Run Identifier
    t3.dv_source_system, -- Data Vault Source System
    t3.dv_source_entity, -- Data Vault Source Entity
    t3.dv_package_version, -- Data Vault Package Version
    t3.dv_load_name, -- Data Vault Load Name
    t3.dv_datahash, -- Data Vault Comparison Hash
    t3.salesorderid,
    t3.revisionnumber,
    t3.orderdate,
    t3.duedate,
    t3.shipdate,
    t3.status,
    t3.onlineorderflag,
    t3.salesordernumber,
    t3.purchaseordernumber,
    t3.accountnumber,
    t3.customerid,
    t3.salespersonid,
    t3.territoryid,
    t3.billtoaddressid,
    t3.shiptoaddressid,
    t3.shipmethodid,
    t3.creditcardid,
    t3.creditcardapprovalcode,
    t3.currencyrateid,
    t3.subtotal,
    t3.taxamt,
    t3.freight,
    t3.totaldue,
    t3.comment,
    t3.rowguid,
    t3.modifieddate,

    -- Attributes from entity: rdv.H_CUSTOMER
    t4.dv_id, -- Data Vault Hash Key
    t4.dv_load_time, -- Data Vault Load Time
    t4.dv_run_id, -- Data Vault Run Identifier
    t4.dv_source_system, -- Data Vault Source System
    t4.dv_source_entity, -- Data Vault Source Entity
    t4.dv_package_version, -- Data Vault Package Version
    t4.dv_load_name, -- Data Vault Load Name
    t4.business_key, -- Business key

    -- Attributes from entity: rdv.H_SALES_ORDER_HEADER
    t5.dv_id, -- Data Vault Hash Key
    t5.dv_load_time, -- Data Vault Load Time
    t5.dv_run_id, -- Data Vault Run Identifier
    t5.dv_source_system, -- Data Vault Source System
    t5.dv_source_entity, -- Data Vault Source Entity
    t5.dv_package_version, -- Data Vault Package Version
    t5.dv_load_name, -- Data Vault Load Name
    t5.business_key, -- Business key

    -- Attributes from entity: rdv.H_ADDRESS
    t6.dv_id, -- Data Vault Hash Key
    t6.dv_load_time, -- Data Vault Load Time
    t6.dv_run_id, -- Data Vault Run Identifier
    t6.dv_source_system, -- Data Vault Source System
    t6.dv_source_entity, -- Data Vault Source Entity
    t6.dv_package_version, -- Data Vault Package Version
    t6.dv_load_name, -- Data Vault Load Name
    t6.business_key, -- Business key

    -- Attributes from entity: rdv.L_SALES_ORDER_HEADER_ADDRESS
    t7.dv_id, -- Data Vault Hash Key
    t7.dv_load_time, -- Data Vault Load Time
    t7.dv_run_id, -- Data Vault Run Identifier
    t7.dv_source_system, -- Data Vault Source System
    t7.dv_source_entity, -- Data Vault Source Entity
    t7.dv_package_version, -- Data Vault Package Version
    t7.dv_load_name, -- Data Vault Load Name
    t7.sales_order_header_dv_id,
    t7.bill_to_address_dv_id,
    t7.ship_to_address_dv_id

-- FROM clause with JOINs based on mapped entities and references
FROM rdv.S_CUSTOMER_C t1

    JOIN rdv.L_SALES_ORDER_HEADER_CUSTOMER t2
    ON t2.<join_column> = t1.<join_column>
    -- Reference: t2.customer_dv_id = H_CUSTOMER.dv_id
    -- Reference: t2.sales_order_header_dv_id = H_SALES_ORDER_HEADER.dv_id

    JOIN rdv.S_SALES_ORDER_HEADER_C t3
    ON t3.<join_column> = t1.<join_column>
    -- Reference: t3.dv_id = H_SALES_ORDER_HEADER.dv_id

    JOIN rdv.H_CUSTOMER t4
    ON t4.<join_column> = t1.<join_column>

    JOIN rdv.H_SALES_ORDER_HEADER t5
    ON t5.<join_column> = t1.<join_column>

    JOIN rdv.H_ADDRESS t6
    ON t6.<join_column> = t1.<join_column>

    JOIN rdv.L_SALES_ORDER_HEADER_ADDRESS t7
    ON t7.<join_column> = t1.<join_column>
    -- Reference: t7.sales_order_header_dv_id = H_SALES_ORDER_HEADER.dv_id
    -- Reference: t7.bill_to_address_dv_id = H_ADDRESS.dv_id
    -- Reference: t7.ship_to_address_dv_id = H_ADDRESS.dv_id

Other examples

JavaScript errors detected

Please note, these errors can depend on your browser setup.

If this problem persists, please contact our support.