OPT_SKIP_DISTINCT_ONLY

OPT_SKIP_DISTINCT_ONLY is a load option that if set to true will skip SELECT DISTINCT in automatically generated loads.

See also:

• Load options

• OPT_SKIP_DUPLICATE_PREVENTION

Usage

OPT_SKIP_DISTINCT_ONLY: Boolean

Default: false

Notes

OPT_SKIP_DISTINCT_ONLY can be used in loads that have a single entity mapping (i.e. one source entity) and no OVERRIDE load steps. OPT_SKIP_DISTINCT_ONLY will not have an effect if the load is overridden with an OVERRIDE load step.

Examples

Hub load

OPT_SKIP_DISTINCT_ONLY: false

In the default situation OPT_SKIP_DISTINCT_ONLY is set to false (or not set at all) a hub load will insert DISTINCT values into the hub.

DML generated by Agile Data Engine:

INSERT INTO
  dv.H_TRIP (
    ...
  )
SELECT
  DISTINCT ...
FROM
  (
    SELECT
      ...
    FROM
      stage.STG_TRIPDATA_YELLOW src_entity
  ) src
WHERE
  NOT EXISTS (
    SELECT
      1
    FROM
      dv.H_TRIP trg
    WHERE
      trg.dv_id = src.dv_id
  );

OPT_SKIP_DISTINCT_ONLY: true

With OPT_SKIP_DISTINCT_ONLY set to true the DISTINCT is dropped from the generated DML:

INSERT INTO
  dv.H_TRIP (
    ...
  )
SELECT
  ...
FROM
  (
    SELECT
      ...
    FROM
      stage.STG_TRIPDATA_YELLOW src_entity
  ) src
WHERE
  NOT EXISTS (
    SELECT
      1
    FROM
      dv.H_TRIP trg
    WHERE
      trg.dv_id = src.dv_id
  );