Skip to main content
Skip table of contents

Configuration of Database Tags

Database Tags in Snowflake

Precondition on Snowflake

Tags are required to be defined on target database. Defining their allowed values are optional.

Do notice that tags are schema specific.

Create TAG | Snowflake Documentation

Database tag configuration:

Entity level TAG

In example we are creating a tag configuration Cost Center with values marketing and development.

CODE 
-- Define TAG in Snowflake (loose definition):

create tag tags.cost_center;

-- or more restrictively:
create tag tags.cost_center allowed_values 'marketing','development';

Grant APPLY TAG permission for the row Agile Data Engine is using

  1. Open CONFIG_ENTITY_DEFAULTS - package in Agile Data Engine

  2. Open Configuration

  3. Choose Database Tags - tab

  4. Click Create Database Tag

  5. Create Database Tag with following information

    1. Label - Label for the Tag shown in Agile Data Engine, for example “Cost Center”

    2. Technical Name - Schema specific TAG name in Snowflake “tags.cost_center”

    3. Description - Short description for which purposes this tag is to be used for

    4. Select options:

      1. Allowed on entity level

      2. has values

    5. Click “Create”

  6. Click the name of freshly created Cost Center from the Database tag list

    1. Click Create Database Tag Value

    2. Type in the wanted Tag value information (marketing)

      1. Value - Tag value in Snowflake

      2. Description - Short description for the tag value

    3. Click “Create”

    4. Click Create Database Tag Value

    5. Type in the wanted Tag value information (development)

      1. Value - Tag value in Snowflake

      2. Description - Short description for the tag value

    6. Click “Create”

  7. Commit the CONFIG_ENTITY_DEFAULT package

Attribute level TAG

CODE 
-- Define TAG in Snowflake (loose definition):

create tag tags.information_type;

-- or more restrictively:
create tag tags.information_type allowed_values 'pii','nationality';

Grant APPLY TAG permission for the row Agile Data Engine is using

  1. Open CONFIG_ENTITY_DEFAULTS - package in Agile Data Engine

  2. Open Configuration

  3. Choose Database Tags - tab

  4. Click Create Database Tag

  5. Create Database Tag with following information

    1. Label - Label for the Tag shown in Agile Data Engine, for example “Information type”

    2. Technical Name - Schema specific TAG name in Snowflake “tags.information_type”

    3. Description - Short description for which purposes this tag is to be used for

    4. Select options:

      1. Allowed on attribute level

      2. has values

    5. Click “Create”

  6. Click the name of freshly created Information type from the Database tag list

    1. Click Create Database Tag Value

    2. Type in the wanted Tag value information (pii)

      1. Value - Tag value in Snowflake

      2. Description - Short description for the tag value

    3. Click “Create”

    4. Click Create Database Tag Value

    5. Type in the wanted Tag value information (nationality)

      1. Value - Tag value in Snowflake

      2. Description - Short description for the tag value

    6. Click “Create”

  7. Commit the CONFIG_ENTITY_DEFAULT package

Generated DDL

DDL generated by Agile Data Engine on Entity level:

CODE 
ALTER TABLE data SET TAG cost_center = 'marketing';

DDL generated by Agile Data Engine on Attribute level:

CODE 
ALTER TABLE data ALTER COLUMN phone_number SET TAG information_type = 'pii';

Database Tags in Databricks SQL

Database tag configuration:

Entity level TAG

In example we are creating a tag configuration Cost Center with values marketing and development.

  1. Open CONFIG_ENTITY_DEFAULTS - package in Agile Data Engine

  2. Open Configuration

  3. Choose Database Tags - tab

  4. Click Create Database Tag

  5. Create Database Tag with following information

    1. Label - Label for the Tag shown in Agile Data Engine, for example “Cost Center”

    2. Technical Name - TAG name in Databricks “cost_center”

    3. Description - Short description for which purposes this tag is to be used for

    4. Select options:

      1. Allowed on entity level

      2. has values

    5. Click “Create”

  6. Click the name of freshly created Cost Center from the Database tag list

    1. Click Create Database Tag Value

    2. Type in the wanted Tag value information (marketing)

      1. Value - Tag value in Databricks

      2. Description - Short description for the tag value

    3. Click “Create”

    4. Click Create Database Tag Value

    5. Type in the wanted Tag value information (development)

      1. Value - Tag value in Databricks

      2. Description - Short description for the tag value

    6. Click “Create”

  7. Commit the CONFIG_ENTITY_DEFAULT package

Attribute level TAG

  1. Open CONFIG_ENTITY_DEFAULTS - package in Agile Data Engine

  2. Open Configuration

  3. Choose Database Tags - tab

  4. Click Create Database Tag

  5. Create Database Tag with following information

    1. Label - Label for the Tag shown in Agile Data Engine, for example “Information type”

    2. Technical Name - TAG name in Databricks “information_type”

    3. Description - Short description for which purposes this tag is to be used for

    4. Select options:

      1. Allowed on attribute level

      2. has values

    5. Click “Create”

  6. Click the name of freshly created Information type from the Database tag list

    1. Click Create Database Tag Value

    2. Type in the wanted Tag value information (pii)

      1. Value - Tag value in Databricks

      2. Description - Short description for the tag value

    3. Click “Create”

    4. Click Create Database Tag Value

    5. Type in the wanted Tag value information (nationality)

      1. Value - Tag value in Databricks

      2. Description - Short description for the tag value

    6. Click “Create”

  7. Commit the CONFIG_ENTITY_DEFAULT package

Generated DDL

DDL generated by Agile Data Engine on Entity level:

CODE 
ALTER TABLE data SET TAGS ('cost_center' = 'marketing');

DDL generated by Agile Data Engine on Attribute level:

CODE 
ALTER TABLE data ALTER COLUMN phone_number SET TAGS ('information_type' = 'pii');

Database Tags in Google BigQuery

Precondition on BigQuery

Resource Tags are required to be defined on target project (https://console.cloud.google.com/iam-admin/tags). These affect on the entity level and only to TABLEs.
https://cloud.google.com/bigquery/docs/tags

Policy Tags are required to be defined on target project https://console.cloud.google.com/bigquery/policy-tags. These affect on the attribute level. No notice that only one Policy Tag is allowed per attribute.

Create Taxonomies and Policy Tags | BigQuery Documentation

Entity level TAG (Resource Tag)

In example we are creating a tag configuration Cost Center with values marketing and development.

  1. Open CONFIG_ENTITY_DEFAULTS - package in Agile Data Engine

  2. Open Configuration

  3. Choose Database Tags - tab

  4. Click Create Database Tag

  5. Create Database Tag with following information

    1. Label - Label for the Tag shown in Agile Data Engine, for example “Cost Center”

    2. Technical Name - TAG name in BigQuery having the project id included

      CODE 
      <project-id>/cost_center

    3. Description - Short description for which purposes this tag is to be used for

    4. Select options:

      1. Allowed on entity level

      2. has values

    5. Click “Create”

  6. Click the name of freshly created Cost Center from the Database tag list

    1. Click Create Database Tag Value

    2. Type in the wanted Tag value information (marketing)

      1. Value - Tag value in BigQuery

      2. Description - Short description for the tag value

    3. Click “Create”

    4. Click Create Database Tag Value

    5. Type in the wanted Tag value information (development)

      1. Value - Tag value in BigQuery

      2. Description - Short description for the tag value

    6. Click “Create”

  7. Commit the CONFIG_ENTITY_DEFAULT package

Attribute level TAG (Policy Tag)

  1. Open CONFIG_ENTITY_DEFAULTS - package in Agile Data Engine

  2. Open Configuration

  3. Choose Database Tags - tab

  4. Click Create Database Tag

  5. Create Database Tag with following information

    1. Label - Label for the Tag shown in Agile Data Engine, for example “Information type”

    2. Technical Name - The ID of the Policy Tag in BigQuery, syntax is of type

      CODE 
      projects/<project_id>/locations/<location>/taxonomies/<taxonomy_id>/policyTags/<policy_tag_id>

    3. Description - Short description for which purposes this tag is to be used for

    4. Select options:

      1. Allowed on attribute level

    5. Click “Create”

  6. Commit the CONFIG_ENTITY_DEFAULT package

Entity level Resource Tags will be updated through BigQuery API call, where the table involved will be updated to contain the desired resource tags.

Attribute level Policy Tags will be updated through BigQuery API call, where the table schema involved will be updated to contain the desired policy tags on desired columns

Required permissions in Google BigQuery

For ADE service account, the following permissions need to be added:

CODE 
-- Permissions to bind or unbind tags
resourcemanager.tagValueBindings.create
resourcemanager.tagValueBindings.delete

-- Permissions to get taxonomies and assign those to columns
datacatalog.taxonomies.get
bigquery.tables.setCategory
-- Permissions to query masked data
bigquery.dataPolicies.maskedGet
datacatalog.categories.fineGrainedGet
JavaScript errors detected

Please note, these errors can depend on your browser setup.

If this problem persists, please contact our support.

Contact Support Close