from_file_list

Target database: Snowflake, Google BigQuery, Databricks SQL

Resolved as a comma-separated source file list in file load execution.

See also:

Usage

VARIABLE

<from_file_list>

Notes

<from_path> can be used with <from_file_list>. When multiple files are queued with a manifest, <from_path> gives the lowest common part of the file paths and <from_file_list> gives the unique part of the file paths. In single file loads <from_file_list> is resolved as the single file name.

Examples

Snowflake

The variable from_file_list represents a list of file names to be loaded from a Snowflake stage.
When used in a COPY INTO statement, ADE injects this list directly into the Snowflake FILES clause.

CODE

FILES=(<from_file_list>)

Example file list in executed COPY INTO statement in Snowflake:

CODE

FILES=('myfile1.csv', 'myfile2.csv')

More information in Snowflake documentation.

BigQuery

The variable from_file_list can be used to pass specific file URIs into a LOAD DATA statement. When referenced, it maps directly to the BigQuery uris parameter:

CODE

uris=<from_file_list>

Example file list in executed LOAD DATA statement in BigQuery:

CODE

uris=['gs://bucket/path1/myfile1.csv', 'gs://bucket/path1/myfile2.csv']

More information in BigQuery documentation.