Notifications

Edition: Private

Notifications are empty files named like data files or manifests that are added into the notification bucket/container to queue the corresponding files or manifests for loading into the target database. Agile Data Engine Private Edition installation creates an S3 bucket (AWS) or a blob container (Azure) for notifications per Runtime environment.

See also:

Private AWS

Notification bucket name is defined in the Runtime installation, buckets are created per Runtime environment. Please refer to your project documentation for the bucket names.

For example:

s3://example-notify-bucket/

In AWS a notification file is automatically created by AWS Lambda into the notification bucket when a data file or a manifest is added into the default file load bucket. If the default file load bucket is not used for source data, a similar notification solution has to be developed to notify Agile Data Engine about new files.

Private Azure

Notification container name and storage account name are defined in the Runtime installation. Storage accounts and notification containers are created per Runtime environment. Please refer to your project documentation for the storage account and container names.

For example:

azure://examplestorageaccount.blob.core.windows.net/example-notify-container/

In Azure a process has to be developed to create notifications triggered by new files or manifests.

Notes

Notification paths and filenames in the notification bucket/container have to match the file/manifest names and paths in the file load bucket/container.

Notification files are empty, i.e. they have no content.

If a file load succeeds, notifications processed in that file load are moved into the archive folder in the notification bucket/container.

If a file load fails, notifications processed in that file load are moved into the error folder in the notification bucket/container.

Examples

See notification examples in the single file load and manifest load articles.