Schema | Agile Data Engine

API_CALL
- request
  - query_parameters
  - headers
- retries
  - rules
    - conditions
- response
  - transformations
    - variables
    - conditions

API_CALL

Name	Type	Format	Required	Description
request	object		Yes	Request to make. (See definition)
retries	object		No	Retry configuration. Delay between retries are calculated as delay_seconds = max(min(backoff_factor * (2^(retry-1)) , backoff_max_seconds), backoff_min_seconds) + random(0, backoff_jitter_seconds) (See definition)
response	object		No	(See definition)
description	string		No	Description of the api call.
type	string	["HTTP"]	Yes	Type of API call. Currently supports 'HTTP'.

request

Request to make.

Name	Type	Format	Required	Default	Description
query_parameters	object		No		Query parameters are key/value pairs and are included to url query parameters with urlencoding. There is no restrictions what user can define here. (See definition)
headers	object		No		(See definition)
method	string	["GET","POST","PUT","DELETE"]	Yes		HTTP method to use.
description	string		No		Description of the request.
timeout_seconds	integer		No	300	Timeout in seconds. Default is 300 seconds. 0 will disable timeout (not recommended)
url	string	uri	Yes		Mandatory URL for the API call. Must use HTTPS.
content	string		No		Request body content, represented as a string.

query_parameters

Query parameters are key/value pairs and are included to url query parameters with urlencoding. There is no restrictions what user can define here.

headers

Name	Type	Required	Default	Description
Authorization	string	No		Authorization header.
Accept	string	No	application/json
Content-Type	string	No	application/json

retries

Retry configuration. Delay between retries are calculated as delay_seconds = max(min(backoff_factor * (2^(retry-1)) , backoff_max_seconds), backoff_min_seconds) + random(0, backoff_jitter_seconds)

Name	Type	Required	Default	Description
total	integer	No	0	Total number of retries. Set 0 to disable.
backoff_jitter_seconds	integer	No	0	Maximum jitter to add to backoff time in seconds.
backoff_min_seconds	integer	No	0	Minimum backoff time in seconds.
backoff_factor	integer	No	2	Backoff multiplier.
description	string	No		Description of the retry logic.
rules	array (object)	No		(See definition)
backoff_max_seconds	integer	No	60	Maximum backoff time in seconds.

rules

Name	Type	Format	Required	Default	Description
description	string		No		Description of the retry rule.
conditions	array (object)		Yes		(See definition)

conditions

Name	Type	Format	Required	Default	Description
regexp	string		No		Regular expression to match against the source. Only feasible with type REGEXP_VALUE_MATCHER.
negate	boolean		No	false	Negate/invert boolean logic.
values	array (["string","integer"])		No		List of values to check against matcher value.
description	string		No		Description of the condition.
source	string		Yes		Source of the value to match. Can refer for example to response status with variable like <http_status_code> or http response content with <http_response_content>.
type	string	["VALUE_MATCHER","REGEXP_VALUE_MATCHER"]	Yes		Type of condition. VALUE_MATCHER direct matcher for value. REGEXP_VALUE_MATCHER can be used to extract certain part of source by using regexp pattern.

response

Name	Type	Format	Required	Default	Description
transformations	array (object)		No		(See definition)
description	string		No		Description of the response. If transformations are not defined then we default to logic than 200 and 201 responses are successful and we don't extract any affected rows info.

transformations

Name	Type	Format	Required	Description
result	string	["SUCCESS","FAILED"]	No
variables	object		No	Variables are key/value pairs where key is name of variable. Variables can be used in later load steps or referenced in result/affected_rows. (See definition)
description	string		No	Description of the transformation.
affected_rows	["string","integer"]		No	Value indicating affected rows.
conditions	array (object)		Yes	(See definition)

variables

Variables are key/value pairs where key is name of variable. Variables can be used in later load steps or referenced in result/affected_rows.

Additional properties values must match following definition

Name	Type	Format	Required	Description
regexp	string		No	Reg exp pattern with group to extract part of source value. Only feasible with REGEXP_VALUE
description	string		No	Description of the variable.
source	["string","integer"]		Yes	Value used as source for variable. Can refer for example to response status with variable like <http_status_code> or http response content with <http_response_content>.
type	string	["VALUE","REGEXP_VALUE"]	Yes	Type of variable extraction. VALUE can be used to directly use source value for variable. REGEXP_VALUE can be used to allow using regular expression to capture part of source content.

conditions

Name	Type	Format	Required	Default	Description
regexp	string		No		Regular expression to match against the source. Only feasible with type REGEXP_VALUE_MATCHER.
negate	boolean		No	false	Negate/invert boolean logic.
values	array (["string","integer"])		No		List of values to check against matcher value.
description	string		No		Description of the condition.
source	string		Yes		Source of the value to match. Can refer for example to response status with variable like <http_status_code> or http response content with <http_response_content>.
type	string	["VALUE_MATCHER","REGEXP_VALUE_MATCHER"]	Yes		Type of condition. VALUE_MATCHER direct matcher for value. REGEXP_VALUE_MATCHER can be used to extract certain part of source by using regexp pattern.