GQLRequest

Query a remote GraphQL endpoint and retrieve the data.

Added in version 1.6.0.

Friendly name: GraphQL Request
Step class: base_steps_syncpack.steps.GQLRequest.GQLRequest
Step version: 1.0.0
Required parameters: prefix_url

Parameters

Name	Type	Required	Default	Description
`sub_key`	string	no	–	Sub Key of the GQL Response to Page over. Example: `constituents`
`save_errors`	boolean	no	`False`	If there are errors found from the GQL response, save them along with the rest of the payload
`follow_cursor`	boolean	no	`False`	Whether or not to continually make calls following the last cursor. If this is enabled, their MUST be an after:”” present in your query. The gql query must also be sure to request the cursor property be present in the response, as well requesting pageInfo with hasNextPage in the response
`edges_only`	boolean	no	`False`	Whether or not to only include edges when parsing through response and saving data. If true, the return result will be a list of all edges foundwhen following cursor (if enabled). If false, the result will be the ‘data’ value of the response
`mutation`	boolean	no	`False`	Set to True for Mutation calls.
`gql_limit_name`	string	no	`first`	The variable name for the max number of records to fetch in one GQL request.
`gql_offset_name`	string	no	`after`	The variable name for the next cursor to be fetched in GQL request.
`gql_chunk_size`	number	no	`500`	The max number of records to fetch in one GQL request.

Required inherited parameters

These are defined on Shared parameters: HTTPBaseStep but are required here, so a working configuration must set them.

Name	Type	Required	Default	Description
`prefix_url`	string	yes	–	used with relative_url to create the full URL. Example: `http://host.example.com`

Other inherited parameters

This step also accepts 31 optional parameter(s) inherited from Shared parameters: HTTPBaseStep.

Every step also accepts the platform parameters common to all steps – see Platform parameters.

Dependencies

Inherits from: GQLRequest -> HTTPBaseStep -> BaseStep -> Serializable
SyncPack modules: base_steps_syncpack.steps.HTTPBaseStep, base_steps_syncpack.util.gql_utils, base_steps_syncpack.util.request_params, base_steps_syncpack.util.request_utils
External packages: ipaascommon

Behaviour

The supported GraphQL step. Fixes gql_query = True and method = "POST", then delegates to Shared parameters: HTTPBaseStep — so it gets the full shared HTTP surface: OAuth, bearer, mTLS, timeouts and token caching. The query itself comes from the inherited query and variables parameters (or from upstream data via query_from_input / variables_from_input).

Cursor paging is driven by the gql_* parameters above rather than the REST chunk_* ones: gql_limit_name (default first) and gql_offset_name (default after) name the variables in your query, and gql_chunk_size is the page size.

Outputs

Saved through HTTPBaseStep (pickled when pickle_output is set). The shape depends on three parameters:

Configuration	Saved data
`follow_cursor`, `edges_only`	One list, the concatenated `edges` from every page.
`follow_cursor` only	A list of per-page `response_data` dicts.
neither, or `mutation`	A single `response_data` object.
`edges_only` only	The `edges` list from the single response.

With save_errors, whichever of the above applies is wrapped:

{"data_out": {"...": "..."}, "errors": [{"message": "..."}]}

An empty response yields {}.

Example

{
  "prefix_url": "https://example.sciencelogic.io",
  "relative_url": "/graphql",
  "auth_method": "OAuth",
  "oauth_client_id": "${config.client_id}",
  "oauth_client_secret": "${config.client_secret}",
  "oauth_token_url": "https://example.sciencelogic.io/oauth/token",
  "query": "query Devices($first: Int, $after: String) { devices(first: $first, after: $after) { edges { node { id name } } } }",
  "sub_key": "devices",
  "follow_cursor": true,
  "edges_only": true,
  "gql_chunk_size": 500
}

Notes

follow_cursor needs sub_key to know which part of the response holds the page info. Setting one without the other is the usual cause of a query that returns only the first page.
Set mutation for writes. Cursor following is skipped for mutations, so leaving it unset on a mutation can produce a spurious second request.
save_errors changes the output shape, not just its contents. A downstream step written against the unwrapped form breaks when you turn it on.

Warning

Do not combine ``follow_cursor`` with ``save_errors``. The wrapping happens inside the paging loop: after the first page all_data is replaced by the {"data_out": ..., "errors": ...} dict, and the next iteration then calls .append() (or +=) on that dict and raises. It works for a single-page result and fails on the second page. Use one or the other until this is fixed at source.

API

class base_steps_syncpack.steps.GQLRequest.GQLRequest

Bases: HTTPBaseStep

process_rest_command(payload)

Processes query results

Parameters:: payload – payload to be posted
Returns:: Processed query results

process_response_counts(response_dict, response_headers)

Process number of responses from query results

Parameters:

response_dict – list or dictionary of formatted results from HTTP call
response_headers (dict) – returned headers from HTTP call

Returns:

Updates total and offset attributes used for paging