Snippet Framework API

exception silo.low_code.ConfigGenerationError
exception silo.low_code.CredAttributeError
exception silo.low_code.CredTypeNotSupported
exception silo.low_code.CredentialError
exception silo.low_code.DeviceError
exception silo.low_code.FrameworkError
exception silo.low_code.InvalidSnippetArgumentException
silo.low_code.read_cached_results(cache_key, reuse_for=None)

Return the most recent cached element for the given cache_key

This function enables the lookup of Snippet Framework cached elements outside of the Snippet Framework. While this function could work with automatic cache keys the Snippet Framework generates, its best used by overriding the cache_key name within the steps configuration.

The functionality of reuse_for operates the same way within the Snippet Framework. This value is specified in minutes before a cached entry is considered stale. If this value is specified, an entry is considered stale if the following criteria is met:

\[collection\_time + reuse\_for <= current\_time\]

If the newest entry is considered stale, the exception silo.apps.cache.CacheDataIsExpired will be raised.

If an invalid object is found (bad formatting within the cache), the exception MetadataError will be raised.

Parameters:

  • cache_key (str) – Specified cache_key for the step

  • reuse_for (int) – Time (in seconds) before considering an entry as stale

Return type:

object

silo.low_code.read_step_cache(step, reuse_for, metadata_key, gmtimestamp)

Check the cache for step data prior to step execution

Parameters:

  • step (int) – Number of the current step

  • reuse_for (int) – Cache use limit in seconds

  • metadata_key (str) – Current step request id or key hashed

  • gmtimestamp (int) – gmtimestamp of the current collection window

Returns:

Cached data if it exists, otherwise None

silo.low_code.register_cacher(*args, **kwargs)

Decorator to register a cacher

All arguments are optional during registration. A Cacher can request any of the Standard Parameters and an additional argument:

  • step_cache: CacheManager from silo-apps for interacting with cache associated to the step

Parameters:

  • get_req_id (callable) – Function used for generating the request id. If one is not provided, the request_id will be <step_name>_<step_args_in_string_form>. This function can utilize any of the Standard Parameters a

  • name (str) – Name for the step that will be referenced in the snippet argument. If a name is not specified, it will use the pythonic name of the callable. For example, if you use have def cool_beans(result): ... the name would be cool_beans.

  • metadata (dict) – Metadata related to the step

  • read (callable) – Callable function that performs a cache read for fast-forwarding. This function can utilize any of the Standard Parameters.

  • required_args (list) – List of required top-level arguments for the step. If these arguments are not present in the step_args, the Snippet Framework will not attempt to execute the collection and log a warning message instead.

  • arg_required (bool) – If an argument is required for a step. This denotes a step must have an argument, regardless of the type. This can be used in conjunction with required_args if your step accepts a dictionary or a single value.

Returns:

Original object

Return type:

object

silo.low_code.register_processor(*args, **kwargs)

Decorator to register a processor

Parameters:

  • get_req_id (callable) –

    Function used for generating the request id. If one is not provided, the request_id will be <step_name>_<step_args_in_string_form>. This function can utilize any of the Standard Parameters and additional arguments:

    • reg_info: Registration information for the step

    • caller: Callable assigned to the step

  • name (str) – Name for the step that will be referenced in the snippet argument. If a name is not specified, it will use the pythonic name of the callable. For example, if you use have def cool_beans(result): ... the name would be cool_beans.

  • metadata (dict) – Metadata related to the step

  • required_args (list) – List of required top-level arguments for the step. If these arguments are not present in the step_args, the Snippet Framework will not attempt to execute the collection and log a warning message instead.

  • arg_required (bool) – If an argument is required for a step. This denotes a step must have an argument, regardless of the type. This can be used in conjunction with required_args if your step accepts a dictionary or a single value.

Returns:

Original object

Return type:

object

silo.low_code.register_requestor(*args, **kwargs)

Decorator for registering Requesters

Parameters:

  • get_req_id (callable) – Function used for generating the request id. This is a very important function and its operation must be understood when developing a requestor. The proper choice for building the requestor is critical to building a highly perfomant dynamic app. If one is not provided, the request_id will be <step_name>_<step_args_in_string_form> and you must ensure this provides the correct level of uniquness. This function can utilize any of the Standard Parameters

  • name (str) – This function allows for overriding the pythonic name for the step. If a name is not specified, it will use the pythonic name of the callable. For example, if you use have def cool_beans(result): ... the name would be cool_beans.

  • metadata (dict) – Metadata related to the step

  • rewind (callable) –

    Callable function that updates the step_args to start the rewinding process. This function can utilize any of the Standard Parameters and an additional argument:

    • data_request: RequestMoreData exception that was previously raised.

  • generate_agent_config (callable) – Function used to generated agent configs. The function must return a ProtocolConfig This function can utilize any of the Standard Parameters.

  • manipulate_agent_data (callable) –

    Function used to perform any pre-processing on the agent results before providing it to the pipeline. This function can utilize any of the Standard Parameters and additional arguments:

    • file_timestamp: Time of the collection

    • pid: ProcessID of the JVM being monitored

    • jvm_name: Name of the JVM being monitored

  • validate_request (callable) – Function used to validate a request prior to execution. If an issue is detected, an exception must be raised and the ResultContainer will not be executed. This function can utilize any of the Standard Parameters.

  • required_args (list) – List of required top-level arguments for the step. If these arguments are not present in the step_args, the Snippet Framework will not attempt to execute the collection and log a warning message instead.

  • arg_required (bool) – If an argument is required for a step. This denotes a step must have an argument, regardless of the type. This can be used in conjunction with required_args if your step accepts a dictionary or a single value.

Returns:

Original object

Return type:

object

silo.low_code.register_rmd(*args, **kwargs)

Decorator to register a RequestMoreData step

A RequestMoreData can request any of the Standard Parameters and additional arguments:

  • set_index: Callable for setting the index of the current iteration. This should be used if you need to easily identify the information in the following step.

  • set_max_iterations: Callable for setting the maximum number of iterations before the Snippet Framework stops performing the collections. The default value for the amount of iterations is 50.

Parameters:

  • get_req_id (callable) – Function used for generating the request id. If one is not provided, the request_id will be <step_name>_<step_args_in_string_form>. This function can utilize any of the Standard Parameters

  • name (str) – Name for the step that will be referenced in the snippet argument. If a name is not specified, it will use the pythonic name of the callable. For example, if you use have def cool_beans(result): ... the name would be cool_beans.

  • metadata (dict) – Metadata related to the step

  • required_args (list) – List of required top-level arguments for the step. If these arguments are not present in the step_args, the Snippet Framework will not attempt to execute the collection and log a warning message instead.

  • arg_required (bool) – If an argument is required for a step. This denotes a step must have an argument, regardless of the type. This can be used in conjunction with required_args if your step accepts a dictionary or a single value.

Returns:

Original object

Return type:

object

silo.low_code.register_syntax(*args, **kwargs)

Decorator for registering a Top-Level or Macro Syntax

Register a Syntax step that will be validated once the Framework is initialized. All arguments are optional during registration.

A Top-Level Syntax is a step that converts a text-based Snippet Argument into an Execution Plan that the Snippet Framework can process. The most familiar example of this is the low_code Syntax, which transforms a YAML-based instruction set into it’s equivalent pythonic Execution Plan. Top-Level Syntaxes can only be used to parse a Snippet Argument - they must not be referenced within another Execution Plan, such as one generated by a Macro Syntax. A Top-Level Syntax step must return a dictionary containing the key execution that maps to the Execution Plan list. Optionally, it can also include a name key that maps to a string representing the identifier of the Collection.

A Top-Level Syntax can request any of the Standard Parameters and the following additional parameters:

  • snippet_arg (str): The Snippet Argument for the Collection

    (after substitution) with the Syntax identifying header

  • snippet_arg_content (str): The Snippet Argument for the Collection

    (after substitution) without the Syntax identifying header

When utilizing a Top-Level Syntax, the Snippet Argument must start with a Syntax Identifier (the name given to the Top-Level Syntax step) followed by a colon (:).

Example 1: Syntax Identifier

low_code:    (Syntax Identifier)
  version: 2
  steps:
    - ...

Example 2: Inline Syntax Identifier

new_syntax_identifier: {"test": True}    (Inline Syntax Identifier)

A Macro Syntax is a nestable Syntax step that creates a set of instructions known as an Execution Plan that expands automatically in-place inside the Execution Plan that contains (references) it. Macro Syntaxes cannot be used to parse a Snippet Argument - they must be referenced (nested) within another Execution Plan, such as one generated by a Top-Level Syntax. A Macro Syntax inherits step arguments (step_args) from the Execution Plan that contains it.

Unlike a Top-Level Syntax, a Macro Syntax cannot request snippet_arg or snippet_arg_content as parameters. This is due to the fact that by the time a Macro Syntax executes, the Snippet Argument has already been transformed into an Execution Plan by the Top-Level Syntax. However, a Macro Syntax can still request any of the Standard Parameters, as well as Substitution Parameters and any Custom Substitution keys defined by the user as parameters.

When utilizing a Macro Syntax, you must reference the Macro Syntax Identifier (the name given to the Macro Syntax step) in the Execution Plan created from it’s containing Syntax. Ex:

low_code:    (Top-Level Syntax Identifier)
  version: 2
  steps:
    - custom_macro    (Macro Syntax Identifier)
    - ...
Parameters:

  • get_req_id (callable) –

    Function used for generating the request id. If one is not provided, the request_id will be <step_name>_<step_args_in_string_form>. This function can utilize any of the Standard Parameters and additional parameters:

    • reg_info: Registration information for the step

    • caller: Callable assigned to the step

  • name (str) – Name for the Syntax that will be referenced in the snippet argument. If a name is not specified, it will use the pythonic name of the callable. For example, if you use have def cool_beans(result): ... the name would be cool_beans.

  • macro (bool) – Indicates if the Syntax step is a Macro or Top-Level Syntax. Setting this flag enables the additional functionality only available to Macro Syntax steps, such as substitution values as parameters.

  • metadata (dict) – Metadata related to the step. The macro flag can also be set here (metadata={"macro": True}) which achieves the same behavior as setting the macro flag directly in the decorator.

  • required_args (list) – (Macro only) List of required step arguments. If these arguments are not present in the step_args, the Snippet Framework will not attempt to expand the Macro and will throw an error instead.

  • arg_required (bool) – (Macro only) If any step argument is required. This denotes a Macro must have an argument, regardless of the type. This can be used in conjunction with required_args if your Macro accepts a dictionary or a single value.

Returns:

Original object

Return type:

object