sl.snippet_api.collection package

This modules provides a simplified interface for interacting with Dynamic Applications. The purpose of this module is to remove the noise regarding the required structure for backend processing but not required for collection. This is accomplished by creating several new data-structure containing only the required information. Those data structures are

The root data-type will be Collection() and hold the other objects containing their relevant information. The collection allows for setting values based off indices as a dictionary rather than the list of tuples required by result handler / app.oids. This simplifies the collection as knowing the structure is no longer needed.

Some benefits of using Collection are:

  • Simplified data structure

  • Easy to initialize by get_collections()

  • Interacting with the data to save / retrieve becomes standard regardless of app type

  • Devices have a consistent format regardless of app type

  • Saving result information is handled by save_collections()

Note

Below is and example of using the collection object inside of a Dynamic Application Snippet

from sl.snippet_api.collection import *
import time
import datetime

collections = create_collections(self)
current_time_obj = find_collection(collections, name="Current Time")
date_obj = find_collection(collections, name="Date")
for did, device in current_time_obj.devices.items():
    cur_time = time.time()
    current_time_obj.add_result(did, int(cur_time), index=1)
    current_time_obj.add_result(did, cur_time % 1, index=2)
    date_obj.add_result(did, datetime.datetime.now())
save_collections(collections, self)
class sl.snippet_api.collection.Collection(**kwargs)

Bases: object

Simplified representation of a Dynamic Application’s Collection Object

This object represents a collection in a Dynamic Application. It holds information about the app, the aligned devices, and the collection object. After collection, the object will also hold results.

Variables:

  • app (sl.snippet_api.collection.dynamic_app.DynamicApp) – Minimal Dynamic Application representation

  • devices (dict) – Dict of Device that need to be collected. The dictionary is indexed by the Device ID.

  • group (int) – Collection group of the collection

  • name (str) – Name for the collection

  • obj_id (int) – Object ID of the collection

  • results (dict) – the collected results for this object

  • argument (str) – Snippet Argument of the collection

  • type (int) – Class representation of the collection

  • error (object) – Error information related to the Collection during the collection process. The value that gets set should be able to convert to a string to display the correct error message. If the value is set to False or None (default) the Collection will be saved through save_collections().

  • snippet_id (int) – ID of the snippet tied to this collection

__init__(**kwargs)

Initialize the Collection

Parameters:

  • app (sl.snippet_api.collection.dynamic_app.DynamicApp) – Application definition

  • obj_id (int) – The ID of the object

  • group (int) – Collection group defined by the Dynamic Application

  • devices (dict) –

    Dictionary of Device objects indexed by Device ID

    >>> {
>>>    50: Device(did=50)
>>> }

  • name (str) – Collection name

  • argument (str) – Collection argument

  • col_class (int) – Class / Type for the collection

  • snippet_id (int) – ID of the snippet tied to this collection

add_device_error()

Save a device communication error.

Only unique exception types are stored. Duplicate types are discarded. This function should only be called within an exception block or a RuntimeError exception will be raised.

Return type:

None

add_did_error(did)

Store errors from processing results for a did

Only unique exception types are stored. Duplicate types are discarded. If the collection is processed by save_collections() a Device Log message will be generated.

Parameters:

did (int) – Device ID used during collection

Return type:

None

add_result(did, value, index=0)

Store the result of a collection

Parameters:

  • did (int) – Device ID used during collection

  • value (Any) – Result of the collection

  • index (Optional[Any]) – Index for the result. If this is not specified 0 will be used for the index

Return type:

None

set_results_dict(results)

Store the results of a completed collection object

The results must be stored with the following structure:

{
    did: {
        index1: value,
        index2: value,
        ...
        indexn: value
    }
}
Parameters:

results (Dict[Any, Any]) – Results for the collection object

Return type:

None

set_results_list(did, results)

Store the results of a completed collection object

The results must be stored with the following as a list or list of tuples:

[val1, val2, val3]
[(ind1, val1), (ind2, val2), (ind3, val3)]
Parameters:

  • did (int) – Device ID used during collection

  • results (List[Tuple[Any, Any]]) – Results for the collection object

Return type:

None

exception sl.snippet_api.collection.CollectionError

Bases: Exception

exception sl.snippet_api.collection.CollectionErrorInvalidDeviceID

Bases: CollectionError

class sl.snippet_api.collection.Credential(secret_fields=None, **kwargs)

Bases: object

Simplified representation of a Credential

Creates a credential object from a credential dictionary. Adds additional functionality such as name lookup and prevention for printing secure fields on accident. Credential fields are accessed by using Credential.fields["cred_field"]. For example, if you wanted to access the credential host, you would use the follow:

credential.fields["cred_host"]
Variables:

  • id (int) – Credential ID

  • name (str) – Credential name

  • cred_type_name (str) – Credential Type Name

  • fields (dict) – Credential dictionary

  • secret_fields (list) – Fields that are considered secret

Parameters:

secret_fields (List[str]) – Fields that are considered secret. If a credential is used in conjunction with sl.snippet_api.collection.LoggingRedactor, these fields will not be printed to the logs. Refer to LoggingRedactor for more information about the redactor.

class sl.snippet_api.collection.Device(did=None, ip=None, name=None, cred=None, root_did=None, parent_did=None, unique_id=None, guid=None, dn=None, pdu_packing=False)

Bases: object

Simplified representation of a Device

This object represents a SL1 device being used in a Snippet Dynamic App.

Variables:

  • cred (sl.snippet_api.collection.Credential) – Credential information for the device

  • dn (str) – Unique identifier for the DCM tree

  • guid (str) – Global identifier for the DCM tree

  • id (int) – Device Identifier

  • ip (str) – Primary IP address for the device

  • name (str) – name of the device

  • parent_id (int) – Parent device ID used in the DCM tree

  • pdu_packing (bool) – PDU Packing support for SNMP

  • root_id (int) – Root device ID used in the DCM tree

  • guid – Universal identifier for the DCM tree

Parameters:

  • did (Optional[int]) – SL1 device ID. Default is None

  • ip (Optional[str]) – ip address of the device, if available. Default is None

  • name (Optional[str]) – name of the device

  • cred (Optional[Credential]) – The credential entry for the device.

  • root_did (Optional[int]) – the device’s root device ID, if this is a component device. Default is None

  • parent_did (Optional[int]) – the device’s parent device ID, if this is a child component. Default is None

  • unique_id (Optional[str]) – the device’s unique identifier in SL1, if a component device. Default is None

  • guid (Optional[str]) – the device’s global identifier in SL1, if a component device. Default is None

  • dn (Optional[str]) – the device’s unique identifier in the component tree, if a component device. Default is None

  • pdu_packing (Optional[bool]) – does this device support PDU packing. Default is False

parent_dn

Lookup the parent device distinguished name for the given device

parent_name

Lookup the parent device name for the given device

parent_uid

Lookup the parent device unique id for the given device

root_dn

Lookup the root device distinguished name for the given device

root_name

Lookup the root device name for the given device

root_uid

Lookup the root device unique id for the given device

class sl.snippet_api.collection.DynamicApp(app_id=None, name=None, guid=None, freq=None, gmtime=None)

Bases: object

Simplified representation of a Dynamic Application

Variables:

  • id – Dynamic Application ID

  • name – Name assigned to the Dynamic Application

  • freq – Time (in minutes) for the collection cycle

  • gmtime – Time jobs were created

  • guid – GUID assigned to the Dynamic Application

class sl.snippet_api.collection.LoggingRedactor(name, level=0)

Bases: Logger

Logger that removes secrets from logs

Custom logger that prevents secrets from being printed. Secrets can be added by calling initialize_secrets() or add_secret() methods.

makeRecord(name, level, fn, lno, msg, args, exc_info, func=None, extra=None, sinfo=None)

Create a new log record

This uses the same signature as logging.Logger, so have fun with params

Return type:

LoggingRedactorRecord

sl.snippet_api.collection.add_secret(secret)

Add a secret to be filtered

Adding a secret to the list will prevent it from being logged within the redacted loggers.

Parameters:

secret (str) – Secret that should not be logged

Return type:

None

sl.snippet_api.collection.create_collections(app, cred_extended_lookup=True)

Builds a list of collection objects related to the Dynamic Application

Convert a Snippet Dynamic Applications self object into a more user-friendly list of Collection’s. These collections abstract the platform to allow for a more pythonic way to interact with the data. This allows for a higher-degree of platform abstraction since it will handle Bulk and Non-Bulk Dynamic Applications and return the name structure regardless of Dynamic Application type.

Note

Journals are not currently supported

Parameters:

  • app – The self object from within the Dynamic Application

  • cred_extended_lookup (Optional[bool]) – Perform extended type name lookup for credentials. This allows lookups for Universal Credentials type names through the database

Return type:

List[Collection]

Returns:

List of all collection objects (list of collection_info objects) with the required information to perform the collection

sl.snippet_api.collection.create_device(da_job_dict, cred)

Creates the device from a given DynamicAppJob

Note

This function should only be used externally when mocking a device during testing.

Parameters:

  • da_job_dict (Dict[str, Any]) – dict_repr from a DynamicApp job

  • cred (Credential) – Credential for the device

Return type:

Device

Returns:

The newly created Device object

sl.snippet_api.collection.credential_factory(cred_details, ip, protected_fields=None)

Builds a Credential object from a credential dictionary

Builds and returns the correctly-classed credential. If a UCF credential is presented, the protected fields will be looked up (if possible). The following modifications are made to the credential:

  • SNMP

  • Update cred_host to the provided IP address

Parameters:

  • cred_details (dict) – Decoded credential

  • ip (str) – IP address associated with the device

  • protected_fields (Optional[List[str]]) – Fields identified as protected fields. By specifying this, the list of protected fields will be extended to include these.

Return type:

Credential

sl.snippet_api.collection.dict_to_result_handler(value)

Converts a dict to be consumed by SL1

Converts the dictionary result into the list of tuples (index, value) to be consumed by SL1. During this process, the index is checked for length, and if required, will shorten the index based on a hash. See get_index_max_length().

Parameters:

value (dict) – The dict (ideally with scalar values) that was retrieved

Return type:

List[Tuple[Any, Any]]

sl.snippet_api.collection.find_collection(collections, name=None, identifier=None, group=None, ignore_labels=False)

Finds a collection object from the collection object list

Parameters:

  • collections (List[Collection]) – Collection objects from create_collections

  • name (Optional[str]) – Name of the object to find

  • identifier (Optional[int]) – Object ID of the object to find

  • group (Optional[int]) – Collection group of the object to find

  • ignore_labels (Optional[bool]) – Do not include labels in the group

Return type:

Optional[Collection]

Returns:

The Collection Object if found, otherwise None

sl.snippet_api.collection.find_collections(collections, name=None, identifier=None, group=None, ignore_labels=False)

Finds all collections that match the required search

Parameters:

  • collections (List[Collection]) – Collection objects from create_collections

  • name (Optional[str]) – Name of the object to find

  • identifier (Optional[int]) – Object ID of the object to find

  • group (Optional[int]) – Collection group of the object to find

  • ignore_labels (Optional[bool]) – Do not include labels in the group

Return type:

List[Collection]

Returns:

All collections that match the search

sl.snippet_api.collection.gather_app_snippets(app_id, state=None, dbc=None)

Get snippets aligned to the Dynamic Application

Note

While this function is part of the API, it should not be used during Dynamic Application development.

Parameters:

  • app_id (int) – Dynamic Application ID

  • state (Optional[bool]) – State of the snippet

  • dbc (Union[silo_cursor, SiloCursor, None]) –

    The database cursor. If none is provided, a local database cursor will be used

    • If state=``True``: all enabled snippets

    • if state=``False``: all disabled snippets

    • if state=``None``: all snippets (default)

{
    "req_id": {
        "collections": [obj_id1, obj_id2],
        "snippet": "Snippet Code",
        "state": 1,

    }
}
Return type:

Dict[int, Dict[str, Any]]

Returns:

All snippets for the expected DA ID of the given state

sl.snippet_api.collection.get_active_collections(collections, alt_logger=None, snippets=None, dbc=None)

Identify all active collections

Note

This is part of the API but not expected to be used by customers

Parameters:

  • collections (List[Collection]) – List of sl.snippet_api.collection.Collection objects

  • alt_logger (Optional[Logger]) – Logger where the messages should be written. If no logger is present, it will use the logger within the module

  • snippets (Optional[Dict[str, str]]) – Snippets that should be checked. If not specified, a database query will gather the snippets

  • dbc (Union[silo_cursor, SiloCursor, None]) – The database cursor. If none is provided, a local database cursor will be used

Return type:

List[Collection]

Returns:

The collections that are active, after filtering

sl.snippet_api.collection.initialize_secrets(cred, secret_keys=None)

Initializes secrets from Credential’s secret fields

Add all known secrets from the credential to the secrets list. The secrets for the credential are identified by the credential itself. If any additional secrets are required, you can call add_secret() or send them in using the secret_keys parameter.

Parameters:

  • cred (Credential) – Credential from silo-apps

  • secret_keys (Optional[List[str]]) – dictionary keys in cred that require filtering

Return type:

None

sl.snippet_api.collection.parse_snippet_uri(uri, keep_plus=False)

Parses a URI into the components.

The returned information will be the results of urlparse and parse_qs:

parse_snippet_uri("label://something?order=123&index=456")
(
     ParseResult(
         scheme='label',
         netloc='something',
         path='',
         params='',
         query='order=123&index=456',
         fragment=''
     ),
     {
         'order': '123',
         'index': '456'
     }
)
Parameters:

  • uri (str) – URI to parsed

  • keep_plus (Optional[bool]) – Replace ‘ ‘ with ‘+’

Return type:

Tuple[ParseResult, Dict]

Returns:

Tuple containing the parsed information.

sl.snippet_api.collection.reset_secrets()

Remove all secrets from filtering

Return type:

None

Note

This is part of the internal API for testing and should not be called directly. Use :py:func`~sl.snippet_api.testing.reset_secrets`.

sl.snippet_api.collection.save_collections(collections, app, **kwargs)

Store the result of collection object in the result_handler by the oid

Saves collections to the app object. Processes any required logic for label auto-generation. If the collection has the attribute error populated, the result will not be saved.

Note

Labels for a collection group can be automatically generated by using the label protocol. The protocol specifies precedence in which labels should be set. The label is generated from the index name of the collection(s). The available methods for the protocol label are:

  • lookup. This method will use the indices and set them as the label.

The following query parameters are available for the labels protocol:

  • order. Specifies the precedence for the collections. If order is not specified the default behavior is the order of the parameter collections

Note

Example for generating labels in default order based on the collection index. Order passed into the function is Collection1, Collection2

label://lookup

  • Collection1 contains the following results {“metric_1”: “100”, “metric_2”: “200”}

  • Collection2 contains the following results {“metric_3”: “300”}

The following labels would be produced:

  • {“metric_1”: “metric_1”, “metric2”: metric2”, “metric3”: metric3”}

Note

Example for generating labels only for Collection1 based on the collection index

label://lookup?order=Collection1

  • Collection1 contains the following results {“metric_1”: “100”, “metric_2”: “200”}

  • Collection2 contains the following results {“metric_3”: “300”}

The following labels would be produced:

  • {“metric_1”: “metric_1”, “metric2”: metric2”}

Parameters:

  • collections (List[Collection]) – List of a collections that allows for a standard interface across Dynamic Application during collection

  • app (object) – The app comes from the EM7 Dynamic Application

  • toggle_limit_indices (bool) – Indicator of feature usage of reduction of indices. If True, the feature is ON and the number of indices will be reviewed and reduced if they exceed the limit. This will imply an index reduction if the amount exceeds the expected.

  • limit_indices_count (int) – The maximum number of indices that will be stored in the database, default value = 50. If the feature of indices reduction is ON, the number of indices returned will be no larger than the provided value.

Return type:

None