silo.low_code.log_policy.log_policy module

LogPolicies allow for targeted debugging while executing. This enables less chatter during production runs to help troubleshoot / diagnose any issues.

There are a few main pieces to log policies:

  • PolicyManager

  • ConfiguredPolicy

  • PolicySearch

  • PolicyFilter

PolicyManager’s job is to store or retrieve data from the database.

ConfiguredPolicy’s define what is being targeted for logging. This class enables a few conditions for determining if the current state matches. It helps determine if two ConfiguredPolicies are equal one-way. Note This class is NOT py3.9+ compatible due to the one-way equality.

PolicySearch enables searching through a list of policies to determine if there is a match. This is used during the spinup of the Snippet Framework to determine which policies match to reduce the overall filter-load while logging. Note This probably should live in silo.low_code.framework.framework_logging instead of this module.

PolicyFilter compares the current message context to the configured policy to see if there is a match. Note This probably should live in silo.low_code.framework.framework_logging instead of this module.

class silo.low_code.log_policy.log_policy.ConfiguredPolicy(name, app_id, cobj_id, did, action_number=None, duration=None, step=None)

Bases: object

A configured logging policy

This represents a logging policy that a user has configured. It can be used to compare two policies together to see if the required entities match. For the policies to be the equal, the policy that is being checked must have the same values. Only the attributes with values will be compared.

Variables

  • name (str) – Name of the policy

  • app_id (int) – Dynamic Application ID

  • cobj_id (int) – Dynamic Application’s Collection Object ID

  • did (int) – Device ID

  • action_number (int) – Action number of the Execution

  • duration (str) – policy duration before to be expired in string format following the format ‘1w, 1d, 1h, 1m, 1s’

  • step (str) – Name of the step

CORE_DATA_POINTS = ['app_id', 'cobj_id', 'did']
get_filter_key()

Generates the filter key used for key-based lookups

get_filter_key_core()

Generates the filter key used for key-based lookups

policy_contained_within(policies)

Determine if the current policy is contained with a list

This function exists due to PEP-590 switching list comprehension from a == b to b == a which broke the partial check of __eq__.

Parameters

policies (list) – List of ConfiguredPolicies

Return type

bool

class silo.low_code.log_policy.log_policy.PolicyFilter(unique_id, policy)

Bases: Filter

filter(record)

Determine if the specified record is to be logged.

Returns True if the record should be logged, or False otherwise. If deemed appropriate, the record may be modified in-place.

class silo.low_code.log_policy.log_policy.PolicyManager(logger, include_expired=True)

Bases: object

Enables easy interaction with Snippet Framework logging policies

Parameters

  • logger (logger) – Logger to use for logging

  • include_expired (bool) – Include expired log policies when gathering data

calculate_ttl(time_string_format)

Sum of times to have the expiration time.

Parameters

time_string_format (str) – expiration time in format ‘1h, 2m, …’.

Returns

time parsed to seconds.

Return type

int

delete_policy(policy_name)

Delete a policy from the cache.

Parameters

policy_name (str) – policy name which will be used to delete the entry from the cache.

filter_data(time_identifier, time_string_list)

Filter string list based on time_identifier.

Having ‘d’ or ‘h’ as time identifier, filter just those that matches with time identifier.

Parameters

  • time_identifier (str) – Regex that will be used to filter values

  • time_string_list (list) – times string list.

Returns

all elements that matches with the time identifier.

Return type

list

get_filter_policies(return_expiry=True)

Retrieve policies from cache.

Parameters

return_expiry (bool) – flag to let the cache get expiration time in the result.

Returns

list filter policies

Return type

list

get_filter_policy(policy_name, return_expiry=False)

Retrieve a policy by name.

Parameters

  • policy_name (str) – name of the policy to search

  • return_expiry (bool) – flag to let the cache get expiration time in the result.

Returns

a single policy

Return type

list

make_duration_time_human_readable(duration_time)
Parse duration time to human readable format taking

into account the singular, plural pronouns for each time units based on the value.

Parameters

duration_time (str) – duration time in short format.

Returns

parsed time un human readable format.

Return type

str

parse_time(time_string)

Time interpreter and parser.

Having “1w, 2d, 3h, 4m, 5s” as time_string, the function will return a dictionary with numbers in the values. The time should be separated by a comma. It is not necessary to have value for all times, you can give just minutes, seconds, or whatever you need. If the time have ‘2d, 2d’, two values for one identifier, it’s going to return the sum of those values in the corresponding key.

Parameters

time_string (str) – expiration time in string format, the time can be one or all of the options.

Returns

weeks, days, hours, minutes and seconds value in a OrderedDict.

Return type

OrderedDict

save_policy(policy)

Store a policy in the cache.

policy = {
    "name": "some name",
    "duration": "1w, 1d, 1h, 1m, 1s",
    "did": "10",
    "app_id": "20",
    "cobj_id": "30"
}

The name field shouldn’t be empty, the other can be empty values, which means that your policy filter will filter all values for that field.

The duration field can use one, two, or more time identifiers, the order does not matter, you should separate by comma the different times.

Parameters

policy (dict) – data with policy configuration.

validate(policy)

Validate a policy values are correct and provided information are part of system steps, devices, collection objects and applications.

Parameters

policy (dict) – Policy values.

Returns

dict containing policy values

Return type

dict

verify_data_in_sys(value, get_data_action, column)

Validate element existence on the database.

Parameters

  • value (str) – value to verify on the db result.

  • get_data_action (callabe) – function to get data from the DB.

  • column (str) – column where should be the value.

Returns

value getting it from DB row, should be the same as value input if the value is empty, there is nothing to see in the DB.

Return type

str

Raises

  • KeyError – if the column does not exist on the result.

  • IndexError – if the result is not on the system DB.

class silo.low_code.log_policy.log_policy.PolicySearch(collections)

Bases: object

Given a list of Collection objects, it will create the logic needed to determine if a policy matches.

__init__(collections)

Create a FilterSearch instance.

Parameters

collections (list) – List of Collection objects.

silo.low_code.log_policy.log_policy.cleanup_expired_policies(policies)

Remove expired policies

Remove any expired policies from the database. Due to the nature of threading, we want to use a mutex to ensure only one thread performs the removal of expired policies. This prevents duplicate messages from appearing in the System Messages. If the thread is unable to get the lock on the file, it will skip the removal phase and proceed with collection.

Parameters

policies (list) – List of expired ConfiguredPolicy

silo.low_code.log_policy.log_policy.get_app_header(app_id, did, cobj_id, step)

Create a header for a AppLogAdapter.

Parameters

  • app_id (int) – Application ID

  • did (int) – Device ID

  • cobj_id (int) – Collection Object ID

  • step (str) – Name of the current step

silo.low_code.log_policy.log_policy.get_filtered_policies(policies, collections)

Determine which policies apply to the collections

Parameters

  • policies (list) – List of active policies

  • collections (list) – List of collections for this iteration

Returns

List of policies that apply to this iteration

Return type

list

silo.low_code.log_policy.log_policy.process_logging_policies(policies)

Processes the policies into non-expired and expired

Parameters

policies (list) – List of configured policies

Return type

ConfiguredPolicies