Journal Dynamic Applications

This section describes how to use journal Dynamic Applications, which collect and store data in log format.

Use the following menu options to navigate the SL1 user interface:

To view a pop-out list of menu options, click the menu icon ().
To view a page containing all the menu options, click the Advanced menu icon ().

This section includes the following topics:

What is a Journal Dynamic Application?

A journal Dynamic Application collects and stores data in log format. Collected data is stored as a series of journal entries, each entry representing a "log". For example, a journal Dynamic Application might collect:

Telephone call records, where each journal entry represents a single call.
System access records, where each journal entry represents a user session.

Journal entries are not static and can change state over multiple collection periods. For example, using the case of telephone call records, a single journal entry might be collected in the following way:

At collection period A, the snippet performs collection and determines that the call has started, but is not yet finished. A journal entry is created with a state of "open", and collection objects that are available, such as the start time, are stored.
At collection period B, the snippet performs collection and determines that the call is still ongoing but additional information about the call is now available. The additional collection objects that are now available are stored in the journal entry that was created during collection period A.
At collection period C, the snippet performs collection and determines that the call has completed. The snippet changes the journal entry that was created during collection period A, changing the state to "closed". Additional collection objects that are now available, such as the end time, are stored.

Each journal entry has the following information associated with it:

Key. A unique key that identifies the journal entry in the list of all journal entries for the Dynamic Application.
State. The current state of the journal entry. A journal entry can be in one of the following states:

Open
Closed
Error
Abandoned
Reopened

Collected Data. Each journal entry can store a value for each collection object defined in the Dynamic Application.
Meta data. A snippet can store additional data about each journal entry that is not displayed in the user interface. Metadata can be any type of information that needs to be preserved between collection periods. For example, you might store information about how the journal entry must be collected during future collection periods.

In the user interface, data for each journal Dynamic Application is displayed in tabular format, where each row displays information about a journal entry. Two default columns are always displayed:

State. The current state of the journal entry.
Collected On. The last time the Dynamic Application updated the journal entry.

All other columns are defined in the Dynamic Application as presentation objects. Each presentation object is a string, into which collected data can be substituted.

SL1 provides a framework that includes multiple variables and functions to allow you to create, modify, and track journal entries in your snippet code. This chapter describes all of the tasks journal Dynamic Application snippets must perform, the variables and functions that can be used to accomplish those tasks, and how to define collection objects and presentation objects for journal Dynamic Application snippets.

The inputs and outputs for journal snippets—for example, credential information and collection object dictionaries—are different from those used by configuration snippets and performance snippets. These differences provide a more robust platform to develop against, where each snippet is provided with its own copy of variables. The change in inputs and outputs also simplifies the interface between a snippet and SL1.

Journal Collection Objects

Collection objects for journal Dynamic Applications are data fields that can be associated with a journal entry. For example, using the case of telephone call records, the collection objects might be:

Dialing Party
Receiving Party
Start Time
End Time

Each collection object can be included in one or more presentation objects to define how the data for each journal entry will be displayed.

Unlike performance and configuration Dynamic Applications, journal collection objects can take only a single value, not a list of values with different indexes. For collection objects in journal Dynamic Applications that return multiple values for a single journal entry (for example, "Call Participants" for telephone call records), you must store this information in a JSON Array, and store it in a "Journal JSON Array" collection object.

To create a collection object for a journal Dynamic Application, perform the following steps:

Go to the Dynamic Applications Manager page (System > Manage > Applications).
Select the wrench icon () for the journal Dynamic Application to which you want to add a collection object. The Dynamic Applications Properties Editor page is displayed.
Select the Collections tab. The Dynamic Applications | Collections Objects page is displayed:

Supply values in the following fields:

Object Name. The name of the collection object.
Variable Name. The name of the collection object as it will appear to the snippet code. This field is analogous to the snippet argument used by configuration and performance Snippet Dynamic Applications.
Class Type. The type of data stored in this collection object. Choices are:

Journal Character. Object that contains information in text format.
Journal Enum. Object that contains a value in enumerated format. Enumerated values associate a text-based label with a numeric value. For example "up(1)", "down(2)", and "testing(3)".
Journal Timeticks. Object whose value represents elapsed time.
Journal Gauge. Object that contains a numeric value. This value can increase and decrease.
Journal Numeric. Object that contains a value in numeric format.
Journal Time Stamp. Object that contains a UNIX timestamp.
Journal MAC Address. Object that contains a MAC address.
Journal IP Address. Object that contains an IP address.
Journal Date & Time. Object that contains a formatted date & time stamp.

For collection objects that store time values, SL1 performs no special processing on the collected values. However, some presentation formula options convert UNIX timestamp values to human-readable strings.

Journal JSON Object. Object that contains JSON object with multiple data values.
Journal JSON Array. Object that contains an array of values in JSON format.
Discovery. For information on discovery objects in journal Snippet Dynamic Applications, see the Journal Discovery Objects section.

Snippet. The snippet responsible for collecting this collection object.
Description. A description of the collection object.
Enum Values. This field is displayed for collection objects with a Class Type of Journal Enum. In this field you can map each possible collected value to a descriptive string that will be displayed in SL1. Use the following format:

#number:value#number:value#number:value

Where number is the collected value and value is the descriptive string. You must include a "#" as the first character in this field and as a separator between number:value pairs.

For example, suppose a Journal Enum object returns an integer that represents a status condition. The possible returned values and the conditions they represent might be:

3 = healthy

4 = minor

5 = major

6 = critical

You would supply the following string in the Enum Values field:

#3:healthy#4:minor# 5:major#6:critical

Unlike collection objects for configuration snippets or performance snippets, journal objects do not have group or index settings. Journal collection objects are grouped for display by the journal entry they are associated with, and indexed by the unique key assigned to the journal entry they are associated with.

Select the Save button.

Journal Snippet Code

The snippet code for a journal Dynamic Application must:

Collect and populate new journal entries.
Re-process any open journal entries, updating and/or closing them if appropriate.
Report all new and updated journal entries for the current poll.
Report the status of the current poll, indicating whether the poll was successful or encountered a problem.

The following sections describe how to use the variables and functions that are provided by SL1 to accomplish these tasks.

Available Local Variables

The following local variables are available to use in your snippet code:

Unlike snippets for performance and configuration Dynamic Applications, the local variables for journal snippets are not referenced using self.

logger. An instance of a ScienceLogic logger object. This object allows you to write debug output to the silo.log file. Debug output is written to silo.log only when debug mode is enabled for the dynamic_collect process. To write a debug message to silo.log, use the following line of code:

logger.debug("Message to write to silo.log")

app_id. The ID number for this Dynamic Application.
did. The device ID for the device that this instance of the snippet is currently performing collection on.
cred_details. A dictionary of values for the credential aligned with this Dynamic Application. The dictionary has the same structure as the self.cred_details dictionary described in the Performance and Configuration Snippets section.
collection_objects. A dictionary that contains information about the collection objects this snippet is responsible for collecting. The dictionary keys are the values specified in the Variable Name field for each collection object this snippet is responsible for collecting. The values associated with each Variable Name are initialized to "None". The structure of the collection_objects dictionary is the same structure that must be passed to the update_collected_data() function. collection_objects has the following structure:

{Variable Name: None, Variable Name: None, .... , Variable Name: None}

Creating & Populating a Journal Entry

Information about journal entries are stored in JournalEntry objects. Your snippet must create and populate a JournalEntry object for each newly collected journal entry. To create a journal entry, you must use the following global function:

create_entry(key, state). Returns a new JournalEntry object with the specified unique key and the specified state.

The following global constants must be used when referring to a journal state:

JOURNAL_STATE_OPEN
JOURNAL_STATE_CLOSED
JOURNAL_STATE_ERROR
JOURNAL_STATE_REOPENED
JOURNAL_STATE_ABANDONED

The entry_key attribute of a JournalEntry object can be used to access the unique key.

JournalEntry objects have the following methods:

get_state(). Returns the current state of the JournalEntry object.
is_open(). Returns TRUE if the current state of the JournalEntry is "open" or "reopened", returns FALSE otherwise.
is_closed(). Returns TRUE if the current state of the JournalEntry is "closed", "error", or "abandoned"; returns FALSE otherwise.
set_state(state). Sets the state of the journal entry to the value of the state parameter.
close(). Sets the state of the journal entry to closed.
abandon(). Sets the state of the journal entry to abandoned.
set_error(error_str). Set the error string for the journal entry.
update_collected_data(collected_data). Set one or more of the collection objects associated with this JournalEntry object to the values in the collected_data parameter. The collected_data parameter must be a dictionary with the following structure:

{Variable Name: Collected Value, Variable Name: Collected Value, .... , Variable Name: Collected Value}

This structure is the same as the structure of the collection_objects dictionary described in the Available Local Variables section.

set_meta_data(meta_data). Set the metadata for this JournalEntry object to the value of the meta_data parameter. You can pass any object that can be pickled by python as the meta_data parameter.
get_meta_data(). Returns the metadata for this JournalEntry object. The returned metadata will be identical to the object passed to the set_meta_data function for this JournalEntry object.

Retrieving Existing Journal Entries

Journal entries are not static, and can change state over multiple collection periods. In addition to creating new JournalEntry objects, your snippet code must also re-process already collected JournalEntry objects that have not been closed. Re-processing existing JournalEntry objects might include updating collection objects with new data and changing the state of the JournalEntry. Eventually, your snippet will close each JournalEntry object when appropriate.

The following global functions are available to access existing JournalEntry objects:

bulk_get_entries(states, keys). Returns a list of JournalEntry objects that are in one of the states specified in the states parameter and have a key that matches one of the keys specified in the keys parameter. The states parameter must be a list of states. The keys parameter must be a list of JournalEntry keys.
bulk_get_open_entries(keys). Returns a list of JournalEntry objects that are in an open state, and optionally have a key that matches one of the keys specified in the keys parameter. The keys parameter is optional. If specified, the keys parameter must be a list of JournalEntry keys.
get_entry(key). Returns the JournalEntry object that matches the key specified in the key parameter.

To modify an existing JournalEntry object you have retrieved, you can use any of the JournalEntry object methods listed in the Creating & Populating a Journal Entry section of this chapter.

Returning New & Updated Journal Entries

After your snippet has processed all new and updated JournalEntry objects, they must be added to the EM7_RESULT list. When your snippet has finished executing, SL1 processes the EM7_RESULT list for storage in the main database. For example, suppose that you have a JournalEntry object called entry. To get SL1 to store the new or updated data you have added to entry, you would use the following line of python code:

EM7_RESULT.append(entry)

To reduce the load on your SL1 System, you should not return JournalEntry objects that have no new or changed data. Continually storing the same JournalEntry object will reduce the performance of your SL1 System.

Reporting Collection Status

Unlike other Dynamic Application types, a successful collection for a journal Dynamic Application might return no new or updated data. Because SL1 cannot determine whether collection was successful based on the presence of new or updated data, your snippet code must report when collection was successful. The following local variables are defined for this purpose:

COLLECTION_PROBLEM. When your snippet code completes, you can set this variable to True to indicate that there was a problem with collection (i.e. indicating a missed poll). When you snippet code completes, you can set this variable to False to indicate that collection was successful. This variable is set to True at the start of each collection; you must set this variable to False to report a successful collection.
PROBLEM_STR. This variable can be used with the COLLECTION_PROBLEM variable, to provide a description of the problem with collection. If you set the COLLECTION_PROBLEM variable to false (indicating a missed poll), you can additionally specify why the collection failed by assigning a string value to the PROBLEM_STR variable. SL1 will use the string value of PROBLEM_STR to generate an alert.

Storing & Retrieving General Metadata

In addition to being able to store and retrieve metadata for each JournalEntry object, two functions are available to store and retrieve general metadata about collection:

get_app_instance_meta_data()

set_app_instance_meta_data(meta_data)

These functions are provided so that information can be passed from one instance of collection to the next instance of collection. For example, you might use these functions to track the point in a given log file at which the next collection should resume.

SL1 creates one entry in the collector database for each device the Dynamic Application is aligned with. For example, if a Dynamic Application that stores metadata is aligned to two devices, metadata for each device is stored separately. If a device is realigned to a different collector, the metadata is transferred to the new collector.

You can pass any object that can be pickled by python to the set_app_instance_meta_data(meta_data) function. SL1 will pickle the object for storage, and unpickle the object before returning the object when get_app_instance_meta_data() is called. If an exception occurs in pickling or unpickling, your snippet code must handle the exception. The following exception globals are provided for handling exceptions when using the metadata functions:

class SerializationError

class DeserializationError

Journal Discovery Objects

If you create a discovery object for a Journal Dynamic Application, the discovery object must be aligned with a snippet that collects only discovery objects. For journal snippets for discovery objects, SL1 expects the EM7_RESULT output to be a dictionary of discovery collection objects with the following structure:

{Variable Name: Discovery Object Value, Variable Name: Discovery Object Value, .... , Variable Name: Discovery Object Value}

When SL1 tests for the "presence" alignment condition (that is, when SL1 has found a discovery object on a device and the Dynamic Application should be aligned to the device), the result is "true" only if the variable name for the discovery object is a key in the EM7_RESULT dictionary.

For example, suppose a journal Dynamic Application has two discovery objects aligned to the same discovery snippet:

Variable name "DiscoveryOne" has Alignment Condition "Align if OID is present" with no Validity Check.
Variable name "DiscoveryTwo" has Alignment Condition "Align if OID is NOT present".

At the end of execution for the discovery snippet, if EM7_RESULT is set to the following dictionary:

{DiscoveryOne: 'Returned'}

The Dynamic Application will be aligned to the device, because DiscoveryOne is returned and DiscoveryTwo is not returned.

At the end of execution for the discovery snippet, if EM7_RESULT is set to the following dictionary:

{DiscoveryOne: 'Returned', DiscoveryTwo: None}

The Dynamic Application will not be aligned to the device, because DiscoveryTwo is returned, even though its value is None.

When defining a snippet that collects discovery objects, in addition to EM7_RESULT, you can use any of the variables listed in the Available Local Variables section of this chapter.

Journal snippets that collect discovery objects do not have access to any methods, constants, or variables listed in the Creating & Populating a Journal Entry, Retrieving Existing Journal Entries, Reporting Collection Status, or Storing & Retrieving General Metadata sections of this chapter. If you reference a method, constant, or variable listed in these four sections in a snippet that collects discovery objects, a traceback will occur when the snippet is executed.

Journal Presentation Objects

Journal Dynamic Applications display collected data on the Journals tab in the Device Reports panel. The Journal tab displays a table of collected data for each journal Dynamic Application. When you define the journal Dynamic Application, you must define presentation objects that populate the table in the Journal tab. Each presentation object is displayed as a column in the Journal table. Each journal entry is displayed as a row in the table.

You must define a formula for each presentation object. This formula tells SL1 which collection object(s) to use to populate the presentation object. The presentation formula is applied to each journal entry, and the resulting value is displayed in the column for that presentation object.

For example, suppose you had a collection object called o_1234 that collects the start time of a phone call. You could create a presentation object called "Start Time". The formula for the presentation object would be "o_1234". The Journal View page will display a column called "Start Time", and for each journal entry row, the column will contain the value for o_1234 stored for that journal entry.

In the definition for a journal presentation object, you can define how the column will be displayed in the table, including the column width, the order in which the columns are displayed, and whether a column can be used to sort the table of journal entries.

To create a presentation object for a journal Dynamic Application, perform the following steps:

Go to the Dynamic Applications Manager page (System > Manage > Dynamic Applications).
Select the wrench icon () for the journal Dynamic Application you want to add a presentation object to. The Dynamic Applications Properties Editor page is displayed.
Select the Presentations tab. The Dynamic Applications Presentation Objects page is displayed:

Supply values in the following fields:

Report Name. The name of the presentation object. This name will be displayed as a column heading in the table of journal entries in the Journal View page.
Active State. Specifies whether SL1 should display this presentation object in the Journal View page. Choices are:

Enabled. SL1 will display this presentation object in the Journal View page.
Disabled. SL1 will not display this presentation object in the Journal View page.

Column Order. Specifies the order in which the columns for each presentation object will be displayed in the Journal View page. Columns are displayed in ascending Column Order, from left to right. Enter a numeric value in this field.
Column Width Type. Specifies whether SL1 should display the column with a width relative to the other columns or using a fixed width in pixels. Choices are:

Relative width (x). SL1 will calculate the width of this column based on the width of the other reports in the Dynamic Application.
Absolute width (px). SL1 will display this column using a fixed column width, in pixels.

Column Width. Specifies the relative or absolute width of this column. Enter a numeric value in this field. If you selected Relative width (x) in the Column Width Type drop-down list, enter the width of this column relative to the other reports in this Dynamic Application. For example, a column with a relative Column Width of 2 will be twice as wide as a column with a relative Column Width of 1. If you selected Absolute width (px) in the Column Width Type drop-down list, enter the width of this column in pixels.
Column data type. Specifies the data type of the values displayed in this column. This data type is used by SL1 to determine how the column can be filtered and sorted. For example, suppose there are three rows in the table of journal entries, and the values for a column are "1", "2", and "10". If the Column data type is set to String template, and the list is sorted in ascending order by the column, the order will be: "1", "10", "2". If the Column data type is set to Integer, and the list is sorted in ascending order by the column, the order will be: "1", "2", "10". Choices are:

String template. The values in the column are text strings. For this data type, the Display formula can contain multiple collection objects and additional text that will be displayed for every journal entry.
Integer. The values in the column are integers. For this data type, the Display formula can contain only one collection object.
Floating-point. The values in the column are floating-point numbers. For this data type, the Display formula can contain only one collection object.
Unix timestamp. The collected values are converted from UNIX timestamp format to a human-readable timestamp. Use this option only if the collection object is a UTC UNIX timestamp. When the data is presented to a user, the timestamp will be formatted according to that user's preferences and in their timezone. For this data type, the Display formula can contain only one collection object.
Date and time. The collected values are converted from UNIX timestamp format to a human-readable timestamp. Use this option only if the collection object is a UTC UNIX timestamp. When the data is presented to a user, the timestamp will be formatted according to that user's preferences and in their timezone. For this data type, the Display formula can contain only one collection object.

Indexing and sorting. Specifies whether the table of journal entries can be sorted by the values in this column. Choices are:

Disabled (column cannot be sorted). The table of journal entries can be sorted by the values in this report.
Enabled (column can be sorted according to data type). The table of journal entries cannot be sorted by the values in this report.

Display formula. This field allows you to define the value that will be displayed in each row of the Journal View for this column. When a collection object is referenced in a journal presentation object, the values for that collection object are inserted in to the Display Formula when the Journal View is displayed.

The scrolling list below the Formula Editor contains a list of all objects in the Dynamic Application. To add an object to the Display Formula, highlight the object in the scrolling list, then select the Add button.

Select the Save button.