Indexing

Download this manual as a PDF file

This section describes Dynamic Application Indexing.

Use the following menu options to navigate the SL1 user interface:

  • To view a pop-out list of menu options, click the menu icon ().
  • To view a page containing all of the menu options, click the Advanced menu icon ().

What is Dynamic Application Indexing?

When SL1 performs collection for a collection object, the response might contain a list of values for that collection object instead of a single value.

For example, suppose a collection object collects the amount of space used in a file system. For that single collection object, a device might respond with a list of values. Each value in the list is the amount of space used for each of the file systems on that device.

When a collection object returns multiple values, SL1 uses Indexing to track each of the values. Each of the values in the list is given an index, which is a numerical value that allows SL1 to:

  • Maintain associations between collection objects in the same Dynamic Application. For example, suppose a collection object collects the amount of space used in a file system and a device responds with multiple values. Suppose a second collection object returns the names of those file systems. The lists of values for each collection object should be assigned the same index, so that the values for space used are associated with the correct file system name in graphs and reports. The first value in the list of amount of space used values is associated with the first value in the list of names, etc.
  • Maintain an association between values collected at different times for the same collection object. Every time SL1 collects a list of values for a collection object, each value that is associated with a value in the previously collected list should have the same index value as the previous value. For example, suppose a collection object collects the amount of space used in a file system and a device responds with multiple values. Suppose that during the first collection, the amount of space used for the file system called "C:\" is assigned index "1". The next time SL1 collects the list of values, the amount of space used for the file system called "C:\" should again be associated with index "1". When SL1 generates a graph of data, each line on the graph is drawn using values with the same index.

Index values are unique and maintain association only for each group of collection objects in each Dynamic Application for each device. For example:

  • In a single Dynamic Application, a collection object in group 1 can have the same index values as a collection object in group 2, but the collected values for these collection objects are not associated with each other.

  • In a single Dynamic Application, for each group of collection objects, SL1 maintains separate sets of indexes for each aligned device. A collection object in group 1 might have different index values for device A and for device B.

SL1 has a default method to assign index values. For some Dynamic Applications, the default method is not sufficient and might create incorrect associations between values. Typically, this is caused when the list of values for a collection object changes order from one collection to the next. If the default indexing method is not sufficient, you can designate a collection object that will control the indexing behavior.

Index values are internal to SL1 and are typically not displayed to users. SL1 displays index values as the labels for performance graphs only when collected labels are not available.

How Indexing Affects Configuration Tables

For each group of collection objects defined in a configuration Dynamic Application, SL1 displays a table in the configuration report for that Dynamic Application:

  • Each collection object is assigned its own column in the table.
  • Each index is assigned its own row in the table.

If a given collection object does not have a collected value for an index, an empty field will be displayed for that row and column.

How Indexing Affects Performance Graphs

For each presentation object defined in a Dynamic Application, SL1 generates a graph that includes one line for each index.

For example, suppose a presentation formula in a Dynamic Application includes a single collection object:

o_123

Suppose that when this Dynamic Application is aligned to a device, SL1 collects a list of three values for o_123.

  • The graph for this presentation object will display three lines, one for each index.
  • When new data is collected, the new poll time will be included on the x-axis of the graph.
  • For each of the three lines, the data point (y-axis) for the new poll time will be the collected value for that index.

For example, suppose SL1 assigns indexes 0, 1, and 2 to the lists of collected values for o_123. Suppose SL1 then performs three polls at 00:00, 00:05, and 00:10. Suppose the indexed collected values look like this:

Index 00:00 00:05 00:10
0 6 8 7
1 15 30 20
2 50 60 55

When SL1 generates the performance graph:

  • One graph line will include the values 6, 8, and 7.
  • One graph line will include the values 15, 30, and 20.
  • One graph line will include the values 50, 60, and 55.

Now suppose that during the 00:15 poll, a list of three values is returned for the collection object, but the values are assigned the indexes 1, 2, and 3:

Index 00:00 00:05 00:10 00:15
0 6 8 7 -
1 15 30 20 9
2 50 60 55 25
3 - - - 70

The graph will now include four lines:

  • One graph line will include the values 6, 8, and 7. This line will not include a data point for the 00:15 poll.
  • One graph line will include the values 15, 30, 20, and 9.
  • One graph line will include the values 50, 60, 55, and 25.
  • One graph line will not include data points for the 00:00, 00:05, and 00:10 polls, and will include the value 70 for the 00:15 poll.

For presentation formulas that contain multiple collection objects, SL1 evaluates the formula for the values at each index. A graph line is drawn for each index for which there are enough collected values to calculate a result. For example, suppose a presentation object has the following formula:

o_123 + o_321

If SL1 collects the following lists of values for o_123 and o_321 and assigns the following index values:

Index o_123 o_321
0 100 -
1 1000 750
2 100 30
3 - 80

SL1 will evaluate the presentation formula for each of the four indexes. However, there are only enough collected values to calculate a result for two indexes (1 and 2). Therefore, two graph lines will appear on the graph.

Suppose the presentation formula was changed to make o_123 optional:

o_123 + {o_321}

For the same set of collected values, the graph would include three graph lines, which would correspond to indexes 0, 1, and 2. Index 3 would not be included, because object o_123 is not optional and does not include a value at index 3.

The indexes assigned to collected values are also used to label the lines in the graph key that appears in the lower right of the Performance page. When a graph line is drawn for an index, SL1 looks for a collected value that matches the following criteria:

  • Was collected for a collection object in the Dynamic Application that has a Class Type of 101 Label (Hourly Polled) or 104 Label (Always Polled).
  • Has the same group value as the collection objects that were used to draw the graph line.
  • Has the same index value as the values used to draw the graph line.

If SL1 finds a collected value that matches the criteria, that value is displayed in the graph key for that graph line. If SL1 does not find a collected value that matches the criteria, the graph line is labeled "Index X", where X is the assigned index for the values used to generate the graph line.

The poll time for the collected value is not considered when SL1 chooses a label for a graph line. For collection objects that have a Class Type of 101 Label (Hourly Polled) or 104 Label (Always Polled), SL1 stores only the last collected value.

How Indexing Affects Alerts

When SL1 evaluates an alert, the formula is evaluated separately for each index assigned to the collected values.

For example, suppose an alert formula in a Dynamic Application uses one collection object:

o_123 > 0

Suppose that when this Dynamic Application is aligned to a device, SL1 collects a list of three values for o_123. SL1 will evaluate the alert formula three times, once for each collected value.

Suppose another alert formula uses two collection objects:

(o_123 / 100) * o_321 > 90

Suppose SL1 collects the following lists of values for o_123 and o_321 and assigns the following index values:

Index o_123 o_321
0 100 80
1 1000 750
2 100 30

During each poll period, SL1 will evaluate the alert formula three times, once for each pair of values for each index.

Now suppose that for a different device, SL1 collects a list of three values for each collection object, but the indexes assigned to the collected values do not match:

Index o_123 o_321
0 200 -
1 300 160
2 100 30
3 - 80

For this set of collected values, the alert formula will be evaluated four times, once for each index assigned to the collected values. However, the evaluation for indexes 0 and 3 will fail because not all the collection objects have a value for that index.

The Default Indexing Method

For each group of collection objects in each Dynamic Application associated with a single device, SL1 maintains a list of index values. When a list of values is collected for a collection object, SL1 uses the following information about each value in the list to determine the index for that value:

  • The ID for the device the data was collected from.
  • The ID for the Dynamic Application the data was collected using.
  • The ID for the group the collection object is included in.
  • Where in the list of values the value appears, called the instance.

Each index in the list of indexes maintained by SL1 has the same four attributes (device ID, Dynamic Application ID, group ID, and instance). During collection, SL1 attempts to match each value in the list of collected values with an index in the list of indexes. If a match is found, i.e. the device ID, Dynamic Application ID, group ID, and instance are all identical, SL1 associates the matching index with the collected value. If a match is not found, SL1 creates a new record in the list of indexes for that device ID, Dynamic Application ID, group ID, and instance and assigns a new index value.

Instance Values and Index Creation for SNMP Dynamic Applications

For collection objects in Dynamic Applications that use the SNMP protocol, the instance of a value in a list of collected values is the SNMP index associated with the value. For example, suppose the SNMP OID used in a collection object is ".1.3.6.1.2.1.25.2.3.1.6". This OID represents the amount of storage space used (as reported by the Host Resources MIB). Suppose a device responds to this OID in the following way:

HOST-RESOURCES-MIB::hrStorageUsed.1 = INTEGER: 6097420

HOST-RESOURCES-MIB::hrStorageUsed.3 = INTEGER: 6097420

HOST-RESOURCES-MIB::hrStorageUsed.6 = INTEGER: 472112

HOST-RESOURCES-MIB::hrStorageUsed.7 = INTEGER: 1984116

The SNMP indexes for the returned values are ".1", ".3", ".6", and ".7". These SNMP indexes are used as the instance for each value when SL1 assigns an index.

If SL1 needs to create a new index record for an SNMP collection object, that is, the device ID, Dynamic Application ID, group ID, and instance do not match an existing index record, SL1 will first attempt to use the numeric equivalent of the instance as the index. In the example above, SL1 will attempt to create indexes 1, 3, 6, and 7.

If SL1 tries to add an index record and determines that the index already exists for the same device ID, Dynamic Application ID, and group ID, SL1 will create the new record with an index equal to one more than the current highest index value for that device ID, Dynamic Application ID, and group ID.

In the example above, suppose that SL1 has already created index records for an instance other than ".1", ".3", ".6", and ".7" for the same device ID, Dynamic Application ID, and group ID. Suppose that it is the only instance that has an existing index record. Suppose that instance has been assigned index 7. When SL1 attempts to create index records for the four instances in the example above, index records for instances ".1", ".3", and ".6" are successfully created with indexes 1, 3, and 6. Because index 7 is already taken for this device ID, Dynamic Application ID, and group ID, the index record for instance ".7" is created with index 8, which is one more than the current highest index value (7).

Instance Values and Index Creation for Snippet Dynamic Applications

For collection objects in Snippet Dynamic Applications, the instance of a value in a list of collected values is defined by the snippet code that collects that collection object. Collection objects are passed back to SL1 by snippet code in a tuple. The first value in the tuple is the instance, the second value in the tuple is the collected value.

SL1 creates index records for Snippet Dynamic Application collection objects with a method similar to that used for SNMP Dynamic Applications:

  • If SL1 needs to create a new index record for a snippet collection object, SL1 will first attempt to use the value of the instance as the value of the index.
  • If SL1 tries to add a record for an index value that already exists for the same device ID, Dynamic Application ID, and group ID, SL1 will create the new record with an index equal to one more than the current highest index value for that device ID, Dynamic Application ID, and group ID.

Instance Values and Index Creation for Other Dynamic Applications

For collection objects in Dynamic Applications that use a protocol of Database, SOAP, WMI, XML, and XSLT, the instance of a value in a list of collected values is the location at which that value appears in the response. The first value in the list is instance "0", the second value in the list is instance "1", and so on.

For example, suppose a collection object for a Database Dynamic Application runs a query that selects the column "speed" from a database table. Suppose the following list of data is returned:

speed
1000
500
1200
700

The instance values would be:

speed

instance

1000 0
500 1
1200 2
700 3

To create a new index record for a Database, SOAP, WMI, XML, or XSLT collection object, SL1 will create the new record with an index equal to one more than the current highest index value for that device ID, Dynamic Application ID, and group ID. The first created index for a device ID, Dynamic Application ID, and group ID is 0.

Designating an Index

In cases where the order or size of the list of values for a collection object might change, the default method of assigning indexes is not sufficient to maintain the associations between collected values.

For example, consider the initial instance values that were assigned to the list of "speed" values in the previous example:

speed instance
1000 0
500 1
1200 2
700 3

Now suppose that the list of values changes order during the next poll:

speed
1000
500
700
1200

The "700" and "1200" values are now matched to different instance values:

speed instance
1000 0
500 1
700 2
1200 3

This means that when SL1 stores the new values "700" and "1200", they are associated with different index values than the previously stored data. If these values are used in a performance graph, the new value of "700" will appear in the graph line for index 2 (which has "1200" as the previous data points), and the new value of "1200" will appear in the graph line for index 3 (which has "700" as the previous data points). If these values are used in a configuration table, SL1 will detect a change from "700" to "1200" for index 2 and a change from "1200" to "700" for index 3.

When the default method of assigning indexes is not sufficient, you can designate a collection object in a group of collection objects as the index. When you designate a collection object as an index, SL1 still uses the same information about each value in the list to determine the index for that value:

  • The ID for the device the data was collected from.
  • The ID for the Dynamic Application the data was collected using.
  • The ID for the group the collection object is included in.
  • The instance associated with the value.

When you designate a collection object as an index, SL1 uses the value and location of the "index" collection object to define the instance. The collected values for the "index" collection object are recorded as the instances. For the collected values for the other collection objects in the same group, the instance is the collected value for the "index" collection object that appears at the same location in the list for that poll period.

For example, suppose that a group contains two collection objects:

  • A collection object that collects the storage space used as reported by the Host Resources MIB (SNMP OID .1.3.6.1.2.1.25.2.3.1.6)
  • A collection object that collects the storage description as reported by the Host Resources MIB (SNMP OID .1.3.6.1.2.1.25.2.3.1.3).

Suppose that the collection object that collects the storage description is designated as the index for the group. Suppose the first time this Dynamic Application collects data from "Device A", the device responds to the storage description OID in the following way:

HOST-RESOURCES-MIB::hrStorageDescr.1 = STRING: C:\

HOST-RESOURCES-MIB::hrStorageDescr.3 = STRING: D:\

HOST-RESOURCES-MIB::hrStorageDescr.6 = STRING: Virtual Memory

HOST-RESOURCES-MIB::hrStorageDescr.7 = STRING: Physical Memory

For this group of collection objects in this Dynamic Application for this device, SL1 will create the following record of instances to internal indexes:

instance index
C:\ 1
D:\ 3
Virtual Memory 6
Physical Memory 7

SL1 will still attempt to use the numeric equivalent of the SNMP index as the index.

For the same first poll of the device, suppose the device responds to the storage used OID in the following way:

HOST-RESOURCES-MIB::hrStorageUsed.1 = INTEGER: 6097420

HOST-RESOURCES-MIB::hrStorageUsed.3 = INTEGER: 0

HOST-RESOURCES-MIB::hrStorageUsed.6 = INTEGER: 24863

HOST-RESOURCES-MIB::hrStorageUsed.7 = INTEGER: 27257

To determine the internal index to associate with the value at SNMP index ".1", 6097420:

  • SL1 determines the value for the designated index (the storage description) at the same location (SNMP index ".1"). This value is "C:\".
  • SL1 uses "C:\" as the instance value to match against the record of internal indexes for this group, Dynamic Application, and device.
  • The instance value of "C:\" matches to internal index 1. When SL1 stores the value "6097420", SL1 associates the value with index 1.

SL1 repeats this process for the other values returned for the storage used OID. For the first poll of the device, the following table shows the values returned for the storage used OID, the instance value SL1 used, and the internal index that is associated with the stored values:

collected value instance index
6097420 C:\ 1
0 D:\ 3
24863 Virtual Memory 6
27257 Physical Memory 7

Every time SL1 uses this Dynamic Application to collect data from the device, SL1 determines the instance values using this method. Suppose that during the next poll, the device responds to the storage description OID in the following way:

HOST-RESOURCES-MIB::hrStorageDescr.1 = STRING: C:\

HOST-RESOURCES-MIB::hrStorageDescr.6 = STRING: Virtual Memory

HOST-RESOURCES-MIB::hrStorageDescr.7 = STRING: Physical Memory

HOST-RESOURCES-MIB::hrStorageDescr.8 = STRING: D:\

And responds to the storage used OID in the following way:

HOST-RESOURCES-MIB::hrStorageUsed.1 = INTEGER: 6097420

HOST-RESOURCES-MIB::hrStorageUsed.6 = INTEGER: 24863

HOST-RESOURCES-MIB::hrStorageUsed.7 = INTEGER: 27257

HOST-RESOURCES-MIB::hrStorageUsed.8 = INTEGER: 0

Notice that the values for the "D:\" drive have moved from SNMP index ".3" to SNMP index ".8".

To determine the internal index to associate with the values at SNMP index ".8":

  • SL1 determines the value for the designated index (the storage description) at SNMP index ".8". This value is "D:\".
  • SL1 uses "D:\" as the instance value to match against the record of internal indexes for this group, Dynamic Application, and device.
  • The instance value of "D:\" matches to internal index 3. When SL1 stores the value "D:\" and "0", SL1 associates the values with index 3.

Therefore, the storage description and storage used values for the "D:\" drive are associated with the same internal index as was associated with the previous values for the storage description and storage used.

System Settings that Affect Indexing

By default, SL1 throttles surges of multi-index Dynamic Application alerts on individual devices.

To this end, it ignores Dynamic Application alert messages when those messages spike beyond 25 alerts per second. This is done to prevent potential system outages in scenarios where a very large number of alerts are occurring more rapidly than the Data Collector can process them.

The "SL1 Default Internal Events" PowerPack that is included in SL1 contains an event policy, "System: Dynamic App Alert Surge," that notifies the system administrator when alerts for a device exceed that 25-per-second threshold and discards the current batch of messages.

You can adjust this default behavior in the Dynamic Alert per-device field on the System Threshold Defaults page (System > Settings > Thresholds > System).

To edit the threshold for Dynamic Application alert throttling:

  1. Go to the System Threshold Defaults page (System > Settings > Thresholds > System).
  2. In the Dynamic Alert per-device field, drag the slider or enter a new value in the text box to edit the threshold for incoming alerts for a Dynamic Application on a given device.
  3. Click Save.