SNMP Traps

This section describes how SL1 handles SNMP traps.

Use the following menu options to navigate the SL1 user interface:

To view a pop-out list of menu options, click the menu icon ().
To view a page containing all of the menu options, click the Advanced menu icon ().

What Happens When a Message Collector Receives an SNMP Trap

When an appliance that performs Message Collection receives an SNMP Trap, it performs the following:

If the trap matches a defined filter, the trap is discarded. See Filtering Traps.
Matches the IP address of the sender to an IP address of a device monitored by a collector group that includes the Appliance.

If the IP address of the sender does not match an IP address of a device monitored by a collector group that includes the Appliance, the message is discarded and a log message is generated. See Traps From Unknown Devices.

Using the MIBs compiled on the SL1 system, translates varbind OIDs to symbolic values.

NOTE: By default, Message Collectors and Data Collectors are not populated with information about all varbind OIDs. The first time a Message Collector or Data Collector attempts to translate a specific varbind OID, that varbind OID will not be translated, but information about that varbind OID will be added to theMessage Collector or Data Collector. All instances of a varbind OID after the first will then be translated correctly. To make SL1 translate the first occurrence of a varbind OID correctly, you can manually run a process that pre-populates Message Collectors and Data Collectors with information about all varbind OIDs. For steps on how to run this process, see the Manually Updating Varbind OIDs.

Compares the trap to the defined trap event policies:

If the trap does not match an event policy, the trap is logged in the Device Logs for the device that sent the trap. See Traps That Do Not Match Event Policies.
If the trap does match an event policy, the Source Host Varbind value for the event policy is evaluated. If the Source Host Varbind value matches a varbind OID in the trap, and the value of the varbind matches an IP address or hostname of a device monitored by a collector group that includes the Message Collector, the event is generated and aligned with the device with that IP address or hostname.
If the trap does match an event policy and is not realigned using the Source Host Varbind value, the event is generated and aligned with the device the trap was matched with in step two.

By default, the event policy "Trap: Unknown trap received" is enabled. This event policy matches all traps that do not match other event policies.

For more information on Trap events, see the section on Events.

Traps That Do Not Match Event Policies

If an Appliance that performs Message Collection receives a trap that:

Is from a device that is monitored by a collector group that includes the Message Collector.
Does not generate an event.

SL1 will log the receipt of the trap in the device logs for the device. If SL1 includes a compiled MIB that contains OIDs used in the received trap, SL1 will include the symbolic translation of those OIDs in the log message. The Device Log will have the following format:

Trap Received | Trap Detail: varbind OID or symbolic translation: varbind data type: varbind data; (Trap OID: trap OID)

NOTE: Device Logs that are not associated with an Event are retrieved from Collection Units at five-minute intervals. It may take up to five minutes for traps that do not match event policies to appear in the Device Logs.

Traps From Unknown Devices

If an Appliance that performs Message Collection receives a trap from an unknown device, a "From unknown device: <ip-address-of-unknown-device>, received the following Trap message:" event will be generated. An unknown device is defined as either:

A device monitored by the SL1 system, but by a collector group that does not include the Appliance.
A device not monitored by the SL1 system.

The "From unknown device: <ip-address-of-unknown-device>, received the following Trap message:" event will appear in the Event Console page associated with the System organization.

For the first trap received from an unknown device, the event will have a Severity value of "Notice". If multiple traps are received from the same unknown device, additional events will be generated at the following thresholds:

10, 25 Traps Received. Severity value of "Minor".
100 Traps Received, and every 100 traps up to and including 900 Traps Received. Severity value of "Minor".
1,000 Traps Received, and every 1,000 traps up to and including 9,000 Traps Received. Severity value of "Minor".
10,000 Traps Received, and every 10,000 traps received thereafter. Severity value of "Major".

NOTE: The counters for the number of traps received from unknown devices will be reset to zero if the Event Engine on the Appliance that performs Message Collection is restarted, or the Appliance is restarted.

NOTE: The default threshold for incoming traps is set to 25 messages per second to prevent degraded performance.

Filtering Traps

In some situations, you might want to filter or limit the traps that are processed by SL1. SNMP Trap Filters allow you to define policies that filter incoming traps to an Appliance that performs Message Collection. When a trap is filtered, the Appliance that performs Message Collection receives the trap, but does not store the trap, does not act on the trap, and does not pass the trap on to be examined by the ScienceLogic event engine.

You can filter incoming SNMP traps using one, multiple, or all of the following parameters:

IP or hostname of the host that sent the trap. You can also specify "all hosts"
Trap OID
Varbind OID
Varbind content

So you can:

Filter all incoming traps from a specific host.
Filter incoming traps with a specific trap OID from all hosts.
Filter incoming traps with a specific trap OID and from a specific host.
Filter traps with a specific trap OID and specific varbind OID from all hosts.
Filter traps with a specific trap OID and specific varbind OID from a specific host.

To create an SNMP Trap Filter, perform the following steps:

Go to Registry > Events > SNMP Trap Filters. The SNMP Trap Filters page is displayed.
In the SNMP Trap Filters page, select the Create button. The SNMP Trap Filter modal page is displayed.
In the SNMP Trap Filter modal page, supply a value in the following fields:

Filter State. Specifies whether the SNMP Trap Filter is currently active. When the SNMP Trap Filter is active, all incoming traps that match the criteria in the filter are dropped, and the Appliance does not act upon them. Choices are "Enabled" or "Disabled".
Host Filter. Specifies hosts to filter-on. All incoming traps sent from the specified host(s) that match the other parameters will be dropped by the Message Collector.

If you select the checkbox next to the field name, you can enter a host name or an IP address. All incoming traps from the specified host that also match the other parameters will be dropped by the Appliance.
If you do not select the checkbox next to the field name, this field will contain the value All. In this case, incoming traps from all hosts that also match the other parameters will be dropped by the Appliance.

Trap OID Filter. Specifies the trap OID to filter on. All incoming traps that are named with the specified OID(s) and match the other parameters will be dropped by SL1.

If you select the checkbox next to the field name, you can enter an OID value in standard dotted-decimal notation in this field. All incoming traps that are named with the specified OID that also match the other parameters will be dropped by the Appliance.
If you do not select the checkbox next to the field name, this field will contain the value All. In this case, all incoming traps named with all OIDs that also match the other parameters will be dropped by the Appliance.

Varbind OID Filter. A varbind consists of an object, specified by an OID, and its value. In this field, you specify the varbind OID to filter on. All incoming traps that contain the specified varbind OID and also match the other parameters will be dropped by the Appliance.

If you select the checkbox next to the field name, you can enter an OID value in standard dotted-decimal notation in this field. All incoming traps that contain that varbind OID and match the other parameters will be dropped by the Appliance.
If you do not select the checkbox next to the field name, this field will contain the value All. In this case, all incoming traps that contain all OIDs will be dropped by the Appliance.

Varbind OID Pattern. A varbind consists of an object, specified by an OID, and its value. In this field, you specify a pattern to search for in the varbind value. All incoming traps that contain a varbind value with this pattern and also match the other parameters will be dropped by the Appliance.

If you select the checkbox next to the field name, you can enter an alpha-numeric pattern, including multi-byte characters, to search for. All incoming traps that contain a varbind with that value and also match the other parameters will be dropped by the Appliance.
If you do not select the checkbox next to the field name, this field will contain the value All. In this case, all incoming traps that contain all varbind values that also match the other parameters will be dropped by the Appliance.

Select the Save button to save the new SNMP Trap Filter.
The new SNMP Trap Filter should now appear in the SNMP Trap Filters page. If the filter is enabled, SL1 will not store or process traps that meet the filter criteria.

To edit an SNMP Trap Filter, perform the following steps:

Go to Registry > Events > SNMP Trap Filters. The SNMP Trap Filters page is displayed.
In the SNMP Trap Filters page, find the filter you want to edit. Select its wrench icon (). The SNMP Trap Filter modal page is displayed.
In the SNMP Trap Filter modal page, change the values in one or more fields.
Select the Save button to save your changes to the SNMP Trap Filter.

To delete an SNMP Trap Filter, perform the following steps:

Go to Registry > Events > SNMP Trap Filters. The SNMP Trap Filters page is displayed.
In the SNMP Trap Filters page, find the filter you want to delete. Select its checkbox (). To select all checkboxes for all filters, select the big checkbox icon () at the top of the page.
In the Select Action drop-down list, select Delete filter definitions. Select the Go button.
The selected SNMP Trap Filters will be deleted. SL1 will stop filtering the incoming SNMP traps that were previously filtered with the deleted SNMP Trap Filters.

Global Settings that Affect SNMP Trap Processing

The following global setting affects how SL1 processes SNMP traps:

use_v1trap_envelope_addr. In environments where Network Address Translation is performed on SNMP v1 trap messages sent to SL1, you can configure SL1 to read the envelope address (the address of the host sending the trap) instead of the agent address (the IP address variable sent as part of the trap). To use the envelope address instead of the agent address for SNMP v1 trap messages, the use_v1trap_envelope_addr=1 configuration option can be added to the [LOCAL] section of silo.conf on Message Collectors, Data Collectors that perform message collection, and All-In-One Appliances. If use_v1trap_envelope_addr is not defined in silo.conf or use_v1trap_envelope_addr=0 is defined, SL1 will use the agent address for SNMP v1 trap messages.

To add a settings to the silo.conf file on an appliance:

Either go to the console of the SL1 appliance or use SSH to access the server.
Login as user em7admin with the password you configured during setup.
At the shell prompt, enter the following:

sudo visilo
On a line of its own, add the new entry.
Save your changes and exit the file (:wq).

System Settings that Affect SNMP Trap Processing

The following system setting affects how SL1 processes SNMP traps:

Ignore trap agent-addr varbind. If you select this checkbox, SL1 will align the SNMP trap with the forwarder (last hop) instead of searching for the IP address of the originator of the trap.
Enhanced OID Translation. If selected, ensures that varbind OIDs that use multi-dimensional indexes are translated correctly. The symbolic translation of the known portion of the OID is included in the log message associated with the trap.

NOTE: Enabling the Enhanced OID Translation option might affect performance on large environments with a large number of traps.

To enable these settings:

Go to the Behavior Settings page (System > Settings > Behavior).
Select the checkbox next to the setting or settings you want to enable:

Click Save to save the settings.

Manually Updating Varbind OIDs

By default, Message Collectors and Data Collectors are not populated with information about all varbind OIDs. The first time a Message Collector or Data Collector attempts to translate a specific varbind OID, that varbind OID will not be translated, but information about that varbind OID will be added to the Message Collector or Data Collector. All instances of a varbind OID after the first will then be translated correctly.

To make SL1 translate the first occurrence of a varbind OID correctly, you can manually run a process that pre-populates Message Collectors and Data Collectors with information about all varbind OIDs. You should run this process after adding new MIBs to SL1.

To manually populates Message Collectors and Data Collectors with information about all varbind OIDs, perform the following steps:

Go to the OID Browser page (System > Tools > OID Browser).
Select the Update button.

Configuring SNMPv3 Traps

To configure a Message Collector or Data Collector to accept an SNMPv3 trap or inform, SL1 automatically configures the trap configuration file on the Message Collector or Data Collector. SL1 automatically populates the SNMPv3 trap and inform credentials including the engine ID of the recipient, the Message Collector or Data Collector.

To configure an SNMPv3 Trap:

Go to the Credentials page (Manage > Credentials.
Click the SNMPv3 Trap Configuration Reset icon ().
SL1 automatically configures the etc/snmptrapd.con file to receive SNMPv3 traps from all monitored devices.

Configuring SNMPv3 Traps in the Classic User Interface

To configure an SNMPv3 Trap in the classic SL1 user interface:

Go to the Credential Management page (System > Manage > Credentials).
Click the Actions button and then select Push SNMPv3 Trap Configuration.
A warning message appears: "Warning: This will push the SNMP V3 trap configuration to all collectors and message collectors and restart the snmptrapd service on the appliance. Are you sure you want to submit this?"

Click OK. SL1 automatically configures the etc/snmptrapd.con file to receive SNMPv3 traps from all monitored devices.

Manually Configuring SNMPv3 Traps

NOTE: These steps are no longer required in SL1 systems later than 8.14.9 or 10.1.4

To configure a Message Collector or Data Collector to accept an SNMPv3 trap or inform, you must edit the trap configuration file on the Message Collector or Data Collector. In the trap configuration file, enter the credentials that allow SL1 to communicate with the device that sends traps to SL1. Enter credentials for each device that sends traps to SL1. This information is configured in a configuration file at the operating-system level.

For SNMPv3 traps, the credential entry must include the engine ID of the device sending the trap. Therefore, there will be an entry in the trap configuration for each device that will send SNMPv3 traps. For SNMPv3 informs, the entry does not need to specify the engine ID. The engine ID of the recipient, the Message Collectoror Data Collector, is used for SNMPv3 informs. Therefore, if all the managed devices use the same credentials to send SNMPv3 informs, you need to add only one entry to the trap configuration file.

NOTE: Existing trap event policies will be triggered by SNMPv3 traps and SNMPv3 informs with no additional configuration.

To add a credential entry to the trap configuration file:

Either go to the console of the the Message Collector or Data Collector or use SSH to access the server.
Log in as user em7admin with the appropriate password.
Open the /etc/snmp/snmptrapd.conf file in a text editor.
At the end of the file, add a new "createUser" line. See the section below for the appropriate syntax.
Save the file.
Restart the trap engine by executing the following command:

sudo service snmptrapd restart

The syntax of createUser is different for each security level and whether you are configuring traps or informs:

Informs, no authentication, no encryption (noAuthNoPriv):

createUser <security name>

For example:

createUser em7defaultv3

Informs, authentication, no encryption (authNoPriv):

createUser <security name> <auth protocol> <security passphrase>

For example:

createUser em7defaultv3 SHA em7authpass

NOTE: For FIPS-compliant systems, authentication with MD5 will fail.

Informs, authentication and encryption (authPriv):

createUser <security name> <auth protocol> <security passphrase> <privacy protocol> <privacy pass phrase>

For example:

createUser em7defaultv3 SHA em7authpass DES em7privpass

Traps, no authentication, no encryption (noAuthNoPriv):

createUser -e <engine ID> <security name>

For example:

createUser -e 0x0102030405 em7defaultv3

Traps, authentication, no encryption (authNoPriv):

createUser -e <engine ID> <security name> <auth protocol> <security passphrase>

For example:

createUser -e 0x0102030405 em7defaultv3 SHA em7authpass

Traps, authentication and encryption (authPriv):

createUser -e <engine ID> <security name> <auth protocol> <security passphrase> <privacy protocol> <privacy pass phrase>

For example:

createUser -e 0x0102030405 em7defaultv3 SHA em7authpass DES em7privpass

Here are some example commands for how to send a test coldStart trap from a SL1 appliance using authPriv and the credential information from the examples above:

snmptrap -e 0x0102030405 -v3 -u em7defaultv3 -a SHA -A em7authpass -x DES -X em7privpass <message collector ip> 0 .1.3.6.1.6.3.1.1.5.1

snmpinform -v3 -u em7defaultv3 -a SHA -A em7authpass -x DES -X em7privpass <message collector ip> 0 .1.3.6.1.6.3.1.1.5.1