Overview of Discovery

This section provides an overview of the device discovery process in SL1.

Use the following menu options to navigate the SL1 user interface:

• To view a pop-out list of menu options, click the menu icon ().
• To view a page containing all of the menu options, click the Advanced menu icon ().

What is Discovery?

SL1 also uses discovery to update existing information about a device and to add to existing information about a device. This type of discovery is called auto-discovery. For each existing device, SL1 automatically runs auto-discovery every night, to keep device data up-to-date.

You can manually trigger discovery at any time and update the data for one device or multiple devices.

What Happens During Discovery?

Discovery is executed:

• During the initial discovery of network devices and applications. This is called initial discovery.
• Automatically runs once a day to update the data on each discovered device. This is called auto-discovery, or sometimes referred to as "nightly discovery".
• On demand, when a user manually asks SL1 to rediscover a group of IP addresses or a list of fully-qualified domain names, rediscover a single device, or rediscover all devices to find those that should align with a selected Dynamic Application. This is called re-discovery.

Discovery uses the following processes:

• Discovery: Auto (discover_iprange.py). This process examines one or more IP addresses or fully-qualified domain names and determines which IP addresses or fully-qualified domain names are aligned with a device. For each device, this process retrieves basic information about the device.
• Discovery: Detail (discover_detail.py). This process is triggered by the Discovery: Auto process and retrieves details about each discovered device.
• Discovery: Dynamic App (discover_app.py). This process is triggered manually from the Dynamic Applications Manager page and checks all existing devices against the selected Dynamic Application.
• Discovery: Nightly Update (discover_update.py). SL1 runs this process each night for already discovered devices. This process updates the information collected during initial discovery.

During discovery of an IP address range (the Discovery: Auto process), SL1 does the following:

1. Performs a DNS lookup to determine the IP address for each fully-qualified domain name supplied in the discovery session. These IP addresses are added to the list of IP addresses supplied in the discovery session.
2. Pings each IP address to determine which are in use.
3. Based on the settings in the discovery session, runs nmap on each IP address to determine which are in use and which ports are open.
4. Searches DNS records to determine the fully qualified domain name at each IP address in range.
5. Tries each selected credential on each IP address to determine if each device is manageable (supports SNMP) or will be a device of type "pingable".
6. For devices that support SNMP, retrieves a system description, SysObject ID, system uptime, system contact, system name, and system location.
7. Assigns a Device Class to each device.

If SL1 cannot determine the appropriate Device Class, it will assign the device to the Generic SNMP Device Class.

8. For devices that can be managed (supports SNMP or if non-SNMP discovery is enabled), assigns a device ID, device name, a primary IP address for use in SL1, and a primary credential.

For each device discovered by the Discovery: Auto process, the Discovery: Detail process does the following:

1. If interface discovery is enabled, finds all network interfaces for devices that support SNMP.
2. If the discovery scan level is set to Initial Population of Apps or higher, SL1 checks each discovered device (both those that support SNMP and those that don't) against the list of already-defined Dynamic Applications. SL1 searches each discovered device to find "discovery objects" and aligns devices with the appropriate Dynamic Application(s).
3. If the discovery scan level is set to Discover SSL Certificates or higher, checks for SSL certificates on port 443 (HTTPS).
4. If the discovery scan level is set to Discover Open Ports, Advanced Port Discovery, or Deep Discovery, SL1 performs a port scan using the settings appropriate for the scan level to determine the open ports for the device.

Immediately after the initial discovery session is completed, SL1 will use the aligned Dynamic Applications to collect additional data from devices. For more information about collection processes, see the section on Data Collection.

What Happens During Discovery when the SL1 Agent is Installed?

If a device is monitored using the agent and is discovered as a SNMP or pingable device using the Discovery tool, the following default data collection methods, data display settings, and monitoring policies are applied during discovery:

• The method SL1 uses to monitor availability of the device is determined by the first method of discovery:
• If the agent is installed and creates a device record before the device is discovered as an SNMP or pingable device, availability is based on whether the agent is reporting data to SL1.
• If the device is discovered as an SNMP or pingable device before the agent is installed, availability is based on the method used to discover the device (SNMP, ICMP, or TCP).
• For Linux devices, the TCP/UDP Ports tab in the Device Reports panel will display open ports detected by the agent and open ports detected by the discovery process (using the NMAP command).
• The Processes tab in the Device Reports panel will display running processes detected by the agent and processes collected using SNMP.
• Port monitoring policies specify whether the policy will be executed using the agent or by a Data Collector using NMAP.
• Process monitoring policies are always executed using the agent.
• Data precedence settings specify whether Dynamic Applications that use data collected by the agent or Dynamic Applications that poll devices for data are used to represent CPU and memory utilization for devices.

For more information about configuring data precedence settings on the agent, or about monitoring ports, processes, and device availability with the agent, see the section on Monitoring with the SL1 agent.

What is a Dynamic Application?

Dynamic Applications are the customizable policies that tell SL1 what data to collect from devices and applications. For example, suppose you want to monitor a MySQL database running on a device in your network. Suppose you want to know how many insert operations are performed on the MySQL database. You can create or edit a Dynamic Application that monitors inserts. Every five minutes (for example), SL1 could check the number of insert operations performed on the MySQL database. SL1 can use the retrieved data to trigger events and/or to create performance reports.

SL1 includes Dynamic Applications for the most common hardware and software. You can customize these default Dynamic Applications to suit your environment. You can also create custom Dynamic Applications.

Dynamic Applications in SL1 support a variety of protocols to ensure that SL1 can always communicate with the devices and applications in your network and retrieve information from them. Dynamic Applications can use the following protocols to communicate with devices:

• SNMP
• SQL
• XML
• SOAP
• XSLT (uses SOAP and XSLT to convert XML data to a new format)
• WMI (Windows Management Instrumentation), including WMI and WBEM
• Windows PowerShell
• Custom Python applications (called "snippets") for proprietary or more complex data retrieval

How Does SL1 Align Dynamic Applications During Discovery?

Most Dynamic Applications include a discovery object. A discovery object enables SL1 to determine which devices to align with a Dynamic Application.

During discovery, SL1:

1. Searches the list of Dynamic Applications.
2. If a Dynamic Application includes a discovery object, SL1 adds that Dynamic Application to the list of Dynamic Applications to try to align during discovery.
3. For each Dynamic Application that includes a discovery object, SL1 checks the current discovery session for an appropriate credential. For example, for each database Dynamic Application, SL1 would look for one or more database credentials that have been selected for the discovery session.
4. For each discovered device, both those that support SNMP and those that don't, discovery tries to determine which Dynamic Applications to align. For each discovered device, SL1 tries to align each Dynamic Application in the list of Dynamic Applications to try during discovery. For each Dynamic Application in the list, SL1 tries to connect to each device with each of the appropriate credentials (until SL1 finds a working credential) and then tries to find the discovery object. If SL1 is able to connect to a device with one of the credentials and can then retrieve the discovery object, SL1 will align the Dynamic Application with the device.

NOTE: SL1 also includes more sophisticated logic that allows you to define multiple discovery objects, validate the value of the discovery object, and to align the Dynamic Application if a discovery object is not available. However, the most common use of a discovery object is as described above (discovery object exists).

5. If discovery aligns a Dynamic Application with a device, immediately after discovery completes SL1 will start the first collection from that device using the aligned Dynamic Application. This step is not performed for Dynamic Applications that meet all of the following three criteria:

• Has a collection frequency of 1 minute, 2 minutes, 3 minutes or 5 minutes.
• Does not have component mapping enabled (does not discover component devices).
• Is aligned with a component device.

NOTE: During discovery, SL1 tries each SNMP credential specified in the discovery session on each discovered device, to determine if SL1 can collect SNMP details from the device. Later in the discovery session, during alignment of Dynamic Applications, discovery again tries each SNMP credential specified in the discovery session. If one of the SNMP credentials times out three times without any response, discovery will stop trying to use that SNMP credential to align SNMP Dynamic Applications. Note that "no response" means that a device did not respond at all. Note that if a device reports that "no OID was found" or "the end of the OID tree was reached", these are considered a legitimate response and would not cause SL1 to abandon the credential.

For details on Dynamic Applications, see the appropriate manual for each type of Dynamic Application.

Before You Run Discovery

To make your initial discovery session as productive as possible, you might want to perform these configuration tasks before running discovery:

• Determine the SNMP credentials for the devices and applications in your network. Define correlating credentials in SL1, to allow discovery to retrieve as much information as possible.
• If you want SL1 to immediately start collecting data from devices using Dynamic Applications, you should make sure that each of those Dynamic Applications includes a discovery object.
• If you want SL1 to immediately start collecting data from devices using Dynamic Applications, you should also define any additional credentials required for those Dynamic Applications. For example, if you want SL1 to immediately start monitoring all MySQL databases in your network, you should define credentials that allow SL1 to communicate with each MySQL database in your network. During discovery, SL1 will determine which devices can be monitored with a Dynamic Application for MySQL. After discovery, SL1 will use the database credential to collect data about each MySQL database in your network.
• Defining the global Behavior Settings that affect discovery and auto-discovery. For a description of the Behavior Settings that affect discovery, see the section System Settings that Affect Discovery. For a detailed description of all the Behavior Settings, see the System Administration section.
• Define one or more organizations. During discovery, all discovered devices will be placed into a single organization. If you do not define an organization, SL1 will place all devices in the System organization. However, you can later assign one or more devices to another organization after discovery.
• If you want to perform bulk configuration of discovered devices by using device groups, device templates, or both, you should define device groups and device templates before performing discovery. However, you can apply device groups, device templates or both to one or more devices after initial discovery. For details on device groups and device templates, see the Device Groups and Device Templates section.

System Settings that Affect Discovery

Some of the parameters in the Behavior Settings page affect discovery functionality (discovery, auto-discovery, and re-discovery) in SL1.

NOTE: You can define global parameters for auto-discovery in this page, but you can always override those parameters on a per-device basis by editing the device settings in a device's Device Properties page. For details on configuring a device, see the Device Management section.

To define or edit the settings that affect discovery in the Behavior Settings page:

1. Go to the Behavior Settings page (System > Settings > Behavior).
2. In the Behavior Settings page, edit the values in one or more of the following fields:

• Ping & Poll Timeout (Msec.). This field specifies the number of milliseconds both the discovery tool and availability checks will wait for a response after pinging a device. After the specified number of milliseconds have elapsed without a response, the poll will timeout. The choices are from 100 to 5000 milliseconds.
• SNMP Poll Timeout (Msec.). This field specifies the number of milliseconds the discovery tool will wait for a response after sending an SNMP query to a device. After the specified number of milliseconds have elapsed without a response, the SNMP poll will timeout. The choices are from 100 to 5000 milliseconds.
• SNMP Failure Retries. This field specifies the number of times the discovery tool will try to communicate with a device after a timeout or failure. After that number of times has been met, the discovery tool will not retry unless the user manually restarts the discovery process. The choices are 0–6.
• DHCP Community Strings. SNMP "read only" community string, to use during discovery. This is required only if DHCP servers and devices use a different SNMP community string than other devices in the network. If the community string specified in the Discovery Control Panel (System > Manage > Classic Discovery) page does not work for DHCP devices, SL1 will automatically use the community string specified in this field.
• NFS Detection Disable. If selected, this checkbox prevents SL1 from monitoring and reporting on NFS "shared" hard drives. SL1 will monitor and report only on local hard drives.
• Port Polling Type. Specifies how SL1 should poll devices to discover open ports. The choices are:

• Half Open. Uses a faster TCP/IP connection method and does not appear on device's logs.
• Full Connect. Uses the standard TCP/IP connection to detect open ports.

• Initial Discovery Scan Level. Specifies the data to be gathered during the initial discovery session. You can override this setting for a single discovery session in the Discovery Session Editor modal page. The options are:

• 0. Model Device Only. Discovery tool will discover if device is up and running and if so, collect the make and model of the device. SL1 will then generate a device ID for the device, so it can be managed by SL1.
• 1. Initial Population of Apps. Discovery tool will search for Dynamic Applications to associate with the device. Discovery tool will attempt to collect data for the aligned Dynamic Applications. Discovery will also perform 0. Model Device Only discovery.
• 2. Discover SSL Certificates. Discovery tool will search for SSL certificates and retrieve SSL data. Discovery tool will also perform 1. Initial Population of Apps and 0. Model Device Only.
• 3. Discover Open Ports. Discovery tool will search for open ports. Discovery tool will also perform 2. Discover SSL Certificates, 1. Initial Population of Apps, and 0. Model Device Only. If your system includes a firewall and you select 3. Discover Open Ports, discovery might be blocked and/or might be taxing to your network.
• 4. Advanced Port Discovery. Discovery tool will search for open ports, using a faster TCP/IP connection method. Discovery tool will also perform 2. Discover SSL Certificates, 1. Initial Population of Apps, and 0. Model Device Only. If your system includes a firewall and you select 4. Advanced Port Discovery, some auto-discovered devices might remain in a pending state (purple icon) for some time after discovery. These devices will achieve a healthy status, but this might take several hours.
• 5. Deep Discovery. Discovery tool will use nmap to retrieve operating system name and version. Discovery will also scan for services running on each open port and can use this information to match devices to device classes. Discovery tool will search for open ports, using a faster TCP/IP connection method. Discovery tool will also perform 2. Discover SSL Certificates, 1. Initial Population of Apps, and 0. Model Device Only. For devices that don't support SNMP, option 5. Deep Discovery allows you to discover devices that don't support SNMP and then align those devices with a device class other than "pingable".

Option 5. Deep Discovery is compute-intensive and might significantly tax your network if used as the default setting. ScienceLogic recommends that you use this option on a per-discovery basis by selecting it in the Discovery Session Editor page.

• Rediscovery Scan Level (Nightly). Specifies the data to be gathered/updated each night during auto-discovery. The auto-discovery process will find any changes to previously discovered devices and will also find any new devices added to the network. The options are the same as for Initial Discovery Scan Level.

ScienceLogic recommends that you delete all unused PowerPacks from your SL1 system to improve the performance of the nightly auto-discovery process.

• Discovery Scan Throttle. Specifies the amount of time a discovery process should pause between each IP address or hostname in a discovery session. (You specify the list of IP addresses or hostnames for a discovery session in the IP Address/Hostname Discovery List field in the Discovery Session Editor page.) Pausing discovery processes between IP addresses or hostnames spreads the amount of network traffic generated by discovery over a longer period of time. The choices are:

• Disabled. Discovery processes will not pause.
• 1000 Msec to 10000 Msec. A discovery process will pause for a random amount of time between half the selected value and the selected value.

• Port Scan All IPs. Specifies whether SL1 should scan all IP addresses on a device for open ports. You can override this setting for a single discovery session in the Discovery Session Editor modal page. The choices are:

• 0. Disabled. SL1 will scan only the primary IP address (the one used to communicate with SL1) for open ports.
• 1. Enabled. SL1 will scan all discovered IP addresses for open ports.

• Port Scan Timeout. Length of time, in milliseconds, after which SL1 should stop trying to scan an IP address for open ports and begin scanning the next IP address (if applicable). You can override this setting for a single discovery session in the Discovery Session Editor modal page. Choices are between 60,000 and 1,800,000 milliseconds.

• Restart Windows Services (Agent required). Specifies whether SL1 should automatically restart failed Windows services that have been defined on the device with a startup type of "automatic". To use this feature, the managed device must be running the agent SNMP Informant, WMI Edition. For assistance or information on purchasing and installing this agent, please contact ScienceLogic. Users must also supply a value in the SNMP Write field in the Device Properties page for the device. The choices are:

• 0. Disabled. SL1 will not automatically restart failed Windows services that have been defined on the device with a startup type of "automatic".
• 1. Enabled. SL1 will automatically restart failed Windows services that have been defined on the device with a startup type of "automatic".

• Hostname Precedence. Specifies which name SL1 will use for each discovered device. Choices are:

• SNMP System Name. Use the device name specified in the device's SNMP System MIB. If SNMP System Name is selected and SL1 cannot find an SNMP name for the device, SL1 will assign the name returned by the DNS Reverse Lookup. If SL1 cannot find a DNS Reverse Lookup name for the device, SL1 will use the device's Admin Primary IP address as the device name in SL1.
• DNS Reverse Lookup. Use the device name specified in the device's reverse-lookup record.

• Event Interface Name Format. Specifies the format of the network interface name that you want to appear in events. If you selected Interface Alias for the deprecated Interface Name Precedence field in a previous release of SL1, the format for existing interfaces is set to {alias}. If you selected Interface Name for the deprecated Interface Name Precedence field in a previous release of SL1, the format for existing interfaces is set to {name}. The default format is {name}. You can use a combination of string text and the following tokens to define the interface name format for events, such as string_{name}, string_{alias}, {name}{alias}, or {ifdesc}:

• {alias}
• {name}
• {state}
• {ifdescr}
• {if_id}
• {did}
• {ifindex}
• {ifphysaddress}
• {iftype}
• {ifspeed}
• {ifhighspeed}
• {ifoperstatus}
• {ifadminstatus}

• DNS Hostnames. If SL1 will use the DNS Reverse Lookup name as the device name (see the description of the field Hostname Precedence), this field specifies whether SL1 will use the fully-qualified domain name or only the hostname for each discovered device. Choices are:

• Strip Device Name (Hostname). SL1 will use only the device name as the DNS hostname for each device.
• Use Full Domain Name (FQDN). SL1 will use the fully-qualified domain name as the device name for each device.

3. Click Save.

Device Settings that Affect Auto-Discovery and Re-Discovery

For each discovered device, the following settings in the Device Properties page (Devices > Device Manager > wrench icon) affect how nightly auto-discovery and rediscovery behaves for that device. These settings override any global settings defined in the Behavior Settings page (System > Settings > Behavior):

• SNMP Read /Write. The first drop-down lets you select an SNMP Read credential for read access to SNMP information on the device. The second drop-down let you select a n SNMP Write credential for read-and-write access to SNMP information on the device.
• Availability Port. Specifies the protocol and specific port SL1 should monitor to determine if the device is available. The list of ports will contain all the ports discovered by SL1. The data collected from this port will be used in device availability reports.
• Run Availability Policy (). When you select this icon, SL1 immediately checks the availability of the device, using the port and protocol specified in the Availability Port fields. SL1 displays a Session Logs modal page that displays a detailed description of each step of the availability policy. This information is helpful when troubleshooting availability problems with a device.
• Latency Port. Specifies the protocol and specific port SL1 should monitor to determine latency for the device. The list of ports will contain all the ports discovered by SL1. The data collected from this port will be used in device latency reports.
• Avail + Latency Alert. Specifies how SL1 should respond when the device fails an availability check, when the device fails a latency check, and when the device fails both. These options allow you to create separate events when SNMP fails on a device and when a device is not up and running.
• User Maintenance. Specifies whether the device will be put into "user maintenance" mode. By default, when a device is in "user maintenance", SL1 will not generate events about the device.
• User Maintenance Collection. Specifies whether SL1 should poll the device during the "user maintenance". During normal operation, SL1 polls each device as specified by each device's policies and aligned Dynamic Applications.
• Collection . Specifies (among other things) if the device will be automatically updated each night with SL1's auto-discovery tool. To edit this field, select one of the following from the drop-down list:
• Enabled. Device will be polled by the auto-discovery tool.
• Disabled. Device will not be polled by the auto-discovery tool.

• Collector Group. Specifies which collector group will gather data from the device. You can select from a list of available collector groups.
• Coll. Type. Specifies how SL1 should perform auto-discovery. The choices are:

• Standard. SL1 will perform auto-discovery of each device based on the device's IP address. This method is appropriate for devices using standard DNS.
• DHCP. SL1 will perform auto-discovery of each device based on the device's MAC address. This method is appropriate for devices using DHCP.

• Critical Ping. Frequency with which SL1 should ping the device in addition to the five minute availability poll. If the device does not respond, SL1 creates an event.
• Event Mask. Events that occur on a single device within the selected time-interval are grouped together. This allows related events that occur in quick succession on a single device to be rolled-up and posted together, under one event description.

How File Systems are Hidden During Discovery

When you hide a file system:

• SL1 stops collecting information about the file system.
• SL1 does not generate events about the file system.
• SL1 does not monitor the file system for thresholds (defined in the Device Thresholds and Global Threshold Settings pages).
• SL1 does not include the file system in the Device Summary page.
• SL1 does not include the file system in file system reports in the Device Performance page.

The following rules are applied during discovery to automatically hide file systems:

• If the NFS Detection Disable checkbox is selected in the Behavior Settings page (System > Settings > Behavior), NFS file systems are automatically hidden during discovery.
• File systems of type "iso9660" are automatically hidden during discovery.
• File systems for which the storage size is not reported or the storage size is less than 1024 MB are automatically hidden during discovery.
• File systems of type "Other" are automatically hidden during discovery.

If the type for a discovered file system changes, the auto-hide rules are re-applied to that file system. For example, suppose a Windows drive letter is initially discovered as a removable disk and is auto-hidden. If that drive-letter is later re-used for a fixed drive, this change will cause the file system to be automatically un-hidden.

To manually hide one or more file systems:

1. Go to the Device Hardware page (Devices > Hardware).
2. Filter the list to display only Comp Type of "file system".
3. Select the checkbox for one or more file systems you would like to hide.
4. From the Select Actions field (in the lower right), select Hide File Systems.
5. Click the Go button.
6. Each selected file system will be hidden in SL1.

To manually unhide one or more file systems:

1. Go to the Device Hardware page (Devices > Hardware).
2. Filter the list to display only Comp Type of "file system".
3. Select the checkbox for one or more file systems you would like to unhide.
4. From the Select Actions field (in the lower right), select Unhide File Systems.
5. Click the Go button.
6. SL1 will resume collection for each selected file system and will include each selected file system in the Device Summary and Device Performance pages.