Configuring the SL1 Agent

This section covers how to configure the agent based on the SL1 architecture you are using. This section also describes additional agent configurations, such as enabling extensible collection with an SL1 agent.

Use the following menu options to navigate the SL1 user interface:

To view a pop-out list of menu options, click the menu icon ().
To view a page containing all of the menu options, click the Advanced menu icon ().

Configuring Agent Monitoring Based on SL1 Architecture

Based on your SL1 architecture, you will have access to different workflows for configuring the monitoring settings on an SL1 agent:

If the SL1 agent is used in an SL1 Extended Architecture (which includes Compute Nodes, Storage Nodes, and a Management node), use the following configuration workflows:

To configure basic settings related to how the agent collects data, see Configuring Agent Settings from the Device Investigator Page.
To configure log file monitoring and Windows event log monitoring with the agent, see Monitoring Device Logs Using an Agent.
To configure and view data collected by the agent, see the relevant tabs on the Device Investigator page, including the following tabs:

Settings
Collections
Interfaces
Logs
Processes
Services

For more information about these tabs, see Using the Device Investigator.

If the SL1 agent is used in an SL1 Distributed Architecture (where the SL1 Agent collects and transfers data to a Message Collector), use the following configuration workflows:

To configure basic settings related to how the agent collects data, see Configuring an SL1 Agent on the Device Manager Page.
To configure log file monitoring and Windows event log monitoring with the agent, see Monitoring Device Logs Using an Agent.
To configure and view data collected by the agent, see the relevant tabs on the Device Investigator page, including the following tabs:

Settings
Collections
Interfaces
Logs
Processes
Services

For more information about these tabs, see Using the Device Investigator.

To configure system vitals monitoring with the agent, see Monitoring Vitals Using an Agent.
To configure port monitoring with the agent, see the section on Monitoring Ports.

Configuring Agent Settings from the Device Investigator Page

If a device does not have an agent running on it, the Settings tab of the Device Investigator page for the device will display the Collection section at top left. However, if the device has an agent running on it, the Agents section appears above the Collection section:

In the Agents section, you can configure the disk space, excludes, includes, and other settings related to how you want the agent to collect data.

The Agents section appears on the Settings tab only when you are using the SL1 Extended Architecture. If your version of SL1 does not have an Agents section, see Configuring an SL1 Agent on the Device Manager Page.

To configure agent settings:

From the Devices page, select the device with the agent that you want to configure. The Device Investigator page appears.

To quickly find all devices on the Devices page that have agents installed on them, type "agent" in the Search field and select the "ANY: agent" search.
Click the Settings tab and click Edit.

In the Agent section, complete the following fields, as needed:

Disk Space. Specify the amount of disk space in MB that the agent can use to store data. If the agent loses connectivity to SL1, this disk space will be used to store collected data until the connection to SL1 is restored. When connectivity is re-established, the agent uploads all of its stored data.
Excludes. Type a list of processes and directories, separated by semi-colons, that you do not want the agent to monitor.
Includes. Type a list of processes and directories, separated by semi-colons, that you want the agent to monitor. This field ensures that specific processes are monitored.

If a process or directory is included in both the Excludes field and the Includes field, the item in the Includes field will override the item in the Excludes field.

Collect File Information. Select this option if you want the agent to report the names of files accessed by each monitored process.
Collect Named Pipe Information. Select this option if you want the agent to collect named pipe information.
Collect Socket Information. Select this option if you want the agent to collect socket information.
Collect Thread Information. Select this option if you want the agent to collect thread information.
Collect Non-Intercepted Processes. Select this option if you want the agent to collect limited information for processes that do not contain the agent library.
Processes Aggregation. Specify how you want the agent to collect limited information for processes that do not have the agent library in them, and how to aggregate short-lived processes. Your options include the following:

All: Aggregate every short-lived process into its parent.
None: Do not aggregate any short-lived process.
Without Sockets: Aggregate short-lived processes unless those processes have sockets.

Upload Interval. Specify how often the agent should upload data. Your options include the following:

20 Seconds. Upload a data snapshot every 20 seconds.
60 Seconds. Upload a data summary every 60 seconds. This is the default setting starting with SL1 version 11.1.0, and version 174 of the Linux agent and version 133 for the Windows agent. This option uses an improved data format that requires fewer SL1 resources. The SL1 agent continues to internally collect and poll data every 20 seconds, but the agent summarizes and uploads that data every 60 seconds. There is no data loss even though the data is uploaded less frequently.

Starting with SL1 version 11.3.0, if you specify 60 seconds for the upload interval, the summary upload now will include "watched" or "monitored" files, just like the snapshot upload does.

Click the Save button to save your configuration settings for the agent.

You can view and configure additional agent settings from the Device Investigator page for the device on which the agent is installed, including the following tabs: Interfaces, Processes, Services, and Collections. For more information about these tabs, see Using the Device Investigator.

Configuring an Agent to Export Data to Skylar

You can send agent data to Skylar AI so it can analyze the data for anomaly detection, predictive alerting, and new data visualizations.

For Gen1 agents, which use the SL1 Distributed Environment, you only need to configure the in the "Getting Started with Skylar Analytics" topic in the Skylar Analyticsmanual.

For Gen3 agents, which use the SL1 Extended Environment, you will need to complete the steps in the "Getting Started with Skylar Analytics" topic, and then complete the following additional procedures.

Enabling Skylar Export Using Deploy

After you complete the steps in the "Getting Started with Skylar Analytics" topic:

Retrieve the following values from the /etc/sl-otelcol/sl-otelcol.conf and /etc/sl-otelcol/sl-otelcol-system-id.conf files:

OTELCOL_SYSTEM_ID
OTELCOL_SKYLAR_ENDPOINT
OTELCOL_SKYLAR_KEY
OTELCOL_SKYLAR_KEY_HEADER

On the Management Node, use the prior values from the Database Server and add them to sl1x.inv.yml:

enable_skylar_integration: true
skylar_otel_endpoint: "<OTELCOL_SKYLAR_ENDPOINT>"
skylar_otel_key:"<OTELCOL_SKYLAR_KEY>"
skylar_otel_key_header: "<OTELCOL_SKYLAR_KEY_HEADER>"
skylar_system_id: "<OTELCOL_SYSTEM_ID>"

Run the following command:

run docker-compose -f sl1x-deploy/docker-compose.external.yml run --rm deploy app

As long as deploy is set to run for SL1 12.3.0 and later, the deploy will pull down the necessary services and configure them to enable the export of data to Skylar.

Manually Enabling Skylar

This procedure enables the export of data to the Skylar platform from SL1 Extended by installing or upgrading helm charts from the command line.

On the Database Server, complete the following steps if you have not done them already:

Generate the Skylar config files:

sudo sl-otelcol-mgmt.py --skylar-api-key <API KEY --skylar-endpoint <SKYLAR ENDPOINT> --skylar-all --simple-logging skylar
Ensure that the Skylar service connection is enabled on the Database Server from the SL1 user interface by going to the Service Connections page (Manage > Service Connections) and clicking Add Service Connection.
Get the following values from /etc/sl-otelcol/sl-otelcol.conf file:

OTELCOL_SYSTEM_ID
OTELCOL_SKYLAR_ENDPOINT
OTELCOL_SKYLAR_KEY

On the Extended Management Node, use the values from sl-otelcol.conf when installing the helm charts that enable Skylar:

helm repo update sl1

helm upgrade --version 1.0.42 agent-vitals-service sl1/agent-vitals-service 
--set env.SKYLAR_EXPORT_ENABLED=true
helm upgrade --version 1.3.8 da-postprocessing-service sl1/da-postprocessing-service 
--set skylarExportEnabled=True -f output-files/da-postprocessing-service-values.yml
helm upgrade --version 2.1.26 avail-store sl1/avail-store 
--set skylarExportEnabled=true 
-f output-files/avail-store-values.yml
helm upgrade --version 2.1.32 interface-store sl1/interface-store 
--set skylarExportEnabled=true -f output-files/interface-store-values.yml

# Might need to run with --install if not in place already.
helm upgrade --version 0.0.2 apl-optel-publisher sl1/apl-optel-publisher 
--set optelSettings.skylar.enabled=true 
--set optelSettings.skylar.endpoint=<OTELCOL_SKYLAR_ENDPOINT> 
--set optelSettings.skylar.key=<OTELCOL_SKYLAR_KEY> 
"--set optelSettings.skylar.systemId=<OTELCOL_SYSTEM_ID>

Line breaks were added to the lines of code, above, to allow the code sample to display properly.

After the helm charts are installed, data will begin exporting to the Skylar platform.

Validating the Skylar Export

Check the logs for any of the services:

agent-vitals-service
da-postprocessing-service
avail-store
interface-store

You should see logging similar to the following:

interface-store-8548989dcc-hjkzz 2024-09-17 15:30:00,861::INFO::1::interface_store.handlers.223:::Sending skylar message for did: 5, sink: skylar.data

apl-optel-publisher

apl-optel-publisher-67b7c6558b-shb2z 2024-09-17 15:31:21,643::INFO::1::apl_optel_publisher.handlers.109::: ** Published 1 payloads apl-optel-publisher-67b7c6558b-shb2z {'__OTLP_Resource__': {'attributes': {'sl.service.name': 'Extended', 'sl.datatype.name': 'dynamic_app'}, 'schema_url': ''}}

If the helm charts were installed prior to 12.3.0, you might need to manually create the Kafka topic:

JMX_PORT=5557 /opt/bitnami/kafka/bin/kafka-topics.sh --bootstrap-server kafka-service-headless:9092 --create --topic skylar.data --partitions 5 --replication-factor 3

Configuring Extensible Collection

You can use the SL1 agent for "extensible collection", where you align an SL1 agent with Dynamic Applications to gather metrics and attributes from other infrastructures and applications.

Dynamic Applications are customizable policies, created for a specific vendor and a specific type of device or system, that tell SL1:

What data to collect from devices
How to present the data that has been collected
When to generate alerts and events based on the data that has been collected

SL1 includes Dynamic Applications for the most common hardware and software. You can customize these default Dynamic Applications to best work in your environment. You can also create custom Dynamic Applications.

You can align the agent with PowerShell Dynamic Applications (for Windows agents) and JMX Dynamic Applications (for Linux agents).

In addition, Dynamic Applications that leverage the Low Code No Code CLI/SSH framework can execute using the agent.

To enable extensible collection:

On the Devices page, search for the device with the agent installed on it.

To quickly find all devices that have agents installed on them, type "agent" in the Search field on the Devices page, and select the "ANY: agent" search.
Select the device. The Device Investigator page appears.
Click the Collections tab, click Edit, and click Align Dynamic App. The Align Dynamic Application window appears:
Click Choose Dynamic Application and search for the Dynamic Application you want to align.
Select the Dynamic Application and click Select. The Align Dynamic Application window appears again.
Click Choose Credential to search for a credential for the agent. The Choose Credential page appears.

The Account Type for the credential must be Local for the agent. Do not use the SNMP credential.
Select the credential and click Select. The Align Dynamic Application window appears again.
Click Align Dynamic App. The Collections tab appears, and a pop-up message displays when the Dynamic Application is aligned.
Click Save.
To view the metrics from that newly aligned Dynamic Application, click the Investigator tab.
Click the EDIT button in the right-hand pane of the Investigator tab.
In the Add a metric field, select a metric from the new Dynamic Application to view its data in the right-hand pane. The widget is added to the tab.

The metrics from the new Dynamic Application might take a minute or two to display in the list of metrics. Click Refresh in your browser if needed.

Configuring Run Book Automations for an SL1 Agent

You can enable an SL1 agent to execute run book automation by customizing the automation and action policies in the following PowerPacks:

"Linux SSH Automations" PowerPack version 103 or later
"Windows PowerShell Automations" PowerPack version 103 or later

Action Type

For your agent-based run book action, you will need to use one of the following action types:

"Execute PowerShell Request " from the "Windows PowerShell Automations PowerPack" version 103 or later
"Execute Shell Commands" from the "Linux SSH Automations" PowerPack version 103 or later

These action types (Registry > Run Book > Action Types) contain the code in the Input Parameters Defintion pane that lets you need to run the actions on the agent:

Configuring a Run Book Automation for an Agent Device

You configure a run book automation with an SL1 agent in the same way you configure an automation on a non-agent device. For more information about creating run book action and automation policies, see Run Book Automation.

The following steps provide an example of configuring a User-initiated Automation with an SL1 agent:

Make sure you have the latest version of the "Linux SSH Automations" or "Windows PowerShell Automations" PowerPacks installed on your SL1 system.
Go to the Automation Policy Manager page (Registry > Run Book > Automation).
Click the Create button to create a new run book automation policy or click the edit icon () to edit an existing PowerShell or Linux SSH automation policy.
In the Aligned Devices field, make sure that the SL1 agent device is selected.
In the Available Events pane, select the SL1 event that will trigger this automation and click the right-arrow button.
In the Available Actions pane, select an action policy by highlighting it and clicking the right arrow-button.
Complete the other fields as needed, and then click Save.
Go to the Actions page (Registry > Run Book > Actions).
Click the Create button to create a new run book action or click the edit icon () to edit an existing PowerShell or Linux SSH action.
For the Action Type field, select Execute PowerShell Request or Execute Shell Commands.
Update the commands in the Input Parameters section as needed.
Click Save and close the Policy Editor window.
When an event occurs on the agent device, you can open the Event Investigator page for that event and run this user-initiated run book action by selecting the action from the Tools pane:

Configuring an SL1 Agent on the Device Manager Page

To configure agent settings for an agent, you must first add the SL Agent column to the Device Manager page. For more information about adding the SL Agent column, see Adding the SL Agent Column to the Device Manager Page.

If you are using the SL1 Extended Architecture, you can access these settings from the Settings tab of the Device Investigator page for the device on which the agent is installed. For more information, see Configuring Agent Settings from the Device Investigator Page.

You can control how an agent runs on a device by configuring the following agent settings on the Device Manager page (Devices > Device Manager or Registry > Devices > Device Manager):

Disk Space. Controls the amount of disk space that the agent can use to store data. If an agent loses connectivity to SL1, this disk space will be used to store collected data until the connection to SL1 is restored.
Data Directory. Defines the directory in which the agent will store temporary data.
Excludes. Defines the list of processes and directories to explicitly exclude from monitoring by the agent.
Includes. Defines the list of processes and directories that must be explicitly monitored by the agent. Use the Includes field to ensure that specific processes are monitored.

If a process or directory is included in both the Excludes field and the Includes field, that process or directory will be monitored by the agent.

Adding the "SL Agent" Column to the Device Manager Page

The SL Agent column allows you to access the configuration settings for the agent on a device. For more information about agent configuration settings, see Configuring Agent Settings on a Device. By default, the SL Agent column is not displayed in the Device Manager page.

To add the SL Agent column to the Device Manager page:

Go to the Device Manager page (Devices > Device Manager or Devices > Classic Devices, or Registry > Devices > Device Manager in the classic SL1 user interface in the classic user interface).

Click Actions, and then select Device Manager Preferences. The Edit Device Manager Preferences modal page appears.

In the Device Manager Columns field, control-click Agent.
Click Save.

Configuring Agent Settings on a Device

To configure agent settings, you must first add the SL Agent column to the Device Manager page. For more information about adding the SL Agent column, see Adding the SL Agent Column to the Device Manager Page.

To configure agent settings on a device:

Go to the Device Manager page (Devices > Device Manager or Devices > Classic Devices, or Registry > Devices > Device Manager in the classic SL1 user interface).

Find the device for which you want to edit agent settings. In the SL Agent column, click the gear icon () for the device. The Agent Configuration page appears.

Supply values in the following fields:

Disk Space. Enter the amount of disk space that the agent can use to store data. If the agent loses connectivity to SL1, this disk space will be used to store collected data until the connection to SL1 is restored.
Data Directory. Enter the directory in which the agent will store temporary data.
Excludes. Enter a semi-colon delimited list of processes and directories to explicitly exclude from monitoring by the agent.
Includes. Enter a semi-colon delimited list of processes and directories that must be monitored by the agent. Use the Includes field to ensure that specific processes are monitored.

If a process or directory is included in both the Excludes field and the Includes field, that process or directory will be monitored by the agent.

Click Save.