Monitoring and Maintaining SL1

This section describes how to manage user access, manage scheduled tasks, and more.

Use the following menu options to navigate the SL1 user interface:

To view a pop-out list of menu options, click the menu icon ().
To view a page containing all of the menu options, click the Advanced menu icon ().

Monitoring and Managing User Access

SL1 lets multiple users log into the same SL1 system at the same time. The time a user spends logged into SL1 is known as a user session. You can end a user's session in SL1, and you can also limit the number of users that can be simultaneously logged into an SL1 system.

The Access Sessions page allows administrators to monitor user logins and logouts to the user interface.

From this page, you can also:

End a user's session.
View a list of accounts that are locked out of the user interface due to invalid username and password.
Unlock accounts that are locked out of the user interface.

Viewing Information about Each Access Session

The Access Sessions page displays a list of recent logins to the user interface. To view the Access Sessions page:

Go to the Access Sessions page (System > Monitor > Access Logs).

For each session, the Access Sessions page displays:

User Account. Username of person logging in to the user interface.
User Display Name. The username, email address, or preferred display name. This value is determined by the user's authentication resource settings.
Last Address. IP address from which the user accessed the user interface.
State. Current status of the user. The choices are:

Active. User is currently logged in to the user interface.
Expired. User's session in the user interface was killed.
Logged Out. User logged out of the user interface.
Never Used. User logged in to the user interface and did not perform any tasks before the session was killed.

Login Time. Date and time at which the user logged in.
Last-Hit Time. Date and time at which the user last loaded a page in the user interface.
Logout Time. Date and time at which the user logged out.
Session Duration. Length of time between login and logout.

Deleting a User's Session

From the Access Sessions page, you can end a user's session in the user interface. The user must log in again to access the user interface. The status of the session will be "expired".

To end a user's session:

Go to the Access Sessions page (System > Monitor > Access Logs).

In the Access Sessions page, find the session you want to end. Click the checkbox () for that session.
Click the Select Actions field (in the lower right of the page) and then select Kill user session. Click the Go button
Each selected session is ended. The user associated with each selected session is logged out of the user interface. The status of the session changes to "expired".

NOTE: After ending a user's session, that user can immediately log in to the user interface again. To prevent a user from logging in to the user interface, you must disable the user's account. For information on user accounts, see the Organizations and Users section.

Limiting the Number of Simultaneous User Sessions

If you get an "HTTP Response code was 429 (Too Many Requests)" error or a "User sessions at maximum" in SL1, you can adjust the USER_MAX_SESSIONS value in the /opt/em7/nextui/nextui.conf file:

SSH to the SL1 appliance and log in as user em7admin.
At the command line, open the nextui.conf file in the vi editor:

sudo vi /opt/em7/nextui/nextui.conf
In the NextUI configuration file, set a new value for USER_MAX_SESSIONS, such as USER_MAX_SESSIONS=1000 for 1,000 concurrent user sessions.
Save your changes and restart the NextUI service:

sudo systemctl restart nextui

Viewing Lockouts and Unlocking Lockouts

If a user enters incorrect login information multiple times in a row, that username, the user's IP address, or both will be locked out of the user interface.

To view lockouts or restore login privileges to locked out users:

Go to the Access Sessions page (System > Monitor > Access Logs).
In the Access Sessions page, click the Lockouts button.
The Account Lockouts modal page allows administrators to view a list of locked-out accounts and to restore login privileges to locked out users.
The Account Lockouts modal page displays the following about each lockout:

Attempt Account. Username that caused the lockout.
From Address. IP address from which the failed login attempts originated.
Attempt Time. Date and time at which lockout occurred.
Tries. Number of times user tried to log in to the user interface.

To remove the lock for the user account and allow logins from the username and/or IP address, click the bomb icon ().

Global Settings for Lockouts

The platform includes global settings that define how lockouts behave. In the Behavior Settings page (System > Settings > Behavior), the following fields affect lock-outs:

Account Lockout Type
Account Lockout Attempts
Account Lockout Duration
Lockout Contact Information

Audit Logs

For additional information about users and their actions in the platform, you can view the Audit Logs page. The Audit Logs page provides a complete audit trail for the platform. The Audit Logs page displays a record of all actions in the platform that are generated by users or by managed elements. For details, see the section on Audit Logs.

Improved User Session Control with tmux

Starting with SL1 version 12.2.1, the tmux utility now runs by default when you access an SL1 system using SSH. The tmux utility is a terminal multiplexer that enables a number of terminals to be created, accessed, and controlled from a single screen.

In previous versions of SL1, session control relied on the default shell TMOUT variable, which logged out user sessions after five minutes of inactivity. Starting with SL1 version 12.2.1, this function is now integrated into tmux, automatically locking sessions after 15 minutes of idleness, or if an unclean SSH disconnect or dropped SSH connection occur. This update ensures that idle work is detached rather than lost, resembling workstation behavior when idle. Upon login, SL1 checks for detached tmux sessions and attaches detached tmux sessions if it finds them. Otherwise SL1 starts a new session.

Beyond improved session control, tmux also introduces advanced features like scroll-back buffering with search, built-in clipboarding, multiple sessions and panes, detaching or attaching sessions, and session supervision or sharing. For more information about tmux shortcuts and usage, see https://tmuxcheatsheet.com/.

When using the command-line interface on an SL1 appliance, the interface is meant for limited administrative purposes only.

Managing Scheduled Tasks

The Schedule Manager page (Registry > Schedules > Schedule Manager) allows you to view and manage all the scheduled processes you have defined in your system.

You can define scheduled processes in the following pages:

Report Scheduler. (For more information, see the section on Scheduling a Report Job.)
My Work Schedule. (For more information, see the section on Editing a User Account Work Schedule.)
Recurring Ticketing Scheduler. (For more information, see the section on Scheduled and Recurring Tickets.)
Discovery Control Panel. (For more information, see the section on Scheduling a Discovery Session.)
Dashboards. (For more information, see the section on Scheduling a Dashboard.)
IT Service Editor. (For more information, see the section on Scheduling Downtime for an IT Service.)
Device Manager. (For more information, see the section on Device Maintenance.)
Backup Management. (For more information, see the section on Configuration Backups.)

Managed schedules are loaded in batches of 1,000 by default to prevent the Process Manager from becoming overloaded and stopping the scheduled jobs prematurely. Previously, all scheduled jobs were loaded simultaneously. You can update the default batch number of scheduled jobs by going to the Database Tool page (System > Tools > DB Tool) and entering the following in the SQL Query field, replacing <batch_number> with the number of scheduled jobs you want to load per batch:

INSERT INTO master.system_custom_config (field, field_value) VALUES ('load_schedules_in_batches', <batch_number>) ON DUPLICATE KEY UPDATE field = 'load_schedules_in_batches', field_value = <batch_number>

The Database Tool page is available only in versions of SL1 prior to 12.2.1 and displays only for users that have sufficient permissions to access the page.

Recommended System Maintenance

ScienceLogic also recommends that you take the following actions on a regular basis to reduce outages as much as possible.

Daily:

Review "SL1 Operational Insights: Database Performance" classic dashboard
Review "SL1 Operational Insights: Collector Performance" classic dashboard
Review "SL1 Operational Insights: System Log Summary" classic dashboard
Review "SL1 Operational Insights: Backup History" classic dashboard

You can find the SL1 Operational Insights PowerPack on the PowerPacks page at the ScienceLogic Support Site: https://support.sciencelogic.com/s/powerpacks.

Weekly:

Run the System Status Script and review:

Address every error item in the report
Read Knowledge Base articles
Open tickets for issues when help from SL1 Support is needed

Monthly:

Review capacity items. You must understand License Usage and how to project future capacity

Quarterly:

Audit User Profile access to verify that it meets expected needs
Audit DNS servers and Timeservers on all collectors

Viewing the List of Schedules

The Schedule Manager page (Registry > Schedules > Schedule Manager) displays the following about each schedule:

Schedule Summary. Displays the name assigned to the scheduled process.
Schedule Description. Displays a description of the scheduled process.
Event ID. Displays a unique, numeric ID for the scheduled process. SL1 automatically creates this ID for each scheduled process.
sch id. Displays a unique, numeric ID for the schedule. SL1 automatically creates this ID for each schedule.
Context. Displays the area of SL1 upon which the schedule works.
Timezone. Displays the time zone associated with the scheduled process.
Start Time. Displays the date and time at which the scheduled process will begin.
Duration. Displays the duration, in minutes, which the scheduled process occurs.
Recurrence Interval. If applicable, displays the interval at which the scheduled process recurs.
End Date. If applicable, displays the date and time on which the scheduled process will recur.
Last Run. If applicable, displays the date and time the scheduled process most recently ran.
Owner. Displays the username of the owner of the scheduled process.
Organization. Displays the organization to which the scheduled process is assigned.
Visibility. Displays the visibility level for the scheduled process. Possible values are "Private", "Organization", or "World".
Enabled. Specifies if the scheduled process is enabled. Possible values are "Yes" or "No".

Enabling or Disabling One or More Schedules

You can enable or disable one or more scheduled process from the Schedule Manager page (Registry > Schedules > Schedule Manager). To do this:

Go to the Schedule Manager page (Registry > Schedules > Schedule Manager).

Select the checkbox icon for each scheduled process you want to enable or disable.
Click the Select Action menu and choose Enable Schedules or Disable Schedules.
Click the Go button.

Deleting One or More Schedules

You can delete one or more scheduled process from the Schedule Manager page (Registry > Schedules > Schedule Manager). To do this:

Go to the Schedule Manager page (Registry > Schedules > Schedule Manager).

Select the checkbox icon for each scheduled process you want to delete.
Click the Select Action menu and choose Delete Schedules.
Click the Go button.

Putting the Database Server into Maintenance Mode

You can now put the Database Server in maintenance mode and stop all pull processes from the Data Collectors. You can then perform database maintenance or network maintenance without generating events.

After maintenance is completed, you can put the put the Database Server out of maintenance mode. Pull processes from the the Data Collectors will resume from the point where they were paused.

The new commands are silostart and silostop.

To put a Database Server in maintenance mode:

Either go to the console of a Database Server or SSH to access the Database Server.
Log in as em7admin with the appropriate password.
At the shell prompt, execute the following:

silostop

To put a Database Server out of maintenance mode:

Either go to the console of a Database Server or SSH to access the Database Server.
Log in as em7admin with the appropriate password.
At the shell prompt, execute the following:

silostart

Monitoring Overall System Usage and Statistics

The System Usage page displays:

Tables that show the type and number of each type of task performed by SL1
An optional line graph that displays system usage. To enable the display of this graph, go to the Behavior Settings page (System > Settings > Behavior) and uncheck the Hide Perpetual License Count checkbox. The graph displays the following metrics over time:

Capacity. The total monitoring capacity of the system. This value is determined by the license(s) for the Database Server(s) or All-In-One Appliance(s) in the system.
Number of Devices. The number of devices currently discovered in the system.
System Usage. The amount of Capacity that the devices in the system are currently using. This value is the sum of the Device Ratings for all devices in the system. The Device Rating for each device is calculated daily and is based on the number of collections performed for that device.

If you have a subscription license, you can also generate reports about subscription licensing.

To view the System Usage page, go to the System Usage page (System > Monitor > System Usage).

Viewing an Overview of All Events

The Event Overview page (System > Monitor > Event Overview) provides a graphical overview of all events in SL1.

The Event Overview page displays the following reports:

Number of Events by Severity. This graph displays event distribution by severity for the last 24 hours and for the last 7 days.

The y-axis displays the number of events.
The x-axis displays severity.
The red line represents events in the last 24 hours.
The blue line represents events in the last 7 days.
Mousing over a data point in the red line displays the number of events of the specified severity in the last 24 hours.
Mousing over a data point on the blue line displays the number of events of the specified severity in the last 7 days.

Most Common Event Types. This pie graph displays the ten most frequently occurring events for the last 7 days.

Each slice of the pie represents an event type. The legend on the left maps each slice color to an event and lists the actual number of events of that type.
The graph displays percent. Compared to the total number of occurrences for the top ten events, each slice displays the percent that belong to a specific event.

Mean Time-to-Resolution. This bar graph displays the number of events generated in the last 24 hours, 7 days, 14 days, and 30 days, and their average resolution time.

The y-axis displays the number of events.
The x-axis displays the time span. There is a bar for 24 hours, 7 days, 14 days, and 30 days.
The red bars represent the actual number of events associated with the time-to-resolution.
The blue bars represent the average number of events associated with the time-to-resolution.
Mousing over a bar displays the number of events associated with the time-to-resolution.

Viewing Events by Appliance and Event Source

The Event Statistics page displays a graph of the number of events processed by a selected Database Server, Data Collector, or Message Collector.

The Event Statistics page displays the following information:

Appliance. In the field in the upper left, select from the list of all Database Servers, Data Collectors, and Message Collectors.
Event Type. In the next field on the upper left, select from the list of event types. The choices are:

API. The event was generated by an external API.
Dynamic. Event was generated by a monitoring application running on the device.
Email. The event was generated by an incoming email.
Internal. Event was generated by SL1.
Syslog. Event was generated from standard system log generated by device.
Trap. Event was generated by an SNMP trap.

The graph displays the average number of events processed by the selected appliance, for the selected duration.

The y-axis displays the average number of events.
The x-axis displays time. The increments vary depending upon the selected data type (from the Options menu) and the date range (from the Date Range Selection pane).
Mousing over any point in any line displays the value at that time-point in the Mouse-over column in the Data Table pane.
You can use your mouse to scroll the report to the left and right.