Configuration and Discovery

The following sections describe how to configure and discover IBM AIX devices for monitoring by SL1 using the IBM: AIX Monitoring PowerPack:

Supported Operating Systems

Versions 7.X of AIX are supported by the PowerPack. Older versions of IBM AIX will work if the commands in this document will work with that version.

Prerequisites

To configure the SL1 system to monitor IBM AIX devices using the IBM: AIX Monitoring PowerPack, you must have the following information about your devices:

• IP addresses/hostnames of the devices you want to monitor.
• A service account with the appropriate privileges:

• Username/Password, OR
• SSH private keys

• You must install the "sudo" package on IBM's online article: AIX Toolbox for Open Source Software.

The "IBM: AIX CPU Utilization (sar)" and "IBM: AIX Total Allocated Storage Value (Bootinfo)" Dynamic Applications execute their commands via "sudo" by default. If you do not wish to install the "sudo" package, you will need specific permissions to execute both sar and bootinfo commands, and to remove "sudo" from the snippet arguments of the aforementioned Dynamic Applications.

• Port 22 (default SSH port) open from the SL1 Data Collector to the end device. Custom ports are supported if they are specified in the credential.
• If you are using the "IBM: AIX CPU Utilization (sar)" Dynamic Application, you will need to execute the sar command with sudo. To do so, add the following permission to the sudoers configuration file (/etc/sudoers): 

<username> ALL=(ALL) NOPASSWD:/usr/sbin/sar

By adding the permission to the sudoers configuration file, the sar command will not request for a password.

• If you are using the "IBM: AIX Total Allocated Storage Value (Bootinfo)" Dynamic Application, you will need to execute the booinfo command with sudo. To do so, add the following permission to the sudoers configuration file (/etc/sudoers):

<username> ALL=(ALL) NOPASSWD:/usr/sbin/bootinfo

By adding the permission to the sudoers configuration file, the bootinfo command will not request for a password.

• If you are upgrading from any previous AIX community PowerPack, you will need to disable all events and alerts from the previous PowerPack before installing and using the IBM: AIX Monitoring PowerPack to ensure that duplicate events do not occur.

To monitor CLI-SSH authentication events or alerts, you must first install the "Linux Base Pack" PowerPack. The "Linux Base Pack" PowerPack contains the following API-type event policies, which trigger events for any device that uses SSH authentication:

• Linux SSH: Authentication Error (CLI-SSH: Authentication Error)
• Linux SSH: Authentication Timeout (CLI-SSH: Authentication Timeout)
• Linux SSH: Connection Error (CLI-SSH: Error Connecting to Host)

Configuring IBM AIX Devices to Collect Data

The following tables list the collection objects included in IBM AIX Dynamic Applications and the AIX commands used by each of those collection objects. You can use these commands to grant or restrict access to certain data types in your IBM AIX user account that you will use to monitor your IBM AIX devices.

Dynamic Application	Collection Object(s)	AIX Command(s) Used
IBM: AIX Device Class Configuration	OS Level	oslevel
	OS Version	oslevel | sed 's/\./ /g' | awk '{ if (NF>=2) print "AIX "$1"."$2; else print "AIX "$1}'
IBM: AIX Disks	Active Disk %	lspv;iostat -D
	Disk Service Queue Full
	Failed Read requests per sec
	Failed Write requests per sec
	Physical Volume Name
IBM: AIX Error Report	TimeStamp	errpt -s <start_time> | head -n 1000
IBM: AIX ICDA Cache	Filesystem	df -kP
	Filesystem Characteristics	lsfs
	Processes	ps auxww
	uptime	uptime
IBM: AIX Inodes	Inodes Used Percentage	df -i
	Mount Directory
IBM: AIX LPAR CPU Configuration	Maximum Memory (MB)	lparstat -i
	Maximum Physical CPUs in system
	Maximum Virtual CPUs
	Minimum Memory (MB)
	Minimum Virtual CPUs
	Mode
	Node Name
	Online Memory
	Partition Name
	Partition Type
IBM: AIX LPAR CPU Stats	Active CPUs in Pool	lparstat -i
	Active Physical CPUs in system
	Entitled Capacity (%entc)	lparstat
	Entitled Capacity (units)	lparstat -i
	LPAR Overall CPU Utilization %	lparstat
	LPAR Type
	Maximum Physical CPUs in system	lparstat -i
	Node Name
IBM: AIX Memory Stats	Available Memory	svmon -G -O summary=basic,unit=MB
	Free Memory
	Memory Size
	Pinned memory consumption
	Real Memory Consumption
	Virtual Memory Consumption
	Paging (Swap) Space Inuse
	Paging (Swap) Space Size
IBM: AIX Network Stats	Packet Errors	netstat
IBM: AIX Page Space Read Write	Page Space Pages Read per Sec	vmstat 1 1
	Page Space Pages Written per Sec
IBM: AIX Path Status	Path Status	lspath | uniq
IBM: AIX Processor States	Processor Name	lsdev -Cc processor
	Processor Status	lsdev -Cc processor; lsattr -El <processorname>
IBM: AIX System Configuration	CPU type	prtconf
	Default Gateway
	Domain Name
	Firmware Version
	FQDN
	Host Name
	IP Address
	Logical CPU Cores	mpstat -s 1 1
	Manufacturer	prtconf
	Name Server
	Netmask
	OS Level	oslevel
	OS Version	oslevel | sed 's/\./ /g' | awk '{ if (NF>=2) print "AIX V"$1"."$2; else print "AIX V"$1}'
	Paging Space (MB)	prtconf
	Physical Memory (MB)
	Physical Processors
	Processor Speed (MHz)
	Processor Type
	Processor Version
	Product Name
	Serial Number
	System Type
	Time Zone	date
	Uptime (days)	uptime
IBM: AIX Uptime & Load Average	CPU Core Count	lsdev -Cc processor
	Load Average (15 min)	uptime
	Load Average (1 min)
	Load Average (5 min)
	Total Uptime Minutes
IBM: AIX Volumes	Free Partition Size	lspv
	Logical Volume Name	lsvg
	Logical Volume State
	Logical Volume Type
	Physical Volume Name	lspv
	Physical Volume State
	Stale Partitions Count
	Total Partition Size
	Used Partition Size
IBM: AIX Zombie Process Count	Zombie Process Count	ps -eo "status" | grep -c Z

 

The following table lists internal collection inventory and internal collection performance Dynamic Applications (ICDA) in the PowerPack:

Dynamic Application	Metrics	AIX Command(s) Used
IBM: AIX IC Detail	uptime	uptime
IBM: AIX IC Filesystem Inventory	Filesystem	df -kP
IBM: AIX IC Filesystem Performance
IBM: AIX IC Process Inventory	Processes	ps auxww
IBM: AIX IC Process Performance

The following table lists Dynamic Applications that are disabled by default and not included as part of the "IBM: AIX Dynamic Applications Template" device template. These Dynamic Applications can be enabled as needed by manually adding them to your devices or the device template:

Dynamic Application	Metrics	AIX Command(s) Used
IBM: AIX CPU Utilization (mpstat)	CPU Utilization %	mpstat
IBM: AIX CPU Utilization (sar)	CPU Utilization %	sudo sar -u 1 3
IBM: AIX Filesystem	Filesystem	df -kP
IBM: AIX Total Allocated Storage Value (Bootinfo)	Total Allocated Storage (GB)	TOTAL=0; for DISK in $(lspv | awk '{ print $1 }');do SIZE=$(sudo bootinfo -s $DISK); TOTAL=$(echo "$TOTAL + $SIZE"|bc); done; echo "$TOTAL / 1024" | bc

Creating an SSH/Key Credential for IBM AIX

To configure SL1 to monitor IBM AIX devices using SSH, you must first create an SSH/Key credential. This credential allows the Dynamic Applications in the IBM: AIX Monitoring PowerPack to connect with an IBM AIX device.

NOTE: If you are using an SL1 system prior to version 11.1.0, the new user interface does not include the Duplicate option for sample credential(s). ScienceLogic recommends that you use the classic user interface and the Save As button to create new credentials from sample credentials. This will prevent you from overwriting the sample credential(s).

To create an SSH/Key credential:

1. Go to the Credentials page (Manage > Credentials).
2. Locate the "IBM: AIX Example Credential" and click on it. The Edit Credential modal page appears:

3. Supply values in the following fields:

• Name. Name of the credential. Can be any combination of alphanumeric characters, up to 64 characters. This field is required.

• All Organizations. Toggle on (blue) to align the credential to all organizations, or toggle off (gray) and then select one or more specific organizations from the What organization manages this service? drop-down field to align the credential with those specific organizations. This field is required.

• Timeout (ms). Time, in milliseconds, after which SL1 will stop trying to communicate with the device from which you want to retrieve data.

• Hostname/IP. Type the hostname or IP address of the IBM AIX device you want to monitor. Alternatively, you can use the following variables:

• You can include the variable %D in this field. SL1 will replace the variable with the IP address of the current device (device that is currently using the credential).
• You can include the variable %N in this field. SL1 will replace the variable with hostname of the current device (device that is currently using the credential). If SL1 cannot determine the hostname, SL1 will replace the variable with the primary, management IP address for the current device.

• Port. Port number associated with the data you want to retrieve. This field is required.

The default TCP port for SSH servers is 22.

• Username. Username for an SSH or user account on the device to be monitored.
• Password. Password for an SSH user account on the device to be monitored.
• Private Key (PEM Format). Type or paste the SSH private key that you want SL1 to use, in PEM format.

The private key must include the lines "BEGIN RSA PRIVATE KEY" and "END RSA PRIVATE KEY", as well as all preceding and following dashes on those lines.

The Private Key (PEM Format) field is only required in the current SL1 user interface. The Private Key (PEM Format) field is not required if you are using the classic SL1 user interface to define a credential.

The private key can have a maximum of 64 characters per line. Therefore, you cannot use keys in the OpenSSH format, because that format uses 70 characters per line. When you attempt to save the credential, SL1 will validate that the private key entered is in the correct format. You will be able to save the credential only if the private key is correctly formatted.

4. Click Save & Close.

Creating an SSH/Key Credential for IBM AIX in the SL1 Classic User Interface

To configure SL1 to monitor IBM AIX devices using SSH, you must first create an SSH/Key credential. This credential allows the Dynamic Applications in the IBM: AIX Monitoring PowerPack to connect with an IBM AIX device.

To define an SSH/Key credential:

1. Collect the information you need to create each credential (usually username and password).
2. Go to the Credential Management page (System > Manage > Credentials).
3. In the Credential Management page, located the "IBM: AIX Example Credential" and click its wrench icon (). The Credential Editor appears.

4. In the Credential Editor, supply values in the following fields:

• Credential Name. Type a name for the credential.
• Hostname/IP. Type the hostname or IP address of the IBM AIX device you want to monitor. Alternatively, you can use the following variables:

• You can include the variable %D in this field. SL1 will replace the variable with the IP address of the current device (device that is currently using the credential).

• You can include the variable %N in this field. SL1 will replace the variable with hostname of the current device (device that is currently using the credential). If SL1 cannot determine the hostname, SL1 will replace the variable with the primary, management IP address for the current device.

• Port. Port number associated with the data you want to retrieve. This field is required.

The default TCP port for SSH servers is 22.

• Timeout (ms). Time, in milliseconds, after which SL1 will stop trying to communicate with the authenticating server.
• Username. Username for an SSH or user account on the device to be monitored.
• Password. Password for an SSH user account on the device to be monitored. (Optional if using a PEM key.)
• Private Key (PEM Format). Type or paste the SSH private key that you want SL1 to use, in PEM format.

The private key must include the lines "BEGIN RSA PRIVATE KEY" and "END RSA PRIVATE KEY", as well as all preceding and following dashes on those lines.

The private key can have a maximum of 64 characters per line. Therefore, you cannot use keys in the OpenSSH format, because that format uses 70 characters per line. When you attempt to save the credential, SL1 will validate that the private key entered is in the correct format. You will be able to save the credential only if the private key is correctly formatted.

5. Click the Save button to save the new SSH/Key credential.

Configuring the AIX Device Template

A device template allows you to save a device configuration and apply it to multiple devices. The Dynamic Applications in the IBM: AIX Monitoring PowerPack do not align automatically. Configuring and applying the "IBM: AIX Dynamic Applications Template" device template when you discover your IBM AIX device will align the appropriate Dynamic Applications.

To configure the IBM AIX device template:

1. Go to the Configuration Templates page (Devices > Templates or Registry > Devices > Templates in the SL1 classic user interface).

2. Locate the "IBM: AIX Dynamic Applications Template", or if the target host is a VIOS type, the "IBM: VIOS Dynamic Applications Template", and click its wrench icon (). The Device Template Editor page appears.

3. Click the Dyn Apps tab. The Editing Dynamic Application Subtemplates page appears.
4. Complete the following fields:

• Template Name. Type a new name for the device template.
• Credentials. Select the SSH/Key credential that you created for IBM AIX.

5. Click the next Dynamic Application listed in the Subtemplate Selection section on the left side of the page and then select the AIX SSH/Key credential in the Credentials field.
6. Repeat step 5 until the you have selected the IBM AIX SSH/Key credential in the Credentials field for all of the Dynamic Applications listed in the Subtemplate Selection section.
7. Click Save As.

 Do not click the Save button, as it will save over the "IBM: AIX Dynamic Applications Template", which you may need for future use.

Preventing IBM AIX Devices from Dynamically Aligning Unwanted Dynamic Applications

As the Dynamic Applications in this PowerPack do not get automatically aligned during discovery, you can choose to disable the "Dynamic Discovery" flag on your IBM AIX device to ensure that other Dynamic Applications do not automatically align to your IBM AIX device and cause increased scale on your SL1 system.

To disable the "Dynamic Discovery" flag from the Device Investigator:

1. In the Devices page, locate your IBM AIX device and click on it.
2. In the Device Investigator page, click the Settings tab.
3. Locate the Dynamic Discovery checkbox and deselect it.
4. Click the Save button.

To disable "Dynamic Discovery" from the device template:

1. In the Configuration Templates page (Devices > Templates), locate your IBM AIX device and click on its wrench icon ().
2. In the Device Template Editor, in the Device Preferences pane of the Config tab, click on Dynamic Discovery to enable the option.
3. Deselect the Dynamic Discovery checkbox.
4. Click the Save button.

Discovering IBM AIX Devices

To create and run a discovery session that will discover IBM AIX devices, perform the following steps:

1. On the Devices page () or the Discovery Sessions page (Devices > Discovery Sessions), click the Add Devices button. The Select page appears:

2. Click the Unguided Network Discovery Workflow button. Additional information about the requirements for discovery appears in the General Information pane to the right.

3. Click Select. The Basic Information page appears:
4. Complete the following fields:

• Name. Type a unique name for this discovery session. This name is displayed in the list of discovery sessions on the Discovery Sessions tab.
• Description. Optional. Type a short description of the discovery session. You can use the text in this description to search for the discovery session on the Discovery Sessions tab.
• Select the organization to add discovered devices to. Select the name of the organization to which you want to add the discovered devices.

5. Click Next. The Credential Selection page of the Add Devices wizard appears:

6. On the Credentials page, locate and select the SSH/Key credential you created.

7. Click Next. The Discovery Session Details page of the Add Devices wizard appears:

8. Complete the following fields:

• List of IPs/Hostnames. Type the IP addresses for the IBM AIX devices you want to monitor, separated by a comma.

• Which collector will monitor these devices?. Select an existing collector to monitor the discovered devices. Required.
• Run after save. Select this option to run this discovery session as soon as you click Save and Close.

In the Advanced options section, click the down arrow icon () to complete the following fields:

• Discover Non-SNMP. Enable this setting.
• Model Devices. Enable this setting.

• Apply Device Template. Select the device template that you created for IBM AIX.

9. Click Save and Close to save the discovery session. The Discovery Sessions page (Devices > Discovery Sessions) displays the new discovery session.
10. If you selected the Run after save option on this page, the discovery session runs, and the Discovery Logs page displays any relevant log messages. If the discovery session locates and adds any devices, the Discovery Logs page includes a link to the Device Investigator page for the discovered device.

Discovering IBM AIX Component Devices in the SL1 Classic User Interface

To create and run a discovery session that will discover IBM AIX devices, perform the following steps:

1. Go to the Discovery Control Panel page (System > Manage > Classic Discovery).
2. Click the Create button to create a new discovery session. The Discovery Session Editor modal page appears.

3. Enter values in the following fields:

• IP Address Discovery List. Type the IP addresses for the IBM AIX devices you want to monitor, separated by a comma.

• Other Credentials. Select the SSH/Key credential that you created for IBM AIX.
• Discover Non-SNMP. Select this checkbox.
• Model Devices. Select this checkbox.
• Apply Device Template. Select the device template that you created for IBM AIX.

4. Optionally, you can enter values in the other fields on this page. For more information about the other fields on this page, see the Discovery & Credentials manual.
5. Click the Save button and then close the Discovery Session Editor modal page.
6. The discovery session you created will appear at the top of the Discovery Control Panel page. Click its lightning-bolt icon () to run the discovery session.
7. The Discovery Session window appears. When the IBM AIX device is discovered, click its device icon () to view the Device Properties page for the IBM AIX device.

Enabling More Than Two Thresholds for Filesystem

The "IBM: AIX IC Filesystem Inventory" and "IBM: AIX IC Filesystem Performance" Dynamic Applications provide two thresholds - Major and Critical. If you need more than two thresholds for Filesystem, you will need to disable the IC Filesystem Dynamic Applications and enable the "IBM: AIX Filesystem" Dynamic Application.

To disable the "IBM: AIX IC Filesystem Inventory" and "IBM: AIX IC Filesystem Performance" Dynamic Applications:

1. Locate your IBM AIX device in the Devices page and click on it.
2. In the Device Investigator page for the IBM AIX device, go to the Collections tab.
3. In the Collections tab, click the Edit button.
4. Locate the "IBM: AIX IC Filesystem Inventory" Dynamic Application, click its Actions icon (), and select Disable Collection.
5. Click the Save button.
6. Repeat these steps to disable the "IBM: AIX IC Filesystem Performance" Dynamic Application.

Next, you will need to enable the "IBM: AIX ICDA Cache" and "IBM: AIX Filesystem" Dynamic Applications:

1. In the Device Investigator page for the IBM AIX device, go to the Collections tab.

2. In the Collections tab, click the Edit button.
3. Locate the "IBM: AIX ICDA Cache" Dynamic Application, click its Actions icon (), and select Enable Collection.
4. Click the Save button.
5. Repeat these steps to enable the "IBM: AIX Filesystem" Dynamic Application.

Disabling the "IBM: AIX IC Filesystem Inventory" and "IBM: AIX IC Filesystem Performance" Dynamic Applications will result in no filesystem data collection in the Hardware page and hiding the filesystem option will not be available.

Enabling both ICDA and non-ICDA Dynamic Applications will result in duplicate filesystem alerts.

Changing the Collection Commands for CPU Utilization

Some IBM AIX administrators prefer to calculate CPU utilization using the SAR command as opposed to the MPSTAT command. SL1 administrators can chose to use the approach that best fits their use case.

There are two CPU utilization Dynamic Applications in the PowerPack:

• IBM: AIX CPU Utilization (mpstat)
• IBM: AIX CPU Utilization (sar)

The "IBM: AIX CPU Utilization (sar)" and "IBM: AIX CPU Utilization (mpstat) Dynamic Applications and their alerts are disabled by default and cannot be aligned using the device template. To enable the "IBM: AIX CPU Utilization (sar)" and "IBM: AIX CPU Utilization (mpstat)" Dynamic Applications:

1. Ensure that the user has the proper privileges to run the SAR command.
2. Go to the Dynamic Applications Manager page (System > Manage > Dynamic Applications).
3. Locate the "IBM: AIX CPU Utilization (sar)"/"IBM: AIX CPU Utilization (mpstat)" Dynamic Application and click its wrench icon ().
4. In the Operational State dropdown field, select Enabled. Click Save.
5. Go to the Event Policies page (Events > Event Policies) and enable the following events:

• IBM: AIX Logical CPU Utilization % has exceeded threshold
• IBM: AIX Logical CPU Utilization % has returned below threshold

6. Go to the Alerts tab for both Dynamic Applications and enable their alerts.
7. The "IBM: AIX CPU Utilization (sar)" and "IBM: AIX CPU Utilization (mpstat)"  Dynamic Applications can now be manually aligned to individual devices.

Updating Virtual Memory Alerts

The "IBM: AIX Memory Stats" Dynamic Application has virtual memory events enabled by default. You must ensure that there are no other previously built custom PowerPacks with virtual memory events running to use these events.

If you are using the AIX Virtual Memory Utilization Monitoring PowerPack, you must disable the alerts and events in that PowerPack or disable the alerts in the "IBM: AIX Memory Stats" Dynamic Application in the IBM: AIX Monitoring PowerPack.

To disable these events:

1. Go to the Dynamic Applications Manager page (System > Manage > Dynamic Applications) .
2. Locate the "IBM: AIX Memory Stats" Dynamic Application and click its wrench icon ().
3. Click the Alerts tab and locate the "IBM: AIX Virtual Memory Utilization Critical" alert and click its wrench icon ().
4. In the Active State dropdown, select Disabled. Click Save.
5. Repeat the above steps for the following event policies:

• IBM: AIX Virtual Memory Utilization Healthy
• IBM: AIX Virtual Memory Utilization Major
• IBM: AIX Virtual Memory Utilization not Critical