Amazon Web Services PowerPack Release Notes, version 123

Version 123 of the Amazon Web Services PowerPack adds new Dynamic Applications and device classes, improves region discovery, and adds monitoring for a number of new services including services in the new AWS Extended Services PowerPack.

  • Minimum Required SL1 Version: 10.2.0

Before You Install or Upgrade

Ensure that you are running version 10.2.0 or later of SL1 before installing "Amazon Web Services" PowerPack version 123.

For details on upgrading SL1, see the relevant SL1 Platform Release Notes.

If your SL1 system is not currently running version 10.2.0 or later, you must upgrade to 10.2.0 or later as part of the upgrade process for version 123 of the PowerPack.

If your SL1 system has been upgraded to a later release, you should go to the PowerPack Manager page (System > Manage > PowerPacks) and ensure that the Amazon Web ServicesPowerPack has been upgraded to the most recent version.

Additionally, the Data Collectors used to monitor the AWS account must be running the Oracle Linux 7.2 operating system.

More AWS services are available to be monitored by installing the AWS Extended Services PowerPack.

Upgrade Process from PowerPack version 100 or Later

This section describes the upgrade process when upgrading from version 100 or later of the Amazon Web Services PowerPack.

By default, installing a new version of a PowerPack will overwrite all content in that PowerPack that has already been installed on the target system. You can use the Enable Selective PowerPack Field Protection setting in the Behavior Settings page (System > Settings > Behavior) to prevent local changes for some commonly customized fields from being overwritten.

To upgrade from version 100 or later of the Amazon Web Services PowerPack:

  1. Familiarize yourself with the Known Issues for this release.
  2. Disable collection for AWS devices. To do so, go to the Device Components page (Registry > Devices > Device Components) and then select the checkbox for all Amazon Web Services root devices. In the Select Actions drop-down list, select Change Collection State: Disabled (recursive), and then click the Go button.
  3. If you are upgrading from a version of the Amazon Web Services PowerPack prior to this version, you must ensure that the Preserve Hostname checkbox is not selected for any Storage Gateway Instances. To do so, go to the Device Manager page (Registry > Devices > Device Manager) and then type "AWS | Storage Gateway Instance" in the Device Class | Sub-class column filter field. For each Storage Gateway Instance component device in the list, click the wrench icon (), uncheck the Preserve Hostname checkbox, and then click Save.

If desired, you can select the Preserve Hostname checkbox again for these devices after you have completed the upgrade to this version of the PowerPack.

  1. Delete the "AWS: Health" Dynamic Application. To do so, go to the Dynamic Applications Manager page (System > Manage > Applications) and then select the checkbox for the "AWS: Health" Dynamic Application. In the Select Actions drop-down list, select DELETE Application, and then click the Go button.

You should not delete any of the AWS Service Health Dynamic Applications, nor any other Dynamic Applications that include "Health" in the name. Only the "AWS: Health" Dynamic Application should be deleted.

  1. If you have not done so already, upgrade your version SL1 system to 10.2.0 or later.

For versions 8.6.0 and later of SL1, the Amazon Web Services PowerPack content library will not update until you enable collection for your AWS devices.

  1. If you are upgrading from a version of the Amazon Web Services PowerPack between versions 104 and 107, you must delete any LightSail Instances that were previously discovered by the "AWS: LightSail EC2 Instance Discovery" Dynamic Application. To do so, go to the Device Manager page (Registry > Devices > Device Manager), type "LightSail EC2 Instance" in the Device Class | Sub-class column search field, and then select the checkboxes for all of the devices listed. In the Select Action drop-down list, select DELETE Selected Devices, and then click the Go button.

Deleting these devices results in the loss of any historical data collected by the beta EC2 LightSail Dynamic Applications between versions 104 and 107.

  1. Download the latest version of the Amazon Web Services PowerPack from the Support Site to a local computer.
  2. Go to the PowerPack Manager page (System > Manage > PowerPacks). Click the Actions menu and choose Import PowerPack. When prompted, import the Amazon Web ServicesPowerPack.
  3. After importing the PowerPack, you will be prompted to install the PowerPack. Click the Install button to install the PowerPack.
  4. If you have updated to this version of the PowerPack and already have an account discovered that is part of an organization, you can use your AssumeRole to have your devices associated with that organization. After you upgrade and the "AWS: Account Creation" run book action runs, you'll see the account name change to reflect the organization account in the Device Manager and Device Components pages.

ScienceLogic recommends turning off collection of the parent device if you are upgrading to use this new feature.

Upgrade Process from PowerPack version 8.1.0 or Earlier

This section describes the upgrade process when upgrading from version 8.1.0 or earlier of the Amazon Web Services PowerPack to this version.

To upgrade from version 8.1.0 or earlier, you must perform the following general steps:

  1. Disable collection for AWS devices.
  2. Uncheck "Preserve Hostname" for Storage Gateway Instances.
  3. Delete the "AWS: Health" Dynamic Application.
  4. Upgrade to SL1 10.2.0 or later release.
  5. If you have made changes to the AWS PowerPack, optionally enable selective PowerPack field protection.
  6. Install this release of the AWS PowerPack.
  7. If you enabled selective PowerPack field protection, edit collection objects.
  8. Clear the cache on all Data Collectors.
  9. Unalign the AWS: Custom Metrics Dynamic Application.
  10. Enable collection for AWS devices.
  11. If you enabled selective PowerPack field protection, optionally disable selective PowerPack field protection after the installation.

Step 1: Disable Collection for AWS Devices

To disable collection for AWS devices:

  1. Go to the Device Components page (Registry > Devices > Device Components).

  1. Select the checkbox for all Amazon Web Services root devices.
  2. In the Select Actions drop-down list, select Change Collection State: Disabled (recursive).
  3. Click the Go button.

Step 2: Uncheck the "Preserve Hostname" Option for Storage Gateway Instances

To ensure that the Preserve Hostname checkbox is not selected for any Storage Gateway Instance component devices:

  1. Go to the Device Manager page (Registry > Devices > Device Manager).
  2. In the Device Class | Sub-class column filter field, type "AWS | Storage Gateway Instance".
  3. For each Storage Gateway Instance component device in the list, click the wrench icon (). The Device Properties page appears.
  4. Uncheck the Preserve Hostname checkbox.
  5. Click Save.

If desired, you can select the Preserve Hostname checkbox again for these devices after you have completed the upgrade to this version.

Step 3: Delete the "AWS: Health" Dynamic Application

To delete the "AWS Health" Dynamic Application:

  1. Go to the Dynamic Applications Manager page (System > Manage > Applications).
  2. Locate the "AWS: Health" Dynamic Application and then select its checkbox.
  3. In the Select Actions drop-down list, select DELETE Application.
  4. Click the Go button.

You should not delete any of the AWS Service Health Dynamic Applications, nor any other Dynamic Applications that include "Health" in the name. Only the "AWS: Health" Dynamic Application should be deleted.

Step 4: Upgrade to the 10.2.0 or Later Release

If you have not previously done so, upgrade or migrate your SL1 system to an 10.2.0 or later release using the documentation applicable to your current version:

  • For systems running an 8.x release, see the 10.2.0 Release Notes.
  • For systems running a 7.x release, see the 10.2.0 Migration Steps document.

For versions 8.6.0 and later of the ScienceLogic platform, the Amazon Web Services PowerPack content library will not update until you enable collection for your AWS devices.

Step 5 (Optional): Enable Selective PowerPack Field Protection

If you have made changes to the Amazon Web Service PowerPack on your system, you can use the Enable Selective PowerPack Field Protection option to preserve changes to some fields. For a full list of fields that are preserved by this option, click the Guidebutton on the Behavior Settings page (System > Settings > Behavior). If you use the Enable Selective PowerPack Field Protection option, you must perform the steps listed in the Step 7 (If Applicable): Edit Collection Objects section after installing this version of the Amazon Web Services PowerPack.

To enable selective PowerPack field protection:

  1. Go to the Behavior Settings page (System > Settings > Behavior).
  2. Enable the Enable Selective PowerPack Field Protection checkbox.
  3. Click the Save button.

Step 6: Install this Version of the Amazon Web Services PowerPack

To install this version of the Amazon Web Services PowerPack:

  1. Go to the PowerPack Manager page (System > Manage > PowerPacks).

  1. Click the Actions button and select Import PowerPack. The Import PowerPack modal page appears.
  2. Click the Browse button and select the PowerPack file in your file browser.
  3. Click the Import button. A confirmation dialog appears.
  4. Click the OK button. The PowerPack Installer modal page appears.
  5. Click the Install button. A confirmation dialog appears.
  6. Click the OK button.

Step 7 (If Applicable): Edit Collection Objects

If you performed the steps listed in the Step 5 (Optional): Enable Selective PowerPack Field Protection section, you must perform the following steps:

  1. Go to the Dynamic Applications Manager page (System > Manage > Application).
  2. Click the wrench icon () for the "AWS: CloudFront Origin Configuration" Dynamic Application. The Dynamic Applications Properties Editor page appears.

  1. Click the Collections tab.
  2. Click the wrench icon () for the Distinguished Name collection object.
  3. Select the Hide Object checkbox.
  4. Click the Save button.

Step 8: Clear Data Collector Cache

Beginning with Amazon Web ServicesPowerPack version 112, ScienceLogic libraries are delivered with the PowerPack. However, you must ensure that Amazon Web ServicesPowerPack versions 108-111 that were installed on Data Collectors prior to updating to SL1 version 8.11.0 still work and that the silo_aws system library is set to 2.9.5.

All versions of the ScienceLogic libraries appear on the ScienceLogic Library Manager page (System > Customize > ScienceLogic Libraries). The default silo_aws library is indicated by a System setting of True.

After validating the ScienceLogic library versions on the Data Collectors, you must also clear the Data Collectors' cache.

Perform the following steps for every Data Collector used to monitor an AWS account:

  1. Log in to the command-line of the appliance as the em7admin user.
  2. Validate the versions of the ScienceLogic libraries on the Data Collector at the paths below:

    3. $ cd /opt/em7/envs/<PP GUID>/lib/python2.7/cl-packages/silo_aws

    $ cd /opt/em7/envs/system/lib/python2.7/cl-packages/silo_aws

    • The library versions might be the same if you have not yet upgraded the Amazon Web ServicesPowerPack to the latest version that was delivered in SL1.
    • The library versions might be different if you have upgraded the Amazon Web ServicesPowerPack to the latest version that was delivered in SL1.

  1. On the Data Collector, ensure that the current content library version matches the version installed in the PowerPack:

    2. $ cat version.txt

For versions 8.6.0 and above of SL1, the content library version listed in the version.txt file will not update until you enable collection for your AWS devices.

  1. Execute the following command to open a MariaDB prompt:

    2. $ sudo bash

    [sudo] password for root:

    # silo_mysql

  1. Execute the following command:

    2. DELETE FROM cache.dynamic_app WHERE `key` LIKE 'AWS_SELF_MONITOR_%';

Step 9: Unalign the AWS: Custom Metrics Dynamic Application

A previous release of the Amazon Web Services PowerPack erroneously aligned the "AWS: Custom Metrics" Dynamic Application to certain types of devices. To unalign the "AWS: Custom Metrics" Dynamic Application from these devices:

  1. Copy the provided aws_unalign_custom_metrics_app.py file to the home directory of the em7admin user on an appliance in your system:
  • If your system includes All-In-One Appliances, use the primary All-In-One Appliance.

  • If your system includes Database Servers where the user interface/API has not been disabled on the Database Servers, use the primary Database Server.
  • If your system includes Database Servers where the user interface/API has been disabled on the Database Servers, use an Administration Portal.

The aws_unalign_custom_metrics_app.py file can be found by clicking the "Contrib Files" link for the most recent version of the Amazon Web Services PowerPack at the ScienceLogic Support site.

  1. Log in to the command-line of the appliance as the em7admin user.

  1. Execute the following command:

    2. sudo python aws_unalign_custom_metrics_app.py --base-url http://[IP address of appliance] --username [username of administrator user] --password [password of administrator user]

    The output will show information about each device from which the "AWS Custom Metrics" Dynamic Application was unaligned.

Step 10: Enable Collection for AWS Devices

To enable collection for AWS devices:

  1. Go to the Device Components page (Registry > Devices > Device Components).
  2. Select the checkbox for all AWS Web Services root devices.
  3. In the Select Actions drop-down list, select Change Collection State: Enabled (recursive).
  4. Click the Go button.

Step 11 (Optional): Disable Selective PowerPack Field Protection

If you performed the steps listed in the Step 5 (Optional): Enable Selective PowerPack Field Protection section and want to disable the option for future PowerPack updates, perform the following steps:

  1. Go to the Behavior Settings page (System > Settings > Behavior).
  2. Disable the Enable Selective PowerPack Field Protection checkbox.
  3. Click the Save button.

Features

This release includes the following features:

  • Dynamic Applications that discover, model, and collect data from AWS component devices
  • Event Policies and corresponding alerts that are triggered when AWS component devices meet certain status criteria

Many of the Event Policies included in this PowerPack are disabled by default. You must manually enable the Event Policies that you want to use. To do so, go to the Event Policy Editor page (Registry > Events > Event Manager > create or edit) and change the Operational State to Enabled.

  • Device Classes for each of the AWS component devices monitored
  • Sample credentials for discovering AWS component devices
  • Reports and dashboards that display information about AWS instances and component devices
  • Run Book Action and Automation policies that can automate certain AWS monitoring processes
  • The ScienceLogic Libraries that are utilized by this PowerPack:
  • aws_explorer
  • boto3
  • content
  • recordreplay
  • silo_apps
  • silo_aws
  • silo_aws_args
  • silo_credentials
  • silo_snippet
  • silo_vmware
  • urllib3

Enhancements and Issues Addressed

The following enhancements and addressed issues are included in version 123 of the "Amazon Web Services" PowerPack:

  • The following Dynamic Applications were added to the PowerPack:
  • AWS: Account Service Inventory
  • AWS: Services Discovery
  • AWS: Service Resources
  • Added service discovery code to snippets in Instance Discovery Dynamic Applications for the following services:
  • API Gateway
  • Auto Scale
  • Cloud Trail
  • Direct Connect
  • DynamoDB
  • ECS
  • Elastic Beanstalk
  • Elasticache
  • ELB/ELB2
  • EMR
  • Glacier
  • KMS (added to the snippet code in the "AWS: KMS Configuration" Dynamic Application)
  • Lambda
  • S3
  • SNS
  • SQS
  • Storage Gateway
  • VPC
  • WAF Regional
  • The following device classes were added to the PowerPack:
  • AmazonMQ Service
  • App Runner Service
  • AppStream Service
  • AppSync Service
  • Athena Service
  • Chime Service
  • CloudHSMv2 Service
  • CloudSearch Service
  • CloudWatch RUM Service
  • CloudWatch Synthetics Service
  • CodeBuild Service
  • Cognito Identity Service
  • Connect Service
  • DAX Service
  • DMS Service
  • DocDB Service
  • Elastic Transcoder Service
  • ELB Service
  • Elemental MediaConnect Service
  • Elemental MediaConvert Service
  • Elemental MediaPackage Live Service
  • Elemental MediaPackage Service
  • Elemental MediaPackage Vod Service
  • Elemental MediaTailor Service
  • FSx Service
  • GameLift Service
  • Glue Service
  • GuardDuty Service
  • HealthLake Service
  • Inspector2 Service
  • IoTAnalytics Service
  • IoTSiteWise Service
  • IoTThings Graph Service
  • IoTTwinMaker Service
  • Kafka Service
  • KeySpaces Service
  • Kinesis Data Firehose Service
  • Kinesis Service
  • Kinesis Video Streams Service
  • Lexv2 Service
  • MQ Service
  • Neptune Service
  • Network Firewall Service
  • OpenSearch Service
  • Polly Service
  • Secrets Manager Service
  • SWF Service
  • TimeStream Write Service
  • Transfer Family Service
  • WorkMail Service
  • WorkSpaces Web Service
  • Region discovery has been updated to discover only regions that have active metrics. Users can restrict discovery of specific regions in the Regions header of the credential.
  • Icons have been added for device classes in the classic and current SL1 user interfaces.
  • Removed unnecessary logging of error messages generated by failures of the CloudWatch API.
  • The "AWS: EKS Virtual Device Creation" event policy was added to the PowerPack.
  • The "AWS: Regional Services Dynamic App Alignment" Run Book action, Run Book automation policy, and event policy were added to the PowerPack.
  • Updated the "AWS: Service API Request Health" Dynamic Application to collect and count specific error codes of failed API requests.
  • The Collector Affinity field of the "AWS: Custom Metrics Performance" Dynamic Application was changed to Root device collector by default due to being a cache consumer.
  • The "silo_aws_args" and the "silo_aws" content libraries were updated to use "silo_logs".
  • The following Dynamic Applications were removed from the PowerPack and replaced with the "AWS: Services Discovery" Dynamic Applicationn:
  • AWS: API Gateway Service Discovery
  • AWS: Auto Scale Service Discovery
  • AWS: CloudTrail Service Discovery
  • AWS: CloudWatch Service Discovery
  • AWS: DDB Service Discovery
  • AWS: Direct Connect Service Discovery
  • AWS: ECS Service Discovery
  • AWS: EFS Service Discovery
  • AWS: EKS Service Discovery
  • AWS: Elastic Beanstalk Service Discovery
  • AWS: ElastiCache Service Discovery
  • AWS: ELB Service Discovery
  • AWS: EMR Service Service
  • AWS: Glacier Service Discovery
  • AWS: IoT Service Discovery
  • AWS: KMS Service Discovery
  • AWS: Lambda Service Discovery
  • AWS: RDS Aurora Service Discovery
  • AWS: S3 Service Discovery
  • AWS: Security Discovery
  • AWS: SES Service Discovery
  • AWS: SNS Service Discovery
  • AWS: SQS Service Discovery
  • AWS: Storage Gateway Service Discovery
  • AWS: Transit Gateway Service Discovery
  • AWS: VPC Service Discovery
  • AWS: WAF Regional Service Discovery
  • AWS: Workspaces Service Discovery
  • The Usage Type field in the "AWS: EFS File System Configuration" Dynamic Application was updated to Standard by default to address an issue in which the Dynamic Application was mapping incorrect relationships. (Support Cases: 00259988 and 00260939; Jira ID: SOL-19408)
  • The "AWS: Account Creation" run book automation policy was updated to address an issue in which it was incorrectly assigning commercial AWS-formatted ARNs to accounts with AWS Gov Cloud devices.
  • A filter has been added to the PowerPack to prevent other database engines from being discovered on the same RDS service.

If you are still using the deprecated Aurora Engine, discovered MySQL 5.6 databases should still collect data in regions where the engine is still available. However, the discovery of new clusters with the new filter will not be possible until the cluster has been updated to MySQL 5.7.

  • The formula of the "AWS: RDS Free Storage Space Has Returned To Normal" alert in the "AWS: RDS Instance Performance" Dynamic Application was updated to address an issue in which it was returning incorrect data. (Support Case: 00254509; Jira ID: SOL-19075)
  • The "silo_aws_args" content library was updated to address an issue which some performance metrics were returned as "0" when a "None" value was returned in CloudWatch.
  • An issue was addressed in the "AWS: KMS Performance" and "AWS: KMS Configuration" Dynamic Applications in which an error was occurring when the cache was empty.
  • The "AWS: Get Devices and Apps Information" Run Book action was added to the PowerPack.
  • Cleaned up the Password field in example credentials.
  • The following content libraries were updated to the following versions:
  • aws_explorer 0.3.7d
  • silo_aws 6.6.8d
  • silo_aws_args 1.4.9d

Known Issues and Workarounds

The following known issues affect version 123 of the Amazon Web Services PowerPack:

  • A reported bug in AWS is causing data gaps in data collected from Aurora RDS cluster volumes on SL1. This issue will be fixed in the next version.

  • When discovering Athena devices, you cannot create Workgroups that have the same name even if the case of the letters is different, for example: "Primary" and "primary". These will be recognized duplicate Workgroups and only one device component will be created between them.
  • Amazon Aurora MySQL-Compatible Edition version 1 (with MySQL 5.6 compatibility) will reach end of life on February 28, 2023. It is recommended to upgrade to newer supported versions before this date.
  • A known issue in the "silo_log" content library may cause a "No such file or directory" error to appear in system logs.
  • Upgrading an IAM account to use Assume Role may cause child device components to have conflicts with parent devices. To prevent this, disable the "AWS: Account Discovery" Dynamic Application before you upgrade the IAM account.
  • To prevent the discovery of DocDB and Neptune databases inside regional RDS services, which will not collect data or align Dynamic Applications, do not enable new permissions for services in the AWS Extended Services PowerPack before installing version 123 of the Amazon Web Services PowerPack.
  • Relationship creation for the following relationships has been disabled temporarily until a fix can be implemented for a circular relationship issue: (Support Case: 00271522; Jira ID: SOL-20134)
  • AWS VPC instances in the same region
  • AWS VPC Instances and other intra-account AWS VPC Instances
  • The following Run Book Actions may fail to run for Assume Role and IAM accounts in SL1 MUD systems:
  • AWS: Discover from EC2 IP
  • AWS: Disable Instance By Tag
  • AWS: Vanish Terminated EC2 Instances
  • In AWS Dynamic Applications that produce API results containing special characters, if a character cannot be normalized, SL1 will display its Unicode values inside brackets as a normalized string.
  • Some disk-related alerts and events were removed from the "AWS: LightSail Instance Performance" Dynamic Application as of Amazon Web ServicesPowerPack version 108. If you are upgrading from a version prior to version 108, then you must manually delete the thresholds relating to these removed alerts and events. To do so, go to the Dynamic Applications Threshold Objects page (System > Manage > Applications > wrench icon > Thresholds) for the "AWS: LightSail Instance Performance" Dynamic Application, and then click the bomb icon () for the following thresholds:
  • AWS: LightSail Disk IOPS High
  • AWS: LightSail Disk GB Usage High
  • AWS does not currently support IPv6 addresses for LightSail services. However, the "AWS: LightSail Instance Configuration" Dynamic Application includes support for IPv6 addresses in the event that AWS adds support in the future.
  • SSL EOF error messages might appear in the system log when connecting to AWS through a proxy server. The error does not seem to prevent or cause issues with data collection.
  • "Read operation timed out" and "Connection reset by peer" error messages might appear in the system log and device logs when upgrading the Amazon Web ServicesPowerPack from versions prior to 108.