Configuration and Discovery

Download this manual as a PDF file

The following sections describe how to configure Google Cloud Platform resources for monitoring by SL1 using the Google Cloud Platform PowerPack:

Creating a Google Cloud Platform Service Account

To monitor Google Cloud Platform (GCP) resources with SL1, you must first create a GCP service account for SL1 in the GCP Console. This service account belongs to SL1 instead of an individual end user, and enables SL1 to communicate with Google APIs when monitoring your GCP resources.

This service account's credentials will include a unique email address and a secret JSON key. You will include this email address and key information when you create the SOAP/XML credential that enables SL1 to monitor your GCP resources.

To create a GCP service account:

  1. Log in to the GCP Console and go to the Service accounts page. If prompted, select a project.

  1. Click the CREATE SERVICE ACCOUNT button.

  1. Complete the following fields on the Create service account page:

  • Service account name. Type a name for the service account.

  • Service account ID. This field auto-populates with a service account ID that is based on your Service account name.
  • Service account description. Type a description for the service account.
  1. Click Create. Your service account is created, and the Service account permissions page displays.

  1. Complete the following fields on the Service account permissions page:

  • Role. Select Project > Viewer.

At a minimum, the service account must have a role of "Project" with "Viewer" permissions for the GCP service that you want to monitor.

  1. Click Continue. The Grant users access to this service account page displays.
  2. Click Create Key. The Create key pane appears.

  1. On the Create key pane, select the JSON radio button and then click Create. The private JSON key is saved to your computer.

  1. Click Close, and then click Done.

  1. Open the JSON file that was downloaded to your computer and copy the following information:
  • client_email
  • private_key

When you copy the private key from the JSON file, it must include the "BEGIN PRIVATE KEY" and "END PRIVATE KEY" lines, including all leading and ending dashes.

If you are discovering GCP resources at the Project level, then you can skip the following steps and continue on to the Enabling Google Cloud APIs section.

However, if you are discovering GCP resources at the Organization level, then you must also do the following:

  1. In the GCP Console, go to the IAM page and select your organization.
  2. Click Add.
  3. Add your service account as a member of the organization, and then add the following mandatory roles:
  • Role > Project > Viewer
  • Role > Resource Manager >Folder Viewer
  • Role > Resource Manager > Organization Viewer
  1. When you are finished, click Save.

Enabling Google Cloud Platform APIs

Before SL1 can monitor GCP, you must also enable two APIs in the GCP portal:

  • Cloud Resource Manager API
  • Compute Engine API

To enable these GCP APIs:

  1. Log in to the GCP Console for your project and go to the API & Services Dashboard page.

  1. Click ENABLE APIS AND SERVICES. The API Library page appears.

  1. In the search bar, type "Cloud Resource Manager API". The page will filter search results while you type.

  1. Click the Cloud Resource Manager API box.

  1. On the Cloud Resource Manager API page, click the Enable button.
  2. Click Dashboard on the API & Services left menu and then repeat steps 2-5 to enable the Compute Engine API.

Dynamic Applications that call Google's monitoring APIs for time-series based data (ScienceLogic performance applications) may experience delay. Google's API is subject to delay that exceed the expected data retrieval latency.

Creating a SOAP/XML Credential for Google Cloud Platform

To configure SL1 to monitor GCP, you must create a SOAP/XML credential that allows the Dynamic Applications in the Google Cloud Platform PowerPack to connect with your GCP service. An example SOAP/XML credential that you can edit for your own use is included in the Google Cloud Platform PowerPack.

NOTE: If you are using an SL1 system prior to version 11.1.0, the new user interface does not include the Duplicate option for sample credential(s). ScienceLogic recommends that you use the classic user interface and the Save As button to create new credentials from sample credentials. This will prevent you from overwriting the sample credential(s).

To define a SOAP/XML credential:

  1. Go to the Credentials page (Manage > Credentials).
  2. Locate the GCP: SOAP Credential Example sample credential, click its Actions icon () and select Duplicate. A copy of the credential, calledGCP: SOAP Credential Example copy appears.
  3. Click the Actions icon () for the GCP: SOAP Credential Example copy credential and select Edit. The Edit Credential modal page appears.

    4. An image of the Edit Credential page.

  1. Supply values in the following fields:
  • Name. Type a new name for your Google Cloud credential.

SOAP Options

  • Embedded Password [%P]. Paste the "private_key" value from the private key JSON file.
  • Embed Value [%1]. Type the "client_email" value from the private key JSON file. For example: myprojectid@myaccount.iam.gserviceaccount.com.

When you copy the "private_key" from the JSON file, it must include the "BEGIN PRIVATE KEY" and "END PRIVATE KEY" lines, including all leading and ending dashes.

  1. For all remaining fields, use the default values.
  2. Click Save & Close .

The SOAP/XML credential tester is not supported by the Google Cloud Platform PowerPack

Creating a SOAP/XML Credential in the Classic SL1 User Interface

To configure SL1 to monitor GCP, you must create a SOAP/XML credential that allows the Dynamic Applications in the Google Cloud Platform PowerPack to connect with your GCP service. An example SOAP/XML credential that you can edit for your own use is included in the Google Cloud Platform PowerPack.

To create a SOAP/XML credential to access GCP:

  1. Go to the Credential Management page (System > Manage > Credentials).

  1. Locate the GCP: SOAP Credential Example and then click its wrench icon (). The Edit SOAP/XML Credential modal page appears.

  1. Complete the following fields:

Basic Settings

  • Profile Name. Type a new name for the credential.

SOAP Options

  • Embedded Password [%P]. Paste the "private_key" value from the private key JSON file.
  • Embed Value [%1]. Type the "client_email" value from the private key JSON file. For example: myprojectid@myaccount.iam.gserviceaccount.com.

When you copy the "private_key" from the JSON file, it must include the "BEGIN PRIVATE KEY" and "END PRIVATE KEY" lines, including all leading and ending dashes.

  1. For all remaining fields, use the default values.
  2. Click the Save As button, and then click OK.

Creating a Google Cloud Platform Virtual Device

Because the GCP service does not have a static IP address, you cannot discover GCP devices using a regular discovery session. Instead, you must create a virtual device that represents the GCP service. A virtual device is a user-defined container that represents a device or service that cannot be discovered by SL1. You can use the virtual device to store information gathered by policies or Dynamic Applications.

To create a virtual device that represents your GCP service:

  1. Go to the Device Manager page (Devices > Classic Devices, or Registry > Devices > Device Manager in the classic SL1 user interface).

  1. Click the Actions button and select Create Virtual Device from the menu. The Virtual Device modal page appears.

  1. Complete the following fields:
  • Device Name. Type a name for the device.

  • Organization. Select the organization for this device. The organization you associate with the device limits the users that will be able to view and edit the device. Typically, only members of the organization will be able to view and edit the device.
  • Device Class. Select GCP | Service.
  • Collector. Select the collector group that will monitor the device.
  1. Click Add to create the virtual device.

Aligning the Google Cloud Platform Dynamic Applications

The Dynamic Applications in the Google Cloud Platform PowerPack are divided into the following types:

  • Discovery. These Dynamic Applications poll GCP for new instances of services or changes to existing instances of services.
  • Configuration. These Dynamic Applications retrieve configuration information about each service instance and retrieve any changes to that configuration information.
  • Performance. These Dynamic Applications poll GCP for performance metrics.

When configuring SL1 to monitor GCP services, you must manually align Dynamic Applications to discover GCP component devices.

Discovering Google Cloud Platform Component Devices

To discover all the components of your GCP service, you must manually align two Dynamic Applications with the GCP virtual device. The specific Dynamic Applications that you must align to the virtual device vary based on whether you are discovering GCP resources from the Organization level or the Project level.

  • If you are discovering an Organization, you must align the following Dynamic Applications:
  • GCP: Token
  • GCP: Organization Discovery

  • If you are discovering GCP resources from the Project level, you must align the following Dynamic Applications:
  • GCP: Token
  • GCP: Project Discovery

To manually align these Dynamic Applications:

  1. Go to the Device Manager page (Devices > Classic Devices, or Registry > Devices > Device Manager in the classic SL1 user interface).

  1. Click the wrench icon () for your GCP virtual device.
  2. In the Device Administration panel, click the Collections tab. The Dynamic Application Collections page appears.
  3. Click the Actions button and select Add Dynamic Application from the menu.

  1. In the Dynamic Application Alignment modal:
  • In the Dynamic Applications field, select GCP Token.
  • In the Credentials field, select the credential you created for your GCP service.

  1. Click Save to align the Dynamic Application with the GCP virtual device.
  2. Repeat steps 2-6 to align the "GCP Project Discovery" or "GCP Project Discovery" Dynamic Application, depending on whether you are discovering an Organization or a Project.

You must align the "GCP: Token" Dynamic Application before you align the "GCP: Organization Discovery" or "GCP: Project Discovery" Dynamic Application.

When you align the Dynamic Applications with the virtual device representing your GCP service, SL1 creates a component device representing your GCP Organization or Project.

SL1 then automatically aligns several other Dynamic Applications to that component device. These Dynamic Applications discover and create additional component devices representing your GCP resources.

SL1 might take several minutes to align these Dynamic Applications and create the component devices in your GCP service.

Configuring the GCP: BigQuery DataSet Discovery Dynamic Application

The "GCP: BigQuery DataSet Discovery" Dynamic Application is set by default to discover only visible datasets. You can optionally edit the Dynamic Application to discover visible and hidden datasets by updating the 'ALL' value in the Dynamic Application snippet to "true".

To edit the snippet:

  1. Go to the Dynamic Applications Manager page (System > Manage > Applications).
  2. Find the "GCP: BigQuery DataSet Discovery" Dynamic Application and click its wrench icon ().
  3. In the Dynamic Applications Properties Editor, click the Snippets tab.
  4. In the Dynamic Applications Snippet Editor & Registry page, click the wrench icon () of the "GCP: BigQuery DataSet Discovery" snippet.
  5. The content of the snippet will appear. Edit the 'false' value in the following snippet text to 'true':

subs ={

'PROJECT_ID':project_id,

'ALL':'false' <----- <FLAG>

}

 

  1. Click Save.

Viewing Google Cloud Platform Component Devices

In addition to the Device Manager page (Devices > Classic Devices, or Registry > Devices > Device Manager in the classic SL1 user interface), you can view the GCP service and all associated component devices in the following places in the user interface:

  • The Device View modal page (click the bar-graph icon [] for a device, then click the Topology tab) displays a map of a particular device and all of the devices with which it has parent-child relationships. Double-clicking any of the devices listed reloads the page to make the selected device the primary device.

  • The Device Components page (Registry > Devices > Device Components) displays a list of all root devices and component devices discovered by SL1 in an indented view, so you can easily view the hierarchy and relationships between child devices, parent devices, and root devices. To view the component devices associated with the GCP service, find the GCP virtual device and click its plus icon (+).

  • The Component Map page (Classic Maps > Device Maps > Components) allows you to view devices by root node and view the relationships between root nodes, parent components, and child components in a map. This makes it easy to visualize and manage root nodes and their components. SL1 automatically updates the Component Map as new component devices are discovered. The platform also updates each map with the latest status and event information. To view the map for the GCP service, go to the Component Map page and select the map from the list in the left NavBar. To learn more about the Component Map page, see the section on Views.

Relationships Between Component Devices

In addition to parent/child relationships between component devices, SL1 also creates relationships between the following component devices:

  • Compute Instances and Storage Persistent Disks
  • Compute Instances and Subnets
  • Compute Instances and VPC Networks
  • Load Balancing Global HTTPS and Backend Buckets
  • Load Balancing Global HTTPS and Backend Services
  • Load Balancing Global HTTPS and Default Backend Services
  • Load Balancing Global SSL Proxy and Backend Services
  • Load Balancing Global TCP Proxy and Backend Services
  • Load Balancing Regional Network TCP/UDP and Compute Instances
  • VPC Subnets and VPC Networks

If an instance is configured in GCP to automatically delete any associated read-write persistent disks when the instance is deleted, then that behavior will also occur in SL1: If the instance is deleted, its related persistent disks will also be deleted. This behavior is controlled in GCP on the VM Instances page by the Delete boot disk when instance is deleted checkbox for boot disks and the When deleting instance field for additional disks.

Additionally, SL1 can also build relationships between GCP VM Instances and Kubernetes Nodes, for users who also have the Kubernetes PowerPack installed.