Configuring PowerFlow Applications for the ELK Stack SyncPack

Download this manual as a PDF file 

This section describes how to set up the run book automations in SL1, and how to set up the PowerFlow applications for the "ELK Stack" SyncPack.

Workflow for Configuring the SyncPack

Configuring SL1

  1. Enable the ELK Stack event policies
  2. Create a SOAP/XML credential to access PowerFlow
  3. Edit the ELK Stack run book actions
  4. Enable the ELK Stack run book automations

Configuring PowerFlow

  1. Create a PowerFlow configuration object
  2. Align and configure the ELK Stack PowerFlow applications
  3. Schedule the PowerFlow applications

Configuring SL1

Enabling the ELK Stack Event Policies

The "ELK Stack Automation" PowerPack includes the following event policies that you can enable to trigger the events detected by the applications included in the "ELK Stack" SyncPack:

  • "ELK ExternalLog"
  • "ELK: Sync SL1 Log"
  • "ELK: Search"

To enable the event policies:

  1. Go to the Event Policies page (Events > Event Policies).
  2. Click the check boxes for all three event policies. A row of buttons appear at the top of the list.
  3. Click Enable. The Status for all three policies is updated to "Enabled".

Creating a SOAP/XML Credential to Access PowerFlow

You will need to create a SOAP/XML credential so that the action policies included in the "ELK Stack Automation" PowerPack can access your PowerFlow system.

To create a SOAP/XML credential:

  1. In SL1, go to the Credentials page (Manage > Credentials).
  2. Locate the "ELK PowerFlow" sample credential, then click its Actions icon () and select Duplicate. A copy of the credential, called ELK PowerFlow copy appears.
  3. Select the new credential and complete the following fields:
  • Name. Type a new name for the credential.
  • All Organizations. Toggle on (blue) to align the credential to all organizations, or toggle off (gray) and then select one or more specific organizations from the What organization manages this service? drop-down field to align the credential with those specific organizations.
  • URL. Type the URL for your PowerFlow system.
  • HTTP Auth User. Type the username for your PowerFlow system.
  • HTTP Auth Password. Type the password for your PowerFlow system.
  1. Click Save & Close .
  2. Take note of the SL1-assigned ID number for the new credential on the Credentials page, in the ID column. You will need the ID number when editing the input parameters of the run book actions included in the "ELK Stack Automation" PowerPack, below.

Editing the ELK Stack Run Book Actions

The "ELK Stack Automation" PowerPack includes three action policies:

  • "ELK: ExtLog"
  • "ELK: Index Create"
  • "ELK: Search"

These action policies use the corresponding "Run Integration Service Application" action type to trigger the PowerFlow application that collects data from the ELK Stack.

To edit the action policies included in the PowerPack:

  1. In SL1, go to the Actions page (Registry > Run Book > Actions).
  2. Locate the ELK action policy that you want to use and click its wrench icon (). The Editing Action page appears.
  1. In the Input Parameters field, change the values of the following parameters:
  • credential_id. Change the value to the credential ID that you noted earlier when creating a credential for your PowerFlow system in the previous procedure. This parameter is required.
  • include_event. Leave the value as "true".
  • application_name. Leave the default application value.
  • params. Leave the default parameter value.
  1. Make sure the Action State is set to Enabled, and then click Save.

Enabling the ELK Stack Run Book Automations

The "ELK Stack Automation" PowerPack includes the following run book automation policies that you can enable:

  • "ELK: ExtLog"
  • "ELK: Index Create"
  • "ELK: Search"

These run book automation policies update the SL1 event with the state of the associated ELK job. When a node is offline, a failure occurs, or a major event is detected in ELK, an SL1 event is created and the associated event is updated with any job details.

To enable the run book automations:

  1. In SL1, go to the Automation page (Registry > Run Book > Automation).
  2. Locate an ELK automation policy and click its wrench icon (). The Automation Policy Editor page appears.
  3. Update the following fields:
  • Policy State. Select Enabled.
  • Policy Priority. Select High to ensure that this PowerFlow automation policy is added to the top of the queue.
  • Available Actions. If it is not already selected, select the "Run Integration Service Application: <name>" action that corresponds with the ELK automation policy you selected in step 2, and click the arrows to move it to Aligned Actions.

ScienceLogic highly recommends that you do not make changes to the Policy Type, Repeat Time, or Align With fields or the And event is NOT acknowledged setting.

  1. Click Save.
  2. Repeat steps 2-4 for the remaining ELK run book automation policies.

Configuring PowerFlow

Creating and Aligning a Configuration Object in PowerFlow

A configuration object supplies the login credentials and other required information needed to execute the steps for a PowerFlow application. The Configurations page () of the PowerFlow user interface lists all available configuration objects for that system.

You can create as many configuration objects as you need. A PowerFlow application can only use one configuration object at a time, but you can use (or "align") the same configuration object with multiple applications.

To use this SyncPack, you will need to use an existing configuration object in the PowerFlow user interface or create a new configuration object. Next, you need to align that configuration object to the relevant applications.

Creating a Configuration Object

For this SyncPack, you should make a copy of the "ElasticSearch Logstash Kibana Configuration" configuration object, which is the sample configuration file that was installed with the  "ELK Stack" SyncPack.

The "ElasticSearch Logstash Kibana Configuration" configuration object contains all of the required variables. Simply update the variables from that object to match your SL1 and ELK settings.

To create a configuration object based on the "ElasticSearch Logstash Kibana Configuration" configuration object:

  1. In the PowerFlow user interface, go to the Configurations page ().
  2. Click the Actions button () for the "ElasticSearch Logstash Kibana Configuration" configuration object and select Edit. The Configuration pane appears.
  3. Click Copy as. The Create Configuration pane appears.
  4. Complete the following fields:
  • Friendly Name. Type a name for the configuration object that will display on the Configurations page.
  • Description. Type a brief description of the configuration object.
  • Author. Type the user or organization that created the configuration object.
  • Version. Type a version of the configuration object.
  1. In the Configuration Data field, include the required block of code to ensure that the applications aligned to the ELK configuration object is successful:
  • sl1_host. Type the hostname or IP address of the SL1 system the alerts will synchronize with.
  • sl1_user. Type the username for your SL1 system.
  • sl1_password. Type the password for your SL1 system.
  • elk_username. Type the username for your ELK system.
  • elk_password. Type the password for your ELK system.
  • elk_document_url. Enter the URL for your ELK document.
  • ELK_Cloud_id. Type the ELK Cloud ID for login.
  • create_index_name. Type the name of the log data.
  • URI_for_log_data. Type the Log Data URL.
  • from_date. Type the start date in dd-mm-yyyy hh:mm:ss.
  • to_date. Type the end date in dd-mm-yyyy hh:mm:ss.

The following fields are required only if you choose to manually create a virtual device for your ELK instance:

  • device_class_id. Type the device class ID for your ELK instance.
  • collector_group_id. Type the collector group ID for your ELK instance.
  • device_id.Type the device ID for your ELK instance.
  1. Click Save. You can now align this configuration object with one or more applications.

Aligning a Configuration Object and Configuring PowerFlow Applications

With this SyncPack, you can create SL1 events based on ELK jobs. You will need to align the PowerFlow applications from the "ELK Stack" SyncPack with the relevant configuration object in PowerFlow, and, if needed, update any other fields on the Configuration pane for the applications.

To align the configuration object with the relevant PowerFlow applications:

  1. On the Applications page of the PowerFlow user interface, open an ELK Stack PowerFlow application and click Configure. The Configurations pane for that application appears.
  1. From the Configurations drop-down, select the configuration object you want to use.

    The values for sl1_hostname and the other parameters that appear in the Configuration pane with a padlock icon () are populated by the configuration object you aligned with the application. Do not modify these values. If you encounter an error, make sure your configuration object is configured properly.

  1. Update the remaining fields on the Configurations pane as needed.
  2. Click Save to align that configuration with the application.
  3. Repeat this process for the other PowerFlow applications.

Scheduling PowerFlow Applications

You can create one or more schedules for a single application in the PowerFlow user interface. When creating each schedule, you can specify the queue and the configuration file for that application.

To create a schedule:

  1. On the Applications page (), click the Schedule button for the application you want to schedule. The Scheduler window appears.
  2. In the Schedule List pane, click the down arrow icon () next to an existing schedule to view the details for that schedule.
  3. In the Schedule Creator pane, complete the following fields for the default Frequency setting:
  • Schedule Name. Type a name for the schedule.
  • Frequency in seconds. Type the number of seconds per interval that you want to run the application.
  • Custom Parameters. Type any JSON parameters you want to use for this schedule, such as information about a configuration file or mappings.
  1. To use a cron expression, click the Switch to Cron Expression toggle to turn it blue. If you select this option, you can create complicated schedules based on minutes, hours, the day of the month, the month, and the day of the week:

As you update the cron expression, the Schedule window displays the results of the expression in more readable language, such as Runs app: "Every 0 and 30th minute past every hour on Sat", based on 0,30 in the Minutes field and 6 in the Day of Week field.

  1. Click Save Schedule. The schedule is added to the Schedule List pane. Also, on the Applications page, the Schedule button now displays with a dark blue background:

After you create a schedule, it continues to run until you delete it. Also, you cannot edit an existing schedule, but you can delete it and create a similar schedule if needed.

To view or delete an existing schedule:

  1. On the Applications page, click the Schedule button for the application that contains a schedule you want to delete. The Scheduler window appears.
  2. Click the down arrow icon () to view the details of an existing schedule.
  3. To delete the selected schedule, click the Actions icon () and selectDelete.

On the Scheduler window for a PowerFlow application, you can click the Copy as button from the Schedule List pane to make a copy of an existing schedule.

When either multiple SL1 instances or multiple ELK instances are involved with PowerFlow, you should create an individual configuration object for each SL1 or ELK instance. Next, create an individual schedule for each configuration object. Each schedule should use a configuration object that is specific to that single SL1 or ELK instance. Creating copies of a PowerFlow application from a SyncPack for the purpose of distinguishing between domains is not supported, and will result in issues on upgrades.