Configuring Applications for the ELK Stack Synchronization PowerPack

Download this manual as a PDF file 

This section describes how to set up the PowerFlow applications for the ELK Stack SyncPack.

You must have an Elastic Cloud account before you integrate the ELK Stack  SyncPack with PowerFlow.

Creating and Aligning a Configuration Object in PowerFlow

A configuration object supplies the login credentials and other required information needed to execute the steps for a PowerFlow application. The Configurations page () of the PowerFlow user interface lists all available configuration objects for that system.

You can create as many configuration objects as you need. A PowerFlow application can only use one configuration object at a time, but you can use (or "align") the same configuration object with multiple applications.

To use this SyncPack, you will need to use an existing configuration object in the PowerFlow user interface or create a new configuration object. Next, you need to align that configuration object to the relevant applications.

Creating a Configuration Object

For this SyncPack, you should make a copy of the "ElasticSearch Logstash Kibana Configuration" configuration object, which is the sample configuration file that was installed with the ELK Stack Automation PowerPack.

The "ElasticSearch Logstash Kibana Configuration" configuration object contains all of the required variables. Simply update the variables from that object to match your SL1 and ELK settings.

To create a configuration object based on the "ElasticSearch Logstash Kibana Configuration" configuration object:

  1. In the PowerFlow user interface, go to the Configurations page ().
  2. Click the Actions button () for the "ElasticSearch Logstash Kibana Configuration" configuration object and select Edit. The Configuration pane appears:

An image of the configuration object page.

  1. Click Copy as. The Create Configuration pane appears.
  2. Complete the following fields:
  • Friendly Name. Type a name for the configuration object that will display on the Configurations page.
  • Description. Type a brief description of the configuration object.
  • Author. Type the user or organization that created the configuration object.
  • Version. Type a version of the configuration object.
  1. In the Configuration Data field, include the required block of code to ensure that the applications aligned to the ELK configuration object is successful:
  • sl1_host. Type the hostname or IP address of the SL1 system the alerts will synchronize with.
  • sl1_user. Type the username for your SL1 system.
  • sl1_password. Type the password for your SL1 system.
  • elk_username. Type the username for your ELK system.
  • elk_password. Type the password for your ELK system.
  • elk_document_url. Enter the URL for your ELK document.
  • ELK_Cloud_id. Type the ELK Cloud ID for login.
  • create_index_name. Type the name of the log data.
  • URI_for_log_data. Type the Log Data URL.
  • from_date. Type the start date in dd-mm-yyyy hh:mm:ss.
  • to_date. Type the end date in dd-mm-yyyy hh:mm:ss.

The following fields are required only if you choose to manually create a virtual device for your ELK instance:

  • device_class_id. Type the device class ID for your ELK instance.
  • collector_group_id. Type the collector group ID for your ELK instance.
  • device_id.Type the device ID for your ELK instance.
  1. Click Save. You can now align this configuration object with one or more applications.

Aligning a Configuration Object and Configuring PowerFlow Applications

With this SyncPack, you can create SL1 events based on ELK jobs. You will need to align the ELK Stack SyncPack applications with the relevant configuration object in PowerFlow, and, if needed, update any other fields on the Configuration pane for the applications.

To align the configuration object with the relevant PowerFlow applications:

  1. On the Applications page of the PowerFlow user interface, open each of the PowerFlow applications listed above and click Configure () one at a time. The Configurations pane for that application appears:

An image of the Configure Application page.

  1. From the Configurations drop-down, select the configuration object you want to use.

    The values for sl1_hostname and the other parameters that appear in the Configuration pane with a padlock icon () are populated by the configuration object you aligned with the application. Do not modify these values. If you encounter an error, make sure your configuration object is configured properly.

  1. Click Save to align that configuration with the application.
  2. Repeat this process for the other PowerFlow applications.

Scheduling PowerFlow Applications

You can create one or more schedules for a single application in the PowerFlow user interface. When creating each schedule, you can specify the queue and the configuration file for that application.

To create a schedule on PowerFlow version 2.5.0 and later:

  1. On the Applications page (), click the Schedule button for the application you want to schedule. The Scheduler window appears:

  1. In the Schedule List pane, click the down arrow icon () next to an existing schedule to view the details for that schedule.
  2. In the Schedule Creator pane, complete the following fields for the default Frequency setting:
  • Schedule Name. Type a name for the schedule.
  • Frequency in seconds. Type the number of seconds per interval that you want to run the application.
  • Custom Parameters. Type any JSON parameters you want to use for this schedule, such as information about a configuration file or mappings.
  1. To use a cron expression, click the Switch to Cron Expression toggle to turn it blue. If you select this option, you can create complicated schedules based on minutes, hours, the day of the month, the month, and the day of the week:

As you update the cron expression, the Schedule window displays the results of the expression in more readable language, such as Runs app: "Every 0 and 30th minute past every hour on Sat", based on 0,30 in the Minutes field and 6 in the Day of Week field.

  1. Click Save Schedule. The schedule is added to the Schedule List pane. Also, on the Applications page, the Schedule button now displays with a dark blue background:

To create a schedule on PowerFlow version 2.4.1 or earlier:

  1. On the Applications page (), click the Schedule button for the application you want to schedule. The Schedule window appears, displaying any existing schedules for that application:

If you set up a schedule using a cron expression, the details of that schedule display in a more readable format in this list. For example, if you set up a cron expression of */4 * * * *, the schedule on this window includes the cron expression along with an explanation of that expression: "Every 0, 4, 8, 12, 16, 20, 24, 28, 32, 36, 40, 44, 48, 52, and 56th minute past every hour".

  1. Select a schedule from the list to view the details for that schedule.
  2. Click the + icon to create a schedule. A blank Schedule window appears:

  1. In the Schedule window, complete the following fields:
  • Schedule Name. Type a name for the schedule.
  • Switch to. Use this toggle to switch between a cron expression and setting the frequency in seconds.
  • Cron expression. Select this option to schedule the application using a cron expression. If you select this option, you can create complicated schedules based on minutes, hours, the day of the month, the month, and the day of the week. As you update the cron expression, the Schedule window displays the results of the expression in more readable language, such as Expression: "Every 0 and 30th minute past every hour on the 1 and 31st of every month", based on */30 * * /30 * *.
  • Frequency in seconds. Type the number of seconds per interval that you want to run the application.
  • Custom Parameters. Type any JSON parameters you want to use for this schedule, such as information about a configuration file or mappings.
  1. Click Save Schedule. The schedule is added to the list of schedules on the initial Schedule window. Also, on the Applications page, the word "Scheduled" appears in the Scheduled column for this application, and the Schedule button contains a check mark:

After you create a schedule, it continues to run until you delete it. Also, you cannot edit an existing schedule, but you can delete it and create a similar schedule if needed.

To view or delete an existing schedule:

  1. On the Applications page, click the Schedule button for the application that contains a schedule you want to delete. The Scheduler window appears.
  2. Click the down arrow icon () to view the details of an existing schedule.
  3. To delete the selected schedule, click the Actions icon () and selectDelete.

On the Scheduler window for a PowerFlow application, you can click the Copy as button from the Schedule List pane to make a copy of an existing schedule.

When either multiple SL1 instances or multiple ELK instances are involved with PowerFlow, you should create an individual configuration object for each SL1 or ELK instance. Next, create an individual schedule for each configuration object. Each schedule should use a configuration object that is specific to that single SL1 or ELK instance. Creating copies of a PowerFlow application from a SyncPack for the purpose of distinguishing between domains is not supported, and will result in issues on upgrades.