Docker Container Log Collector

The Skylar Automated RCA Docker container log collector, ze-docker-log-collector, collects container logs and sends logs to Skylar Automated RCA for automated incident detection. The collector uses the Fluentd logging driver for Docker and the Fluentd output plugin.

The GitHub repository for the collector is located at https://github.com/zebrium/ze-docker-log-collector.

Getting Started

When sending your logs from your docker daemon to Skylar Automated RCA, there are two configuration options for where your log collector can be installed and configured. The collector can be installed within the docker daemon context that you are sending all the logs from, or it could be installed on an external host, and route the logs to it by each docker daemon.

Deploying the Collector

Regardless on the installation method, you will start the collector using the following command, substituting the token and URL in for the values found in your Skylar Automated RCA Integration and Collectors page.

docker run -p 24224:24224 -e ZE_LOG_COLLECTOR_URL=<URL> -e ZE_LOG_COLLECTOR_TOKEN=<TOKEN> --restart always zebrium/docker-log-collector:latest

Additional environment variables can be specified to the collector to further extend the functionality.

Configuring the Docker Daemon

After the collector has been deployed and configured, modify the docker daemon configuration to start sending logs to the collector. For a complete list of configuration options, see the Docker documentation.

The docker daemon is located in /etc/docker/daemon.json on the Linux host and in C:\ProgramData\docker\config\daemon.json on the Windows host. For more about the docker daemon.json, see the Docker documentation.

Add the following configuration to your daemon.json file, substituting <fluentd-address> for the address of your log collector. If your log collector is deployed in the same docker daemon, then use 127.0.0.1:24224 as your address.

{
"log-driver": "fluentd",
  "log-opts": {
    "fluentd-address": "<fluentd-address>",
    "fluentd-async": "true"
  }
}

After the daemon file is updated, restart the docker daemon for the new changes to take effect. After this, you should be able to view the logs of the log collector and verify that it is receiving and forwarding logs to Skylar Automated RCA.

Environment Variables

Below is a list of environment variables that are available for configuration of the Fluentd container:

Environment Variables	Default	Description	Required?
ZE_LOG_COLLECTOR_URL	""	Skylar Automated RCA URL Endpoint for log ingestion.	Yes
ZE_LOG_COLLECTOR_TOKEN	""	Skylar Automated RCA ZAPI token for log ingestion.	Yes
ZE_DEPLOYMENT_NAME	"default"	Skylar Automated RCA Service Group Name.	No
FLUSH_INTERVAL	"60s"	Buffer Flush Interval.	No
ZE_LOG_LEVEL	"info"	Sets the log level for the output plugin.	No
VERIFY_SSL	"true"	Enables or disables SSL verification on endpoint.	No

Testing your Installation

After the Docker log collector software has been deployed in your environment, your container logs and incident detection will be available in the Skylar Automated RCA user interface.