This
If you are using a new SL1 system or a system that has not previously used PhoneHome communication for collectors, you or your SL1 administrator will need to configure each Database Server in the SL1 system to accept these connections.
Use the following menu options to navigate the SL1 user interface:
- To view a pop-out list of menu options, click the menu icon ().
- To view a page containing all of the menu options, click the Advanced menu icon ().
What is PhoneHome Communication?
SL1 supports two methods for communication between a Database Server (an SL1 Central Database or an SL1 Data Engine) and the SL1 Collectors:
- Traditional
- PhoneHome
In the Traditional method, the SL1 services on the Database Server initiate a new connection to the MariaDB port on the collector to read and write data. The connection request traverses the network, including the Internet if necessary, eventually reaching the collector. For this approach to work, the collector administrator must allow ingress communication from the Database Server on TCP port 7707, which is the MariaDB port on the collector. The communication is encrypted using SSL whenever possible.
The benefit of the traditional method is that communication to the Database Server is extremely limited, so the Database Server remains as secure as possible.
In the PhoneHome method, the collectors initiate an outbound connection to the Database Server over SSH. The connection requests originate from edge to core via TCP, using port 7705 by default.
After authenticating, the client forwards the local MariaDB port onto the Database Server using a loopback remote IP address. A corresponding SL1 appliance is added using the loopback IP. When the SL1 services on the database try to make a connection to the collector's MariaDB, they connect locally to the loopback IP address, in contrast to reaching out to the collector's IP or DNS name. The communication is encrypted.
The benefits of this method are that no ingress firewall rules need to be added, as the collector initiates an outbound connection, and no new TCP ports are opened on the network that contains the Data Collectors.
While you do not need to add any ingress firewall rules, a best practice is to add an egress firewall rule that allows SSH traffic from the collector on the server's port to either all available destination addresses on the DB or to the specific address on the DB that you know the collector will be able to reach. Starting with SL1 12.1.0, custom firewall rules must use the rich rules syntax and added to /etc/siteconfig/firewalld-rich-rules.siteconfig.
The PhoneHome configuration uses public key/private key authentication to maintain the security of the Database Server. Each Data Collector is aligned with an SSH account on the Database Server and uses SSH to communicate with the Database Server. Each SSH account on the Database Server is highly restricted, has no login access, and cannot access a shell or execute commands on the Database Server.
Important Notes about PhoneHome Communication
Before attempting to configure PhoneHome communication for your SL1 system, be advised of the following:
- If you are using a proxy in your PhoneHome configuration, you should configure the proxy before you configure SL1 Collectors. For more information, see the section on Adding a Proxy Configuration.
- If you are using a high-availability (HA) or disaster recovery (DR) setup, you can configure up to three PhoneHome Database Servers.
- PhoneHome communication uses secure shell (SSH). You cannot use PhoneHome over HTTP(S) or an HTTP(S) proxy.
- ScienceLogic does not recommend putting a PhoneHome Database Server behind a load balancer or a NAT gateway, as PhoneHome communication is designed to enable active connections to all Database Servers at any given time. If you must use a load balancer or NAT gateway, make sure each Database Server is behind a separate load balancer or NAT gateway.
- For destination addresses, use IP addresses whenever possible. Use a DNS name only if it uniquely identifies one host and does not point to a load balancer or is a round-robin for multiple hosts.
- If you have an AWS configuration, set up AWS hosts for the Database Server using an Elastic IP. In the event of a disaster recovery, this will make it easier to rebuild the Database Server without needing to change the IP address.
- Most intrusion detection/prevention systems will flag and drop SSH traffic on ports other than 22, which is the default SSH port. Since the PhoneHome server listens on ports other than 22, this often causes issues with onboarding PhoneHome collectors. You should ensure that your intrusion detection/prevention systems are configured to allow SSH traffic on the server's port.
Prerequisites for Configuring PhoneHome Communication
Before configuring PhoneHome communication in your ScienceLogic environment, you must:
-
Have installed and licensed the Database Server and SL1 Collectors.
-
Have SSH access or console access to each Database Server.
-
On each ScienceLogic appliance, know the username and password for access to the console. Note that the MySQL password matches the login password for SL1 unless one or both of the passwords were manually changed after installation.
-
Ensure that all SL1 appliances are running the same version of SL1 that the Database Server is running.
-
Ensure that the Database Server opens a port for PhoneHome communication. The default port used by the Configuration Utility is 7705. If you are on a SaaS SL1 system, you must use port 7705. If you are on a non-SaaS system, you can use other ports besides 7705, but make sure those ports are not already being used.
Allow-listing port 7705 in the firewall is not enough. If the firewall does a layer 7 (application layer) filtering, you must create an exception rule to allow any outgoing traffic from the Data Collector to all the Database Servers on the control port, which is port 7705 by default. Some firewalls enable this by default and, as a result, those firewalls will drop SSH traffic on a non-standard port like 7705 in this situation.
If you use a proxy in your PhoneHome configuration, you must add the proxy configuration before configuring the SL1 Collectors for PhoneHome communication.
Overview of the PhoneHome Configuration
For a configuration that includes one or more Database Servers, perform the following steps in the SL1 user interface to use PhoneHome communications:
-
Add a proxy connection, if applicable to your configuration. Otherwise, you can skip this step.
-
Configure the SL1 Collectors for PhoneHome. If needed, update the collector to the same version of SL1 that the Database Server is running.
After you have configured PhoneHome communications for your SL1 system, you can also:
-
Familiarize yourself with the phd and phc PhoneHome components.
-
Learn how to use the command-line interface for PhoneHome communications.
-
As needed, perform administrative functions on your PhoneHome system, such as:
- View a list of PhoneHome devices
- View information about a single PhoneHome device
- Rename a PhoneHome device
- Check the status of a PhoneHome collector
- Check the connection between PhoneHome devices
- Sync the configuration of a PhoneHome system
- Define port forwarding for each collector to use SSH from the Database Server to access that collector
- Associate a new destination address with a PhoneHome Database Server
- View logs relating to PhoneHome configuration
- Tune various PhoneHome settings
- Clear a PhoneHome device
- Delete a PhoneHome collector or Database Server
-
See the Troubleshooting section for additional help.
Configuring the Database Server for PhoneHome Communication
The first step in establishing PhoneHome communication is to configure a PhoneHome Database Server. This can be either a Central Database (CDB) appliance or a Data Engine (DE) appliance.
In PhoneHome communication, the Database Server communicates with the SL1 Collectors. The Database Server stores all the configuration information for the PhoneHome configuration. Server-client authentication happens over the configuration store in MariaDB on the Database Server.
Setting up a Database Server prepares the server to listen to incoming connections from a PhoneHome collector. This process also opens the firewall rules on the configured port and labels the configured port for SSH traffic in the SE Linux subsystem.
PhoneHome configuration is stored in tables on the Database Server. The information is accessible to all Database Servers in the SL1 system. Any Database Server in the SL1 system can provide network access.
Before Configuring the Database Server for PhoneHome Communication
Make sure you have answers to the following questions before setting up the Database Server for PhoneHome communication:
-
Is the Database Server a single CDB or DE, or is there a High Availabilty (HA) or Disaster Recovery (DR) pair?
-
Is the CDB or DE behind a NAT gateway?
-
Do you want the PhoneHome server to listen on to the default port 7705, or do you want to customize the port?
SaaS SL1 systems must use port 7705.
-
Does the Database Server have multiple routable IP addresses to it, and do you plan to have PhoneHome collectors from different subnets connect to the Database Server?
Each Database Server must have SL1 installed, have an IP address assigned to it, and be licensed with ScienceLogic. For more information about licensing, see Licensing and Configuring a Database Server.
Understanding Database Server PhoneHome Configuration Options
The following sections explain how to configure the Database Server based on your SL1 environment.
After you configure the Database Server for PhoneHome communication, you must add a proxy host (if necessary for your setup) and then configure the Data Collectors and Message Collectors in your network. For more information, see Configuring SL1 Collectors for PhoneHome Communication.
Configuring a Single Database Server
The most basic SL1 environment contains a single Database Server. This setup makes the following assumptions:
- The Database Server has a public IP address assigned to one of its network interfaces or has a private IP address.
- All the PhoneHome collectors will be on the same network and will be able to reach the private IP address of the PhoneHome Database Server.
- The PhoneHome Database Server will be configured to listen on port 7705.
- The PhoneHome Database Server will be named "ph-db-1". Naming the PhoneHome collector is optional, but recommended.
To configure a single Database Server for PhoneHome communication:
-
Go to the console of the Database Server or use SSH to access the server and log in as user em7admin with the password you configured during setup.
-
Run the following command:
sudo phonehome setup -n ph-db-1
The setup command creates a PhoneHome device in the config store along with its corresponding RSA host key. It also adds the default non-loopback IP address, corresponding to the hostname, as the default destination address. However, you can define a custom destination address if required. The command also adds a firewall rule to allow incoming connections on the specified port and labels it as SSH port (ssh_port_t) in the SELinux subsystem.
Configuring a Database with a Non-default Address or Port
If you are configuring an SL1 system in a SaaS environment, you must use 7705 as the port for PhoneHome communication. Custom ports are not supported for PhoneHome communication on SaaS systems.
You can configure a PhoneHome Database Server to use a non-default address or port in the following situations:
- You want the PhoneHome server to listen on a non-default port, or on an address that is different than the output of the getaddrbyhostname syscall.
- The database appliance is behind a NAT gateway
- The database appliance is set up on a cloud host, like AWS, where the public IP is not assigned directly to the network interface of the virtual host.
To configure a Database Server with a non-default address or port:
-
Go to the console of the Database Server or use SSH to access the server and log in as user em7admin with the password you configured during setup.
-
Run the following command:
sudo phonehome setup -n ph-db-1 -a <addr>
where <addr> is an IPv4 address or DNS name in "host:port" format, such as 203.65.33.22:7809 or ph-db1.example.com:8899.
The PhoneHome server process runs as an unprivileged user that will not be able to bind to a privileged port (1-1023). Therefore, when you choose a custom port, you must choose port 1024 or higher.
Configuring a Database with Multiple IP Addresses
You can assign multiple addresses to a destination if required. The list of addresses can be a mix of IPv4 addresses and DNS names.
To configure a Database Server with multiple IP addresses:
-
Go to the console of the Database Server or use SSH to access the server and log in as user em7admin with the password you configured during setup.
-
Run the following command:
sudo phonehome setup -n ph-db-1
-
Run the following command:
sudo phonehome destination add <id> <addr>
where <id> is the resulting device ID for the PhoneHome Database Server and <addr> is an actual address string in "host:port" format.
The port must be the same for all addresses, because a PhoneHome server is not capable of listening on multiple ports.
-
Repeat this command for every address that you want to add to the destination.
Configuring PhoneHome Database Servers for High Availability and Disaster Recovery
If you are using a high-availability (HA) or disaster recovery (DR) setup, you can configure up to three PhoneHome Database Servers.
In an HA/DR PhoneHome configuration, there is no notion of a control node. Every Database Server in an HA/DR setup can participate in all operations.
ScienceLogic does not recommend putting a PhoneHome Database Server behind a load balancer or a NAT gateway, as PhoneHome communication is designed to enable active connections to all Database Servers at any given time. If you are configuring PhoneHome communication for HA/DR and you must use a load balancer or NAT gateway, make sure each Database Server is behind a separate load balancer or NAT gateway.
You can use the same Database Servers in both a PhoneHome configuration and a traditional configuration.
To configure PhoneHome Database Servers for HA/DR:
-
To configure the primary Database Server for PhoneHome communication, follow the instructions in the section Configuring a Single Database Server.
-
To add a secondary Database Server, run the following command:
sudo phonehome setup -n <name>
where <name> is a customized name other than ph-db-1.
Optionally, you can add -a <addr> to the above command if you want to specify the listening address, where <addr> is an IPv4 address or DNS name in "host:port" format. If you do not add that flag, the system will attempt to pick up the private IP address from the assigned IP list.
-
Repeat step 2 if you are adding a third Database Server. Otherwise, proceed to step 4.
-
To add SL1 Collectors to your PhoneHome setup, follow the instructions in the section Configuring SL1 Collectors for PhoneHome Communication.
Alternatively, you can configure the SL1 Collectors for PhoneHome communication using the command line.
Managing Proxy Connections for PhoneHome Communication
If your organization requires that you use a proxy for outbound requests, you can configure one or more proxy connections between the SL1 Collectors and the Database Server.
If you use a proxy in your PhoneHome configuration, you must perform the steps in the section about Adding a Proxy Connection before you configure SL1 Collectors. The other steps in the PhoneHome configuration setup will require the proxy for communication.
Otherwise, if you are configuring PhoneHome communication and do not require a proxy connection, you can skip ahead to the section on Configuring SL1 Collectors for PhoneHome Communication.
For example, you might use a proxy connection if your SL1 Collector does not have a direct outbound internet connection to reach the Database Server. A PhoneHome proxy configuration includes the destination address—either the address of the Database Server or that of the next proxy host—and the address of the proxy server to which the client must connect to reach the destination.
There can be one or more proxy hosts in between an SL1 Collector and a Database Server, thus forming a proxy chain.
Adding a Proxy Configuration
If you are using a proxy in your PhoneHome configuration, you should configure the proxy before you configure SL1 Collectors. The other steps in the PhoneHome configuration will require the proxy for communication.
To add a proxy connection between an SL1 Collector to the Database Server:
-
Go to the console of the SL1 Collector.
-
Run the following command on the SL1 Collector, replacing <Destination Address> with the address and port of the Database Server appliance to which you want to connect, <Proxy Address> with the proxy host address and port you want to use as a tunnel, and <Proxy User> with the username used to log in to the proxy host:
phonehome proxy new <Destination Address> <Proxy Address> <Proxy User>
Addresses should be in the format <host>:<port>. The host can be either an IP address or a DNS name.
For example, if you want to configure the SL1 Collector to connect to the Database Server with an address of 202.35.52.71 through a proxy host with the address 10.1.17.68 with the user em7admin, you would run the following command:
phonehome proxy new 202.35.52.71:7705 10.1.17.68:22 em7admin
If you are connecting to the Database Server through a chain consisting of multiple proxies, you should add the proxy configurations in reverse order, starting with the destination address and last proxy host address, then the last proxy host address and previous proxy host address, and so forth, until you get to the first proxy host.
For example, if you want to connect to the Database Server with an address of 202.42.63.79 through proxy host A with an address of 192.168.0.3 with the user proxyuser, and also proxy host B with an address of 10.2.13.79 with the user em7admin, then you would run the following commands:
phonehome proxy new 202.42.63.79:7705 10.2.13.79:22 em7admin
phonehome proxy new 10.2.13.79:22 192.168.0.3 proxyuser
New proxy configurations do not take effect until the PhoneHome client is restarted or the next watchdog cycle occurs.
When you run the command, the system prompts you for a password for the proxy host. The system uses this password to automatically configure and validate SSH key-based authentication to the host; the next time you need to run anything via the proxy host, it will use the collector's private key for authentication rather than prompting you for the password. Optionally, you can disable this behavior by adding "-n" to the end of the command. If you do so, you must then manually configure the proxy's SSH key-based authentication.
If you get a "handshake failed: ssh..." error message when adding a new proxy:
-
In SL1, go to the Appliances page (System > Settings > Appliances) and click the edit button () for that appliance.
-
Review the collector's MariaDB credentials. This error can occur if the collector and the Database Server (CDB) use different credentials.
For example, if the Database Server has been updated and the ISO for the Database Server is before SL1 version 11.3.0, while the collector was deployed with SL1 version 11.3.0 or later, the Database Server will be using root/<password>, and the collector would be using clientdbuser/<password>.
Viewing a List of Proxy Connections
To view a list of proxy connections from an SL1 Collector to the Database Server:
-
Go to the console of the SL1 Collector.
-
Run the following command on the SL1 Collector:
phonehome proxy list
Deleting a Proxy Configuration
To add a proxy configuration between an SL1 Collector to the Database Server:
-
Go to the console of the SL1 Collector.
-
Run the following command on the SL1 Collector, replacing <Destination Address> with the address and port of the Database Server appliance to which the proxy is connecting:
phonehome proxy delete <Destination Address>
Addresses should be in the format <host>:<port>. The host can be either an IP address or a DNS name.
Removed proxy configurations do not take effect until the PhoneHome client is restarted or the next watchdog cycle occurs.
Configuring SL1 Collectors for PhoneHome Communication
After you install an SL1 Collector, use the Add Node wizard on the Nodes page (Manage > Nodes > Add Nodes) to configure your new SL1 Collector. This configuration process:
- Registers the SL1 Collector in SL1
- Connects the SL1 Collector to the Database Server so it can share its collected data
- Aligns the SL1 Collector to a new or existing Collector Group.
While navigating through the Add Node wizard, the Choose Connection Type window appears. This window enables you to determine the method in which the SL1 Collector and Database Server will communicate. The options are:
Connection Type |
Used For |
---|---|
Token-based PhoneHome Communication |
|
Password/secret-based PhoneHome Communication |
|
Traditional Communication |
Part of the setup for SL1 Collectors takes place in the Node Configuration Utility, which has its own user interface separate from the SL1 user interface. The Nodes page and the Node Configuration Utility replace some of the functionality previously found in the Web Configuration Utility in earlier versions of SL1.
All connection types require a token that SL1 generates as part of the wizard. A token is a JSON web token (JWT) that contains a set of secure data that SL1 uses to establish communication between the SL1 Collector and the Database Server. This token expires after a predefined time from the time of generation; by default, this expiration time is 30 minutes, but it can be extended to a maximum of 2 hours. The token encodes all destination addresses.
The processes for setting up the two PhoneHome communication types—"Collector Initiates | System Accepts" and "Collector Initiates | User Accepts"—through the SL1 user interface and the Node Configuration Utility are described below. Alternatively, you can also configure these communication types using the command line.
Configuring Collector Initiates | System Accepts Communication
This section describes how to register and connect an SL1 Collector to the Database Server using the Collector Initiates | System Accepts option. This is a token-based PhoneHome collector connection type.
To connect an SL1 Collector to the Database Server for token-based PhoneHome communication:
-
On the Nodes page (Manage > Nodes), click . The Choose Connection Type window of the Add Node wizard appears.
tab on the -
Select Collector Initiates | System Accepts and click Next. The Define Collector Properties window appears.
-
Complete the following fields as needed:
-
Collector Name. Type the name the collector used when registering the collector. SL1 will update this value with the collector hostname.
-
Collector IP Address or Hostname. Type the IP address or the hostname of the collector. This information is optional but recommended, as it is used in Step 3 of the wizard to create a link to the collector's Node Configuration Utility, where you will input the token you generate.
-
Collector Description. Type a description of the collector. This field is optional.
-
Collector Group. The new collector must be aligned to an SL1 Collector Group. You have the following options for this field:
- Select an existing Collector Group from the drop-down.
- Create a new Collector Group for the collector by clicking the plus icon (+). On the Add Collector Group modal, you can name the new group and choose to make that Collector Group available to all current and future organizations. You can also limit the Collector Group to specific organizations.
The All current and future organizations toggle is enabled by default. If you want to limit Organization access to the new Collector Group, disable this toggle and select the organization or organizations from the drop-down.
-
Collector Type. Your options include:
- Data Collector. This is the most commonly used type. A Data Collector retrieves a specific set of information from monitored devices. A Data Collector can also work as a Message Collector.
- Message Collector. A Message Collector receives and processes inbound, asynchronous syslog and trap messages from monitored devices.
-
Click Configure Collector window appears.
. TheYou can go back to a previous step at any point in the wizard, but when you click the SL1 always generates a new token. You cannot retrieve this particular token if you close the Add Node wizard. The generated token expires after 30 minutes.
button,
-
In the Token field, click the Copy icon () to copy the token .
-
Open the Node Configuration Utility by clicking the Open icon () in the Node Configuration Utility field. The login page for the Node Configuration Utility opens in a new browser window.
If you did not specify an IP address or a hostname in step 2 of this wizard, you will need to open a new browser window and type the IP address or hostname for the collector, followed by ":7700/node-config", such as "https://10.1.1.100:7700/node-config".
If the node type is not a collector, the Node Configuration Utility will display the following message: "This page will only be visible if you are on a collector."
-
Log in to the Node Configuration Utility using the same username and password that you used when you installed the collector. After you log in, the collector and the SL1 Database Server attempt to connect. The connection will fail, which is expected. The Connect Collector page appears with an empty Paste token text field.
-
Paste the token you copied in step 5 in the Paste token field.
If you did not generate a token, you can click User Accepted Connection Request, and add the IP addresses for the Database Servers (CMDBs) in the text box.
, select -
After pasting the token, click Success dialog states that the collector was registered and the connection to the database was initiated.
or , based on your choices in the previous step. When the connection is made, a -
Click Success dialog. The Collector Connection Status page displays details about the collector and the Database Server, along with the connection state, which can be "Connected", "Not Connected", or "Unknown". "Unknown" indicates that SL1 has not yet completed its first check of the connection state; click after a few moments and the status should update to "Connected".
on the -
On the Collector Connection Status page, click the expand icon () to view the connection path. The health of each hop in the connection is reported separately, but hops after an unresponsive hop will not be checked. This "Connection Path" information can be useful in diagnosing collector-database connection issues.
-
To view any changes to the connection status, click
.If you want to disconnect the collector and close the SSH tunnel between the collector and the Database Server, click Disconnect & Clear Configuration. This action will close the outgoing connection from the collector to all configured destinations, and it will also clear all local configuration. This action cannot be undone.
-
Close the Node Configuration Utility.
-
In SL1, go to the tab on the Nodes page, where you can now see the new collector in the list, aligned with the Collector Group you specified in the Add Node wizard. The new collector also displays on the Appliance Manager page (System > Settings > Appliances).
-
On Step 3 of the Add Node wizard, click Nodes page appears with the pending request.
. The tab on the
Configuring Collector Initiates | User Accepts Communication
This section describes how to register and connect an SL1 Collector to the Database Server using the Collector Initiates | User Accepts option. This is a password/secret key PhoneHome collector connection type.
To connect an SL1 Collector to the Database Server for password/secret key PhoneHome communication:
-
On the Nodes page (Manage > Nodes), click . The Choose Connection Type window of the Add Node wizard appears.
tab on the -
Select Collector Initiates | User Accepts and click Next. The Define Collector Properties window appears.
-
Complete the following fields as needed:
-
Collector Name. Type the name the collector used when registering the collector. SL1 will update this value with the collector hostname.
-
Collector IP Address or Hostname. Type the IP address or the hostname of the collector. This information is optional but recommended, as it is used to create a link to the collector's Node Configuration Utility, where you will input the token you generate.
-
Collector Description. Type a description of the collector. This field is optional.
-
Collector Group. The new collector must be aligned to an SL1 Collector Group. You have the following options for this field:
-
Select an existing Collector Group from the drop-down.
-
Create a new Collector Group for the collector by clicking the plus icon (+). On the Add Collector Group modal, you can name the new group and choose to make that Collector Group available to all current and future organizations. You can also limit the Collector Group to specific organizations.
The All current and future organizations toggle is enabled by default. If you want to limit Organization access to the new Collector Group, disable this toggle and select the organization or organizations from the drop-down.
-
-
Collector Type. Your options include:
-
Data Collector. This is the most commonly used type. A Data Collector retrieves a specific set of information from monitored devices. A Data Collector can also work as a Message Collector.
-
Message Collector. A Message Collector receives and processes inbound, asynchronous syslog and trap messages from monitored devices.
-
-
Click Configure Collector window appears.
. TheYou can go back to a previous step at any point in the wizard, but when you click the SL1 always generates a new token. You cannot retrieve this particular token if you close the Add Node wizard. The generated token expires after 30 minutes.
button,
-
Click the Copy icon () to copy the token in the Token field.
-
Open the Node Configuration Utility by clicking the Open icon () in the Node Configuration Utility field. The login page for the Node Configuration Utility opens in a new browser window.
If you did not specify an IP address or a hostname in step 2 of this wizard, you will need to open a new browser window and type the IP address or hostname for the collector, followed by ":7700/node-config", such as "https://10.1.1.100:7700/node-config".
If the node type is not a collector, the Node Configuration Utility will display the following message: "This page will only be visible if you are on a collector."
-
Log in to the Node Configuration Utility using the same username and password that you used when you installed the collector. After you log in, the collector and the SL1 Database Server attempt to connect. The connection will fail, which is expected. The Connect Collector page appears with an empty Paste token text field.
-
Paste the token you copied in step 5 in the Paste token field.
If you did not generate a token, you can click User Accepted Connection Request, and add the IP addresses for the Database Servers (CMDBs) in the text box.
, select -
After pasting the token, click Success dialog contains a six-digit confirmation code. Click the Copy icon () to copy the confirmation code.
or , based on your choices in the previous step. When the connection is made, the -
Click Success dialog. The Collector Connection Status page displays details about the connection request and the same six-digit confirmation code.
on the -
In SL1, click on Step 3 of the Add Node wizard. The tab on the Nodes page appears with the pending request.
-
Select the Actions icon () next to the pending request for the new collector and select Accept. The Accept Request dialog appears.
-
Paste the six-digit confirmation code you copied in step 9 from the Connect Collector page of the Node Configuration Utility and click . The Configure Collector dialog displays a summary of the collector information you entered in the Add Node wizard.
-
Edit the collector information and collector group as needed, and then click Configure Collector dialog displays a summary of your information.
. The -
Click Nodes page displays the new collector, aligned with the collector group you specified. The new collector also displays on the Appliance Manager page (System > Settings > Appliances).
. The tab on the
Connecting an SL1 Collector to the SL1 Database Server using the Command-line Interface
As an alternative to onboarding SL1 Collectors via the user interface, you can instead choose to onboard SL1 Collectors using the command-line interface if you prefer to do so. This section describes how to onboard SL1 Collectors based on whether you want a "system accepted" connection type or a "user accepted" connection type.
System Accepted
In this connection method, the database administrator creates a new token on the database appliance.
To connect a collector using the System Accepted method with the command-line interface:
-
Go to the console of the Database Server or use SSH to access the server and log in as user em7admin with the password you configured during setup.
-
Run the following command:
phonehome token new <model_type> <name> <CUG_ID ><description>
where:
- <model_type> is either a cu for a Data Collector or mc for a Message Collector.
- <name> is the name of the PhoneHome collector. You can use any name you want.
- <CUG_ID> is the numeric ID of a collector group from SL1.
- <description> is the descriptive text about the collector.
-
Make a note of the resulting token and share it with the collector administrator.
-
The collector administrator registers the collector using the token value by running the following command on the SL1 collector:
sudo phonehome register <token>
User Accepted
In this connection method, the collector administrator sends a registration request from the collector.
To connect a collector using the User Accepted method with the command-line interface:
-
Go to the console of the SL1 collector or use SSH to access the collector and log in as user em7admin with the password you configured during setup.
-
Run the following command on the collector:
sudo phonehome request send <address_1> [<address_2> <address_3> ... <address_n>] [-l <label>]
where:
-
<address> is the destination address of the database server, in "host:port" format. You can include multiple addresses to one or multiple databases. Separate multiple addresses with a space.
-
<label> is an optional field you can use to associate a human-friendly identifier with the request. Every request is identified by a random string on the server side, and it might be confusing for the database administrator to find a specific request if numerous requests are coming from other collectors.
-
-
Make a note of the one-time secret and share it with the database administrator.
-
The Database administrator accepts the incoming request using the one-time secret by running the following command on the Database Server:
phonehome request accept <uuid> <model_type> <name> <CUG_ID> <description> <one_time_secret>
where:
- <uuid> is the unique ID of the request.
- <model_type> is either a cu for a Data Collector or mc for a Message Collector.
- <name> is the name of the PhoneHome collector. You can use any name you want.
- <CUG_ID> is the numeric ID of a collector group from SL1 to which you want to assign this collector.
- <description> is the descriptive text about the collector.
- <one_time_secret> is the secret generated when sending a request from the collector that you made note of in step 3.
Understanding PhoneHome Components
This section describes two important PhoneHome components, phd and phc.
phd
The phd PhoneHome server daemon is installed and managed as a systemd service that is enabled on PhoneHome Database Servers. The server daemon listens to a port (7705 by default) and accepts incoming SSH connections from the PhoneHome client (phc) as well as OpenSSH clients. This service supports public key authentication for registered PhoneHome clients and collectors, as well as challenge-response authentication for the initial registration. The authentication-related configuration is stored in MariaDB; as such, it does not require creating local (Linux) users on the Database Server. Some aspects of the phd configuration will be stored on the local filesystem.
phc
The phc PhoneHome client runs as a service in systemd on PhoneHome SL1 Collectors. It is responsible for establishing a tunnel with the phd that is running on the Database Server and forwarding the local MariaDB port from the SL1 Collector to the Database Server.
Using the Command-Line Interface for PhoneHome Collection
If you have access to the console for each appliance in the PhoneHome configuration, or if you have SSH access to each appliance in the PhoneHome configuration, you can use the phonehome command to configure and troubleshoot your PhoneHome configuration.
To use the phonehome command:
-
Either go to the console of the SL1 appliance or use SSH to access the server. Log in as "root".
-
At the command prompt, type the following:
phonehome <command>
where <command> is one of the following commands:
Command |
Used For |
See Also |
---|---|---|
clear | Clears the PhoneHome configuration on a PhoneHome device. The clear command will also disable the PhoneHome phd service. You can use the clear command on a Database Server to block future connection requests from Data Collectors and secondary Database Servers in an HA/DR configuration. | Clearing a PhoneHome Device |
check | Checks the state of the connection from an SL1 Collector to the Database Server, visualizing the network path from the SL1 Collector to the Database Server as well as any proxy hops in between, if applicable. The output indicates any failures connecting to any hop. | Checking the Connection Between PhoneHome Devices |
client | Runs the PhoneHome client (installed as a systemd service phc). | Understanding PhoneHome Components |
config | Displays and enables you to edit PhoneHome configuration related to the server and client. | Tuning PhoneHome Settings |
delete | Deletes a PhoneHome SL1 Collector. This argument prevents you from deleting any SL1 Collector with an associated SL1 appliance. | Deleting a PhoneHome Collector |
destination | Enables you to add, remove, or view addresses to a PhoneHome Database Server. | Managing Destinations |
forwards | Enables you to add, remove, or view ports forwarded from an SL1 Collector to the Database Server. | Managing Port Forwarding for PhoneHome Communication |
list | Displays a list of PhoneHome devices (Database Servers and Collectors). | Viewing a List of PhoneHome Devices |
migrate | Migrates the configuration from the classic PhoneHome setup to the new PhoneHome setup. This is done automatically during upgrade, if you are upgrading from a version of SL1 prior to 11.2.0. | Running the Pre-upgrade Test for Existing PhoneHome Connections |
proxy | Enables you to add, remove, or view proxy configurations along the network path from an SL1 Collector to the Database Server. | Managing Proxy Connections for PhoneHome Communication |
register | Registers a new SL1 Collector as a PhoneHome collector with a token. | Connecting an SL1 Collector to the SL1 Database Server using the Command-line Interface |
rename | Renames an existing Phone Home device: phonehome rename <id> <new_name>. | Renaming a PhoneHome Device |
request | Enables you to send, view, accept, or reject an SL1 Collector registration request. | Connecting an SL1 Collector to the SL1 Database Server using the Command-line Interface |
server | Runs the PhoneHome server (installed as a systemd service phd). | Understanding PhoneHome Components |
setup | Configures a new PhoneHome Database Server. | Configuring the Database Server for PhoneHome Communication |
status | Displays the status of the PhoneHome SL1 Collectors. The output is tabular by default but supports JSON output as well. The output does not contain the remote loopback IP address of PhoneHome SL1 Collectors, nor does it list PhoneHome Database Servers. | Checking the Status of a PhoneHome Collector |
sync | Syncs the configuration from the Database Server. | Syncing the Configuration of a PhoneHome System |
token | Enables you to create, view, or delete registration tokens. | Connecting an SL1 Collector to the SL1 Database Server using the Command-line Interface |
view | Displays the state of an SL1 Collector. This argument must be run on a Database Server. | Viewing Information about a Single PhoneHome Device |
Additionally, after configuring communication between a Database Server and an SL1 Collector, you can go to the console of the SL1 Collector or Database Server and run the following commands to view more information about your servers and collectors:
-
To ensure that the PhoneHome service is active on the Database Server and view additional configuration information about the server:
systemctl status phd.service
-
If the PhoneHome service is disconnected on a Database Server or SL1 Collector and you want to start it:
systemctl start phc
Viewing a List of PhoneHome Devices
The phonehome list command lists all of the PhoneHome devices in your SL1 system, including the Database Server and SL1 Collector, including the addresses for the Database Server and the remote IP address corresponding to the collectors.
To view a list of PhoneHome devices:
-
Go to the console of the SL1 Collector or Database Server.
-
Run the following command on the SL1 Collector or Database Server:
sudo phonehome list
To view a list of only the PhoneHome Database Servers, run the following command:
sudo phonehome destination list
To view information about a specific PhoneHome Database Servers, run the following command:
sudo phonehome destination list --id <id>
where <id> is the PhoneHome device ID for the Database Server.
Viewing Information about a Single PhoneHome Device
The phonehome view command displays the state of a single PhoneHome device. This command must be run on a Database Server.
To view information about the PhoneHome configuration of a specific PhoneHome device:
-
Go to the console of the Database Server.
-
Run the following command on the SL1 Collector or Database Server:
sudo phonehome view <id>
where <id> is the PhoneHome device ID for the Database Server or SL1 Collector.
Renaming a PhoneHome Device
The phonehome rename command enables you to rename a PhoneHome device. You can run this command only from a Database Server, and you must know the PhoneHome device ID of the device that you want to rename.
To rename a PhoneHome device:
-
Go to the console of the Database Server.
-
Run the following command on the SL1 Collector or Database Server:
sudo phonehome rename <id><new_name>
where <id> is the PhoneHome device ID for the Database Server or SL1 Collector that you want to rename and <new_name> is the new name that you want to apply to the device.
Checking the Status of a PhoneHome Collector
The phonehome status command displays the status of the PhoneHome SL1 Collectors against all available databases. The output is tabular by default but supports JSON output as well. In the color output mode, the command will print the status of disconnected collectors in red.
The output does not contain the remote loopback IP address of PhoneHome SL1 Collectors, nor does it list PhoneHome Database Servers.
To check the status of a PhoneHome SL1 Collectors:
-
Go to the console of the SL1 Collector.
-
Run the following command:
sudo phonehome status
where you can optionally add the following parameters to the command:
-
-n to disable live probing to the collector and instead use the periodic server check results, which happens every minute by default
-
-x to enable extended output that includes a column indicating the last change timestamp
-
-c to disable color output
-
-j to output the data in JSON instead of a table
-
Checking the Connection Between PhoneHome Devices
The phonehome check command indicates the state of the connection from an SL1 Collector to the Database Server, visualizing the network path from the SL1 Collector to the Database Server as well as any proxy hops in between, if applicable. The output reports back any failures connecting to any hop.
To check the connection between PhoneHome devices:
-
Go to the console of the SL1 Collector or Database Server.
-
Run the following command on the SL1 Collector or Database Server:
sudo phonehome check -x
Syncing the Configuration of a PhoneHome System
The phonehome sync command syncs the configuration from the Database Server. This command can be run on the SL1 Collector.
To sync the configuration of a PhoneHome system:
-
Go to the console of the SL1 Collector.
-
Run the following command:
sudo phonehome sync
Managing Port Forwarding for PhoneHome Communication
A port forward is a configuration that allows a PhoneHome client to "copy" a local port from the SL1 Collector to the Database Server, essentially making the local port available on the Database Server appliance as if it was physically present on that appliance itself.
The local MariaDB port 7707 on the collector is forwarded to the Database Server by default.
Viewing a List of Port Forwards
To view a list of ports forwarded from an SL1 Collector to the Database Server:
-
Go to the console of the SL1 Collector.
-
Run the following command on the SL1 Collector:
sudo phonehome forwards list
This list will not include the MariaDB port 7707, which is forwarded by default.
Adding a Port Forward
To add a port forward:
-
Go to the console of the SL1 Collector.
-
Run the following command on the SL1 Collector, replacing <Remote Port> with the port on the Database Server onto which the local port will be forwarded and <Local Port> with the local port to forward from the SL1 Collector:
sudo phonehome forward add <Remote Port> <Local Port>
Ports should be in the format :<port>.
The remote port should be an unprivileged port greater than 1023.
For example, if you want to forward SSH port 22 from the SL1 Collector to the Database Server appliance as port 10022 to enable a Database Server administrator to SSH into the SL1 Collector from the Database Server appliance, you would run the following command:
sudo phonehome forward add :10022 :22
New forwards do not take effect until the PhoneHome client is restarted or the next watchdog cycle occurs.
Removing a Port Forward
To remove a port forward:
-
Go to the console of the SL1 Collector.
-
Run the following command on the SL1 Collector, replacing <Remote Address> with the port on the Database Server appliance onto which the local port was forwarded and <Local Address> with the local port that was forwarded from the SL1 Collector:
sudo phonehome forward remove <Remote Address> <Local Address>
Addresses should be in the format :<port>.
For example, if you want to no longer forward SSH port 22 from the SL1 Collector to the Database Server appliance as port 10022, run the following command:
sudo phonehome forward remove :10022 :22
Deleted forwards do not take effect until the PhoneHome client is restarted or the next watchdog cycle occurs.
Managing Destinations
A destination is a list of addresses associated with a Database Server. A PhoneHome Database Server can have one or more destination addresses associated with it.
Destination addresses can be IPv4 addresses or DNS names, or a combination of both.
Viewing a List of Destinations
To view a list of all destinations in your stack:
-
Go to the console of the SL1 Collector.
-
Run the following command on the SL1 Collector:
sudo phonehome destination list
This command provides a list of all Database Servers with their device IDs, addresses, and priorities. Priorities define the order in which an SL1 Collector will try to connect to the individual addresses. It will keep trying until it can connect to one of them.
To view a list of destinations on a specific PhoneHome Database Server, run the following command, where <Device ID> is the ID of the PhoneHome Database Server:
phonehome destination list --id <Device ID>
Adding a Destination Address
To add a new destination address:
-
Go to the console of the SL1 Collector.
-
Run the following command on the SL1 Collector, where <Device ID> is the ID of the device to which you want to add a new address and <Address> is the destination address:
sudo phonehome destination add <Device ID> <Address>
Addresses should be in the format <host>:<port>.
Host addresses can be IPv4 addresses or DNS names.
If successful, you will get a message confirming that the new address was successfully added to the destination.
For example, if you wanted to add the destination address 192.168.0.13, with port 7705 open, to the device with the device ID 2, run the following command:
phonehome destination add 2 192.168.0.13:7705
The port you open must match the port that is open for the original device. Otherwise, you will receive an error.
Optionally, you can add the suffix --priority <Priority> to establish the destination's priority, or use the suffix --force to force add a destination address, even if the port does not match with the device's listed port.
Removing a Destination
To remove an existing address from a destination:
-
Go to the console of the SL1 Collector.
-
Run the following command on the SL1 Collector, where <Device ID> is the ID of the device from which you want to remove a destination address and <Address> is the destination address:
sudo phonehome destination remove <Device ID> <Address>
Addresses should be in the format <host>:<port>.
Host addresses can be IPv4 addresses or DNS names.
You cannot remove an address from a destination if it is the destination's only address.
Logging PhoneHome Configuration Information
In SL1, the server hosts are stored in the journald log for the phd service on the Database Server and in the journald log for the phc service on the Collector.
To view those logs, run the following commands on the Database Server or Collector:
sudo journalctl -u phd.service
sudo journalctl -u phc.service
Tuning PhoneHome Settings
A PhoneHome setting is a customizable configuration that impacts how a PhoneHome server or client behaves. Some settings impact both the server and client; others are localized to either just the server or just the client.
Updated PhoneHome settings do not take effect until the PhoneHome server or client is restarted or the next watchdog cycle occurs.
Viewing a List of Current PhoneHome Settings
To view a list of current PhoneHome settings:
-
Go to the console of the SL1 Collector or Database Server.
-
Run the following command on the SL1 Collector or Database Server:
sudo phonehome config list
When you run the command, the system returns a list that includes each configuration setting, its value, a description, and an indication of whether the setting affects the client, the server, or both.
Updating PhoneHome Settings
To set a new value for an existing PhoneHome setting:
-
Go to the console of the SL1 Collector or Database Server.
-
Run the following command on the SL1 Collector or Database Server:
sudo phonehome config set <setting_name> <new_value>
For example, if you want to change the client timeout value to 30 seconds, you would run the following command:
sudo phonehome config set client_timeout 30s
You can update the following settings:
Configuration | Setting | Description | Default Value | Affects |
---|---|---|---|---|
Client Timeout | client_timeout | Maximum amount of time allowed for the client to connect to a Database Server, after which the connection times out. The value is an actual time value, such as 30s, 5m, or 2h. | 30s | Client |
Exit on Forward Failure | exit_on_forward_failure | Indicates whether to close the connection to the Database Server if any custom ports fail to forward. This is not applicable to MariaDB port forwarding (port 7707). If the MariaDB port fails to forward, the client closes the connection regardless of this setting. The value is either true or false. | false | Client |
Watchdog Frequency Duration | watchdog_freq | Amount of time between watchdog service cycles. The value is an actual time value, such 30s, 5m, or 2h. | 1m0s | Both |
Port Ping Timeout | port_ping_timeout | Maximum allowed time for a Database Server's watchdog to connect to the forwarded port before it marks the SL1 Collector as disconnected and closes the incoming client connection. The value is an actual time value, such as 30s, 5m, or 2h. | 10s | Server |
Token Time to Live (TTL) | token_ttl | Default amount of time a token is valid before it expires. The value is an actual time value, such as 30s, 5m, or 2h. The maximum value is 2h. | 30m0s | Server |
Expired Token Cleanup Frequency | expired_token_cleanup_freq | Amount of time after which an expired token is deleted by the server. The value is an actual time value, such as 30s, 5m, or 2h. | 48h0m0s | Server |
Clearing a PhoneHome Device
The phonehome clear command clears the PhoneHome configuration on a PhoneHome device. This command will also disable the PhoneHome phd service if it is run on the Database Server.
For PhoneHome SL1 Collectors, the phonehome clear command clears the PhoneHome configuration, stops the PhoneHome client, and deletes the client keys. However, it will not delete the collector's configuration that is stored on the Database Server. To delete the Database Server's configuration related to the client, you must use the phonehome clear command on the SL1 Collector and then execute the phonehome delete command on the Database Server.
For PhoneHome Database Servers, the phonehome clear command clears the PhoneHome configuration and stops the PhoneHome server. You can also use the phonehome clear command on a Database Server to block future connection requests from Data Collectors and secondary Database Servers in an HA/DR configuration.
To clear a PhoneHome device:
-
Go to the console of the SL1 Collector or Database Server.
-
Run the following command on the SL1 Collector or Database Server:
sudo phonehome clear
For PhoneHome Database Servers, you can alternatively use the command phonehome clear -d. This deletes the device record associated with the Database Server, including the host key. For more information, see the section on Deleting a PhoneHome Database Server.
Deleting a PhoneHome Collector
You can delete a PhoneHome SL1 Collector only if it has no corresponding SL1 appliance.
Therefore, to delete a PhoneHome SL1 Collector, you must also perform the following steps, if applicable:
- If the SL1 Collector has a corresponding SL1 appliance, you must delete that appliance before you can delete the SL1 Collector.
- If the corresponding SL1 appliance is included in a collector group, you must delete that collector group before you can delete the appliance and then the SL1 Collector. If there are more than one collectors in the collector group, you can edit the collector group to deselect that collector instead of deleting the collector group.
- If the SL1 appliance's collector group includes other devices, you must move those devices to a different collector group before you can delete the appliance's collector group, then the appliance, and finally the SL1 Collector.
Once you delete a PhoneHome SL1 Collector, you cannot undelete it. Deleting an SL1 Collector will delete all configuration related to the device and cause all Database Servers to close incoming connections from the device.
To delete a PhoneHome SL1 Collector:
-
Go to the console of the SL1 Collector.
-
Run the following command on the SL1 Collector, replacing <id> with the PhoneHome device ID of the device you want to delete:
sudo phonehome delete <id>
This command only works for deleting a collector. You cannot use this command to delete a Database Server.
One of the following will occur:
-
If the device does not have a corresponding SL1 appliance on the stack, a confirmation prompt appears, asking you to confirm that you want to delete the device. Type "Y" and press Enter. The device is deleted and you can skip the rest of this section.
-
If the device does have a corresponding SL1 appliance, a message similar to the following appears:
Error: Cannot delete a phonehome device that has a corresponding appliance: [Module ID: 10, Name: example-device-cu1, CUG(s): cug-dc09]
If you receive an error message, proceed to the next step.
- Go to the Appliance Manager page (System > Settings > Appliances).
- Locate the device with the ID that matches the Module ID value that was returned in the error message in step 2, and then do one of the following:
- If the appliance is not part of a collector group, click its bomb icon () to delete it. You can then repeat steps 1 and 2 to delete the SL1 Collector.
- If the appliance is part of a collector group, the bomb icon is disabled. Proceed to the next step.
- Go to the Collector Group Management page (System > Settings > Collector Groups).
- Locate the collector group with the name that matches the CUG value that was returned in the error message in step 2, and do one of the following:
-
If the collector group does not contain any devices, click its bomb icon () to delete it. You can then repeat steps 3 and 4 to delete the appliance.
-
If the collector group contains devices, the bomb icon is disabled. Proceed to the next step.
- Go to the Device Manager page (Registry > Devices > Device Manager).
- Select the checkbox for each device that you want to move to a different collector group.
- In the Select Action field (in the lower right), select Change Collector Group and then select a collector group.
- Click the button. The selected devices will now be aligned with the selected collector group.
- Repeat steps 5 and 6, and then work your way backwards as needed, completing steps 3 and 4, followed by steps 1 and 2. Repeat these steps as needed until the device is deleted successfully in step 2.
Deleting a PhoneHome Database Server
To delete a PhoneHome Database Server:
-
Go to the console of the Database Server that you want to delete.
-
Run the following command:
sudo phonehome clear -d
A confirmation prompt appears, asking you to confirm that you want to delete the device. Type "delete" and press Enter.
You must run this command from the Database Server that you want to delete. You cannot run it from any other Database Server or the Administration Portal.
Once you delete a PhoneHome Database Server, you cannot undelete it. Deleting a Database Server will delete all configuration related to the device and close all incoming connections from PhoneHome SL1 Collectors.
Troubleshooting PhoneHome Configurations
This section describes how to troubleshoot issues some users experience when configuring PhoneHome communications.
Connectivity Issues from a Collector
You can run the following command on the SL1 Collector or Database Server to check connectivity issues:
sudo phonehome check -x
This command visualizes the network path from the SL1 Collector to the Database Server as well as any proxy hops in between, if applicable. The output reports back any failures connecting to any hop.
These are some of the common error messages seen with the disconnected host:
ssh: handshake failed: ssh: unable to authenticate, attempted methods [none publickey], no supported methods remain
There are two possible causes if the disconnected error is shown on the database host:
- Client keys have been reconfigured on the collector.
- The server does not have a valid record of the client. This would happen if a database administrator would delete the device record, but would not run clear on the collector itself.
If this happens on an intermediary proxy host, this means that the SSH key-based authentication has not been set properly with the proxy host.
ssh: handshake failed: knownhosts: key mismatch
This means there is an old entry for the given destination (or proxy) in /etc/phonehome/known_hosts that needs to be deleted from the file.
dial TCP <database_host_addr>:<port>: i/o timeout
This issue can be caused due to any of the following reasons:
- The Database Server is inaccessible or shut down.
- The Database Server is up but the phd service is down.
- A firewall rule has been added that prevents a connection from the SL1 Collector to the Database Server.
dial TCP <database_host_addr>:<port>: connect: no route to host
This error means that either the Database Server is shut down or it is experiencing a network connectivity issue.
dial TCP <database_host_addr>:<port>: connect: connection refused
This error means that the phd service on the Database Server host is not active/running.
Register Command Complains that the Token Has Expired
A PhoneHome token has a default time to live of 30 minutes, although this can be extended up to two hours using the command-line interface to generate the token. After this set time, the token expires. The register command lets you know that the token is expired and the Database Server will reject the request if you attempt to use it.
If this happens, you have two options:
- Ask the database administrator to issue you a new token since the old one has expired.
- Send a request from the SL1 Collector instead and let the database administrator know the one-time secret so they can accept the request on the Database Server.
You Cannot See a Request You Sent on the Server and You Cannot Send Another Request
When you send a request, the request is stored on the Database Server for an administrator to accept or reject. A request never expires.
If there is any failure with storing the request, the phonehome request send command will fail and display an error. This can happen if a database administrator deletes or rejects the request by mistake. The SL1 Collector does not get any feedback when an administrator rejects a request on the Database Server, and the tool prevents you from sending duplicate requests because it thinks that there is already a queued request.
You can override this by using the -f|--force flag with the phonehome request send command.
Status Shows Disconnected but the Check Succeeds
This means that the SL1 Collector is able to connect to the Database Server successfully but is failing to forward the ports.
Status changes are not immediate. To determine a collector's status, the Database Server needs to run a watchdog cycle, which happens every minute by default. Therefore, if you have very recently registered an SL1 Collector or restarted the phc service, wait for another watchdog cycle to see if the status changes from disconnected to forwarded. If this does not happen, you can check the logs for more details on the forwarding issue. To do so, use the following commands:
- On the Database Server: journalctl -u phd.service -f -n
- Client SL1 Collector: journalctl -u phc.service -f -n