Working with Managed Servers

Image Modified

The Managed Server page is where you can add a new server, view the list of existing servers, and view monitoring data on each server. From here, you may also access server settings and details for each server.

...

The Managed Server Section List provides server and monitoring data for each managed DNS / DHCP server.

Image Modified

The list provides the following information:

...

Action Menu (gear icon): Right click on the action menu (or anywhere on the row) to display available server actions

Working with the Managed Server List

The following actions may be performed when interacting with the server list:

Sort the list by clicking on the "Server Name" column, to view by Ascending / Descending order

...

Info

Before you begin, you will need to know following connection information for the new managed server:

Server FQDN or IP (only required for automatic installation)
Desired server OS
SSH credentials and port

...

To set up a new managed server, click "Add Server" at the top right of the Managed Server Tab.

Image Modified

From there, proceed through each of the five steps of the Add New Server wizard, below:

...

Expand

title	Step 2: Communication Settings...

Step 2: Communication Settings

Managed servers require a ProVision user account and API IP in order to communicate configuration, data, and monitoring information.

ProVision Username / Password: You may enter a specific ProVision username and password, or allow one to be automatically generated. If you enter an existing ProVision user, please ensure that the user is included under the 6cservers group and/or has full C/R/U/D resource permissions on the "6c server" resource.

IP API Address: The API address the managed server should use. There are cases when the DNS servers are in different networks and should communicate with ProVision on different IPs.

Click "Next" to proceed.

...

View or Edit Managed Server Details

Click on a server name, or open the server action menu to view details for a managed server.

The details of a server are organized into tabs: Settings, Services, Monitoring, and RPS.

...

Image Added

Settings

By default, only "Display Name", "Server OS" and "FQDN or IP" fields are shown.

Expand

Toggle on "Show Advanced Settings" to see the ssh credentials, "IP Api Address" and the max versions for each component:

Image Modified

Toggle on "Custom Monitoring Settings" to view/edit the monitoring settings:

Image Modified

This will be toggled if the user has set custom settings earlier or during the setup. The fields are the same as in the Monitoring Settings step during setup and are described there.

At the bottom left of the settings module, two buttons are available: "Run Actions" (including Push Install, Push Update, and Run Diagnostic options) and "Download Setup".

On the bottom right, you may click "Test Connection" to test the SSH connection to the server.

Image Modified

When any edits are complete, click "Save Changes".

Services

Under Services, you can view the enabled components for this server.

Expand

Image Modified

Click on a component to open a new tab with component details. In case of a "child" DHCP service, the link will lead to its parent.

Image Modified

Add Service

The "Add Service" button shows a popup to choose the service.

Image Modified

For DNS there is nothing to select:

Image Modified

For DHCP, the interface is the same as during the setup wizard:

Image Modified

Here, you may edit your settings if changes are needed.

Image Modified

When done, click "Save" to ensure any changes are saved, or click "Close" to exit without saving.

Monitoring

Under Monitoring, you can see the aggregated system statistics + "DNS aggregate":

Expand

Image Modified

Select a certain time interval (by default, the last 24h) and aggregation interval (5 min, 15 min, 30 min, 1 hour). Note - the charts are different for the different metrics.

For example, selecting "Dns Aggregate" shows a different set of information:

Image Modified

For the DNS records, we aggregate the overall total hits and the last time it was accessed. This is in order to be able to see the most/least used records and the ones which haven't been used for a long time and could be deleted. "Date From" and "Date To" here refer to the "Last Access".

If the request is for a record, for which we don't have resource, the ID will be 0. We have "pvverify.test" two times, because one of them is for "A" request and the other is for "TXT".

RPS (requests per second)

Under RPS, you can search for data for a given dns record.

Expand

Image Modified

The following search filters are available:

Date From/ Date To: Time interval to search

Action: Request or response

DNS Record: Search for a specific resource

Host: Search by host

Type: DNS request type (A, AAAA, CNAME, MX, TXT etc.)

...

Update / Install Managed Server (Existing Component)

The installation can be run more than once. If we already have running components, they will be stopped before the real installation continues.

If there is newer version of any component, it will be downloaded and used.

The overall process is:

Image Modified

Requirements

Python 3.6+ and Docker are the only requirements. If we run the installation with root (see below) and we have internet access, the installation process will install Docker.

The overall process is always the same - ProVision generates an install bundle, it is transferred to the server, extracted there and run some scripts from it. This can be done by ProVision (automatic install) or by the user (manual). In case of automatic install ProVision will automate the steps, which the user would do manually. All the components can run without root access, but we need it for user creation, folders creation, network and heartbeat service setup.

Automatic (Recommended)

This is the recommended and easiest setup method. If you have root access, ProVision can do all the necessary actions.

Go to the details page → Settings tab → "Run Actions" → "Push Install"

Image Modified

Confirmation dialog will be shown:

Image Modified

Popup with the progress will be shown.:

Image Modified

Each command/step is shown together with the overall progress.

...

Manual (from the command line)

...

This case is usually when the user doesn't want to input the root user in ProVision, but he has root access. The setup bundle should be downloaded and saved on your local computer:
Image Modified
The setup bundle name will be in format "setup-XXX.tar" where XXX is the resource ID of the managed server. The rest of the process is:
copy the file to the managed server (with SCP or any other suitable way)
extract it with "tar -xf setup-XXX.tar"
execute "sudo /bin/bash install_python3.sh" - this should finish with "PVOK"

Without root

This is the used if root access to the server is not available. In this case, an admin with root access should first do the following:

Install docker with the relevant commands for the OS. This can be done by running "install_docker.sh" from the setup bundle with root or any other suitable way

...

Code Block

language	bash
title	Install docker

...

Create the relevant user, add it to "docker" group, create "/provision" directory and make the created user owner of this directory.
User and directory setup Expand source

...

Make the necessary network changes. This can be done by running "network_config.sh" from the install bundle
Network config CentOS Expand source

Network config Ubuntu Expand source

...

Create service file and edit ﹤user﹥ with the correct system user.
pv_heartbeat.service Expand source

Make symlink to this file (replace /provision/libs/pv_heartbeat.service with the correct file path):
pv_heartbeat symlink Expand source
CentOS sudoers Expand source
Ubuntu sudoers Expand source

...

Reload the systemct daemon:
Reload services Expand source

From this point automatic or manual setup can be done with the non-root user

Update Managed Server

Updates may be automatic or manually performed.

Automatic:

In ProVision's Managed Servers page, open the server details. Then, under the settings tab, click "Run Actions, and select "Push Update".

Image Removed

Manual Update:

From the command line on the server, execute "python3 /provision/libs/pv_update.py"

The process of update of a component is as follow:

...

CentOS
collapse	true

yum update -y
yum install -y yum-utils bind-utils

yum-config-manager \
    --add-repo \
    https://download.docker.com/linux/centos/docker-ce.repo

yum install -y docker-ce docker-ce-cli containerd.io docker-compose-plugin

systemctl enable docker
systemctl start docker

docker run hello-world

Code Block

language	bash
title	Install docker Ubuntu
collapse	true

sudo apt-get update

sudo apt-get install -y \
    ca-certificates \
    curl \
    gnupg \
    lsb-release

# default start
KEYRING_FILE=/usr/share/keyrings/docker-archive-keyring.gpg
[ -f "$KEYRING_FILE" ] && rm $KEYRING_FILE

curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo gpg --dearmor -o $KEYRING_FILE

echo \
  "deb [arch=$(dpkg --print-architecture) signed-by=$KEYRING_FILE] https://download.docker.com/linux/ubuntu \
  $(lsb_release -cs) stable" | sudo tee /etc/apt/sources.list.d/docker.list > /dev/null

sudo apt-get update
sudo apt-get -y install docker-ce docker-ce-cli containerd.io
sudo docker run hello-world

Create the relevant user, add it to "docker" group, create "/provision" directory and make the created user owner of this directory.

Code Block

language	bash
title	User and directory setup
collapse	true

useradd -m <user>
passwd <user>
usermod -a -G docker <user>
mkdir -p /provision
chown -R <user>:<user> /provision

Make the necessary network changes. This can be done by running "network_config.sh" from the install bundle

Code Block

language	bash
title	Network config CentOS
collapse	true

yum update -y
yum install -y yum-utils bind-utils iptables-services

systemctl enable iptables
#iptables -I INPUT -p udp -m udp --dport 53 -j ACCEPT
#service iptables save

iptables -P INPUT ACCEPT
iptables -P FORWARD ACCEPT
iptables -P OUTPUT ACCEPT
iptables -t nat -F
iptables -t mangle -F
iptables -F
iptables -X
ip6tables -P INPUT ACCEPT
ip6tables -P FORWARD ACCEPT
ip6tables -P OUTPUT ACCEPT
ip6tables -t nat -F
ip6tables -t mangle -F
ip6tables -F
ip6tables -X
service iptables save

Code Block

language	bash
title	Network config Ubuntu
collapse	true

#!/usr/bin/sh
if [ "$(readlink /etc/resolv.conf)" != "/run/systemd/resolve/resolv.conf" ]; then
    ln -sf /run/systemd/resolve/resolv.conf  /etc/resolv.conf
fi

sed -i 's/^#DNSStubListener/DNSStubListener/;s/DNSStubListener=yes/DNSStubListener=no/' /etc/systemd/resolved.conf

systemctl stop systemd-resolved
systemctl start systemd-resolved

Create service file and edit ﹤user﹥ with the correct system user.

Code Block

language	bash
title	pv_heartbeat.service
collapse	true

# https://www.cloudsavvyit.com/3092/how-to-add-your-own-services-to-systemd-for-easier-management/
# https://www.opentechguides.com/how-to/article/centos/169/systemd-custom-service.html
# https://computingforgeeks.com/how-to-run-systemd-service-without-root-sudo/
# https://unix.stackexchange.com/a/497011

[Unit]
Description=pv_heartbeat Service, which reports the current status.
After=network.target
# StartLimitIntervalSec=0

[Service]
Type=simple
Restart=always
RestartSec=1
User=<user>
ExecStart=/usr/bin/python3 /provision/libs/pv_heartbeat.py

[Install]
# WantedBy=multi-user.target
WantedBy=deault.target
Alias=pv_heartbeat.service

Make symlink to this file (replace /provision/libs/pv_heartbeat.service with the correct file path):

Code Block

title	pv_heartbeat symlink
collapse	true

ln -s /etc/systemd/system/pv_heartbeat.service /provision/libs/pv_heartbeat.service

Create sudoers file (﹤user﹥ with the correct system user) and place it in /etc/sudoers.d:

Code Block

language	bash
title	CentOS sudoers
collapse	true

# https://unix.stackexchange.com/a/497011

Cmnd_Alias MANAGE_PV_HEARTBEAT = \
    /bin/systemctl enable  pv_heartbeat, \
    /bin/systemctl disable pv_heartbeat, \
    /bin/systemctl start   pv_heartbeat, \
    /bin/systemctl stop    pv_heartbeat, \
    /bin/systemctl restart pv_heartbeat, \
    /bin/systemctl status  pv_heartbeat

<user> ALL = (root) NOPASSWD: MANAGE_PV_HEARTBEAT

Code Block

language	bash
title	Ubuntu sudoers
collapse	true

# https://unix.stackexchange.com/a/497011

Cmnd_Alias MANAGE_PV_HEARTBEAT = \
    /usr/bin/systemctl enable  pv_heartbeat, \
    /usr/bin/systemctl disable pv_heartbeat, \
    /usr/bin/systemctl start   pv_heartbeat, \
    /usr/bin/systemctl stop    pv_heartbeat, \
    /usr/bin/systemctl restart pv_heartbeat, \
    /usr/bin/systemctl status  pv_heartbeat

<user> ALL = (root) NOPASSWD: MANAGE_PV_HEARTBEAT

Reload the systemctl daemon:

Code Block

language	bash
title	Reload services
collapse	true

systemctl daemon-reload

From this point automatic or manual setup can be done with the non-root user.

Update Managed Server

In ProVision's Managed Servers page, open the server details. Then, under the settings tab, click "Run Actions, and select "Push Update". ProVision will automatically create a backup file in /provision/backups/.

Image Added

...

Managed Server Diagnostics / Debug

...

Manual - Command Line Scripts

All the scripts are in "/provision/libs/".

All the scripts can be started with "-h" parameter to show relevant help.

pv_install.py - used for the whole setup. The process itself is broken into separate steps, which can be executed individually with "-e/--exec-only=". The steps can be seen from the help (-h). This should be used by developers and for debugging purposes
start_pv_*.sh - all these scripts are used to start given component. All of them support "-v/–version" to specify the version of the component to start and "-b/–bash" to start the container in debug mode. In debug mode the user will be logged into the container but without starting /start.sh (the entrypoint).
stop_pv_*.sh - all these scripts are used to properly stop given component/container

Local

In addition to logging into any of the containers (starting the relevant script with -b/–bash) few other files are included into the monitoring component:

mysql client for browsing the database
dhtest (link) for DHCP testing
dnsutils - dig, nslookup, nsupdate
telnet

...

Page tree

Versions Compared

Old Version 3

New Version 4

Key

Working with Managed Servers

Working with the Managed Server List

Step 2: Communication Settings

View or Edit Managed Server Details

Settings

Services

Add Service

Monitoring

RPS (requests per second)

Update / Install Managed Server (Existing Component)

Requirements

Automatic (Recommended)

Manual (from the command line)

Without root

Update Managed Server

Automatic:

Manual Update:

Update Managed Server

Managed Server Diagnostics / Debug

Manual - Command Line Scripts

Local

Page tree

Page History

Versions Compared

Old Version 3

New Version 4

Key

Working with Managed Servers

Working with the Managed Server List

Step 2: Communication Settings

View or Edit Managed Server Details

Settings

Services

Add Service

Monitoring

RPS (requests per second)

Update / Install Managed Server (Existing Component)

Requirements

Automatic (Recommended)

Manual (from the command line)

Without root

Update Managed Server

Automatic:

Manual Update:

Update Managed Server

Managed Server Diagnostics / Debug

Manual - Command Line Scripts

Local