19. Upgrading Robin Platform¶

Upgrading a Robin cluster involves several steps, updating multiple components and rigorous checks before after the process is complete to ensure all the necessary components are functioning appropriately. In order to ease this process, Robin ships two native scripts, the installer script and the GoROBIN utility tool, that can complete the upgrade process with as little as one command.

Review the details below and follow the necessary instructions given to upgrade a Robin cluster in a fast, reliable and painless manner.

19.1. Robin Upgrade Path Matrix¶

Robin Upgrade	Kubernetes Upgrade
5.3.0-198 -> 5.3.3-xxx	Not required

Note

Please use this as a reference document for performing upgrades to the 5.3.3 version of Robin.

19.2. Pre Robin Upgrade Checklist¶

If you are planning to use the Robin installer script to upgrade a Robin cluster ensure it is present on all nodes within the cluster alongside the Kubernetes/Robin Images tar files. If you are planning on using GoROBIN to perform the upgrade, this is not necessary.
Credentials for the super admin within the cluster (both username and password) must be used for all upgrade steps.
All nodes in the cluster must be correctly NTP time synced. If this is not the case, it could cause the upgade to fail.
If a Kubernetes upgrade is required as part of the upgrade flow any pods with a Pod Disruption Budget configured must be deleted beforehand.
No control plane operations, including those associated with applications and volumes, can be performed by any user(s) until the entire upgrade process has been completed successfully.

19.3. Manual upgrade via the Robin installer¶

19.3.1. Load Upgrade Images¶

This step downloads all the required Kubernetes/Robin images with regard to the upgrade process and can be executed by issuing the command described below.

Note

This step needs to be executed on all nodes within the Robin cluster and can be run in parallel on all of the aforementioned nodes.

# ./<Robin Installer> prepare-upgrade-hosts --robin-image-archive=<Location for Robin Images tar file> --k8s-image-archive=<Location for Kubernetes Images tar file>

19.3.2. Pre Robin Upgrade I¶

This step ensures all file collections within the cluster are offline as well as disabling Robin autopilot and HA before starting the upgrade. It can be executed with the command described below.

Note

This step needs to be executed only on the Master manager node of the Robin cluster.

# ./<Robin Installer> pre-robin-upgrade --username=<Robin Admin Username> --password=<Robin Admin Password>

19.3.3. Pre Robin Upgrade II¶

This step ensures all nodes within the cluster are ready for the upgrade. This involves making sure Robin watchdog is stopped on every node and all PGSQL instances on master nodes are synced. It can be executed with the command described below.

Note

This step needs to be executed on all nodes within the Robin cluster and can be run in parallel on all of the aforementioned nodes.

# ./<Robin Installer> pre-robin-upgrade-all-hosts --username=<Robin Admin Username> --password=<Robin Admin Password>

Note

When executing on the Master manager node of the Robin cluster the --primary-cp-node option needs to be included with the above command.

19.3.4. Robin Upgrade¶

This step upgrades the Calico, Multus, CSI, Robin master and Robin worker pods in the cluster. It can be executed with the command described below.

Note

This step needs to be executed only on the Master manager node of the Robin cluster.

# ./<Robin Installer> upgrade-robin --username=<Robin Admin Username> --password=<Robin Admin Password>

19.3.5. Kubernetes Upgrade¶

A Kubernetes upgrade is only required if indicated for your selected upgrade path in the matrix detailed here. The Kubernetes upgrade steps listed below need to be executed for every higher Kubernetes version in a sequential manner. For example when upgrading Kubernetes from 1.16.3 to 1.18.6, the command(s) listed below need to be executed twice; once when upgradring Kubernetes from 1.16.3 to 1.17.5 and another time when upgrading Kubernetes from 1.17.5 to 1.18.6. In order for the the Kubernetes upgrade for the cluster to be successful, it needs to be run and complete with no errors on the Master manager node of the cluster. After this is complete, the upgrade can be performed on the remaining nodes in the cluster in parallel. This step can be executed with the command described below.

Note

The Kubernetes upgrade step is not necessary for all Robin upgrades but instead is only required for a select few upgrade paths.

# ./<Robin Installer> upgrade-k8s --username=<Robin Admin Username> --password=<Robin Admin Password> --k8s-upgrade-version=<K8S version>

Note

The above command should be executed on the Master manager node of the Robin cluster first and as a result the --primary-cp-node option needs to be included when it is ran.

19.3.6. Post Robin Upgrade I¶

This step ensures all the nodes are in a healthy state after the upgrade and can be executed by issuing the command described below.

Note

This step needs to be executed on all nodes within the Robin cluster and can be run in parallel on all of the aforementioned nodes.

# ./<Robin Installer> post-robin-upgrade-all-hosts --username=<Robin Admin Username> --password=<Robin Admin Password>

19.3.7. Post Robin Upgrade II¶

This step ensures all file collections are brought online and re-enables Robin HA and Autopilot. It can be executed with the command described below.

Note

This step needs to be executed only on the Master manager node of the Robin cluster.

# ./<Robin Installer> post-robin-upgrade --username=<Robin Admin Username> --password=<Robin Admin Password>

19.4. Automated Upgrade via GoROBIN¶

All the upgrade steps listed above can be performed in an automated manner using the Robin utility script, GoROBIN. There are two options available when using this utility; namely one can either perform the entire upgrade with a single command or break it down into stages.

Note

Performing upgrades using GoROBIN requires root access to all nodes within the Robin cluster. This is because the script initiates an SSH session to access the aforementioned nodes and run commands during the upgrade process. Also, it is necessary that all nodes in the Robin cluster are reachable before starting the upgrade process using GoROBIN. Moreover, if all hosts in the Robin cluster have been configured to allow passwordless SSH access, users can utilize the --sshnopwd option in all commands to indicate no root password is needed for access.

19.4.1. Prepare cluster for upgrade via GoROBIN¶

In order to prepare a cluster for an upgrade, users can copy the GoRobin tar onto all nodes within the cluster beforehand with the below command. This is recommended when upgrading clusters that contain 15 or more nodes since the transfer of the tarfile to all nodes can take considerable amount of time.

# ./gorobin onprem copy-gorobin-tar --hosts <IP address of the Robin Primary Master Node> --gorobintar <Upgrade version GoRobin tar> --robin-admin-user <Robin Super Admin Username> --robin-admin-passwd <Robin Super Admin Password>

Note

This step is entirely optional and should only be run in the interest of time.

19.4.2. Single command upgrade via GoROBIN¶

Utilizing the below command enables Robin users to perform all the steps listed as part of manual upgrade with one action. Moreover, this command also performs the Kubernetes upgrade if it is required for the Robin upgrade.

# ./gorobin onprem upgrade-cluster --hosts <IP address of the Robin Primary Master Node> --gorobintar <Upgrade version GoRobin tar> --robin-admin-user <Robin Super Admin Username> --robin-admin-passwd <Robin Super Admin Password>

Note

If the GoROBIN tar is already copied to all nodes within the cluster use the --skip-gorobintar-push parameter to avoid repeating the process. Also specify the --dir option to specify the destination tar folder if GoROBIN tar was manually copied to all nodes.

GoROBIN can also be used to perform upgrades on hosts with different login credentials using a simple JSON file similar to the one shown below. Users can specify host SSH usernames, passwords and ports via this JSON file. Make sure to specify the role attribute for each of the master nodes within the cluster. Specify the value as ‘primary’ for the Master manager within the cluster and ‘secondary’ for the other master nodes. The role attribute does not need to be specified for worker nodes in the cluster. In addition, the path to the host PEM file can be specified within the JSON file instead of the SSH password. An example of this is also shown below.

{
    "10.10.10.2":{
        "password":"xxxxxxxxx",
        "user":"root",
        "port":"22",
        "role":"secondary"
    },
    "10.10.10.3":{
        "password":"/home/host-2.pem",
        "user":"root",
        "port":"22",
        "role":"primary"
    },
    "10.10.10.4":{
        "password":"/home/host-3.pem",
        "user":"root",
        "port":"22",
        "role":"secondary"
    },
    "10.10.10.5":{
        "password":"/home/host-4.pem",
        "user":"root",
        "port":"22"
    }
    "10.10.10.6":{
        "password":"/home/host-5.pem",
        "user":"root",
        "port":"22"
    }
}

To specify the hosts login credentials via a JSON file, issue the following GoROBIN command to execute the upgrade for the cluster:

# ./gorobin onprem upgrade-cluster --hosts-json <Hosts login credentials and ports JSON file path> --gorobintar <Upgrade version GoRobin tar> --robin-admin-user <Robin Super Admin Username> --robin-admin-passwd <Robin Super Admin Password>

Note

Only one of either the --hosts-json or --hosts options can be specified.

19.4.3. Multistage upgrade via GoROBIN¶

Using GoROBIN, users are able to split their upgrade process into four main parts: pre-upgrade , upgrade, kubernetes-upgrade and post-upgrade. Given the flexibility this feature provides, the user inherently gains more control over the upgrade process.

Listed below are the four GoROBIN options that correspond to each of the aforementioned sections. These commands must be executed sequentially in order to result in a successful upgrade.

Note

For each of the given commands GoROBIN will copy the necessary Robin tar files to all nodes in the Robin cluster while executing the upgrade process. The user can skip this copying process (assuming the required files are already present on the respective machines) using the --skip-gorobintar-push parameter. Similarly, the --hosts-json option can be used for each of the commands in order to specify the login credentials of each host via a JSON file.

19.4.3.1. Pre-Robin Upgrade¶

This step will perform all the pre-upgrade steps on the Robin cluster and can be executed with the following command:

# ./gorobin onprem pre-upgrade-robin --hosts <IP address of the Robin Primary Master Node> --gorobintar <Upgrade version GoRobin tar> --robin-admin-user <Robin Super Admin Username> --robin-admin-passwd <Robin Super Admin Password>

19.4.3.2. Robin Upgrade¶

This step will perform the actual upgrade on the Robin cluster and can be executed with the following command:

# ./gorobin onprem upgrade-robin --hosts <IP address of the Robin Primary Master Node> --gorobintar <Upgrade version GoRobin tar> --robin-admin-user <Robin Super Admin Username> --robin-admin-passwd <Robin Super Admin Password>

19.4.3.3. Kubernetes Upgrade¶

This step performs the Kubernetes upgrade on the Robin cluster. If there are multiple Kubernetes versions involved in the Kubernetes upgrade then the command detailed below needs to be executed for each intermediate Kubernetes version.

Note

This step can be skipped for upgrade paths where a Kubernetes upgrade is not required.

# ./gorobin onprem upgrade-k8s --hosts <IP address of the Robin Primary Master Node> --gorobintar <Upgrade version GoRobin tar> --robin-admin-user <Robin Super Admin Username> --robin-admin-passwd <Robin Super Admin Password> --k8s-upgrade-version <K8S Version i.e 1.17.5>

19.4.3.4. Post-Robin Upgrade¶

This step will perform all post upgrade steps on the Robin cluster and can be executed with the following command:

# ./gorobin onprem post-upgrade-robin --hosts <IP address of the Robin Primary Master Node> --gorobintar <Upgrade version GoRobin tar>  --robin-admin-user <Robin Super Admin Username> --robin-admin-passwd <Robin Super Admin Password>

19.5. Post Upgrade Cluster Audit¶

After the upgrade process is complete, users can obtain a brief summary of all the updated components and services by executing the command shown below.

Note

This command can be run from any node with the Robin cluster using the installer script.

# ./<Robin Installer> audit-cluster --username=<robin admin username> --password=<robin admin password>

19.6. Upgraded Component Versions¶

Component	Upgraded Version
Calico	v3.12.3
Docker	19.03.6
SR-IOV device plugin	nfvpe/sriov-device-plugin:amd64
Multus	nfvpe/multus:v3.6
Containerd	1.2.13-3.2.el7.x86_64
MetalLB	v0.9.3