2. Installing Robin Platform¶
2.1. Minimum physical requirements¶
All nodes should be DNS resolvable resulting in internet connectivity between them.
All nodes should have a minimum of 4GB memory.
All nodes should have a minimum of 8 cores.
All storage requirements detailed below are met.
The interconnect network between the host supports 10G network bandwidth.
2.2. General configuration requirements¶
2.2.1. Mandatory requirements¶
The following operating systems are supported for Robin installation:
CentOS 7 or CentOS 8
Redhat Enterprise Linux (RHEL)
Oracle Enterprise Linux (OEL)
The minimum supported kernel version is 3.10.0-1062. To confirm that you are running a supported kernel, run this command:
# uname -r
In order to upgrade to the latest available kernel run this command:
# yum update kernel
After the above step is run the respective machine needs to be rebooted to ensure it has been updated to the correct kernel version.
Highly available deployments of Robin require a minimum of three nodes (five nodes are recommended), or a minimum of one node for non highly available deployments.
Note
For HA installations a virtual IP address (VIP) is required. The IP address must be in the same subnet as that of the hosts on which Robin will be installed.
Check the status of the following services:
Firewall: Disabled
SELinux: Disabled (Only during installation)
NTP: Enabled, Up and Running
Swap must be disabled.
If the directory
/var/lib/dockeris present, it must be on an XFS filesystem.The locations
/,/varand/var/crashshould be on separate partitions.The
rootuser needs to be present in the sudoers file located at/etc/sudoersAny conflicting packages must be removed before installation or when indicated by an installation failure. In particular, on machines using CentOS 8 the
podmanandbuildahpackages should be removed. Note there can be more conflicting packages then the ones previously mentioned. Once the conflicting packages are removed rerun the installer if needed.Currently Robin supports GPU allocations only via the NVIDIA GPU operator. This operator works only on CentOS Kernel version 3.10.0-1160 and above.
Automatic detection of isolated CPUs only occurs when the respective hosts have been configured to have isolated cores via tuned and/or tuna settings. The cores which are not part of this isolated set will be set as the reserved CPUs for Kubelet. If the isolated cores are configured in an alternative manner they will have to be passed to the installer explicitly.
2.2.2. Storage requirements¶
For automatic storage configuration on the nodes, create /var and
/home folders which are at least 60GB and 240GB in size,
respectively. The Robin installer will create the folders underneath.
If you prefer to manually configure storage, separate volumes can be created following these requirements:
/var/lib/docker |
Directory in which the Docker images and metadata will be stored. Minimum 50GB in size, but can be sized according to the application spread. |
/var/lib/robin |
Directory in which the locally cached bundles and images will be stored. Minimum 20GB in size. Should be at least 50GB if LXC images will be used. |
/home/robinds |
Directory in which Robin config and Consul data will be stored. Minimum 20GB in size. |
/home/robinds/var/log |
Directory in which all the Robin log files will be stored. Minimum 60GB in size. Robin log files are capped at ~55G on master nodes and 30G on worker nodes. |
/home/robinds/var/crash |
Directory in which Robin core dump files will be stored. Minimum 100GB in size. This is sufficient to store data for at least 4 crashes. |
/home/robinds/var/lib/pgsql |
Directory in which the Robin database will be stored. Minimum 80GB in size. Needs to have sufficient space to hold the contents of the database, as well as a backup to support failover. |
Note
Robin platform only discovers and initializes unpartitioned drives for pod deployments. These drives should not be tagged or labeled.
2.2.3. Environment specific requirements¶
2.2.3.1. VMWware ESX requirements¶
Promiscuous Mode must be enabled
The MAC Address Change Policy setting must be set to Accept
The Forged Transmissions setting must be set to Accept
2.2.3.2. Port requirements on cloud platforms¶
If you are installing Robin on any cloud platform, the following ports need to be accessible for the installation to succeed and for Robin to operate correctly:
Ports |
Description |
22 * |
SSH |
179 |
BGP |
443 * |
UI Access |
2379 - 2380 |
ETCD Ports |
6443 ** |
K8s API Server |
29442 * |
Robin Server |
29443 * |
Robin UI |
29444 - 29446 |
Robin services |
29447 * |
NodeJS |
29448 - 29467 |
Robin services |
10250 |
Kubelet |
Note
The above list of ports also apply to on-premises clusters.
The ports above which are highlighted with an asterisk need to be exposed to all sources or at least the relevant range of sources that plan to access the Robin Cluster including the machines within the cluster. Other ports which do not have this distinction only need to be exposed to machines within the Robin cluster. The K8s API server port has a special distinction as it only needs to be exposed to all external sources when installing Robin on Google Cloud Platform via GoRobin.
Note
When creating each rule to make the port accessible, ensure that the protocol is TCP. In addition, all cloud nodes within the cluster should be able to communicate with one another via the IP-in-IP (4) Protocol.
To manage ports:
2.2.3.3. AWS EC2 requirements¶
A security group that conforms to the port requirements detailed here must be assigned to the instances where Robin will be installed or passed to the GoROBIN tool for AWS.
The smallest set of IAM permissions needed to install Robin are detailed below in JSON format. This can be pasted into the JSON tab when creating a Policy for the user:
{ "Version": "2012-10-17", "Statement": [ { "Sid": "VisualEditor0", "Effect": "Allow", "Action": [ "iam:CreateServiceLinkedRole", "iam:SimulatePrincipalPolicy", "iam:GetUser" ], "Resource": [ "arn:aws:iam::*:user/*", "arn:aws:iam::*:role/*" ] }, { "Sid": "VisualEditor1", "Effect": "Allow", "Action": [ "ec2:AttachVolume", "ec2:DescribeInstances", "connect:DescribeInstance", "ec2:RequestSpotInstances", "connect:ModifyInstance", "elasticloadbalancing:ConfigureHealthCheck", "ec2:DescribeRegions", "ec2:DescribeSpotInstanceRequests", "connect:DestroyInstance", "ec2:DescribeSpotPriceHistory", "ec2:DeleteVolume", "connect:CreateInstance", "ec2:DescribeNetworkInterfaces", "ec2:StartInstances", "ec2:DescribeAvailabilityZones", "connect:ListInstances", "ec2:DescribeVolumes", "ec2:ModifyInstanceAttribute", "elasticloadbalancing:RegisterInstancesWithLoadBalancer", "ec2:DetachVolume", "ec2:RebootInstances", "ec2:ModifyVolume", "elasticloadbalancing:CreateLoadBalancer", "ec2:TerminateInstances", "ec2:CreateTags", "ec2:ModifyNetworkInterfaceAttribute", "ec2:RunInstances", "ec2:StopInstances", "ec2:DescribeSecurityGroups", "ec2:CreateVolume", "ec2:EnableVolumeIO", "ec2:DescribeHosts", "ec2:DescribeImages", "ec2:CancelSpotInstanceRequests", "ec2:DescribeSubnets", "ec2:ModifyInstanceAttribute", "elasticloadbalancing:DescribeLoadBalancers", "elasticloadbalancing:DescribeTags", "elasticloadbalancing:DeleteLoadBalancer" ], "Resource": "*" } ] }
Note
These permissions must be assigned to the user whose credentials are passed to Robin during installation.
Robin requires both the access key and secret key of a user to be passed during both manual installation and automated installations via GoROBIN. Details on how to create/manage AWS credentials can be found here.
Note
The user which is identified by the given keys cannot have Multi-Factor Authentication enabled.
For the automated installation of Robin via the GoROBIN tool for AWS, a PEM key is required. Details on how to create/manage PEM keys via AWS can be found here.
2.2.3.4. Google Cloud platform requirements¶
Firewall rules that conform to the port requirements detailed here must be assigned to the instances where Robin will be installed, or associated with the project which will be passed to the GoROBIN tool for GCP.
Instances on which Robin will be installed need to have full access to cloud APIs. This can be set during instance creation.
For the automated installation of Robin via the GoROBIN tool for GCP, the following permissions need to be assigned to the service account that the tool will utilize: - Compute Instance Admin - Editor - Service Account Admin - Service Account User
2.2.3.5. GPU Based hosts requirements¶
Robin CNP supports only NVIDIA GPUs via the NVIDIA GPU operator.
In order to utilize the GPUs available on the host the operator must be installed. In order to setup the operator via Robin the
--install-gpu-operatoroption must be used when installing the initial master node.Currently, Robin only supports one type of GPU per host. In the case where there are multiple types of GPU, the most available resource at the time of installation will be selected.
2.2.4. Automated precheck¶
Robin provides a validation script that checks for the configuration requirements described above, and creates a detailed system check report. Please run this script before installation in order to make sure the hosts on which Robin is installed are setup correctly:
# robin-install-k8s-<VERSION>.sh precheck
Example:
# ./robin-install-k8s-dev_5.2.0-9103.sh precheck
ROBIN K8S Installer Precheck
Checking if Robin is currently installed... : DONE
Discovering Host Type... : DONE
Inspecting System Configuration : DONE
Checking CRI... : PASSED
Checking kernel version... : PASSED
Checking for curl ... : PASSED
Checking curl version >= 7.29.0 ... : PASSED
Checking SELinux status... : PASSED
Checking system FQDN... : PASSED
Checking localhost entry in /etc/hosts file... : PASSED
Checking system FQDN format in /etc/hosts... : PASSED
Checking if fqdn is resolvable... : PASSED
Checking / usage is less than 90%... : PASSED
Checking space in /tmp... : PASSED
Checking /var/lib/docker is on xfs... : PASSED
Checking /var/lib xfs dtype set... : PASSED
Checking space in /var/lib... : PASSED
Checking if /var/lib/docker is a partition... : WARNING
Checking space in /var/lib/docker... : WARNING
Checking for conflicting packages... : PASSED
Checking if directory /var/run/udev exists... : PASSED
Checking /tmp folder for permissions... : PASSED
Checking firewalls... : PASSED
Checking if network ports are in use... : PASSED
Checking hostname resolution... : PASSED
Checking if /var/lib/robin is a partition... : WARNING
Checking space in /var/lib/robin... : WARNING
Checking if /home/robinds/var/log is a partition... : WARNING
Checking space in /home/robinds/var/log... : WARNING
Checking if /home/robinds/var/crash is a partition... : WARNING
Checking space in /home/robinds/var/crash... : WARNING
Checking if /home/robinds is a partition... : WARNING
Checking space in /home/robinds... : PASSED
Checking if /home/robinds/var/lib/pgsql is a partition... : WARNING
Checking space in /home/robinds/var/lib/pgsql... : WARNING
Checking if Swap is disabled... : FAILED
Checking for incomplete YUM transactions... : PASSED
Robin Precheck warnings/errors
WARNING: /var/lib/docker not a partition (Folder not present)
WARNING: /var/lib/docker folder not present. Required space: 40G
WARNING: /var/lib/robin not a partition (Folder not present)
WARNING: /var/lib/robin folder not present. Required space: 20G
WARNING: /home/robinds/var/log not a partition (Folder not present)
WARNING: /home/robinds/var/log folder not present. Required space: 60G
WARNING: /home/robinds/var/crash not a partition (Folder not present)
WARNING: /home/robinds/var/crash folder not present. Required space: 100G
WARNING: /home/robinds not a partition
WARNING: /home/robinds/var/lib/pgsql not a partition (Folder not present)
WARNING: /home/robinds/var/lib/pgsql folder not present. Required space: 50G
Robin Precheck failures
FAILED: Swap is enabled on the host
Note
Please note that the Robin installer also runs the pre-check script by default and will not proceed with the install if there are any unsatisfied requirements (reported as pre-check failures in the report).
2.3. Installation files¶
The Robin installer is shipped as a package of 3 files. These files are necessary for the manual installation of Robin.
robin-install-k8s-<VERSION>.sh– the actual install scriptrobinimg-<VERSION>.tar.gz– contains Docker images of Robin softwarek8s-images-<VERSION>.tar.gz– contains Docker images of all Kubernetes components used by Robin
Note
To download Robin CNP installation files, contact your Robin Account Manager, Robin Support Team, or request for a demo here.
The installer itself is a single file, but the other tar files are provided so that the installation can proceed without requiring access to the Internet. Refer to the manual installation steps on how to use these files.
2.4. Installing Robin on-premises¶
2.4.1. Manual installation¶
2.4.1.1. Non-HA installation (for non-production deployments)¶
In a non-HA installation, the first machine is installed as a server, while the next machines are installed as agents. Run this command to install the Robin server on the first node:
# ./robin-install-k8s-<VERSION>.sh server --username=<admin_username> --password=<password> --robin-image-archive=/path_to/robinbinimg-<VERSION>.tar.gz --k8s-image-archive=/path_to/k8s-images-<VERSION>.tar.gz
Note
The Robin installer provides several command-line parameters to configure the installation for your environment. Please run the installation script with --help option to get a complete list of supported options.
Example:
# ./robin-install-k8s_5.0.6-479.sh server --username=robin --password=Robin123 --robin-image-archive=/tmp/robinbinimg-5.0.6-479.tar.gz --k8s-image-archive=/tmp/k8s-images-5.0.6-479.tar.gz
ROBIN K8S Server Installer
Inspecting System Configuration : DONE
Checking if Robin is currently installed.... : PASSED
Checking CRI... : PASSED
Checking kernel version... : PASSED
Checking for curl ... : PASSED
Checking SELinux status... : PASSED
Checking system FQDN... : PASSED
Checking localhost entry in /etc/hosts file... : PASSED
Checking system FQDN format in /etc/hosts... : PASSED
Checking if fqdn is resolvable... : PASSED
Checking /var/lib/docker is on xfs... : PASSED
Checking /var/lib/docker xfs dtype set... : PASSED
Checking /tmp folder for permissions... : PASSED
Checking / usage is less than 90%... : PASSED
Checking space in /tmp... : PASSED
Checking space in /var/lib/docker... : PASSED
Extracting Installer Payload : DONE
Installing Host Packages : DONE
Loading Kubernetes Images : DONE
Loading Robin Image : DONE
Setting up devmode : DONE
Setting up Host : DONE
Setting up Kubernetes Components : DONE
Registering Robin CRI : DONE
Setting up Networking Services : DONE
Setting up Helm : DONE
Creating Robin Master Service : DONE
Creating Robin Daemonset : DONE
Setting up Robin CSI : DONE
Waiting for deployment of Robin Daemonset: DONE
Successfully installed ROBIN Server
You can verify that the master was installed successfully by running these commands:
# robin login robin --password Robin123
User robin is logged in
# robin host list
Id | Hostname | Version | Status | State | Resource Pool | Roles | Cores | GPUs | Memory(Used/Alloc/Total) | HDD(#/Alloc/Total) | SSD(#/Alloc/Total) | Instances | Joined Time
-------------+--------------------------------+----------+--------+--------+---------------+-------+-------+------+--------------------------+--------------------+--------------------+-----------+----------------------
1548753123:1 | cscale-82-103.robinsystems.com | 5.0.2-65 | Ready | ONLINE | None | M* | 1/12 | 0/0 | 3.88 GB/1.14 GB/39.13 GB | 2/0 GB/600.0 GB | 0/0 GB/0 B | 0 | 28 Jan 2019 17:12:26
The next step is to install the Robin agent on all remaining nodes. Copy the three installation files to all nodes that are going to be part of the Robin cluster and run this command to install the agent:
# ./robin-install-k8s-<VERSION>.sh agent --server=<ip_address> --username=<admin_username> --password=<password> --robin-image-archive=/path_to/robinbinimg-<VERSION>.tar.gz --k8s-image-archive=/path_to/k8s-images-<VERSION>.tar.gz
Note
The variable <ip_address> refers to the IP address of the Robin MASTER Manager installed in the prior step.
Example:
# ./robin-install-k8s_5.0.6-479.sh agent --server=cscale-82-103.robinsystems.com --username=robin --password=Robin123 --k8s-image-archive=/root/k8s-images-5.0.6-479.tar.gz --robin-image-archive=/root/robinbinimg-5.0.6-479.tar.gz
ROBIN K8S Agent Installer
Extracting Installer Payload : DONE
Inspecting System Configuration : DONE
Installing Host Packages : DONE
Setting up Host : DONE
Loading Kubernetes Images : DONE
Loading Robin Image : DONE
Setting up Kubernetes Components : DONE
Registering Robin CRI : DONE
Joining Kubernetes Master : DONE
Creating Robin Daemonset : DONE
Setting up Robin Daemonset : DONE
Successfully joined this node to the ROBIN cluster
2.4.1.2. HA installation (for production deployments)¶
For production deployments, we highly recommend installing Robin in high availability mode to avoid single points of failure. The Robin cluster typically runs in HA mode with 3 master nodes.
Run this command to install Robin on the first master node:
# ./robin-install-k8s-<VERSION>.sh master --vip=<virtual_ip_address> --vip-type=robin --vrid=<vrid> --username=<admin_username> --password=<password> --robin-image-archive=/path_to/robinbinimg-<VERSION>.tar.gz --k8s-image-archive=/path_to/k8s-images-<VERSION>.tar.gz
Note
The variable <virtual_ip_address> is a virtual IP address that will be utilized by Kubernetes HAproxy. In addition the variable <vrid> is a unique virtual router ID in the VIP subnet used by Keepalived (must be an integer from 1 to 255).
Example:
# ./robin-install-k8s_5.0.6-479.sh master --vip=10.9.20.23 --vip-type=robin --vrid=88 --robin-image-archive=robinbinimg-5.0.6-479.tar.gz --k8s-image-archive=k8s-images-5.0.6-479.tar.gz --username=robin --password=Robin123
k8s master node/Robin Server Installer
Extracting Installer Payload : DONE
Inspecting System Configuration : DONE
Installing Host Packages : DONE
Setting up Host : DONE
Loading Kubernetes Images : DONE
Loading Robin Image : DONE
Setting up Kubernetes : DONE
Registering Robin CRI : DONE
Setting up Networking Services : DONE
Setting up Helm : DONE
Creating Robin Daemonset : DONE
Setting up Robin CSI : DONE
Setting up Robin Daemonset : DONE
Successfully installed ROBIN Master Server
For the installation on the subsequent master nodes, run this command:
# ./robin-install-k8s-<VERSION>.sh master --server=<ip_address> --username=<admin_username> --password=<password> --robin-image-archive=/path_to/robinbinimg-<VERSION>.tar.gz --k8s-image-archive=/path_to/k8s-images-<VERSION>.tar.gz
Note
Do NOT run this command simultaneously on two or more nodes. In addition the variable <ip_address> refers to the IP address of the Robin MASTER Manager node installed in the prior step.
Example:
# ./robin-install-k8s_5.0.6-479.sh master --server=centos-60-21.robinsystems.com --robin-image-archive=robinbinimg-5.0.6-479.tar.gz --k8s-image-archive=k8s-images-5.0.6-479.tar.gz --username=robin --password=Robin123
k8s master node/Robin Server Installer
Extracting Installer Payload : DONE
Inspecting System Configuration : DONE
Installing Host Packages : DONE
Setting up Host : DONE
Loading Kubernetes Images : DONE
Loading Robin Image : DONE
Setting up Kubernetes HA : DONE
Registering Robin CRI : DONE
Setting up Helm : DONE
Creating Robin Daemonset : DONE
Setting up Robin Daemonset : DONE
Successfully joined this node to the ROBIN cluster
You can verify that the three node cluster (currently made of three master nodes) is ready, by querying Robin and Kubernetes with these commands:
# robin login robin --password Robin123
User robin is logged in
# robin host list
Id | Hostname | Version | Status | State | Resource Pool | Roles | Cores | GPUs | Memory | HDD(Alloc/Total) | SSD(Alloc/Total) | Instances | Joined Time
-------------+-------------------------------+-----------+--------+--------+---------------+-------+-------+------+-----------------+------------------+------------------+-----------+----------------------
1536771934:1 | centos-60-21.robinsystems.com | 5.0.6-479 | Ready | ONLINE | None | M* | 0/12 | 0/0 | 1.0 GB/18.02 GB | 0 GB/600.0 GB | 0 GB/0 B | 0 | 12 Sep 2018 10:06:02
1536771934:2 | centos-60-22.robinsystems.com | 5.0.6-479 | Ready | ONLINE | None | M | 0/12 | 0/0 | 1.0 GB/18.02 GB | 0 GB/600.0 GB | 0 GB/0 B | 0 | 12 Sep 2018 10:23:00
1536771934:3 | centos-60-23.robinsystems.com | 5.0.6-479 | Ready | ONLINE | None | M | 0/12 | 0/0 | 1.0 GB/18.02 GB | 0 GB/600.0 GB | 0 GB/0 B | 0 | 12 Sep 2018 10:33:33
# kubectl get nodes
NAME STATUS ROLES AGE VERSION
centos-60-21 Ready master 31m v1.17.0
centos-60-22 Ready master 13m v1.17.0
centos-60-23 Ready master 2m v1.17.0
In order to add worker nodes to the Robin cluster, run this command:
# ./robin-install-k8s-<VERSION>.sh agent --server=<ip_address> --username=<admin_username> --password=<password> --robin-image-archive=/path_to/robinbinimg-<VERSION>.tar.gz --k8s-image-archive=/path_to/k8s-images-<VERSION>.tar.gz
Note
The variable
<ip_address>refers to the IP address of the Robin MASTER Manager node.If the new worker node is with GPU hardware, you can run the
--install-gpu-operatorcommand to install the GPU operator.
Example:
# ./robin-install-k8s_5.0.6-479.sh agent --server=centos-60-21.robinsystems.com --username=robin --password=Robin123 --k8s-image-archive=/root/k8s-images-5.0.6-479.tar.gz --robin-image-archive=/root/robinbinimg-5.0.6-479.tar.gz
ROBIN K8S Agent Installer
Extracting Installer Payload : DONE
Inspecting System Configuration : DONE
Installing Host Packages : DONE
Setting up Host : DONE
Loading Kubernetes Images : DONE
Loading Robin Image : DONE
Setting up Kubernetes Components : DONE
Registering Robin CRI : DONE
Joining Kubernetes Master : DONE
Creating Robin Daemonset : DONE
Setting up Robin Daemonset : DONE
Successfully joined this node to the ROBIN cluster
You can verify that the worker node has been added to the Robin cluster by running these commands:
# robin login robin --password Robin123
User robin is logged in
# robin host list
Id | Hostname | Version | Status | State | Resource Pool | Roles | Cores | GPUs | Memory | HDD(Alloc/Total) | SSD(Alloc/Total) | Instances | Joined Time
-------------+-------------------------------+-----------+--------+--------+---------------+-------+-------+------+-----------------+------------------+------------------+-----------+----------------------
1536771934:1 | centos-60-21.robinsystems.com | 5.0.6-479 | Ready | ONLINE | None | M* | 0/12 | 0/0 | 1.0 GB/18.02 GB | 0 GB/600.0 GB | 0 GB/0 B | 0 | 12 Sep 2018 10:06:02
1536771934:2 | centos-60-22.robinsystems.com | 5.0.6-479 | Ready | ONLINE | None | M | 0/12 | 0/0 | 1.0 GB/18.02 GB | 0 GB/600.0 GB | 0 GB/0 B | 0 | 12 Sep 2018 10:23:00
1536771934:3 | centos-60-23.robinsystems.com | 5.0.6-479 | Ready | ONLINE | None | M | 0/12 | 0/0 | 1.0 GB/18.02 GB | 0 GB/600.0 GB | 0 GB/0 B | 0 | 12 Sep 2018 10:33:33
1536771934:3 | centos-60-24.robinsystems.com | 5.0.6-479 | Ready | ONLINE | None | | 0/12 | 0/0 | 1.0 GB/18.02 GB | 0 GB/600.0 GB | 0 GB/0 B | 0 | 12 Sep 2018 10:44:57
2.4.2. Automated installation via GoROBIN¶
2.4.2.1. Non-HA installation¶
Robin provides a utility, GoROBIN, which automates the deployment, setup and teardown of Robin clusters on-premises. In addition to the script, you need the tarball with the three files required for an on-prem installation (see above). You can download the GoROBIN utility here and the accompanying tarball here.
You can use this command to create a non-HA Robin cluster:
# ./gorobin onprem install-nonha --hosts <hostnames> --gorobintar <path-to-gorobin-tarball> --pemfile <pemfile-path>
|
Operation to peform. In this case it is |
|
Comma-separated list of fully qualified hostnames to deploy Robin software on. |
|
Path to the Hosts login credentials and SSH ports JSON file. |
|
Path to the GoROBIN tarball containing installation files. |
|
Virtual IP address of the Kubernetes control plane (must be an IP address). |
|
Cluster-unique virtual router ID used by Keepalived (must be an integer from 1 to 255). |
|
Path to a JSON file containing extra configuration parameters for each host. |
|
License ID, obtained from https://get.robin.io, for cluster activation. |
|
PEM key file to access specified hosts. |
|
Root password to access specified hosts. |
|
Installation of GPU operator. |
|
Topology manager policy type. Options: none, best-effort, restricted, single-numa-node. |
Note
The required parameters for non-HA installations via GoROBIN are --hosts and --gorobintar along with the positional arguments. Other parameters such as --vip and vrid will be ignored as they dont pertain to this type of installation. In addition, at least one of the following parameters are required: --password or --pemfile. If necessary the --hosts-json option can be used instead of --hosts if the hosts within the Robin cluster have non-homogenous SSH password and port configurations.
Sample --hosts-json file
{
"<hostip_or_hostname>":{
"password":"xxxxxxxxx",
"user":"root",
"port":"22"
},
"<hostip_or_hostname>":{
"password":"/home/host-2.pem",
"user":"root",
"port":"22"
},
"<hostip_or_hostname>":{
"password":"/home/host-3.pem",
"user":"root",
"port":"22"
}
}
Example:
# ./gorobin onprem install-nonha --hosts cscale-82-34,cscale-82-35,centos-60-212 --gorobintar gorobintar-5.2.1-9567.tar --ignore-warnings
Please Enter the sshpassword for hosts in your Robin cluster. GoRobin will use this sshpassword to access all hosts in your cluster
Password:
Checking network connectivity to host cscale-82-34.robinsystems.com ... DONE (0 secs)
Checking network connectivity to host cscale-82-35.robinsystems.com ... DONE (0 secs)
Checking network connectivity to host centos-60-212.robinsystems.com ... DONE (0 secs)
Copying ROBIN tarball to '3' node(s) in the cluster ... DONE (50 secs)
Running Precheck on host cscale-82-34.robinsystems.com ... DONE (12 secs)
Running Precheck on host cscale-82-35.robinsystems.com ... DONE (10 secs)
Running Precheck on host centos-60-212.robinsystems.com ... DONE (15 secs)
Installing ROBIN software on '3' node(s) in the cluster ... DONE (609 secs)
Configuring 'cscale-82-34' as Master Node of cluster ... DONE (174 secs)
Adding '2' additional agent node(s) 'cscale-82-35.robinsystems.com, centos-60-212.robinsystems.com' to cluster ... DONE (492 secs)
Initializing Compute and Storage services on 3 node(s) ... DONE (50 secs)
Setting up file collection ... DONE (61 secs)
Adding Application Bundles ... DONE (100 secs)
Cluster with 3 node(s) is ready for use:
1. cscale-82-34.robinsystems.com
2. cscale-82-35.robinsystems.com
3. centos-60-212.robinsystems.com
Robin Cluster Name ..... cluster-r23g
Robin Admin Username ... admin
Robin Admin Password ... Robin123
Robin Admin Access ..... https://cscale-82-34.robinsystems.com
Note: Since a license was not provided at runtime, the cluster is not fully activated.
ROBIN was installed on the following hosts which had precheck warnings: cscale-82-34.robinsystems.com, cscale-82-35.robinsystems.com, centos-60-212.robinsystems.com.
2.4.2.2. HA installation¶
Robin provides a utility, GoROBIN, which automates the deployment, setup and teardown of Robin clusters on-premises. In addition to the script, you need the tarball with the three files required for an on-prem installation (see above). You can download the GoROBIN utility here and the accompanying tarball here.
You can use this command to create an HA Robin cluster:
# ./gorobin onprem install-ha --hosts <hostnames> --gorobintar <path-to-gorobin-tarball> --pemfile <pemfile-path> --vip <virtual-ip> --vrid <virtual-router-id>
|
Operation to peform. In this case it is |
|
Comma-separated list of fully qualified hostnames to deploy Robin software on. |
|
Path to the Hosts login credentials and SSH ports JSON file. |
|
Path to GoROBIN tarball containing installation files. |
|
Virtual IP address of the Kubernetes control plane (must be an IP address). |
|
Cluster-unique virtual router ID used by Keepalived (must be an integer from 1 to 255). |
|
Path to a JSON file containing extra configuration parameters for each host. |
|
License ID, obtained from https://get.robin.io, for cluster activation. |
|
PEM key file to access specified hosts. |
|
Root password to access specified hosts. |
|
Installation of GPU operator. |
|
Topology manager policy type. Options: none, best-effort, restricted, single-numa-node. |
Note
The required parameters for HA installations via GoROBIN are --hosts, --gorobintar, --vip, --vrid along with the positional arguments. In addition, at least one of the following parameters are required: --password or --pemfile. If necessary the --hosts-json option can be used instead of --hosts if the hosts within the Robin cluster have non-homogenous SSH password and port configurations.
Example:
# ./gorobin onprem install-ha --hosts cscale-82-34,cscale-82-35,centos-60-212 --gorobintar gorobintar-5.2.1-9567.tar --ignore-warnings --vrid 28 --vip 10.9.93.51
Please Enter the sshpassword for hosts in your Robin cluster. GoRobin will use this sshpassword to access all hosts in your cluster
Password:
Checking network connectivity to host cscale-82-34.robinsystems.com ... DONE (0 secs)
Checking network connectivity to host cscale-82-35.robinsystems.com ... DONE (0 secs)
Checking network connectivity to host centos-60-212.robinsystems.com ... DONE (0 secs)
Copying ROBIN tarball to '3' node(s) in the cluster ... DONE (50 secs)
Running Precheck on host cscale-82-34.robinsystems.com ... DONE (12 secs)
Running Precheck on host cscale-82-35.robinsystems.com ... DONE (10 secs)
Running Precheck on host centos-60-212.robinsystems.com ... DONE (15 secs)
Installing ROBIN software on '3' node(s) in the cluster ... DONE (609 secs)
Configuring 'cscale-82-34' as Master Node of cluster ... DONE (174 secs)
Adding '2' additional master node(s) 'cscale-82-35.robinsystems.com, centos-60-212.robinsystems.com' to cluster ... DONE (492 secs)
Initializing Compute and Storage services on 3 node(s) ... DONE (50 secs)
Setting up file collection ... DONE (61 secs)
Adding Application Bundles ... DONE (100 secs)
Cluster with 3 node(s) is ready for use:
1. cscale-82-34.robinsystems.com
2. cscale-82-35.robinsystems.com
3. centos-60-212.robinsystems.com
Robin Cluster Name ..... cluster-r23g
Robin Admin Username ... admin
Robin Admin Password ... Robin123
Robin Admin Access ..... https://cscale-82-34.robinsystems.com
Note: Since a license was not provided at runtime, the cluster is not fully activated.
ROBIN was installed on the following hosts which had precheck warnings: cscale-82-34.robinsystems.com, cscale-82-35.robinsystems.com, centos-60-212.robinsystems.com.
2.4.3. VLAN-based installation¶
VLANs allow a user to logically group a set of devices in the same L2 domain irrespective of how they are physically connected. Consequently this provides a variety of benefits from a networking prespective including isolation, security and flexibility. More details on VLANs can be found here.
A Robin cluster can be configured during the installation process so that VLAN support is natively enabled. This is done by providing the --nics option to installer. Here’s a list of all possible types of physical interface configurations and the corresponding options that need to be passed:
Interface without VLANs: To have VLAN support enabled in this case, this option should be provided during installation:
--nics=<nic>:<vlanNo>:untagged. The VLAN number specified here should be based on what is configured on the upstream switch.Interface with an IP Address and VLAN: To have VLAN support enabled in this case, this option should be provided during installation
--nics=<nic>:<vlanNo>:untaggedfor the interface without a VLAN configured. Its counterparts tagged with the VLANs will be automatically detected and registered.Interface with only VLANs: To have VLAN support enabled in this case, this option should be provided during installation:
--nics=<nic>:<vlanNo>. This is because all traffic leaving the interface is tagged with a VLAN number. Here the untagged option need not be specified, as the upstream switch expects only tagged traffic.
Providing the appropriate installer option (given the physical network configuration) along with options specified in the standard installation process, will result in a Robin cluster with VLAN support enabled.
2.4.4. Manual installation with HashiCorp Vault¶
Robin Cloud Native Platform (CNP) supports the integration of HashiCorp Vault as a key management service (KMS). Vault is a tool for securely accessing secrets. This includes but is not limited to API keys, passwords, and certificates. Vault provides a unified interface to any secret, while providing tight access control and recording a detailed audit log. More information on the product can be found here.
2.4.4.1. Prerequisites¶
The following prerequisites must be met to install Robin CNP with HashiCorp Vault integration.
All prerequisites that apply to general installations, detailed here, must be met.
A Vault Server must be pre-installed and functioning correctly.
The IP address of the Vault Server must be reachable from the Kubernetes node on which the Robin CNP installation will take place.
Signed x509 CA Certificates for the Vault Server and Vault Client must be present on the Kubernetes node where the Robin CNP installation will be initiated.
The key for the Vault Client CA Certificate must be present on the Kubernetes node where the Robin CNP installation will be initiated.
A Robin CNP folder, in which the master key will be placed, must be present on the Vault Server.
Note
HashiCorp Vault can only be integrated as part of the initial Robin CNP installation, it is not possible to configure it post-deployment.
2.4.4.2. How the Integration works¶
As part of the initial Robin CNP installation a one-time Kubernetes Job is spawned that sends an API request (using the given Vault parameters) to the Vault Server in order to create a key that is stored within the Vault Server database. Following this, the robin-kms init container fetches this key and encodes it in order to create the Robin CNP master key. This is then stored in-memory within the Robin CNP master Pod. When a PersistentVolume is provisioned on the Robin CNP cluster, a volume key is generated based on this master key in order to encrypt the aforementioned PV, and is stored within the Robin CNP database.
Some points to consider with regards to Vault integration include:
Either certificates or keys can be used to complete the integration
Integration with existing CNP clusters is not supported.
The in-memory master key will be deleted when the Robin CNP master Pod is deleted.
2.4.4.3. Necessary Parameters¶
The below parameters must be passed to the installation script in order to integrate Robin CNP with HashiCorp Vault:
vault-addr- URL at which to contact the Vault server consisting of its IP Address and the port number at which it can be reached.vault-keys-path- The path to the folder at which the key for Robin CNP will be stored in the Vault Server.vault-ca-cert- The path to the Vault Server CA certificate.vault-client-cert- The path to the Vault Client CA certificate.vault-client-key- The path to the Vault Client key.
Note
The above parameters all map to respective installer options that can be passed during the initial master installation. For example, --vault-addr is a valid option so on and so forth.
2.5. Uninstalling Robin on-premises¶
On bare metal nodes, Robin can be uninstalled either manually or via GoROBIN.
2.5.1. Manually uninstalling Robin¶
Run this command to uninstall Robin from a physical node:
# ./robin-install-k8s-<VERSION>.sh uninstall --yes
This command should be run on all machines within a Robin cluster to teardown the entire cluster.
Note
The same script that is used to install Robin can be used to uninstall it. In addition, if you are tearing down the entire cluster, use the --force option to ease the uninstall process.
Example:
# ./robin-install-k8s_5.0.6-479.sh uninstall --force --yes
ROBIN K8S Uninstaller
Extracting Installer Payload : DONE
Cleaning up Kubernetes : DONE
Removing Host Packages : DONE
Cleaning up Robin Daemonset folders : DONE
Cleaning up environment : DONE
Successfully uninstalled ROBIN Master Server
2.5.2. Automatically uninstalling Robin via GoROBIN¶
Robin provides a utility, GoROBIN, which automates the deployment, setup and teardown of Robin clusters on-premises. In addition to the script, you need the tarball with the three files required for an on-prem installation (see above). You can download the GoROBIN utility here and the accompanying tarball here.
You can use this command to create a HA Robin cluster:
# ./gorobin onprem teardown --hosts <hostnames> --gorobintar <path-to-gorobin-tarball> --pemfile <pemfile-path>
|
Operation to peform. In this case it is |
|
Comma-separated list of fully qualified hostnames to remove ROBIN software from. |
|
Path to the Hosts login credentials and SSH ports JSON file. |
|
Path to the GoROBIN tarball containing installation files. |
|
PEM key file to access specified hosts. |
|
Root password to access specified hosts. |
Example:
# ./gorobin onprem teardown --hosts centos-60-243.robinsystems.com,centos-60-244.robinsystems.com,centos-60-242.robinsystems.com --gorobintar gorobintar-5.2.0-34.tar
Please Enter the sshpassword for hosts in your Robin cluster. GoRobin will use this sshpassword to access all hosts in your cluster
Password:
- Checking network connectivity to host centos-60-243.robinsystems.com ... DONE (0 secs)
- Checking network connectivity to host centos-60-244.robinsystems.com ... DONE (0 secs)
- Checking network connectivity to host centos-60-242.robinsystems.com ... DONE (0 secs)
- Copying ROBIN tarball to '3' node(s) in the cluster ... DONE (50 secs)
- Uninstalling ROBIN software from '3' node(s) in the cluster ... DONE (180 secs)
ROBIN successfully uninstalled from hosts: centos-60-243.robinsystems.com, centos-60-244.robinsystems.com, centos-60-242.robinsystems.com
2.6. Installing Robin on AWS EC2¶
Robin can be installed from AWS instances either manually or via GoROBIN. The commands that need to be run are detailed below. However before issuing any of the given commands ensure all the prerequistes detailed here are met.
2.6.1. Manual installation¶
2.6.1.1. Non-HA installation (for non-production deployments)¶
In a non-HA installation, the first provisioned instance is installed as a server while the subsequent ones are installed as agents. Run this command to install the Robin server on the first instance:
# ./robin-install-k8s-<VERSION>.sh server --username=<admin_username> --password=<password> --robin-image-archive=/path_to/robinbinimg-<VERSION>.tar.gz --k8s-image-archive=/path_to/k8s-images-<VERSION>.tar.gz --host-type=ec2 --access-key=<access_key> --secret-key=<secret_key>
Note
The Robin installer provides several command-line parameters to configure the installation for your environment. Please run the installation script with --help option to get a complete list of supported options. The options displayed are mandatory however.
Example:
# ./robin-install-k8s_5.0.6-479.sh server --username=robin --password=Robin123 --robin-image-archive=/tmp/robinbinimg-5.0.6-479.tar.gz --k8s-image-archive=/tmp/k8s-images-5.0.6-479.tar.gz --host-type=ec2 --access-key=AKASDJ123JA --secret-key=Jqqqqiusan/1slla21/gg12WFG/AG
ROBIN K8S Server Installer
Inspecting System Configuration : DONE
Checking if ROBIN is currently installed.... : PASSED
Checking CRI... : PASSED
Checking kernel version... : PASSED
Checking for curl ... : PASSED
Checking SELinux status... : PASSED
Checking system FQDN... : PASSED
Checking localhost entry in /etc/hosts file... : PASSED
Checking system FQDN format in /etc/hosts... : PASSED
Checking if fqdn is resolvable... : PASSED
Checking /var/lib/docker is on xfs... : PASSED
Checking /var/lib/docker xfs dtype set... : PASSED
Checking /tmp folder for permissions... : PASSED
Checking / usage is less than 90%... : PASSED
Checking space in /tmp... : PASSED
Checking space in /var/lib/docker... : PASSED
Extracting Installer Payload : DONE
Installing Host Packages : DONE
Loading Kubernetes Images : DONE
Loading Robin Image : DONE
Setting up devmode : DONE
Setting up Host : DONE
Setting up Kubernetes Components : DONE
Registering Robin CRI : DONE
Setting up Networking Services : DONE
Setting up Helm : DONE
Creating Robin Master Service : DONE
Creating Robin Daemonset : DONE
Setting up Robin CSI : DONE
Waiting for deployment of Robin Daemonset: DONE
Successfully installed ROBIN Server
We can verify that the master installation was successful by running the following commands:
# robin login robin --password Robin123
User robin is logged in
# robin host list
Id | Hostname | Version | Status | State | Resource Pool | Roles | Cores | GPUs | Mem(Free/Alloc/Total) | HDD(#/Alloc/Total) | SSD(#/Alloc/Total) | Instances | Joined Time
-------------+---------------------------------------------+-----------+--------+--------+---------------+--------+-------+------+-----------------------+--------------------+--------------------+-----------+----------------------
1580933557:1 | ip-172-31-19-54.us-west-2.compute.internal | 5.0.5-693 | Ready | ONLINE | default | M*,C,S | 1/16 | 0/0 | 56G/6G/62G | -/-/- | 2/10G/200G | 0 | 05 Feb 2020 20:13:01
The next step is to install the Robin agent on all remaining nodes. Copy the three installation files to all nodes that are going to be part of the Robin cluster and run this command to install the agent:
# ./robin-install-k8s-<VERSION>.sh agent --server=<ip_address> --username=<admin_username> --password=<password> --robin-image-archive=/path_to/robinbinimg-<VERSION>.tar.gz --k8s-image-archive=/path_to/k8s-images-<VERSION>.tar.gz --host-type=ec2
Note
The variable <ip_address> refers to the private IP address of the instance which was installed as the Robin master node.
Example:
# ./robin-install-k8s_5.0.6-479.sh agent --server=172.31.19.54 --username=robin --password=Robin123 --k8s-image-archive=/root/k8s-images-5.0.6-479.tar.gz --robin-image-archive=/root/robinbinimg-5.0.6-479.tar.gz --host-type=ec2
ROBIN K8S Agent Installer
Extracting Installer Payload : DONE
Inspecting System Configuration : DONE
Installing Host Packages : DONE
Setting up Host : DONE
Loading Kubernetes Images : DONE
Loading Robin Image : DONE
Setting up Kubernetes Components : DONE
Registering Robin CRI : DONE
Joining Kubernetes Master : DONE
Creating Robin Daemonset : DONE
Setting up Robin Daemonset : DONE
Successfully joined this node to the Robin cluster
2.6.1.2. HA installation (for production deployments)¶
For production deployments, we highly recommend installing Robin in high availability mode to avoid single points of failure. The Robin cluster typically runs in HA mode with 3 master nodes.
Before starting the installation of ROBIN, an internal elastic loadbalancer will need to be created. The steps shown below detail this process:
Upon landing at the loadbalancer creation page, pick the type of the loadbalancer to be ‘Classic’, select the VPC where instances will be deployed, and select the option to create an internal load balancer.
Select Advanced VPC configuration and edit the listener configuration such that the Instance/Load Balancer Protocol is ‘TCP’ and the Instance/Load Balancer Port is 6443.
Select subnets, pick all 3 zones for better reliability. Note: this has nothing to do with the zone where your instances are deployed. Elastic Load Balancers are region-tied resources.
On the Assign Security Groups page, select or create a security group which permits access to all ports for protocol ‘TCP’.
On the Configure Health Check page, choose ‘TCP’ as Ping Protocol and ‘6443’ as Ping Port.
On the Add EC2 instances page, add the instances on which you are planning to install Robin and enable cross-zone load balancing.
Upon creation of the ELB, it will provide a DNS name (e.g internal-test-elb-1267768627.us-west-2.elb.amazonaws.com) through which the Kubernetes API Server can be accessed (post Robin installation).
Run this command to install Robin on the first master node:
# ./robin-install-k8s-<VERSION>.sh master --vip=<load_balancer_dns_name> --vip-type=external --vip-port=6443 --username=<admin_username> --password=<password> --robin-image-archive=/path_to/robinbinimg-<VERSION>.tar.gz --k8s-image-archive=/path_to/k8s-images-<VERSION>.tar.gz --host-type=ec2 --access-key=<access_key> --secret-key=<secret_key>
Note
The variable <load_balancer_dns_name> is the DNS name of the loadbalancer that was previously created. You can find this on the ELB details page on the AWS UI.
Example:
# ./robin-install-k8s_5.0.6-479.sh master --vip=internal-test-elb-1267768627.us-west-2.elb.amazonaws.com --vip-type=external --vip-port=6443 --robin-image-archive=robinbinimg-5.0.6-479.tar.gz --k8s-image-archive=k8s-images-5.0.6-479.tar.gz --username=robin --password=Robin123 --host-type=ec2 --access-key=AKASDJ123JA --secret-key=Jqqqqiusan/1slla21/gg12WFG/AG
k8s master node/Robin Server Installer
Extracting Installer Payload : DONE
Inspecting System Configuration : DONE
Installing Host Packages : DONE
Setting up Host : DONE
Loading Kubernetes Images : DONE
Loading Robin Image : DONE
Setting up Kubernetes : DONE
Registering Robin CRI : DONE
Setting up Networking Services : DONE
Setting up Helm : DONE
Creating Robin Daemonset : DONE
Setting up Robin CSI : DONE
Setting up Robin Daemonset : DONE
Successfully installed ROBIN Master Server
For installation on subsequent master nodes, run this command:
# ./robin-install-k8s-<VERSION>.sh master --server=<ip_address> --username=<admin_username> --password=<password> --robin-image-archive=/path_to/robinbinimg-<VERSION>.tar.gz --k8s-image-archive=/path_to/k8s-images-<VERSION>.tar.gz --host-type=ec2
Note
Do NOT run this command simultaneously on two or more
nodes. In addition the variable <ip_address> refers to the
private IP address of the MASTER Robin Manager instance installed in
the prior step.
Example:
# ./robin-install-k8s_5.0.6-479.sh master --server=172.31.19.54 --robin-image-archive=robinbinimg-5.0.6-479.tar.gz --k8s-image-archive=k8s-images-5.0.6-479.tar.gz --username=robin --password=Robin123 --host-type=ec2
k8s master node/Robin Server Installer
Extracting Installer Payload : DONE
Inspecting System Configuration : DONE
Installing Host Packages : DONE
Setting up Host : DONE
Loading Kubernetes Images : DONE
Loading Robin Image : DONE
Setting up Kubernetes HA : DONE
Registering Robin CRI : DONE
Setting up Helm : DONE
Creating Robin Daemonset : DONE
Setting up Robin Daemonset : DONE
Successfully joined this node to the ROBIN cluster
You can verify that the three node cluster (all master nodes) is ready by running these Robin and Kubernetes commands after logging into the AWS instance:
# robin login robin --password Robin123
User robin is logged in
# robin host list
Id | Hostname | Version | Status | State | Resource Pool | Roles | Cores | GPUs | Mem(Free/Alloc/Total) | HDD(#/Alloc/Total) | SSD(#/Alloc/Total) | Instances | Joined Time
-------------+---------------------------------------------+-----------+--------+--------+---------------+--------+-------+------+-----------------------+--------------------+--------------------+-----------+----------------------
1580933557:1 | ip-172-31-19-54.us-west-2.compute.internal | 5.0.5-693 | Ready | ONLINE | default | M*,C,S | 1/16 | 0/0 | 56G/6G/62G | -/-/- | 2/10G/200G | 0 | 05 Feb 2020 20:13:01
1580933557:2 | ip-172-31-7-111.us-west-2.compute.internal | 5.0.5-693 | Ready | ONLINE | default | M,C,S | 1/16 | 0/0 | 56G/6G/62G | -/-/- | 2/0G/200G | 0 | 05 Feb 2020 20:16:56
1580933557:3 | ip-172-31-41-159.us-west-2.compute.internal | 5.0.5-693 | Ready | ONLINE | default | M,C,S | 1/16 | 0/0 | 56G/6G/62G | -/-/- | 2/0G/200G | 0 | 05 Feb 2020 20:21:26
# kubectl get nodes
NAME STATUS ROLES AGE VERSION
ip-172-31-19-54.us-west-2.compute.internal Ready master 24h v1.17.0
ip-172-31-7-111.us-west-2.compute.internal Ready master 24h v1.17.0
ip-172-31-41-159.us-west-2.compute.internal Ready master 24h v1.17.0
To add worker nodes to the Robin cluster, run this command:
# ./robin-install-k8s-<VERSION>.sh agent --server=<ip_address> --username=<admin_username> --password=<password> --robin-image-archive=/path_to/robinbinimg-<VERSION>.tar.gz --k8s-image-archive=/path_to/k8s-images-<VERSION>.tar.gz
Note
The variable
<ip_address>refers to the private IP address of the instance which was installed as the MASTER Robin Manager node.If the new worker node is with GPU hardware, you can run the
--install-gpu-operatorcommand to install the GPU operator.
Example:
# ./robin-install-k8s_5.0.6-479.sh agent --server=172.31.19.54 --username=robin --password=Robin123 --k8s-image-archive=/root/k8s-images-5.0.6-479.tar.gz --robin-image-archive=/root/robinbinimg-5.0.6-479.tar.gz --host-type=ec2
ROBIN K8S Agent Installer
Extracting Installer Payload : DONE
Inspecting System Configuration : DONE
Installing Host Packages : DONE
Setting up Host : DONE
Loading Kubernetes Images : DONE
Loading Robin Image : DONE
Setting up Kubernetes Components : DONE
Registering Robin CRI : DONE
Joining Kubernetes Master : DONE
Creating Robin Daemonset : DONE
Setting up Robin Daemonset : DONE
Successfully joined this node to the ROBIN cluster
You can verify that the worker node has been added to the Robin cluster by running these commands:
# robin login robin --password Robin123
User robin is logged in
# robin host list
Id | Hostname | Version | Status | State | Resource Pool | Roles | Cores | GPUs | Mem(Free/Alloc/Total) | HDD(#/Alloc/Total) | SSD(#/Alloc/Total) | Instances | Joined Time
-------------+---------------------------------------------+-----------+--------+--------+---------------+--------+-------+------+-----------------------+--------------------+--------------------+-----------+----------------------
1580933557:1 | ip-172-31-19-54.us-west-2.compute.internal | 5.0.5-693 | Ready | ONLINE | default | M*,C,S | 1/16 | 0/0 | 56G/6G/62G | -/-/- | 2/10G/200G | 0 | 05 Feb 2020 20:13:01
1580933557:2 | ip-172-31-7-111.us-west-2.compute.internal | 5.0.5-693 | Ready | ONLINE | default | M,C,S | 1/16 | 0/0 | 56G/6G/62G | -/-/- | 2/0G/200G | 0 | 05 Feb 2020 20:16:56
1580933557:3 | ip-172-31-41-159.us-west-2.compute.internal | 5.0.5-693 | Ready | ONLINE | default | M,C,S | 1/16 | 0/0 | 56G/6G/62G | -/-/- | 2/0G/200G | 0 | 05 Feb 2020 20:21:26
1580933557:4 | ip-172-31-45-169.us-west-2.compute.internal | 5.0.5-693 | Ready | ONLINE | default | M,C,S | 1/16 | 0/0 | 56G/6G/62G | -/-/- | 2/0G/200G | 0 | 05 Feb 2020 20:33:26
2.6.2. Automated installation via GoROBIN¶
Robin provides a utility, GoROBIN, which automates the deployment, setup and teardown of Robin clusters on AWS. You can download the GoROBIN utility here.
Note
The prerequisites mentioned here still have to be met in order for the installation to succeed.
Detailed in the sections below are the command line parameters for different operations for the AWS flavor of GoROBIN.
2.6.2.1. HA installation¶
You can use this command to create an HA Robin cluster on AWS:
# ./gorobin aws install-ha --access-key <access_key> --secret-key <secret_key> --region <region> --zone <zone1:num,zone2:num...> --ami <ami_id> --pem-keyname <pem_keyname> --security-group <sec_group_id>
Note
The options shown above are all mandatory. For more options please refer to the section below.
|
Operation to peform. In this case it is |
|
AWS IAM Access Key. |
|
AWS IAM Secret Key. |
|
AWS Region to deploy in. |
|
AWS Availability Zones to deploy in. |
|
AMI ID to use. |
|
AWS Security Group ID. |
|
AWS PEM Key Name. |
|
AWS PEM Key File. |
|
Robin Cloud Platform Customer Name. |
|
AWS instance type. |
|
Number of EC2 instances to create. |
|
Number of EBS vols to attach per EC2 instance. |
|
EBS volume type. Options: gp2, io1, st2. |
|
Size of volume . |
|
Provision spot instances instead of regular ones. |
|
AWS Subnet ID to deploy instances in. |
|
Delete volumes upon termination of instances. |
|
User to login in as using the specified PEM key. |
|
Installation of GPU operator. |
|
Topology manager policy type. Options: none, best-effort, restricted, single-numa-node. |
Example:
# ./gorobin aws install-ha --access-key=AKASDJ123JA --secret-key=Jqqqqiusan/1slla21/gg12WFG/AG --region us-west-2 --zone us-west-2a:3 --security-group sg-05ac4f58104db6172 --pem-keyname robin-key --ami ami-015a728ae3ed933e8 --instance-type m4.4xlarge --instance-count 3 --volume-count 2 --volume-size 100GB
- Validating User Permissions ... DONE (0 secs)
- Validating security group 'sg-05ac4f58104db6172' meets port requirements ... DONE (0 secs)
- Creating 3 EC2 instance(s) of type 'm4.4xlarge' in zone us-west-2a ... DONE (17 secs)
- Creating Internal Elastic Load Balancer ... DONE (181 secs)
- Attaching 2x100.0 GiB EBS volumes to 'ec2-52-12-21-207' ... DONE (12 secs)
- Attaching 2x100.0 GiB EBS volumes to 'ec2-34-221-31-252' ... DONE (12 secs)
- Attaching 2x100.0 GiB EBS volumes to 'ec2-54-202-216-231' ... DONE (12 secs)
- Checking network connectivity to host 'i-02be2cbb9fd037a5c' 'ec2-52-12-21-207' ... DONE (1 secs)
- Checking network connectivity to host 'i-0101291a6d8e9adb8' 'ec2-34-221-31-252' ... DONE (0 secs)
- Checking network connectivity to host 'i-0b8018681694df01d' 'ec2-54-202-216-231' ... DONE (0 secs)
- Configuring Hosts for ROBIN installation ... DONE (10 secs)
- Configuring 'ec2-52-12-21-207' as Master Node of cluster ... DONE (324 secs)
- Adding '2' additional master nodes 'ec2-34-221-31-252.us-west-2.compute.amazonaws.com, ec2-54-202-216-231.us-west-2.compute.amazonaws.com' to cluster ... DONE (523 secs)
- Initializing Compute and Storage services on 3 nodes ... DONE (104 secs)
- Setting up file collection ... DONE (62 secs)
- Adding Application Bundles ... DONE (198 secs)
-----------------------------------------------------------------
Cluster with 3 node(s) is ready for use
1. ec2-52-12-21-207.us-west-2.compute.amazonaws.com 52.12.21.207 i-02be2cbb9fd037a5c
2. ec2-34-221-31-252.us-west-2.compute.amazonaws.com 34.221.31.252 i-0101291a6d8e9adb8
3. ec2-54-202-216-231.us-west-2.compute.amazonaws.com 54.202.216.231 i-0b8018681694df01d
Robin Cluster Name ..... lakshay@robin.io-robin-2h4tu0xf
Elastic Load Balancer .. elb-robin-2h4tu0xf
Robin Admin Username ... admin
Robin Admin Password ... Robin123
Robin Admin Access ..... https://ec2-52-12-21-207.us-west-2.compute.amazonaws.com
2.6.2.2. Non-HA installation¶
You can use this command to create a Non-HA Robin cluster on AWS:
# ./gorobin aws install-nonha --access-key <access_key> --secret-key <secret_key> --region <region> --zone <zone1:num,zone2:num...> --ami <ami_id> --pem-keyname <pem_keyname> --security-group <sec_group_id>
Note
The options shown above are all mandatory. For more options please refer to the section below.
|
Operation to peform. In this case it is |
|
AWS IAM Access Key. |
|
AWS IAM Secret Key. |
|
AWS Region to deploy in. |
|
AWS Availability Zones to deploy in. |
|
AMI ID to use. |
|
AWS Security Group ID. |
|
AWS PEM Key Name. |
|
AWS PEM Key File. |
|
Robin Cloud Platform Customer Name. |
|
AWS instance type. |
|
Number of EC2 instances to create. |
|
Number of EBS vols to attach per EC2 instance. |
|
EBS volume type. Options: gp2, io1, st2. |
|
Size of volume. |
|
Provision spot instances instead of regular ones. |
|
AWS Subnet ID to deploy instances in. |
|
Delete volumes upon termination of instances. |
|
User to login in as using the specified PEM key. |
|
Installation of GPU operator. |
|
Topology manager policy type. Options: none, best-effort, restricted, single-numa-node. |
Example:
# ./gorobin aws install-nonha --access-key=AKASDJ123JA --secret-key=Jqqqqiusan/1slla21/gg12WFG/AG --region us-west-2 --zone us-west-2a:3 --security-group sg-05ac4f58104db6172 --pem-keyname robin-key --ami ami-015a728ae3ed933e8 --instance-type m4.4xlarge --instance-count 3 --volume-count 2 --volume-size 100GB
- Validating User Permissions ... DONE (0 secs)
- Validating security group 'sg-05ac4f58104db6172' meets port requirements ... DONE (0 secs)
- Creating 3 EC2 instance(s) of type 'm4.2xlarge' ... DONE (17 secs)
- Attaching 2x100.0 GiB EBS volumes to 'ec2-52-27-16-48' ... DONE (11 secs)
- Attaching 2x100.0 GiB EBS volumes to 'ec2-34-221-220-2' ... DONE (5 secs)
- Attaching 2x100.0 GiB EBS volumes to 'ec2-54-184-55-50' ... DONE (5 secs)
- Finalizing AWS Configuration ... DONE (0 secs)
- Checking network connectivity to host 'i-03a6af0b3909f7004' 'ec2-34-221-220-2' ... DONE (32 secs)
- Checking network connectivity to host 'i-0d26ecbc54a81b739' 'ec2-52-27-16-48' ... DONE (0 secs)
- Checking network connectivity to host 'i-0554dda0e0c3fd61a' 'ec2-54-184-55-50' ... DONE (0 secs)
- Configuring 'ec2-52-27-16-48' as Master Node of cluster ... DONE (215 secs)
- Adding '2' additional agent nodes 'ec2-34-221-220-2.us-west-2.compute.amazonaws.com, ec2-54-184-55-50.us-west-2.compute.amazonaws.com' to cluster ... DONE (181 secs)
- Initializing Compute and Storage services on 3 nodes ... DONE (109 secs)
- Setting up file collection ... DONE (6 secs)
- Adding Application Bundles ... DONE (298 secs)
-----------------------------------------------------------------
Cluster with 3 node(s) is ready for use
1. ec2-34-221-220-2.us-west-2.compute.amazonaws.com 34.221.220.2 i-03a6af0b3909f7004
2. ec2-52-27-16-48.us-west-2.compute.amazonaws.com 52.27.16.48 i-0d26ecbc54a81b739
3. ec2-54-184-55-50.us-west-2.compute.amazonaws.com 54.184.55.50 i-0554dda0e0c3fd61a
Robin Cluster Name ..... lakshay-test-robin-6ghum936
Robin Admin Username ... admin
Robin Admin Password ... Robin123
Robin Admin Access ..... https://ec2-52-27-16-48.us-west-2.compute.amazonaws.com
-----------------------------------------------------------------
Cluster with 3 node(s) is ready for use
1. ec2-52-12-21-207.us-west-2.compute.amazonaws.com 52.12.21.207 i-02be2cbb9fd037a5c
2. ec2-34-221-31-252.us-west-2.compute.amazonaws.com 34.221.31.252 i-0101291a6d8e9adb8
3. ec2-54-202-216-231.us-west-2.compute.amazonaws.com 54.202.216.231 i-0b8018681694df01d
Robin Cluster Name ..... lakshay@robin.io-robin-2h4tu0xf
Elastic Load Balancer .. elb-robin-2h4tu0xf
Robin Admin Username ... admin
Robin Admin Password ... Robin123
Robin Admin Access ..... https://ec2-52-12-21-207.us-west-2.compute.amazonaws.com
2.7. Uninstalling Robin on AWS¶
Robin can be uninstalled from AWS instances either manually or via GoROBIN. The commands that need to be run are detailed below.
Note
If the AWS instances are terminated, the Robin cluster will obviously be torn down along with them.
2.7.1. Manual uninstallation of Robin¶
Run this command to uninstall Robin from an AWS instance:
# ./robin-install-k8s-<VERSION>.sh uninstall --yes
This command should be run on all the machines within a Robin cluster to teardown the entire cluster.
Note
The same script that is used to install Robin can be used to uninstall it. In addition, if you are tearing the down the entire cluster, use the --force option to ease the uninstall process.
Example:
# ./robin-install-k8s_5.0.6-479.sh uninstall --force --yes
ROBIN K8S Uninstaller
Extracting Installer Payload : DONE
Cleaning up Kubernetes : DONE
Removing Host Packages : DONE
Cleaning up Robin Daemonset folders : DONE
Cleaning up environment : DONE
Successfully uninstalled ROBIN Master Server
2.7.2. Automated uninstallation of ROBIN via GoROBIN¶
Robin provides a utility, GoROBIN, which automates the deployment, setup and teardown of Robin clusters in AWS. You can download the GoROBIN utility here.
This command can be run to teardown a Robin cluster via GoROBIN:
# ./gorobin aws teardown --access-key=<access_key> --secret-key=<secret_key> --region <region> --cluster-id <cluster_id>
Note
All parameters listed above are mandatory. In addition, the cluster-id for an installation can also be found within the tags of any instance in the cluster.
|
Operation to peform. In this case it is |
|
AWS IAM Access Key. |
|
AWS IAM Secret Key. |
|
AWS Region cluster is deployed in. |
|
Unique identifier for cluster, displayed at the end of a successful installation. |
Example:
# ./gorobin aws teardown --access-key=AKASDJ123JA --secret-key=Jqqqqiusan/1slla21/gg12WFG/AG --region us-west-2 --cluster-id demo-gorobin-ufqiy42x
- Validating User Permissions ... DONE (0 secs)
- Terminating Robin cluster 'demo-gorobin-ufqiy42x' and all its attached resources ... DONE (42 secs)
-----------------------------------------------------------------
Robin cluster 'demo-gorobin-ufqiy42x' and all its attached resources have been terminated.
2.8. Installing Robin on GCP¶
2.8.1. Manual non-HA installation (for non-production deployments)¶
In a non-HA installation, the first provisioned instance is installed as a server while the subsequent ones are installed as agents. Run this command to install the Robin server on the first instance:
# ./robin-install-k8s-<VERSION>.sh server --username=<admin_username> --password=<password> --robin-image-archive=/path_to/robinbinimg-<VERSION>.tar.gz --k8s-image-archive=/path_to/k8s-images-<VERSION>.tar.gz --host-type=gcp
Note
The Robin installer provides several command-line parameters to configure the installation for your environment. Please run the installation script with the --help option to get a complete list of supported options. The options displayed are mandatory however.
Example:
# ./robin-install-k8s_5.2.0-111.sh server --username=robin --password=Robin123 --robin-image-archive=/tmp/robinbinimg-5.2.0-111.tar.gz --k8s-image-archive=/tmp/k8s-images-5.2.0-111.tar.gz --host-type=gcp
ROBIN K8S Server Installer
Inspecting System Configuration : DONE
Checking if ROBIN is currently installed.... : PASSED
Checking CRI... : PASSED
Checking kernel version... : PASSED
Checking for curl ... : PASSED
Checking SELinux status... : PASSED
Checking system FQDN... : PASSED
Checking localhost entry in /etc/hosts file... : PASSED
Checking system FQDN format in /etc/hosts... : PASSED
Checking if fqdn is resolvable... : PASSED
Checking /var/lib/docker is on xfs... : PASSED
Checking /var/lib/docker xfs dtype set... : PASSED
Checking /tmp folder for permissions... : PASSED
Checking / usage is less than 90%... : PASSED
Checking space in /tmp... : PASSED
Checking space in /var/lib/docker... : PASSED
Extracting Installer Payload : DONE
Installing Host Packages : DONE
Loading Kubernetes Images : DONE
Loading Robin Image : DONE
Setting up devmode : DONE
Setting up Host : DONE
Setting up Kubernetes Components : DONE
Registering Robin CRI : DONE
Setting up Networking Services : DONE
Setting up Helm : DONE
Creating Robin Master Service : DONE
Creating Robin Daemonset : DONE
Setting up Robin CSI : DONE
Waiting for deployment of Robin Daemonset: DONE
Successfully installed ROBIN Server
You can verify that the master installation was successful by running the following commands:
# robin login robin --password Robin123
User robin is logged in
# robin host list
Id | Hostname | Version | Status | LastOpr | Resource Pool | Roles | Cores | GPUs | Mem(Free/Alloc/Total) | HDD(#/Alloc/Total) | SSD(#/Alloc/Total) | Instances | Joined Time
-------------+------------------------------------------------------------+-----------+--------+---------+---------------+--------+-------+------+-----------------------+--------------------+--------------------+-----------+----------------------
1582150592:1 | robin-f1m5ck1p-us-west1-b-000.c.rock-range-207622.internal | 5.2.0-111 | Ready | ONLINE | default | M*,C,S | 2/16 | 0/0 | 51G/7G/58G | 3/-/600G | -/-/- | 0 | 18 Feb 2020 20:17:04
The next step is to install the Robin agent on all remaining nodes. Copy the three installation files to all nodes that are going to be part ofc the Robin cluster and run this command to install the agent:
# ./robin-install-k8s-<VERSION>.sh agent --server=<ip_address> --username=<admin_username> --password=<password> --robin-image-archive=/path_to/robinbinimg-<VERSION>.tar.gz --k8s-image-archive=/path_to/k8s-images-<VERSION>.tar.gz --host-type=gcp
Note
The variable <ip_address> refers to the private IP address of the instance which was installed as the Robin master node.
Example:
# ./robin-install-k8s_5.2.0-211.sh agent --server=10.138.0.7 --username=robin --password=Robin123 --k8s-image-archive=/root/k8s-images-5.2.0-211.tar.gz --robin-image-archive=/root/robinbinimg-5.2.0-211.tar.gz --host-type=gcp
ROBIN K8S Agent Installer
Extracting Installer Payload : DONE
Inspecting System Configuration : DONE
Installing Host Packages : DONE
Setting up Host : DONE
Loading Kubernetes Images : DONE
Loading Robin Image : DONE
Setting up Kubernetes Components : DONE
Registering Robin CRI : DONE
Joining Kubernetes Master : DONE
Creating Robin Daemonset : DONE
Setting up Robin Daemonset : DONE
Successfully joined this node to the ROBIN cluster
2.8.2. Manual HA installation (for production deployments)¶
For production deployments, we highly recommend installing Robin in high availability mode to avoid single points of failure. The Robin cluster typically runs in HA mode with 3 master nodes.
Before starting the installation of ROBIN, an internal loadbalancer will need to be created. The steps below detail this process:
Create an Instance Group containing only the instance which is intended to be the first master of the Robin cluster. This in turn will require the instance group to be in same region and zone as the aforementioned instance. For every additional instance that is intended to be a master node, the instance group will have to be updated with the instance (only after Robin has been installed on it) if its in the same zone (a new instance group will have to be created if it’s in a different zone from the original).
Create a Health Check configuration, with the following parameters: TCP as the protocol and 6443 as the port with the interval being 10 seconds.
Upon landing at the loadbalancer creation page, pick the type of the loadbalancer to be ‘TCP’, and select the option to create a load balancer that balances traffic only between your VMs.
Create a backend service in the region where all your instances are located with the aforementioned instance group and health check configuration.
Create a frontend service whose IP address is automatically assigned and which only accesses a single port, 6443. A service label can be added but it is optional - it will create a DNS entry with the provided name.
Review and create the loadbalancer.
Upon creation of the load balancer, it will provide a DNS name (e.g demo.demo-docs-fip.il4.us-west1.lb.rock-range-207622.internal) through which the Kubernetes API Server can be accessed (post Robin installation).
Run this command to install Robin on the first master node:
# ./robin-install-k8s-<VERSION>.sh master --vip=<load_balancer_dns_name> --vip-type=external --vip-port=6443 --username=<admin_username> --password=<password> --robin-image-archive=/path_to/robinbinimg-<VERSION>.tar.gz --k8s-image-archive=/path_to/k8s-images-<VERSION>.tar.gz --host-type=gcp
Note
The variable <load_balancer_dns_name> is the DNS name of the loadbalancer that was previously created. You can find this on the loadbalancer details page on the GCP UI.
Example:
# ./robin-install-k8s_5.2.0-211.sh master --vip=demo.demo-docs-fip.il4.us-west1.lb.rock-range-207622.internal --vip-type=external --vip-port=6443 --robin-image-archive=robinbinimg-5.2.0-211.tar.gz --k8s-image-archive=k8s-images-5.2.0-211.tar.gz --username=robin --password=Robin123 --host-type=gcp
k8s master node/Robin Server Installer
Extracting Installer Payload : DONE
Inspecting System Configuration : DONE
Installing Host Packages : DONE
Setting up Host : DONE
Loading Kubernetes Images : DONE
Loading Robin Image : DONE
Setting up Kubernetes : DONE
Registering Robin CRI : DONE
Setting up Networking Services : DONE
Setting up Helm : DONE
Creating Robin Daemonset : DONE
Setting up Robin CSI : DONE
Setting up Robin Daemonset : DONE
Successfully installed ROBIN Master Server
For the installation on the subsequent master nodes, run this command:
# ./robin-install-k8s-<VERSION>.sh master --server=<ip_address> --username=<admin_username> --password=<password> --robin-image-archive=/path_to/robinbinimg-<VERSION>.tar.gz --k8s-image-archive=/path_to/k8s-images-<VERSION>.tar.gz --host-type=gcp
Note
Do NOT run this command simultaneously on two or more nodes. In addition, the variable <ip_address> refers to the private IP address of the MASTER Robin Manager instance installed in the prior step. Moreover, an instance group will have to be updated with this instance (a new one should be created along with a backend service if the instance is in a different zone) after the installation.
Example:
# ./robin-install-k8s_5.2.0-211.sh master --server=10.138.0.8 --robin-image-archive=robinbinimg-5.2.0-211.tar.gz --k8s-image-archive=k8s-images-5.2.0-211.tar.gz --username=robin --password=Robin123 --host-type=gcp
k8s master node/Robin Server Installer
Extracting Installer Payload : DONE
Inspecting System Configuration : DONE
Installing Host Packages : DONE
Setting up Host : DONE
Loading Kubernetes Images : DONE
Loading Robin Image : DONE
Setting up Kubernetes HA : DONE
Registering Robin CRI : DONE
Setting up Helm : DONE
Creating Robin Daemonset : DONE
Setting up Robin Daemonset : DONE
Successfully joined this node to the ROBIN cluster
You can verify that the three node cluster (all master nodes) is ready by running these Robin and Kubernetes commands after logging into the AWS instance:
# robin login robin --password Robin123
User robin is logged in
# robin host list
Id | Hostname | Version | Status | LastOpr | Resource Pool | Roles | Cores | GPUs | Mem(Free/Alloc/Total) | HDD(#/Alloc/Total) | SSD(#/Alloc/Total) | Instances | Joined Time
-------------+------------------------------------------------------------+-----------+--------+---------+---------------+--------+-------+------+-----------------------+--------------------+--------------------+-----------+----------------------
1582150592:1 | robin-z18ledep-us-west1-b-000.c.rock-range-207622.internal | 5.2.0-111 | Ready | ONLINE | default | M*,C,S | 2/16 | 0/0 | 51G/7G/58G | 3/-/600G | -/-/- | 0 | 19 Feb 2020 22:17:04
1582150592:2 | robin-z18ledep-us-west1-c-000.c.rock-range-207622.internal | 5.2.0-111 | Ready | ONLINE | default | M,C,S | 1/16 | 0/0 | 51G/6G/58G | 3/30G/600G | -/-/- | 0 | 19 Feb 2020 22:21:43
1582150592:3 | robin-z18ledep-us-west1-a-000.c.rock-range-207622.internal | 5.2.0-111 | Ready | ONLINE | default | M,C,S | 1/16 | 0/0 | 51G/6G/58G | 3/-/600G | -/-/- | 0 | 19 Feb 2020 22:25:46
# kubectl get nodes
NAME STATUS ROLES AGE VERSION
robin-z18ledep-us-west1-a-000 Ready master 24m v1.16.3
robin-z18ledep-us-west1-b-000 Ready master 38m v1.16.3
robin-z18ledep-us-west1-c-000 Ready master 28m v1.16.3
To add worker nodes to the Robin cluster, run this command:
# ./robin-install-k8s-<VERSION>.sh agent --server=<ip_address> --username=<admin_username> --password=<password> --robin-image-archive=/path_to/robinbinimg-<VERSION>.tar.gz --k8s-image-archive=/path_to/k8s-images-<VERSION>.tar.gz
Note
The variable <ip_address> refers to the private IP address of the instance which was installed as the MASTER ROBIN Manager node.
Example:
# ./robin-install-k8s_5.2.0-211.sh agent --server=10.138.0.8 --username=robin --password=Robin123 --k8s-image-archive=/root/k8s-images-5.2.0-211.tar.gz --robin-image-archive=/root/robinbinimg-5.2.0-211.tar.gz --host-type=gcp
ROBIN K8S Agent Installer
Extracting Installer Payload : DONE
Inspecting System Configuration : DONE
Installing Host Packages : DONE
Setting up Host : DONE
Loading Kubernetes Images : DONE
Loading Robin Image : DONE
Setting up Kubernetes Components : DONE
Registering Robin CRI : DONE
Joining Kubernetes Master : DONE
Creating Robin Daemonset : DONE
Setting up Robin Daemonset : DONE
Successfully joined this node to the ROBIN cluster
You can verify that the worker node has been added to the Robin cluster by running these commands:
# robin login robin --password Robin123
User robin is logged in
# robin host list
Id | Hostname | Version | Status | LastOpr | Resource Pool | Roles | Cores | GPUs | Mem(Free/Alloc/Total) | HDD(#/Alloc/Total) | SSD(#/Alloc/Total) | Instances | Joined Time
-------------+------------------------------------------------------------+-----------+--------+---------+---------------+--------+-------+------+-----------------------+--------------------+--------------------+-----------+----------------------
1582150592:1 | robin-z18ledep-us-west1-b-000.c.rock-range-207622.internal | 5.2.0-111 | Ready | ONLINE | default | M*,C,S | 2/16 | 0/0 | 51G/7G/58G | 3/-/600G | -/-/- | 0 | 19 Feb 2020 22:17:04
1582150592:2 | robin-z18ledep-us-west1-c-000.c.rock-range-207622.internal | 5.2.0-111 | Ready | ONLINE | default | M,C,S | 1/16 | 0/0 | 51G/6G/58G | 3/30G/600G | -/-/- | 0 | 19 Feb 2020 22:21:43
1582150592:3 | robin-z18ledep-us-west1-a-000.c.rock-range-207622.internal | 5.2.0-111 | Ready | ONLINE | default | M,C,S | 1/16 | 0/0 | 51G/6G/58G | 3/-/600G | -/-/- | 0 | 19 Feb 2020 22:25:46
1582150592:4 | robin-z18ledep-us-west1-a-001.c.rock-range-207622.internal | 5.2.0-111 | Ready | ONLINE | default | C,S | 1/16 | 0/0 | 51G/6G/58G | 3/-/600G | -/-/- | 0 | 19 Feb 2020 22:31:46
2.8.3. Automated installation via GoROBIN¶
Robin provides a utility, GoROBIN, which automates the deployment, setup and teardown of Robin clusters on GCP. You can download the GoROBIN utility here.
Note
The prerequisites mentioned here still have to be met in order for the installation to succeed.
Detailed below are the command line parameters for different operations for the GCP flavor of GoROBIN:
2.8.3.1. HA installation¶
You can run this command to create an HA Robin cluster on GCP:
# ./gorobin gcp install-ha --project-id <project_id> --service-email <service_email> --auth_json <path_to_auth_json> --region <region> --zone <zone1:num,zone2:num...> --image-name <image_name> --pem-keyname <pem_keyname> --pem-keyfile <path_to_pemkey>
Note
The options shown above are all mandatory. For more options please refer to the section below.
|
Operation to peform. In this case it is |
|
Project ID where instances are to be spun up, |
|
Email address associated with service account to be used. |
|
Path to Service Account Authentication JSON. |
|
GCP Region to deploy in. |
|
GCP Availability Zones to deploy in. |
|
Name of GCP Image to use. |
|
GCP PEM Key Name. |
|
Path to GCP PEM Key File. |
|
Robin Cloud Platform Customer Name. |
|
GCE instance type. |
|
Number of GCE instances to create. |
|
Number of disks to attach per GCE instance. |
|
GCE disk type. Options: pd-standard, pd-ssd. |
|
Size of volume. |
|
Provision preemptible instances instead of regular ones. |
|
GCE Subnet ID to deploy instances in. |
|
Delete volumes upon termination of instances. |
Example:
# ./gorobin gcp install-ha --project-id rock-range-207622 --service-email demogorobin@rock-range-207622.iam.gserviceaccount.com --auth_json ~/auth.json --region us-west1 --zone us-west1-a:1,us-west1-b:1,us-west1-c:1
--image-name robin-5-2-0-111 --pem-keyname robin-key.pem --pem-keyfile ~/.ssh/robin-key.pem --instance-count 3 --volume-count 3
- Creating 1 GCE instance(s) of type 'n1-standard-16' in zone us-west1-a ... DONE (14 secs)
- Creating 1 GCE instance(s) of type 'n1-standard-16' in zone us-west1-b ... DONE (56 secs)
- Creating 1 GCE instance(s) of type 'n1-standard-16' in zone us-west1-c ... DONE (59 secs)
- Creating GCE Load Balancer ... DONE (24 secs)
- Attaching 3x200.0 GiB volumes to instance 'demogorobin-robin-z18ledep-us-west1-b-000' ... DONE (31 secs)
- Attaching 3x200.0 GiB volumes to instance 'demogorobin-robin-z18ledep-us-west1-a-000' ... DONE (26 secs)
- Attaching 3x200.0 GiB volumes to instance 'demogorobin-robin-z18ledep-us-west1-c-000' ... DONE (28 secs)
- Checking network connectivity to host 'demogorobin-robin-z18ledep-us-west1-b-000' ... DONE (0 secs)
- Checking network connectivity to host 'demogorobin-robin-z18ledep-us-west1-c-000' ... DONE (0 secs)
- Checking network connectivity to host 'demogorobin-robin-z18ledep-us-west1-a-000' ... DONE (0 secs)
- Configuring 'demogorobin-robin-z18ledep-us-west1-b-000' as Master Node of cluster ... DONE (568 secs)
- Updating health check on node demogorobin-robin-z18ledep-us-west1-b-000 ... DONE (2 secs)
- Adding '2' additional master nodes 'demogorobin-robin-z18ledep-us-west1-c-000, demogorobin-robin-z18ledep-us-west1-a-000' to cluster ... DONE (510 secs)
- Updating health check on node demogorobin-robin-z18ledep-us-west1-c-000 ... DONE (2 secs)
- Updating health check on node demogorobin-robin-z18ledep-us-west1-a-000 ... DONE (3 secs)
- Initializing Compute and Storage services on 3 nodes ... DONE (47 secs)
- Setting up file collection ... DONE (41 secs)
- Adding Application Bundles ... DONE (121 secs)
-----------------------------------------------------------------
Cluster with 3 node(s) is ready for use
1. demogorobin-robin-z18ledep-us-west1-b-000 35.197.85.243
2. demogorobin-robin-z18ledep-us-west1-c-000 34.83.86.255
3. demogorobin-robin-z18ledep-us-west1-a-000 34.83.213.129
Robin Cluster Name ..... demogorobin-robin-z18ledep
Forwarding Rule (LoadBalancer) .. demogorobin-robin-z18ledep-elb
Robin Admin Username ... admin
Robin Admin Password ... Robin123
Robin Admin Access ..... https://35.197.85.243
2.8.3.2. Non-HA installation¶
You can use this command to create a non-HA Robin cluster on GCP:
# ./gorobin gcp install-nonha --project-id <project_id> --service-email <service_email> --auth_json <path_to_auth_json> --region <region> --zone <zone1:num,zone2:num...> --image-name <image_name> --pem-keyname <pem_keyname> --pem-keyfile <path_to_pemkey>
Note
The options shown above are all mandatory. For more options please refer to the section below.
|
Operation to peform. In this case it is |
|
Project ID where instances are to be spun up. |
|
Email address associated with service account to be used. |
|
Path to Service Account Authentication JSON. |
|
GCP Region to deploy in. |
|
GCP Availability Zones to deploy in. |
|
Name of GCP Image to use. |
|
GCP PEM Key Name. |
|
Path to GCP PEM Key File. |
|
Robin Cloud Platform Customer Name. |
|
GCE instance type. |
|
Number of GCE instances to create. |
|
Number of disks to attach per GCE instance. |
|
GCE disk type. Options: pd-standard, pd-ssd. |
|
Size of volume. |
|
Provision preemptible instances instead of regular ones. |
|
GCE Subnet ID to deploy instances in. |
|
Delete volumes upon termination of instances. |
Example:
# ./gorobin gcp install-nonha --project-id rock-range-207622 --service-email demogorobin@rock-range-207622.iam.gserviceaccount.com --auth_json ~/auth.json --region us-west1 --zone us-west1-a:1,us-west1-b:1,us-west1-c:1 --image-name robin-5-2-0-111 --pem-keyname robin-key.pem --pem-keyfile ~/.ssh/robin-key.pem --instance-count 3 --volume-count 3
- Creating 1 GCE instance(s) of type 'n1-standard-16' in zone us-west1-a ... DONE (14 secs)
- Creating 1 GCE instance(s) of type 'n1-standard-16' in zone us-west1-b ... DONE (56 secs)
- Creating 1 GCE instance(s) of type 'n1-standard-16' in zone us-west1-c ... DONE (59 secs)
- Attaching 3x200.0 GiB volumes to instance 'demogorobin-robin-z18ledep-us-west1-b-000' ... DONE (31 secs)
- Attaching 3x200.0 GiB volumes to instance 'demogorobin-robin-z18ledep-us-west1-a-000' ... DONE (26 secs)
- Attaching 3x200.0 GiB volumes to instance 'demogorobin-robin-z18ledep-us-west1-c-000' ... DONE (28 secs)
- Checking network connectivity to host 'demogorobin-robin-z18ledep-us-west1-b-000' ... DONE (0 secs)
- Checking network connectivity to host 'demogorobin-robin-z18ledep-us-west1-c-000' ... DONE (0 secs)
- Checking network connectivity to host 'demogorobin-robin-z18ledep-us-west1-a-000' ... DONE (0 secs)
- Configuring 'demogorobin-robin-z18ledep-us-west1-b-000' as Master Node of cluster ... DONE (568 secs)
- Adding '2' additional agent nodes 'demogorobin-robin-z18ledep-us-west1-c-000, demogorobin-robin-z18ledep-us-west1-a-000' to cluster ... DONE (510 secs)
- Initializing Compute and Storage services on 3 nodes ... DONE (47 secs)
- Setting up file collection ... DONE (41 secs)
- Adding Application Bundles ... DONE (121 secs)
-----------------------------------------------------------------
Cluster with 3 node(s) is ready for use
1. demogorobin-robin-z18ledep-us-west1-b-000 35.197.85.243
2. demogorobin-robin-z18ledep-us-west1-c-000 34.83.86.255
3. demogorobin-robin-z18ledep-us-west1-a-000 34.83.213.129
Robin Cluster Name ..... demogorobin-robin-z18ledep
Robin Admin Username ... admin
Robin Admin Password ... Robin123
Robin Admin Access ..... https://35.197.85.243
2.9. Uninstalling Robin on GCP¶
Robin can be uninstalled from GCP instances either manually or via GoROBIN. The commands that need to be run are detailed below.
Note
If the GCP instances are terminated, the Robin cluster will obviously be torn down alongside them.
2.9.1. Manual Uninstallation of Robin¶
Run this command to uninstall Robin from an GCP instance:
# ./robin-install-k8s-<VERSION>.sh uninstall --yes
This command should be run on all machines within a Robin cluster to teardown the entire cluster.
Note
The same script that is used to install Robin can be used to uninstall it. In addition, if you are tearing down the entire cluster, use the --force option to ease the uninstall process.
Example:
# ./robin-install-k8s_5.0.6-479.sh uninstall --force --yes
ROBIN K8S Uninstaller
Extracting Installer Payload : DONE
Cleaning up Kubernetes : DONE
Removing Host Packages : DONE
Cleaning up Robin Daemonset folders : DONE
Cleaning up environment : DONE
Successfully uninstalled ROBIN Master Server
2.9.2. Automated uninstallation of ROBIN via GoROBIN¶
Robin provides a utility, GoROBIN, which automates the deployment, setup and teardown of Robin clusters in GCP. You can download the GoROBIN utility here.
You can use this command to teardown a Robin cluster via GoROBIN:
# ./gorobin gcp teardown --cluster-id <cluster_id> --project-id <project_id> --service-email <service_email> --auth_json <path_to_auth_json> --region <region>
|
Operation to peform. In this case it is |
|
Project ID in which cluster is deployed. |
|
Email address associated with service account to be used. |
|
Path to Service Account Authentication JSON. |
|
GCP Region in which cluster is deployed. |
|
Unique identifier for cluster, displayed at the end of a successful installation. |
Note
The parameters shown above are all mandatory. In addition, the cluster-id for an installation can be found within the tags of any instance in the cluster.
Example:
# ./gorobin gcp teardown --cluster-id demo-gorobin-ufqiy42x --project-id rock-range-207622 --service-email demogorobin@rock-range-207622.iam.gserviceaccount.com --auth_json ~/auth.json --region us-west1
- Terminating Robin cluster 'demo-gorobin-ufqiy42x' and all its attached resources ... DONE (238 secs)
-----------------------------------------------------------------
Robin cluster 'demo-gorobin-ufqiy42x' and all its attached resources have been terminated.
2.10. Load Balancer Support via MetalLB¶
Robin utilizes the layer 2 mode of MetalLB in order to provide support for network load balancing on bare metal clusters. As a result, this allows users to deploy and effectively use Kubernetes services of type LoadBalancer in a production bare metal environment.
To enable this, pass the following option, --loadbalancer-iprange=<range_of_ips>, at the time of Robin software installation alongside the other parameters described in the various sections above. The IP range specified for the aforementioned option is seperate from
Robin IP Pool ranges and should be in the expanded format (eg: 192.168.10.20-192.168.10.30).
Moreover, MetalLB can be set up and removed on an existing Robin Cluster using installer script with the primary options described below.
2.10.1. Setup MetalLB Post Robin Installation¶
In order to setup MetalLB after Robin has been installed run the following command:
# ./robin-install-k8s-<VERSION>.sh setup-metallb --loadbalancer-iprange=<range_of_ips> --host-type=<host_type>
Example:
# ./robin-install-k8s_5.3.3-111.sh setup-metallb --loadbalancer-iprange=192.168.10.20-192.168.10.30 --host-type=physical
Extracting Installer Payload : DONE
Successfully setup MetalLB, Verify using command 'kubectl get pods -n metallb-system'
# kubectl get pods -n metallb-system
NAME READY STATUS RESTARTS AGE
controller-65895b47d4-lzqkt 1/1 Running 0 7s
speaker-xrxnq 1/1 Running 0 8s
2.10.2. Cleanup MetalLB¶
In order to remove MetalLB from the Robin Cluster run the following command:
# ./robin-install-k8s-<VERSION>.sh cleanup-metallb
Example:
# ./robin-install-k8s_5.3.3-111.sh cleanup-metallb
Extracting Installer Payload : DONE
Successfully cleaned up MetalLB
2.11. Robin Calico Typha Configuration¶
As a part of the Robin installation, Calico is deployed as a daemonset and the associated Calico Felix agent pods created on each node watch for events from the Kubernetes API server. When the size of the Kuberenetes cluster exceeds 50 nodes, Calico recommends deploying the Typha daemonset to act as an intermediatary between the datastore (the Kuberenetes API server in this case) and the aforementioned Calico Felix agents. This is because it enables increased scale by reducing each node’s impact on the datastore. Documentation on Calico Typha can be found here.
2.11.1. Configure Calico Typha¶
Users should be able to configure Calico Typha post Robin Installation/Upgrade using the following command:
# ./robin-install-k8s-<VERSION>.sh spawn-calico-typha --ip-protocol=<ipv4|ipv6>
Example:
# ./robin-install-k8s_5.3.3-111.sh spawn-calico-typha --ip-protocol=ipv4
Extracting Installer Payload : DONE
Configuring Calico Typha : DONE
Successfully configured Calico Typha
# kubectl get pods -A | grep calico-typha
kube-system calico-typha-55dc6794d7-hppq4 1/1 Running 0 12h
kube-system calico-typha-55dc6794d7-w8kcg 1/1 Running 0 12h
Note
It is also possible to configure Calico Typha as a part of the initial Robin install by passing the --spawn-calico-typha parameter during the first master installation.
2.12. High availability of Robin services¶
Robin manages the high availability of all management services that are deployed as part of a Robin installation. Robin pods are deployed as part of a daemonset. Some pods are designated to run master services. Robin configures 3 of these pods as manager pods which can host these master services.
If one of the Kubernetes nodes goes down, Robin seamlessly starts master services on other manager pods.
If a pod hosting Robin master services is removed from the Kuberenetes cluster, Robin will automatically designate another pod as manager pod, so that it always has 3 master pods.