3. Setting Up Your Robin Cluster¶
Before your cluster becomes fully operational, there are additional steps that need to be performed in order to configure Robin correctly.
Note
If your Robin cluster was created by GoROBIN, many of these steps will not apply.
3.1. Robin license activation¶
After installing Robin software on any platform, whenever you run a command a notice will appear stating that the Robin license is not currently activated. This is an indication that no license has currently been applied to the cluster and thus certain commands will not be usable. In order to utilize the cluster to its full capability a license must be applied to it. Described below are the types of licenses that can be activated in order to enable your cluster to function completely and how to acquire/activate each one.
Note
You must renew your license within 15 days after it expires. If you do not renew it, your cluster is locked after 15 days from the license expiry.
3.1.1. Types of License¶
Robin offers the following types of license for the Robin CNP:
Robin Enterprise trial license - This license is an evaluation license with unlimited capacity, 24x7 enterprise support, and a 30-day free trial.
Robin Express license - This license is a community license, free for a lifetime with limited capacity (only five nodes with five TB of storage within a cluster) and community support.
Robin Production license - This license is a custom production license for built specifically for customers that have a paid contract with Robin.io.
3.1.2. Activate Robin Enterprise trial license¶
The Robin Enterprise trial license is an evaluation license that offers unlimited capacity, 24x7 enterprise support, and a 30-day free trial. It allows full access to all the features of the Robin platform and enables users to experience the product fully.
After the successful installation of a Robin cluster, the Robin Enterprise trial license can be activated in the following two ways:
Using an auto-generated User ID
Using a license key
3.1.2.1. Method 1: Using a User ID¶
Complete the following steps to activate the Robin Enterprise trial license using a User ID:
Sign into the get.robin.io website by following the appropriate instructions.
After logging in, hover over the user icon in the upper righthand corner and copy the displayed User ID.
Login to your Robin CNP cluster and run the
robin license activatecommand with copied User ID as shown in the example below:
# robin license activate --userid 3D87AHGE
License has been applied successfully
3.1.2.2. Method 2: Using a License Key¶
The Robin Enterprise trial license can also be activated using a license key. In order to generate a license key on get.robin.io the cluster license ID must be provided. The complete steps are outlined below:
Login to your Robin CNP cluster and save the output of the following command:
# robin license id
Sign into the get.robin.io website by following the appropriate instructions.
After logging in, click the ACTIVATE tab and navigate to the Activate entering your license ID section.
In the ROBIN LICENSE ID field, enter the license ID from step 1 and select Robin Enterprise trial for the license type.
Click Activate in order to generate a license key and copy it.
Login to your Robin CNP cluster and run the
robin license applycommand with copied license key as shown in the example below:
# robin license apply lg48ugNqMOnHTelEE7KrnaY1IlGFeFlowPkXt6MQkoK3zJ4ULZvNZEKsq34gS5x6B7CrCEPP8t78xkya2xfj6XlVWHGo5/zFt6gEKGeugHef0r3RbJC4zMuGdiucREHMCbAFyZ/dYHGPSnod2I8pG5/adxdOtEFUwReYBIPgNBJ3Sh7WCt8d3xW/jLPVUUPfyqToAv2m6C8SZwlI1QDhp0/NQD8cJ5CJoI4bCJpOM6pNgfbMhY9nTjORLNERujdrYhrWojs/kasySwH4uOnIFzF9GVLD/HgYR8viZOrUs1yq72V0MP7CLAsxtjnMiNeIKxgnXSrHzJIgLZvqEm80Q5ZImTREtJU53KHv5xJ7EvflFzWa8JBzXhrOXywuI/CJpbFCPA+s7BbWEneG/7aw0M6zxFR6s7ItYAmXryEW3cXnyM19o5oHswrHY9WX/67sMXBmdhzUa/zVugJmDUSvtuKV6X8aXpp/hPFzMOTzkOfk/4/ACMNZB2wWPl9Wg9hyFfx1+JitUeKhuUmqMt7grLrCY2SP9wHAUXOwPUnrpmk88sJHWDpBKeVrSApaTHuH5lJFIMw7iatxQT7nw8hsJ5sz4sRoq0iyV/EFA1k+UjVyvP+oLRgUJur6VqalpoEo0oTlJ4pZX1g7iM5dhLn5lLn7RMpzylqs2/Lo2j8dQexE7kbFV34vSlL0/oqg6YjsjpC1SVd/L3X72PXydBM9eCv6P82Zj8nARVEMq0yGGUTkjPervGcNg0VAFyeL1mA3K0KeduzRKuM5cyixrfB3M7BdxJyJ3Ob0YHoHwKzGRCY0w9yhtUhR/mGMklXomjgjFdc+L92qi2QA/fZz4qbhSFTUxY/fzOdPZKZvEXYXR9oSOAOqDjfuOk8GPDAE2H6RDN5AJk3nn3ViNI8JGW3Tf+dLiFWHxfieIVMlpT45Fy5/24K1QfoSv4HQ/p5qaOdq8Ygw5kXmoQw8fujXdqRA8dnIEcxk9fgXoAVOjSSGe9tllqy1mzMGEpQxMC4REH4qNwQMJDk6TOjEKijwpHHLDvsirJoyx6wm65geBTUxJVjmQrh568fou2BuCJhkKLfKiRd0Pj+yjkDqtCjw21v8yh1H+gkIJIshBrAz3jQJ2O6DV0g3ka/akaVvmBhk8vUxyDFC2d6ixurmdGz44Cz3cDA1JHG3D4iciiErA+K+/Iz1RavVmwp/FGrI2W2QMOSzMRqJKTsq2zoGOYWWEdIIvYRaISAEmsKa4eSHnl5vjlf/4lhTLP/Il12nV1cG3zUXUlMiadEww4//NWhet91LInGr+R1Pjb1yYFPmx2
License has been applied successfully
3.1.3. Activate Robin Express license¶
The Robin Express license is a free-for-life license that offers a maximum of 5 nodes and a total of 5 TB of storage within a Robin cluster with community support. It allows users to have a small Robin cluster running for an unlimited period of time.
After the successful installation of a Robin cluster, the Robin Express license can be activated using a license key by completing the following steps:
Login to your Robin CNP cluster and save the output of the following command:
# robin license id
Sign into the get.robin.io website by following the appropriate instructions.
After logging in, click the ACTIVATE tab and navigate to the Activate entering your license ID section.
In the ROBIN LICENSE ID field, enter the license ID from step 1 and select Robin Express for the license type.
Click Activate in order to generate a license key and copy it.
Login to your Robin CNP cluster and run the
robin license applycommand with copied license key as shown in the example below:
# robin license apply lg48ugNqMOnHTelEE7KrnaY1IlGFeFlowPkXt6MQkoK3zJ4ULZvNZEKsq34gS5x6B7CrCEPP8t78xkya2xfj6XlVWHGo5/zFt6gEKGeugHef0r3RbJC4zMuGdiucREHMCbAFyZ/dYHGPSnod2I8pG5/adxdOtEFUwReYBIPgNBJ3Sh7WCt8d3xW/jLPVUUPfyqToAv2m6C8SZwlI1QDhp0/NQD8cJ5CJoI4bCJpOM6pNgfbMhY9nTjORLNERujdrYhrWojs/kasySwH4uOnIFzF9GVLD/HgYR8viZOrUs1yq72V0MP7CLAsxtjnMiNeIKxgnXSrHzJIgLZvqEm80Q5ZImTREtJU53KHv5xJ7EvflFzWa8JBzXhrOXywuI/CJpbFCPA+s7BbWEneG/7aw0M6zxFR6s7ItYAmXryEW3cXnyM19o5oHswrHY9WX/67sMXBmdhzUa/zVugJmDUSvtuKV6X8aXpp/hPFzMOTzkOfk/4/ACMNZB2wWPl9Wg9hyFfx1+JitUeKhuUmqMt7grLrCY2SP9wHAUXOwPUnrpmk88sJHWDpBKeVrSApaTHuH5lJFIMw7iatxQT7nw8hsJ5sz4sRoq0iyV/EFA1k+UjVyvP+oLRgUJur6VqalpoEo0oTlJ4pZX1g7iM5dhLn5lLn7RMpzylqs2/Lo2j8dQexE7kbFV34vSlL0/oqg6YjsjpC1SVd/L3X72PXydBM9eCv6P82Zj8nARVEMq0yGGUTkjPervGcNg0VAFyeL1mA3K0KeduzRKuM5cyixrfB3M7BdxJyJ3Ob0YHoHwKzGRCY0w9yhtUhR/mGMklXomjgjFdc+L92qi2QA/fZz4qbhSFTUxY/fzOdPZKZvEXYXR9oSOAOqDjfuOk8GPDAE2H6RDN5AJk3nn3ViNI8JGW3Tf+dLiFWHxfieIVMlpT45Fy5/24K1QfoSv4HQ/p5qaOdq8Ygw5kXmoQw8fujXdqRA8dnIEcxk9fgXoAVOjSSGe9tllqy1mzMGEpQxMC4REH4qNwQMJDk6TOjEKijwpHHLDvsirJoyx6wm65geBTUxJVjmQrh568fou2BuCJhkKLfKiRd0Pj+yjkDqtCjw21v8yh1H+gkIJIshBrAz3jQJ2O6DV0g3ka/akaVvmBhk8vUxyDFC2d6ixurmdGz44Cz3cDA1JHG3D4iciiErA+K+/Iz1RavVmwp/FGrI2W2QMOSzMRqJKTsq2zoGOYWWEdIIvYRaISAEmsKa4eSHnl5vjlf/4lhTLP/Il12nV1cG3zUXUlMiadEww4//NWhet91LInGr+R1Pjb1yYFPmx2
License has been applied successfully
3.1.4. Activate Robin Production license¶
When you purchase the production license for Robin CNP from Robin.io, a custom license contract is created based on the sales agreement made. This contract acts a template from which production licenses are generated. It contains the following details:
Authorized users
Owners
Contract end date
Maximum number of nodes
Renewal period, and so on.
Each contract is saved on get.robin.io which in turn is where all production licenses are created. Once the production license is generated, it can be activated in the following ways:
License proxy server - As the name suggests this option entails the usage of a stateful proxy server that contacts the get.robin.io website on behalf of Robin clusters that are otherwise on isolated networks. It can be setup on a low resource machine with Kuberentes installed and can be connected to multiple Robin clusters through the usage of a config variable. It enables the intial license activation of the aforementioned Robin clusters whilst also introducing the possibility of automatic renewal. Details on how to setup the proxy server are given below.
Manually - A production license can also be applied manually after it is generated on get.robin.io. Details on how to do so are described below. Note the same process can be utilized when the license needs to be renewed.
3.1.4.1. Method 1: Using a License Proxy server¶
A License Proxy server enables you to automatically activate and renew the licenses of the linked Robin CNP clusters, without any manual intervention.
Points to consider for License Proxy server
Each License Proxy server can handle multiple Robin CNP clusters but it can associate with only one license contract.
The License Proxy server is made known to each Robin CNP cluster through the config variables.
There must be constant connectivity between each respective Robin CNP cluster and the License Proxy server. If there is a drop in connectivity, alerts will be generated on the Robin CNP cluster until the issue is resolved.
Make sure that there is a constant connectivity between the License Proxy server and the get.robin.io website to activate and renew the licenses of all associated Robin CNP clusters automatically.
Although It is not mandatory but it is highly recommended that the License Proxy server should deploy on a machine that is not part of any of the Robin CNP clusters to be associated with it.
Prerequisites
Kubernetes v1.8 or above must be deployed on the machine where the proxy server will be deployed.
A request must be submitted to the Robin.io Customer Success team for access to the
license_proxy.tarandlicense_deployment.yamlfiles which contain the image and object spec for the proxy server.The contract to be associated with the proxy server must already be created. This can be confirmed with the Robin.io Customer Success team.
Deploy and activate License Proxy server
You can deploy and activate a License Proxy server either before installing Robin CNP or after installing Robin CNP.
Note
The deployment of the license proxy server is supported on Rocky Linux 8 or RHEL 8 machines.
The following are the steps to deploy, activate a License Proxy server and link it to the relevant Robin clusters:
Create server certificate for License Proxy server:
./license-proxy-cert.bin initialize-certs /root/license-proxy-certs --namespace default --ip-addrs 192.2.11.141 CA cert files created at the following locations: /etc/ssl/license_proxy_ca_ssl.key, /etc/ssl/license_proxy_ca_ssl.crt Successfully created file: /root/license-proxy-certs/license-proxy-server.cert Successfully created file: /root/license-proxy-certs/license-proxy-server.config Successfully created file: /root/license-proxy-certs/license-proxy-server.key Successfully created file: /root/license-proxy-certs/license-proxy-server.request Successfully created license-proxy-server secret in default
Create Client certificate and its key:
./license-proxy-cert.bin create-client-cert lak-test /root/license-proxy-certs --force Client key/cert files created at the following locations: /root/license-proxy-certs/lak-test.key, /root/license-proxy-certs/lak-test.crt
Using the
license_proxy.tarfile, load the docker image for the proxy server by running the following command:# docker load -i license-proxy.tar
To deploy the Kubernetes objects needed for the proxy server, including the Pod, PVC etc. run the following command:
# kubectl apply -f license-deployment.yaml
Ensure the License Proxy Pod is in a running state by issuing the following command:
# kubectl describe pod -n default license-proxy
After confirming the License Proxy is in a healthy state, execute into the main container and fetch the ID associated with it as shown in the example below. This will be used to activate the License Proxy server thus allowing it to serve activation and renewal requests.
# kubectl exec -ti -n default license-proxy -- robin proxy id
Sign into the get.robin.io website by following the appropriate instructions.
On the Activate page, click the ACTIVATE PROXY tab, and navigate to the Link a proxy to a contract section.
Select the contract to associate with the proxy, enter a name to help easily identify the proxy and input the proxy ID from Step 4.
To generate a proxy key, click Activate and copy it.
Activate the proxy with the generated key:
# kubectl exec -ti -n default license-proxy -- bash # robin proxy activate U7oF1y9WMpPRubRvFRDThbqIrel1TrflAkqsXqrRNmn3l6PH5BMQoNC2CyO1dwQqh8HpdWeAEeY3Wl8a0b6inU6nSEzya7KiTYftcfyryCX1vLEz9Eg20AJNMICOurbrQuENhMYxnAXfDotKhg6yAM8BZ51HpiVZq3P6WyHz00JnlzcPR+DAB4JN9rTkttn64CBApT6tUCpTzCqODfZqk55o3RnXCTMnuC4clRKJw6Ro1obQEpQiOMWTx0IC8RI1m1sik+ANESG4ap6Sn0xN9wFZZGgK6xqlI+GW8Ftr9Z5VFa+cD0L7tuVMSCNQihFV3jVk9KF77Ok7meCT1cbeAY1SHxW/l9ArcT4crieFzIXi1VeU8vjejFpPYO/6znmEPIKjlYgUDQFYojFvJXBoW2uTyNoPZIFgFCq3EWxs9jxM5o9KXCiFGZvDZpBrWJ9FWAUV6E+FeunEAcmPASqKYn2X0RCq5Y7MumgL+3OHdD5BL//koUWl/yS+ioi04IjXt+p/wbpYoqRF4tWgONLJhnvg9xQi/mCqGRhioL4DHuuvWSjCCb9T8F50ZszLTF15qMQMArSRhvAh3N/t3r56DnjBZbgc+ebKsnvlJT5SSE0+lrnfTPhl8tqQMhw3GaNTASjON4PuYVBD= Successfully activated Robin Proxy
Activate license using License Proxy server
After deploying and activating License Proxy server, you can activate license of a Robin CNP cluster.
Complete the following steps to activate license of a Robin CNP cluster:
Login to Robin CNP cluster using your username and password.
Update the license_server_url config variable with the License Proxy server URL to link the Robin CNP cluster with the License Proxy server to automatically activate or renew its license:
# robin config update cluster license_server_url http://192.0.2.55:31123 The 'cluster' attribute 'license_server_url' has been updated
If you have already specified the License Proxy server URL in the
--license-server-urloption as part of Robin CNP installation command, follow the step 3.Note
If the standard deployment YAMLs are used for the License Proxy server, the URL through which it can be contacted is: <host_ipaddress>:<svc_node_port>. The default NodePort used is 31123.
Copy the following certificates and key from the machine where you have setup License Proxy server to the cluster created during the deploy and activate License Proxy server:
CA certificate
Client certificate
Client key
Create certificate group for license proxy server:
# robin cert-group create license-proxy robin-server /etc/robin/rcm/ssl/license-proxy robin-master "ca.cert:/etc/ssl/license_proxy_ca_ssl.crt,client.cert:/root/license-proxy-certs/lak-test.crt,client.key:/root/license-proxy-certs/lak-test.key" {"response":"Cert Group 'license-proxy' created"} # robin cert-group list NAME | APP | DIRECTORY | PODS | NUM_FILES | VERSION --------------+--------------+----------------------------------+--------------+-----------+--------- license-proxy | robin-server | /etc/robin/rcm/ssl/license-proxy | robin-server | 3Bounce Robin Master Pod. Once Robin master Pod is bounced, License will be activated automatically.
Note
To activate or renew the license of a Robin CNP cluster on demand which is already linked to a License Proxy server, run the robin license renew command inside the Robin master Pod.
Important
Further details on the config variables, namely the license_server_url and license_renew_period attributes, that control the behavior of the License Proxy server from the perspective of the Robin cluster can be found Cluster wide Attributes.
3.1.4.2. Method 2: Apply a Production license manaully¶
If a license proxy server cannot be setup in the given environment, the Production license can also be generated and applied manually with the use of an authorization token. In order to generate a token on get.robin.io a contract must be available for use by the given user. The complete steps are outlined below:
Sign into the get.robin.io website by following the appropriate instructions.
After logging in, hover over the user icon in the upper right corner and click the Generate Token link.
Within the presented section, select the contract to associate with the token, and enter the number of days the token should be valid for.
Click Generate Token in order to generate a token and copy it.
Login to your Robin CNP cluster and run the
robin license renewcommand with copied token as shown in the example below:
# robin license renew --token vF6KtruI1mdDhm6ASbAMhMiuhHaR9J/iMUR/FtyMnXbDmkOyZr+Zu4Fb6BdnRLNyPtliHteH/I1+s51h/AWeQ==
License has been renewed successfully
3.2. License management¶
3.2.1. View License ID¶
Each Robin cluster has a unique, auto-generated identification number associated with it that enables license keys to be created specifically for the given cluster. In order to view the license ID of a cluster run the following command:
# robin license id --json
|
Output in JSON |
Example:
# robin license id
License Id: YGVkYWBjYGFiZWkhZ00lZk0gX1NcYWkDQDM5P0wENjNHNjNcQzAyOi75QF0ERyRcND0GREU1Q0z8QDQxPWjg.S_
Returns the unique, auto-generated identification number associated with a cluster that enables license keys to be created specifically for the given cluster.
End Point: /api/v3/robin_server/license
Method: GET
URL Parameters:
id=true: This mandatory parameter specifies that the License ID of the cluster should be returned.
Data Parameters: None
Port: RCM Port (default value is 29442)
Headers:
Authorization: <auth_token>: Authorization token to identify which user is sending the request. The token can be acquired from the login API.
Success Response Code: 200
Error Response Code: 500 (Internal Server Error), 401 (Unauthorized Error), 400 (Invalid API Usage Error)
Example Response:
Output
{
"license_id": "YGVkZ1BjZlFlX1khZ00lZk0gX1NcYWkDQDM5P0wENjNHNjNcQzAyOi75QF0ERyRcND0GREU1Q0z8\r\nQDQxPWjg.SFoI1kjXWNcYFwiZWk3"
}
3.2.2. View license information¶
To view additional details about the current applied license, run the following command:
# robin license info --usage
|
View cluster usage with regard to license restrictions |
Example:
# robin license info
ROBIN License Information:
ID ......................... YGVkYWBjYGFiZWkhZ00lZk0gX1NcYWkDQDM5P0wENjNHNjNcQzAyOi75QF0ERyRcND0GREU1Q0z8\r\nQDQxPWjg.S_
Type ....................... Evaluation
Created on ................. 2022-09-14T10:38:30 UTC
ROBIN Type ................. Platform
Host Type .................. PHYSICAL
Version .................... 5.4.1-4998
K8S Provider ............... ROBIN
Deployment Source .......... Installer
Expires on ................. 2022-10-14T10:38:30 UTC (18 day(s) remaining)
Status ..................... OK
Returns additional details about the license currently applied on a cluster.
End Point: /api/v3/robin_server/license
Method: GET
URL Parameters: None
Data Parameters: None
Port: RCM Port (default value is 29442)
Headers:
Authorization: <auth_token>: Authorization token to identify which user is sending the request. The token can be acquired from the login API.
Success Response Code: 200
Error Response Code: 500 (Internal Server Error), 401 (Unauthorized Error), 400 (Invalid API Usage Error)
Example Response:
Output
{
"license_info": {
"license": {
"sku": "platform",
"features": {
"snapshot": true,
"clone": true,
"backup": true
},
"expireson": "2022-12-11T09:06:15",
"creator": "Robin",
"issuedto": {
"name": "",
"email": "internal_robin@robin.io"
},
"pernode": {
"disks": 72,
"storage": 1266637395197952,
"cpu_cores": 96,
"containers": 128,
"memory": 2199023255552
},
"percluster": {
"gpu_cores": 120000,
"users": 1000,
"apps": 10000,
"disks": 360000,
"storage": 6333186975989760000,
"snapshots": 2560000,
"cpu_cores": 480000,
"memory": 10995116277760000,
"clones": 10000,
"nodes": 5000,
"tenants": 1000,
"containers": 50000
},
"activated": true,
"createtime": "2022-11-11T09:06:15",
"version": "5.4.1-46",
"installkey": "385e43c7-bf16-4768-9b77-0e09ddb7263c",
"host_type": "physical",
"k8s_provider": "robin",
"type": "Evaluation",
"id": "YGVkZ1BjZlFlX1khZ00lZk0gX1NcYWkDQDM5P0wENjNHNjNcQzAyOi75QF0ERyRcND0GREU1Q0z8\r\nQDQxPWjg.SFoI1kjXWNcYFwiZWk3",
"deployment_source": "installer",
"robin_type": "platform",
"num_days_expire": 12,
"num_days_lockdown": 27
},
"profile": {
"apps": 0,
"snapshots": 0,
"clones": 0,
"containers": 0,
"users": 3,
"tenants": 1,
"disks": 3,
"nodes": 3,
"node_def": {
"3": {
"primary_hostname": "vnode-109-179.robinsystems.com",
"disks": 1,
"storage": 85899345920,
"cpu_cores": 8,
"memory": 16655675392
},
"1": {
"primary_hostname": "vnode-109-177.robinsystems.com",
"disks": 1,
"storage": 85899345920,
"cpu_cores": 8,
"memory": 16655675392
},
"2": {
"primary_hostname": "vnode-109-178.robinsystems.com",
"disks": 1,
"storage": 85899345920,
"cpu_cores": 8,
"memory": 16655675392
}
},
"cpu_cores": 24,
"gpu_cores": 0,
"memory": 49967026176,
"storage": 257698037760
},
"status": "OK",
"status_message": "License is in a good state."
}
}
3.2.3. Activate a license¶
In order to activate a license and thus allow a cluster to be fully operational, run the following command:
# robin license activate --userid <userid>
|
Custom User ID generated after logging into the get.robin.io website. Note this parameter is mandatory whenn the |
Example:
# robin license activate --userid 3D87AHGE
License has been applied successfully
Returns a URL which can be used to activate a license in order to allow the cluster to become fully operational.
End Point: /api/v3/robin_server/license
Method: GET
URL Parameters:
url=activate: This mandatory parameter specifies that the URL for activation should be returned.userid=<userid>: Utilizing this parameter results in the URL returned pointing directly to get.robin.io. It is mandatory when thelicense_server_urlparameter is not set to the URL for a Robin License Proxy server.
Data Parameters: None
Port: RCM Port (default value is 29442)
Headers:
Authorization: <auth_token>: Authorization token to identify which user is sending the request. The token can be acquired from the login API.
Success Response Code: 200
Error Response Code: 500 (Internal Server Error), 401 (Unauthorized Error), 400 (Invalid API Usage Error)
Example Response:
3.2.4. Apply a license key¶
In order to apply a generated license key manually, run the following command:
# robin license apply <license_key> --desc <desc>
--force
|
Raw license key to be used or path to file containing license key |
|
Description to associate with license being applied |
|
Override inferior license check |
Example:
# robin license apply lg48ugNqMOnHTelEE7KrnaY1IlGFeFlowPkXt6MQkoK3zJ4ULZvNZEKsq34gS5x6B7CrCEPP8t78xkya2xfj6XlVWHGo5/zFt6gEKGeugHef0r3RbJC4zMuGdiucREHMCbAFyZ/dYHGPSnod2I8pG5/adxdOtEFUwReYBIPgNBJ3Sh7WCt8d3xW/jLPVUUPfyqToAv2m6C8SZwlI1QDhp0/NQD8cJ5CJoI4bCJpOM6pNgfbMhY9nTjORLNERujdrYhrWojs/kasySwH4uOnIFzF9GVLD/HgYR8viZOrUs1yq72V0MP7CLAsxtjnMiNeIKxgnXSrHzJIgLZvqEm80Q5ZImTREtJU53KHv5xJ7EvflFzWa8JBzXhrOXywuI/CJpbFCPA+s7BbWEneG/7aw0M6zxFR6s7ItYAmXryEW3cXnyM19o5oHswrHY9WX/67sMXBmdhzUa/zVugJmDUSvtuKV6X8aXpp/hPFzMOTzkOfk/4/ACMNZB2wWPl9Wg9hyFfx1+JitUeKhuUmqMt7grLrCY2SP9wHAUXOwPUnrpmk88sJHWDpBKeVrSApaTHuH5lJFIMw7iatxQT7nw8hsJ5sz4sRoq0iyV/EFA1k+UjVyvP+oLRgUJur6VqalpoEo0oTlJ4pZX1g7iM5dhLn5lLn7RMpzylqs2/Lo2j8dQexE7kbFV34vSlL0/oqg6YjsjpC1SVd/L3X72PXydBM9eCv6P82Zj8nARVEMq0yGGUTkjPervGcNg0VAFyeL1mA3K0KeduzRKuM5cyixrfB3M7BdxJyJ3Ob0YHoHwKzGRCY0w9yhtUhR/mGMklXomjgjFdc+L92qi2QA/fZz4qbhSFTUxY/fzOdPZKZvEXYXR9oSOAOqDjfuOk8GPDAE2H6RDN5AJk3nn3ViNI8JGW3Tf+dLiFWHxfieIVMlpT45Fy5/24K1QfoSv4HQ/p5qaOdq8Ygw5kXmoQw8fujXdqRA8dnIEcxk9fgXoAVOjSSGe9tllqy1mzMGEpQxMC4REH4qNwQMJDk6TOjEKijwpHHLDvsirJoyx6wm65geBTUxJVjmQrh568fou2BuCJhkKLfKiRd0Pj+yjkDqtCjw21v8yh1H+gkIJIshBrAz3jQJ2O6DV0g3ka/akaVvmBhk8vUxyDFC2d6ixurmdGz44Cz3cDA1JHG3D4iciiErA+K+/Iz1RavVmwp/FGrI2W2QMOSzMRqJKTsq2zoGOYWWEdIIvYRaISAEmsKa4eSHnl5vjlf/4lhTLP/Il12nV1cG3zUXUlMiadEww4//NWhet91LInGr+R1Pjb1yYFPmx2
License has been applied successfully
Applies a specified license key to the cluster.
End Point: /api/v3/robin_server/license
Method: PUT
URL Parameters: None
Data Parameters:
license: <license_key>- This mandatory field within the payload specifies the license key to apply.desc: <desc>- Utilizing this parameter within the payload, by specifying by specifying a string, results in the description for the applied license being set to the string given.force: [true|false]- Utilizing this parameter within the payload, by specifying a boolean value, determines whether or not the inferior license check is overriden.
Port: RCM Port (default value is 29442)
Headers:
Authorization: <auth_token>: Authorization token to identify which user is sending the request. The token can be acquired from the login API.
Success Response Code: 200
Error Response Code: 500 (Internal Server Error), 401 (Unauthorized Error), 400 (Invalid API Usage Error)
Example Response:
3.2.5. Renew a license¶
In order to renew a license such that its expiration date is extended, run the following command:
# robin license renew --token <token>
|
Authorization token generated on the get.robin.io website. Note this parameter is mandatory whenn the |
Example:
# robin license renew
License has been renewed successfully
Returns a URL which can be used to renew a license in order to extend the expiration date for a license.
End Point: /api/v3/robin_server/license
Method: GET
URL Parameters:
url=renew: This mandatory parameter specifies that the URL for renewal should be returned.token=<token>: Utilizing this parameter results in the URL returned utilizing the specified authorization token and its associated contract. It is mandatory when thelicense_server_urlparameter is not set to the URL for a Robin License Proxy server.
Data Parameters: None
Port: RCM Port (default value is 29442)
Headers:
Authorization: <auth_token>: Authorization token to identify which user is sending the request. The token can be acquired from the login API.
Success Response Code: 200
Error Response Code: 500 (Internal Server Error), 401 (Unauthorized Error), 400 (Invalid API Usage Error)
Example Response:
3.3. Resource pool assignment¶
Resource pools are a construct in Robin which allow you to group nodes in the cluster together for allocation purposes. Pools provide resource isolation. More details on resource pools and commands to manage them can be found here.
Upon installation, a resource pool named default is created. If you wish to create a custom resource pool run this command:
# robin rpool add <name>
Example:
# robin rpool add demo
Added resource pool demo.
In order to add nodes to a resource pool run this command:
# robin host assign-rpool <hostnames> <rpool_name> --wait
Example:
# robin host assign-rpool centos-60-212,centos-60-214 demo --wait
Job: 11 Name: HostAddResourcePoolMulti State: VALIDATED Error: 0
Job: 11 Name: HostAddResourcePoolMulti State: WAITING Error: 0
Job: 11 Name: HostAddResourcePoolMulti State: COMPLETED Error: 0
Note
The default resource pool is not assigned to any nodes upon installation. Thus, if you want to utilize the precreated resource pool you still have to explicitly assign it.
3.4. Role assignment¶
There are three Robin roles that can be assigned to hosts: Manager, Compute and Storage. Depending on the resources present on a node you can initialize it to have one or more of these roles. This allows for a more granular control over what resources a node can provide for allocation. More details on the specific roles, what they entail, and how to manage them can be found here.
Note
Upon installation, nodes which that installed as masters will have the Manager role already assigned to them.
To add a role, run this command:
# robin host add-role <hostnames> <roles> --wait
Example:
# robin host add-role centos-60-212,centos-60-214 Compute,Storage --wait
Job: 18 Name: HostAddRoles State: VALIDATED Error: 0
Job: 18 Name: HostAddRoles State: WAITING Error: 0
Job: 18 Name: HostAddRoles State: COMPLETED Error: 0
To verify the roles were appropriately added, run this command:
# robin host list
Id | Hostname | Version | Status | State | Resource Pool | Roles | Cores | GPUs | Memory | HDD(Alloc/Total) | SSD(Alloc/Total) | Instances | Joined Time
-------------+----------------------------------+-----------+--------+--------+---------------+--------+-------+------+-------------------+------------------+------------------+-----------+----------------------
1539379146:1 | centos-60-212.robinsystems.com | 5.0.5-548 | Ready | ONLINE | demo | M*,S,C | 1/6 | 0/0 | 7.76 GB/15.5 GB | 396 GB/43.66 TB | 0 GB/111.79 GB | 0 | 12 Oct 2019 14:19:32
1539379146:2 | centos-60-214.robinsystems.com | 5.0.5-548 | Ready | ONLINE | demo | C,S | 1/24 | 0/0 | 22.81 GB/94.25 GB | 0 GB/0 B | 50 GB/894.25 GB | 0 | 12 Oct 2019 14:27:07
3.5. IP-Pool addition¶
An IP-Pool is a construct in Robin that groups together a set of IP addresses. Given that these IP addresses are then allocated by Robin during the application creation process, an IP-Pool allows you to have some control over what IP addresses are assigned to application containers. More details on IP-pools and commands to manage them can be found here.
Upon installation, an IP-Pool named default is created. If you wish to create a custom IP-Pool run this command:
# robin ip-pool add <name> --driver=<driver> --ranges <ip_range> --netmask <netmask>
Example:
# robin ip-pool add ovs-p1 --driver=ovs 10.10.1.82-128 255.255.255.0
Registered IP pool ovs-p1.
Note
The subnet for the IP-Pool is required to be present on at least one of the nodes with the compute role.
3.6. File collection addition¶
A file collection is an NFS like server that is backed by a volume allocated from Robin managed storage. Bundles and images used to create applications will be stored on File Collections, in addition to any logs that are collected. More details on File Collections and commands to manage them can be found here.
3.6.1. Considerations for file collection creation¶
Here are some factors that should be taken into account when creating a new File Collection:
The size of the collection. This will vary per use case and will be dependent on the number and size of the images that will be used for application creation.
The replication factor of the volumes backing the collection. This is vital as it helps ensure the collection is recoverable after failovers.
The collection type, which can be either log or file. This depends on the use case.
The resource pool from which to draw storage for the allocated volume.
The media type for the volume which will provide storage for the collection (either HDD or SSD). This will depend on the cluster.
File Collections will only be created on Robin nodes with the Manager role.
Moreover additional planning is required to ensure that there is sufficient capacity should a Master node fail. Operators should either:
Ensure that the replication factor on the File Collection is N-1 where N is the number of Master nodes
Ensure that there is sufficient space available on the Master nodes to allow the creation of a temporary File Collection should one of the master nodes fail in such a way that
The entire server has to be replaced
The operating system has to be reinstalled.
Note
The considerations regarding node failures also apply to Robin volumes in a situation where the number of replicas is equal to the number of hard drives available for Storage. For example a situation where they apply is a three node cluster, wherein which there is one hard drive available for storage on each node.
3.6.2. File collection creation¶
To create a file collection run this command:
# robin collection create <media_type> <rpool_name> --collection_type <type> --replicas <count> --size <size>
Example:
# robin collection create HDD demo --collection_type FILE_COLLECTION --replicas 3 --size 50G
Job: 189 Name: CollectionAdd State: VALIDATED Error: 0
Job: 189 Name: CollectionAdd State: PROCESSED Error: 0
Job: 189 Name: CollectionAdd State: WAITING Error: 0
Job: 189 Name: CollectionAdd State: FINALIZED Error: 0
Job: 189 Name: CollectionAdd State: COMPLETED Error: 0
3.7. Enabling Metrics¶
Robin provides metrics collection for both nodes and applications via Prometheus. Statistics such as CPU usage, disk utilization, memory etc. are collected for these entities. Robin uses Prometheus for three main reasons:
Given its affinity for recording numeric time series, it is an excellent tool for machine-centric monitoring as all the data that is exposed is numeric.
Prometheus supports multidimensional data collection for microservices making it very easy to leverage for Robin application monitoring.
Resilience: each Prometheus server can be run completely independent of network storage or remote services.
The data collected is stored on disk in Prometheus’s local time series database. The time series data is stored in a custom format and is retained for a variable number of days. The amount of storage needed for the created PVC is dependent on the retention policy. More details on how the scraped data is stored are available here.
3.7.1. Calico Felix Prometheus metrics¶
Starting with Robin CNP v5.9.0, Prometheus metrics from Calico Felix are enabled by default. The built-in Prometheus service automatically discovers and scrapes these metrics when Robin metrics collection is enabled.
Felix metrics are exposed by the calico-node DaemonSet on each node through the /metrics endpoint on port 9091.
3.7.1.1. Verify metrics collection¶
To verify that Felix metrics collection is enabled, follow these steps:
Check that the environment variable is enabled:
kubectl get daemonset calico-node -n kube-system -o yaml | grep FELIX_PROMETHEUSMETRICSENABLED
Verify that the output shows the following setting:
FELIX_PROMETHEUSMETRICSENABLED=true
Query the Felix metrics endpoint on each node:
curl -s http://<node_ip>:9091/metrics | head -5
The response should contain
felix_metrics, such asfelix_active_local_endpoints.
Robin’s built-in Prometheus service scrapes Felix metrics through the Felix metrics service. Verify that the metric is available in Prometheus using the Prometheus query interface or API, for example:
felix_active_local_endpoints
3.7.1.2. Example metrics¶
The following are examples of available Felix metrics:
felix_active_local_endpointsfelix_active_local_policiesfelix_ipsets_totalfelix_iptables_rules_totalfelix_resync_state
3.7.1.3. Disable metrics collection¶
Warning
Running kubectl set env updates the Calico DaemonSet and triggers a pod rollout restart. Perform this operation during a scheduled maintenance window or a low-traffic period.
To disable Felix metrics collection, run the following command:
kubectl set env daemonset/calico-node -n kube-system FELIX_PROMETHEUSMETRICSENABLED=false
Metrics collection stops after the DaemonSet rollout completes. To re-enable metrics, set the value to true.
3.7.2. Considerations for setting up metrics¶
Here are some factors that should be taken into account when enabling metrics collection:
The retention time period. This will depend on the amount of historical data you wish to keep. The default is 7 days.
The size of the storage volume. This will be automatically calculated by multiplying the retention period by 1.5Gi but can be overridden. It is important to create a volume large enough to cover the whole retention period.
The replication factor of the volumes backing the collection. We recommend 3 to ensure the data previously collected is always available even after failure.
The fault domain for replicas. This will be cluster dependent and can be disk, host, or rack.
The resource pool from which to draw storage for the allocated volume.
The media type for the volume that will provide storage for the collection (either HDD or SSD). This will be cluster dependent.
3.7.3. Starting Metrics¶
Robin brings up the Prometheus server and node exporter alongisde Grafana via Helm. In order to deploy the Prometheus Helm chart, run the following command:
# robin metrics start --media <media_type> --resource-pool <rpool_name> --faultdomain <fault_domain> --replicas <count> --retention-period <period>
Example:
# robin metrics start --media HDD --resource-pool demo --faultdomain disk --replicas 3 --retention-period 2
Job: 192 Name: MetricsStartCollect State: PREPARED Error: 0
Job: 192 Name: MetricsStartCollect State: DONE Error: 0
Job: 192 Name: MetricsStartCollect State: AGENT_WAIT Error: 0
Job: 192 Name: MetricsStartCollect State: NOTIFIED Error: 0
Job: 192 Name: MetricsStartCollect State: COMPLETED Error: 0
Note
The default username and password needed to access the Grafana dashboard is admin/admin.
3.7.4. Viewing Metrics Status¶
You can view the status of metrics collection by Robin CNP for Prometheus and Grafana. If you want to view cAdvisor status, you can run the robin metrics cadvisor-status command.
To view the status of metrics collection, run the following command:
# robin metrics status
Example
[root@hypervvm-63-114 ~]# robin metrics status
Prometheus:
-- Status........ Running
-- URL........... http://[fd74:ca9b:3a09:868c:0010:0009:0063:0114]:31266
Grafana:
-- Status........ Running
-- URL........... https://[fd74:ca9b:3a09:868c:0010:0009:0063:0114]:31412
3.7.5. Stopping Metrics¶
You can stop metrics collection if you do need metrics from Robin CNP.
When you stop metric collection, Robin CNP stops metric collection for all parameters.
It stops the Prometheus, Grafana, and cAdvisor monitoring systems.
If you do not want to disable cAdvisor, you must use the --skip-cadvisor option. And, if you need to delete previously collected data, you can use the --delete option while stopping the metrics collection.
To stop metrics, run the following command:
# robin metrics stop --wait
Example
[root@hypervvm-63-114 ~]# robin metrics stop --wait
Job: 202 Name: MetricsStopCollect State: PREPARED Error: 0
Job: 202 Name: MetricsStopCollect State: COMPLETED Error: 0
3.7.6. cAdvisor as a DaemonSet¶
Robin CNP v5.4.3 HF5 contains the cAdvisor as a standalone DaemonSet. By default, cAdvisor (Container Advisor) is enabled when you enable Robin Metrics on your Robin Cluster. However, if you do not want to use Robin Metrics but still need cAdvisor for gathering node-level metrics, you can enable it and manage cAdvisor independently. You can start, stop, and view the status of the cAdvisor services without enabling Robin Metrics. This allows you to view metrics on any observability framework (OBF) of your choice.
Note
When disabling metrics in Robin CNP using the robin metrics stop command, CNP also disables the CAdvisor. If you want to continue to use CAdvisor without using the complete metrics feature, you can use the --skip -cadvisor option with the robin metrics stop command
3.7.6.1. Starting cAdvisor Metrics¶
You can enable cAdvisor for viewing node-level metrics. You can enable this without enabling Robin Metrics.
Note
You do not need to enable cAdvisor separately if Robin Metrics is already enabled.
To start cAdvisor, run the following command:
# robin metrics start-cadvisor
Example
[root@hypervvm-63-114 ~]# robin metrics start-cadvisor --wait
Job: 179 Name: MetricsStartCadvisor State: PREPARED Error: 0
Job: 179 Name: MetricsStartCadvisor State: AGENT_WAIT Error: 0
Job: 179 Name: MetricsStartCadvisor State: FINALIZED Error: 0
Job: 179 Name: MetricsStartCadvisor State: COMPLETED Error: 0
3.7.6.2. Viewing cAdvisor Metrics Status¶
You can check the status of the cAdvisor after enabling it. The status displays as Running or Stopped.
To view the status of the cAdvisor, run the following command:
# robin metrics status --cadvisor
Example
[root@hypervvm-63-114 ~]# robin metrics status --cadvisor
Cadvisor Status........ Running/Total Pods - 3/3
3.7.6.3. Stopping cAdvisor¶
You can stop the cAdvisor when you do not need node level metrics.
To stop cAdvisor, run the following command:
# robin metrics stop-cadvisor
Example
[root@hypervvm-63-114 ~]# robin metrics stop-cadvisor --wait
Job: 198 Name: MetricsStopCadvisor State: PREPARED Error: 0
Job: 198 Name: MetricsStopCadvisor State: COMPLETED Error: 0
3.8. HAProxy Prometheus exporter¶
3.8.1. Overview¶
Robin CNP includes a built-in Prometheus exporter module in the HAProxy binary on all master nodes. The exporter provides HAProxy performance and health metrics in Prometheus exposition format through an HTTP /metrics endpoint. This eliminates the need for an external Prometheus exporter.
The HAProxy binary is compiled with the USE_PROMEX flag, which enables the native exporter module. The exporter runs as a frontend block in haproxy.cfg.
The exporter has the following characteristics:
Metrics are available on all master nodes by default.
The default port is
29470.The port is configurable using a port file or an environment variable.
Port settings persist across container restarts.
Upgrades from versions earlier than 5.9.0 automatically configure the exporter.
The exporter supports both IPv4 and IPv6.
3.8.2. Prerequisites¶
To use and verify the HAProxy Prometheus exporter, you need the following:
Robin CNP version 5.9.0 or later.
SSH access to the master nodes.
3.8.3. Verify the exporter¶
To verify that the exporter is running and accessible, follow these steps:
On any master node, run the following command to check the
/metricsendpoint:curl -Ss localhost:29470/metrics | head -5
Expected output:
# HELP haproxy_process_nbthread Number of started threads (global.nbthread) # TYPE haproxy_process_nbthread gauge haproxy_process_nbthread 16 # HELP haproxy_process_nbproc Number of started worker processes (historical, always 1) # TYPE haproxy_process_nbproc gauge
Verify the exporter port in the
robin-bootstrap-configConfigMap:kubectl get cm -n robinio robin-bootstrap-config -oyaml | grep HAPROXY_PROMETHEUS_PORT
Expected output:
HAPROXY_PROMETHEUS_PORT: "29470"
Confirm the port is listening by running the following command:
ss -nltp | grep 29470
3.8.4. Configure the exporter port¶
By default, the exporter listens on port 29470 on each master node. You can change this port using a port file or an environment variable. The system determines the port using the following priority order:
Environment variable
Port file
Default port (
29470)
3.8.4.1. Method 1: Use a port file¶
To change the default Prometheus port, follow these steps on all master nodes. On startup, the exporter uses the port number defined in the /root/robin/robink8s-monitor/HAPROXY_PROMETHEUS_PORT file.
Set the desired port in the port file:
echo 8404 > /root/robin/robink8s-monitor/HAPROXY_PROMETHEUS_PORT
Confirm the value in the port file:
cat /root/robin/robink8s-monitor/HAPROXY_PROMETHEUS_PORT 8404
Restart the
haproxy-keepalived-robink8s_monitorcontainer using either Docker or Podman:docker restart haproxy-keepalived-robink8s_monitor podman restart haproxy-keepalived-robink8s_monitor
Confirm that HAProxy is listening on the new port:
ss -nultp | grep 8404 tcp LISTEN 0 2048 0.0.0.0:8404 0.0.0.0:* users:(('haproxy',pid=4
Warning
Enter only a valid port number in the HAPROXY_PROMETHEUS_PORT file. Entering garbage or invalid values can cause the service to malfunction.
3.8.4.2. Method 2: Use an environment variable¶
Set the HAPROXY_PROMETHEUS_PORT environment variable on the container. This overrides both the port file and the default value.
--env HAPROXY_PROMETHEUS_PORT=9090
When you set this environment variable, the exporter listens only on the specified port. The default port (29470) becomes inactive.
3.8.4.3. Revert to the default port¶
To revert to the default port, follow these steps:
Delete the port file:
rm /root/robin/robink8s-monitor/HAPROXY_PROMETHEUS_PORT rm: remove regular file '/root/robin/robink8s-monitor/HAPROXY_PROMETHEUS_PORT'? y
Restart the container:
podman restart haproxy-keepalived-robink8s_monitor haproxy-keepalived-robink8s_monitor
3.8.5. Metrics reference¶
The exporter provides standard HAProxy metrics. The following table describes key metric families:
Category |
Metric |
Description |
|---|---|---|
Process |
|
The number of started threads. |
Process |
|
The number of seconds since the worker process started. |
Process |
|
The current number of connections on the worker process. |
Process |
|
The maximum number of connections allowed per process, as configured or restricted by |
Frontend |
|
The current status of the service based on the state label. |
Frontend |
|
The number of active sessions on the frontend, backend, or server. |
Frontend |
|
The total number of HTTP requests processed by the object since the worker process started. |
Frontend |
|
The total number of invalid requests since the process started. |
Backend |
|
The current status of the service based on the state label. |
Backend |
|
The number of connections currently in the queue. |
Backend |
|
The total number of active servers with a status of UP and a weight greater than zero. |
Backend |
|
The average response time, in seconds, for the last 1024 successful connections. |
Backend |
|
The total number of failed connections to the server since the worker process started. |
Server |
|
The current status of the service based on the state label. |
Server |
|
The total number of failed health checks for the server or backend since the worker process started. |
The exporter provides 168 metrics. To view the complete list, run the following command:
curl -Ss localhost:29470/metrics | grep "^# HELP"
3.8.6. IPv6 support¶
The /metrics endpoint is accessible over both IPv4 and IPv6. To access the endpoint using the IPv6 loopback address, run the following command:
curl -Ss "http://[::1]:29470/metrics" | head -5
3.9. Robin client¶
You can download the Robin client on your Linux or macOS machine to access your Robin cluster remotely. To retrieve the client binary, issue the following command:
# curl -k 'https://<master_ip>:<port>/api/v3/robin_server/download?file=robincli&os=<os>' -o robin
Note
The possiible values for the os URL parameter include: linux and mac. Please select the appropriate value for your desired operating system when retrieving the client.
Example:
# curl -k 'https://vnode42:29442/api/v3/robin_server/download?file=robincli&os=linux' -o robin
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 10.1M 100 10.1M 0 0 1421k 0 0:00:07 0:00:07 --:--:-- 1483k
# ls -lart
-rw-r--r-- 1 demo staff 10655536 Mar 26 14:12 robin
After running the curl command a file should have been downloaded to your current working directory. Make it an executable and setup the appropriate context based on the type of installation:
3.9.1. Non-HA Deployments¶
# chmod +x robin
# robin client add-context <master_ip> --set-current
3.9.2. HA deployments¶
Before setting up the context for highly available clusters make sure the robincp_mode config attribute is enabled by logging into the appropriate node and issuing the following command:
# robin config list | grep "robincp_mode"
manager | robincp_mode | False
If the aformentioned attribute is not enabled, issue the following command:
# robin config update manager robincp_mode True --wait
Job: 163 Name: HostConfigUpdateMulti State: VALIDATED Error: 0
Job: 163 Name: HostConfigUpdateMulti State: COMPLETED Error: 0
# robin config list | grep "robincp_mode"
manager | robincp_mode | True
Once this step is complete, finish setting up the client by issuing the following commands:
# chmod +x robin
# ./robin client add-context <cluster_vip> --port 29465 --file-port 29465 --event-port 29465 --set-current
Note
The examples above for client context configurations assume default port usage. For custom port deployments please set the custom port for each respective service explicitly. More information about client contexts please refer to the documentation here.
3.9.3. Access using Whitelisted IP address¶
You can access your Robin CNP cluster remotely by adding the IP address of the machine from which you want to access your cluster as whitelisted IP address.
3.9.3.1. Prerequisites¶
You must install your Robin CNP cluster with the
zero-trust=trueoption.
Perform the following steps to access your Robin CNP cluster using the whitelisted IP address:
Login to your Robin CNP cluster using your username and password.
Add the whitelisted IP address to the
robin-iptables-cfgin the/bindirectory.# cd bin/ # vi robin-iptables-cfg ZEROTRUST=1 #Robin Control Ports CONTROLPORTS="6443,29442,29443,29445,29449" #Application Ports APPPORTS="" IP_VERSION=4 HAINSTALL=0 #Comma separated list of IPs or subnets that are whitelisted WHITELIST_IPS="<IP addresses of machines>"
Run the
robin-k8s-iptablesscript:# ./robin-k8s-iptables
Create a network policy yaml with the whitelisted IP addresses.
apiVersion: crd.projectcalico.org/v1 kind: NetworkPolicy metadata: name: allow-ingress-external namespace: robinio spec: selector: app == 'robin-master' types: - Ingress - Egress ingress: action: Allow protocol: TCP source: nets: - <K8S cluster CIDR> - <Whitelisted IP address> egress: action: Allow
Run the following command to apply the network policy yaml to your Robin CNP cluster:
# kubectl apply -f <network policy>.yaml
Run the robin-k8s-iptables script again:
# ./robin-k8s-iptables
Access the machine where you have already downloaded the Robin client’s binary.
Run the following command to make the Robin client executable:
# chmod +x robin
Run the following command to add context to the Robin client:
# robin client add-context <IP address of node where Robin master Pod is running> --set-current
Example
# ./robincli-5.4.3-83 client add-context 192:0:2:24 --set-current Context robin-cluster-192:0:2:24 updated successfully and set as the current context
Note
To know the IP address of node where Robin master Pod is running, run the
kubectl get pod -n robinio -o wide | grep robin-mastercommand.When the Robin master Pod moves from one node to another node,
robin client add-contextmust be added with the new node again.
Before you can use the CLI you will have to login with the username and password (by default it is admin/Robin123):
# ./robin login <username>
Enter the password when prompted.
3.10. Robin Auth Utility¶
Before downloading and configuring the Kubernetes client (see below),
you first must download the robin-auth utility. The
robin-auth utility returns the credentials of the currently
logged in Robin user. The credentials are passed to the Kubernetes
API server when sending requests using the Kubernetes client. You can download the robin-auth
utility on your Linux or macOS machine by issuing the following command:
# curl -k 'https://<master_ip>:<port>/api/v3/robin_server/download?file=robinauth&os=<os>' -o robin-auth
Note
The possiible values for the os URL parameter include: linux and mac. Please select the appropriate value for your desired operating system when retrieving the client.
Example:
# curl -k 'https://vnode42:29442/api/v3/robin_server/download?file=robinauth&os=linux' -o robin-auth
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 46.8M 100 46.8M 0 0 36.1M 0 0:00:01 0:00:01 --:--:-- 36.1M
# ls -lart robin-auth
-rw-r--r-- 1 root root 6145568 Sep 1 14:58 robin-auth
After running the curl command a file should have been downloaded to your current working directory. Make it an executable and move it to a directory that is in your current PATH.
# chmod +x robin-auth
Check if a bin directory is present by running the ls command. If the directory is not present, create the bin directory.
# mkdir ~/bin
# mv robin-auth ~/bin
# robin login user1
Password:
User user1 is logged into t1 tenant
# robin-auth
{"kind": "ExecCredential", "status": {"token": "eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJ0ZW5hbnRfaWQiOjIsImV4cCI6MTU5OTA4NjI1MiwidXNlcl9pZCI6NX0._fN7JSEdRvNqHtdHJiMog5USAiTEnD4xiAMRdOy_NLw"}, "apiVersion": "client.authentication.k8s.io/v1beta1"}
Note
The robin-auth utility works in conjunction with the Robin client (see above).
The robin-auth utility must be located in your current PATH.
3.11. Kubernetes client¶
You can download the Kubernetes client on your Linux or macOS machine and setup the configuration via Robin in order to access the Kubernetes cluster externally. To retrieve the Kubernetes client, issue the following command:
# curl -k 'https://<master_ip>:<port>/api/v3/robin_server/download?file=kubectl&os=<os>' -o kubectl
Example:
# curl -k 'https://vnode31:29442/api/v3/robin_server/download?file=kubectl&os=linux' -o kubectl
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 44.5M 100 44.5M 0 0 28.9M 0 0:00:01 0:00:01 --:--:-- 28.9M
# ls -lart kubectl
-rw-r--r-- 1 demo staff 46682408 Apr 18 23:43 kubectl
After running the curl command a file should have been downloaded to your current working directory. Make it an executable and configure the client by issuing the following commands:
# chmod +x kubectl
# mkdir ~/.kube
# ./robin k8s get-kube-config --save-as-file --dest-dir ~/.kube
# kubectl get nodes
NAME STATUS ROLES AGE VERSION
vnode31.robinsystems.com Ready master 9d v1.16.3
vnode35.robinsystems.com Ready <none> 9d v1.16.3
vnode52.robinsystems.com Ready <none> 9d v1.16.3
Note
The above steps assume that the Robin client and robin-auth utility have already been downloaded and setup correctly on the remote machine. More details on how to setup your Robin client can be found in the above section.
The downloaded kubeconfig file contains an entry for the currently logged in Robin user, which is set as the default. The credentials for the currently logged in Robin user will be used when accessing the Kubernetes API server.
3.12. Helm client¶
You can download the Helm client on your Linux or macOS machine by issuing the following command:
# curl -k 'https://<master_ip>:<port>/api/v3/robin_server/download?file=helm&os=<os>' -o helm
Example:
# curl -k 'https://vnode31:29442/api/v3/robin_server/download?file=helm&os=linux' -o helm
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 38.5M 100 38.5M 0 0 29.3M 0 0:00:01 0:00:01 --:--:-- 29.4M
# ls -lart
-rw-r--r-- 1 demo staff 40460288 Apr 18 23:48 helm
After running the curl command, a file should have been downloaded to your current working directory. Make the file an executable by issuing the following commands:
# chmod +x helm
3.13. Helm 2 client¶
Note
Support for Helm 2 and Tiller is deprecated and will be removed in the next release.
You can download the Helm 2 client on your Linux or macOS machine and configure the necessary Helm/Tiller objects via Robin in order to utilize Helm to deploy charts to be managed by the platform. To retrieve the Helm 2 client, issue the following comman:
# curl -k 'https://<master_ip>:<port>/api/v3/robin_server/download?file=helm2&os=<os>' -o helm2
Example:
# curl -k 'https://vnode31:29442/api/v3/robin_server/download?file=helm2&os=linux' -o helm2
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 38.5M 100 38.5M 0 0 29.3M 0 0:00:01 0:00:01 --:--:-- 29.4M
# ls -lart
-rw-r--r-- 1 demo staff 40460288 Apr 18 23:48 helm2
After running the curl command a file should have been downloaded to your current working directory. Make it an executable, configure the necessary files and deploy the appropriate Tiller objects by issuing the following commands:
# chmod +x helm2
# ./robin whoami
username: robin
tenant: Administrators
tenants: ['Administrators']
namespace: t001-u000003
session_expires: 2020-04-19T23:13:10
# ./robin k8s deploy-tiller t001-u000003
Tiller objects were successfully deployed for namespace 't001-u000003'
# robin k8s helm-setup t001-u000003
Helm initialized successfully
Note
The above steps assume that the Robin client has already been
setup correctly on the remote machine. More details on how to
setup your Robin client can be found in the above section. In
addition when running the final two commands ensure the Robin
namespace displayed in the output of the whoami command is
passed as parameter.
3.14. Robin Certificate Management¶
Robin certificate management feature allows you to manage all certificates in your cluster without manual intervention. Robin CNP leverages cert-manager to achieve this.
cert-manager is a native Kubernetes certificate management controller. It automates the issuance, renewal, and management of certificates from both external Certificate Authorities (CA) such as Let’s Encrypt, DigiCert, HashiCorp Vault and local Certificate Authorities (self-signed certificates).
cert-manager adds Certificate and Issuer resources to Kubernetes clusters, simplifies the process of obtaining, generating, and renewing certificates for the cluster. For more information, see, cert-manager.
Robin certificate management feature manages certificates only for Robin internal services deployed in the robinio namespace. It ensures that all certificates are valid and up-to-date by automatically renewing them before they expire.
Certificate Authorities in Robin CNP Cluster
A Robin CNP cluster has the following certificate authorities (CAs):
Cluster CA – It signs all certificates for internal communication within the cluster.
Identity CA – It signs the Cluster Identity certificate for all external-facing services such as Kubernetes API Server, Robin client, GUI, etc.
Certificate Issuers in Robin Certificate Management
Robin certificate management feature contains the following certificate issuers:
cluster-issuer - It issues and renews all certificates used internally by the various control plane services.
ident-issuer - It issues and renews the Cluster Identity certificate used for all external-facing services such as Kubernetes API Server, Robin client, GUI, etc.
These two certificate issuers are configured and managed independently. Therefore, you can reset the Identity CA certificate without affecting certificates issued by the cluster-issuer, which are used by the control plane services.
Certificate Issuer Operation and Control Modes
A certificate issuer has the following operation and control modes for managing certificates:
Local control - Certificates and keys are generated by internal functions of the control planes.
External control - Certificates and keys are generated by an external CA and manually deployed in the cluster.
Note
Robin CNP supports external control only for the ident-issuer certificate issuer type. If you use external control, you are responsible for renewing the certificate.
cert-manager control - Certificates and keys are generated and automatically renewed by cert-manager.
Note
Currently, the cert-manager supports only self-signed cluster CA.
external-csr control - Certificates and keys are generated and automatically renewed by an external CA using the external PKI server, such as EJBCA or Keyfactor. Robin CNP sends Certificate Signing Requests (CSRs) to the registered external PKI server over the network and receives signed certificates. The External PKI feature can be configured for external (outward facing) server certificates and/or certificates used internally by the CNP control plane services.
Points to consider for Robin certificate management
When you install or upgrade to the latest Robin CNP version, cert-manager is deployed by default. A new service called
robin-cert-monitoris also deployed to monitor the state of all certificates required by various Pods and containers in the Robin CNP cluster, ensuring that all required certificates exist and are valid.During install or upgrade, only the cert-manager option is supported. If you want to manage certificates using the local control mode, use the
robin cert reset-cluster-certsto enable it.After upgrading to the latest Robin CNP version, if your cluster is already installed with a Cluster Identity certificate signed by an external CA, you must reconfigure it using the
robin cert reset-cluster-identitycommand.If you want to use a Cluster Identity certificate signed by an external CA after installing Robin CNP, use the
robin cert reset-cluster-identitycommand to configure it.To install Robin CNP with both (Cluster Identity certificate signed by an external CA and cert-manager), you need to pass the following options in the
config.jsonfile for one of the master nodes.ident-ca-path
ident-cert-path
ident-key-path
For more information, see Installation with custom Cluster Identity certificate.
Only one cert-manager instance can be deployed in a cluster.
You cannot install your own cert-manager on a Robin CNP cluster. If you want to use cert-manager functionality, use the cert-manager instance deployed as part of the Robin certificate management feature to create Issuers and Certificates in other namespaces.
The following commands are described in this section:
|
List all certificates for a cluster |
|
Check the status of all certificates for a cluster |
|
Reset Cluster Identity CA and its key |
|
Reset Cluster CA and its key |
3.14.1. Manage external PKI server¶
To manage Robin certificates using the external-csr, you must integrate an external Public Key Infrastructure (PKI). The external PKI Integration feature allows Robin CNP to communicate with an external Public Key Infrastructure (PKI) server such as EJBCA or Keyfactor to issue or renew all Robin certificates. The external PKI feature can be configured for external (outward facing) server certificates and/or certificates used internally by the CNP control plane services.
By default, Robin CNP uses self-signed certificates. Instead of using self-signed certificates, Robin CNP can send Certificate Signing Requests (CSRs) to the external PKI server over the network and use the signed certificates it receives back.
The following commands are described in this section:
|
Register a PKI server |
|
List registered PKI servers |
|
Update a registered PKI server |
|
Remove a registered PKI server |
3.14.1.1. Register PKI server¶
You can register an external Public Key Infrastructure (PKI) server with the Robin CNP cluster. Once you register a PKI server, Robin CNP sends Certificate Signing Requests (CSRs) to this server to issue and renew certificates.
To register a PKI server, run the following command:
# robin pki add <name> <backend> <url>
--credentials <credentials>
--ca-cert <ca_cert>
--client-cert <client_cert>
--client-key <client_key>
--server-cert-profile <server_cert_profile>
--client-cert-profile <client_cert_profile>
--client-server-cert-profile <client_server_cert_profile>
|
Unique name for the Public Key Infrastructure (PKI) server. |
|
Type of PKI backend. Valid values are |
|
Endpoint URL for the PKI server. |
|
Credentials as a JSON string, such as {“auth_type”:”bearer”,”token”:”…”}. These credentials will be stored in a K8s secret. |
|
Path to the PEM file containing the PKI server’s TLS Certificate Authority (CA) certificate. It will be stored in a K8s secret. |
|
Path to the PEM file containing the client certificate for mutual TLS (mTLS). You must use this option with the |
|
Path to the PEM file containing the client key for mTLS. You must use this option with the |
|
Name of PKI profile or template for server certificates only, such as |
|
Name of PKI profile or template for client certificates only, such as |
|
Name of PKI profile or template for combined client and server certificates, such as |
Example
# robin pki add demo-pki ejbca "https://[fd74:ca9b:3a09:868c:172:18:0:cfb3]:18443" --ca-cert /tmp/demo-ca-chain.pem --client-cert /tmp/demo-client.crt --client-key /tmp/demo-client.key
Successfully added PKI server 'demo-pki'
3.14.1.2. List PKI servers¶
You can view the list of all PKI servers registered with your Robin CNP cluster.
To view the list of PKI servers, run the following command:
# robin pki list <name>
--json
|
Name of the PKI server. Note: Do not specify any PKI server name to list all PKI servers. |
|
Display output in JSON format |
Example
# robin pki list
NAME | BACKEND | URL | SECRET | CERT PROFILES
----------------+---------+---------------------------------------------------+---------------------------+---------------
test-broken-pki | ejbca | https://[fd74:ca9b:3a09:868c:172:18:0:cfb3]:18443 | robin-pki-test-broken-pki | -
3.14.1.3. Update PKI server¶
You can update the following parameters for an already registered PKI server:
Endpoint URL for PKI server
PKI backend type
Credentials
CA certificate
Client certificate
Client key
Server certificate profile
Client certificate profile
Client and server certificate profile
To update any parameter for a PKI server, run the following command:
# robin pki update <name>
--url <url>
--backend <backend>
--credentials <credentials>
--ca-cert <ca_cert>
--client-cert <client_cert>
--client-key <client_key>
--server-cert-profile <server_cert_profile>
--client-cert-profile <client_cert_profile>
--client-server-cert-profile <client_server_cert_profile>
|
Name of the PKI server to be updated. |
|
New endpoint URL for the PKI server. |
|
New type of PKI backend. Valid values are |
|
New credentials as a JSON string, such as {“auth_type”:”bearer”,”token”:”…”}. To clear existing credentials, use |
|
Path to the new PEM file for the PKI server’s TLS Certificate Authority (CA) certificate. To remove the existing certificate, use |
|
Path to the new PEM file for the client certificate for mutual TLS (mTLS). You must use this option with the |
|
Path to the new PEM file for the client key for mTLS. You must use this option with the |
|
New name of PKI profile or template for server certificates only, such as |
|
New name of PKI profile or template for client certificates only, such as |
|
New name of PKI profile or template for combined client and server certificates, such as |
Example
# robin pki update demo-pki "https://[fd74:ca9b:3a09:868c:172:18:0:cfb3]:18443"
Successfully updated PKI server 'demo-pki'
3.14.1.4. Remove PKI server¶
You can remove an already registered PKI server from your Robin CNP cluster if the PKI server is no longer available.
To remove a PKI server, run the following command:
# robin pki remove <name>
--yes
|
Name of the PKI server to be removed. |
|
Do not prompt for confirmation. |
Example
# robin pki remove demo-pki -y
Successfully removed PKI server 'demo-pki'
3.14.2. List all cluster certificates¶
You can view all certificates in a cluster issued by both the cluster-issuer and ident-issuer certificate issuers.
It provides the following additional information about certificates:
Certificate authority (CA)
Issuer type
Certificate’s name
Certificate’s expiry
Run the following command to view all certifictes for a cluster:
# robin cert list
--json
|
Output in JSON format. |
Example
# robin cert list
CERTIFICATE AUTHORITY | EXPIRES | ISSUER_TYPE
----------------------+----------------------+--------------
cluster-ca | 2035-03-26T04:33:07Z | cert-manager
robin-ssl | 2035-03-26T04:33:07Z | cert-manager
CERTIFICATE | EXPIRES | CERTIFICATE AUTHORITY | ISSUER_TYPE
----------------------------------------------+----------------------+-----------------------+--------------
cluster-tls | 2028-03-27T04:33:08Z | cluster-ca | cert-manager
file-server-tls | 2028-03-27T04:33:12Z | robin-ssl | cert-manager
iomgr-client-hypervvm-63-34.robinsystems.com | 2028-03-27T04:33:21Z | robin-ssl | cert-manager
iomgr-client-hypervvm-63-35.robinsystems.com | 2028-03-27T04:33:25Z | robin-ssl | cert-manager
iomgr-client-hypervvm-63-36.robinsystems.com | 2028-03-27T04:33:30Z | robin-ssl | cert-manager
iomgr-server-hypervvm-63-34.robinsystems.com | 2028-03-27T04:33:19Z | robin-ssl | cert-manager
iomgr-server-hypervvm-63-35.robinsystems.com | 2028-03-27T04:33:23Z | robin-ssl | cert-manager
iomgr-server-hypervvm-63-36.robinsystems.com | 2028-03-27T04:33:28Z | robin-ssl | cert-manager
log-server-tls | 2028-03-27T04:33:13Z | robin-ssl | cert-manager
master-tls | 2028-03-27T04:33:10Z | robin-ssl | cert-manager
node-tls-hypervvm-63-34.robinsystems.com | 2028-03-27T04:33:18Z | robin-ssl | cert-manager
node-tls-hypervvm-63-35.robinsystems.com | 2028-03-27T04:33:22Z | robin-ssl | cert-manager
node-tls-hypervvm-63-36.robinsystems.com | 2028-03-27T04:33:26Z | robin-ssl | cert-manager
robin-auth | 2035-03-26T04:33:14Z | robin-ssl | cert-manager
stormgr-client-tls | 2028-03-27T04:33:16Z | robin-ssl | cert-manager
stormgr-server-tls | 2028-03-27T04:33:14Z | robin-ssl | cert-manager
3.14.3. Status of Cluster Certificates¶
You can check the current state of all certificates in a cluster using the robin cert renewal-check command. This command also helps you to start, stop, and update the CertRenewalCheck CronJob.
The robin cert renewal-check command launches a robin-certs-check Kubernetes job, which runs inside its own Pod. The robin-certs-check job runs a script that checks the status of all cluster certificates. If the certificates of Pods or containers are about to expire, the robin-cert-monitor service renews these certificates. The robin cert renewal-check command also restarts all Pods or containers impacted by recently renewed certificates for new certificates to take effect, which results in the reloading of all certificates used by Pods or containers. The HAProxy and Keepalived containers, deployed on each master node in the cluster, are containers that utilize Robin CNP-managed certificates.
The Kubernetes API server Pods are configured with Robin CNP cluster certificates and may be restarted during a certificate renewal check.
Note
Restarting control plane Pods may cause temporary disruption to the cluster. Execute this command with caution.
Run the following command to check the current state of all certificates for a cluster:
# robin cert renewal-check
--ttl-after-finished <ttl_after_finished>
--start-cronjob
--stop-cronjob
--update-cronjob
--schedule <schedule>
--force
--dry-run
|
Number of seconds the CertRenewalCheck job stays active after its completion. If you set its value to 0, the CertRenewalCheck job will never be deleted. |
|
Start a CronJob that launches a CertRenewalCheck job on a periodic basis. |
|
Stop the CronJob that stops the CertRenewalCheck job, if running. |
|
Update the currently running CronJob to change the time period of CertRenewalCheck job. |
|
String to schedule a CronJob. For information, see Crontab. |
|
It is a mandatory option to check cluster certificates for renewal. |
|
Perform a dry run without actually triggering Pod or container restarts. You cannot use this option with the |
Example
# robin cert renewal-check --force
Cluster Certificate Check: Started (jobid=764)
# robin job info 764
ID | Type | Desc | State | Start | End | Duration | Dependson | Error | Message
----+---------------------------+-------------------------------+--------------------+-----------------+----------+----------+-----------+-------+----------------------------------------
764 | CertRenewalCheck | Checking cluster certificates | COMPLETED | 15 May 02:45:03 | 02:45:04 | 0:00:01 | [] | 0 | K8s job 'robin-certs-check' started...
3.14.4. Reset Cluster Identity CA and its certificate¶
You can reset the Cluster Identity CA and its certificate for a cluster using the robin cert reset-cluster-identity command. Once you reset the Cluster Identity CA certificate, it generates a new Cluster Identity certificate and its key for external facing services such as Kubernetes API Server, Robin client, GUI, etc.
You can use this command to configure or update the Cluster Identity certificate signed by an external CA for a cluster in the following scenarios:
A Cluster Identity certificate was not provided during Robin CNP installation, and you want to configure it post installation.
The existing Cluster Identity CA certificate is expired.
The existing Cluster Identity CA is changed.
To configure or reset a Cluster Identity certificate signed by an external CA, specify the following options as part of the robin cert reset-cluster-identity command:
–ident-ca-path
–ident-cert-path
–ident-key-path
To configure or reset a Cluster Identity certificate signed by an external PKI server, specify the following options as part of the robin cert reset-cluster-identity command:
–pki
–ca-profile
Note
To apply new certificates, you must run the robin cert renewal-check command to restart the affected Pods and containers for new certificates to take effect.
Run the following command to reset the Cluster Identity CA and its certificate for a cluster:
# robin cert reset-cluster-identity [{cert-manager,local,external,external-csr}]
--ident-ca-path <ident_ca_path>
--ident-cert-path <ident_cert_path>
--ident-key-path <ident_key_path>
--pki <pki_server>
--ca-profile <ca_profile>
--force
|
Type of certificate Issuer. Valid values are |
|
Path to the external CA certificate that signs the Cluster Identity certificate. This is required when the certificate issuer type is |
|
Path to the Cluster Identity certificate. This is required when the certificate issuer type is |
|
Path to the Cluster Identity certificate’s private key. This is required when the certificate issuer type is |
|
Name of the registered Public Key Infrastructure (PKI) server. This option is required when the issuer type is |
|
Name of the CA profile for the external PKI. It is used to fetch the CA certificate and sign the identity certificate. This option is optional when the issuer type is |
|
Forcibly reset the cluster identity CA and its certificate. This option is mandatory to reset the cluster identity CA and its certificate. |
Example
# robin cert reset-cluster-identity --force
Cluster identity CA and certificate have been reset
3.14.5. Reset Cluster CA and its certificates¶
You can reset the Cluster CA and its certificate for a cluster using the robin cert reset-cluster-certs command. It generates a new Cluster CA certificate and its key. The robin-cert-monitor service then generates new certificates and keys for all currently issued certificates in a cluster.
Note
To apply new certificates, you must run the robin cert renewal-check command to restart the affected Pods and containers for new certificates to take effect.
Run the following command to reset the Cluster CA and its certificate for a cluster:
# robin cert reset-cluster-certs [{cert-manager,local,external-csr}]
--pki <pki_server>
--ca-profile <ca_profile>
--force
|
Type of certificate Issuer. Valid values are |
|
Name of the registered Public Key Infrastructure (PKI) server. This option is required when the issuer type is |
|
Name of the CA profile for the external PKI. It is used to fetch the CA certificate and sign the identity certificate. This option is optional when the issuer type is |
|
Forcibly reset the cluster identity CA and its certificate. This option is mandatory to reset the cluster CA and its certificate. |
Example
# robin cert reset-cluster-certs local --force
Cluster CA and certificates have been reset
3.15. Manage Kubernetes Control-Plane Certificates¶
Robin CNP v5.9.0 provides a feature to manage the lifecycle of Kubernetes control-plane certificates. You can use the Kubernetes certificate management feature to view certificate expiry dates, configure automatic renewal windows, and manually trigger renewals when necessary.
The following certificates are supported:
Certificates located in
/etc/kubernetes/pki/on control-plane nodes.Certificates typically managed by the
kubeadm certs renew allcommand.
The following are list of commands available to manage Kubernetes certificates:
Section |
Details |
|---|---|
|
List K8s control-plane certificates with expiration date, remaining days, and issuer type. |
|
Trigger a one-time K8s control-plane certificate renewal Job. |
|
Update K8s certificate renewal configuration. |
|
Display current K8s certificate renewal configuration. |
3.15.1. View Kubernetes certificate status¶
The certificate status command enables you to view the following:
Total certificates and their details
Expired certificates
Certificates expiring soon
Number of healthy and unhealthy nodes with the certificate details
Certificates require renewal
To check the status and expiry dates of certificates across all control-plane nodes, run the following command:
robin k8s-cert list
The output includes the node name, certificate name, expiry date, and remaining validity.
Example
# robin k8s-cert list
Cluster Certificate Summary:
Total Certificates: 40
Expired Certificates: 0
Certificates Expiring Soon: 0
Healthy Nodes: 3
Unhealthy Nodes: 0
Renewal Required: False
Node: vnode-103-152 (Status: success, Health: healthy)
Certificate Authorities:
CA NAME | EXPIRES | REMAINING DAYS | EXTERNALLY MANAGED
---------------+------------------------+----------------+--------------------
ca | Jul 07, 2036 21:57 UTC | 3285 | No
etcd-ca | Jul 07, 2036 21:57 UTC | 3285 | No
front-proxy-ca | Jul 07, 2036 21:57 UTC | 3285 | No
Certificates:
CERTIFICATE NAME | EXPIRES | REMAINING DAYS | CERTIFICATE AUTHORITY | EXTERNALLY MANAGED
-------------------------+------------------------+----------------+-----------------------+--------------------
admin.conf | Jul 10, 2027 21:57 UTC | 351 | ca | No
apiserver | Jul 10, 2027 21:57 UTC | 351 | ca | No
apiserver-etcd-client | Jul 10, 2027 21:57 UTC | 351 | etcd-ca | No
apiserver-kubelet-client | Jul 10, 2027 21:57 UTC | 351 | ca | No
controller-manager.conf | Jul 10, 2027 21:57 UTC | 351 | ca | No
etcd-healthcheck-client | Jul 10, 2027 21:57 UTC | 351 | etcd-ca | No
etcd-peer | Jul 10, 2027 21:57 UTC | 351 | etcd-ca | No
etcd-server | Jul 10, 2027 21:57 UTC | 351 | etcd-ca | No
front-proxy-client | Jul 10, 2027 21:57 UTC | 351 | front-proxy-ca | No
scheduler.conf | Jul 10, 2027 21:57 UTC | 351 | ca | No
super-admin.conf | Jul 10, 2027 21:57 UTC | 351 | ca | No
Node: vnode-103-159 (Status: success, Health: healthy)
Certificate Authorities:
CA NAME | EXPIRES | REMAINING DAYS | EXTERNALLY MANAGED
---------------+------------------------+----------------+--------------------
ca | Jul 07, 2036 21:57 UTC | 3285 | No
etcd-ca | Jul 07, 2036 21:57 UTC | 3285 | No
front-proxy-ca | Jul 07, 2036 22:00 UTC | 3285 | No
Certificates:
CERTIFICATE NAME | EXPIRES | REMAINING DAYS | CERTIFICATE AUTHORITY | EXTERNALLY MANAGED
-------------------------+------------------------+----------------+-----------------------+--------------------
admin.conf | Jul 10, 2027 21:57 UTC | 351 | ca | No
apiserver | Jul 10, 2027 22:00 UTC | 351 | ca | No
apiserver-etcd-client | Jul 10, 2027 22:00 UTC | 351 | etcd-ca | No
apiserver-kubelet-client | Jul 10, 2027 22:00 UTC | 351 | ca | No
controller-manager.conf | Jul 10, 2027 22:00 UTC | 351 | ca | No
etcd-healthcheck-client | Jul 10, 2027 22:00 UTC | 351 | etcd-ca | No
etcd-peer | Jul 10, 2027 22:00 UTC | 351 | etcd-ca | No
etcd-server | Jul 10, 2027 22:00 UTC | 351 | etcd-ca | No
front-proxy-client | Jul 10, 2027 22:00 UTC | 351 | front-proxy-ca | No
scheduler.conf | Jul 10, 2027 22:00 UTC | 351 | ca | No
Node: vnode-103-168 (Status: success, Health: healthy)
Certificate Authorities:
CA NAME | EXPIRES | REMAINING DAYS | EXTERNALLY MANAGED
---------------+------------------------+----------------+--------------------
ca | Jul 07, 2036 21:57 UTC | 3285 | No
etcd-ca | Jul 07, 2036 21:57 UTC | 3285 | No
front-proxy-ca | Jul 07, 2036 22:03 UTC | 3285 | No
Certificates:
CERTIFICATE NAME | EXPIRES | REMAINING DAYS | CERTIFICATE AUTHORITY | EXTERNALLY MANAGED
-------------------------+------------------------+----------------+-----------------------+--------------------
admin.conf | Jul 10, 2027 21:57 UTC | 351 | ca | No
apiserver | Jul 10, 2027 22:03 UTC | 351 | ca | No
apiserver-etcd-client | Jul 10, 2027 22:03 UTC | 351 | etcd-ca | No
apiserver-kubelet-client | Jul 10, 2027 22:03 UTC | 351 | ca | No
controller-manager.conf | Jul 10, 2027 22:03 UTC | 351 | ca | No
etcd-healthcheck-client | Jul 10, 2027 22:03 UTC | 351 | etcd-ca | No
etcd-peer | Jul 10, 2027 22:03 UTC | 351 | etcd-ca | No
etcd-server | Jul 10, 2027 22:03 UTC | 351 | etcd-ca | No
front-proxy-client | Jul 10, 2027 22:03 UTC | 351 | front-proxy-ca | No
scheduler.conf | Jul 10, 2027 22:03 UTC | 351 | ca | No
3.15.2. View Kubernetes certificate current settings¶
To see your current auto-renewal configuration, run the following command:
robin k8s-cert show
Parameter |
Description |
Default |
|---|---|---|
|
How long before expiry to start the renewal. |
14 days |
|
How often the system checks for expiring certificates. |
24 hours |
|
|
|
Example
# robin k8s-cert show
renewal_offset: 1209600s
check_interval: 86400s
schedule_enabled: False
3.15.3. Update Kubernetes certificate auto-renewal settings¶
To modify the auto-renewal behavior, run the following command:
robin k8s-cert update
Option |
Description |
|---|---|
|
Enable automatic K8s certificate renewal in |
|
Disable automatic K8s certificate renewal in |
|
Seconds before expiry to trigger renewal, for example |
|
How often |
|
Reset all K8s certificate configuration to default values. |
To enable or disable the schedule:
robin k8s-cert update --enable or --disable
To change the renewal window:
robin k8s-cert update --renewal-offset <duration>
Example:
robin k8s-cert update --renewal-offset 30dTo change the check frequency:
robin k8s-cert update --check-interval <duration>
Example:
robin k8s-cert update --check-interval 12hTo reset to default values:
robin k8s-cert update --defaults
Note
Duration suffixes include d (days), h (hours), and m (minutes). The renewal_offset must be between 10 and 365 days.
3.15.4. Renew Kubernetes certificates manually¶
In some cases, you might need to renew certificates before the scheduled time.
You can use the following command to manually renew Kubernetes certificates:
robin k8s-cert renew
Option |
Description |
|---|---|
|
Seconds the renewal Job stays active after completion. A value of |
|
Required to trigger renewal. Confirms control-plane Pod restarts. |
|
Renews all certificates immediately, regardless of expiry. This option implies |
|
Minimum time before actual certificate expiration when certificate renewal is triggered. The renewal offset can be specified as seconds, minutes. |
3.15.4.1. Perform a dry run¶
To simulate the renewal process without making changes or restarting services, run the following command:
robin k8s-cert renew --dry-run
3.15.4.2. Trigger a renewal¶
To renew certificates that are within the configured renewal_offset window, run the following command:
robin k8s-cert renew --force
Warning
This command restarts control-plane components, including the API server and Etcd. This causes a brief disruption of approximately 10 to 30 seconds per master node.
3.15.4.3. Perform an emergency renewal¶
To renew certificates regardless of their current expiry date, run the following command:
robin k8s-cert renew --force-renew
3.15.5. Troubleshooting¶
Symptom |
Action |
|---|---|
A node shows an error status. |
Verify that the Robin agent is running on that specific control-plane node. |
A renewal job is stuck. |
Use |
Automatic renewal did not start. |
Run |
Permission denied error. |
Ensure you are logged in with an account that has administrator privileges. |
3.16. Secure communication between kubelet and kube-apiserver¶
In a Kubernetes cluster, the kubelet and kube-apiserver communicate with each other securely using TLS certificates. The kubelet, running on each node, authenticates with the kube-apiserver to register itself and manage containers. This communication is secured through mutual TLS, meaning both the kubelet and kube-apiserver present their certificates to verify each other’s identity. This ensures that only authorized kubelets connect to the kube-apiserver and communication between them is secure.
The kubelet presents its server certificate to clients that connect to it. By default, kubelet’s server certificate is self-signed meaning it is signed by a temporary Certificate Authority (CA) that is created on the fly and then discarded.
To enable secure communication between the kubelet and kube-apiserver, you must configure the kubelet to obtain its server certificate by issuing a Certificate Signing Request (CSR), rather than using a server certificate signed by a self-signed CA. After configuring the kubelet, you must also configure the kube-apiserver to process and approve the CSR.
Complete the following steps to configure secure communication between kubelet and kube-apiserver:
Configure kubelet on all nodes of a cluster, starting with the master nodes and then followed by the worker nodes:
# ./k8s-script-el8-5.5.0-1529.sh config-kubelet-ca
The above
k8s-scriptconfigures the kubelet on each node to request its server certificate. The kubelet sends a Certificate Signing Request (CSR) to thekube-apiserver when requesting or renewing its server certificate. After submitting the CSR. the kubelet waits for CSR approval.Configure the kube-apiserver:
# ./k8s-script-el8-5.5.0-1529.sh config-kubelet-apiserver
The
robin-cert-monitorservice, which runs automatically in the cluster, monitors for CSRs issued by the kubelets, and approves them if any CSRs are available. When the kubelet issues a CSR, it waits for CSR approval. Once it is approved, kubelet is restarted to reload its new server certificate.
3.17. Rotate Robin master key using Vault¶
You can rotate the robin-master key using HashiCorp Vault as an external Key Management Service (KMS). The robin-master key is generated during Robin CNP installation and stored in the Vault KV store.
Robin CNP uses the robin-master key as a Key Encryption Key (KEK) to encrypt and decrypt other encryption keys at the node and volume level. This feature replaces the existing master key with a new one and re-encrypts all dependent key references (keyrefs) in the database.
3.17.1. Points to consider for robin-master key rotation¶
The
robin-masterkey rotation is only supported for HashiCorp Vault KMS. It is not supported for Robin native, Google, or Kubernetes-based KMS.Rotating the
robin-masterkey improves security by ensuring that keys are periodically replaced. In Robin CNP, this process occurs during the control plane bootstrap to ensure that no storage operations are active during the re-encryption of metadata.The key rotation process re-encrypts all keyrefs in the database, including host keyrefs, encrypted volume keyrefs, and backup keyrefs. This operation can take time depending on the number of keyrefs in the database.
Backup keyrefs on remote repositories are also updated during the
robin-masterkey rotation. Ensure that the secondary cluster is accessible before starting the rotation.Do not interrupt the
robin-masterpod during key rotation. Interrupting the rotation can leave the database in an inconsistent state. If the rotation is interrupted, restart therobin-masterpod to resume the rotation from the last committed state.
3.17.2. Prerequisites¶
HashiCorp Vault must be configured as the external KMS.
All backup repositories must be online and accessible. If a repository is offline, the rotation process fails because the backup encryption keys cannot be updated.
3.17.3. Rotate the Robin master key¶
You can rotate the robin-master key. When you run the robin kms rotate-key command, it sets the rotate_kms_key flag to true in the bootstrap-config ConfigMap.
You must bounce the active Robin master Pod for the changes to take effect. After rotating the robin-master key, Robin CNP sets the rotate_kms_key flag to false in the bootstrap-config ConfigMap.
To rotate the robin-master key, run the following command:
# robin kms rotate-key
--unset
|
Reset the key rotation flag for the |
Example
# robin kms rotate-key
Master key rotation scheduled. Bounce the active robin-master pod for it to take effect. After the rotation completes, run 'robin kms sync-backup-keyrefs' to sync any re-encrypted backup key references to their backup repos.
Note
After rotating the robin-master key, you must bounce the active Robin master Pod using the kubectl delete pod <active-master-pod-name> -n robinio command for the changes to take effect. To know the active Robin master Pod, run the kubectl get lease -n robinio | grep master command.
You must run the robin kms sync-backup-keyrefs command after rotating the robin-master key.
3.17.4. View Robin master key rotation history¶
To view the history of robin-master key rotations, run the following command:
# robin kms rotation-history
--json
|
Output in JSON format. |
Example
# robin kms key-rotation-history
Version | Created At
--------+----------------------------------
v1 | 2026-09-23 06:16:19.702607+00:00
v2 | 2026-09-23 13:07:05.917337+00:00
v3 | 2026-09-25 07:27:37.632731+00:00
v4 | 2026-09-25 08:06:44.609164+00:00
3.17.5. Sync backup key references for backup repos¶
After bouncing the active Robin master Pod, you must synchronize the backup key references with their respective backup repositories. It syncs and updates each backup’s encrypted master key reference within the repository metadata.
To sync backup key references to their backup repos, run the following command:
# robin kms sync-backup-keyrefs
Example
# robin kms sync-backup-keyrefs --wait
Job: 377 Name: BackupKeyrefRepoSync State: COMPLETED Error: 0
Note
You must run the robin kms sync-backup-keyrefs command after each robin-master key rotation.