********************************************* Installing Robin Cloud Native Storage (CNS) ********************************************* In this section, we will cover the pre-requisites for installing Robin CNS, accessing the necessary software and installing Robin CNS on various Kubernetes distributions/versions. ========================= Prerequisites ========================= Detailed below are the general prerequisites that must be met for all Robin CNS installations in any environment. Environment specific prerequisites can be found within their individual sections. ---------------------------------- Minimum Resource Requirements ---------------------------------- The minimum compute requirements in order to install Robin CNS on a Kubernetes cluster are listed below: - Each node must have at least 4 CPUs. - Each node must have at least 4GB of memory. In order to determine the resource capacity on an existing Kubernetes cluster the following command can be used: .. code-block:: text $ kubectl get nodes -o custom-columns=Name:.metadata.name,OSImage:.status.nodeInfo.osImage,KubeVersion:.status.nodeInfo.kubeProxyVersion,CPU:.status.capacity.cpu,Memory:.status.capacity.memory,providerID:.spec.providerID ------------------------------------ Supported Kubernetes versions ------------------------------------ Robin CNS can be installed on the following Kubernetes Distributions/versions. +------------------------+-------------------------------+ | **Kubernetes** | **Version Supported** | +------------------------+-------------------------------+ | Google Anthos | 1.14 and 1.15 | +------------------------+-------------------------------+ | GKE | 1.16 through to 1.27 | +------------------------+-------------------------------+ ------------------------------------ Supported OS for worker nodes ------------------------------------ - CentOS 7/8 - Redhat 7/8 - Ubuntu - Red Hat Enterprise Linux CoreOS - Container optimized OS (For GKE) - EdgeOS .. Note:: Windows based OS are not supported by Robin CNS ------------------------------------ Kernel dependency ------------------------------------ 1. Minimum supported kernel version for Redhat/CentOS based OS is `3.10.0-1062.xx.el7.x86_64`. Run the command below to confirm that you are running the minimum supported kernel version:: $ uname -r or run ``yum update`` to upgrade to the latest available kernel. 2. Ensure that the ``target_core_user`` and ``tcm_loop`` kernel modules are loaded on all nodes within the cluster before starting the installation process. .. code-block:: text $ lsmod | egrep '^target_core_user|^tcm_loop' target_core_user 35043 0 tcm_loop 21028 3 In order to load the modules on an Ubuntu based machine, run the following commands: .. code-block:: text $ sudo apt-get update $ sudo apt-get install linux-modules-extra- $ sudo depmod -a $ sudo modprobe tcm_loop $ lsmod | grep tcm tcm_loop 28672 3 target_core_mod 360448 5 tcm_loop,target_core_user .. Note:: The kernel version can be determined from the output of the ``uname -r`` command. ------------------------------------- Asynchronous I/O (AIO) configuration ------------------------------------- For production environments, Robin recommends setting the ``aio-max-nr`` parameter value to ``1048576`` for each node of the cluster. For more information about the ``aio-max-nr`` parameter, see `here `__. The ``aio-nr`` parameter displays the current number of asynchronous non-blocking I/O (AIO) requests running on the node and the ``aio-max-nr`` parameter displays the maximum allowable limit of AIO concurrent requests per node. You can view the ``aio-nr`` parameter value in the ``/proc/sys/fs/aio-nr`` virtual file and the ``aio-max-nr`` parameter value in the ``/proc/sys/fs/aio-max-nr`` virtual file for a node. You can update the ``aio-max-nr`` parameter value for a node by adding the following line in the ``/etc/sysctl.conf`` file: ``fs.aio-max-nr = 1048576`` Complete the following steps to update the ``aio-max-nr`` parameter value for a node: 1. Update the ``aio-max-nr`` parameter value to ``1048576`` in the ``/etc/sysctl.conf`` by adding the following line: ``fs.aio-max-nr = 1048576`` 2. Run the following command for the changes to take effect: .. code-block:: text # sysctl --system 3. Run the following command to verify the updated ``aio-max-nr`` parameter value: .. code-block:: text # sysctl -p /etc/sysctl.conf ------------------------------------ User permissions ------------------------------------ Ensure the user installing Robin CNS is the Cluster Administrator within the Kubernetes cluster. ----------------------------------------------- Port and access requirements ----------------------------------------------- If you are installing Robin CNS on any platform, the following ports need to be accessible for the installation to succeed and for Robin to operate correctly: +---------------------+---------------------+ | **Ports** | **Description** | +---------------------+---------------------+ | 29442 - 29470 | Robin services | +---------------------+---------------------+ * These ports are not mandatory for Robin CNS installations but are necessary when applications have to be made accessible outside of the IKS cluster. .. Note:: The above list of ports also apply to on-premises clusters. For the cloud platforms, when creating each rule to make the port accessible ensure that the protocol is TCP. In addition the cloud nodes should be able to communicate with one another on all ports via the IPv4 protocol. Detailed below are the resources to manage ports and access requirements: .. **AWS** .. * Ports can be made accessible via security groups on AWS. Details on how to create security groups can be found `here `__. .. * Robin requires both the access key and secret key of a user to be passed during installation. Details on how to create/manage AWS credentials can be found `here `__. **GCP** * Ports can be made accessible via firewall rules on GCP. Details on how to create appropriate firewall rules can be found `here `__. * Instances on which Robin will be installed need to have full access to cloud APIs. This can be set during instance creation. .. **Azure** .. * Ports can be made accessible via network security groups on Azure. Details on how to create network groups can be found `here `__. .. **IKS** .. * Ports can be made accessible via network Network interfaces section on IKS. Details on how to create configure network interfaces can be found `here `__. .. * Robin requires an API key to be passed during installation. Details on how to create/manage such keys can be found `here `__. ------------------------ Supported Volumes Types ------------------------ The following are the supported volume types: - GCP: pd-ssd, pd-standard - Google Anthos : independent-persistent .. - IBM cloud Platform : IBM include general-purpose (3iops-tier), 5iops-tier, 10iops-tier, custom .. - AWS: gp2, gp3, io1, io2, st1 ------------------------ Storage requirements ------------------------ The disks available for use by Robin CNS should meeting the following requirements: - A minimum of 25 GB of disk size is required for Robin CNS usage - Should not have any filesystem - Should not have any partitions created on the disk. If the partitions should be used for robin storage, there are manual steps involved to make this work. - Robin recommends having direct access to the storage disks without RAID controller in between. If RAID is to be used and caching is enabled at RAID controller level, then cache should be battery backed. This is to guarantee data consistency when power goes off. Overall, Robin does not recommend using RAID configuration and is discussed in more detail at `here `__. .. Note:: Within the cluster, there needs to be at least one disk available for use by Robin CNS. .. ------------------------------------------- .. Networking requirements (Removed as per review comment from Tushar) .. ------------------------------------------- .. By default the Robin CNS installer assumes that nodes within the Kubernetes cluster are DNS resolvable. If this is not the case the ``--update-coredns`` installer option must be used when installing the product in order for the installation to succeed. ------------------------------------------- Tolerations requirements ------------------------------------------- Robin CNS provides tolerations for the following DaemonSets: - Robin - CSI-nodeplugin .. Note:: These DaemonSets can now tolerate any taint. As a result, you can now deploy Robin CNS on control plane nodes with master tolerations. ---------------- Node Selector ---------------- To Install Robin CNS on only selected worker nodes, you must add labels to nodes before installation and pass those labels to the ``node-selector`` field of the installer. **Example:** .. code-block:: text # kubectl label nodes key1=value1 # ./install-robin.sh --node-selector="key1:value1" =========================== Install options =========================== You can specify the install parameters for Robin CNS in the following ways: - In ``robin.yaml`` file - With Robin installer command .. Note:: The ``robin.yaml`` file has more install parameters than the Robin installer command ``./install-robin.sh``. -------------------------------------------- Specify parameters in ``robin.yaml`` -------------------------------------------- The ``robin.yaml`` file enables you to provide the install options for Robin CNS. You can use the ``robin.yaml`` file when installing Robin CNS to pass configuration details for hosts. You need to provide the ``robin.yaml`` file when installing Robin CNS using ``--cr`` or ``-f`` command option. The ``robin.yaml`` file has more install parameters than the Robin installer command. The following is the format of ``robin.yaml`` file. You can use the following format to include any configurable option in the ``robin.yaml`` file. .. code-block:: yaml apiVersion: manage.robin.io/v1 kind: RobinCluster metadata: name: robin namespace: robinio labels: app.kubernetes.io/instance: robin app.kubernetes.io/managed-by: robin.io app.kubernetes.io/name: robin spec: image_robin: # for example, robinsys/robinimg:5.4.4-103 k8s_provider: K8S_PROVIDER # possible values openshift,eks,gke,aks,iks,rke,upstream host_type: HOST_TYPE # possible values physical,ec2,gcp,azure,ibm,anthos namespace: NAMESPACE reporting_secret: REPORTING_SECRET source: DEPLOYMENT_SOURCE # possible values operatorhub,gcpmarketplace image_ubbagent: # for exapmle, robinsys/ubbagent:5.4.4-ent-1 image_nfs_server: # for example, robinsys/nfs-server:v1 # image_registry_path: IMAGE_REGISTRY_PATH # image_pull_secret: IMAGE_PULL_SECRET # options: # robinlog_dir: ROBINLOG_DIR # Path where robin logs should be stored on the host, default: /home/robinds # robincrash_dir: ROBINCRASH_DIR # Path where robin crash are stored on the host, default: /home/robinds # robinds_dir: ROBINDS_DIR # Path where robin database will be storaed on the host, default: /home/robinds # cloud_cred_secret: CLOUD_CRED_SECRET # generate_password: "true" # vault_addr: "VAULT_ADDR" # VAULT_CONFIG # vault_robin_keys_path: "VAULT_ROBIN_KEYS_PATH" # VAULT_CONFIG # vault_token: "VAULT_TOKEN" # VAULT_CONFIG # vault_skip_verify: "VAULT_SKIP_VERIFY" # VAULT_CONFIG # vault_namespace: "VAULT_NAMESPACE" # VAULT_CONFIG # http_proxy: HTTP_PROXY # https_proxy: HTTPS_PROXY # ftp_proxy: FTP_PROXY # proxy_certificate: PROXY_CERTIFICATE_PATH # gcp_key_secret: GCP_KEY_FILE # skip_k8s_auto_registration: "SKIP_K8S_AUTO_REGISTRATION" # auto_move_disks: "AUTO_MOVE_DISKS" # auto_add_storage_disks: "AUTO_ADD_STORAGE_DISKS" # disk_discovery: "DISK_DISCOVERY" # pv_folder_path: "PV_FOLDER_PATH" # pv_sc_name: "PV_SC_NAME" # pv_labels: "PV_LABELS" # Should be robin.io/enabled:true # skip_setup_iptables: "1" # If this option is set, robin will not open required ports # csi_attacher_max_backoff: "ATTACHER_BACKOFF" # storage_encryption_enabled: "1" # fast_failover_enabled: "0" # patroni_options: # volume_size: "VOLUME_SIZE" # default value 10GB # storage_class: "STORAGE_CLASS" # default value local-storage # patroni_dir: "PATRONI_DIR" # default value /home/robinds/var/lib/pgsql/patroni # cpu_requests: "CPU_REQUESTS" # default 100m # cpu_limits: "CPU_LIMITS" # default 500m # memory_requests: "MEMORY_REQUESTS" # default 4G # memory_limits: "MEMORY_LIMITS" # default 4G # skip_postgres_operator: "0" # storage_disks: STORAGE_DISKS "count:1,type:pd-ssd,size:200" # node_selector: # NODE_SELECTOR # master_node_selector: # MASTER_NODE_SELECTOR # patroni_node_selector: # tolerations: # DS_TOLERATIONS # patroni_tolerations: # PATRONI_TOLERATIONS - key: baremetal.cluster.gke.io/maintenance operator: Exists effect: NoSchedule - key: baremetal.cluster.gke.io/maintenance operator: Exists effect: NoExecute # master_tolerations: # MASTER_TOLERATIONS - key: node.kubernetes.io/not-ready effect: NoExecute tolerationSeconds: 60 - key: node.kubernetes.io/unreachable effect: NoExecute tolerationSeconds: 0 # kms: k8s/google/vault/ibmprotect # kms_keyring: # ports: # SERVER_REST_PORT: 29442 # requests: # cpu: "2" # memory: "2Gi" # limits: # cpu: "2" # memory: "2Gi" # images: # csi-attacher: k8s.gcr.io/sig-storage/csi-attacher:v3.2.0 # csi-provisioner: k8s.gcr.io/sig-storage/csi-provisioner:v2.2.0 # csi-node-driver-registrar: k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.1.0 # livenessprobe: k8s.gcr.io/sig-storage/livenessprobe:v2.5.0 # csi-resizer: k8s.gcr.io/sig-storage/csi-resizer:v1.1.0 # csi-snapshotter: k8s.gcr.io/sig-storage/csi-snapshotter:v4.2.1 # snapshot-controller: k8s.gcr.io/sig-storage/snapshot-controller:v3.0.3 # coredns: k8s.gcr.io/coredns/coredns:v1.8.0 Each option shown above is described below alongside whether or not they must to be present in the YAML file. ================================= ========== ================================================================================= ``image_robin`` Required Name and version of the Robin CNS image as per docker registry. The format of Robin CNS image: ``robinsys/robinimg:`` **Example:** ``robinsys/robinimg:5.4.4-103`` ``k8s_provider`` Required Name of the Kubernetes distribution. The following values are supported: - ``gke`` (Google Kubernetes Engine), - ``openshift`` (Redhat Openshift), - ``eks`` (Amazon Elastic Kubernetes Engine), - ``aks`` (Azure Kubernetes Service), - ``pks`` (Pivotal Container Service), - ``rke`` (Rancher Kubernetes Engine), - ``upstream`` (Upstream Kubernetes) ``host_type`` Required Type of host on which the Kubernetes distribution is running. The following values are supported: - ``physical`` (Private On-premises Datacenter) - ``ec2`` (Amazon EC2 Instances), - ``gcp`` (Google Compute Engine Instances), - ``azure`` (Azure Virtual Machines), - ``ibm`` (IBM Virtual Server Instances), - ``anthos`` (Google Cloud Anthos) ``namespace`` Required Namespace in which all Robin CNS-related objects will be spawned. Default value: ``robinio`` ``reporting_secret`` Required Secret name created for the license key generated by Google marketplace pertaining to the service account. ``source`` Required Source of deployment yaml. Only specify this value if Robin CNS is deployed through one of the following portals: ``operatorhub`` or ``gcpmarketplace`` ``image_ubbagent`` Required Version of UBB agent image. The format of the UBB agent image: ``robinsys/ubbagent:`` **Example:** ``robinsys/ubbagent:5.4.4-ent-1`` ``image_nfs_server`` Required Version of NFS server image. The format of the NFS server image: ``robinsys/nfs-server:`` **Example:** ``robinsys/nfs-server:v1`` ``image_registry_path`` Optional Path to the registry from which images will be pulled. ``image_pull_secret`` Optional K8s secret containing the credentials to pull images. ``robinlog_dir`` Optional Path to the directory where Robin logs will be stored on the host. Default value: ``/home/robinds`` ``robincrash_dir`` Optional Path to the directory where Robin crash logs will be stored on the host. Default value: ``/home/robinds`` ``robinds_dir`` Optional Path to the directory where Robin database will be stored on the host. Default value: ``/home/robinds`` ``cloud_cred_secret`` Required Name of secret containing cloud specific credentials. This is required for Azure and IBM, optional for AWS if EC2 instances are configured with the necessary IAM roles, and optional for GCP deployments. ``generate_password`` Optional Automatically generate the random password for the admin user. To know the generated password for the admin user, run the following command: .. code-block:: text # kubectl get secret --namespace robinio default-admin-user -o jsonpath=" {.data.password}" | base64 --decode; echo If this option is not provided, then the default password for the admin user will be ``Robin123``. ``vault_addr`` Optional URL of the Vault server. The format of the Vault address: ``https://:`` **Example:** ``https://192.0.2.1:8080`` ``vault_robin_keys_path`` Optional Path to the folder where the key for Robin CNS is stored. ``vault_token`` Optional Token for the Vault server. ``vault_skip_verify`` Optional Skip the verification for the Vault server. ``vault_namespace`` Optional Namespace where the Vault server is deployed. ``http_proxy`` Optional Proxy server to be used for cloud APIs over HTTP. By default, it is not set. ``https_proxy`` Optional Proxy server to be used for cloud APIs over HTTPS. By default, it is not set. ``ftp_proxy`` Optional Proxy server to be used for cloud APIs over FTP. By default, it is not set. ``proxy_certificate`` Optional Certificate for the Proxy server. ``gcp_key_secret`` Optional Key secret for Google Cloud Platform (GCP). ``skip_k8s_auto_registration`` Optional Skip the auto registration of the created applications to the Robin CNS. Apps created will not get auto-registered in Robin CNS. Default value: ``true`` ``auto_move_disks`` Optional Enable automatically attach/detach of disks on cloud-based clusters when nodes become unreachable. Default value: ``true`` ``auto_add_storage_disks`` Optional Automatically add storage disks to Robin CNS. Default value: ``true`` The following are the valid values: - ``true`` - To add storage disks automatically to Robin CNS. - ``false`` - Not to add storage disks automatically to Robin CNS. ``disk_discovery`` Optional Type of disk-discovery to be used by Robin CNS for storage. Default value: ``disk`` The following are the valid values: - ``disk`` - to use all disks available on the host. - ``pv`` - to use local PersistenceVolumes (PVs) available on the host and claim them as PersistenceVolumeClaims (PVCs). ``pv_folder_path`` Optional Path to PV folder. ``pv_sc_name`` Optional Name of StorageClass for PV. ``pv_labels`` Optional Labels for PV. **Example:** ``robin.io/enabled:true`` ``skip_setup_iptables`` Optional Skip to set up IP tables rule. If this option is set, Robin will not open the required ports. ``csi_attacher_max_backoff`` Optional Set the backoff time for CSI attacher Pods. **Example:** ``“30s”`` ``storage_encryption_enabled`` Optional Enable to secure the in-transit data while transferring from one node to another node. By default, this option is disabled. ``fast_failover_enabled`` Optional Disable auto fast failover of Pods that use Robin Storage. By default, this option is enabled. ``storage_disks`` Optional Disk for storage. If disks need to be provisioned as part of the installation, a string containing the count, type and size (in GB) of each disk can be specified. Valid disk types for GCP based machines include: ``pd-ssd`` and ``pd-standard``. Valid disk types for AWS based machines include: ``gp2``, ``gp3``, ``io1``, ``io2``, and ``st1``. Valid disk types for Azure include: ``premium`` and ``standard``. ``node_selector`` Optional Specify a selector to pick the nodes on which Robin CNS should be installed. For OpenShift v3 specify "node-role.kubernetes.io/compute=true" or for OpenShift v4 specify "node-role.kubernetes.io/master=''" ``master_node_selector`` Optional Specify a selector to pick the node for the master node role on which Robin CNS should be installed and Patroni instance will come up. **Example:** ``master-node-selector="color:red,org:robin"`` ``patroni_node_selector`` Optional This parameter is applicable to Patroni PV creation. Specify this parameter to schedule Patroni Pods on nodes with labels for Patroni. It will create Patroni Pods on the nodes with local PV. **Example:** ``patroni_node_selector= "color:green"`` ``tolerations`` Optional Tolerations for Robin DaemonSet Pods to be deployed on nodes with master taints ``patroni_tolerations`` Optional Specify this parameter if any nodes have a taint for Patroni pods and you want the Robin Patroni Pods to scheduled on that node. **Note**: You must add the custom Patroni tolerations after the default Patroni tolerations as provided in the ``robin.yaml`` file. **Example**: If you have nodes A,B, and C and you apply a taint to node A as patroni:NoSchedule., however, you want Robin Patroni Pod to schedule on any of the nodes, then you should use the patroni_tolerations parameter. ``master_tolerations`` Optional Specify this parameter if a master node has a taint for pods and you want Robin to schedule Pods on that node. **Note**: You must add the custom master tolerations after the default master tolerations as provided in the ``robin.yaml`` file. ``kms`` Optional Name of the key management service to use. The following are the valid values: - K8s - google - vault - ibmprotect ``kms_keyring`` Optional Name of the keyring associated with the KMS if Google KMS is provided. ``ports`` Optional Customize ports for using Robin services. ``requests`` Optional Request resource for Robin Pods. ``limits`` Optional Set resource limits for Robin Pods. **Patroni options** ``volume_size`` Optional The size of the volume for one replica, default value: ``10GB`` ``storage_class`` Optional Storage class for the PV. Default and only supported value is ``local-storage`` ``patroni_dir`` Optional Local directory to be used if storage class is ``local-storage``. Default value: ``/home/robinds/var/lib/pgsql/patroni`` ``cpu_requests`` Optional CPU requests for each Patroni Pod. Default value: ``100m`` ``cpu_limits`` Optional CPU limits for each Patroni Pod. Default value: ``500m`` ``memory_requests`` Optional Memory requests for each Patroni Pod. Default value: ``4G`` ``memory_limits`` Optional Memory limits for each Patroni Pod. Default value: ``4G`` ``skip_postgres_operator`` Optional Use this parameter if Zalando Postgres operator is already installed on your cluster. If the value is set to 1, Robin CNS install or upgrade operation skips Postgres operator installation. **Note**: You must make sure that the existing Zalando Postrgres operator is running version 1.10.0 or higher. With the install script, you can use the ``--skip-postgres-operator`` option during install or upgrade. **Images** ``csi-attacher`` Optional Version of CSI attacher image. **Example:** csi-attacher: k8s.gcr.io/sig-storage/csi-attacher:v3.2.0 ``csi-provisioner`` Optional Version of CSI provisioner image. **Example:** csi-provisioner: k8s.gcr.io/sig-storage/csi-provisioner:v2.2.0 ``csi-node-driver-registrar`` Optional Version of CSI node driver image. **Example:** csi-node-driver-registrar: k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.1.0 ``livenessprobe`` Optional Version of Liveness probe image. **Example:** livenessprobe: k8s.gcr.io/sig-storage/livenessprobe:v2.5.0 ``csi-resizer`` Optional Version of CSI resizer image. **Example:** csi-resizer: k8s.gcr.io/sig-storage/csi-resizer:v1.1.0 ``csi-snapshotter`` Optional Version of CSI snapshotter image. **Example:** csi-snapshotter: k8s.gcr.io/sig-storage/csi-snapshotter:v4.2.1 ``snapshot-controller`` Optional Version of Snapshot controller image. **Example:** snapshot-controller: k8s.gcr.io/sig-storage/snapshot-controller:v3.0.3 ``coredns`` Optional Version of core DNS image. **Example:** coredns: k8s.gcr.io/coredns/coredns:v1.8.0 ================================= ========== ================================================================================= ------------------------------------------------ Specify parameters with Robin installer ------------------------------------------------ The following parameters can be specified with the Robin CNS install command ``./install-robin.sh``: ================================ ============================================= **Parameter** **Description** ================================ ============================================= ``-a|--access-key`` Access key for Amazon web service (AWS). ``-s|--secret-key`` Secret key for AWS. ``-p|--app-id`` App ID for Azure. ``-t|--tenant-id`` Tenant ID for Azure. ``-c|--cluster-name`` Cluster name of Google Kubernetes Engine (GKE). ``-z|--zone-name`` Zone name where GKE cluster is running. ``-g|--generate-yaml`` Generate a yaml. ``-f|--cr`` Path to ``robin.yaml`` to install Robin CNS using the custom parameters. ``-n|--node-selector`` Specify a selector to pick up the nodes on which Robin CNS will be installed. ``-r|--image-registry-path`` Path to the registry where Robin images are stored. ``-l|--image-pull-secret`` K8s secret containing the credentials to pull images. ``--skip-lb`` Skip the load balancer. ``--storage-disks`` Disk for storage. If disks need to be provisioned as part of the installation, a string containing the count, type and size (in GB) of each disk can be specified. Valid disk types for GCP based machines include: ``pd-ssd`` and ``pd-standard``. Valid disk types for AWS based machines include: ``gp2``, ``gp3``, ``io1``, ``io2``, and ``st1``. Valid disk types for Azure include: ``premium`` and ``standard``. **Example:** count:1,type:pd-ssd,size:200 ``-u|--uninstall`` Uninstall Robin CNS. ``-U|--upgrade`` Upgrade Robin CNS. ``-v|--version`` Version of Robin CNS. ``-y|--yes`` Do not prompt the user for confirmation of evacuation. ``--vault-addr`` URL of the Vault server. The format of the Vault address: ``https://:``. **Example:** ``https://192.0.2.1:8080`` ``--vault-keys-path`` Path to the folder where the key for Robin CNS is stored in the Vault Server. ``--vault-token`` Token for the Vault server. ``--vault-ca-cert`` Path to the CA certificate for the Vault server. ``--vault-client-cert`` Path to the client certificate for the Vault server. ``--vault-client-key`` Path to the key of the client certificate for Vault server. ``--vault-namespace`` Namespace in which Vault server is installed. ``--vault-skip-verify`` Skip verification of Vault server. Default value: ``false``. ``--kms-keyring`` Keyring name associated with the key management service (KMS) if Google KMS is provided. ``--gcp-key-file`` Path to the JSON file containing the Service account key for the appropriate Service Account on Google Cloud. ``--proxy-cert`` Path to certificate of the Proxy server. ``--repo`` Path to registry to pull Robin images. ``--host-check`` Check TCMU kernel module and network ports required by CNS. ``--kernel-check`` Check the Kernel version. ``--ip-protocol`` Type of node’s IP address. Default value: ``ipv4``. Valid values are ``ipv4`` and ``ipv6``. ``--logs`` Creates a tar of logs files. ``--robinds-dir`` Path to the folder where Robin database will be stored on the host. Default: ``/home/robinds`` ``--robinlog-dir`` Path to the folder where Robin logs will be stored on the host. Default: ``/home/robinds`` ``--robincrash-dir`` Path to the folder where Robin crash logs will be stored on the host. Default: ``/home/robinds`` ``--skip-k8s-auto-registration`` Skip auto registration of the created applications to the Robin CNS. Apps created will not get auto-registered in Robin CNS. Default value: ``True`` ``--execute-prechecks`` Execute prechecks on nodes before installing Robin CNS. ``--skip-auto-move-disks`` Skip to move storage disks automatically. ``--skip-auto-add-disks`` Skip to add storage disks automatically to Robin CNS. ``--validate-all-nodes`` Validate all nodes of the cluster. ``--disk-discovery`` Type of disk-discovery to be used by Robin CNS for storage. Default value: ``disk``. The following are the valid values: - ``disk`` - to use all disks available on the host. - ``pv`` - to use local PVs available on the host and claim them as PVCs. ``--pv-sc-name`` Name of StorageClass for PV. ``--pv-folder-path`` Path to PV folder. ``--pv-labels`` Labels for PV. For example, ``robin.io/enabled:true`` ``--reporting-secret`` Secret name created for the license key generated by Google marketplace pertaining to the service account. ``--generate-password`` Automatically generates the random password for the admin user. To know the generated password for the admin user, run the following command: .. code-block:: text # kubectl get secret --namespace robinio default-admin-user -o jsonpath="{.data.password}" | base64 --decode; echo If this option is not provided, then the default password for the admin user will be Robin123. ``--skip-setup-iptables`` Skip to set up IP tables rule. If this option is set, Robin will not open the required ports. ``-m|--master-node-selector`` Specify a selector to pick the node for the master node role on which Robin CNS should be installed and Patroni instance will come up. You can label a node and use the label to select the node. For example, --master-node-selector="color:red,org:robin" ``--enable-metadata-encryption`` Enables mTLS enabled metadata encryption in transit. ``--skip-postgres-operator`` Use this option if Zalando Postgres operator is already installed on your cluster. ================================ ============================================= .. Note:: The ``robin.yaml`` file has more install parameters than the Robin installer command. =========================== Pre-install Checks =========================== Robin CNS provides the automatic pre-installation checks feature. You can run these checks before installing Robin CNS. The Pre-installation checks are done at the cluster level and node level. **Cluster Level Checks** The following are the cluster-level checks: - Supported Kubernetes distribution. - Minimum CPU - Minimum memory **Node Level Checks** - Checks whether the following Kernel modules are present: - tcm_loop - target_core_user To run the pre-install checks, run the following command: .. code-block:: text ./install-robin.sh --execute-prechecks **Example** .. code-block:: text # ./install-robin.sh --execute-prechecks --repo=robineng Creating robin-precheck namespace and service account ............ Done Creating cluster-wide prechecks job ............ Done Creating node prechecks daemonset ............ Done Waiting for prechecks to complete ............ Done Generating precheck report ............ Done Cleaning up precheck deployments ............ Done Cluster/Node prechecks report is generated at /tmp/cns-prechecks-report-1685433893.txt . (Note: please use cat or less -R to view the file) CLUSTER WIDE PRECHECKS Checking kubernetes distribution (GKE IKS AKS RKE OpenShift) ...... OK Checking minimum memory (each node should have minimum 4Gi memory) ...... OK Checking minimum cpus (each node should have minimum 4 cpus) ...... OK NODE hypervvm-72-42.robinsystems.com PRECHECKS Checking target_core_user kernel module ...... OK Checking tcm_loop kernel module ...... OK NODE hypervvm-72-44.robinsystems.com PRECHECKS Checking target_core_user kernel module ...... OK Checking tcm_loop kernel module ...... OK NODE hypervvm-72-45.robinsystems.com PRECHECKS Checking target_core_user kernel module ...... OK Checking tcm_loop kernel module ...... OK =========================== Install on Google Anthos =========================== Anthos is a managed platform from Google that allows for the unification of multiple environments across on-premises, edge, and in multiple public clouds with a Google Cloud-backed control plane. Robin Cloud Native Storage is available as an offering to bring enterprise grade Storage and Data Management capabilities to run Stateful applications on Anthos. You can download Robin CNS software and install using a shipped installation script or install it manually via the CLI. In this section we will cover both. .. Note:: You can install Robin CNS on Anthos installed on a bare metal system. The same prerequisites and steps applicable for installing Robin CNS on Cloud are applicable for Anthos installed on bare metal setup. -------------- Prerequisites -------------- - All general prequisites detailed `here `__ must be met. - The ports detailed `here `__ need to be accessible for the installation to succeed and for Robin to operate correctly. - Ensure the user installing Robin CNS is the Cluster Administrator within the Kubernetes cluster. For Anthos installations, run following commands to add a user as a Cluster Administrator: .. code-block:: text $ account=$(gcloud info | grep Account | awk -F'[\\[\\]]' '{print $2}') $ kubectl create clusterrolebinding robin-cluster-admin-binding --clusterrole=cluster-admin --user=$account ------------------------------------ Install using Robin Installer ------------------------------------ Follow the steps below to get and install Robin CNS for Google Anthos. 1. Contact Robin Support team for Robin CNS software binaries. 2. After getting the Robin CNS from the support team, run the following command to untar the contents into a new directory. .. code-block:: text $ mkdir -p ./robin && tar -xf robin-install.tar -C ./robin 3. Move into the newly created directory, such that it is now your current working directory, and install Robin CNS via the following command: .. Note:: To customize the installation with more parameters, see the various installation parameters detailed `here `__. .. code-block:: text # cd robin # ./install-robin.sh -y Validating Kubernetes cluster......Done Confirm that ports 29442-29470 are open between kubernetes nodes Make sure the cluster meets prerequisites mentioned at https://docs.robin.io/latest/install.html#minimum-node-requirements Type yes to confirm that cluster meets the prerequisites: yes Validating kubernetes cluster permissions......Done Validating robin cluster yaml......Done Installing Robin operator...........Done Installing Robin cluster............secret/cloud-cred-secret created Done Setting up robin client............Done Activate robin license............Required # robin license activate --userid Note: You can get your User ID after registering on https://get.robin.io. Above command will only work if the host on which the ROBIN client is running on has an internet connection. If this is not the case please retrieve the license key by using the below link and apply it using the command 'robin license apply '. https://get.robin.io/activate?clusterid= For logging into robin cluster, Default username: admin Default password: Robin123 4. This concludes the install. Continue `here `__ to verify the installation and set up the product. ------------------------------- Install via CLI ------------------------------- Follow the steps below to manually install Robin CNS for Google Anthos using CLI commands. 1. Clone the ``robin-storage`` repository using the command shown below: .. code-block:: text $ git clone https://github.com/RobinSystems/robin-storage-gke-marketplace.git Cloning into 'robin-storage-gke-marketplace'... remote: Enumerating objects: 142, done. remote: Counting objects: 100% (12/12), done. remote: Compressing objects: 100% (11/11), done. remote: Total 142 (delta 2), reused 9 (delta 1), pack-reused 130 Receiving objects: 100% (142/142), 84.59 KiB | 0 bytes/s, done. Resolving deltas: 100% (64/64), done. 2. Move into the newly created directory, such that it is now your current working directory, and create the application CRD within the Anthos cluster if it is not already available with the following command: .. code-block:: text $ cd robin-storage-gke-marketplace/ $ kubectl apply -f app-crd.yaml customresourcedefinition.apiextensions.k8s.io/applications.app.k8s.io configured 3. Create the ``robinio`` namespace if it is not present using the below command: .. code-block:: text $ export NAMESPACE=robinio $ kubectl create ns ${NAMESPACE} namespace/robinio created 4. Visit the following `link `__ and download a license key from the `Deploy via Command Line` tab. .. figure:: ./images/license-key-gke.png :width: 700 5. Once the license key is downloaded, run the below command to apply it. .. code-block:: text $ kubectl apply -f license.yaml -n robinio secret/robin-cns-license-335206 created $ export REPORTING_SECRET="robin-cns-license-335206" .. Note:: The value for the ``REPORTING_SECRET`` environment variable should be the name of the Kubernetes secret created after the license YAML is applied. 6. Run the following commands in order to setup the appropriate variables, apply the necessary YAML files and install Robin: .. code-block:: text $ export APP_INSTANCE_NAME=robin $ export HOST_TYPE=physical $ export TAG=5.3.10 $ export IMAGE_ROBIN_OPERATOR="gcr.io/robinio-public/robin-storage/robin-operator:${TAG}" $ export IMAGE_ROBIN="gcr.io/robinio-public/robin-storage:${TAG}" $ export IMAGE_UBBAGENT="gcr.io/robinio-public/robin-storage/ubbagent:${TAG}" $ export IMAGE_ROBIN_DEPLOYER="gcr.io/robinio-public/robin-storage/deployer:${TAG}" $ export IMAGE_CRJOB="gcr.io/robinio-public/robin-storage/crjob:${TAG}" $ export CLUSTER_REQUIREMENTS="yes" $ export STORAGE_DISKS="" $ export ROBINDS_DIR="/home/robinds" $ export ROBINLOG_DIR="/home/robinds" $ export ROBINCRASH_DIR="/home/robinds" $ export UPDATE_COREDNS="0" $ export DNS_SERVICE="kube-system/kube-dns" $ awk 'FNR==1 {print "---"}{print}' sa.yaml.template | envsubst '$APP_INSTANCE_NAME $NAMESPACE $IMAGE_ROBIN_OPERATOR $IMAGE_ROBIN $SERVICE_ACCOUNT $IMAGE_PROVISIONER_V04 $IMAGE_PROVISIONER_V10 $REPORTING_SECRET $IMAGE_UBBAGENT $IMAGE_ROBIN_DEPLOYER $CLUSTER_REQUIREMENTS $STORAGE_DISKS $HOST_TYPE $IMAGE_CRJOB $UPDATE_COREDNS $DNS_SERVICE' > "sa_manifest.yaml" $ kubectl apply -f "sa_manifest.yaml" --namespace "${NAMESPACE}" serviceaccount/robin-deployer-sa created clusterrole.rbac.authorization.k8s.io/robin:robinio:deployer-cr0 created clusterrolebinding.rbac.authorization.k8s.io/robin:robinio:deployer-crb0 created $ export SERVICE_ACCOUNT="${APP_INSTANCE_NAME}-deployer-sa" $ awk 'FNR==1 {print "---"}{print}' manifest/* | envsubst '$APP_INSTANCE_NAME $NAMESPACE $IMAGE_ROBIN_OPERATOR $IMAGE_ROBIN $SERVICE_ACCOUNT $IMAGE_PROVISIONER_V04 $IMAGE_PROVISIONER_V10 $REPORTING_SECRET $IMAGE_UBBAGENT $IMAGE_ROBIN_DEPLOYER $CLUSTER_REQUIREMENTS $STORAGE_DISKS $HOST_TYPE $ROBINDS_DIR $ROBINLOG_DIR $ROBINCRASH_DIR $IMAGE_CRJOB $UPDATE_COREDNS $DNS_SERVICE' > "${APP_INSTANCE_NAME}_manifest.yaml" $ kubectl apply -f "${APP_INSTANCE_NAME}_manifest.yaml" --namespace "${NAMESPACE}" application.app.k8s.io/robin created customresourcedefinition.apiextensions.k8s.io/robinclusters.manage.robin.io created configmap/robin-cr created deployment.apps/robin-operator created job.batch/robin-cr created deployment.apps/robin-monitor created 7. This concludes the install. Continue `here `__ to verify the installation and set up the product. ------------------------------------------------------------ Install with Google KMS integration ------------------------------------------------------------ Google KMS can be integrated with Robin CNS at the time installation on Google Anthos. Detailed instructions can be found `here `__. =========================== Install on Google GKE =========================== GKE is a managed Kubernetes offering from Google that runs in the Google Cloud Platform. Robin Cloud Native Storage is available as a marketplace offering to bring enterprise grade Storage and Data Management capabilities to run Stateful applications on GKE. You can also download Robin CNS software and install using the shipped installation script or manually via the CLI. In this section we will cover all these options. -------------- Prerequisites -------------- - All general prequisites detailed `here `__ must be met. - The ports detailed `here `__ need to be accessible for the installation to succeed and for Robin to operate correctly. .. - The kubernetes auto-upgrade option must be disabled at the time of install or immediately after a successful installation. Failure to do so could lead to future disruptions. - Ensure the user installing Robin CNS is the Cluster Administrator within the Kubernetes cluster. For GKE installations, run following commands to add a user as a Cluster Administrator: .. code-block:: text $ account=$(gcloud info | grep Account | awk -F'[\\[\\]]' '{print $2}') $ kubectl create clusterrolebinding robin-cluster-admin-binding --clusterrole=cluster-admin --user=$account .. Note:: For **GKE** installations, Robin needs cluster with ``Compute Engine: Read Write`` and ``Storage: Full`` API being enabled. Compute Engine API access is required to make sure disks are always accessible and easy way to provision disks within ROBIN. Storage API access is required for backing up applications in GCS. You can select the ``Access scope`` as one of the following while creating GKE cluster. - Set access for each API and select individual permissions - Allow full access to all Cloud APIs ------------------------------- Install via Google Marketplace ------------------------------- .. _marketplace: https://console.cloud.google.com/marketplace .. |marketplace| replace:: **marketplace** 1. Access the |marketplace|_ and search for ``Robin Cloud Native storage``. .. figure:: ./images/GKE-1.png :width: 700 2. From the search results, select *Robin Cloud Native Storage*. .. figure:: ./images/GKE-3.png :width: 700 3. In the **Product details** page, click **Configure**. .. figure:: ./images/GKE-4.png :width: 700 4. In the **Deploy Robin Cloud Native Storage** page, select an existing GKE cluster or create a new cluster to deploy Robin CNS. Fill each of the options appropriately and type 'YES' for **Cluster requirements** parameter. Accept the **GCP Marketplace Terms of Service** and click **Deploy**. .. figure:: ./images/GKE-5.jpg :width: 700 5. You will be taken to the **Application details** page. You should see a similar output as displayed in the following screenshot. Wait until all the components turn to **OK** or **Running** status. .. figure:: ./images/GKE-7.jpg :width: 700 After installing Robin CNS, `verify the installation `__ and set up your cluster. .. Note:: Billing for Robin CNS when deploying on Google Marketplace is done on an hourly basis and is dependant on the total number of CPUs in the cluster. The exact formula to calculate the total cost is as follows: ``#TotalCPUs x #Hours x $ per CPU per hour``. In addition, to find the number of CPUs on a single node the ``cat /proc/cpuinfo`` command can be used. ------------------------------------ Install using Robin Installer ------------------------------------ Follow the steps below to download and install Robin CNS for GKE. 1. Contact Robin Support team for Robin CNS software binaries. 2. After getting the Robin CNS from the support team, run the following command to untar the contents into a new directory. .. code-block:: text $ mkdir -p ./robin && tar -xf robin-install.tar -C ./robin 3. Move into the newly created directory, such that it is now your current working directory, and install Robin via the following commands: .. Note:: To customize the installation with more parameters, see the various installation parameters detailed `here `__. .. code-block:: text # cd robin # ./install-robin.sh --cluster-name= --zone-name= -y Validating Kubernetes cluster......Done Confirm that ports 29442-29470 are open between kubernetes nodes Make sure the cluster meets prerequisites mentioned at https://docs.robin.io/latest/install.html#minimum-node-requirements Type yes to confirm that cluster meets the prerequisites: yes Validating kubernetes cluster permissions......Done Validating robin cluster yaml......Done Installing Robin operator...........Done Installing Robin cluster............secret/cloud-cred-secret created Done Setting up robin client............Done Activate robin license............Required # robin license activate --userid Note: You can get your User ID after registering on https://get.robin.io. Above command will only work if the host on which the ROBIN client is running on has an internet connection. If this is not the case please retrieve the license key by using the below link and apply it using the command 'robin license apply '. https://get.robin.io/activate?clusterid= For logging into robin cluster, Default username: admin Default password: Robin123 4. This concludes the install. Continue `here `_ to verify the installation and set up the product. ------------------------------- Install via CLI ------------------------------- Follow the steps below to manually install Robin CNS for GKE using CLI commands. 1. Clone the ``robin-storage`` repository using the command shown below: .. code-block:: text $ git clone https://github.com/RobinSystems/robin-storage-gke-marketplace.git Cloning into 'robin-storage-gke-marketplace'... remote: Enumerating objects: 142, done. remote: Counting objects: 100% (12/12), done. remote: Compressing objects: 100% (11/11), done. remote: Total 142 (delta 2), reused 9 (delta 1), pack-reused 130 Receiving objects: 100% (142/142), 84.59 KiB | 0 bytes/s, done. Resolving deltas: 100% (64/64), done. 2. Move into the newly created directory, such that it is now your current working directory, and create the application CRD within the Anthos cluster if it is not already available with the following command: .. code-block:: text $ cd robin-storage-gke-marketplace/ $ kubectl apply -f app-crd.yaml customresourcedefinition.apiextensions.k8s.io/applications.app.k8s.io configured 3. Create the ``robinio`` namespace if it is not present using the below command: .. code-block:: text $ export NAMESPACE=robinio $ kubectl create ns ${NAMESPACE} namespace/robinio created 4. Visit the following `link `__ and download a license key from the `Deploy via Command Line` tab. .. figure:: ./images/license-key-gke.png :width: 700 5. Once the license key is downloaded, run the below command to apply it. .. code-block:: text $ kubectl apply -f license.yaml -n robinio secret/robin-cns-license-335206 created $ export REPORTING_SECRET="robin-cns-license-335206" .. Note:: The value for the ``REPORTING_SECRET`` environment variable should be the name of the Kubernetes secret created after the license YAML is applied. 6. Run the following commands in order to setup the appropriate variables, apply the necessary YAML files and install Robin: .. code-block:: text $ export APP_INSTANCE_NAME=robin $ export HOST_TYPE=gcp $ export TAG=5.3.10 $ export IMAGE_ROBIN_OPERATOR="gcr.io/robinio-public/robin-storage/robin-operator:${TAG}" $ export IMAGE_ROBIN="gcr.io/robinio-public/robin-storage:${TAG}" $ export IMAGE_UBBAGENT="gcr.io/robinio-public/robin-storage/ubbagent:${TAG}" $ export IMAGE_ROBIN_DEPLOYER="gcr.io/robinio-public/robin-storage/deployer:${TAG}" $ export IMAGE_CRJOB="gcr.io/robinio-public/robin-storage/crjob:${TAG}" $ export CLUSTER_REQUIREMENTS="yes" $ export STORAGE_DISKS="" $ export ROBINDS_DIR="/home/robinds" $ export ROBINLOG_DIR="/home/robinds" $ export ROBINCRASH_DIR="/home/robinds" $ export UPDATE_COREDNS="0" $ export DNS_SERVICE="kube-system/kube-dns" $ awk 'FNR==1 {print "---"}{print}' sa.yaml.template | envsubst '$APP_INSTANCE_NAME $NAMESPACE $IMAGE_ROBIN_OPERATOR $IMAGE_ROBIN $SERVICE_ACCOUNT $IMAGE_PROVISIONER_V04 $IMAGE_PROVISIONER_V10 $REPORTING_SECRET $IMAGE_UBBAGENT $IMAGE_ROBIN_DEPLOYER $CLUSTER_REQUIREMENTS $STORAGE_DISKS $HOST_TYPE $IMAGE_CRJOB $UPDATE_COREDNS $DNS_SERVICE' > "sa_manifest.yaml" $ kubectl apply -f "sa_manifest.yaml" --namespace "${NAMESPACE}" serviceaccount/robin-deployer-sa created clusterrole.rbac.authorization.k8s.io/robin:robinio:deployer-cr0 created clusterrolebinding.rbac.authorization.k8s.io/robin:robinio:deployer-crb0 created $ export SERVICE_ACCOUNT="${APP_INSTANCE_NAME}-deployer-sa" $ awk 'FNR==1 {print "---"}{print}' manifest/* | envsubst '$APP_INSTANCE_NAME $NAMESPACE $IMAGE_ROBIN_OPERATOR $IMAGE_ROBIN $SERVICE_ACCOUNT $IMAGE_PROVISIONER_V04 $IMAGE_PROVISIONER_V10 $REPORTING_SECRET $IMAGE_UBBAGENT $IMAGE_ROBIN_DEPLOYER $CLUSTER_REQUIREMENTS $STORAGE_DISKS $HOST_TYPE $ROBINDS_DIR $ROBINLOG_DIR $ROBINCRASH_DIR $IMAGE_CRJOB $UPDATE_COREDNS $DNS_SERVICE' > "${APP_INSTANCE_NAME}_manifest.yaml" $ kubectl apply -f "${APP_INSTANCE_NAME}_manifest.yaml" --namespace "${NAMESPACE}" application.app.k8s.io/robin created customresourcedefinition.apiextensions.k8s.io/robinclusters.manage.robin.io created configmap/robin-cr created deployment.apps/robin-operator created job.batch/robin-cr created deployment.apps/robin-monitor created 7. This concludes the install. Continue `here `__ to verify the installation and set up the product. ------------------------------------------------------------ Install with Google KMS integration ------------------------------------------------------------ Google KMS can be integrated with Robin CNS at the time installation on GKE. Detailed instructions can be found `here `__. =========================================================== Install with Google KMS using Robin Installer =========================================================== Robin Cloud Native Storage (CNS) supports the integration of Google Cloud Key Management as a key management service (KMS). `Googke Cloud KMS `__ is a product that allows for fast and centralized key management in Google Cloud. It enables the storage of API keys, passwords, certificates and encryption of Kuberenetes secrets at the application layer on GKE and Google Anthos. More information on the product can be found `here `__. --------------- Prerequisites --------------- - All general prequisites detailed `here `__ must be met. - Access to a Kubernetes cluster running a supported version of GKE or Google Anthos must be available. - All prerequisites for the base Kuberentes distribution must be met. For GKE clusters this is available `here `__, whilst for Anthos cluster this is available `here `__. - A JSON file containing the Service Account key for the appropriate Service Account on Google Cloud is needed. To create and download the Service account key, follow the steps available `here `__. - The keyring name of the respective KMS must be known. .. Note:: Google KMS can only be integrated as part of the initial Robin CNS installation using the Robin installer, it is not possible to configure it post-deployment. --------------------- Necessary Parameters --------------------- The below parameters must be passed to the installation script in order to integrate Robin CNS with Google KMS: - ``kms-keyring`` - The name of the keyring associated with the desired KMS. - ``gcp-key-file`` - The fully qualified pathname of the JSON file containing the Service account key to use. ------------------- Installation Steps ------------------- Complete the following steps to download and install Robin CNS on Google Anthos or GKE and integrate Google KMS. Proceed to Step 5 if you already have the installer downloaded. 1. Contact Robin Support team for Robin CNS software binaries. 2. After getting the Robin CNS from the support team, run the following command to untar the contents into a new directory. .. code-block:: text $ mkdir -p ./robin && tar -xf robin-install.tar -C ./robin 3. Move into the newly created directory, such that it is now your current working directory, and install Robin via the following commands: .. Note:: To customize the installation with more parameters, see the various installation parameters detailed `here `__. .. code-block:: text # cd robin # ./install-robin.sh --zone-name=us-central1=us-central1 --cluster-name=test-cluster-1 --kms-keyring=TESTKR --gcp-key-file=/home/kms/5314183/rock-range-2076xxxx3193a74e.json Validating Kubernetes cluster......Done Confirm that ports 29442-29470 are open between kubernetes nodes Make sure the cluster meets prerequisites mentioned at https://docs.robin.io/latest/install.html#minimum-node-requirements Type yes to confirm that cluster meets the prerequisites: yes Validating kubernetes cluster permissions......Done Validating robin cluster yaml......Done Installing Robin operator...........Done Installing Robin cluster............secret/cloud-cred-secret created Done Setting up robin client............Done Activate robin license............Required # robin license activate --userid Note: You can get your User ID after registering on https://get.robin.io. Above command will only work if the host on which the ROBIN client is running on has an internet connection. If this is not the case please retrieve the license key by using the below link and apply it using the command 'robin license apply '. https://get.robin.io/activate?clusterid= For logging into robin cluster, Default username: admin Default password: Robin123 4. This concludes the install. Continue `here `__ to verify the installation and set up the product. ================================= Install on an air-gapped Cluster ================================= An air-gapped Kubernetes cluster is a cluster that is behind a firewall without access to the Internet. You can install Robin on an air-gapped cluster by following the instructions provided in this topic. --------------- Prerequisites --------------- - All general prequisites detailed `here `__ must be met. - A driver node with Docker and the ``kubectl`` client installed. - The aforementioned driver node should also have access to the Internet for downloading the following: container images required to install Robin, Robin installation script, and Robin deployment YAML files. - A Kubernetes or OpenShift cluster already deployed on your choice of infrastructure. - A container registry with sufficient space for the Robin image and other images. ---------------------- Installation Steps ---------------------- 1. From the designated driver node, download the following Docker images: - `Robin Image `__ - `Robin Operator `__ Use the suggested ``docker pull`` commands to download each image respectively from the above links. .. Note:: From the **Tags** tab on the above links, you can copy specific commands for the required Robin CNS version and download the images. **Example** .. code-block:: text $ docker pull robinsys/robinimg:5.4.4-146 2. From the same driver node, download the following CSI and other images: - ``registry.k8s.io/sig-storage/csi-attacher:v4.3.0`` - ``registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.8.0`` - ``registry.k8s.io/sig-storage/csi-provisioner:v3.5.0`` - ``registry.k8s.io/sig-storage/csi-resizer:v1.8.0`` - ``registry.k8s.io/sig-storage/csi-snapshotter:v6.2.1`` - ``registry.k8s.io/sig-storage/csi-snapshotter:v6.2.2`` - ``registry.k8s.io/sig-storage/livenessprobe:v2.10.0`` - ``robineng/robin-image-puller:v1`` - ``robineng/robin-operator:5.4.4-146`` - ``ghcr.io/zalando/spilo-15:3.0-p1`` - ``ghcr.io/zalando/spilo-15:3.0-p1`` - ``ghcr.io/zalando/spilo-15:3.0-p1`` - ``registry.opensource.zalan.do/acid/postgres-operator:v1.10.0`` - ``robineng/robinimg:5.4.4-146`` .. Note:: Use the ``docker pull`` command to download all the images individually. **Example** .. code-block:: text $ docker pull registry.k8s.io/sig-storage/csi-attacher:v4.3.0 .. Note:: You must download all the above images. They are listed with the repository name and image version. 3. Push the Robin, CSI and other images from the driver node to a cloud container registry that is accessible by all nodes within the target Kubernetes cluster. You can use the ``docker push`` command to push the images. **Example** .. code-block:: text $ docker push artifactory.robinsystems.com/docker/robinsys/robinimg:5.4.4-146 $ docker push artifactory.robinsystems.com/docker/registry.k8s.io/sig-storage/csi-attacher:v4.3.0 .. Note:: Be sure to include the registry path, image name, and version tag when pushing the images to cloud container registry. In the above example, the registry is at ``artifactory.robinsystems.com`` 4. Access one of the nodes within the Kubernetes cluster that has access to the previously utilized container registry and create the ``robinio`` namespace as shown below: .. code-block:: text $ kubectl create ns robinio .. Note:: A namespace with the name ``robinio`` exactly must be created. 5. (Optional) If your cloud registry is only accessible using login credentials, then a secret needs to be created in the ``robinio`` namespace with the appropriate credentials as shown below: .. code-block:: text $ kubectl create secret docker-registry regcred --docker-server=artifactory.robinsystems.com --docker-username= --docker-password= -n robinio --docker-email=a@a.com .. Note:: The steps to create secret might differ depending on the cloud platform that you are using. 6. (Optional) If there already exists a secret that can be used to access cloud container registry in which all the necessary images are stored, copy it to the ``robinio`` namespace using the given commands: .. code-block:: text $ kubectl get secret --export -o yaml > $ kubectl create -f -n robinio 7. From the driver node with ``kubectl`` installed, access `get.robin.io `__ and follow the on-screen instructions to download the ``robin-install.tar`` file for the desired Robin version. Ensure the tarfile downloads successfully onto your driver node. 8. Copy the preciously download installation tarfile onto a node within your Kubernetes cluster. Next, create a new directory and extract the contents of the tar into it via the following command: .. code-block:: text $ mkdir -p ./robin && tar -xf robin-install.tar -C ./robin 9. Move into the newly created directory, such that it is now your current working directory, and install Robin via the following commands: .. Note:: To customize the installation with more parameters, see the various installation parameters detailed `here `__. .. code-block:: text # cd ./robin # ./install-robin.sh --version=5.4.4-146 --image-registry-path=us.icr.io/robin-5-4-4 -y --image-pull-secret=all-icr-io Validating Kubernetes cluster......Done Validating kubernetes cluster permissions......Done Validating robin cluster yaml......Done Installing Robin operator...........Done Installing Robin cluster............Done Setting up robin client............Done Activate robin license............Required # robin license activate --userid Note: You can get your User ID after registering on https://get.robin.io. Above command will only work if the host on which the ROBIN client is running on has an internet connection. If this is not the case please retrieve the license key by using the below link and apply it using the command 'robin license apply '. https://get.robin.io/activate?clusterid= For logging into robin cluster, Default username: admin Default password: Robin123 .. Note:: The command used in the final step of this process can have the options associated with specific environment type appended to it if needed. This is particularly the case when integrating with an external KMS. -------------------------------------------------- Verify Robin Installation on an air-gapped cluster -------------------------------------------------- Check if the Pods created in the ``robinio`` namespace are in the ``Running`` Status other than Patroni pre-install Pods by issuing the following command: .. code-block:: text root@eqx04-flash06:~# kubectl get pods -n robinio NAME READY STATUS RESTARTS AGE csi-attacher-robin-7568fc89cf-fwtst 3/3 Running 0 11h csi-nodeplugin-robin-5bnk2 3/3 Running 0 11h csi-nodeplugin-robin-d8f7p 3/3 Running 0 11h csi-nodeplugin-robin-hrwzj 3/3 Running 0 11h csi-nodeplugin-robin-kdvfc 3/3 Running 0 11h csi-nodeplugin-robin-vxhgn 3/3 Running 0 11h csi-provisioner-robin-67c96bbc5-pp6nt 3/3 Running 0 11h csi-resizer-robin-7c498f6665-kkkwt 3/3 Running 0 11h csi-snapshotter-robin-744ddb9d47-mt779 3/3 Running 0 11h robin-iomgr-429vm 1/1 Running 0 11h robin-iomgr-cwld2 1/1 Running 0 11h robin-iomgr-j4t66 1/1 Running 0 11h robin-iomgr-j7gqx 1/1 Running 0 11h robin-iomgr-jq5df 1/1 Running 0 11h robin-master-6f84c6fdb8-mcf9v 1/1 Running 0 11h robin-operator-6798bdcd85-rqltq 1/1 Running 0 11h robin-patroni-0 1/1 Running 0 11h robin-patroni-1 1/1 Running 0 11h robin-patroni-2 1/1 Running 0 11h robin-patroni-postgres-operator-c6f65cdf5-p2x54 1/1 Running 0 11h robin-patroni-pre-install-hook-j8576 0/1 Completed 0 11h robin-patroni-pre-install-hook-pbrzn 0/1 Completed 0 11h robin-patroni-pre-install-hook-wkrbd 0/1 Completed 0 11h robin-prejob-5d4p2 0/1 Completed 0 11h robin-worker-7blv6 1/1 Running 0 11h robin-worker-8dkpf 1/1 Running 0 11h robin-worker-nclcl 1/1 Running 0 11h robin-worker-smzzz 1/1 Running 0 11h robin-worker-vxxcp 1/1 Running 0 11h Additional verification steps alongside the one above can be found `here `__. ========================================================= Install with local PersistentVolume using Robin Installer ========================================================= Robin CNS supports disk and local PersistentVolume (PV) based discovery for storage. You can select the local PVs during Robin CNS installation. **Disk -** When you choose the disk-based discovery during Robin CNS installation, Robin discovers all disks on the host and starts formatting these disks for its use. **Local PersistentVolume (PV) -** When you choose local PV-based discovery during Robin CNS installation, Robin creates PVCs for the local PVs on the host and claims them for its use. Robin does not discover disks attached to the nodes but only uses local PV disks for its use. Local PV enables you to access local storage through the standard PVC interface. For more information about local PV, see `local PersistentVolume `__. --------------- Prerequisites --------------- The following prerequisites must be met to install Robin CNS with local PV: - All general prerequisites detailed `here `__ must be met. - `Local PVs `__ must be created in the cluster for Robin consumption. --------------------- Necessary Parameters --------------------- You must pass the following parameters with Robin installation command to use local PV-based discovery: - ``disk-discovery`` - Type of disk-discovery to be used by Robin CNS for storage. Valid values are ``disk`` and ``pv``. The default value is ``disk``. - ``pv-sc-name`` - Name of the StorageClass for local PV. - ``pv-folder-path`` - Path to the PersistentVolume (PV) folder created on all nodes which contain symlinks to the devices. - ``pv-labels`` - (optional) Label selector for selecting local PV to use by Robin CNS. If you provide any key-value pair then Robin claims and adds that specific local PV only. For example, ``robin.io/enabled:true``. If you do not provide any key-value pair then Robin claims and adds all local PVs for its use. ------------ How it works ------------ When you install Robin CNS, Robin creates PVCs for the local PVs and claims them to be used for Software Defined Storage. After creating PVCs, the status of PVs moves from the Available state to the Bound state so other applications cannot use these PVs. ------------------ Installation steps ------------------ Perform the following steps to install Robin CNS using the local PV: 1. Contact Robin Support team for Robin CNS software binaries. 2. After getting the tar file, run the following command to untar the contents into a new directory. .. code-block:: text # mkdir -p ./robin && tar -xf robin-install.tar -C ./robin 3. Move into the newly created directory, so that it is your current working directory and run the following command to install Robin CNS using the local PV: .. code-block:: text # cd robin # ./install-robin.sh --disk-discovery=pv --pv-folder-path=/mnt/mnt-disks --pv-labels=robin.io/enabled:true --pv-sc-name=mnt-disks Validating Kubernetes cluster......Done Validating kubernetes cluster permissions......Done Validating robin cluster yaml......Done Installing Robin operator...........Done Installing Robin cluster............Done Setting up robin client............Done Activate robin license............Required ***** ***** Your ROBIN License is NOT ACTIVATED... ***** Please register at https://get.robin.io/activate ***** to activate your license and access all ROBIN features. ***** $ robin license activate Note: You can get your User ID after registering on https://get.robin.io. Above command will only work if the host on which the ROBIN client is running on has an internet connection. If this is not the case please retrieve the license key by using the below link and apply it using the command 'robin license apply '. https://get.robin.io/activate?clusterid=XXXXXXXXXXXXXXXXXXXXXXXXXXXXXX.XXXXXXX.XXXXXXXXXXXXXXX.XX For logging into robin cluster, Default username: admin Default password: Robin123 You can access Robin UI at https://192.0.2.100:29443 4. After installing Robin CNS, `verify the installation `__ and set up your cluster. =================== Verify Installation =================== Once the installation is complete, verify the Robin cluster state and status by referring to the ``robincluster`` custom resource as shown beelow. After a successful installation, the state and status of each node within the cluster should be ``ONLINE`` and ``Ready`` respectively. .. code-block:: text $ kubectl describe robinclusters -n robinio Name: robin Namespace: Labels: app.kubernetes.io/instance=robin app.kubernetes.io/managed-by=robin.io app.kubernetes.io/name=robin Annotations: API Version: manage.robin.io/v1 Kind: RobinCluster Metadata: Creation Timestamp: 2023-08-22T16:04:18Z Generation: 1 Managed Fields: API Version: manage.robin.io/v1 Fields Type: FieldsV1 fieldsV1: f:metadata: f:annotations: .: f:kubectl.kubernetes.io/last-applied-configuration: f:labels: .: f:app.kubernetes.io/instance: f:app.kubernetes.io/managed-by: f:app.kubernetes.io/name: f:spec: .: f:host_type: f:image_nfs_server: f:image_pull_secret: f:image_robin: f:image_ubbagent: f:k8s_provider: f:namespace: f:options: .: f:skip_setup_iptables: f:source: Manager: kubectl-client-side-apply Operation: Update Time: 2023-08-22T16:04:18Z API Version: manage.robin.io/v1 Fields Type: FieldsV1 fieldsV1: f:status: .: f:connect_command: f:get_robin_client: f:master_ip: f:phase: f:pod_status: f:robin_node_status: Manager: robin-operator Operation: Update Subresource: status Time: 2023-08-23T07:02:34Z Resource Version: 29682064 UID: 687fdd2b-863c-43c1-9165-5860196eae45 Spec: host_type: physical image_nfs_server: robineng/nfs-server:v1 image_pull_secret: mgcred image_robin: robineng/robinimg:5.4.4-121 image_ubbagent: robineng/ubbagent:5.4.4-ent-1 k8s_provider: gke Namespace: robinio Options: skip_setup_iptables: 1 Source: spec Status: connect_command: kubectl exec -it robin-master-5b597c567d-5cvnr -n robinio -- bash get_robin_client: curl -k 'https://192.0.2.106:29442/api/v3/robin_server/download?file=robincli&os=linux' > robin master_ip: 10.9.140.106 Phase: Ready pod_status: robin-master-5b597c567d-5cvnr eqx03-flash06 Running 192.0.2.106 robin-worker-gzxkv eqx04-flash06 Running 192.0.2.106 robin-worker-26xhb eqx03-flash06 Running 192.0.2.106 robin-worker-jn4rh eqx03-flash07 Running 192.0.2.107 robin_node_status: current_version: 5.4.4-121 host_name: eqx04-flash06 join_time: 1692720588 k8s_node_name: eqx04-flash06 Roles: S Rpool: default State: ONLINE Status: Ready current_version: 5.4.4-121 host_name: eqx03-flash06 join_time: 1692720594 k8s_node_name: eqx03-flash06 Roles: S Rpool: default State: ONLINE Status: Ready current_version: 5.4.4-121 host_name: eqx03-flash07 join_time: 1692720597 k8s_node_name: eqx03-flash07 Roles: S Rpool: default State: ONLINE Status: Ready Events: .. Note:: If you have deployed Robin in a different namespace, then specify it instead of ``robinio`` in the above command. Use the **Connect Command** value from the above output to connect to Robin Pod. **Example** .. code-block:: text $ kubectl exec -it robin-master-5b597c567d-5cvnr -n robinio -- bash After logging into Robin container you can login as "admin" user with the default password of 'Robin123' and run Robin commands as shown below: .. code-block:: text $ robin login admin --password Robin123 $ robin host list In addition if a key management service (KMS) was specified as part of the installation run the below command to verify it is registered with Robin. .. code-block:: text # robin config list cluster kms Section | Attribute | Value --------+-----------+------------ cluster | kms | ibmprotect Lastly in order to ensure the Robin storage class is available and can be used for provisioning, issue the following command. .. code-block:: text $ kubectl get sc NAME PROVISIONER RECLAIMPOLICY VOLUMEBINDINGMODE ALLOWVOLUMEEXPANSION AGE local-storage kubernetes.io/no-provisioner Delete WaitForFirstConsumer false 14h robin (default) robin Delete WaitForFirstConsumer true 13h robin-immediate robin Delete Immediate true 13h robin-repl-3 robin Delete WaitForFirstConsumer true 13h robin-rwx robin Delete WaitForFirstConsumer true 13h ======================== ROBIN License Activation ======================== After installing ROBIN software on any platform, a notice will appear stating that the ROBIN license will expire in approximately two days whenever a command is run. This is an indication that the license is yet to be activated. In order to avoid a cluster lockdown you can activate your ROBIN cluster via one of the following methods. If the license is not activated your ROBIN cluster will be locked after the allocated time. ------------------------ Web License Activation ------------------------ Activate your ROBIN license by clicking `here `__. After following the instructions detailed on the website your ROBIN license should be activated and your cluster ready to use. .. Note:: You will be asked to provide the Robin CNS cluster id when activating your license. This can be discovered by running the ``robin license id`` command via the client or when you are connected to the Robin pod. In addition you can apply the generated key via the command ``robin license apply `` ------------------------ CLI License Activation ------------------------ In order to activate your license via the CLI, first register `here `__ and retrieve your User ID. Next run the following command: .. code-block:: text # robin license activate --userid ===================== ============================================================ ``--userid `` User ID generated after registering at https://get.robin.io ===================== ============================================================ This will activate your ROBIN license and ensure your cluster is ready to use. .. =================================== .. High Availability of ROBIN Services .. =================================== .. Robin CNS manages High Availability of all management services that are deployed as part of Robin CNS installation. Robin CNS pods are deployed as part of a daemonset. .. Some pods are designated to run master services. Robin CNS configures 3 of these pods as manager pods which can host these master services. .. * If one of the kubernetes node goes down, ROBIN seamlessly starts master services on other manager pods. .. * If pod hosting ROBIN master services is removed from kuberenetes cluster, ROBIN will automatically designate another pod as manager pod so that it always has 3 master pods. ============== Data Integrity ============== Checksum is a simple mathematical operation performed on a set of data to generate a fixed-size value that represents the integrity of the data. The generated checksum is often compared with a reference checksum to detect errors or inconsistencies that might have occurred during data transmission and storage. - Robin CNS does over-the-wire inline checksum validation for all read and write IOs to detect corruption that could happen during data transfers. - Robin calculates the checksum of every data block persisted on a storage device and stores it along with the metadata for the data block. In order to improve performance by minimizing disk IO, the Robin CNS storage stack pulls a batch of metadata blocks into memory at a time. While it reads the block's metadata from the disk, it doesn't store them in-memory. If the checksums were stored in-memory for the blocks, then the memory footprint for this metadata would increase. Also, modern flash media already performs checksums, and error recovery is automatically done at the FTL layer. Due to these reasons we have intentionally disabled inline checksum validation during the read operations. - Data blocks can be validated for checksum correctness offline by a utility called devck (currently available via Robin Customer Support). A Volume rebuild command can rebuild corrupted blocks from healthier replica copies in the event of corruption. ================== Uninstalling ROBIN ================== -------------------------------------------------- Deleting objects created via the Robin installer -------------------------------------------------- If the Robin CNS operator was deployed on an existing Kubernetes cluster via the Robin installer in the steps described previously, one can run the following command to uninstall: .. code-block:: text $ ./install-robin.sh --uninstall Uninstalling Robin cluster......Done Some Robin managed namespaces (t001-u000001 t001-u000002 t001-u000003 ) are not deleted, please run following command to delete those after making sure there are no required kubernetes objects present in these namespaces "kubectl delete ns -l robin.io/domain=ROBIN" .. Note:: The ``install-robin.sh`` script is shipped within the downloadable tarfile. -------------------------------------------------- Deleting objects created via Kubernetes -------------------------------------------------- Cleanup for automated installations ------------------------------------ For uninstalling Robin CNS from Kubernetes cluster via ``kubectl``, run the following commands: .. code-block:: text $ kubectl delete -f robin.yaml $ kubectl delete clusterrolebinding -l robin.io/domain=ROBIN $ kubectl delete rolebinding -l robin.io/domain=ROBIN -A Run the below command to get a list of namespaces created with Robin and delete the namespaces one by one. Please make sure you are not accidentally deleting namespaces that contain non-robin applications. .. code-block:: text $ kubectl get ns -l robin.io/domain=ROBIN After reviewing the list, if you want to delete all the namespaces at once, you can run .. code-block:: text $ kubectl delete ns -l robin.io/domain=ROBIN Cleanup for manual installations ------------------------------------ .. Note:: The below instructions for cleanup only apply to Anthos/GKE clusters where Robin CNS was setup using the commands detailed `here `__ and NOT via the Robin installer. In order to uninstall Robin CNS from a Anthos/GKE Kubernetes cluster via ``kubectl``, run the following commands: .. code-block:: text $ export APP_INSTANCE_NAME=robin $ export NAMESPACE=robinio $ kubectl delete -f ${APP_INSTANCE_NAME}_manifest.yaml --namespace "${NAMESPACE}" $ kubectl delete serviceaccount "${APP_INSTANCE_NAME}-deployer-sa" -n "${NAMESPACE}" $ kubectl delete clusterrolebinding "$APP_INSTANCE_NAME:$NAMESPACE:deployer-crb0" $ kubectl delete clusterrole "$APP_INSTANCE_NAME:$NAMESPACE:deployer-cr0" $ kubectl delete application,deployment --namespace ${NAMESPACE} --selector app.kubernetes.io/name=$APP_INSTANCE_NAME $ kubectl delete -f license.yaml -n ${NAMESPACE} $ kubectl delete ns robinio ------------------------------- Deleting objects created by OLM ------------------------------- If the ROBIN operator was deployed on an OpenShift cluster via the OLM method described above, run the following command to delete all the objects created within the ``robinio`` namespace: .. code-block:: text $ oc delete namespace robinio .. Note:: The commands starting with ``oc`` are applicable for the OpenShift environment. -------------------------------------------------- Deleting objects created via GCP UI -------------------------------------------------- .. Note:: The below instructions for cleanup only apply to GKE clusters where Robin CNS was setup using the steps detailed `here `__. In order to uninstall Robin CNS from a GKE Kubernetes cluster via the Google Cloud Platform UI navigate to the Application details page, select the instance of Robin CNS to be deleted, and click the delete action at the top of the screen. You should see a screen similar to the one below. .. figure:: ./images/Delete-GKE-UI.png :width: 700 After confirming the deletion of the application, run the following command on the GKE cluster: .. code-block:: text $ export APP_INSTANCE_NAME=robin $ kubectl delete robincluster ${APP_INSTANCE_NAME}