5. Managing Storage

Robin discovers disks attached to the nodes and uses them for providing storage to applications. This applies to local disks, cloud volumes, and SAN stoage that are available to nodes which have the storage role assigned to them. During the process Robin collects all the required metadata about the disks and identifies them as HDD, SSD or NVMes. All the elligible disks (this excludes disks with parititions and/or without WWNs) are marked as Robin storage.

Cloud volumes like EBS volumes on AWS and Persistent disks on GCP are not physically tied to a cloud compute instance. They can be detached from one cloud compute instance and attached to another instance at any time. Logical Units from a Storage Array Network can similarly be visible from multiple physical servers in the same storage fabric or network. Robin assigns a primary host for these devices and ensures that all IO access to a device only happens through one server at a given time. As with local disks, these devices are automatically discovered by Robin and registered. However they are considered to be re-attachable given their properties. Robin makes sure these re-attachable disks are always accessible even in the event of node failure. When a node goes down, Robin detaches the disk from failed node and attaches it to a healthy node so that application data remains accessible.

Note

Whilst installing if it is known that certain disks will not have a WWN, the --set-uuid option can be provided via the installer alongside a list of disks on which there is no WWN. This in turn will result in Robin stamping a UUID on the disk and thus enabling it to be used as Robin storage.

Robin allows multiple operations, detailed below, to be performed on registered disks.

robin disk attach

Attach an existing disk

robin disk detach

Detach a registered disk

robin disk create

Provision and attach a cloud disk

robin disk list

List disks

robin disk info

Display detailed information about a disk

robin disk evacuate

Evacuate allocations off a disk

robin disk unregister

Unregister a disk

robin disk remove

Delete a disk

robin disk update

Update the attributes of a disk

robin disk unfault

Unfault a disk

5.1. Attaching a disk

If a disk was previously detached from a host due to the host undergoing decommissioning or to rebalance the storage available on the cluster one will need to attach the disk back to another host in the cluster. Issue the following command to do so:

Note

This command is only for supported for re-attachable disks. Moreover if a host became unreachable Robin will automatically detach the disk and choose a new server to reattach it to based on the accessibility of the disk and the load on the other servers.

# robin disk attach <wwn>
                    --hostname <hostname>
                    --force
                    --mount-path <mount_path>

wwn

WWN of disk to attach

--hostname <hostname>

Name of host to attach disk to. Note this is a mandatory parameter.

--force

Forcibly re-attach a disk that is already ONLINE

--mount-path <mount_path>

Specify the mount path of the disk to be attached. Note this option is only valid for AWS disks and the specified path must start with ‘/dev/sd’.

Example:

# robin disk detach 0xQEMU_QEMU_HARDDISK_3c71c872-fe13-4fd5-b --wait
Job:  187 Name: DiskAttach     State: VALIDATED       Error: 0
Job:  187 Name: DiskAttach     State: WAITING         Error: 0
Job:  187 Name: DiskAttach     State: FINALIZED       Error: 0
Job:  187 Name: DiskAttach     State: COMPLETED       Error: 0

Attaches a disk, which might have previously been detached due to the host undergoing decommissioning or a rebalancing of storage disks, to a particular host.

End Point: /api/v3/robin_server/disks/<disk_wwn>

Method: PUT

URL Parameters: None

Data Parameters:

  • action: attach - This mandatory field within the payload specifies that the attach operation is to be performed.

  • hostname: <hostname> - Utilizing this parameter results in the disk being attached to the specified host.

  • force: true - Utilizing this parameter enables one to force re-attach an ONLINE disk.

  • mount_path: <mount_path> - Utilizing this parameter results in the given disk being mounted at the path specified. Note this option is only valid for AWS disks and the specified path must start with ‘/dev/sd’.

Port: RCM Port (default value is 29442)

Headers:

  • Authorization: <auth_token> : Authorization token to identify which user is sending the request. The token can be acquired from the login API.

Success Response Code: 202

Error Response Code: 500 (Internal Server Error), 404 (Not Found Error), 401 (Unauthorized Error), 400 (Invalid API Usage Error)

Example Response:

Output
{
   "jobid": 128
}

5.2. Detaching a disk

If a physical host is temporarily or permanently taken down and one can detach its storage disks to ensure future access. Issue the following command to do so:

Note

This command is only for supported for re-attachable disks.

# robin disk detach <wwn>
                    --hostname <hostname>
                    --force

wwn

WWN of disk to detach

--hostname <hostname>

Name of host to which disk is attached to. Note this is an optional parameter

--force

Forcibly detach a disk that is currently ONLINE

Example:

# robin disk detach 0xQEMU_QEMU_HARDDISK_3c71c872-fe13-4fd5-b --wait
Job:  187 Name: DiskDetach     State: VALIDATED       Error: 0
Job:  187 Name: DiskDetach     State: WAITING         Error: 0
Job:  187 Name: DiskDetach     State: FINALIZED       Error: 0
Job:  187 Name: DiskDetach     State: COMPLETED       Error: 0

Detaches a disk from a host.

End Point: /api/v3/robin_server/disks/<disk_wwn>

Method: PUT

URL Parameters: None

Data Parameters:

  • action: detach - This mandatory field within the payload specifies that the detach operation is to be performed.

  • hostname: <hostname> - Utilizing this parameter results in the disk being detached from the specified host. This is optional as the host to which the disk is attached can be discovered implicitly.

  • force: true - Utilizing this parameter enables one to force detach an ONLINE disk.

Port: RCM Port (default value is 29442)

Headers:

  • Authorization: <auth_token> : Authorization token to identify which user is sending the request. The token can be acquired from the login API.

Success Response Code: 202

Error Response Code: 500 (Internal Server Error), 404 (Not Found Error), 401 (Unauthorized Error), 400 (Invalid API Usage Error)

Example Response:

Output
{
   "jobid": 130
}

5.3. Provisioning a disk

Robin provides a utility through which it can provision disks of any size, attach them to hosts and discover them automatically on multiple cloud platforms. This enables users to expand the storage available for their cluster with convenience and ease. Detailed below are the general options for the robin drive create command, followed by specific examples for each supported cloud environment.

# robin disk create <hostname>
                    --type <type>
                    --number <number>
                    --size <size>
                    --iops <iops>
                    --mount-path <mount_path>

hostname

Name of host to attach disk to after creation

--type <type>

Type of disks. Choices for GCP include: pd-ssd, pd-standard. Choices for AWS include: gp2, io1, st1. Choices for Anthos include: independent-persistent. Choices for Azure include: premium, standard.

--number <number>

Number of disks to be created. The default value is 1

--size <size>

Size of disk to be created in GB. The default value is 500 GB

--iops <iops>

IOPs for AWS ‘io1’ disk type. Can be between 100-160000

--mount-path <mount_path>

Specify the mount path of the disk to be created. Note this option is only valid for AWS disks and the specified path must start with ‘/dev/sd’.

Provisions disks of any size, attaches and discovers them automatically on cloud based nodes.

End Point: /api/v3/robin_server/disks/

Method: POST

URL Parameters: None

Data Parameters:

  • hostname: <hostname> - This mandatory field within the payload specifies the host to which the provisioned disk should be attached.

  • type: <disk_type> - This mandatory field within the payload specifies the type of disk to be created. Supported types include: pd-ssd, pd-standard (for GCP); gp2, io1, st1 (for AWS); independent-persistent (for Anthos).

  • number: <num_of_disks> - This mandatory field within the payload specifies the number of disks to be created. It should be an integer value.

  • size: <size_of_disk> - This mandatory field within the payload specifies the size of the disks to be created. It should be a string. An example value is ‘200GB’.

  • iops: <iops_of_disk> - Utilizing this parameter results in disks that can handle a maximum number of IOPs equal to <iops_of_disks> being created. Note this parameter is only valid for the ‘io1’ disk type for AWS nodes.

  • mount_path: <mount_path> - Utilizing this parameter results in disk(s) created being mounted at the paths specified. Note this option is only valid for AWS disks and the specified path must start with ‘/dev/sd’.

Port: RCM Port (default value is 29442)

Headers:

  • Authorization: <auth_token> : Authorization token to identify which user is sending the request. The token can be acquired from the login API.

Success Response Code: 202

Error Response Code: 500 (Internal Server Error), 404 (Not Found Error), 401 (Unauthorized Error), 400 (Invalid API Usage Error)

Example Response:

Output
{
   "jobid": 196
}

5.3.1. Disk Provisioning On GCP

On Google Cloud Platform, you can attach disks to your instance via the UI or Google APIs and have them discovered by Robin by running the below command:

$ robin host probe <hostname> --rediscover

In order to verify the disk has been properly detected run the following command:

# robin disk list

Finally in order to format and initialize the disk such that it is available for use by Robin, issue the following command:

# robin host add-role <hostname> storage

Note

The hostname parameter in the example commands above refer the hostname of the node where the disk is attached.

On the other hand you can utilize Robin to provision disks in GCP to use for application deployment. To create 100 GB disk in GCP, run following command:

$ robin disk create <hostname> --type <pd-standard | pd-ssd> --size 100

These disks will be attached automatically and auto discovered by Robin so they will be ready to use straightaway.

Note

Due to Robin’s advanced feature to make sure disks are always accessible, it needs the manage disks permission to be selected while deploying cluster on GCP.

5.3.2. Disk Provisioning On Google Anthos

On the Google Anthos platform, you can add disks to cluster VMs from vSphere and have them discovered by Robin by running the below command:

$ robin host probe <hostname> --rediscover

In order to verify the disk has been properly detected run the following command:

# robin disk list

Finally in order to format and initialize the disk such that it is available for use by Robin, issue the following command:

# robin host add-role <hostname> storage

Note

The hostname parameter in the example commands above refer the hostname of the node where the disk is attached.

On the other hand you can utilize Robin to provision virtual disks to use for application deployment. To create 100 GB disk for Anthos, run following command:

$ robin disk create <hostname> --type independent-persistent --size 100

These disks will be attached automatically and auto discovered by Robin so they will be ready to use straightaway.

Note

Due to Robin’s advanced feature to make sure disks are always accessible, it needs credentials, provided via Kubernetes secret, to have all cluster and disk level API privileges.

5.3.3. Disk Provisioning On AWS

On AWS, you can attach disks to your EC2 instance via the UI or AWS CLI/APIs and have them discovered by Robin by running the below command:

$ robin host probe <hostname> --rediscover

In order to verify the disk has been properly detected run the following command:

# robin disk list

Finally in order to format and initialize the disk such that it is available for use by Robin, issue the following command:

# robin host add-role <hostname> storage

Note

The hostname parameter in the example commands above refer the hostname of the node where the disk is attached.

On the other hand you can utilize Robin to provision disks in AWS to use for application deployment. To create 100 GB disk in AWS, run following command:

$ robin disk create <hostname> --type <gp2 | io1 | st1> --size 100

These disks will be attached automatically to the EC2 instances and auto discovered by Robin so they will be ready to use straightaway.

Note

Due to Robin’s advanced feature to make sure disks are always accessible, IAM Profiles associated with the host (or permissions granted to a user) must contain all Volume write and list actions.

5.3.4. Disk Provisioning On Azure

On Azure, you can attach disks to your Azure Virtual Machines via the UI or Azure CLI/APIs and have them discovered by Robin by running the below command:

$ robin host probe <hostname> --rediscover

In order to verify the disk has been properly detected run the following command:

# robin disk list

Finally in order to format and initialize the disk such that it is available for use by Robin, issue the following command:

# robin host add-role <hostname> storage

Note

The hostname parameter in the example commands above refer the hostname of the node where the disk is attached.

On the other hand you can utilize Robin to provision disks in Azure to use for application deployment. To create 100 GB disk in Azure, run following command:

$ robin disk create <hostname> --type <premium | standard> --size 100

These disks will be attached automatically to the Azure Virtual Machines and auto discovered by Robin so they will be ready to use straightaway.

Note

Due to Robin’s advanced feature to make sure disks are always accessible, the Azure role(s) assigned to the user, whom the deployed virtual machines are associated with, must allow for management of data disks.

5.4. Listing all disks

In order to view all disks currently present on the cluster and some additional details such as the number of volumes allocated from each, their size, media type etc, issue the following command:

# robin disk list --host <hostname>
                  --role <role>
                  --media <media>
                  --reattachable
                  --eligible
                  --tags
                  --json

--host <hostname>

Filter list by hostname

--role <role>

Filter list by role. Valid choices include: all, storage, rootdisk and reserved

--media <media>

Filter list by media type. Valid choices include: HDD and SSD

--reattachable

Filter list to only display reattachable disks

--eligible

Filter list to only display disks that have free capacity and have not reached the maximum volume count

--tags

Display tags for each disk

--json

Output in JSON

Example:

# robin disk list
ID | WWN                                       | Host    | RPool   | Path /dev/disk/by-id                          | Size(GB) | Movable | Type | Free/Max(GB) | Vols | Role    | Status | LastOpr | Write Unit
---+-------------------------------------------+---------+---------+--------------------------------------+----------+---------+------+--------------+------+---------+--------+--------+---------+------------
1  | 0xQEMU_QEMU_HARDDISK_3c71c872-fe13-4fd5-b | vnode36 | default | scsi-0QEMU_QEMU_HARDDISK_3c71c872-fe13-4fd5-b | 50       | N       | HDD  | 38/38 (100%) | 0/10 | Storage | ONLINE | READY   | 512
2  | 0xQEMU_QEMU_HARDDISK_f12b1f33-8b71-4a8c-a | vnode36 | default | scsi-0QEMU_QEMU_HARDDISK_f12b1f33-8b71-4a8c-a | 50       | N       | HDD  | 28/38 (74%)  | 1/10 | Storage | ONLINE | READY   | 512
3  | 0xQEMU_QEMU_HARDDISK_e54d6149-0a4e-48ce-b | vnode88 | default | scsi-0QEMU_QEMU_HARDDISK_e54d6149-0a4e-48ce-b | 100      | N       | HDD  | 77/77 (100%) | 0/10 | Storage | ONLINE | READY   | 4096
4  | 0xQEMU_QEMU_HARDDISK_89fc0488-2050-4f44-a | vnode88 | default | scsi-0QEMU_QEMU_HARDDISK_89fc0488-2050-4f44-a | 100      | N       | HDD  | 63/77 (82%)  | 2/10 | Storage | ONLINE | READY   | 4096
5  | 0xQEMU_QEMU_HARDDISK_19f9ac67-5e7e-4f00-8 | vnode89 | default | scsi-0QEMU_QEMU_HARDDISK_19f9ac67-5e7e-4f00-8 | 100      | N       | HDD  | 77/77 (100%) | 0/10 | Storage | ONLINE | READY   | 4096
6  | 0xQEMU_QEMU_HARDDISK_d523b7f2-eba7-4edc-b | vnode89 | default | scsi-0QEMU_QEMU_HARDDISK_d523b7f2-eba7-4edc-b | 100      | N       | HDD  | 77/77 (100%) | 0/10 | Storage | ONLINE | READY   | 4096

Only storage disks are shown. Issue `robin disk list --role all` to view all disks

Returns all disks currently present on the cluster and some additional details such as the number of volumes allocated from each, their size, and media type.

End Point: /api/v5/robin_server/disks

Method: GET

URL Parameters:

  • details=tags : Utilizing this parameter results in tag information for each disk being present in the response payload.

  • host=<hostname> : Utilizing this parameter results in only disks attached to the specified host being returned.

Data Parameters: None

Port: RCM Port (default value is 29442)

Headers:

  • Authorization: <auth_token> : Authorization token to identify which user is sending the request. The token can be acquired from the login API.

Success Response Code: 200

Error Response Code: 500 (Internal Server Error)

Example Response:

Output
{
   "items":[
      {
         "node_hostname":"cscale-82-140.robinsystems.com",
         "spf":0.8,
         "state":"READY",
         "type":"HDD",
         "nvols":3,
         "dev":"\/dev\/sdb",
         "aslices":7,
         "stormgrid":1,
         "pused":234881024,
         "nodeid":1,
         "maintenance_mode":"DISABLED",
         "availability_zone":null,
         "max_alloc_slices":77,
         "role":"Storage",
         "wwn":"0x600224804c48fd7e16c608dea0919064",
         "status":"ONLINE",
         "free_alloc_slices":68,
         "lused_size":0,
         "devpath":"\/dev\/disk\/by-id\/scsi-3600224804c48fd7e16c608dea0919064",
         "alloc_slices":9,
         "reattachable":0,
         "max_volumes_per_disk":10,
         "protected":0,
         "slices":6390,
         "capacity":107374182400,
         "pfree":104287174656
      },
      {
         "node_hostname":"cscale-82-140.robinsystems.com",
         "spf":0.8,
         "state":"READY",
         "type":"HDD",
         "nvols":1,
         "dev":"\/dev\/sdc",
         "aslices":20,
         "stormgrid":2,
         "pused":939524096,
         "nodeid":1,
         "maintenance_mode":"DISABLED",
         "availability_zone":null,
         "max_alloc_slices":77,
         "role":"Storage",
         "wwn":"0x600224803bcdafde95b1f5cd27ceb5fb",
         "status":"ONLINE",
         "free_alloc_slices":53,
         "lused_size":0,
         "devpath":"\/dev\/disk\/by-id\/scsi-3600224803bcdafde95b1f5cd27ceb5fb",
         "alloc_slices":24,
         "reattachable":0,
         "max_volumes_per_disk":10,
         "protected":0,
         "slices":6390,
         "capacity":107374182400,
         "pfree":103582531584
      },
      {
         "node_hostname":"cscale-82-140.robinsystems.com",
         "spf":0.8,
         "state":"INIT",
         "type":"HDD",
         "nvols":0,
         "dev":"\/dev\/dm-1",
         "aslices":0,
         "stormgrid":0,
         "pused":0,
         "nodeid":1,
         "maintenance_mode":"DISABLED",
         "availability_zone":null,
         "max_alloc_slices":5,
         "role":"RootDisk",
         "wwn":"0x600224801d3ac9b6650afd3280aa5898-centos-swap",
         "status":"UNKNOWN",
         "free_alloc_slices":5,
         "lused_size":0,
         "devpath":"\/dev\/disk\/by-id\/dm-uuid-LVM-vI83PDTxV3H0dWyAXfH5ef7rxTOuYyphaFy4aq3EUo1yluonS8FG0LF16ycBrdEw",
         "alloc_slices":0,
         "reattachable":0,
         "max_volumes_per_disk":10,
         "protected":0,
         "slices":0,
         "capacity":8254390272,
         "pfree":0
      },
      {
         "node_hostname":"cscale-82-140.robinsystems.com",
         "spf":0.8,
         "state":"INIT",
         "type":"HDD",
         "nvols":0,
         "dev":"\/dev\/dm-0",
         "aslices":0,
         "stormgrid":0,
         "pused":0,
         "nodeid":1,
         "maintenance_mode":"DISABLED",
         "availability_zone":null,
         "max_alloc_slices":38,
         "role":"RootDisk",
         "wwn":"0x600224801d3ac9b6650afd3280aa5898-centos-root",
         "status":"UNKNOWN",
         "free_alloc_slices":38,
         "lused_size":0,
         "devpath":"\/dev\/disk\/by-id\/dm-uuid-LVM-vI83PDTxV3H0dWyAXfH5ef7rxTOuYyphgpZcvqGdfOKaXbEbOZzNthc6btsoSXDj",
         "alloc_slices":0,
         "reattachable":0,
         "max_volumes_per_disk":10,
         "protected":0,
         "slices":0,
         "capacity":53687091200,
         "pfree":0
      },
      {
         "node_hostname":"cscale-82-140.robinsystems.com",
         "spf":0.8,
         "state":"INIT",
         "type":"HDD",
         "nvols":0,
         "dev":"\/dev\/dm-2",
         "aslices":0,
         "stormgrid":0,
         "pused":0,
         "nodeid":1,
         "maintenance_mode":"DISABLED",
         "availability_zone":null,
         "max_alloc_slices":32,
         "role":"RootDisk",
         "wwn":"0x600224801d3ac9b6650afd3280aa5898-centos-home",
         "status":"UNKNOWN",
         "free_alloc_slices":32,
         "lused_size":0,
         "devpath":"\/dev\/disk\/by-id\/dm-uuid-LVM-vI83PDTxV3H0dWyAXfH5ef7rxTOuYyphQObDlS6eMUSpSxH5zsvyg9I5a0Gpuj5W",
         "alloc_slices":0,
         "reattachable":0,
         "max_volumes_per_disk":10,
         "protected":0,
         "slices":0,
         "capacity":44350570496,
         "pfree":0
      },
      {
         "node_hostname":"cscale-82-140.robinsystems.com",
         "spf":0.8,
         "state":"INIT",
         "type":"HDD",
         "nvols":0,
         "dev":"\/dev\/sda",
         "aslices":0,
         "stormgrid":0,
         "pused":0,
         "nodeid":1,
         "maintenance_mode":"DISABLED",
         "availability_zone":null,
         "max_alloc_slices":77,
         "role":"RootDisk",
         "wwn":"0x600224801d3ac9b6650afd3280aa5898",
         "status":"UNKNOWN",
         "free_alloc_slices":77,
         "lused_size":0,
         "devpath":"\/dev\/disk\/by-id\/scsi-3600224801d3ac9b6650afd3280aa5898",
         "alloc_slices":0,
         "reattachable":0,
         "max_volumes_per_disk":10,
         "protected":0,
         "slices":0,
         "capacity":107374182400,
         "pfree":0
      }
   ]
}

5.5. Show information about a specific disk

In order to view detailed information about a disk such as the breakdown of its available capacity, current allocations including the associated applications, write unit etc, issue the following command:

# robin disk info <wwn>
                  --json

wwn

WWN of disk to detach

--json

Output in JSON

Example:

# robin disk info 0xQEMU_QEMU_HARDDISK_80cb8b56-aef2-48a8-8
Drive: 0xQEMU_QEMU_HARDDISK_80cb8b56-aef2-48a8-8
  Id: 2
  Role: Storage
  Type: HDD
  Make: None
  Model: None
  Write Unit: 512
  Availability Zone: None
  Zone Id: 1613675775
  Path: /dev/disk/by-id/scsi-0QEMU_QEMU_HARDDISK_80cb8b56-aef2-48a8-8
  Node: vnode-95-28.robinsystems.com
  Protected: No
  Reattachable: No
  State: READY
  Status: MAINTENANCE
  Maintenance: ON
  Allocations:
    Current: 15G
    Maximum: 38G
    Free: 23G
    Capacity: 50G
    Factor: 30%
    AllocScore: 83
    Physical Usage: 0.16G
    Physical Free: 48G
  Volumes: 3
    Max volumes: 10
    Max latency volumes: 2
    Max throughput volumes: 1
  Applications: 3
    file-collection-1613696655681
      Instance: file-collection-1613696655681.b9307d9d-f1fe-4731-a815-99800f3811a2

          Id      | Volume Name                                                        | Workload | Size (GB)      | Allocated (GB)
          --------+--------------------------------------------------------------------+----------+----------------+----------------
          1       | file-collection-1613696655681.b9307d9d-f1fe-4731-a815-99800f3811a2 | ordinary | 5              | 5


    pvc-9369013d-cba4-41a9-b9b8-15228e5ea09d
      Instance: pvc-9369013d-cba4-41a9-b9b8-15228e5ea09d

          Id       | Volume Name                              | Workload | Size (GB)      | Allocated (GB)
          ---------+------------------------------------------+----------+----------------+----------------
          21       | pvc-9369013d-cba4-41a9-b9b8-15228e5ea09d | ordinary | 5              | 5

Returns detailed information about a disk such as the breakdown of its available capacity, current allocations including the associated applications and write unit.

End Point: /api/v3/robin_server/disks/<wwn>

Method: GET

URL Parameters: None

Data Parameters: None

Port: RCM Port (default value is 29442)

Headers:

  • Authorization: <auth_token> : Authorization token to identify which user is sending the request. The token can be acquired from the login API.

Success Response Code: 200

Error Response Code: 500 (Internal Server Error), 404 (Not Found Error), 401 (Unauthorized Error)

Example Response:

Output
{
   "items":[
      {
         "wwn":"0xQEMU_QEMU_HARDDISK_80cb8b56-aef2-48a8-8",
         "dev":"\/dev\/sdb",
         "make":null,
         "model":null,
         "devpath":"\/dev\/disk\/by-id\/scsi-0QEMU_QEMU_HARDDISK_80cb8b56-aef2-48a8-8",
         "capacity":53687091200,
         "type":"HDD",
         "role":"Storage",
         "state":"READY",
         "status":"MAINTENANCE",
         "maintenance_mode":"ON",
         "alloc_score":83,
         "slices":3190,
         "aslices":15,
         "pused":167772160,
         "pfree":51774488576,
         "nodeid":1,
         "stormgrid":2,
         "reattachable":0,
         "protected":0,
         "availability_zone":null,
         "node_hostname":"vnode-95-28.robinsystems.com",
         "reattachable_nodes":[
            [
               "vnode-95-28.robinsystems.com",
               "ONLINE"
            ]
         ],
         "reattachpolicy":{
            "id":2,
            "burst_start_time":0,
            "burst_count":0,
            "burst_interval":600,
            "restart_limit":5,
            "restarts_done":0
         },
         "zoneid":1613675775,
         "allocations":[
            {
               "name":"file-collection-1613696655681.b9307d9d-f1fe-4731-a815-99800f3811a2.0.e4623953-263e-4b91-b66b-6be69ab60018",
               "slices":5,
               "volume_group":"file-collection-1613696655681.b9307d9d-f1fe-4731-a815-99800f3811a2.72.1.d9d9726c-8052-48b1-b7ee-b953dbe254ff",
               "vols":[
                  {
                     "id":"1",
                     "name":"file-collection-1613696655681.b9307d9d-f1fe-4731-a815-99800f3811a2",
                     "size":5368709120,
                     "state":"ONLINE",
                     "media":"HDD",
                     "workload_str":"ordinary"
                  }
               ],
               "volume":{
                  "id":"1",
                  "name":"file-collection-1613696655681.b9307d9d-f1fe-4731-a815-99800f3811a2",
                  "size":5368709120,
                  "state":"ONLINE",
                  "media":"HDD",
                  "workload_str":"ordinary"
               }
            },
            {
               "name":"pvc-9369013d-cba4-41a9-b9b8-15228e5ea09d.0",
               "slices":5,
               "volume_group":"pvc-9369013d-cba4-41a9-b9b8-15228e5ea09d.72.1",
               "vols":[
                  {
                     "id":"21",
                     "name":"pvc-9369013d-cba4-41a9-b9b8-15228e5ea09d",
                     "size":5368709120,
                     "state":"ONLINE",
                     "media":"HDD",
                     "workload_str":"ordinary"
                  }
               ],
               "volume":{
                  "id":"21",
                  "name":"pvc-9369013d-cba4-41a9-b9b8-15228e5ea09d",
                  "size":5368709120,
                  "state":"ONLINE",
                  "media":"HDD",
                  "workload_str":"ordinary"
               }
            },
            {
               "name":"pvc-71561296-4699-4b79-bf92-3dd5470929cb.0",
               "slices":5,
               "volume_group":"pvc-71561296-4699-4b79-bf92-3dd5470929cb.72.1",
               "vols":[
                  {
                     "id":"22",
                     "name":"pvc-71561296-4699-4b79-bf92-3dd5470929cb",
                     "size":5368709120,
                     "state":"ONLINE",
                     "media":"HDD",
                     "workload_str":"throughput"
                  }
               ],
               "volume":{
                  "id":"22",
                  "name":"pvc-71561296-4699-4b79-bf92-3dd5470929cb",
                  "size":5368709120,
                  "state":"ONLINE",
                  "media":"HDD",
                  "workload_str":"throughput"
               }
            }
         ],
         "tags":{

         },
         "max_volumes_per_disk":10,
         "max_latency_sensitive_vols_per_disk":2,
         "max_throughput_intensive_vols_per_disk":1,
         "nvols":3,
         "lused_size":0,
         "preserved":0,
         "write_unit":512
      }
   ]
}

5.6. Evacuating volumes from a disk

This command allows users to move Robin storage volumes from one disk to another. As a result, it can be utilized to free up space on a disk when it is getting full, or to move data out of a disk before it is decommissioned. To evacuate a volume, issue the following command:

Note

Robin’s placement algorithm will determine the best disks to evacuate volumes to if target disks are not specified.

# robin disk evacuate <wwn>
                      --volume <volume>
                      --to-disks <target_disks>
                      --exclude-disks <excluded_disks>

wwn

WWN of disk to evacuate

--volume <volume>

Name of volume to be evacuated. Note if not provided all the volumes will be evacuated

--to-disks <target_disks>

List of WWNs representing disks to evacuate to

--exclude-disks <excluded_disks>

List of WWNs representing disks to avoid evacuating to

--yes

Do not prompt the user for confirmation of evacuation

Example:

# robin disk evacuate 0xQEMU_QEMU_HARDDISK_3c71c872-fe13-4fd5-b --wait --yes
Job:   65 Name: DiskEvacuate         State: VALIDATED       Error: 0
Job:   65 Name: DiskEvacuate         State: WAITING         Error: 0
Job:   65 Name: DiskEvacuate         State: COMPLETED       Error: 0

Evacuates volumes residing on one disk to another.

End Point: /api/v3/robin_server/disks/<disk_wwn>

Method: PUT

URL Parameters: None

Data Parameters:

  • action: evacuate - This mandatory field within the payload specifies that the evacuate operation is to be performed.

  • volume: <volume_name> - Utilizing this parameter results in only the specified volume being evacuated from the disk. If this is not specified all volumes on the disk are evacuated.

  • target_drives: <list_of_target_drives> - Utilizing this parameter, by specifying a list of WWNs representing respective target drives, ensures the evacuated volumes will be transferred to one of the specified disks.

  • exclude_drives: <list_of_excluded_drives> - Utilizing this parameter, by specifying a list of WWNs representing respective excluded drives, ensures the evacuated volumes will not transferred to one of the specified disks.

Port: RCM Port (default value is 29442)

Headers:

  • Authorization: <auth_token> : Authorization token to identify which user is sending the request. The token can be acquired from the login API.

Success Response Code: 202

Error Response Code: 500 (Internal Server Error), 404 (Not Found Error), 401 (Unauthorized Error), 400 (Invalid API Usage Error)

Example Response:

Output
{
   "jobid": 145
}

5.7. Unregistering a disk

In order to unregister disk from the Robin cluster due to the fact it has physically malfunctioned or it is faulted, issue the following command:

Note

A disk can only be unregistered if there are no storage volumes allocated from it and it is still attached to the node from the perspective of the platform.

# robin disk unregister <wwn>
                        --yes

wwn

WWN of disk to unregister

--yes

Do not prompt the user for confirmation of disk unregistration

Example:

# robin disk unregister 0xQEMU_QEMU_HARDDISK_3c71c872-fe13-4fd5-b --wait --yes
Job:   101 Name: DiskUnregister       State: VALIDATED       Error: 0
Job:   101 Name: DiskUnregister       State: WAITING         Error: 0
Job:   101 Name: DiskUnregister       State: COMPLETED       Error: 0

Unregisters a disk from the Robin cluster.

End Point: /api/v3/robin_server/disks/<disk_wwn>

Method: DELETE

URL Parameters: None

Data Parameters: None

Port: RCM Port (default value is 29442)

Headers:

  • Authorization: <auth_token> : Authorization token to identify which user is sending the request. The token can be acquired from the login API.

Success Response Code: 202

Error Response Code: 500 (Internal Server Error), 404 (Not Found Error), 401 (Unauthorized Error), 400 (Invalid API Usage Error)

Example Response:

Output
{
   "jobid": 130
}

5.8. Deleting a disk

Robin provides a utility through which it detaches and deletes disks on multiple cloud platforms. This enables users to remove the cloud storage present within their clusters with convenience and ease. One caveat is that a disk can only be removed if there are no storage volumes allocated from it. In order to remove disks via the aforementioned utility, issue the following command:

Note

This command is only valid for cloud disks.

# robin disk remove <wwn>
                    --yes

wwn

WWN of disk to remove

--yes

Do not prompt the user for confirmation of disk removal

Example:

# robin disk remove 0x087648bb1c5b68a7a --wait --yes
Job:   101 Name: DiskRemove         State: VALIDATED       Error: 0
Job:   101 Name: DiskRemove         State: WAITING         Error: 0
Job:   101 Name: DiskRemove         State: COMPLETED       Error: 0

Removes a cloud disk from the Robin cluster.

End Point: /api/v3/robin_server/disks/<disk_wwn>

Method: DELETE

URL Parameters:

  • remove=true : This mandatory field within the URL indicates that the disk should be removed from the cluster.

Data Parameters: None

Port: RCM Port (default value is 29442)

Headers:

  • Authorization: <auth_token> : Authorization token to identify which user is sending the request. The token can be acquired from the login API.

Success Response Code: 202

Error Response Code: 500 (Internal Server Error), 404 (Not Found Error), 401 (Unauthorized Error), 400 (Invalid API Usage Error)

Example Response:

Output
{
   "jobid": 134
}

5.9. Updating disk properties

In certain circumstances, due to the hardware or environmental reasons, Robin may not correctly discover all the attributes of a device correctly. As a result, in order to modify a disk properties such that they are correct, issue the following command:

# robin disk update <wwn>
                    --maxvolumes <max_volumes>
                    --maxlatencysensitivevolumes <max_latency_volumes>
                    --maxthroughputintensivevolumes <max_throughput_volumes>
                    --role <role>
                    --type <type>
                    --all
                    --set-reattachable
                    --unset-reattachable
                    --set-maintenance
                    --unset-maintenance

wwn

WWN of disk to update

--maxvolumes <max_volumes>

Max number of volumes allowed on the disk

--maxthroughputintensivevolumes <max_latency_volumes>

Max number of latency sensitive volumes allowed on the disk

--maxthroughputinstensivevolumes <max_throughput_volumes>

Max number of throughput intensive volumes allowed on the disk

--role <role>

Role of the disk to update to

--type <type>

Type of the disk to update to

--all

Run the update for all disks. Should be specified if no WWN is given

--set-reattachable

Mark the disk as re-attachable

--unset-reattachable

Mark the disk as not re-attachable

--set-maintenance

Put the disk into maintenance mode

--unset-maintenance

Remove the disk from maintenance mode

Example:

# robin disk update 0xQEMU_QEMU_HARDDISK_3c71c872-fe13-4fd5-b --maxvolumes 100 --type SSD --wait --yes
Job:   101 Name: DiskModify         State: VALIDATED       Error: 0
Job:   101 Name: DiskModify         State: WAITING         Error: 0
Job:   101 Name: DiskModify         State: COMPLETED       Error: 0

Modifies a disk’s discovered and/or Robin specific properties.

End Point: /api/v3/robin_server/disks/<disk_wwn>

Method: PUT

URL Parameters: None

Data Parameters:

  • action: update - This mandatory field within the payload specifies that the update operation is to be performed.

  • role: <role> - Utilizing this parameter results in the role of the disk being set to the value specified. Options include: Storage, Swap, RootDisk, and Reserved.

  • type: <type> - Utilizing this parameter results in the type of the disk being set to the value specified. Options include: HDD and SSD.

  • maxvolumesperdisk: <max_vols_on_disk> - Utilizing this parameter results in the maximum number of volumes allowed on the disk being set to the value specified.

  • maxlatencysensitivevolumesperdisk: <max_lat_sens_vols_on_disk> - Utilizing this parameter results in the maximum number of latency sensitive volumes allowed on the disk being set to the value specified.

  • maxthroughputintensivevolumesperdisk: <max_through_ints_vols_on_disk> - Utilizing this parameter results in the maximum number of throughput intensive volumes allowed on the disk being set to the value specified.

  • reattachable: [0,1] - Utilizing this parameter results in the reattachable attribute of the disk being set. By specifying a value of 1, the disk is said to be reattachable and vice versa for a value of 0.

  • maintenance: [0,1] - Utilizing this parameter results in the maintenance mode attribute of the disk being set. By specifying a value of 1, the disk is said to be in maintenance mode and vice versa for a value of 0.

Port: RCM Port (default value is 29442)

Headers:

  • Authorization: <auth_token> : Authorization token to identify which user is sending the request. The token can be acquired from the login API.

Success Response Code: 202

Error Response Code: 500 (Internal Server Error), 404 (Not Found Error), 401 (Unauthorized Error), 400 (Invalid API Usage Error)

Example Response:

Output
{
   "jobid": 54
}

5.10. Unfaulting a disk

When the Robin storage stack encounters an IO error on a disk, it will mark the disk as FAULTED. Sometimes the error is NOT due to the disk, but due to environmental or other hardware errors such as controller errors. In this case one can determine that the disk is actually healthy. In turn this command allows a user to inform Robin that the disk which is presumed to be FAULTED is actually healthy and the external issues that caused the IO failure have been resolved. Robin will proceed to mark the disk as ONLINE and resume normal access to the disk. In order to an unfault a disk, issue the following command:

Note

This functionality should be used with extreme care. If the disk is actually bad, and a user utilizes this command to reverse the error reported by the Robin storage stack, it could result in data loss.

# robin disk unfault <wwn>
                     --yes

wwn

WWN of disk to unfault

--yes

Do not prompt the user for confirmation of unfaulting the drive

Example:

# robin disk unfault 0xQEMU_QEMU_HARDDISK_3c71c872-fe13-4fd5-b --wait --yes
Job:   101 Name: DiskUnfault         State: VALIDATED       Error: 0
Job:   101 Name: DiskUnfault         State: WAITING         Error: 0
Job:   101 Name: DiskUnfault         State: COMPLETED       Error: 0

Unfaults a disk that has been marked as FAULTED due to an IO error. Note this functionality should be used with extreme care and only in situations where one has determined that external issues such as hardware/controller errors, that have since been resolved, caused the IO failure. If this condition is not met, unfaulting the disk could result in data loss.

End Point: /api/v3/robin_server/disks/<disk_wwn>

Method: PUT

URL Parameters: None

Data Parameters:

  • action: unfault - This mandatory field within the payload specifies that the unfault operation is to be performed.

  • wwn: <disk_wwn> - This mandatory field within the payload specifies the WWN of the disk on which the unfault operation should be performed on.

Port: RCM Port (default value is 29442)

Headers:

  • Authorization: <auth_token> : Authorization token to identify which user is sending the request. The token can be acquired from the login API.

Success Response Code: 202

Error Response Code: 500 (Internal Server Error), 404 (Not Found Error), 401 (Unauthorized Error), 400 (Invalid API Usage Error)

Example Response:

Output
{
   "jobid": 187
}

5.11. Managing Volumes

A Robin volume is the unit of storage allocation from Robin storage. A volume is reflective of the PersistentVolume object within the native Kubernetes world. It provides logical storage for an application to persist data. Robin allows one to manipulate several properties of these base volumes at the time of creation including it’s replication factor (in order to avoid data loss) and the encryption/compression algorithm applied to the data stored within.

In order to aid the management of volumes, the following commands are described in this section:

robin volume list

View all volumes

robin volume info

Retrieve information about a particular volume

robin volume clone-list

List all volume clones

robin volume snapshot-list

List all volume snapshots

robin volume snapshot-space-limit

Configure snapshot space limits for a volume

5.11.1. Listing all volumes

In order to view all volumes currently present on the cluster and some additional details such as the physical usage of the volume, replica state, and mount details, issue the following command:

# robin volume list --all
                    --repl

--all

Display all volumes including those marked for deletion

--repl

Filter list to only display replicated volumes

Example:

# robin volume list
ID | Name                                                                | Media | Type    | BlockSz | Size (GB) | Used (GB) | Storage Status | SnapshotSpaceUsed (GB) | Replicas | Replica State | Compression | Encryption | Mount                                          | Marked for deletion
---+---------------------------------------------------------------------+-------+---------+---------+-----------+-----------+----------------+------------------------+----------+---------------+-------------+------------+------------------------------------------------+---------------------
1  | file-collection-1602783389182.edc29729-1bfe-4da2-87af-3e0e1fbbe8bd  | HDD   | REGULAR | 4096    | 5         | 0.44      | ONLINE         | -/-                    | 2        | SYNCED        | LZ4         | NONE       | cscale-82-35.robinsystems.com:/dev/sdd MOUNTED | False
20 | mdcap-dev.engine.01.data.1.d0710617-4e5c-419a-9453-b77ea32d6a78     | HDD   | REGULAR | 4096    | 10        | 0.53      | ONLINE         | 0/0.0                  | 1        | SYNCED        | NONE        | NONE       | cscale-82-35.robinsystems.com:/dev/sdf MOUNTED | False
21 | mdcap-dev.engine.01.scripts.1.5c434f33-7e2a-421b-bc04-044d42202b75  | HDD   | REGULAR | 4096    | 10        | 0.53      | ONLINE         | 0/2                    | 1        | SYNCED        | NONE        | NONE       | cscale-82-35.robinsystems.com:/dev/sdg MOUNTED | False
22 | mdcap-dev.postgres.01.data.1.22af4bf6-12d5-4ee1-83b3-0361f15c9d13   | HDD   | REGULAR | 4096    | 10        | 1         | ONLINE         | 0/2                    | 1        | SYNCED        | NONE        | NONE       | cscale-82-35.robinsystems.com:/dev/sde MOUNTED | False
23 | mdcap-dev.playground.01.home.1.a266bca7-8169-410a-ab58-529b58d4b319 | HDD   | REGULAR | 4096    | 10        | 0.53      | ONLINE         | 0/2                    | 1        | SYNCED        | NONE        | NONE       | cscale-82-35.robinsystems.com:/dev/sdh MOUNTED | False
24 | mdcap-dev.logstore.01.data.1.3ee94e27-7368-4e77-90f3-d506a73bb29c   | HDD   | REGULAR | 4096    | 10        | 0.53      | ONLINE         | 0/20                   | 1        | SYNCED        | NONE        | NONE       | cscale-82-34.robinsystems.com:/dev/sdd MOUNTED | False
25 | mdcap-dev.nginx.01.data.1.802a6117-1dd4-4c2a-b3a7-a8b622b08ac8      | HDD   | REGULAR | 4096    | 10        | 0.53      | ONLINE         | 0/2                    | 1        | SYNCED        | NONE        | NONE       | cscale-82-34.robinsystems.com:/dev/sde MOUNTED | False

Returns all volumes currently present on the cluster and some additional details such as the physical usage of the volume, replica state, and mount details.

End Point: /api/v3/robin_server/volumes

Method: GET

URL Parameters: None

Data Parameters: None

Port: RCM Port (default value is 29442)

Headers:

  • Authorization: <auth_token> : Authorization token to identify which user is sending the request. The token can be acquired from the login API.

Success Response Code: 200

Error Response Code: 500 (Internal Server Error), 401 (Unauthorized Error)

Example Response:

Output
{
   "items":[
      {
         "media":72,
         "respoolid":1,
         "replication":1,
         "storstatus_str":"ONLINE",
         "volgroupname":"mdcap-dev.logstore.01.72.1.f2c3a591-a34c-40fe-9277-4dd743098c19",
         "volumeid":24,
         "qgroupid":24,
         "tags":[

         ],
         "rsize":10737418240,
         "slice_size":1073741824,
         "appname":"default",
         "storstatus":1,
         "current_snapshotid":1,
         "compress_str":"NONE",
         "lsize":10737418240,
         "name":"mdcap-dev.logstore.01.data.1.3ee94e27-7368-4e77-90f3-d506a73bb29c",
         "snapshot_space_used":0,
         "snapshot_space_limit":21474836480,
         "active_snapshot_count":0,
         "ctime":1602802778,
         "volgroupid":24,
         "volume_ops":{
            "volops_list":[

            ]
         },
         "state":2,
         "genkey":1602802778089095,
         "fsfree":0,
         "alloc":[
            {
               "nodeid":2,
               "zonename":"default",
               "devid":2,
               "zoneid":1602807544,
               "devname":"\/dev\/disk\/by-id\/scsi-3600224800aba82fe29ee8770cc6440f5",
               "nodename":"cscale-82-34.robinsystems.com"
            }
         ],
         "total_snapshot_count":1,
         "status":1,
         "partitions":0,
         "fssize":0,
         "compress_algo":0,
         "asize":10737418240,
         "marked_for_deletion":false,
         "vtype_str":"REGULAR",
         "mounts":[
            {
               "nodeid":2,
               "zonename":"default",
               "state":14,
               "zoneid":1602807544,
               "readonly":0,
               "state_str":"MOUNTED",
               "nodename":"cscale-82-34.robinsystems.com",
               "mntpath":"\/dev\/sdd"
            }
         ],
         "data_state":7,
         "block_size":4096,
         "protection":0,
         "vtype":0,
         "psize":570425344,
         "resync_state":1,
         "media_str":"HDD",
         "next_backupid":1,
         "multinode_mounting":0,
         "appid":1,
         "resync_progress":100,
         "next_snapshotid":2,
         "encrypt_str":"NONE",
         "parent_alloc":[

         ],
         "cloud_psize":0,
         "encrypt_algo":0
      },
      {
         "media":72,
         "respoolid":1,
         "replication":1,
         "storstatus_str":"ONLINE",
         "volgroupname":"mdcap-dev.engine.01.72.1.6c3cddfa-dea8-408f-bea8-a08ec489711c",
         "volumeid":21,
         "qgroupid":21,
         "tags":[

         ],
         "rsize":10737418240,
         "slice_size":1073741824,
         "appname":"default",
         "storstatus":1,
         "current_snapshotid":1,
         "compress_str":"NONE",
         "lsize":10737418240,
         "name":"mdcap-dev.engine.01.scripts.1.5c434f33-7e2a-421b-bc04-044d42202b75",
         "snapshot_space_used":0,
         "snapshot_space_limit":2147483648,
         "active_snapshot_count":0,
         "ctime":1602802777,
         "volgroupid":22,
         "volume_ops":{
            "volops_list":[

            ]
         },
         "state":2,
         "genkey":1602802777582450,
         "fsfree":0,
         "alloc":[
            {
               "nodeid":1,
               "zonename":"default",
               "devid":4,
               "zoneid":1602807544,
               "devname":"\/dev\/disk\/by-id\/scsi-360022480b410ddebc69ecdea69c2bf34",
               "nodename":"cscale-82-35.robinsystems.com"
            }
         ],
         "total_snapshot_count":1,
         "status":1,
         "partitions":0,
         "fssize":0,
         "compress_algo":0,
         "asize":10737418240,
         "marked_for_deletion":false,
         "vtype_str":"REGULAR",
         "mounts":[
            {
               "nodeid":1,
               "zonename":"default",
               "state":14,
               "zoneid":1602807544,
               "readonly":0,
               "state_str":"MOUNTED",
               "nodename":"cscale-82-35.robinsystems.com",
               "mntpath":"\/dev\/sdg"
            }
         ],
         "data_state":7,
         "block_size":4096,
         "protection":0,
         "vtype":0,
         "psize":570425344,
         "resync_state":1,
         "media_str":"HDD",
         "next_backupid":1,
         "multinode_mounting":0,
         "appid":1,
         "resync_progress":100,
         "next_snapshotid":2,
         "encrypt_str":"NONE",
         "parent_alloc":[

         ],
         "cloud_psize":0,
         "encrypt_algo":0
      }
   ]
}

5.11.2. Show information about a specific volume

In order to view detailed information about a volume such as the breakdown of its available replicas, descriptions of its intrinsic properties including backing media type, encryption/compression applied etc., and tags associated with it issue the following command:

# robin volume info <name>
                    --pvc-name <pvc_name>
                    --namespace <namespace>
                    --json

name

Name of the volume to retrieve information of.

--pvc-name <pvc_name>

The name of PVC that maps to a Robin volume. Note this option can only be used if the Robin volume name is not specified.

--namespace <namespace>

Namespace the PVC is bound to. Note this option is only valid and is mandatory if the PVC name is specified.

--json

Output in JSON

Note

Either one of volume_name or pvc-name options must be specified.

Example 1 (Querying a volume via volume name):

# robin volume info file-collection-1602783389182.edc29729-1bfe-4da2-87af-3e0e1fbbe8bd
Name                        : file-collection-1602783389182.edc29729-1bfe-4da2-87af-3e0e1fbbe8bd
ID                          : 1
State                       : READY
Storage Status              : ONLINE
Media                       : HDD
Compression                 : LZ4
Encryption                  : NONE
Volume Type                 : REGULAR
Workload Type               : ordinary
Block Size                  : 4096
Logical Size                : 5G
Physical Size               : 448M
Cloud Tiered Size           : -
Multi Node Mounting         : False
Replica Count               : 2
Replica State               : SYNCED
QGroup ID                   : 1
Resource Pool Id            : 1
Tags                        : []
Creation Time               : 15 Oct 2020 10:36:29

Mounts:
+-------------------------------+------------+------------+---------+-----------+
| Host Name                     | Mount Path | Zone ID    | State   | Read-Only |
+-------------------------------+------------+------------+---------+-----------+
| cscale-82-35.robinsystems.com | /dev/sdd   | 1602807544 | MOUNTED | False     |
+-------------------------------+------------+------------+---------+-----------+

Allocations:
-------

Replica 1
+----+-------------------------------------+-------------------------------+--------------------------------------------------------+----------------+
| ID | WWN                                 | Host Name                     | Devpath                                                | Allocated (GB) |
+----|-------------------------------------+-------------------------------+--------------------------------------------------------+----------------+
| 1  | 0x3600224800aba82fe29ee8770cc6440f5 | cscale-82-34.robinsystems.com | /dev/disk/by-id/scsi-3600224800aba82fe29ee8770cc6440f5 | 5              |
+----+-------------------------------------+-------------------------------+--------------------------------------------------------+----------------+

Replica 2
+----+-------------------------------------+-------------------------------+--------------------------------------------------------+----------------+
| ID | WWN                                 | Host Name                     | Devpath                                                | Allocated (GB) |
+----+-------------------------------------+-------------------------------+--------------------------------------------------------+----------------+
| 2  | 0x360022480b410ddebc69ecdea69c2bf34 | cscale-82-35.robinsystems.com | /dev/disk/by-id/scsi-360022480b410ddebc69ecdea69c2bf34 | 5              |
+----+-------------------------------------+-------------------------------+--------------------------------------------------------+----------------+

Snapshot Info:
-------
Snapshot Count              : 0
Snapshot Space Used         : 0G/-

Example 2 (Querying a volume via PVC name):

# robin volume info --pvc-name data-my-release-mysql-0 --namespace default
Name                        : pvc-35855033-3505-4dba-bf74-b8746322c528
ID                          : 6
State                       : READY
Status                      : ONLINE
Storage Status              : ONLINE
Media                       : HDD
Compression                 : NONE
Encryption                  : NONE
Volume Type                 : REGULAR
Workload Type               : ordinary
Block Size                  : 4096
Logical Size                : 8G
Physical Size               : -
Cloud Tiered Size           : -
Multi Node Mounting         : True
Replica Count               : 1
Replica State               : SYNCED
Fault Domain                : N/A
QGroup ID                   : 6
Resource Pool Id            : 1
Tags                        : []
Creation Time               : 03 Oct 2021 09:27:08

Mounts:
+-------------------------------+------------+------------+---------+-----------+
| Host Name                     | Mount Path | Zone ID    | State   | Read-Only |
+-------------------------------+------------+------------+---------+-----------+
| cscale-82-83.robinsystems.com | /dev/sde   | 1633270151 | MOUNTED | False     |
+-------------------------------+------------+------------+---------+-----------+

Allocations:
-------

Replica 1
+----+------------------------------------+-------------------------------+--------------------------------------------------------+-----------+
| ID | WWN                                | Host Name                     | Devpath                                                | Size (GB) |
+----+------------------------------------+-------------------------------+--------------------------------------------------------+-----------+
| 5  | 0x60022480a889d05f4df285d75cf253e0 | cscale-82-83.robinsystems.com | /dev/disk/by-id/scsi-360022480a889d05f4df285d75cf253e0 | 8         |
+----+------------------------------------+-------------------------------+--------------------------------------------------------+-----------+

Snapshot Info:
-------
Snapshot Count              : 0
Snapshot Space Used         : 0G/-

Returns detailed information about a volume such as the breakdown of its available replicas, descriptions of its intrinsic properties including backing media type, encryption/compression applied etc., and tags associated with it.

End Point: /api/v3/robin_server/volumes/

Method: GET

URL Parameters: None

Data Parameters:

  • name: <volume_name> - This mandatory field within the payload specifies the name of the volume for which information should be retrieved for.

Port: RCM Port (default value is 29442)

Headers:

  • Authorization: <auth_token> : Authorization token to identify which user is sending the request. The token can be acquired from the login API.

Success Response Code: 200

Error Response Code: 500 (Internal Server Error), 404 (Not Found Error), 401 (Unauthorized Error)

Example Response:

Output
{
   "items":{
      "volumeid":32,
      "resync_progress":100,
      "total_snapshot_count":1,
      "psize":503316480,
      "ctime":1615831965,
      "snapshot_space_used":0,
      "appid":1,
      "multinode_mounting":0,
      "protection":1,
      "name":"file-collection-1615831964492.12f41f8a-1cd9-451d-b719-010832f8521d",
      "snapshot_space_limit":0,
      "active_snapshot_count":0,
      "parent_alloc":[

      ],
      "status":1,
      "media":72,
      "resync_state":1,
      "next_backupid":1,
      "vtype":0,
      "current_snapshotid":1,
      "genkey":1615831965016901,
      "volgroupname":"file-collection-1615831964492.12f41f8a-1cd9-451d-b719-010832f8521d.72.1.01ceef4e-d5e9-417e-8c62-e3352d26643d",
      "appname":"default",
      "next_snapshotid":2,
      "qgroupid":1,
      "block_size":4096,
      "slice_size":1073741824,
      "compress_algo":1,
      "encrypt_algo":0,
      "tags":[

      ],
      "replication":3,
      "state":2,
      "data_state":7,
      "mounts":[
         {
            "zoneid":1613675775,
            "zonename":"default",
            "nodename":"vnode-95-28.robinsystems.com",
            "nodeid":1,
            "mntpath":"\/dev\/sdd",
            "readonly":0,
            "state":14,
            "state_str":"MOUNTED"
         }
      ],
      "partitions":0,
      "lsize":5368709120,
      "fssize":0,
      "fsfree":0,
      "volume_ops":{
         "volops_list":[

         ]
      },
      "rsize":16106127360,
      "asize":16106127360,
      "cloud_psize":0,
      "respoolid":1,
      "volgroupid":32,
      "alloc":[
         [
            {
               "id":2,
               "wwn":"0xQEMU_QEMU_HARDDISK_80cb8b56-aef2-48a8-8",
               "allocation":5,
               "node":"vnode-95-28.robinsystems.com",
               "devpath":"\/dev\/disk\/by-id\/scsi-0QEMU_QEMU_HARDDISK_80cb8b56-aef2-48a8-8"
            }
         ],
         [
            {
               "id":1,
               "wwn":"0xQEMU_QEMU_HARDDISK_40b30feb-02eb-4a5a-b",
               "allocation":5,
               "node":"vnode-95-12.robinsystems.com",
               "devpath":"\/dev\/disk\/by-id\/scsi-0QEMU_QEMU_HARDDISK_40b30feb-02eb-4a5a-b"
            }
         ],
         [
            {
               "id":3,
               "wwn":"0xQEMU_QEMU_HARDDISK_50c07412-3cbb-428e-8",
               "allocation":5,
               "node":"vnode-95-83.robinsystems.com",
               "devpath":"\/dev\/disk\/by-id\/scsi-0QEMU_QEMU_HARDDISK_50c07412-3cbb-428e-8"
            }
         ]
      ],
      "storstatus":1,
      "media_str":"HDD",
      "vtype_str":"REGULAR",
      "compress_str":"LZ4",
      "encrypt_str":"NONE",
      "storstatus_str":"ONLINE",
      "storstate_str":"READY",
      "rwx_vol":false,
      "workload":0,
      "workload_str":"ordinary",
      "nfs_exports":[

      ]
   }
}

5.11.3. Listing all volume clones

In order to view all clone volumes currently present on the cluster and some additional details such as the physical usage of the volume, parent volume, and mount details, issue the following command:

# robin volume clone-list --json

--json

Output in JSON format

Example:

ID | Name                                                                | ParentSnap          | ParentVolume                                                     | Media | BlockSz | Size (GB) | Psize (GB) | Storage Status | SnapshotSpaceUsed (GB) | Replicas | Replica State | Compression | Encryption | Mount
---+---------------------------------------------------------------------+---------------------+------------------------------------------------------------------+-------+---------+-----------+------------+----------------+------------------------+----------+---------------+-------------+------------+-------------------------------------------------
9  | clone-demo.server.01.block.1.2c69deaa-52bc-40ba-b455-5271e4fae4a1   | snapshot-1603479973 | centos1.server.01.block.1.c90c49c0-74ab-4459-be69-fea9f730cc9e   | HDD   | 4096    | 1         | -          | ONLINE         | 0/0.2                  | 1        | SYNCED        | NONE        | NONE       | cscale-82-140.robinsystems.com:/dev/sdm MOUNTED
10 | clone-demo.server.01.root_fs.1.63262a2c-b8b2-4f03-9fea-65b47a95a5a8 | snapshot-1603479973 | centos1.server.01.root_fs.1.2d59b0b3-c33a-4594-9d3d-44def625a38b | HDD   | 4096    | 1         | 0.03       | ONLINE         | 0/0.2                  | 1        | SYNCED        | NONE        | NONE       | cscale-82-140.robinsystems.com:/dev/sdn MOUNTED
11 | clone-demo.server.01.data.1.dd86ccb2-ce0a-4f39-a041-66ec4d492fbc    | snapshot-1603479973 | centos1.server.01.data.1.8dec6c3f-839a-439c-b830-130386ff40f4    | HDD   | 4096    | 1         | 0.03       | ONLINE         | 0/0.2                  | 1        | SYNCED        | NONE        | NONE       | cscale-82-140.robinsystems.com:/dev/sdl MOUNTED

Returns all clone volumes currently present on the cluster and some additional details such as the physical usage of the volume, parent volume, and mount details.

End Point: /api/v3/robin_server/volumes/clone

Method: GET

URL Parameters: None

Data Parameters: None

Port: RCM Port (default value is 29442)

Headers:

  • Authorization: <auth_token> : Authorization token to identify which user is sending the request. The token can be acquired from the login API.

Success Response Code: 200

Error Response Code: 500 (Internal Server Error), 401 (Unauthorized Error)

Example Response:

Output
{
   "items":[
      {
         "qgroupid":10,
         "rsize":1073741824,
         "multinode_mounting":0,
         "volgroupname":"clone-demo.server.01.72.1.3a122025-f911-4ecf-b689-f5492bbefd72",
         "parent_volumeid":6,
         "encrypt_algo":0,
         "parent_alloc":[
            {
               "devid":1,
               "zoneid":1603415748,
               "nodeid":1
            }
         ],
         "volgroupid":9,
         "fsfree":0,
         "psize":33554432,
         "next_snapshotid":2,
         "replication":1,
         "cloud_psize":0,
         "lsize":1073741824,
         "status":1,
         "encrypt_str":"NONE",
         "volumeid":11,
         "vtype":1,
         "ctime":1603480455,
         "snapshot_space_used":0,
         "storstatus_str":"ONLINE",
         "respoolid":1,
         "marked_for_deletion":false,
         "appid":1,
         "appname":"default",
         "protection":0,
         "media":72,
         "media_str":"HDD",
         "genkey":1603480455171116,
         "next_backupid":1,
         "tags":[

         ],
         "block_size":4096,
         "asize":1073741824,
         "partitions":0,
         "parent_snapshotid":1,
         "slice_size":1073741824,
         "storstatus":1,
         "name":"clone-demo.server.01.data.1.dd86ccb2-ce0a-4f39-a041-66ec4d492fbc",
         "total_snapshot_count":1,
         "vtype_str":"CLONE",
         "alloc":[
            {
               "devid":1,
               "nodeid":1,
               "nodename":"cscale-82-140.robinsystems.com",
               "zoneid":1603415748,
               "zonename":"default",
               "devname":"\/dev\/disk\/by-id\/scsi-3600224803bcdafde95b1f5cd27ceb5fb"
            }
         ],
         "volume_ops":{
            "volops_list":[

            ]
         },
         "data_state":7,
         "mounts":[
            {
               "nodeid":1,
               "nodename":"cscale-82-140.robinsystems.com",
               "state_str":"MOUNTED",
               "state":14,
               "zoneid":1603415748,
               "zonename":"default",
               "mntpath":"\/dev\/sdl",
               "readonly":0
            }
         ],
         "parent_appname":"default",
         "active_snapshot_count":0,
         "current_snapshotid":1,
         "fssize":0,
         "parent_volumename":"centos1.server.01.data.1.8dec6c3f-839a-439c-b830-130386ff40f4",
         "compress_str":"NONE",
         "resync_state":1,
         "snapshot_space_limit":214748364,
         "resync_progress":100,
         "parent_snapshotname":"snapshot-1603479973",
         "state":2,
         "parent_appid":1,
         "compress_algo":0
      },
      {
         "qgroupid":9,
         "rsize":1073741824,
         "multinode_mounting":0,
         "volgroupname":"clone-demo.server.01.72.1.bc0f9bf3-3df2-4ec8-8a03-2edad0d4d176",
         "parent_volumeid":8,
         "encrypt_algo":0,
         "parent_alloc":[
            {
               "devid":1,
               "zoneid":1603415748,
               "nodeid":1
            }
         ],
         "volgroupid":11,
         "fsfree":0,
         "psize":33554432,
         "next_snapshotid":2,
         "replication":1,
         "cloud_psize":0,
         "lsize":1073741824,
         "status":1,
         "encrypt_str":"NONE",
         "volumeid":10,
         "vtype":1,
         "ctime":1603480455,
         "snapshot_space_used":0,
         "storstatus_str":"ONLINE",
         "respoolid":1,
         "marked_for_deletion":false,
         "appid":1,
         "appname":"default",
         "protection":0,
         "media":72,
         "media_str":"HDD",
         "genkey":1603480455157228,
         "next_backupid":1,
         "tags":[

         ],
         "block_size":4096,
         "asize":1073741824,
         "partitions":0,
         "parent_snapshotid":1,
         "slice_size":1073741824,
         "storstatus":1,
         "name":"clone-demo.server.01.root_fs.1.63262a2c-b8b2-4f03-9fea-65b47a95a5a8",
         "total_snapshot_count":1,
         "vtype_str":"CLONE",
         "alloc":[
            {
               "devid":2,
               "nodeid":1,
               "nodename":"cscale-82-140.robinsystems.com",
               "zoneid":1603415748,
               "zonename":"default",
               "devname":"\/dev\/disk\/by-id\/scsi-3600224804c48fd7e16c608dea0919064"
            }
         ],
         "volume_ops":{
            "volops_list":[

            ]
         },
         "data_state":7,
         "mounts":[
            {
               "nodeid":1,
               "nodename":"cscale-82-140.robinsystems.com",
               "state_str":"MOUNTED",
               "state":14,
               "zoneid":1603415748,
               "zonename":"default",
               "mntpath":"\/dev\/sdn",
               "readonly":0
            }
         ],
         "parent_appname":"default",
         "active_snapshot_count":0,
         "current_snapshotid":1,
         "fssize":0,
         "parent_volumename":"centos1.server.01.root_fs.1.2d59b0b3-c33a-4594-9d3d-44def625a38b",
         "compress_str":"NONE",
         "resync_state":1,
         "snapshot_space_limit":214748364,
         "resync_progress":100,
         "parent_snapshotname":"snapshot-1603479973",
         "state":2,
         "parent_appid":1,
         "compress_algo":0
      },
      {
         "qgroupid":11,
         "rsize":1073741824,
         "multinode_mounting":0,
         "volgroupname":"clone-demo.server.01.72.1.f366e12b-ae61-4f1d-a441-0258d764d5f3",
         "parent_volumeid":7,
         "encrypt_algo":0,
         "parent_alloc":[
            {
               "devid":1,
               "zoneid":1603415748,
               "nodeid":1
            }
         ],
         "volgroupid":10,
         "fsfree":0,
         "psize":0,
         "next_snapshotid":2,
         "replication":1,
         "cloud_psize":0,
         "lsize":1073741824,
         "status":1,
         "encrypt_str":"NONE",
         "volumeid":9,
         "vtype":1,
         "ctime":1603480455,
         "snapshot_space_used":0,
         "storstatus_str":"ONLINE",
         "respoolid":1,
         "marked_for_deletion":false,
         "appid":1,
         "appname":"default",
         "protection":0,
         "media":72,
         "media_str":"HDD",
         "genkey":1603480455077815,
         "next_backupid":1,
         "tags":[

         ],
         "block_size":4096,
         "asize":0,
         "partitions":0,
         "parent_snapshotid":1,
         "slice_size":1073741824,
         "storstatus":1,
         "name":"clone-demo.server.01.block.1.2c69deaa-52bc-40ba-b455-5271e4fae4a1",
         "total_snapshot_count":1,
         "vtype_str":"CLONE",
         "alloc":[

         ],
         "volume_ops":{
            "volops_list":[

            ]
         },
         "data_state":7,
         "mounts":[
            {
               "nodeid":1,
               "nodename":"cscale-82-140.robinsystems.com",
               "state_str":"MOUNTED",
               "state":14,
               "zoneid":1603415748,
               "zonename":"default",
               "mntpath":"\/dev\/sdm",
               "readonly":0
            }
         ],
         "parent_appname":"default",
         "active_snapshot_count":0,
         "current_snapshotid":1,
         "fssize":0,
         "parent_volumename":"centos1.server.01.block.1.c90c49c0-74ab-4459-be69-fea9f730cc9e",
         "compress_str":"NONE",
         "resync_state":1,
         "snapshot_space_limit":214748364,
         "resync_progress":100,
         "parent_snapshotname":"snapshot-1603479973",
         "state":2,
         "parent_appid":1,
         "compress_algo":0
      }
   ]
}

5.11.4. Listing all volume snapshots

In order to view all volume snapshots currently present on the cluster and some additional details for each snapshot such as the state, number of clones/backups that have been created etc., issue the following command:

# robin volume snapshot-list --json

--json

Output in JSON format

Example:

# robin volume snapshot-list
Snap ID | Name                | Volume                                                           | Read Only | State | ParentSnap ID | Clones | Backups | Creation Time
--------+---------------------+------------------------------------------------------------------+-----------+-------+---------------+--------+---------+---------------
1       | snapshot-1603479973 | centos1.server.01.data.1.8dec6c3f-839a-439c-b830-130386ff40f4    | True      | READY | -             | 0      | 0       | -
1       | snapshot-1603479973 | centos1.server.01.block.1.c90c49c0-74ab-4459-be69-fea9f730cc9e   | True      | READY | -             | 0      | 0       | -
1       | snapshot-1603479973 | centos1.server.01.root_fs.1.2d59b0b3-c33a-4594-9d3d-44def625a38b | True      | READY | -             | 0      | 0       | -

In order to view all volume snapshots currently present on the cluster and some additional details for each snapshot such as the state, number of clones/backups that have been created etc., issue the following command:

End Point: /api/v3/robin_server/volumes/snap

Method: GET

URL Parameters: None

Data Parameters: None

Port: RCM Port (default value is 29442)

Headers:

  • Authorization: <auth_token> : Authorization token to identify which user is sending the request. The token can be acquired from the login API.

Success Response Code: 200

Error Response Code: 500 (Internal Server Error), 401 (Unauthorized Error)

Example Response:

Output
{
   "items":{
      "media":72,
      "respoolid":1,
      "replication":2,
      "storstatus_str":"ONLINE",
      "volgroupname":"file-collection-1602783389182.edc29729-1bfe-4da2-87af-3e0e1fbbe8bd.72.1.0a6b2335-0b82-4c88-815c-c47f0a7e308d",
      "volumeid":1,
      "qgroupid":1,
      "tags":[

      ],
      "rsize":10737418240,
      "slice_size":1073741824,
      "appname":"default",
      "storstatus":1,
      "current_snapshotid":1,
      "compress_str":"LZ4",
      "lsize":5368709120,
      "name":"file-collection-1602783389182.edc29729-1bfe-4da2-87af-3e0e1fbbe8bd",
      "snapshot_space_used":0,
      "snapshot_space_limit":0,
      "active_snapshot_count":0,
      "ctime":1602783389,
      "volgroupid":1,
      "volume_ops":{
         "volops_list":[

         ]
      },
      "state":2,
      "genkey":1602783389367989,
      "fsfree":0,
      "alloc":[
         [
            {
               "devpath":"\/dev\/disk\/by-id\/scsi-3600224800aba82fe29ee8770cc6440f5",
               "node":"cscale-82-34.robinsystems.com",
               "allocation":5,
               "wwn":"0x3600224800aba82fe29ee8770cc6440f5"
            }
         ],
         [
            {
               "devpath":"\/dev\/disk\/by-id\/scsi-360022480b410ddebc69ecdea69c2bf34",
               "node":"cscale-82-35.robinsystems.com",
               "allocation":5,
               "wwn":"0x360022480b410ddebc69ecdea69c2bf34"
            }
         ]
      ],
      "total_snapshot_count":1,
      "status":1,
      "partitions":0,
      "storstate_str":"READY",
      "fssize":0,
      "compress_algo":1,
      "asize":10737418240,
      "vtype_str":"REGULAR",
      "mounts":[
         {
            "nodeid":1,
            "zonename":"default",
            "state":14,
            "zoneid":1602807544,
            "readonly":0,
            "state_str":"MOUNTED",
            "nodename":"cscale-82-35.robinsystems.com",
            "mntpath":"\/dev\/sdd"
         }
      ],
      "data_state":7,
      "block_size":4096,
      "protection":1,
      "vtype":0,
      "psize":469762048,
      "resync_state":1,
      "media_str":"HDD",
      "next_backupid":1,
      "multinode_mounting":0,
      "appid":1,
      "resync_progress":100,
      "next_snapshotid":2,
      "encrypt_str":"NONE",
      "parent_alloc":[

      ],
      "cloud_psize":0,
      "encrypt_algo":0
   }
}

5.11.5. Configure the snapshot space limit of a volume

Robin stores snapshot(s) of a volume within a dedicated space, referred to as the snapshot space of the volume. The snapshot space is considered to be a percentage of the allocated space for a volume (by default this percentage is 40%). However after the volume is provisioned the size of the snapshot might need to be modified. In order to achieve this, issue the following command:

# robin volume snapshot-space-limit <volume_name> <snapshot_space_limit>

Note

Only the snapshot space limit of individual volumes can be configured at this time. To configure the snapshot space limit for an application, update the snapshot space limit of each of the volumes it consists of.

volume_name

Name of the volume to configure the snapshot space limit for

snapshot_space_limit

Size of the snapshot space limit in bytes

Example:

# robin volume snapshot-space-limit pvc-bd01bcd9-9e4d-4a34-ac13-5a98395935b3 4G --wait
Job:  163 Name: VolumeUpdateSnapshotLimit State: PROCESSED       Error: 0
Job:  163 Name: VolumeUpdateSnapshotLimit State: FINALIZED       Error: 0
Job:  163 Name: VolumeUpdateSnapshotLimit State: COMPLETED       Error: 0

Configures the snapshot space limit of a volume.

End Point: /api/v3/robin_server/volumes/<volume_name>

Method: PUT

URL Parameters: None

Data Parameters:

  • action: snapshot_limits - This mandatory field within the payload specifies that the snapshot limit of the volume is to be modified.

  • name: <volume_name> - This mandatory field within the payload specifies the name of the volume for which the snapshot limits are to be modified.

  • snapshot_space_limit: <snap_limit> - This mandatory field within the payload specifies the size, in bytes, the snapshot space limit of the aforementioned volume should be set to.

Port: RCM Port (default value is 29442)

Headers:

  • Authorization: <auth_token> : Authorization token to identify which user is sending the request. The token can be acquired from the login API.

Success Response Code: 202

Error Response Code: 500 (Internal Server Error), 404 (Not Found Error), 401 (Unauthorized Error), 400 (Invalid API Usage Error)

Example Response:

Output
{
   "jobid": 163
}

5.12. ReadWriteMany (RWX) Volumes

Robin supports the ReadWriteMany (RWX) access mode of Persistent Volumes (PVs). An RWX PVC can be used by any Pod on any node in the same namespace for read and write operations. More information on these types of volumes can be found here.

Robin provides support for RWX volumes by utilizing a shared file system. Specifically the network file system (NFS) is used. These volumes can be mounted within any Pod deployed via Helm or YAML files and consequently can support multiple read/write clients. In addition, support for RWX volumes also extends to non-root application Pods, details for which can be found here.

Note

RWX volumes are not supported within Robin Bundle applications.

5.12.1. Default replication factor and fault domain parameters for RWX PVCs

For RWX PVCs the default replication factor is 2 and the default faultdomain is host.

If you want to change the replication factor to 1, you can use the following parameter under annotations in the RWX PVC: robin.io/rwx_force_single_replica: "1".

Note

You must not use the robin.io/replication and robin.io/rwx_force_single_replica annotations together in a PVC.

The following is the example PVC file:

apiVersion: v1
kind: PersistentVolumeClaim
metadata:
   name: nfs-shared-1
   annotations:
      robin.io/nfs-server-type: "shared"
      robin.io/replication: "2"
      robin.io/faultdomain: "host"
spec:
storageClassName: robin
accessModes:
   - ReadWriteMany
resources:
   requests:
      storage: 500Gi

5.12.2. NFS server Pod

Every RWX PVC results in an NFS export from an NFS server Pod. When you create a PVC, Robin automatically creates an NFS server Pod and configures it on demand. All Robin NFS server Pods are High Availability (HA) compliant. Robin monitors the health of the NFS server Pods and executes a failover automatically if any NFS server Pod is offline.

There are two types of NFS server Pod, shared and exclusive, which are described in the sections below. The default NFS server Pod type is shared. In order to update this value, the following command can be used:

# robin config update nfs default_server_type <pod_type>

Example

# robin config update nfs default_server_type exclusive

5.12.2.1. Shared NFS server Pod

With a shared NFS server Pod, multiple RWX PVC exports can be allocated to one NFS server Pod. To allocate a shared NFS server Pod for a PVC, use the following annotation. An example of its usage is shown below as well.

robin.io/nfs-server-type: "shared"

Example

apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: nfs-shared-1
  annotations:
   robin.io/nfs-server-type: "shared"
spec:
  accessModes:
     - ReadWriteMany
  resources:
    requests:
      storage: 1Gi

Note

Shared NFS server Pods are not deleted when associated volumes are unmounted or deleted.

5.12.2.2. Exclusive NFS server Pod

With an exclusive NFS server Pod, only one RWX PVC is associated the Pod resulting in a dedicated NFS server Pod for the PVC. To allocate an exclusive NFS server Pod for a PVC, use the following annotation. An example of its usage is shown below as well.

robin.io/nfs-server-type: "exclusive"

Example

apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: nfs-excl-1
  annotations:
   robin.io/nfs-server-type: "exclusive"
spec:
  accessModes:
     - ReadWriteMany
  resources:
    requests:
      storage: 1Gi

Note

Exclusive NFS server Pods are deleted when associated volumes are unmounted or deleted.

5.12.3. Set resource limits for exclusive NFS Pods

The requests and limits for the CPU and memory consumption of exclusive NFS server Pods can be set using PVC annotations. This allows users to control the resource utilization of dedicated RWX volumes and override the default config values for each of the attributes.

To set the requests and limits for CPU, use the respective annotation shown below.

robin.io/nfs-cpu-limit: <limit_value>
robin.io/nfs-cpu-request: <request_value>

To set the requests and limits for memory, use the respective annotation shown below.

robin.io/nfs-memory-limit: <limit_value>
robin.io/nfs-memory-request: <request_value>

Example

apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: CSI-PVC-robin
  labels:
    app.kubernetes.io/instance: robin
    app.kubernetes.io/managed-by: robin.io
    app.kubernetes.io/name: robin
  annotations:
    robin.io/media: HDD
    robin.io/compression: LZ4
    robin.io/nfs-server-type: "exclusive"
    robin.io/nfs-cpu-limit: 300m
    robin.io/nfs-memory-limit: 300Mi
    robin.io/nfs-cpu-request: 240m
    robin.io/nfs-memory-request: 240Mi
spec:
  accessModes:
     - ReadWriteMany
  resources:
    requests:
      storage: 1Gi
  StorageClassName: robin

Note

The annotations above must be specified during the initial creation of the PVC otherwise they will not take affect even if given at a later time.

5.12.4. Configure NFS server Pod attributes

If you want to create an NFS server Pod other than the default configuration, you can change the NFS server Pod attributes.

The modified values are applied only to the next NFS server Pod creation. These values are not applied to the existing NFS server Pods.

View NFS server Pod attributes and values

To view the list of NFS server Pod attributes and their values, run the following command:

# robin config list nfs

NFS server Pod attributes and their values

Attribute

Default Value

Valid Value

default_server_type

shared

shared - multiple RWX PVC exports are allocated to one NFS server Pod.
exclusive - each RWX PVC results in a dedicated NFS server Pod.

excl_pod_cpu

100m

As per your requirement.

excl_pod_memory

200Mi

As per your requirement.

exclusive_pod_cpu_limit

UNLIMITED

As per your requirement.

exclusive_pod_memory_limit

UNLIMITED

As per your requirement.

failover_enabled

1

1 – to enable the failover.
0 – to disable the failover.

max_exports_per_pod

8

As per your requirement.

nfs_server_storage_affinity

none

none - an NFS server Pod is created on a node where sufficient storage and other resources are available, without checking for allocated volume.
preferred - an NFS server Pod is created on the same node as the volume is allocated. If no sufficient storage and other resources are available on the node, the NFS server Pod is created on the available node.
required - an NFS server Pod is created on the same node as the volume is allocated. If no sufficient storage and other resources are available on the node, the NFS server Pod creation fails.

pod_creation_timeout

600

As per your requirement.

service_creation_timeout

60

As per your requirement.

shared_pod_cpu

100m

As per your requirement.

shared_pod_cpu_limit

UNLIMITED

As per your requirement.

shared_pod_failover_serialized

1

1 – to enable the shared Pod failover.
0 – to disable the shared Pod failover.

shared_pod_memory

200Mi

As per your requirement.

shared_pod_memory_limit

UNLIMITED

As per your requirement.

shared_pod_placement

PACK

PACK – selects the NFS server Pod that is assigned to the most and under the max limit.
SPREAD – selects the NFS server Pod with minimum PVC assignments and ensures that all NFS server Pods are equally loaded.

Update the NFS server Pod attributes

To update the NFS server Pod attributes, run the following command:

# robin config update nfs <attribute> <valid value>

Example

To update the shared_pod_placement attribute of the NFS server Pod.

# robin config update nfs shared_pod_placement SPREAD
The 'nfs' attribute 'shared_pod_placement' has been updated

5.12.5. View NFS export list for RWX volume

You can view the list of NFS exports and their state for the RWX volumes available in your cluster. You get the following information about the NFS exports:

  • Export state

  • Export ID

  • Name of the volume

  • Name of the NFS server Pod

  • Name of the client hosts for the respective NFS export

To view the list of NFS exports, run the following command:

# robin nfs export-list
                    --verbose
                    --server <server>
                    --volume <volume>
                    --json <json>

--verbose

Include additional information in the output.

--server <server>

Filter the list of NFS exports for the requested NFS server Pod. Provide either the NFS server ID or NFS server Pod name as an argument.

--volume <volume>

Filter the list of NFS exports for the requested volume. Provide either the volume ID or volume name as an argument.

--json <json>

Output in JSON format.

Example

# robin nfs export-list
+--------------+-----------+------------------------------------------+-------------------------+-----------------------------------------------------------------------+
| Export State | Export ID |                  Volume                  |      NFS Server Pod     |                             Export Clients                            |
+--------------+-----------+------------------------------------------+-------------------------+-----------------------------------------------------------------------+
|    READY     |     18    | pvc-2985f5f7-fd32-4645-95d1-551bce9ed002 | robin-nfs-excl-v112-175 |                  ["hypervvm-62-33.robinsystems.com"]                  |
|    READY     |     21    | pvc-9b1ea557-f0f5-419d-ad2a-54eed25c22c1 | robin-nfs-excl-v115-179 |                  ["hypervvm-62-33.robinsystems.com"]                  |
|    READY     |     2     | pvc-6ec4aa2d-6b56-4746-85c2-4bc8bf494115 |   robin-nfs-shared-181  |                  ["hypervvm-62-35.robinsystems.com"]                  |
|    READY     |     5     | pvc-6f5747b5-88c9-4e92-9547-0cf7cdf7b12a |   robin-nfs-shared-181  | ["hypervvm-62-35.robinsystems.com","hypervvm-62-33.robinsystems.com"] |
|    READY     |     4     | pvc-a5e14684-761b-4f3e-8f0a-a1828de3fa3f |   robin-nfs-shared-181  |                  ["hypervvm-62-35.robinsystems.com"]                  |
|    READY     |     13    | pvc-7fb597c8-f976-4ea6-9422-5349e37e544d | robin-nfs-excl-v107-170 |                  ["hypervvm-62-34.robinsystems.com"]                  |
|    READY     |     19    | pvc-33827f5a-4004-49fb-b6ef-22f3b0ff6c40 | robin-nfs-excl-v113-182 |                  ["hypervvm-62-34.robinsystems.com"]                  |
+--------------+-----------+------------------------------------------+-------------------------+-----------------------------------------------------------------------+

5.12.6. View NFS server Pod list

You can view the list of NFS server Pods available in your cluster. You can view the following information about the NFS server Pods available in your cluster:

  • NFS server Pod ID

  • Name of the NFS server Pod

  • Name of the host on which NFS server Pod is scheduled

  • Type of NFS server Pod

  • State of the NFS server Pod

To view the list of NFS server Pods, run the following command:

# robin nfs server-list
                    --json <json>

--json <json>

output in JSON format.

Example

# robin nfs server-list
+--------+-------------------------+---------------------------------+---------------------+--------+
| Pod ID |      NFS Server Pod     |             Hostname            | NFS Server Pod Type | State  |
+--------+-------------------------+---------------------------------+---------------------+--------+
|  180   | robin-nfs-excl-v108-180 | hypervvm-62-35.robinsystems.com |      EXCLUSIVE      | ONLINE |
|  181   |   robin-nfs-shared-181  | hypervvm-62-35.robinsystems.com |        SHARED       | ONLINE |
|  170   | robin-nfs-excl-v107-170 | hypervvm-62-33.robinsystems.com |      EXCLUSIVE      | ONLINE |
|  185   | robin-nfs-excl-v104-185 | hypervvm-62-34.robinsystems.com |      EXCLUSIVE      | ONLINE |
|  184   | robin-nfs-excl-v110-184 | hypervvm-62-35.robinsystems.com |      EXCLUSIVE      | ONLINE |
+--------+-------------------------+---------------------------------+---------------------+--------+

5.12.7. View list of applications using RWX volume

You can view the list of applications using the RWX volume in your cluster. You can view the following information about the application:

  • Name of the application

  • Type of application

  • Name of the NFS server Pod

To view the list of applications, run the following command

# robin nfs app-list
                 --server <server>
                 --json <json>

--server <server>

Filter applications that are using the respective NFS server Pod.

--json <json>

Output in JSON format.

Example

# robin nfs app-list
+-------------+------------------+----------------------+
| Application | Application Type |      NFS Server      |
+-------------+------------------+----------------------+
|    myapp1   |       helm       | robin-nfs-shared-181 |
|    myapp2   |       helm       | robin-nfs-shared-181 |
|    myapp3   |       helm       | robin-nfs-shared-181 |
+-------------+------------------+----------------------+

5.12.8. View NFS server Pod used by application

You can view the following information about the NFS server Pod being used by the application:

  • Name of application

  • Name of volume

  • Name of NFS server Pod

To view the NFS server Pod being used by an application, run the following command:

# robin nfs server-info [name]
                         --json <json>

name

Name of the Application.

--json <json>

Output in JSON format.

Example

# robin nfs server-info myapp1
+-------------+------------------------------------------+----------------------+
| Application | Volume Name                                    | NFS Server Pod       |
+-------------+------------------------------------------+----------------------+
| myapp1        | pvc-6ec4aa2d-6b56-4746-85c2-4bc8bf494115 | robin-nfs-shared-181 |
+-------------+------------------------------------------+----------------------+

5.13. Robin StorageClass with GID and UID to Run Non-Root App Pods

In Kubernetes, only the root user can access all the persistent volumes. However, using new parameters in Robin’s StorageClass, you can allow a specific set of users to access the persistent volume. You can provide read and write access for a non-root user to a persistent volume by providing GID and UID when creating a new Robin StorageClass. Use this StorageClass in the PVC and set Pod’s security context with the runAsUser value. When you provide a GID for read and write access to the persistent volumes, any non-root user that belongs to the group ID, including a Pod, is granted access to the file storage.

The following is the sample YAML file:

allowVolumeExpansion: true
apiVersion: storage.k8s.io/v1
kind: StorageClass
labels:
  app.kubernetes.io/instance: robin
  app.kubernetes.io/managed-by: robin.io
  app.kubernetes.io/name: robin
metadata:
        name: robin-gid
parameters:
  gidAllocate: "true"
  gidFixed: "1001"
  media: HDD
  uidFixed: "1001"
provisioner: robin
reclaimPolicy: Delete
volumeBindingMode: WaitForFirstConsumer

Note

There are Kubernetes known permission issues when you mount a volume with subPath for non-root users. As a workaround, you can remove the subPath parameter or use init container to chown the path. For details about the known issue, see here.

5.14. Robin StorageClass with runAsAny parameter

Starting with Robin CNP 5.3.11 HF2, you can use a new parameter runAsAny in the StorageClass object to enable any user other than the root user to read or write to an NFS mountpoint of an RWX volume.

You can use this parameter in a scenario with multiple containers and different users, and you want to allow any user accessing the Pod(containers) to read or write to an NFS mountpoint of an RWX volume.

In the StorageClass object file, set the runAsAny parameter to true.

The following is an example of the StorageClass with runAsAny parameter:

   apiVersion: storage.k8s.io/v1
   kind: StorageClass
   metadata:
      name: run-as-any-imm
      labels:
         app.kubernetes.io/instance: robin
         app.kubernetes.io/managed-by: robin.io
         app.kubernetes.io/name: robin
   provisioner: robin
   reclaimPolicy: Delete
   allowVolumeExpansion: true
   volumeBindingMode: Immediate
   parameters:
      replication: '2'
      media: HDD
      runAsAny: "true"

5.15. Managing Application Ephemeral Volumes

Robin CNP supports Application Ephemeral Volumes (AEVs). An AEV is non-persistent temporary storage that is attached to a Robin bundle application and consequently available on-demand for use. They are mounted inside application container(s) at a specified mount point which in turn allows the application to access the files inside the AEV filesystem. When an application is stopped or deleted the storage space used by an AEV will be reclaimed such that it can be used by other applications on the cluster. On the other hand, when an application is restarted the requested AEV(s) are provisioned again.

Note

The storage space consumed by an AEV is not reserved by a stopped application. As a result, there is no guarantee that the application will be able to restart given the fact that there might not be enough storage space to satisfy the AEV(s) requirements.

Robin provides multiple ways to provision an application with AEVs; namely AEV(s) can either be configured within the manifest file of a Robin bundle, within the template of an application to be deployed or can be set at runtime using the Robin UI. Each of these methods are described in the sections below.

5.15.1. Configuring Application Ephemeral Volumes

You can only configure the Application Ephemeral Volumes for a Robin Bundle Application.

You can add AEV entries in the manifest of an application, and also you can add them to a bundle template that describes a particular application instance or configuration.

To configure AEV in a Robin bundle, you need to add the following ephemeral_storage section inside of each vnode where you want to mount AEV volumes. For more information about Robin bundles, see

Each ephemeral_storage section inside the Robin bundle can contain zero or more AEV entries. You need to provide various attributes in each ephemeral_storage section to create an AEV and mount it on a mount point in the application.

The following is the example of the ephemeral_storage section that you can add to any Robin bundle application:

"ephemeral_storage": [
                  {
                      "size": 1073741824,
                      "path": "/mnt/scratch",
                      "media": "HDD",
                      "replication": 1,
                      "faultdomain": "host",
                      "compute_storage_affinity": true,
                      "disk_tag": {
                          "key": "aev_storage",
                          "value": "white"
                      }
                  }
              ],

.. Note:: : You must add the ephemeral_storage attributes under the storage attribute in the manifest.yaml file.

The following table provides the details of the Ephemeral storage attributes that you can add in the manifest file of a Robin Bundle Application:

Attribute

Description

size

Use this attribute to set the size of an AEV volume.

Note

The size attribute specifies the logical (usable) size of the AEV volume in bytes. It must be exactly divisible by 1GiB.

path

Provide the mount point where you want the AEV volume to be mounted in the container.

replication

Provide the number of copies of the AEV data set to maintain. You can set 1, 2, or 3. By default, it is 1 (no replica copies).

Note

You cannot not use the compute_storage_affinity and replication attributes together as they are mutually exclusive.

falutdomain

Use this attribute along with the replication attribute to save replicas fault tolerance. The valid choices are host and disk.

Note

The faultdomain attribute is only relevant when the replication factor is 2 or 3.

compute_storage_affinity

This is an optional attribute.

Set this attribute to true to keep compute and storage resources on the same physical node.

Set this attribute to false to keep compute and storage resources on different physical nodes.

Note

You cannot not use the compute_storage_affinity and replication attributes together as they are mutually exclusive.

Note

When you set compute_storage_affinity to true and if there is not enough storage space to honor the size requirements on the node where compute resources are located, Robin CNP relaxes the affinity rules to allow storage space usage from devices on other nodes.

disk_tag

Identifies which disk devices to allocate storage from (optional).

Note

Before you add a disk_tag to an AEV entry, you must first create the tag (key and value) and add the tag to the required number of disks to satisfy size, replication, fault domain, and storage_compute_affinity requirements.

5.15.2. Create Application Ephemeral Volumes from Robin UI

The Robin UI contains full support for provisioning AEVs. When creating an application from a bundle, there is an option of adding one or more AEVs for each role defined in the application manifest. If the required AEVs are already specified within the manifest file, the AEV section automatically appears populated with the preconfigured AEVs. However, these can be modified to suit the needs of the application deployment at. runtime. On the other hand if no AEVs have been specified within the manifest, the APPLICATION EPHEMERAL STORAGE option must be enabled and the respective AEV(s) configured in order for them to be provisioned alongside the application.

Note

A maximum of 10 AEVs per application can be provisioned via the Robin UI.

5.15.3. AEV Snapshots and Backups

The Application Ephemeral Volumes are not included in application snapshots and backups. However, the AEVs are part of the application specification that is captured as metadata with each snapshot and backup.

When you create a clone of an application from an application snapshot or new application from the application backup, Robin CNP creates a new volume for each configured AEV

Note

When creating a clone or new application from a snapshot or backup that contains AEVs, check if the disk_tag is specified for any of the AEVs. If the disk_tag is specified, ensure there is are an adequate number of tagged disks to satisfy the storage requirement for the new application instance. If it is not specified, you might need to make necessary modifications to the tag or remove it entirely before deploying the new application instance.

5.16. Auto Disk Rebalance

Robin supports an Auto Disk Rebalance feature which manages the storage space of all disks in order to ensure that the space used on any disk never exceeds a certain threshold. This feature is enabled by default, but can be disabled.

5.16.1. How it works

The storage space on a disk is used by one or more volumes. Over time, some disks in the cluster might use storage space to their maximum threshold capacity, while other disks might still have free space.

Robin Central Manager (RCM) continuously collects the physical usage of all disks in the cluster from the Storage Manager (SM) and updates its database. When a disk reaches a high watermark, the disk rebalance job automatically starts to move the volumes from the high watermark disk to the below low watermark disk.

SM selects the volume slices from a high watermark disk and moves these slices to the below low watermark disk. The target disk is chosen based on the following scenarios:

  • The current physical usage of the disk is below the low watermark threshold.

  • The physical usage after the volume slices movement must not exceed the low watermark threshold.

5.16.2. Watermark threshold

The watermark threshold is an attribute that enables you to set the maximum and minimum limits on the disk’s space usage.

There are two types of watermarks:

  • High watermark - the maximum threshold of the used disk space that, when exceeded, automatically triggers the disk rebalancing. The default value is 90%.

  • Low watermark - the minimum threshold of the used disk space that, when reached, automatically stops the disk rebalancing. The default value is 80%.

Note

The low watermark value cannot be greater than or equal to the high watermark value.

5.16.3. Configure high watermark for disk

You can configure the high watermark threshold percentage value for a disk other than the default value (90%) by updating the high watermark attribute in your cluster.

To configure the high watermark threshold value, run the following command:

# robin config update server disk_used_space_high_watermark <value>

Example

# robin config update server disk_used_space_high_watermark 90
The 'server' attribute 'disk_used_space_high_watermark' has been updated

5.16.4. Configure low watermark for disk

You can configure the low watermark threshold percentage value for a disk other than the default value (80%) by updating the low watermark attribute in your cluster.

To configure the low watermark threshold value, run the following command:

# robin config update server disk_used_space_low_watermark <value>

Example

# robin config update server disk_used_space_low_watermark 80
The 'server' attribute 'disk_used_space_low_watermark' has been updated

5.16.5. Disable Auto Disk Rebalance

As an administrator, you can disable the Auto Disk Rebalance feature when you want to manage the storage space and disks manually. To disable the Auto Disk Rebalance feature, you need to set the value of the high watermark attribute to 100%.

To disable the Auto Disk Rebalance feature, run the following command:

# robin config update server disk_used_space_high_watermark 100
The 'server' attribute 'disk_used_space_high_watermark' has been updated

Note

By default, the Auto Disk Rebalance feature is enabled.

5.17. Thin Provisioning

Robin supports thin provisioning of the Volumes so that large storage volumes can be created without upfront physical capacity. Thin provisioning is disabled by default and can be enabled on a cluster by updating the storage_provision_factor config attribute to value greater than 1.

However precaution must be taken when thin provisioning is enabled especially if there exists application(s) on the cluster that write data to disk resulting in physical space usage. This is because out of space issues may be encountered by said application(s) due to the aforementioned writes. For example, a drive of size 100GB can host 2 volumes with a total size of 200GB when thin provisioning is enabled but if the combined size of the data written to those volumes goes above 100GB, the volumes would see ERR_NOSPACE errors and filesystem would be moved to read-only state. As a result, thin provsioned clusters must be actively managed.

5.17.1. View current Thin Provisioning factor

In order to check the current value for storage_provision_factor config attribute and also the current storage capacity on the cluster, run the following commands. Note that the max amount that can be allocated from the drives is roughly 80% of their capacity given the value of the storage_provision_factor.

$ robin config list server storage_provision_factor
Section | Attribute                | Value
--------+--------------------------+-------
server  | storage_provision_factor | 0.8

$ robin drive list
ID | WWN                                       | Host        | Path /dev/disk/by-id                          | Size(GB) | Movable | Type | Free/Max(GB) | Vols | Role    | Status | LastOpr | Write Unit
---+-------------------------------------------+-------------+-----------------------------------------------+----------+---------+------+--------------+------+---------+--------+---------+------------
1  | 0xQEMU_QEMU_HARDDISK_4a92cb43-9402-417a-9 | vnode-95-30 | scsi-0QEMU_QEMU_HARDDISK_4a92cb43-9402-417a-9 | 50       | N       | HDD  | 38/38 (100%) | 0/10 | Storage | ONLINE | READY   | 512
3  | 0xQEMU_QEMU_HARDDISK_276268ff-b7e4-4a9c-a | vnode-95-39 | scsi-0QEMU_QEMU_HARDDISK_276268ff-b7e4-4a9c-a | 50       | N       | HDD  | 38/38 (100%) | 0/10 | Storage | ONLINE | READY   | 512
2  | 0xQEMU_QEMU_HARDDISK_e85533b3-da77-42ba-9 | vnode-95-96 | scsi-0QEMU_QEMU_HARDDISK_e85533b3-da77-42ba-9 | 50       | N       | HDD  | 30/38 (79%)  | 1/10 | Storage | ONLINE | READY   | 512

5.17.2. Update Thin Provisioning factor

In order to update the thin provisioning factor and validate that the max amount that can be allocated. from the drives has been adjusted accordingly, run the following commands. Note that the max allocatable amount is roughly 200% of each drive’s respective capacity given the value of the storage_provision_factor.

Note

When the storage_provision_factor attribute has a value of greater than 1, thin provisioning is enabled. In the case that it is lower than 1, thin provisioning is disabled.

$ robin config update server storage_provision_factor 2
The 'server' attribute 'storage_provision_factor' has been updated

$ robin config list server storage_provision_factor
Section | Attribute                | Value
--------+--------------------------+-------
server  | storage_provision_factor | 2

$ robin drive list
ID | WWN                                       | Host        | Path /dev/disk/by-id                          | Size(GB) | Movable | Type | Free/Max(GB) | Vols | Role    | Status | LastOpr | Write Unit
---+-------------------------------------------+-------------+-----------------------------------------------+----------+---------+------+--------------+------+---------+--------+---------+------------
1  | 0xQEMU_QEMU_HARDDISK_4a92cb43-9402-417a-9 | vnode-95-30 | scsi-0QEMU_QEMU_HARDDISK_4a92cb43-9402-417a-9 | 50       | N       | HDD  | 96/96 (100%) | 0/10 | Storage | ONLINE | READY   | 512
3  | 0xQEMU_QEMU_HARDDISK_276268ff-b7e4-4a9c-a | vnode-95-39 | scsi-0QEMU_QEMU_HARDDISK_276268ff-b7e4-4a9c-a | 50       | N       | HDD  | 96/96 (100%) | 0/10 | Storage | ONLINE | READY   | 512
2  | 0xQEMU_QEMU_HARDDISK_e85533b3-da77-42ba-9 | vnode-95-96 | scsi-0QEMU_QEMU_HARDDISK_e85533b3-da77-42ba-9 | 50       | N       | HDD  | 88/96 (92%)  | 1/10 | Storage | ONLINE | READY   | 512

Only storage disks are shown. Issue `robin disk list --role all` to view all disks

5.18. Storage Metrics

By default, Robin exposes certain storage metrics that can be scraped and manipulated by end users to suit their needs. These metrics cover a wide variety of objects, ranging from device based statistics such as their usage to details on individual application read and write operations on a volume. Each section detailed below is a grouping of related metrics with a description of each metric attached.

5.18.1. Disk based Metrics

Listed below are several disk related metrics that are exposed by Robin:

Metric

Description

robin_disk_freeslices

The number of free slices on a disk for a new storage request from an application. Each slice is 1GB in size.

robin_disk_maxsegs

The number of segments in a disk. Each segment is 32MB in size.

robin_disk_nsegs

The number of used segments in a disk.

robin_disk_rawused

The used space of a disk, represented in bytes.

robin_disk_size

The capacity of a disk, represented in bytes.

robin_disk_freespace

The free space on a disk that can be used for allocations, represented in bytes.

robin_disk_maxvols

The maximum number of Kubernetes volumes that can be stored in a disk. The default size is 10.

robin_disk_nvols

The number of Kubernetes volumes currently stored on a disk.

robin_disk_freevols

The number of additional Kubernetes volumes that can be accommodated on a disk.

robin_disk_maxslices

The maximum number of slices that can be accommodated on a disk.

robin_disk_nslices

The number of used slices of a disk.

5.18.2. Read/Write based Metrics

Listed below are several read and write based metrics related to volumes that are exposed by Robin:

Metric

Description

robin_rdvm_vol_curr_bytes_read

The number of bytes read from a volume.

robin_rdvm_vol_curr_bytes_written

The number of bytes written to a volume.

robin_rdvm_vol_curr_read_ios

The number of read requests on a volume.

robin_rdvm_vol_curr_read_qdepth

The number of pending reads in the queue for a volume.

robin_rdvm_vol_curr_read_usecs

The cumulative time in microseconds taken for all read requests on a volume.

robin_rdvm_vol_curr_write_ios

The number of write requests to a volume.

robin_rdvm_vol_curr_write_qdepth

The number of pending writes in the queue for a volume.

robin_rdvm_vol_curr_write_usecs

The cumulative time in microseconds taken for all write requests on a volume.

Listed below are several read and write based metrics related to disks that are exposed by Robin:

Metric

Description

robin_rdvm_dev_curr_bytes_read

The number of bytes read from a specified storage device for a specified volume.

robin_rdvm_dev_curr_bytes_written

The number of bytes written to a specified storage device for a specified volume.

robin_rdvm_dev_curr_read_ios

The number of reads from a specified storage device for a specified volume.

robin_rdvm_dev_curr_read_qdepth

The number of pending reads in the queue on a specified storage device for a specified volume.

robin_rdvm_dev_curr_read_usecs

The cumulative time in microseconds taken by all read requests on a specified storage device for a specified volume.

robin_rdvm_dev_curr_write_ios

The number of writes to the specified storage device for a specified volume.

robin_rdvm_dev_curr_write_qdepth

The number of pending writes in the queue on a specified storage device for a specified volume.

robin_rdvm_dev_curr_write_usecs

The cumulative time in microseconds taken by all write requests on a specified storage device for a specified volume.

Listed below are several read and write based metrics related to applications that are exposed by Robin:

Metric

Description

robin_rio_vol_curr_bytes_read

The number of bytes that an application reads from a volume.

robin_rio_vol_curr_bytes_written

The number of bytes that an application wrote to a volume.

robin_rio_vol_curr_read_ios

The number of the read requests that an application performed on a volume.

robin_rio_vol_curr_read_qdepth

The number of the pending read requests from an application on a volume.

robin_rio_vol_curr_read_usecs

The cumulative time in microseconds taken for all read requests on a volume.

robin_rio_vol_curr_write_ios

The number of writes requested by an application on a volume.

robin_rio_vol_curr_write_qdepth

The number of pending writes from an application on a volume.

robin_rio_vol_curr_write_usecs

The cumulative time in microseconds taken for all write requests on a volume.

5.18.3. Snapshot based Metrics

Listed below are several snapshot related metrics that are exposed by Robin:

Metric

Description

robin_snap_dsegs

The number of on-disk segments written by a snapshot.

robin_snap_nclones

The number of clones of a snapshot.

robin_snap_osegs

The total number of segments owned by a snapshot.

robin_snap_state

The snapshot state of a volume.

5.18.4. Volume based Metrics

Listed below are several volume related metrics that are exposed by Robin:

Metric

Description

robin_vol_nsnaps

The number of snapshots of a volume.

robin_vol_rawused

The number of used bytes for a volume.

robin_vol_size

The allocated size of a volume.

5.18.5. Accessing Storage Metrics

All the metrics detailed above can be accessed by running the following command:

# curl -k "https://<ipaddress>:29446/metrics"

The ipaddress parameter showcased above can be one of three options:

  • The Cluster IP associated with the robin-master service. This can be attained by running the command: kubectl get svc -n robinio robin-master.

  • The IP Address of the node which is currently running as the master manager node.

  • The Virtual IP Address associated with cluster. Note this option is only applicable to highly available clusters with multiple manager nodes.

Example

# curl -k "https://172.19.47.240:29446/metrics"
  # Robin storage/server metrics
  robin_server{instance="master"} 1
  robin_disk_freeslices{wwn="0x60022480db8a635ee6764e60418f9ef7",devpath="/dev/sda",node_idx="1", node_name="telxvm-53-150"} 0
  robin_disk_freespace{wwn="0x60022480db8a635ee6764e60418f9ef7",devpath="/dev/sda",node_idx="1", node_name="telxvm-53-150"} 107374182400
  robin_disk_freevols{wwn="0x60022480db8a635ee6764e60418f9ef7",devpath="/dev/sda",node_idx="1", node_name="telxvm-53-150"} 10
  robin_disk_maxsegs{wwn="0x60022480db8a635ee6764e60418f9ef7",devpath="/dev/sda",node_idx="1", node_name="telxvm-53-150"} 0
  robin_disk_maxvols{wwn="0x60022480db8a635ee6764e60418f9ef7",devpath="/dev/sda",node_idx="1", node_name="telxvm-53-150"} 10
  robin_disk_maxslices{wwn="0x60022480db8a635ee6764e60418f9ef7",devpath="/dev/sda",node_idx="1", node_name="telxvm-53-150"} 0
  robin_disk_nsegs{wwn="0x60022480db8a635ee6764e60418f9ef7",devpath="/dev/sda",node_idx="1", node_name="telxvm-53-150"} 0
  robin_disk_nslices{wwn="0x60022480db8a635ee6764e60418f9ef7",devpath="/dev/sda",node_idx="1", node_name="telxvm-53-150"} 0
  robin_disk_nvols{wwn="0x60022480db8a635ee6764e60418f9ef7",devpath="/dev/sda",node_idx="1", node_name="telxvm-53-150"} 0
  robin_disk_size{wwn="0x60022480db8a635ee6764e60418f9ef7",devpath="/dev/sda",node_idx="1", node_name="telxvm-53-150"} 107374182400
  robin_disk_rawused{wwn="0x60022480db8a635ee6764e60418f9ef7",devpath="/dev/sda",node_idx="1", node_name="telxvm-53-150"} 0
  robin_disk_freeslices{wwn="0x600224803f4fe2dbd4d0bca550800af9",devpath="/dev/sdb",node_idx="1", node_name="telxvm-53-150"} 0
  robin_disk_freespace{wwn="0x600224803f4fe2dbd4d0bca550800af9",devpath="/dev/sdb",node_idx="1", node_name="telxvm-53-150"} 107374182400
  robin_disk_freevols{wwn="0x600224803f4fe2dbd4d0bca550800af9",devpath="/dev/sdb",node_idx="1", node_name="telxvm-53-150"} 10
  robin_disk_maxsegs{wwn="0x600224803f4fe2dbd4d0bca550800af9",devpath="/dev/sdb",node_idx="1", node_name="telxvm-53-150"} 0
  robin_disk_maxvols{wwn="0x600224803f4fe2dbd4d0bca550800af9",devpath="/dev/sdb",node_idx="1", node_name="telxvm-53-150"} 10
  robin_disk_maxslices{wwn="0x600224803f4fe2dbd4d0bca550800af9",devpath="/dev/sdb",node_idx="1", node_name="telxvm-53-150"} 0
  robin_disk_nsegs{wwn="0x600224803f4fe2dbd4d0bca550800af9",devpath="/dev/sdb",node_idx="1", node_name="telxvm-53-150"} 0
  robin_disk_nslices{wwn="0x600224803f4fe2dbd4d0bca550800af9",devpath="/dev/sdb",node_idx="1", node_name="telxvm-53-150"} 0
  robin_disk_nvols{wwn="0x600224803f4fe2dbd4d0bca550800af9",devpath="/dev/sdb",node_idx="1", node_name="telxvm-53-150"} 0
  robin_disk_size{wwn="0x600224803f4fe2dbd4d0bca550800af9",devpath="/dev/sdb",node_idx="1", node_name="telxvm-53-150"} 107374182400
  robin_disk_rawused{wwn="0x600224803f4fe2dbd4d0bca550800af9",devpath="/dev/sdb",node_idx="1", node_name="telxvm-53-150"} 0
  robin_disk_freeslices{wwn="0x60022480e965d11b83cd0d074f84fe4f",devpath="/dev/sdc",node_idx="1", node_name="telxvm-53-150"} 0
  robin_disk_freespace{wwn="0x60022480e965d11b83cd0d074f84fe4f",devpath="/dev/sdc",node_idx="1", node_name="telxvm-53-150"} 107374182400
  robin_disk_freevols{wwn="0x60022480e965d11b83cd0d074f84fe4f",devpath="/dev/sdc",node_idx="1", node_name="telxvm-53-150"} 10
  robin_disk_maxsegs{wwn="0x60022480e965d11b83cd0d074f84fe4f",devpath="/dev/sdc",node_idx="1", node_name="telxvm-53-150"} 0
  robin_disk_maxvols{wwn="0x60022480e965d11b83cd0d074f84fe4f",devpath="/dev/sdc",node_idx="1", node_name="telxvm-53-150"} 10
  robin_disk_maxslices{wwn="0x60022480e965d11b83cd0d074f84fe4f",devpath="/dev/sdc",node_idx="1", node_name="telxvm-53-150"} 0
  robin_disk_nsegs{wwn="0x60022480e965d11b83cd0d074f84fe4f",devpath="/dev/sdc",node_idx="1", node_name="telxvm-53-150"} 0
  robin_disk_nslices{wwn="0x60022480e965d11b83cd0d074f84fe4f",devpath="/dev/sdc",node_idx="1", node_name="telxvm-53-150"} 0
  robin_disk_nvols{wwn="0x60022480e965d11b83cd0d074f84fe4f",devpath="/dev/sdc",node_idx="1", node_name="telxvm-53-150"} 0
  robin_disk_size{wwn="0x60022480e965d11b83cd0d074f84fe4f",devpath="/dev/sdc",node_idx="1", node_name="telxvm-53-150"} 107374182400
  robin_disk_rawused{wwn="0x60022480e965d11b83cd0d074f84fe4f",devpath="/dev/sdc",node_idx="1", node_name="telxvm-53-150"} 0
  robin_disk_freeslices{wwn="0x60022480db8a635ee6764e60418f9ef7-centos-root",devpath="/dev/dm-0",node_idx="1", node_name="telxvm-53-150"} 0
  robin_disk_freespace{wwn="0x60022480db8a635ee6764e60418f9ef7-centos-root",devpath="/dev/dm-0",node_idx="1", node_name="telxvm-53-150"} 53687091200
  robin_disk_freevols{wwn="0x60022480db8a635ee6764e60418f9ef7-centos-root",devpath="/dev/dm-0",node_idx="1", node_name="telxvm-53-150"} 10
  robin_disk_maxsegs{wwn="0x60022480db8a635ee6764e60418f9ef7-centos-root",devpath="/dev/dm-0",node_idx="1", node_name="telxvm-53-150"} 0
  robin_disk_maxvols{wwn="0x60022480db8a635ee6764e60418f9ef7-centos-root",devpath="/dev/dm-0",node_idx="1", node_name="telxvm-53-150"} 10
  robin_disk_maxslices{wwn="0x60022480db8a635ee6764e60418f9ef7-centos-root",devpath="/dev/dm-0",node_idx="1", node_name="telxvm-53-150"} 0
  robin_disk_nsegs{wwn="0x60022480db8a635ee6764e60418f9ef7-centos-root",devpath="/dev/dm-0",node_idx="1", node_name="telxvm-53-150"} 0
  robin_disk_nslices{wwn="0x60022480db8a635ee6764e60418f9ef7-centos-root",devpath="/dev/dm-0",node_idx="1", node_name="telxvm-53-150"} 0
  robin_disk_nvols{wwn="0x60022480db8a635ee6764e60418f9ef7-centos-root",devpath="/dev/dm-0",node_idx="1", node_name="telxvm-53-150"} 0
  robin_disk_size{wwn="0x60022480db8a635ee6764e60418f9ef7-centos-root",devpath="/dev/dm-0",node_idx="1", node_name="telxvm-53-150"} 53687091200
  robin_disk_rawused{wwn="0x60022480db8a635ee6764e60418f9ef7-centos-root",devpath="/dev/dm-0",node_idx="1", node_name="telxvm-53-150"} 0
  robin_disk_freeslices{wwn="0x60022480db8a635ee6764e60418f9ef7-centos-swap",devpath="/dev/dm-1",node_idx="1", node_name="telxvm-53-150"} 0
  robin_disk_freespace{wwn="0x60022480db8a635ee6764e60418f9ef7-centos-swap",devpath="/dev/dm-1",node_idx="1", node_name="telxvm-53-150"} 8254390272
  robin_disk_freevols{wwn="0x60022480db8a635ee6764e60418f9ef7-centos-swap",devpath="/dev/dm-1",node_idx="1", node_name="telxvm-53-150"} 10
  robin_disk_maxsegs{wwn="0x60022480db8a635ee6764e60418f9ef7-centos-swap",devpath="/dev/dm-1",node_idx="1", node_name="telxvm-53-150"} 0
  robin_disk_maxvols{wwn="0x60022480db8a635ee6764e60418f9ef7-centos-swap",devpath="/dev/dm-1",node_idx="1", node_name="telxvm-53-150"} 10
  robin_disk_maxslices{wwn="0x60022480db8a635ee6764e60418f9ef7-centos-swap",devpath="/dev/dm-1",node_idx="1", node_name="telxvm-53-150"} 0
  robin_disk_nsegs{wwn="0x60022480db8a635ee6764e60418f9ef7-centos-swap",devpath="/dev/dm-1",node_idx="1", node_name="telxvm-53-150"} 0
  robin_disk_nslices{wwn="0x60022480db8a635ee6764e60418f9ef7-centos-swap",devpath="/dev/dm-1",node_idx="1", node_name="telxvm-53-150"} 0
  robin_disk_nvols{wwn="0x60022480db8a635ee6764e60418f9ef7-centos-swap",devpath="/dev/dm-1",node_idx="1", node_name="telxvm-53-150"} 0
  robin_disk_size{wwn="0x60022480db8a635ee6764e60418f9ef7-centos-swap",devpath="/dev/dm-1",node_idx="1", node_name="telxvm-53-150"} 8254390272
  robin_disk_rawused{wwn="0x60022480db8a635ee6764e60418f9ef7-centos-swap",devpath="/dev/dm-1",node_idx="1", node_name="telxvm-53-150"} 0
  robin_disk_freeslices{wwn="0x60022480db8a635ee6764e60418f9ef7-centos-home",devpath="/dev/dm-2",node_idx="1", node_name="telxvm-53-150"} 0
  robin_disk_freespace{wwn="0x60022480db8a635ee6764e60418f9ef7-centos-home",devpath="/dev/dm-2",node_idx="1", node_name="telxvm-53-150"} 44350570496
  robin_disk_freevols{wwn="0x60022480db8a635ee6764e60418f9ef7-centos-home",devpath="/dev/dm-2",node_idx="1", node_name="telxvm-53-150"} 10
  robin_disk_maxsegs{wwn="0x60022480db8a635ee6764e60418f9ef7-centos-home",devpath="/dev/dm-2",node_idx="1", node_name="telxvm-53-150"} 0
  robin_disk_maxvols{wwn="0x60022480db8a635ee6764e60418f9ef7-centos-home",devpath="/dev/dm-2",node_idx="1", node_name="telxvm-53-150"} 10
  robin_disk_maxslices{wwn="0x60022480db8a635ee6764e60418f9ef7-centos-home",devpath="/dev/dm-2",node_idx="1", node_name="telxvm-53-150"} 0
  robin_disk_nsegs{wwn="0x60022480db8a635ee6764e60418f9ef7-centos-home",devpath="/dev/dm-2",node_idx="1", node_name="telxvm-53-150"} 0
  robin_disk_nslices{wwn="0x60022480db8a635ee6764e60418f9ef7-centos-home",devpath="/dev/dm-2",node_idx="1", node_name="telxvm-53-150"} 0
  robin_disk_nvols{wwn="0x60022480db8a635ee6764e60418f9ef7-centos-home",devpath="/dev/dm-2",node_idx="1", node_name="telxvm-53-150"} 0
  robin_disk_size{wwn="0x60022480db8a635ee6764e60418f9ef7-centos-home",devpath="/dev/dm-2",node_idx="1", node_name="telxvm-53-150"} 44350570496
  robin_disk_rawused{wwn="0x60022480db8a635ee6764e60418f9ef7-centos-home",devpath="/dev/dm-2",node_idx="1", node_name="telxvm-53-150"} 0

5.19. Handling Disruptions

With Robin, highly available applications can be deployed on Kubernetes as Robin can handle failures of drives, rack or hosts automatically. On a Baremetal setup, volumes can be setup with a replication factor of 2 or 3 to ensure that storage is available even if a drive fails. Users can also choose the fault domain to be ‘host’ to protect against node reboots or lost.

However, in a public cloud environment the cloud disks can be detached from one cloud node and reattached to another one. For example, in AWS an EBS volume can be detached on one EC2 host and reattached to a different EC2 host. Same with GCP where a PD can be moved across GCE nodes. If a cloud node (EC2, GCE, Azure VM) is terminated or rebooted, one would want any cloud drive attached to them (EBS, PD, Block) to be moved to the one or more of the remaining healthy nodes automatically. This is not limited to just cloud disks, but also SAN LUNS that are offered to Robin as disks. The SAN LUNS can also be multi-mounted onto multiple nodes or moved around from node to node. User can still choose to replicate volume on public cloud as it takes sometime to detach and attach drives on cloud platforms.

Just having the storage available during a disruption will not help if Kubernetes can not access it from the Pod. For example a Kubernetes StatefulSet serializes the mounting and unmounting of a volume to protect against possible corruptions. Robin utilizes smart detection techniques to ensure that even if a volume is mounted on multiple nodes, it can differentiate the IOs issued from the previous stale mount and the new mount. With this consistency guarantees, Robin enables the Kubernetes StatefulSet to unmount a volume from a dead node and remount it on a healthy node where the Pod is scheduled to run. Robin actively monitors these events to allow for the fast failover of the Pods without user intervention and consequently enables users to reliably deploy highly available stateful applications on Kubernetes.

5.20. Hardware RAID Controllers

5.20.1. Robin Recommendations

Robin recommends not using Redundant Array of Independent Disks (RAID) configurations for the physical storage devices that you plan to format with Robin storage. To understand the reasons for this recommendation, review the explanations presented here. However if RAID usage cannot be avoided, Robin recommends the following:

  • Use the hardware RAID controllers only to set up RAID-1 protection for boot and OS devices. This ensures that the failure of one device does not take the entire node down.

  • To protect Robin volumes against node and rack failures, you must configure replication across the fault domains. All RAID configurations (RAID-0, RAID-1, RAID-5, and RAID-6) protect data only against device failure on the node. They reduce the storage capacity of the cluster, thus increasing the data storage costs.

  • Robín’s replication provides maximum protection against all hardware failures including storage devices, storage nodes, and rack failures.

  • If a RAID controller has a cache, it must have a battery backup. If the cache on the RAID controller does not have a battery backup, Robin recommends disabling the cache as the data on the cache could be lost during a power outage.

5.20.2. Advantages and Disadvantages

The following are the advantages and disadvantages of RAID usage with Robin:

  • RAID-0 - In RAID-0, data is striped across all devices and provides the best performance. However, if one of the physical devices fails, the data on the RAID volume is lost.

  • RAID-1 - In RAID-1, data is mirrored across two devices. It gives the most redundancy. However, the performance of the volume is determined by the performance of the slowest device in the mirror and the data capacity is reduced by half.

  • RAID-5 - In RAID-5, data is striped across “n-1” devices. The controller calculates and writes parity data to the nth device. The RAID-5 configuration can tolerate one device failure and provides good performance. However, if one of the devices fails, the admin has to replace the device immediately and start a RAID rebuild to maintain the redundancy. All devices should be of the same size or the size of the smallest device decides the total usable capacity.

  • RAID-6 - In RAID-6, data is striped across “n-2” devices. The controller calculates and writes parity to two devices. The RAID-6 configuration can tolerate two device failures and provides good performance. However, the disadvantages are the same as RAID-5.

  • The striped RAID configurations (RAID-0, RAID-5, and RAID-6) provide better IO performance. However, RAID-0 does not give data protection, and RAID-5 and RAID-6 configurations reduce the storage capacity.

  • All RAID configurations are managed outside of Robin CNP. Therefore, the administrator must manually manage the device failures and it leads to an increase in operations cost.

  • If you enable hardware RAID, the RAID controller becomes a single point of failure for all the storage devices on the node. If the RAID controller fails, all data on all the storage devices in the node is lost.

Note

For more details on Robin recommendations for RAID controllers, review the points made here.