6. Managing Storage¶
Robin discovers disks attached to the nodes and uses them for providing storage to applications. This applies to local disks, cloud volumes, and SAN stoage that are available to nodes which have the storage role assigned to them. During the process Robin collects all the required metadata about the disks and identifies them as HDD, SSD or NVMes. All the elligible disks (this excludes disks with parititions and/or without WWNs) are marked as Robin storage.
Cloud volumes like EBS volumes on AWS and Persistent disks on GCP are not physically tied to a cloud compute instance.
They can be detached from one cloud compute instance and attached to another instance at any time. Logical Units from a
Storage Array Network can similarly be visible from multiple physical servers in the same storage fabric or network. Robin
assigns a primary host for these devices and ensures that all IO access to a device only happens through one server at a
given time. As with local disks, these devices are automatically discovered by Robin and registered. However they are considered
to be re-attachable given their properties. Robin makes sure these re-attachable disks are always accessible even in the event of node failure.
When a node goes down, Robin detaches the disk from failed node and attaches it to a healthy node so that application data remains accessible.
Note
Whilst installing if it is known that certain disks will not have a WWN, the --set-uuid option can be provided via the installer alongside a list of disks on which there is no WWN. This in turn will result in Robin stamping a UUID on the disk and thus enabling it to be used as Robin storage.
Robin allows multiple operations, detailed below, to be performed on registered disks.
|
Provision and attach a cloud disk |
|
Attach an existing disk |
|
Detach a registered disk |
|
List disks |
|
Display detailed information about a disk |
|
Evacuate allocations off a disk |
|
Remove a disk |
|
Update the attributes of a disk |
|
Unfault a disk |
6.1. Provisioning a disk¶
Robin provides a utility through which it can provision disks of any size, attach them to hosts and discover them automatically on multiple cloud platforms. This enables users to expand the storage available for their cluster with convenience and ease.
Detailed below are the general options for the robin drive create command, followed by specific examples for each supported cloud environment.
# robin disk create <hostname>
--type <type>
--number <number>
--size <size>
--iops <iops>
|
Name of host to attach disk to after creation |
|
Type of disks. Choices for GCP include: pd-ssd, pd-standard. Choices for AWS include: gp2, io1, st1. Choices for Anthos include: independent-persistent. Choices for IBM include general-purpose (3iops-tier), 5iops-tier, 10iops-tier, custom. Choices for AKS include: premium, standard. |
|
Number of disks to be created. The default value is 1 |
|
Size of disk to be created in GB. The default value is 500 GB |
|
IOPs for AWS ‘io1’ disk type, for IBM ‘custom’ disk type. Can be between 100-160000 |
Provisions disks of any size, attaches and discovers them automatically on cloud based nodes.
End Point: /api/v3/robin_server/disks/
Method: POST
URL Parameters: None
Data Parameters:
hostname: <hostname>- This mandatory field within the payload specifies the host to which the provisioned disk should be attached.type: <disk_type>- This mandatory field within the payload specifies the type of disk to be created. Supported types include: pd-ssd, pd-standard (for GCP); gp2, io1, st1 (for AWS); independent-persistent (for Anthos); general-purpose, 5iops-tier, 10iops-tier, custom (for IBM).number: <num_of_disks>- This mandatory field within the payload specifies the number of disks to be created. It should be an integer value.size: <size_of_disk>- This mandatory field within the payload specifies the size of the disks to be created. It should be a string. An example value is ‘200GB’.iops: <iops_of_disk>- Utilizing this parameter results in disks that can handle a maximum number of IOPs equal to <iops_of_disks> being created. Note this parameter is valid for the ‘io1’ disk type for AWS hosts, ‘custom’ disk type for IBM hosts.
Port: RCM Port (default value is 29442)
Headers:
Authorization: <auth_token>: Authorization token to identify which user is sending the request. The token can be acquired from the login API.
Success Response Code: 202
Error Response Code: 500 (Internal Server Error), 404 (Not Found Error), 401 (Unauthorized Error), 400 (Invalid Api Usage Error)
Example Response:
Output
{
"jobid": 196
}
On Google Cloud Platform, you can attach disks to your instance via the UI or Google APIs and have them discovered by Robin by running the below command:
# robin host probe <hostname> --rediscover
In order to verify the disk has been properly detected run the following command:
# robin disk list
Finally in order to format and initialize the disk such that it is available for use by Robin, issue the following command:
# robin host add-role <hostname> storage
Note
The hostname parameter in the example commands above refer the hostname of the node where the disk is attached.
On the other hand you can utilize Robin to provision disks in GCP to use for application deployment. To create 100 GB disk in GCP, run following command:
# robin disk create <hostname> --type <pd-standard | pd-ssd> --size 100
These disks will be attached automatically and auto discovered by Robin so they will be ready to use straightaway.
Note
Due to Robin’s advanced feature to make sure disks are always accessible, it needs the manage disks permission to be selected while deploying cluster on GCP.
On the Google Anthos platform, you can add disks to cluster VMs from vSphere and have them discovered by Robin by running the below command:
# robin host probe <hostname> --rediscover
In order to verify the disk has been properly detected run the following command:
# robin disk list
Finally in order to format and initialize the disk such that it is available for use by Robin, issue the following command:
# robin host add-role <hostname> storage
Note
The hostname parameter in the example commands above refer the hostname of the node where the disk is attached.
On the other hand you can utilize Robin to provision virtual disks to use for application deployment. To create 100 GB disk for Anthos, run following command:
# robin disk create <hostname> --type independent-persistent --size 100
These disks will be attached automatically and auto discovered by Robin so they will be ready to use straightaway.
Note
Due to Robin’s advanced feature to make sure disks are always accessible, it needs credentials, provided via Kubernetes secret, to have all cluster and disk level API privileges.
On AWS, you can attach disks to your EC2 instance via the UI or AWS CLI/APIs and have them discovered by Robin by running the below command:
# robin host probe <hostname> --rediscover
In order to verify the disk has been properly detected run the following command:
# robin disk list
Finally in order to format and initialize the disk such that it is available for use by Robin, issue the following command:
# robin host add-role <hostname> storage
Note
The hostname parameter in the example commands above refer the hostname of the node where the disk is attached.
On the other hand you can utilize Robin to provision disks in AWS to use for application deployment. To create 100 GB disk in AWS, run following command:
# robin disk create <hostname> --type <gp2 | io1 | st1> --size 100
These disks will be attached automatically to the EC2 instances and auto discovered by Robin so they will be ready to use straightaway.
Note
Due to Robin’s advanced feature to make sure disks are always accessible, IAM Profiles associated with the host (or permissions granted to a user) must contain all Volume write and list actions.
On IBM Cloud Platform, you can create and attach disks to your instance via the UI or IBM Cloud APIs and have them discovered by Robin by running the below command:
# robin host probe <hostname> --rediscover
In order to verify the disk has been properly detected run the following command:
# robin disk list
Finally in order to format and initialize the disk such that it is available for use by Robin, issue the following command:
# robin host add-role <hostname> storage
Note
The hostname parameter in the example commands above refer the hostname of the node where the disk is attached.
Alternatively you can utilize Robin to provision and attach disks in IBM Cloud to use for application deployment. To create 100 GB disk in IBM Cloud, run following command:
# robin disk create <hostname> --type <general-purpose | 5iops-tier | 10iops-tier | custom > --size 100
These disks will be attached automatically and auto discovered by Robin so they will be ready to use straightaway.
On Azure, you can attach disks to your Azure Virtual Machines via the UI or Azure CLI/APIs and have them discovered by Robin by running the below command:
$ robin host probe <hostname> --rediscover
In order to verify the disk has been properly detected run the following command:
# robin disk list
Finally in order to format and initialize the disk such that it is available for use by Robin, issue the following command:
# robin host add-role <hostname> storage
Note
The hostname parameter in the example commands above refer the hostname of the node where the disk is attached.
On the other hand you can utilize Robin to provision disks in Azure to use for application deployment. To create 100 GB disk in Azure, run following command:
$ robin disk create <hostname> --type <premium | standard> --size 100
These disks will be attached automatically to the Azure Virtual Machines and auto discovered by Robin so they will be ready to use straightaway.
Note
Due to Robin’s advanced feature to make sure disks are always accessible, the Azure role(s) assigned to the user, whom the deployed virtual machines are associated with, must allow for management of data disks.
6.2. Attaching a disk¶
If a disk was previously detached from a host due to the host undergoing decommissioning or to rebalance the storage available on the cluster one will need to attach the disk back to another host in the cluster. Issue the following command to do so:
Note
This command is only for supported for re-attachable disks. Moreover if a host became unreachable Robin will automatically detach the disk and choose a new server to reattach it to based on the accessibility of the disk and the load on the other servers.
# robin disk attach <wwn>
--hostname <hostname>
--force
|
WWN of disk to attach |
|
Name of host to attach disk to. Note this is a mandatory parameter. |
|
Forcibly re-attach a disk that is already ONLINE |
Example:
# robin disk detach 0xQEMU_QEMU_HARDDISK_3c71c872-fe13-4fd5-b --wait
Job: 187 Name: DiskAttach State: VALIDATED Error: 0
Job: 187 Name: DiskAttach State: WAITING Error: 0
Job: 187 Name: DiskAttach State: FINALIZED Error: 0
Job: 187 Name: DiskAttach State: COMPLETED Error: 0
Attaches a disk, which might have previously been detached due to the host undergoing decommissioning or a rebalancing of storage disks, to a particular host.
End Point: /api/v3/robin_server/disks/<disk_wwn>
Method: PUT
URL Parameters: None
Data Parameters:
action: attach- This mandatory field within the payload specifies that the attach operation is to be performed.hostname: <hostname>- Utilizing this parameter results in the disk being attached to the specified host.force: true- Utilizing this parameter enables one to force re-attach an ONLINE disk.
Port: RCM Port (default value is 29442)
Headers:
Authorization: <auth_token>: Authorization token to identify which user is sending the request. The token can be acquired from the login API.
Success Response Code: 202
Error Response Code: 500 (Internal Server Error), 404 (Not Found Error), 401 (Unauthorized Error), 400 (Invalid Api Usage Error)
Example Response:
Output
{
"jobid": 128
}
6.3. Detaching a disk¶
If a physical host is temporarily or permanently taken down and one can detach its storage disks to ensure future access. Issue the following command to do so:
Note
This command is only for supported for re-attachable disks.
# robin disk detach <wwn>
--hostname <hostname>
--force
|
WWN of disk to detach |
|
Name of host to which disk is attached to. Note this is an optional parameter |
|
Forcibly detach a disk that is currently ONLINE |
Example:
# robin disk detach 0xQEMU_QEMU_HARDDISK_3c71c872-fe13-4fd5-b --wait
Job: 187 Name: DiskDetach State: VALIDATED Error: 0
Job: 187 Name: DiskDetach State: WAITING Error: 0
Job: 187 Name: DiskDetach State: FINALIZED Error: 0
Job: 187 Name: DiskDetach State: COMPLETED Error: 0
Detaches a disk from a host.
End Point: /api/v3/robin_server/disks/<disk_wwn>
Method: PUT
URL Parameters: None
Data Parameters:
action: detach- This mandatory field within the payload specifies that the detach operation is to be performed.hostname: <hostname>- Utilizing this parameter results in the disk being detached from the specified host. This is optional as the host to which the disk is attached can be discovered implicitly.force: true- Utilizing this parameter enables one to force detach an ONLINE disk.
Port: RCM Port (default value is 29442)
Headers:
Authorization: <auth_token>: Authorization token to identify which user is sending the request. The token can be acquired from the login API.
Success Response Code: 202
Error Response Code: 500 (Internal Server Error), 404 (Not Found Error), 401 (Unauthorized Error), 400 (Invalid Api Usage Error)
Example Response:
Output
{
"jobid": 130
}
6.4. Listing all disks¶
In order to view all disks currently present on the cluster and some additional details such as the number of volumes allocated from each, their size, media type etc, issue the following command:
# robin disk list --host <hostname>
--role <role>
--media <media>
--reattachable
--eligible
--tags
--json
--sort <sort>
|
Filter list by hostname |
|
Filter list by role. Valid choices include: all, storage, rootdisk, and reserved |
|
Filter list by media type. Valid choices include: HDD and SSD |
|
Filter list to only display reattachable disks |
|
Filter list to only display disks that have free capacity and have not reached the maximum volume count |
|
Display tags for each disk |
|
Output in JSON |
|
Sort the output based on the column. Valid choices include: ID, WWN, Host, Rpool, Size, Allocated, and Free |
Example:
# robin disk list
ID | WWN | Host | RPool | Path /dev/disk/by-id | Size(GB) | Movable | Type | Free/Max(GB) | Vols | Role | Status | LastOpr | Write Unit
---+-------------------------------------------+---------+---------+-----------------------------------------------+----------+---------+------+--------------+------+---------+--------+---------+---------
1 | 0xQEMU_QEMU_HARDDISK_3c71c872-fe13-4fd5-b | vnode36 | default | scsi-0QEMU_QEMU_HARDDISK_3c71c872-fe13-4fd5-b | 50 | N | HDD | 38/38 (100%) | 0/10 | Storage | ONLINE | READY | 512
2 | 0xQEMU_QEMU_HARDDISK_f12b1f33-8b71-4a8c-a | vnode36 | default | scsi-0QEMU_QEMU_HARDDISK_f12b1f33-8b71-4a8c-a | 50 | N | HDD | 28/38 (74%) | 1/10 | Storage | ONLINE | READY | 512
3 | 0xQEMU_QEMU_HARDDISK_e54d6149-0a4e-48ce-b | vnode88 | default | scsi-0QEMU_QEMU_HARDDISK_e54d6149-0a4e-48ce-b | 100 | N | HDD | 77/77 (100%) | 0/10 | Storage | ONLINE | READY | 4096
4 | 0xQEMU_QEMU_HARDDISK_89fc0488-2050-4f44-a | vnode88 | default | scsi-0QEMU_QEMU_HARDDISK_89fc0488-2050-4f44-a | 100 | N | HDD | 63/77 (82%) | 2/10 | Storage | ONLINE | READY | 4096
5 | 0xQEMU_QEMU_HARDDISK_19f9ac67-5e7e-4f00-8 | vnode89 | default | scsi-0QEMU_QEMU_HARDDISK_19f9ac67-5e7e-4f00-8 | 100 | N | HDD | 77/77 (100%) | 0/10 | Storage | ONLINE | READY | 4096
6 | 0xQEMU_QEMU_HARDDISK_d523b7f2-eba7-4edc-b | vnode89 | default | scsi-0QEMU_QEMU_HARDDISK_d523b7f2-eba7-4edc-b | 100 | N | HDD | 77/77 (100%) | 0/10 | Storage | ONLINE | READY | 4096
Only storage disks are shown. Issue `robin disk list --role all` to view all disks
Returns all disks currently present on the cluster and some additional details such as the number of volumes allocated from each, their size, and media type.
End Point: /api/v5/robin_server/disks
Method: GET
URL Parameters:
details=tags: Utilizing this parameter results in tag information for each disk being present in the response payload.host=<hostname>: Utilizing this parameter results in only disks attached to the specified host being returned.
Data Parameters: None
Port: RCM Port (default value is 29442)
Headers:
Authorization: <auth_token>: Authorization token to identify which user is sending the request. The token can be acquired from the login API.
Success Response Code: 200
Error Response Code: 500 (Internal Server Error)
Example Response:
Output
{
"items":[
{
"node_hostname":"cscale-82-140.robinsystems.com",
"spf":0.8,
"state":"READY",
"type":"HDD",
"nvols":3,
"dev":"\/dev\/sdb",
"aslices":7,
"stormgrid":1,
"pused":234881024,
"nodeid":1,
"maintenance_mode":"DISABLED",
"availability_zone":null,
"max_alloc_slices":77,
"role":"Storage",
"wwn":"0x600224804c48fd7e16c608dea0919064",
"status":"ONLINE",
"free_alloc_slices":68,
"lused_size":0,
"devpath":"\/dev\/disk\/by-id\/scsi-3600224804c48fd7e16c608dea0919064",
"alloc_slices":9,
"reattachable":0,
"max_volumes_per_disk":10,
"protected":0,
"slices":6390,
"capacity":107374182400,
"rpool": "default",
"pfree":104287174656
},
{
"node_hostname":"cscale-82-140.robinsystems.com",
"spf":0.8,
"state":"READY",
"type":"HDD",
"nvols":1,
"dev":"\/dev\/sdc",
"aslices":20,
"stormgrid":2,
"pused":939524096,
"nodeid":1,
"maintenance_mode":"DISABLED",
"availability_zone":null,
"max_alloc_slices":77,
"role":"Storage",
"wwn":"0x600224803bcdafde95b1f5cd27ceb5fb",
"status":"ONLINE",
"free_alloc_slices":53,
"lused_size":0,
"devpath":"\/dev\/disk\/by-id\/scsi-3600224803bcdafde95b1f5cd27ceb5fb",
"alloc_slices":24,
"reattachable":0,
"max_volumes_per_disk":10,
"protected":0,
"slices":6390,
"capacity":107374182400,
"rpool": "default",
"pfree":103582531584
},
{
"node_hostname":"cscale-82-140.robinsystems.com",
"spf":0.8,
"state":"INIT",
"type":"HDD",
"nvols":0,
"dev":"\/dev\/dm-1",
"aslices":0,
"stormgrid":0,
"pused":0,
"nodeid":1,
"maintenance_mode":"DISABLED",
"availability_zone":null,
"max_alloc_slices":5,
"role":"RootDisk",
"wwn":"0x600224801d3ac9b6650afd3280aa5898-centos-swap",
"status":"UNKNOWN",
"free_alloc_slices":5,
"lused_size":0,
"devpath":"\/dev\/disk\/by-id\/dm-uuid-LVM-vI83PDTxV3H0dWyAXfH5ef7rxTOuYyphaFy4aq3EUo1yluonS8FG0LF16ycBrdEw",
"alloc_slices":0,
"reattachable":0,
"max_volumes_per_disk":10,
"protected":0,
"slices":0,
"capacity":8254390272,
"rpool": "default",
"pfree":0
},
{
"node_hostname":"cscale-82-140.robinsystems.com",
"spf":0.8,
"state":"INIT",
"type":"HDD",
"nvols":0,
"dev":"\/dev\/dm-0",
"aslices":0,
"stormgrid":0,
"pused":0,
"nodeid":1,
"maintenance_mode":"DISABLED",
"availability_zone":null,
"max_alloc_slices":38,
"role":"RootDisk",
"wwn":"0x600224801d3ac9b6650afd3280aa5898-centos-root",
"status":"UNKNOWN",
"free_alloc_slices":38,
"lused_size":0,
"devpath":"\/dev\/disk\/by-id\/dm-uuid-LVM-vI83PDTxV3H0dWyAXfH5ef7rxTOuYyphgpZcvqGdfOKaXbEbOZzNthc6btsoSXDj",
"alloc_slices":0,
"reattachable":0,
"max_volumes_per_disk":10,
"protected":0,
"slices":0,
"capacity":53687091200,
"rpool": "default",
"pfree":0
},
{
"node_hostname":"cscale-82-140.robinsystems.com",
"spf":0.8,
"state":"INIT",
"type":"HDD",
"nvols":0,
"dev":"\/dev\/dm-2",
"aslices":0,
"stormgrid":0,
"pused":0,
"nodeid":1,
"maintenance_mode":"DISABLED",
"availability_zone":null,
"max_alloc_slices":32,
"role":"RootDisk",
"wwn":"0x600224801d3ac9b6650afd3280aa5898-centos-home",
"status":"UNKNOWN",
"free_alloc_slices":32,
"lused_size":0,
"devpath":"\/dev\/disk\/by-id\/dm-uuid-LVM-vI83PDTxV3H0dWyAXfH5ef7rxTOuYyphQObDlS6eMUSpSxH5zsvyg9I5a0Gpuj5W",
"alloc_slices":0,
"reattachable":0,
"max_volumes_per_disk":10,
"protected":0,
"slices":0,
"capacity":44350570496,
"rpool": "default",
"pfree":0
},
{
"node_hostname":"cscale-82-140.robinsystems.com",
"spf":0.8,
"state":"INIT",
"type":"HDD",
"nvols":0,
"dev":"\/dev\/sda",
"aslices":0,
"stormgrid":0,
"pused":0,
"nodeid":1,
"maintenance_mode":"DISABLED",
"availability_zone":null,
"max_alloc_slices":77,
"role":"RootDisk",
"wwn":"0x600224801d3ac9b6650afd3280aa5898",
"status":"UNKNOWN",
"free_alloc_slices":77,
"lused_size":0,
"devpath":"\/dev\/disk\/by-id\/scsi-3600224801d3ac9b6650afd3280aa5898",
"alloc_slices":0,
"reattachable":0,
"max_volumes_per_disk":10,
"protected":0,
"slices":0,
"capacity":107374182400,
"rpool": "default",
"pfree":0
}
]
}
6.5. Show information about a specific disk¶
In order to view detailed information about a disk such as the breakdown of its available capacity, current allocations including the associated applications, write unit etc, issue the following command:
# robin disk info <wwn>
--json
|
WWN of disk to detach |
|
Output in JSON |
Example:
# robin disk info 0xQEMU_QEMU_HARDDISK_80cb8b56-aef2-48a8-8
Drive: 0xQEMU_QEMU_HARDDISK_80cb8b56-aef2-48a8-8
Id: 2
Role: Storage
Type: HDD
Make: None
Model: None
Write Unit: 512
Availability Zone: None
Zone Id: 1613675775
Path: /dev/disk/by-id/scsi-0QEMU_QEMU_HARDDISK_80cb8b56-aef2-48a8-8
Node: vnode-95-28.robinsystems.com
Protected: No
Reattachable: No
State: READY
Status: MAINTENANCE
Maintenance: ON
Allocations:
Current: 15G
Maximum: 38G
Free: 23G
Capacity: 50G
Factor: 30%
AllocScore: 83
Physical Usage: 0.16G
Physical Free: 48G
Volumes: 3
Max volumes: 10
Max latency volumes: 2
Max throughput volumes: 1
Applications: 3
file-collection-1613696655681
Instance: file-collection-1613696655681.b9307d9d-f1fe-4731-a815-99800f3811a2
Id | Volume Name | Workload | Size (GB) | Allocated (GB)
--------+--------------------------------------------------------------------+----------+----------------+----------------
1 | file-collection-1613696655681.b9307d9d-f1fe-4731-a815-99800f3811a2 | ordinary | 5 | 5
pvc-9369013d-cba4-41a9-b9b8-15228e5ea09d
Instance: pvc-9369013d-cba4-41a9-b9b8-15228e5ea09d
Id | Volume Name | Workload | Size (GB) | Allocated (GB)
---------+------------------------------------------+----------+----------------+----------------
21 | pvc-9369013d-cba4-41a9-b9b8-15228e5ea09d | ordinary | 5 | 5
Returns detailed information about a disk such as the breakdown of its available capacity, current allocations including the associated applications and write unit.
End Point: /api/v3/robin_server/disks/<wwn>
Method: GET
URL Parameters: None
Data Parameters: None
Port: RCM Port (default value is 29442)
Headers:
Authorization: <auth_token>: Authorization token to identify which user is sending the request. The token can be acquired from the login API.
Success Response Code: 200
Error Response Code: 500 (Internal Server Error), 404 (Not Found Error), 401 (Unauthorized Error)
Example Response:
Output
{
"items":[
{
"wwn":"0xQEMU_QEMU_HARDDISK_80cb8b56-aef2-48a8-8",
"dev":"\/dev\/sdb",
"make":null,
"model":null,
"devpath":"\/dev\/disk\/by-id\/scsi-0QEMU_QEMU_HARDDISK_80cb8b56-aef2-48a8-8",
"capacity":53687091200,
"type":"HDD",
"role":"Storage",
"state":"READY",
"status":"MAINTENANCE",
"maintenance_mode":"ON",
"alloc_score":83,
"slices":3190,
"aslices":15,
"pused":167772160,
"pfree":51774488576,
"nodeid":1,
"stormgrid":2,
"reattachable":0,
"protected":0,
"availability_zone":null,
"node_hostname":"vnode-95-28.robinsystems.com",
"reattachable_nodes":[
[
"vnode-95-28.robinsystems.com",
"ONLINE"
]
],
"reattachpolicy":{
"id":2,
"burst_start_time":0,
"burst_count":0,
"burst_interval":600,
"restart_limit":5,
"restarts_done":0
},
"zoneid":1613675775,
"allocations":[
{
"name":"file-collection-1613696655681.b9307d9d-f1fe-4731-a815-99800f3811a2.0.e4623953-263e-4b91-b66b-6be69ab60018",
"slices":5,
"volume_group":"file-collection-1613696655681.b9307d9d-f1fe-4731-a815-99800f3811a2.72.1.d9d9726c-8052-48b1-b7ee-b953dbe254ff",
"vols":[
{
"id":"1",
"name":"file-collection-1613696655681.b9307d9d-f1fe-4731-a815-99800f3811a2",
"size":5368709120,
"state":"ONLINE",
"media":"HDD",
"workload_str":"ordinary"
}
],
"volume":{
"id":"1",
"name":"file-collection-1613696655681.b9307d9d-f1fe-4731-a815-99800f3811a2",
"size":5368709120,
"state":"ONLINE",
"media":"HDD",
"workload_str":"ordinary"
}
},
{
"name":"pvc-9369013d-cba4-41a9-b9b8-15228e5ea09d.0",
"slices":5,
"volume_group":"pvc-9369013d-cba4-41a9-b9b8-15228e5ea09d.72.1",
"vols":[
{
"id":"21",
"name":"pvc-9369013d-cba4-41a9-b9b8-15228e5ea09d",
"size":5368709120,
"state":"ONLINE",
"media":"HDD",
"workload_str":"ordinary"
}
],
"volume":{
"id":"21",
"name":"pvc-9369013d-cba4-41a9-b9b8-15228e5ea09d",
"size":5368709120,
"state":"ONLINE",
"media":"HDD",
"workload_str":"ordinary"
}
},
{
"name":"pvc-71561296-4699-4b79-bf92-3dd5470929cb.0",
"slices":5,
"volume_group":"pvc-71561296-4699-4b79-bf92-3dd5470929cb.72.1",
"vols":[
{
"id":"22",
"name":"pvc-71561296-4699-4b79-bf92-3dd5470929cb",
"size":5368709120,
"state":"ONLINE",
"media":"HDD",
"workload_str":"throughput"
}
],
"volume":{
"id":"22",
"name":"pvc-71561296-4699-4b79-bf92-3dd5470929cb",
"size":5368709120,
"state":"ONLINE",
"media":"HDD",
"workload_str":"throughput"
}
}
],
"tags":{
},
"max_volumes_per_disk":10,
"max_latency_sensitive_vols_per_disk":2,
"max_throughput_intensive_vols_per_disk":1,
"nvols":3,
"lused_size":0,
"preserved":0,
"write_unit":512
}
]
}
6.6. Evacuating volumes from a disk¶
This command allows users to move Robin storage volumes from one disk to another. As a result, it can be utilized to free up space on a disk when it is getting full, or to move data out of a disk before it is decommissioned. To evacuate a volume, issue the following command:
Note
Robin’s placement algorithm will determine the best disks to evacuate volumes to if target disks are not specified.
# robin disk evacuate <wwn>
--volume <volume>
--to-disks <target_disks>
--exclude-disks <excluded_disks>
--freespace <space_to_free>
|
WWN of disk to evacuate |
|
Name of volume to be evacuated. Note if not provided all the volumes will be evacuated |
|
List of WWNs representing disks to evacuate to |
|
List of WWNs representing disks to avoid evacuating to |
|
Do not prompt the user for confirmation of evacuation |
|
Evacuate a partial amount of space from a volume to free up space from a disk. |
Use the --freespace option to provide the amount of space you need to free up on a volume.
You can provide the source disk’s WWN and the destination disk’s WWN along with the volume name.
If you do not provide the destination disk’s WWN, Robin CNS will automatically move the data to the available disk. You can free up space in units of GB or TB.
Example:
# robin disk evacuate 0xQEMU_QEMU_HARDDISK_3c71c872-fe13-4fd5-b --wait --yes
Job: 65 Name: DiskEvacuate State: VALIDATED Error: 0
Job: 65 Name: DiskEvacuate State: WAITING Error: 0
Job: 65 Name: DiskEvacuate State: COMPLETED Error: 0
Evacuates volumes residing on one disk to another.
End Point: /api/v3/robin_server/disks/<disk_wwn>
Method: PUT
URL Parameters: None
Data Parameters:
action: evacuate- This mandatory field within the payload specifies that the evacuate operation is to be performed.volume: <volume_name>- Utilizing this parameter results in only the specified volume being evacuated from the disk. If this is not specified all volumes on the disk are evacuated.target_drives: <list_of_target_drives>- Utilizing this parameter, by specifying a list of WWNs representing respective target drives, ensures the evacuated volumes will be transferred to one of the specified disks.exclude_drives: <list_of_excluded_drives>- Utilizing this parameter, by specifying a list of WWNs representing respective excluded drives, ensures the evacuated volumes will not transferred to one of the specified disks.
Port: RCM Port (default value is 29442)
Headers:
Authorization: <auth_token>: Authorization token to identify which user is sending the request. The token can be acquired from the login API.
Success Response Code: 202
Error Response Code: 500 (Internal Server Error), 404 (Not Found Error), 401 (Unauthorized Error), 400 (Invalid Api Usage Error)
Example Response:
Output
{
"jobid": 145
}
6.7. Unregistering a disk¶
In order to unregister disk from the Robin cluster due to the fact it has physically malfunctioned or it is faulted, issue the following command:
Note
A disk can only be unregistered if there are no storage volumes allocated from it and it is still attached to the node from the perspective of the platform.
# robin disk unregister <wwn>
--yes
|
WWN of disk to unregister |
|
Do not prompt the user for confirmation of disk unregistration |
Example:
# robin disk unregister 0xQEMU_QEMU_HARDDISK_3c71c872-fe13-4fd5-b --wait --yes
Job: 101 Name: DiskUnregister State: VALIDATED Error: 0
Job: 101 Name: DiskUnregister State: WAITING Error: 0
Job: 101 Name: DiskUnregister State: COMPLETED Error: 0
Unregisters a disk from the Robin cluster.
End Point: /api/v3/robin_server/disks/<disk_wwn>
Method: DELETE
URL Parameters: None
Data Parameters: None
Port: RCM Port (default value is 29442)
Headers:
Authorization: <auth_token>: Authorization token to identify which user is sending the request. The token can be acquired from the login API.
Success Response Code: 202
Error Response Code: 500 (Internal Server Error), 404 (Not Found Error), 401 (Unauthorized Error), 400 (Invalid API Usage Error)
Example Response:
Output
{
"jobid": 130
}
6.8. Removing a disk¶
Robin provides a utility through which it detaches and deletes disks on multiple cloud platforms. This enables users to remove the cloud storage present within their clusters with convenience and ease. One caveat is that a disk can only be removed if there are no storage volumes allocated from it. In order to remove disks via the aforementioned utility, issue the following command:
Note
This command is only valid for cloud disks.
# robin disk remove <wwn>
--yes
|
WWN of disk to remove |
|
Do not prompt the user for confirmation of disk removal |
Example:
# robin disk remove 0xQEMU_QEMU_HARDDISK_3c71c872-fe13-4fd5-b --wait --yes
Job: 101 Name: DiskRemove State: VALIDATED Error: 0
Job: 101 Name: DiskRemove State: WAITING Error: 0
Job: 101 Name: DiskRemove State: COMPLETED Error: 0
Removes a cloud disk from the Robin cluster.
End Point: /api/v3/robin_server/disks/<disk_wwn>
Method: DELETE
URL Parameters:
remove=true: This mandatory field within the URL indicates that the disk should be removed from the cluster.
Data Parameters: None
Port: RCM Port (default value is 29442)
Headers:
Authorization: <auth_token>: Authorization token to identify which user is sending the request. The token can be acquired from the login API.
Success Response Code: 202
Error Response Code: 500 (Internal Server Error), 404 (Not Found Error), 401 (Unauthorized Error), 400 (Invalid API Usage Error)
Example Response:
Output
{
"jobid": 134
}
6.9. Updating disk properties¶
In certain circumstances, due to the hardware or environmental reasons, Robin may not correctly discover all the attributes of a device correctly. As a result, in order to modify a disk properties such that they are correct, issue the following command:
# robin disk update <wwn>
--maxvolumes <max_volumes>
--maxlatencysensitivevolumes <max_latency_volumes>
--maxthroughputintensivevolumes <max_throughput_volumes>
--role <role>
--type <type>
--all
--set-reattachable
--unset-reattachable
--set-maintenance
--unset-maintenance
|
WWN of disk to update |
|
Max number of volumes allowed on the disk |
|
Max number of latency sensitive volumes allowed on the disk |
|
Max number of throughput intensive volumes allowed on the disk |
|
Role of the disk to update to |
|
Type of the disk to update to |
|
Run the update for all disks. Should be specified if no WWN is given |
|
Mark the disk as re-attachable |
|
Mark the disk as not re-attachable |
|
Put the disk into maintenance mode |
|
Remove the disk from maintenance mode |
Example:
# robin disk update 0xQEMU_QEMU_HARDDISK_3c71c872-fe13-4fd5-b --maxvolumes 100 --type SSD --wait --yes
Job: 101 Name: DiskModify State: VALIDATED Error: 0
Job: 101 Name: DiskModify State: WAITING Error: 0
Job: 101 Name: DiskModify State: COMPLETED Error: 0
Modifies a disk’s discovered and/or Robin specific properties.
End Point: /api/v3/robin_server/disks/<disk_wwn>
Method: PUT
URL Parameters: None
Data Parameters:
action: update- This mandatory field within the payload specifies that the update operation is to be performed.role: <role>- Utilizing this parameter results in the role of the disk being set to the value specified. Options include: Storage, Swap, RootDisk, and Reserved.type: <type>- Utilizing this parameter results in the type of the disk being set to the value specified. Options include: HDD and SSD.maxvolumesperdisk: <max_vols_on_disk>- Utilizing this parameter results in the maximum number of volumes allowed on the disk being set to the value specified.maxlatencysensitivevolumesperdisk: <max_lat_sens_vols_on_disk>- Utilizing this parameter results in the maximum number of latency sensitive volumes allowed on the disk being set to the value specified.maxthroughputintensivevolumesperdisk: <max_through_ints_vols_on_disk>- Utilizing this parameter results in the maximum number of throughput intensive volumes allowed on the disk being set to the value specified.reattachable: [0,1]- Utilizing this parameter results in the reattachable attribute of the disk being set. By specifying a value of 1, the disk is said to be reattachable and vice versa for a value of 0.maintenance: [0,1]- Utilizing this parameter results in the maintenance mode attribute of the disk being set. By specifying a value of 1, the disk is said to be in maintenance mode and vice versa for a value of 0.
Port: RCM Port (default value is 29442)
Headers:
Authorization: <auth_token>: Authorization token to identify which user is sending the request. The token can be acquired from the login API.
Success Response Code: 202
Error Response Code: 500 (Internal Server Error), 404 (Not Found Error), 401 (Unauthorized Error), 400 (Invalid Api Usage Error)
Example Response:
Output
{
"jobid": 54
}
6.10. Unfaulting a disk¶
When the Robin storage stack encounters an IO error on a disk, it will mark the disk as FAULTED. Sometimes the error is NOT due to the disk, but due to environmental or other hardware errors such as controller errors. In this case one can determine that the disk is actually healthy. In turn this command allows a user to inform Robin that the disk which is presumed to be FAULTED is actually healthy and the external issues that caused the IO failure have been resolved. Robin will proceed to mark the disk as ONLINE and resume normal access to the disk. In order to an unfault a disk, issue the following command:
Note
This functionality should be used with extreme care. If the disk is actually bad, and a user utilizes this command to reverse the error reported by the Robin storage stack, it could result in data loss.
# robin disk unfault <wwn>
--yes
|
WWN of disk to unfault |
|
Do not prompt the user for confirmation of unfaulting the drive |
Example:
# robin disk unfault 0xQEMU_QEMU_HARDDISK_3c71c872-fe13-4fd5-b --wait --yes
Job: 101 Name: DiskUnfault State: VALIDATED Error: 0
Job: 101 Name: DiskUnfault State: WAITING Error: 0
Job: 101 Name: DiskUnfault State: COMPLETED Error: 0
Unfaults a disk that has been marked as FAULTED due to an IO error. Note this functionality should be used with extreme care and only in situations where one has determined that external issues such as hardware/controller errors, that have since been resolved, caused the IO failure. If this condition is not met, unfaulting the disk could result in data loss.
End Point: /api/v3/robin_server/disks/<disk_wwn>
Method: PUT
URL Parameters: None
Data Parameters:
action: unfault- This mandatory field within the payload specifies that the unfault operation is to be performed.wwn: <disk_wwn>- This mandatory field within the payload specifies the WWN of the disk on which the unfault operation should be performed on.
Port: RCM Port (default value is 29442)
Headers:
Authorization: <auth_token>: Authorization token to identify which user is sending the request. The token can be acquired from the login API.
Success Response Code: 202
Error Response Code: 500 (Internal Server Error), 404 (Not Found Error), 401 (Unauthorized Error), 400 (Invalid Api Usage Error)
Example Response:
Output
{
"jobid": 187
}
6.11. Auto Disk Rebalance¶
Robin supports an Auto Disk Rebalance feature which manages the storage space of all disks in order to ensure that the space used on any disk never exceeds a certain threshold. This feature is enabled by default, but can be disabled.
6.11.1. How it works¶
The storage space on a disk is used by one or more volumes. Over time, some disks in the cluster might use storage space to their maximum threshold capacity, while other disks might still have free space.
Robin Central Manager (RCM) continuously collects the physical usage of all disks in the cluster from the Storage Manager (SM) and updates its database. When a disk reaches a high watermark, the disk rebalance job automatically starts to move the volumes from the high watermark disk to the below low watermark disk.
SM selects the volume slices from a high watermark disk and moves these slices to the below low watermark disk. The target disk is chosen based on the following scenarios:
The current physical usage of the disk is below the low watermark threshold.
The physical usage after the volume slices movement must not exceed the low watermark threshold.
6.11.2. Watermark threshold¶
The watermark threshold is an attribute that enables you to set the maximum and minimum limits on the disk’s space usage.
There are two types of watermarks:
High watermark - the maximum threshold of the used disk space that, when exceeded, automatically triggers the disk rebalancing. The default value is 90%.
Low watermark - the minimum threshold of the used disk space that, when reached, automatically stops the disk rebalancing. The default value is 80%.
Note
The low watermark value cannot be greater than or equal to the high watermark value.
6.11.3. Configure high watermark for disk¶
The high watermark threshold for a disk can be configured by updating the respective attribute as shown in the following command:
# robin config update server disk_used_space_high_watermark <value>
Example
# robin config update server disk_used_space_high_watermark 87
The 'server' attribute 'disk_used_space_high_watermark' has been updated
Note
The value set above is a percentage and by default the high watermark threshold is 90%.
6.11.4. Configure low watermark for disk¶
The low watermark threshold for a disk can be configured by updating the respective attribute as shown in the following command:
# robin config update server disk_used_space_low_watermark <value>
Example
# robin config update server disk_used_space_high_watermark 75
The 'server' attribute 'disk_used_space_low_watermark' has been updated
Note
The value set above is a percentage and by default the low watermark threshold is 80%.
6.11.5. Disable Auto Disk Rebalance¶
The Auto Disk Rebalance feature can be disabled in order to allow users to manage the storage space on disks manually. This can be achieved by setting the high watermark threshold needs to 120% as shown in the example below:
# robin config update server disk_used_space_high_watermark 120
The 'server' attribute 'disk_used_space_high_watermark' has been updated
Note
By default, the Auto Disk Rebalance feature is enabled.
6.12. Managing Volumes¶
A Robin volume is the unit of storage allocation from Robin storage. A volume is reflective of the PersistentVolume object within
the native Kubernetes world. It provides logical storage for an application to persist data. Robin allows one to manipulate several properties
of these base volumes at the time of creation including it’s replication factor (in order to avoid data loss) and the encryption/compression algorithm applied to the data stored within.
In order to aid the management of volumes, the following commands are described in this section:
|
View all volumes |
|
Retrieve information about a particular volume |
|
List all volume clones |
|
Configure snapshot space limits for a volume |
|
Import a volume from a volume backup |
|
Attach a repository to a volume |
|
Detach a repository from a volume |
|
Hydrate a volume |
6.12.1. Robin Volume Modes¶
Robin volume has a new parameter named as Mode. The mode of Robin volume depends on the health of its slices. You can view the mode of a volume by running the robin volume list and robin volume info commands. The following are the different types of volume modes:
READ_WRITE (RW) - A volume usually be in the READ_WRITE (RW) mode when all slices of a volume are up and healthy.
READ_ONLY_PARTIAL (ROP) - A volume will be in the READ_ONLY_PARTIAL (ROP) mode when some of the slices of a volume are unhealthy and the volume goes out of quorum. However, the volume serves all read requests and only some write requests if they fall in the region of the volume that is in quorum.
READ_ONLY (RO) - A volume will be in the READ_ONLY (RO) mode when all slices of a volume are out of quorum.
6.12.2. List all volumes¶
In order to view all volumes currently present on the cluster and some additional details such as the physical usage of the volume, replica state, and mount details, issue the following command:
# robin volume list --all
--repl
--nfs
--pvcinfo
--deferred-deletion
--json
|
Display all volumes including those marked for deletion. |
|
Filter list to only display replicated volumes. |
|
Filter list to only display volumes with NFS exports. |
|
Display information about each volume’s associated PVC. |
|
Filter list to only display volumes that are marked for deferred deletion. The deferred deletion is the state of volume that are deleted from user’s perspective, but are still present in the cluster and occupy the storage space. A volume will be in the deferred deletion state until all the associated objects, such as volume snapshots, and clones, are deleted. The deferred deletion flag is set to True for volumes that are in deferred deletion state. |
|
Output in JSON format. |
Example:
# robin volume list
ID | Name | Media | Type | BlockSz | Size (GB) | FsFree (GB) | State | Status | Storage Status | SnapshotSpaceUsed (GB) | Protection | Replicas | Replica State | Mode | Compression | Encryption | Mount | Marked for deletion | Deferred deletion
----+------------------------------------------+-------+---------+---------+-----------+-------------+-------+--------+----------------+------------------------+------------+----------+---------------+------+-------------+------------+--------------------------------------------------+---------------------+-------------------
126 | pvc-8ced7253-5359-44e8-9fd2-b9bbec8279e0 | HDD | REGULAR | 4096 | 10 | 9 | READY | ONLINE | ONLINE | -/- | REPL | 2 | SYNCED | RW | LZ4 | CHACHA20 | | False | True
127 | pvc-80563184-9d85-4421-acb9-9361d7539491 | HDD | REGULAR | 4096 | 10 | 9 | READY | ONLINE | ONLINE | 0.5/5 | REPL | 2 | SYNCED | RW | LZ4 | CHACHA20 | hypervvm-72-70.robinsystems.com:/dev/sdd MOUNTED | False | False
128 | pvc-ae2bf210-0c13-4765-8496-eb97ed518b96 | HDD | REGULAR | 4096 | 10 | 9 | READY | ONLINE | ONLINE | 0.94/7 | REPL | 3 | SYNCED | RW | LZ4 | CHACHA20 | hypervvm-72-70.robinsystems.com:/dev/sde MOUNTED | False | False
129 | pvc-01664fb4-672c-40ea-b3c9-51e93ca87659 | HDD | REGULAR | 4096 | 10 | - | READY | ONLINE | ONLINE | 0/7 | REPL | 3 | SYNCED | RW | LZ4 | CHACHA20 | | False | False
130 | pvc-f9d4c1c6-0c2f-465d-9a38-3815d95e9bde | HDD | REGULAR | 4096 | 10 | - | READY | ONLINE | ONLINE | 0/7 | REPL | 3 | SYNCED | RW | LZ4 | CHACHA20 | | False | False
131 | pvc-9d7df819-cfd7-4ce3-9db3-30f4b02bcc61 | HDD | REGULAR | 4096 | 10 | - | READY | ONLINE | ONLINE | -/- | REPL | 2 | SYNCED | RW | LZ4 | CHACHA20 | | False | False
132 | pvc-a1408d78-6e6a-4cff-8316-4ba2865bf6e1 | HDD | REGULAR | 4096 | 10 | - | READY | ONLINE | ONLINE | -/- | REPL | 2 | SYNCED | RW | LZ4 | CHACHA20 | | False | False
133 | pvc-bffc5103-cda2-4ab9-8bd0-6336cedd9625 | HDD | REGULAR | 4096 | 10 | - | READY | ONLINE | ONLINE | 0/7 | REPL | 3 | SYNCED | RW | LZ4 | CHACHA20 | | False | False
Returns all volumes currently present on the cluster and some additional details such as the physical usage of the volume, replica state, and mount details.
End Point: /api/v3/robin_server/volumes
Method: GET
URL Parameters: None
Data Parameters: None
Port: RCM Port (default value is 29442)
Headers:
Authorization: <auth_token>: Authorization token to identify which user is sending the request. The token can be acquired from the login API.
Success Response Code: 200
Error Response Code: 500 (Internal Server Error), 401 (Unauthorized Error)
Example Response:
Output
{
"items":[
{
"media":72,
"respoolid":1,
"replication":1,
"storstatus_str":"ONLINE",
"volgroupname":"mdcap-dev.playground.01.72.1.a966a57c-8de5-4090-bed3-71d7a214c7d6",
"volumeid":23,
"qgroupid":23,
"tags":[
],
"rsize":10737418240,
"slice_size":1073741824,
"appname":"default",
"storstatus":1,
"current_snapshotid":1,
"compress_str":"NONE",
"lsize":10737418240,
"name":"mdcap-dev.playground.01.home.1.a266bca7-8169-410a-ab58-529b58d4b319",
"snapshot_space_used":0,
"snapshot_space_limit":2147483648,
"active_snapshot_count":0,
"ctime":1602802778,
"volgroupid":23,
"volume_ops":{
"volops_list":[
]
},
"state":2,
"genkey":1602802778083861,
"fsfree":0,
"alloc":[
{
"nodeid":1,
"zonename":"default",
"devid":4,
"zoneid":1602807544,
"devname":"\/dev\/disk\/by-id\/scsi-360022480b410ddebc69ecdea69c2bf34",
"nodename":"cscale-82-35.robinsystems.com"
}
],
"total_snapshot_count":1,
"status":1,
"partitions":0,
"fssize":0,
"compress_algo":0,
"asize":10737418240,
"marked_for_deletion":false,
"vtype_str":"REGULAR",
"mounts":[
{
"nodeid":1,
"zonename":"default",
"state":14,
"zoneid":1602807544,
"readonly":0,
"state_str":"MOUNTED",
"nodename":"cscale-82-35.robinsystems.com",
"mntpath":"\/dev\/sdh"
}
],
"data_state":7,
"block_size":4096,
"protection":0,
"vtype":0,
"psize":570425344,
"resync_state":1,
"media_str":"HDD",
"next_backupid":1,
"multinode_mounting":0,
"appid":1,
"resync_progress":100,
"next_snapshotid":2,
"encrypt_str":"NONE",
"parent_alloc":[
],
"cloud_psize":0,
"encrypt_algo":0
},
{
"media":72,
"respoolid":1,
"replication":2,
"storstatus_str":"ONLINE",
"volgroupname":"file-collection-1602783389182.edc29729-1bfe-4da2-87af-3e0e1fbbe8bd.72.1.0a6b2335-0b82-4c88-815c-c47f0a7e308d",
"volumeid":1,
"qgroupid":1,
"tags":[
],
"rsize":10737418240,
"slice_size":1073741824,
"appname":"default",
"storstatus":1,
"current_snapshotid":1,
"compress_str":"LZ4",
"lsize":5368709120,
"name":"file-collection-1602783389182.edc29729-1bfe-4da2-87af-3e0e1fbbe8bd",
"snapshot_space_used":0,
"snapshot_space_limit":0,
"active_snapshot_count":0,
"ctime":1602783389,
"volgroupid":1,
"volume_ops":{
"volops_list":[
]
},
"state":2,
"genkey":1602783389367989,
"fsfree":0,
"alloc":[
{
"nodeid":2,
"zonename":"default",
"devid":2,
"zoneid":1602807544,
"devname":"\/dev\/disk\/by-id\/scsi-3600224800aba82fe29ee8770cc6440f5",
"nodename":"cscale-82-34.robinsystems.com"
},
{
"nodeid":1,
"zonename":"default",
"devid":4,
"zoneid":1602807544,
"devname":"\/dev\/disk\/by-id\/scsi-360022480b410ddebc69ecdea69c2bf34",
"nodename":"cscale-82-35.robinsystems.com"
}
],
"total_snapshot_count":1,
"status":1,
"partitions":0,
"fssize":0,
"compress_algo":1,
"asize":10737418240,
"marked_for_deletion":false,
"vtype_str":"REGULAR",
"mounts":[
{
"nodeid":1,
"zonename":"default",
"state":14,
"zoneid":1602807544,
"readonly":0,
"state_str":"MOUNTED",
"nodename":"cscale-82-35.robinsystems.com",
"mntpath":"\/dev\/sdd"
}
],
"data_state":7,
"block_size":4096,
"protection":1,
"vtype":0,
"psize":469762048,
"resync_state":1,
"media_str":"HDD",
"next_backupid":1,
"multinode_mounting":0,
"appid":1,
"resync_progress":100,
"next_snapshotid":2,
"encrypt_str":"NONE",
"parent_alloc":[
],
"cloud_psize":0,
"encrypt_algo":0
},
{
"media":72,
"respoolid":1,
"replication":1,
"storstatus_str":"ONLINE",
"volgroupname":"mdcap-dev.postgres.01.72.1.c25ba799-60e1-4881-8784-ec4e0936d1da",
"volumeid":22,
"qgroupid":25,
"tags":[
],
"rsize":10737418240,
"slice_size":1073741824,
"appname":"default",
"storstatus":1,
"current_snapshotid":1,
"compress_str":"NONE",
"lsize":10737418240,
"name":"mdcap-dev.postgres.01.data.1.22af4bf6-12d5-4ee1-83b3-0361f15c9d13",
"snapshot_space_used":0,
"snapshot_space_limit":2147483648,
"active_snapshot_count":0,
"ctime":1602802778,
"volgroupid":25,
"volume_ops":{
"volops_list":[
]
},
"state":2,
"genkey":1602802778074578,
"fsfree":0,
"alloc":[
{
"nodeid":1,
"zonename":"default",
"devid":4,
"zoneid":1602807544,
"devname":"\/dev\/disk\/by-id\/scsi-360022480b410ddebc69ecdea69c2bf34",
"nodename":"cscale-82-35.robinsystems.com"
}
],
"total_snapshot_count":1,
"status":1,
"partitions":0,
"fssize":0,
"compress_algo":0,
"asize":10737418240,
"marked_for_deletion":false,
"vtype_str":"REGULAR",
"mounts":[
{
"nodeid":1,
"zonename":"default",
"state":14,
"zoneid":1602807544,
"readonly":0,
"state_str":"MOUNTED",
"nodename":"cscale-82-35.robinsystems.com",
"mntpath":"\/dev\/sde"
}
],
"data_state":7,
"block_size":4096,
"protection":0,
"vtype":0,
"psize":1644167168,
"resync_state":1,
"media_str":"HDD",
"next_backupid":1,
"multinode_mounting":0,
"appid":1,
"resync_progress":100,
"next_snapshotid":2,
"encrypt_str":"NONE",
"parent_alloc":[
],
"cloud_psize":0,
"encrypt_algo":0
},
{
"media":72,
"respoolid":1,
"replication":1,
"storstatus_str":"ONLINE",
"volgroupname":"mdcap-dev.nginx.01.72.1.4b086eca-88c2-4168-b678-01923b1f7b3d",
"volumeid":25,
"qgroupid":20,
"tags":[
],
"rsize":10737418240,
"slice_size":1073741824,
"appname":"default",
"storstatus":1,
"current_snapshotid":1,
"compress_str":"NONE",
"lsize":10737418240,
"name":"mdcap-dev.nginx.01.data.1.802a6117-1dd4-4c2a-b3a7-a8b622b08ac8",
"snapshot_space_used":0,
"snapshot_space_limit":2147483648,
"active_snapshot_count":0,
"ctime":1602802778,
"volgroupid":20,
"volume_ops":{
"volops_list":[
]
},
"state":2,
"genkey":1602802778091988,
"fsfree":0,
"alloc":[
{
"nodeid":2,
"zonename":"default",
"devid":2,
"zoneid":1602807544,
"devname":"\/dev\/disk\/by-id\/scsi-3600224800aba82fe29ee8770cc6440f5",
"nodename":"cscale-82-34.robinsystems.com"
}
],
"total_snapshot_count":1,
"status":1,
"partitions":0,
"fssize":0,
"compress_algo":0,
"asize":10737418240,
"marked_for_deletion":false,
"vtype_str":"REGULAR",
"mounts":[
{
"nodeid":2,
"zonename":"default",
"state":14,
"zoneid":1602807544,
"readonly":0,
"state_str":"MOUNTED",
"nodename":"cscale-82-34.robinsystems.com",
"mntpath":"\/dev\/sde"
}
],
"data_state":7,
"block_size":4096,
"protection":0,
"vtype":0,
"psize":570425344,
"resync_state":1,
"media_str":"HDD",
"next_backupid":1,
"multinode_mounting":0,
"appid":1,
"resync_progress":100,
"next_snapshotid":2,
"encrypt_str":"NONE",
"parent_alloc":[
],
"cloud_psize":0,
"encrypt_algo":0
},
{
"media":72,
"respoolid":1,
"replication":1,
"storstatus_str":"ONLINE",
"volgroupname":"mdcap-dev.engine.01.72.1.bcf3834d-9c17-463a-bf60-032c9371e4ee",
"volumeid":20,
"qgroupid":22,
"tags":[
],
"rsize":10737418240,
"slice_size":1073741824,
"appname":"default",
"storstatus":1,
"current_snapshotid":1,
"compress_str":"NONE",
"lsize":10737418240,
"name":"mdcap-dev.engine.01.data.1.d0710617-4e5c-419a-9453-b77ea32d6a78",
"snapshot_space_used":0,
"snapshot_space_limit":2,
"active_snapshot_count":0,
"ctime":1602802777,
"volgroupid":21,
"volume_ops":{
"volops_list":[
]
},
"state":2,
"genkey":1602802777561630,
"fsfree":0,
"alloc":[
{
"nodeid":1,
"zonename":"default",
"devid":4,
"zoneid":1602807544,
"devname":"\/dev\/disk\/by-id\/scsi-360022480b410ddebc69ecdea69c2bf34",
"nodename":"cscale-82-35.robinsystems.com"
}
],
"total_snapshot_count":1,
"status":1,
"partitions":0,
"fssize":0,
"compress_algo":0,
"asize":10737418240,
"marked_for_deletion":false,
"vtype_str":"REGULAR",
"mounts":[
{
"nodeid":1,
"zonename":"default",
"state":14,
"zoneid":1602807544,
"readonly":0,
"state_str":"MOUNTED",
"nodename":"cscale-82-35.robinsystems.com",
"mntpath":"\/dev\/sdf"
}
],
"data_state":7,
"block_size":4096,
"protection":0,
"vtype":0,
"psize":570425344,
"resync_state":1,
"media_str":"HDD",
"next_backupid":1,
"multinode_mounting":0,
"appid":1,
"resync_progress":100,
"next_snapshotid":2,
"encrypt_str":"NONE",
"parent_alloc":[
],
"cloud_psize":0,
"encrypt_algo":0
},
{
"media":72,
"respoolid":1,
"replication":1,
"storstatus_str":"ONLINE",
"volgroupname":"mdcap-dev.logstore.01.72.1.f2c3a591-a34c-40fe-9277-4dd743098c19",
"volumeid":24,
"qgroupid":24,
"tags":[
],
"rsize":10737418240,
"slice_size":1073741824,
"appname":"default",
"storstatus":1,
"current_snapshotid":1,
"compress_str":"NONE",
"lsize":10737418240,
"name":"mdcap-dev.logstore.01.data.1.3ee94e27-7368-4e77-90f3-d506a73bb29c",
"snapshot_space_used":0,
"snapshot_space_limit":21474836480,
"active_snapshot_count":0,
"ctime":1602802778,
"volgroupid":24,
"volume_ops":{
"volops_list":[
]
},
"state":2,
"genkey":1602802778089095,
"fsfree":0,
"alloc":[
{
"nodeid":2,
"zonename":"default",
"devid":2,
"zoneid":1602807544,
"devname":"\/dev\/disk\/by-id\/scsi-3600224800aba82fe29ee8770cc6440f5",
"nodename":"cscale-82-34.robinsystems.com"
}
],
"total_snapshot_count":1,
"status":1,
"partitions":0,
"fssize":0,
"compress_algo":0,
"asize":10737418240,
"marked_for_deletion":false,
"vtype_str":"REGULAR",
"mounts":[
{
"nodeid":2,
"zonename":"default",
"state":14,
"zoneid":1602807544,
"readonly":0,
"state_str":"MOUNTED",
"nodename":"cscale-82-34.robinsystems.com",
"mntpath":"\/dev\/sdd"
}
],
"data_state":7,
"block_size":4096,
"protection":0,
"vtype":0,
"psize":570425344,
"resync_state":1,
"media_str":"HDD",
"next_backupid":1,
"multinode_mounting":0,
"appid":1,
"resync_progress":100,
"next_snapshotid":2,
"encrypt_str":"NONE",
"parent_alloc":[
],
"cloud_psize":0,
"encrypt_algo":0
},
{
"media":72,
"respoolid":1,
"replication":1,
"storstatus_str":"ONLINE",
"volgroupname":"mdcap-dev.engine.01.72.1.6c3cddfa-dea8-408f-bea8-a08ec489711c",
"volumeid":21,
"qgroupid":21,
"tags":[
],
"rsize":10737418240,
"slice_size":1073741824,
"appname":"default",
"storstatus":1,
"current_snapshotid":1,
"compress_str":"NONE",
"lsize":10737418240,
"name":"mdcap-dev.engine.01.scripts.1.5c434f33-7e2a-421b-bc04-044d42202b75",
"snapshot_space_used":0,
"snapshot_space_limit":2147483648,
"active_snapshot_count":0,
"ctime":1602802777,
"volgroupid":22,
"volume_ops":{
"volops_list":[
]
},
"state":2,
"genkey":1602802777582450,
"fsfree":0,
"alloc":[
{
"nodeid":1,
"zonename":"default",
"devid":4,
"zoneid":1602807544,
"devname":"\/dev\/disk\/by-id\/scsi-360022480b410ddebc69ecdea69c2bf34",
"nodename":"cscale-82-35.robinsystems.com"
}
],
"total_snapshot_count":1,
"status":1,
"partitions":0,
"fssize":0,
"compress_algo":0,
"asize":10737418240,
"marked_for_deletion":false,
"vtype_str":"REGULAR",
"mounts":[
{
"nodeid":1,
"zonename":"default",
"state":14,
"zoneid":1602807544,
"readonly":0,
"state_str":"MOUNTED",
"nodename":"cscale-82-35.robinsystems.com",
"mntpath":"\/dev\/sdg"
}
],
"data_state":7,
"block_size":4096,
"protection":0,
"vtype":0,
"psize":570425344,
"resync_state":1,
"media_str":"HDD",
"next_backupid":1,
"multinode_mounting":0,
"appid":1,
"resync_progress":100,
"next_snapshotid":2,
"encrypt_str":"NONE",
"parent_alloc":[
],
"cloud_psize":0,
"encrypt_algo":0
}
]
}
6.12.3. Show information about a specific volume¶
In order to view detailed information about a volume such as the breakdown of its available replicas, descriptions of its intrinsic properties including backing media type, encryption/compression applied, file system space usage etc., and tags associated with it issue the following command:
# robin volume info <volume_name>
--pvc-name <pvc_name>
--namespace <namespace>
|
Name of the volume to retrieve information of |
|
The name of PVC that maps to a Robin volume. Note this option can only be used if the Robin volume name is not specified |
|
Namespace the PVC is bound to. Note this option is only valid and is mandatory if the PVC name is specified |
Note
Either one of volume_name or pvc-name options must be specified.
Example 1 (Querying a volume via volume name):
# robin volume info pvc-94a9b4f7-e82c-475c-ab25-842300b259d0
Name : pvc-94a9b4f7-e82c-475c-ab25-842300b259d0
ID : 4
State : READY
Status : ONLINE
Storage Status : ONLINE
Media : SSD
Compression : LZ4
Compression Ratio : 26 %
Encryption : CHACHA20
Volume Type : REGULAR
Clone Type : NONE
Hydrate : False
Workload Type : ordinary
Block Size : 4096
Logical Size : 341.0G
Physical Size* : 640.88G
FS Free : 214.61G
FS Type : ext4
Cloud Tiered Size : -
Multi Node Mounting : False
Protection Type : QUORUM
Replica Count : 3
Replica State : SYNCED
Fault Domain : rack
Mode : READ_WRITE
QGroup ID : 5
Resource Pool Id : 1
Volume Repos :
Namespace : default
PVC Name : ubuntu-vm100-boot-disk
UUID : 98854c92cac211ee93265bbcdc3bed82
Tags : []
Creation Time : 13 Feb 2024 14:52:42
Mounts:
+-----------------------------+------------+------------+---------+-----------+
| Host Name | Mount Path | Zone ID | State | Read-Only |
+-----------------------------+------------+------------+---------+-----------+
| rakuten-40.robinsystems.com | /dev/sda | 1707795823 | MOUNTED | False |
+-----------------------------+------------+------------+---------+-----------+
Allocations:
-------
Replica 1
+----+---------------------------------------+-----------------------------+------------------------------------------------+--------------------+
| ID | WWN | Host Name | Devpath | Reserved Size (GB) |
+----+---------------------------------------+-----------------------------+------------------------------------------------+--------------------+
| 3 | 0xui.00000000000000008ce38ee209aaad01 | rakuten-31.robinsystems.com | /dev/disk/by-id/nvme-KCD6XVUL3T20_91T0A00GT4T8 | 341 |
+----+---------------------------------------+-----------------------------+------------------------------------------------+--------------------+
Replica 2
+----+---------------------------------------+-----------------------------+------------------------------------------------+--------------------+
| ID | WWN | Host Name | Devpath | Reserved Size (GB) |
+----+---------------------------------------+-----------------------------+------------------------------------------------+--------------------+
| 2 | 0xui.00000000000000008ce38ee209aabf01 | rakuten-40.robinsystems.com | /dev/disk/by-id/nvme-KCD6XVUL3T20_91T0A00VT4T8 | 341 |
+----+---------------------------------------+-----------------------------+------------------------------------------------+--------------------+
Replica 3
+----+---------------------------------------+-----------------------------+------------------------------------------------+--------------------+
| ID | WWN | Host Name | Devpath | Reserved Size (GB) |
+----+---------------------------------------+-----------------------------+------------------------------------------------+--------------------+
| 1 | 0xui.00000000000000008ce38ee20a102801 | rakuten-37.robinsystems.com | /dev/disk/by-id/nvme-KCD6XVUL3T20_X1W0A08KT4T8 | 341 |
+----+---------------------------------------+-----------------------------+------------------------------------------------+--------------------+
Snapshot Info:
-------
Snapshot Count : 1
Snapshot Space Used* : 325G
Snapshot Space Reserved* : N/A
*Size calculation is inclusive of all replicas.
Example 2 (Querying a volume via PVC name):
# robin volume info --pvc-name ubuntu-vm24-boot-disk --namespace default
Name : pvc-b9fb1af5-d16f-43b9-8fea-8621f5cae371
ID : 345
State : READY
Status : ONLINE
Storage Status : ONLINE
Media : HDD
Compression : LZ4
Compression Ratio : 12 %
Encryption : NONE
Volume Type : REGULAR
Workload Type : ordinary
Block Size : 4096
Logical Size : 30.0G
Physical Size* : 12G
Fs Free : 25.8G
Fs Type : ext4
Cloud Tiered Size : -
Multi Node Mounting : False
Replica Count : 2
Replica State : SYNCED
Fault Domain : host
QGroup ID : 349
Resource Pool Id : 1
Volume Repos :
UUID : 7323a4c667a211ee9f4cd1371a995c71
Namespace : default
PVC Name : ubuntu-vm24-boot-disk
UUID : 7323a4c667a211ee9f4cd1371a995c71
Tags : []
Creation Time : 15 May 2023 12:19:57
Mounts:
+-------------------------+------------+------------+---------+-----------+
| Host Name | Mount Path | Zone ID | State | Read-Only |
+-------------------------+------------+------------+---------+-----------+
| qct-42.robinsystems.com | /dev/sdh | 1683209288 | MOUNTED | False |
+-------------------------+------------+------------+---------+-----------+
Allocations:
-------
Replica 1
+----+------------------------------------+-------------------------+--------------------------------------------------------+--------------------+
| ID | WWN | Host Name | Devpath | Reserved Size (GB) |
+----+------------------------------------+-------------------------+--------------------------------------------------------+--------------------+
| 8 | 0x6001636002ff8c602ba95579c126abf0 | qct-44.robinsystems.com | /dev/disk/by-id/scsi-36001636002ff8c602ba95579c126abf0 | 30 |
+----+------------------------------------+-------------------------+--------------------------------------------------------+--------------------+
Replica 2
+----+------------------------------------+-------------------------+--------------------------------------------------------+--------------------+
| ID | WWN | Host Name | Devpath | Reserved Size (GB) |
+----+------------------------------------+-------------------------+--------------------------------------------------------+--------------------+
| 5 | 0x6001636002ff8f202904367bcacc3cb7 | qct-42.robinsystems.com | /dev/disk/by-id/scsi-36001636002ff8f202904367bcacc3cb7 | 30 |
+----+------------------------------------+-------------------------+--------------------------------------------------------+--------------------+
Snapshot Info:
-------
Snapshot Count : 0
Snapshot Space Used* : 0G
Snapshot Space Reserved* : N/A
NFS info:
--------
NFS Server : robin-nfs-shared-15
NFS Server Host : qct-42.robinsystems.com
NFS Clients : ["qct-43.robinsystems.com"]
NFS Export State : READY
*Size calculation is inclusive of all replicas.
Returns detailed information about a volume such as the breakdown of its available replicas, descriptions of its intrinsic properties including backing media type, encryption/compression applied etc., and tags associated with it.
End Point: /api/v3/robin_server/volumes/
Method: GET
URL Parameters: None
Data Parameters:
name: <volume_name>- Utilizing this parameter within the payload by specifying the name of a volume results in information for the aforementioned volume being retrieved.pvc_name: <pvc_name>- Utilizing this parameter within the payload by specifying the name of a PVC results in information for the aforementioned volume being retrieved.namespace: <namespace>- Utilizing this parameter within the payload by specifying the name of a namespace results in only PVC’s bound to that namespace being queried. Note this option is only valid and mandatory when thepvc_nameparameter is utilized.
Note
Either one of the name or pvc_name parameters must be provided.
Port: RCM Port (default value is 29442)
Headers:
Authorization: <auth_token>: Authorization token to identify which user is sending the request. The token can be acquired from the login API.
Success Response Code: 200
Error Response Code: 500 (Internal Server Error), 404 (Not Found Error), 401 (Unauthorized Error)
Example Response:
Output
{
"items": {
"volumeid": 8,
"appid": 1,
"name": "pvc-99404455-b84f-4847-bd78-6dcf2ed2a8b1",
"snapshot_space_used": 0,
"snapshot_space_limit": 0,
"active_snapshot_space": 1140850688,
"active_snapshot_count": 0,
"total_snapshot_count": 1,
"media": 72,
"vtype": 0,
"protection": 0,
"current_snapshotid": 1,
"next_snapshotid": 2,
"next_backupid": 1,
"qgroupid": 9,
"appname": "default",
"ctime": 1696965790,
"genkey": 1696965790904809,
"block_size": 4096,
"slice_size": 1073741824,
"compress_algo": 1,
"encrypt_algo": 0,
"multinode_mounting": 0,
"replication": 1,
"state": 2,
"data_state": 7,
"status": 1,
"partitions": 0,
"resync_progress": 100,
"resync_state": 1,
"lsize": 2147483648,
"fssize": 0,
"fsfree": 0,
"rsize": 2147483648,
"asize": 2147483648,
"psize": 1140850688,
"cloud_psize": 0,
"respoolid": 1,
"volgroupid": 8,
"volgroupname": "pvc-99404455-b84f-4847-bd78-6dcf2ed2a8b1.72.1",
"mounts": [
{
"zoneid": 1696976139,
"zonename": "default",
"nodeid": 2,
"nodename": "hypervvm-61-81",
"mntpath": "/dev/sdd",
"readonly": 0,
"state": 14,
"state_str": "MOUNTED"
}
],
"mountgenids": [
{
"nodeid": 2,
"zoneid": 1696976139,
"genid": 1
}
],
"tags": [],
"compress_ratio": 12,
"volume_ops": {
"volops_list": []
},
"alloc": [
[
{
"id": 3,
"wwn": "0x6002248022fdd8b14f905a8a568290a6",
"allocation": 2,
"node": "hypervvm-61-82",
"k8s_node_name": "hypervvm-61-82",
"devpath": "/dev/disk/by-id/scsi-36002248022fdd8b14f905a8a568290a6"
}
]
],
"parent_alloc": [],
"storstatus": 1,
"media_str": "HDD",
"vtype_str": "REGULAR",
"compress_str": "LZ4",
"encrypt_str": "NONE",
"state_str": "READY",
"status_str": "ONLINE",
"storstatus_str": "ONLINE",
"stordatastate_str": "COMPLETE",
"volstatus_str": "ONLINE",
"rwx_vol": false,
"workload": 0,
"faultdomain": "N/A",
"pvcname": "default/comp-test",
"workload_str": "ordinary",
"nfs_exports": [],
"volume_repos": [],
"uuid": "7323a4c667a211ee9f4cd1371a995c71",
"mnt_stats": {
"volid": 8,
"devpath": "/dev/sdd",
"mntpath": "/var/lib/pods/9667ed86-0533-499a-8c9e-5c6a987df99d/volumes/kubernetes.io~csi/pvc-99404455-b84f-4847-bd78-6dcf2ed2a8b1/mount",
"fstype": "ext4",
"fstotal": 2046640128,
"fsfree": 842452992,
"fsused": 1204187136,
"time_updated": 1696967037
}
}
}
6.12.4. List all volume clones¶
In order to view all clone volumes currently present on the cluster and some additional details such as the physical usage of the volume, parent volume, and mount details, issue the following command:
# robin volume clone-list --json
|
Output in JSON format |
Example:
ID | Name | ParentSnap | ParentVolume | Media | BlockSz | Size (GB) | Psize (GB) | Storage Status | SnapshotSpaceUsed (GB) | Replicas | Replica State | Compression | Encryption | Mount
---+---------------------------------------------------------------------+---------------------+------------------------------------------------------------------+-------+---------+-----------+------------+----------------+------------------------+----------+---------------+-------------+------------+-------------------------------------------------
9 | clone-demo.server.01.block.1.2c69deaa-52bc-40ba-b455-5271e4fae4a1 | snapshot-1603479973 | centos1.server.01.block.1.c90c49c0-74ab-4459-be69-fea9f730cc9e | HDD | 4096 | 1 | - | ONLINE | 0/0.2 | 1 | SYNCED | NONE | NONE | cscale-82-140.robinsystems.com:/dev/sdm MOUNTED
10 | clone-demo.server.01.root_fs.1.63262a2c-b8b2-4f03-9fea-65b47a95a5a8 | snapshot-1603479973 | centos1.server.01.root_fs.1.2d59b0b3-c33a-4594-9d3d-44def625a38b | HDD | 4096 | 1 | 0.03 | ONLINE | 0/0.2 | 1 | SYNCED | NONE | NONE | cscale-82-140.robinsystems.com:/dev/sdn MOUNTED
11 | clone-demo.server.01.data.1.dd86ccb2-ce0a-4f39-a041-66ec4d492fbc | snapshot-1603479973 | centos1.server.01.data.1.8dec6c3f-839a-439c-b830-130386ff40f4 | HDD | 4096 | 1 | 0.03 | ONLINE | 0/0.2 | 1 | SYNCED | NONE | NONE | cscale-82-140.robinsystems.com:/dev/sdl MOUNTED
Returns all clone volumes currently present on the cluster and some additional details such as the physical usage of the volume, parent volume, and mount details.
End Point: /api/v3/robin_server/volumes/clone
Method: GET
URL Parameters: None
Data Parameters: None
Port: RCM Port (default value is 29442)
Headers:
Authorization: <auth_token>: Authorization token to identify which user is sending the request. The token can be acquired from the login API.
Success Response Code: 200
Error Response Code: 500 (Internal Server Error), 401 (Unauthorized Error)
Example Response:
Output
{
"items":[
{
"qgroupid":10,
"rsize":1073741824,
"multinode_mounting":0,
"volgroupname":"clone-demo.server.01.72.1.3a122025-f911-4ecf-b689-f5492bbefd72",
"parent_volumeid":6,
"encrypt_algo":0,
"parent_alloc":[
{
"devid":1,
"zoneid":1603415748,
"nodeid":1
}
],
"volgroupid":9,
"fsfree":0,
"psize":33554432,
"next_snapshotid":2,
"replication":1,
"cloud_psize":0,
"lsize":1073741824,
"status":1,
"encrypt_str":"NONE",
"volumeid":11,
"vtype":1,
"ctime":1603480455,
"snapshot_space_used":0,
"storstatus_str":"ONLINE",
"respoolid":1,
"marked_for_deletion":false,
"appid":1,
"appname":"default",
"protection":0,
"media":72,
"media_str":"HDD",
"genkey":1603480455171116,
"next_backupid":1,
"tags":[
],
"block_size":4096,
"asize":1073741824,
"partitions":0,
"parent_snapshotid":1,
"slice_size":1073741824,
"storstatus":1,
"name":"clone-demo.server.01.data.1.dd86ccb2-ce0a-4f39-a041-66ec4d492fbc",
"total_snapshot_count":1,
"vtype_str":"CLONE",
"alloc":[
{
"devid":1,
"nodeid":1,
"nodename":"cscale-82-140.robinsystems.com",
"zoneid":1603415748,
"zonename":"default",
"devname":"\/dev\/disk\/by-id\/scsi-3600224803bcdafde95b1f5cd27ceb5fb"
}
],
"volume_ops":{
"volops_list":[
]
},
"data_state":7,
"mounts":[
{
"nodeid":1,
"nodename":"cscale-82-140.robinsystems.com",
"state_str":"MOUNTED",
"state":14,
"zoneid":1603415748,
"zonename":"default",
"mntpath":"\/dev\/sdl",
"readonly":0
}
],
"parent_appname":"default",
"active_snapshot_count":0,
"current_snapshotid":1,
"fssize":0,
"parent_volumename":"centos1.server.01.data.1.8dec6c3f-839a-439c-b830-130386ff40f4",
"compress_str":"NONE",
"resync_state":1,
"snapshot_space_limit":214748364,
"resync_progress":100,
"parent_snapshotname":"snapshot-1603479973",
"state":2,
"parent_appid":1,
"compress_algo":0
},
{
"qgroupid":9,
"rsize":1073741824,
"multinode_mounting":0,
"volgroupname":"clone-demo.server.01.72.1.bc0f9bf3-3df2-4ec8-8a03-2edad0d4d176",
"parent_volumeid":8,
"encrypt_algo":0,
"parent_alloc":[
{
"devid":1,
"zoneid":1603415748,
"nodeid":1
}
],
"volgroupid":11,
"fsfree":0,
"psize":33554432,
"next_snapshotid":2,
"replication":1,
"cloud_psize":0,
"lsize":1073741824,
"status":1,
"encrypt_str":"NONE",
"volumeid":10,
"vtype":1,
"ctime":1603480455,
"snapshot_space_used":0,
"storstatus_str":"ONLINE",
"respoolid":1,
"marked_for_deletion":false,
"appid":1,
"appname":"default",
"protection":0,
"media":72,
"media_str":"HDD",
"genkey":1603480455157228,
"next_backupid":1,
"tags":[
],
"block_size":4096,
"asize":1073741824,
"partitions":0,
"parent_snapshotid":1,
"slice_size":1073741824,
"storstatus":1,
"name":"clone-demo.server.01.root_fs.1.63262a2c-b8b2-4f03-9fea-65b47a95a5a8",
"total_snapshot_count":1,
"vtype_str":"CLONE",
"alloc":[
{
"devid":2,
"nodeid":1,
"nodename":"cscale-82-140.robinsystems.com",
"zoneid":1603415748,
"zonename":"default",
"devname":"\/dev\/disk\/by-id\/scsi-3600224804c48fd7e16c608dea0919064"
}
],
"volume_ops":{
"volops_list":[
]
},
"data_state":7,
"mounts":[
{
"nodeid":1,
"nodename":"cscale-82-140.robinsystems.com",
"state_str":"MOUNTED",
"state":14,
"zoneid":1603415748,
"zonename":"default",
"mntpath":"\/dev\/sdn",
"readonly":0
}
],
"parent_appname":"default",
"active_snapshot_count":0,
"current_snapshotid":1,
"fssize":0,
"parent_volumename":"centos1.server.01.root_fs.1.2d59b0b3-c33a-4594-9d3d-44def625a38b",
"compress_str":"NONE",
"resync_state":1,
"snapshot_space_limit":214748364,
"resync_progress":100,
"parent_snapshotname":"snapshot-1603479973",
"state":2,
"parent_appid":1,
"compress_algo":0
},
{
"qgroupid":11,
"rsize":1073741824,
"multinode_mounting":0,
"volgroupname":"clone-demo.server.01.72.1.f366e12b-ae61-4f1d-a441-0258d764d5f3",
"parent_volumeid":7,
"encrypt_algo":0,
"parent_alloc":[
{
"devid":1,
"zoneid":1603415748,
"nodeid":1
}
],
"volgroupid":10,
"fsfree":0,
"psize":0,
"next_snapshotid":2,
"replication":1,
"cloud_psize":0,
"lsize":1073741824,
"status":1,
"encrypt_str":"NONE",
"volumeid":9,
"vtype":1,
"ctime":1603480455,
"snapshot_space_used":0,
"storstatus_str":"ONLINE",
"respoolid":1,
"marked_for_deletion":false,
"appid":1,
"appname":"default",
"protection":0,
"media":72,
"media_str":"HDD",
"genkey":1603480455077815,
"next_backupid":1,
"tags":[
],
"block_size":4096,
"asize":0,
"partitions":0,
"parent_snapshotid":1,
"slice_size":1073741824,
"storstatus":1,
"name":"clone-demo.server.01.block.1.2c69deaa-52bc-40ba-b455-5271e4fae4a1",
"total_snapshot_count":1,
"vtype_str":"CLONE",
"alloc":[
],
"volume_ops":{
"volops_list":[
]
},
"data_state":7,
"mounts":[
{
"nodeid":1,
"nodename":"cscale-82-140.robinsystems.com",
"state_str":"MOUNTED",
"state":14,
"zoneid":1603415748,
"zonename":"default",
"mntpath":"\/dev\/sdm",
"readonly":0
}
],
"parent_appname":"default",
"active_snapshot_count":0,
"current_snapshotid":1,
"fssize":0,
"parent_volumename":"centos1.server.01.block.1.c90c49c0-74ab-4459-be69-fea9f730cc9e",
"compress_str":"NONE",
"resync_state":1,
"snapshot_space_limit":214748364,
"resync_progress":100,
"parent_snapshotname":"snapshot-1603479973",
"state":2,
"parent_appid":1,
"compress_algo":0
}
]
}
6.12.5. Configure the snapshot space limit of a volume¶
Robin stores snapshot(s) of a volume within a dedicated space, referred to as the snapshot space of the volume. The snapshot space is considered to be a percentage of the allocated space for a volume. However after the volume is provisioned the size of the snapshot might need to be modified.
Note
If you do not configure any limit for snapshots space usage, a snapshot can consume complete space available in the volume and beyond.
Also, the configured snapshot limit depends on the replication count of the volume.
For example, if the replication count is 2 for a volume and if you set the snapshot space limit as 10 GB, then the total space limit would be 10 GB * 2=20 GB.
In order to configure the snapshot space limit, issue the following command:
# robin volume snapshot-space-limit <volume_name> <snapshot_space_limit>
Note
Only the snapshot space limit of individual volumes can be configured at this time. To configure the snapshot space limit for an application, update the snapshot space limit of each of the volumes it consists of.
|
Name of the volume to configure the snapshot space limit for |
|
Size of the snapshot space limit. Can be specified as total bytes or a human readable format, such as ‘1GB’ |
Example:
# robin volume snapshot-space-limit pvc-bd01bcd9-9e4d-4a34-ac13-5a98395935b3 4G --wait
Job: 163 Name: VolumeUpdateSnapshotLimit State: PROCESSED Error: 0
Job: 163 Name: VolumeUpdateSnapshotLimit State: FINALIZED Error: 0
Job: 163 Name: VolumeUpdateSnapshotLimit State: COMPLETED Error: 0
Configures the snapshot space limit of a volume.
End Point: /api/v3/robin_server/volumes/<volume_name>
Method: PUT
URL Parameters: None
Data Parameters:
action: snapshot_limits- This mandatory field within the payload specifies that the snapshot limit of the volume is to be modified.name: <volume_name>- This mandatory field within the payload specifies the name of the volume for which the snapshot limits are to be modified.snapshot_space_limit: <snap_limit>- This mandatory field within the payload specifies the size, in bytes, the snapshot space limit of the aforementioned volume should be set to.
Port: RCM Port (default value is 29442)
Headers:
Authorization: <auth_token>: Authorization token to identify which user is sending the request. The token can be acquired from the login API.
Success Response Code: 202
Error Response Code: 500 (Internal Server Error), 404 (Not Found Error), 401 (Unauthorized Error), 400 (Invalid API Usage Error)
Example Response:
Output
{
"jobid": 163
}
6.12.6. Import a volume¶
Robin enables users to import data from a volume backup and create a new volume on the source cluster of the volume backup or on a different cluster that has access to the same external cloud repository. After the import process is complete, the new volume will contain the same data as the source volume. However, there is no relationship between the original volume and the volume created using the backup. If a volume backup contains multiple volumes, each volume must be imported in a serial and seperate manner. With regards to the objects created as part of the import process, there are two options available:
Create only a Robin volume. This is the default behavior and is intended for those who will be directly handling the creation of PersistentVolumeClaim, PersistentVolume, and volume, manually stitching them all together. With the option, the user can specify the volume name, as well as various attributes pertaining to the volume itself (e.g., replication, faultdomain, access mode, etc.). The one caveat is, the PersistentVolume name and Robin volume name must be the same.
A PersistentVolumeClaim, PersistentVolume, and Robin volume will be created. This option is intended for those who want to create or recreate a PersistentVolumeClaim that can can be referenced by a Pod for storage. With this option, the user cannot specify the volume name. They can, however, specify the name of the PersistentVolumeClaim, as well as the various volume attributes (e.g., replication, faultdomain, access mode, etc.).
To import a volume, run the following command:
# robin volume import <backupid> <import_volume> <rpool>
--name <name>
--namespace <namespace>
--media <media>
--protection <protection>
--replication <replication>
--faultdomain <faultdomain>
--workload <workload>
--access-mode <access_mode>
--pvc-name <pvc_name>
--storage-class <storage_class>
--username <username>
--tenant <tenant>
--timeout <timeout>
--start-hydration
--create-pvc
|
ID of volume backup containing the volume to import |
|
Name of the volume to import data from. The volume must be included in the referenced volume backup |
|
Name of the resource pool in which to create the volume |
|
Custom name for the new volume. If a custom volume name is not specified, then a standardized volume name will be generated. Note that a custom volume name cannot be used in conjunction with the |
|
The namespace to bind the newly created PVC to. By default, the current user’s current namespace is used. Note this option can only be used in conjunction with the |
|
Specify which type of drives to allocate storage for the new volume from. Valid choices include ‘HDD’ and ‘SSD’. Default media type is HDD. |
|
Type of Protectin for new volume. Valid options are |
|
Replication level for the new volume. The default is same as the source volume |
|
Fault domain for replicas. Valid choices include: ‘disk’, ‘host’, and ‘rack’. This option is only valid when |
|
Workload type of the new volume. Valid choices include ‘latency’, ‘throughput’, ‘dedicated’, or ‘ordinary’. |
|
Storage access mode. Valid choices include: ‘ReadWriteOnce’, ‘ReadOnlyOnce’, ‘ReadWriteOncePod’, and ‘ReadWriteMany’. If access mode is not specified then the access mode of the PVC bound to the imported volume will be used. If there is no bound PVC, then ‘ReadWriteOnce’ will be used. |
|
Name to set for the newly created PVC. Note this option can only be used with the |
|
Name of storage class for the new volume. Valid choices include: ‘robin’ and ‘robin-immediate’. By default, ‘robin-immediate’ is used. Note this option can only be used in conjunction with the |
|
Username of user the volume will be bound to. By default, the current user’s username is used. Note this option can only be used in conjunction with the |
|
Name of tenant the volume backup will be bound to |
|
The number of seconds to wait for the volume import operation to finish (default is no timeout) |
|
Start the hydration of imported volume in the background |
|
Instead of creating the imported volume directly, create a PVC that binds to a PV and the resulting Robin volume. |
Imports data from a volume backup and create a new volume on the source cluster of the volume backup or on a different cluster that has access to the same external cloud repository. The new volume will contain the same data as the source volume. However, there is no relationship between the original volume and the volume created using the backup.
End Point: /api/v6/robin_server/volumes
Method: POST
URL Parameters: None
Data Parameters:
action: import_volume- This mandatory field within the payload specifies the import volume operation is to be performed.backupid: <backupid>- This mandatory field within the payload specifies a string representing the ID of volume backup containing the volume to import.import_volume: <import_volume>- This mandatory field within the payload specifies the name of the volume to import data from. The volume must be included in the aforementioned volume backup.rpool: <rpool>- This mandatory field within the payload specifies the name of the resource pool in which to create the volume.media: <media>- This mandatory field within the payload specifies the type of drives to allocate storage for the new volume from. Valid choices include ‘HDD’ and ‘SSD’.name: <name>- Utilizing this parameter within the payload, by specifying by specifying a string, results in the new volume’s name being set to the string given. If a custom volume name is not specified, then a standardized volume name will be generated. Note that this paraemeter cannot be used if thecreate_pvcparameter is enabled.namespace: <namespace>- Utilizing this parameter within the payload, by specifying a string representing the name of a namespace, results the newly created PVC being bound to the respective namespace. By default, the current user’s current namespace is used. Note this parameter is only valid if thecreate_pvcparameter is enabled.replication: <replication>- Utilizing this parameter within the payload, by specifying a string representing an integer, results in the replication level for the new volume being set to the given factor. By default, the replication factor of the source volume is used. Valid choices include: ‘1’, ‘2’, or ‘3’.faultdomain: <faultdomain>- Utilizing this parameter within the payload, by specifying a string, results in the fault domain of the replicas being set to the value given. Valid choices include: ‘disk’, ‘host’, and ‘rack’. Note this option is only valid when the replication factor is greater than 1.blocksize: <blocksize>- Utilizing this parameter within the payload, by specifying an integer, results in the write unit for the volume being set to the value given. Valid choices include: 512 and 4096.workload: <workload>- Utilizing this parameter within the payload, by specifying a string, results in the newly created volume having the workload type set to the value specified. Valid choices include ‘latency’, ‘throughput’, ‘dedicated’, or ‘ordinary’.compression: <compression>- Utilizing this parameter within the payload, by specifying a string, results in specified compression algorithm being applied to the data stored in the newly created volume. Valid choices include ‘LZ4’.access_mode: <access_mode>- Utilizing this parameter within the payload, by specifying a string representing a Kubernetes access mode, results the the storage access mode of the new volume being set to the value given. Valid choices include: ‘ReadWriteOnce’, ‘ReadOnlyOnce’, and ‘ReadWriteMany’. By default, the access mode of the PVC bound to the imported volume will be used. If there is no bound PVC, then ‘ReadWriteOnce’ will be used. Note this parameter is only valid if thecreate_pvcparameter is enabled.storage_class: <storage_class>- Utilizing this parameter within the payload, by specifying a string representing the name of Kubernetes storage class, results the the storage class of the new volume being set to the value given. Valid choices include ‘robin’ or ‘robin-immediate’. By default, ‘robin-immediate’ is used. Note this parameter is only valid if thecreate_pvcparameter is enabled.username: <username>- Utilizing this parameter within the payload, by specifying a string representing the name of a user, results in the volume being bound to the respective user. Note if this parameter is utilized thetenantparameter must also be specified in order for it to be valid.tenant: <tenant>- Utilizing this parameter within the payload, by specifying a string representing the name of a tenant, results in the volume being bound to the respective tenant. Note this option is only valid if theusernameparameter is specified and thecreate_pvcparameter is enabled.timeout: <timeout>- Utilizing this parameter within the payload, by specifying a integer representing a duration in seconds, results in the job waiting the specified number of seconds before marking the import operation as incomplete and failed. If not specified, the default timeout of 600 seconds is used.hydrate: true- Utilizing this parameter within the payload results in the hydration of the imported volume being started in the background. The default value is false.create_pvc: true- Utilizing this parameter within the payload results in a PVC being created which in turn that to a PV and the resulting Robin volume, instead of creating the imported volume directly. The default value is false. Note this parameter cannot be used if thenameparameter is specified.
Port: RCM Port (default value is 29442)
Headers:
Authorization: <auth_token>: Authorization token to identify which user is sending the request. The token can be acquired from the login API.
Success Response Code: 202
Error Response Code: 500 (Internal Server Error), 401 (Unauthorized Error), 400 (Invalid API Usage Error), 409 (Duplicate Resource Error)
Example Response:
6.12.7. Attach a volume to an external repository¶
In order to attach a volume to an external repository such that any volume snapshot containing the volume can be backed up, issue the following command:
# robin volume attach-repo <volume_name> <repo_name>
|
Name of volume to be attached |
|
Name of the already registered external respositry to attach to |
Note
A volume can only be attached to one repository at a time.
Example:
# robin volume attach-repo pvc-bd01bcd9-9e4d-4a34-ac13-5a98395935b3 testrepo --wait
Job: 331 Name: VolumeAddRepo State: VALIDATED Error: 0
Job: 331 Name: VolumeAddRepo State: WAITING Error: 0
Job: 331 Name: VolumeAddRepo State: COMPLETED Error: 0
6.12.8. Detach a volume from an external repository¶
In order to detach a volume from an external repository such that it cannot be backed up using a volume snapshot, issue the following command:
Note
Imported volumes created from volume or application backups need to be hydrated first before they can be detached. Volumes in other states can be detached regardless as long as no backup operation involving the volumes is in progress.
# robin volume detach-repo <volume_name> <repo_name>
|
Name of volume to be detached |
|
Name of the external respositry the aforementioned volume is currently attached to |
Example:
# robin volume detach-repo pvc-bd01bcd9-9e4d-4a34-ac13-5a98395935b3 testrepo --wait
Job: 333 Name: VolumeRemoveRepo State: VALIDATED Error: 0
Job: 333 Name: VolumeRemoveRepo State: WAITING Error: 0
Job: 333 Name: VolumeRemoveRepo State: COMPLETED Error: 0
6.12.9. Hydrate a volume¶
In order to hydrate an imported volume such that it becomes a standalone volume with no need to be associated with an external repository, issue the following command:
# robin volume hydrate <volume_name>
|
Name of volume to be hydrated |
Note
Only imported volumes created from volume or application backups can be hydrated.
Example:
# robin volume hydrate pvc-bd01bcd9-9e4d-4a34-ac13-5a98395935b3 --wait
Job: 332 Name: VolumeHydrate State: VALIDATED Error: 0
Job: 332 Name: VolumeHydrate State: WAITING Error: 0
Job: 332 Name: VolumeHydrate State: COMPLETED Error: 0
6.12.10. Identify unused volumes¶
When PVC is created with a ReclaimPolicy of Retain, Kubernetes will not call the CSI driver to delete the backing PersistentVolume when the linked PVC is removed. As a result, in such cases the Robin Adminstrator might need to manually clean up the associated volumes in order complete the deletion process. To aid the process, a utility named get-unbound-vols is provided within the Robin daemonset Pods. As its name suggests it discovers any unbound volumes which do not have any associated PVCs and thus enables the Adminstrator to simply delete the identified PV objects using the kubectl delete pv command. Moreover the associated Robin volume with the same name will also have to be deleted using the robin volume delete command. More information on the ReclaimPolicy concept can be found here.
To run the utility, execute into any Robin daemonset pod, login as the Adminstrator and issue the get-unbound-vols command as shown in the example below:
# kubectl describe robinclusters -n robinio
Name: robin
Namespace: robinio
Labels: app.kubernetes.io/instance=robin
app.kubernetes.io/managed-by=robin.io
app.kubernetes.io/name=robin
Annotations: API Version: manage.robin.io/v1
Kind: RobinCluster
Metadata:
Creation Timestamp: 2021-02-04T09:41:06Z
Generation: 1
Resource Version: 378697
Self Link: /apis/manage.robin.io/v1/namespaces/robinio/robinclusters/robin
UID: 7533d2da-8521-47e2-bb52-426df5a07a2b
Spec:
host_type: ibm
image_pull_secret: all-icr-io
image_registry_path: uk.icr.io/docker_registry/docker
image_robin: robinsys/robinimg:5.3.4-75
k8s_provider: iks
Options:
cloud_cred_secret: cloud-cred-secret
update_coredns: 1
Status:
connect_command: kubectl exec -it robin-nfbxx -n robinio -- bash
get_robin_client: curl -k https://10.242.64.22:29442/api/v3/robin_server/download?file=robincli&os=linux > robin
master_ip: 10.242.64.22
Phase: Ready
pod_status:
robin-nfbxx 10.242.64.22 Running 10.242.64.22
robin-drdzx 10.242.64.21 Running 10.242.64.21
robin-57w8h 10.242.64.20 Running 10.242.64.20
robin_node_status:
host_name: kube-c0d69ael0c0ce2a8b6b0-asitclus-default-00000b25
join_time: 1612431722
k8s_node_name: 10.242.64.22
Roles: M*,S
Rpool: default
State: ONLINE
Status: Ready
host_name: kube-c0d69ael0c0ce2a8b6b0-asitclus-default-00000a68
join_time: 1612431745
k8s_node_name: 10.242.64.20
Roles: S,M
Rpool: default
State: ONLINE
Status: Ready
host_name: kube-c0d69ael0c0ce2a8b6b0-asitclus-default-00000c21
join_time: 1612431760
k8s_node_name: 10.242.64.21
Roles: S,M
Rpool: default
State: ONLINE
Status: Ready
Events: <none>
# kubectl exec -it robin-nfbxx -n robinio -- bash
[robinds@hypervvm-72-43]#
[robinds@hypervvm-72-43]# robin login admin --p Robin123
User admin is logged into Administrators tenant
[robinds@hypervvm-72-43]# get-unbound-vols
Unbound volumes (without PVCs):
pvc-6732bdf4-58ce-40da-abff-c93089dcdcdf
pvc-15b32f8f-4e85-4d97-a28e-9d760f63769b
6.13. Managing Volume Snapshots¶
The Volume Snapshot feature allows you to create crash-consistent snapshots of one or more Robin volumes as a group. Volumes in a volume snapshot may all be from the same application, or they may be from multiple, unrelated applications. Unlike with application snapshots, no application metadata, such as its configuration and state, is saved when creating a volume snapshot. To ensure data consistency across all volumes in a volume snapshot, the storage layer waits for I/O operations to complete on all volumes before copying the volume data.
Before you can make use of of a volume snapshot, it first needs to be pushed to an external storage repository. From there, it can use it to recover (recreate) volumes that have become corrupted or lost due to human error or catastrophic failure. A replacement volume can be created on the same cluster as the original volume. Or, it can be imported to an entirely different cluster.
Volume snapshots can be backed up to any supported external storage repository in the cloud. For example, Amazon S3 Cloud, Google Cloud Storage, or Azure Cloud. Once a volume snapshot has been pushed to an external storage repository (repo), it can be removed from the local cluster, freeing up local storage space.
Note
You can use the volume snapshot and volume backup features only for volumes that are not part of a Robin bundle application.
Important
Only users with the superadmin role can create volume snapshots, however, they can be shared to users within tenants.
Topics covered in this section:
|
Create a volume snapshot contaning data from one or more volumes |
|
List volume snapshots |
|
Delete a volume snapshot |
6.13.1. Create a volume snapshot¶
Run the following command to create a volume snapshot containing data from one or more volumes:
# robin volume-snapshot create <volume_list>
--snapname <snapshot_name>
|
Comma separated list of names of volumes to snapshot |
|
Custom volume snapshot name |
Note
The robin volume list command, detailed here, can be used to view a list of available volumes and their respective names.
Example
# robin volume-snapshot create pvc-fc5e7624-05f2-4b31-b266-386fa0771458 --snapname test135 --wait
Job: 16352 Name: VolumeSnapshotCreate State: VALIDATED Error: 0
Job: 16352 Name: VolumeSnapshotCreate State: PROCESSED Error: 0
Job: 16352 Name: VolumeSnapshotCreate State: PREPARED Error: 0
Job: 16352 Name: VolumeSnapshotCreate State: FINALIZED Error: 0
Job: 16352 Name: VolumeSnapshotCreate State: NOTIFIED Error: 0
Job: 16352 Name: VolumeSnapshotCreate State: COMPLETED Error: 0
6.13.2. List all volume snapshots¶
Run the following command to view a list of volume snapshots on your cluster:
# robin volume-snapshot list [<name>] --snapshotid <snapshot ID> --volume-name <volume name> --full --wide --json
nameSnapshot name
--snapshotidList the volume snapshot for a given Id (You can’t be use in conjunction with ‘name’)
--volume-nameOnly list volume snapshots that contain a given volume
--fullDisplay additional information about each volume snapshot (e.g., volumes)
--wideInclude the list of volumes snapshots were generated for at the end of each output row. Note that the
--wideand--fullcommand line options cannot be used together.
--jsonOutput in JSON format
Example:
# robin volume-snapshot list +----------------------------------+-----------------------------------------------+--------+---------+----------------------+----------+ | Snapshot ID | Name | State | Backups | Creation Time | Restored | +----------------------------------+-----------------------------------------------+--------+---------+----------------------+----------+ | 14491694374f11eeb110adb2fc9c12ca | snapshot-014d958a-16ff-4019-8bc4-488cc986813a | ONLINE | 0 | 10 Aug 2023 00:25:27 | False | | 3fc99c48375511eeb6c2d3c3fc260aa7 | snapshot-01bf79ad-0809-43db-9e55-69abbd2eb985 | ONLINE | 0 | 10 Aug 2023 01:09:37 | False | | 80cfce80375711ee864add69607308f7 | snapshot-04142a4b-e932-47ad-a086-b75db3dade71 | ONLINE | 0 | 10 Aug 2023 01:25:45 | False | | 9078c3f6375111ee8126adcdc15ea3cd | snapshot-05ab57ef-669d-4445-9cb9-dcc763fb86f3 | ONLINE | 0 | 10 Aug 2023 00:43:15 | False | | 4be83758374e11ee94ffe1228ff4120a | snapshot-0666fad4-4ebc-4cf5-9f94-53efc9b0ae8b | ONLINE | 0 | 10 Aug 2023 00:19:51 | False | =========================================================================================================================================
In order to view all volume snapshots currently present on the cluster and some additional details for each snapshot such as the state, number of clones/backups that have been created etc., issue the following command:
End Point: /api/v3/robin_server/volumes/snap
Method: GET
URL Parameters: None
Data Parameters: None
Port: RCM Port (default value is 29442)
Headers:
Authorization: <auth_token>: Authorization token to identify which user is sending the request. The token can be acquired from the login API.
Success Response Code: 200
Error Response Code: 500 (Internal Server Error), 401 (Unauthorized Error)
Example Response:
Output
{
"items":{
"media":72,
"respoolid":1,
"replication":2,
"storstatus_str":"ONLINE",
"volgroupname":"file-collection-1602783389182.edc29729-1bfe-4da2-87af-3e0e1fbbe8bd.72.1.0a6b2335-0b82-4c88-815c-c47f0a7e308d",
"volumeid":1,
"qgroupid":1,
"tags":[
],
"rsize":10737418240,
"slice_size":1073741824,
"appname":"default",
"storstatus":1,
"current_snapshotid":1,
"compress_str":"LZ4",
"lsize":5368709120,
"name":"file-collection-1602783389182.edc29729-1bfe-4da2-87af-3e0e1fbbe8bd",
"snapshot_space_used":0,
"snapshot_space_limit":0,
"active_snapshot_count":0,
"ctime":1602783389,
"volgroupid":1,
"volume_ops":{
"volops_list":[
]
},
"state":2,
"genkey":1602783389367989,
"fsfree":0,
"alloc":[
[
{
"devpath":"\/dev\/disk\/by-id\/scsi-3600224800aba82fe29ee8770cc6440f5",
"node":"cscale-82-34.robinsystems.com",
"allocation":5,
"wwn":"0x3600224800aba82fe29ee8770cc6440f5"
}
],
[
{
"devpath":"\/dev\/disk\/by-id\/scsi-360022480b410ddebc69ecdea69c2bf34",
"node":"cscale-82-35.robinsystems.com",
"allocation":5,
"wwn":"0x360022480b410ddebc69ecdea69c2bf34"
}
]
],
"total_snapshot_count":1,
"status":1,
"partitions":0,
"storstate_str":"READY",
"fssize":0,
"compress_algo":1,
"asize":10737418240,
"vtype_str":"REGULAR",
"mounts":[
{
"nodeid":1,
"zonename":"default",
"state":14,
"zoneid":1602807544,
"readonly":0,
"state_str":"MOUNTED",
"nodename":"cscale-82-35.robinsystems.com",
"mntpath":"\/dev\/sdd"
}
],
"data_state":7,
"block_size":4096,
"protection":1,
"vtype":0,
"psize":469762048,
"resync_state":1,
"media_str":"HDD",
"next_backupid":1,
"multinode_mounting":0,
"appid":1,
"resync_progress":100,
"next_snapshotid":2,
"encrypt_str":"NONE",
"parent_alloc":[
],
"cloud_psize":0,
"encrypt_algo":0
}
}
6.13.3. Delete a volume snapshot¶
To delete a volume snapshot run the following command:
# robin volume-snapshot delete <name>
--yes
|
Volume snapshot name |
|
Do not prompt the user for confirmation |
Example
# robin volume-snapshot delete test123
Are you sure you want to delete [y/n]? y
6.14. Managing Volume Backups¶
As was covered in the section on Managing Volume Snapshots, volume snapshots contain crash-consistent snapshots of one or more storage volumes. Volume backups allow users to save storage space and provide a means of recreating (recovering) the volumes when required. Robin CNS allows you to push existing volume snapshots to a registered external cloud storage repository. Alternatively, a volume snapshot will be created on the fly as part of the volume backup creation process.
Volume backups can coexist with application backups in a cloud storage repository. The Robin CNS CLI displays the Volume backups as Type VOLUME. Individual volumes from a volume backup can be imported to the same cluster as the original volume, or they can be imported to an entirely different cluster.
Starting with Robin CNS v5.4.6, you can also create volume backups containing encrypted volumes (you can also create application backups containing encrypted volumes). Details on how this is done is provided in the Backup and Restore Applications and Volumes section.
Note
You can use the volume snapshot and volume backup features only for volumes that are not part of Robin bundle applications.
When you import volumes from a backup containing data from multiple volumes, you can only import one volume at a time. Also, there is no relationship/dependency between backed-up volumes, imported volumes, and applications.
Topics covered in this section:
|
Push a volume snapshot to an external cloud storage repo |
|
List all volume snapshot backup |
|
Delete a volume snapshot backup |
6.14.1. Create a volume backup¶
As described previously a volume backup is essentially a volume snapshot that is pushed a registered external cloud repository. As a result, either a pre-existing volume snapshot needs to be specified as part of the creation command or a list of volumes from which to create a volume snapshot on the fly needs to be given. You can back up regular and encrypted volumes. To create a volume backup, run the following command:
# robin volume-backup create <repo_name>
--snapname <snapshot_name>
--snapshotid <snapshotid>
--volume-list <volume_list>
--backup-name <backup_name>
--username <username>
--tenant <tenant>
--retain-snapshot
--timeout <timeout>
|
Registered external cloud repository name |
|
Name of volume snapshot to backup. Note this option cannot be used if either |
|
ID of volume snapshot to backup. Note this option cannot be used if either |
|
Comma separated the list of names for volumes to include in the auto-generated snapshot. Note this option cannot be used if either |
|
Name to assign to the newly created volume backup. |
|
Username of user the volume backup is bound to. |
|
Name of tenant the volume backup is bound to. |
|
Don’t delete the temporary snapshot that gets created when |
|
Number of minutes to wait for the backup to finish. (default is no timeout) |
Important
Before a pre-existing volume snapshot can be backed up, all volumes associated with the aforementioned snapshot must be attached to the respective external repo where the volume snapshot will be pushed. The robin volume attach-repo command, detailed here, can be used to achieve this.
Example 1 (Creating a volume backup directly by specifying a list of volumes):**
# robin volume-backup create remote-repo --volume-list pvc-6d1fbf4b-13ab-419e-8fc1-0c6c250959b3,pvc-5e414c69-b922-4caa-8c95-fab4e8b21924 --backup-name test
Example 2 (Creating a volume backup using a pre-existing volume snapshot):**
# robin volume-backup create testrepo --snapshotid 836b180aaa7a11ecbf5e61eb92e252a8 --backup-name testbackup
Creating volume backup 'testbackup' from snapshot '836b180aaa7a11ecbf5e61eb92e252a8'
6.14.2. List volume backups¶
In order to view a list of all volume backups the current user is entitled to view alongside details such as the ID, name, associated external repo, state, and creation time for each backup, run the following command:
# robin volume-backup list [<name>]
--backupid <backupid>
--snapshot-name <snapshot_name>
--volume-name <volume_name>
--full
|
List the volume backup for a given name. Note this parameter is optional |
|
List the volume backup for a given ID. Note this option can’t be used in conjunction with |
|
Only list volume backups that contain a given snapshot |
|
Only list volume backups that contain a given volume |
|
Display additional information about each volume backup (e.g., volumes) |
Example
# robin volume-backup list
+----------------------------------+------------+----------+------------------+---------------+----------------------+
| Backup ID | Name | Repo | State | Snapshot | Creation Time |
+----------------------------------+------------+----------+------------------+---------------+----------------------+
| 270e5b24aa7b11eca35ff9a1e6fb8ced | testbackup | testrepo | Push in progress | Deleted | 19 Mar 2022 06:19:13 |
+----------------------------------+------------+----------+------------------+---------------+----------------------+
6.14.3. Delete a volume backup¶
To delete a volume backup, run the following command:
# robin volume-backup delete <backup_name>
--yes
|
Name of volume backup to delete |
|
Do not prompt the user for confirmation |
Example
# robin volume-backup delete testbackup --yes --wait
Job: 115 Name: VolumeBackupDelete State: VALIDATED Error: 0
Job: 115 Name: VolumeBackupDelete State: PREPARED Error: 0
Job: 115 Name: VolumeBackupDelete State: COMPLETED Error: 0
6.15. Encrypted volume backup and recovery¶
Volume backups contain snapshots of one or more storage volumes. The storage volumes can be unencrypted, or they can contain encrypted data. The procedures for creating a volume backup are the same, regardless of volume data protection.
When creating a volume backup, you can specify the name or Id of a volume snapshot, or you can supply a list of volumes to include in the backup (with this option, a volume snapshot to be created dynamically). Unless otherwise specified, all dynamically generated volume snapshots are removed on successful completion of the backup operation. Following are examples of how to crate a volume backup and how to import a volume from that backup, on the same cluster as the original volume, and on a different cluster.
Prerequisites
You must have a registered external repository for backup purposes. For more information, see Register a repo.
You must have the same external repository registered on both source and destination clusters.
Source and destination clusters must be running Robin CNS v5.4.6.
6.15.1. Create a volume backup containing an encrypted volume¶
Create a volume backup by running the following command:
# robin volume-backup create <repo_name> --volume-list <volume_list>Note
You can run
robin repo listto view a list of registered repos androbin volume listto view a list of volumes. For more information, see Managing Volume Backups.Example
# robin volume-backup create testrepo --volume-list pvc-8aa8aefa-b5d6-4b37-888c-3ce4495dff13 --wait Creating volume backup Job: 188 Name: VolumeBackupCreate State: PROCESSED Error: 0 Job: 188 Name: VolumeBackupCreate State: WAITING Error: 0 Job: 188 Name: VolumeBackupCreate State: AGENT_WAIT Error: 0 Job: 188 Name: VolumeBackupCreate State: FINALIZED Error: 0 Job: 188 Name: VolumeBackupCreate State: COMPLETED Error: 0
Verify the created backup by running robin backup list and robin backup info <backupid> commands. For more information about the backup commands, see Backup and Restore Applications.
6.15.2. Import an encrypted volume on the same cluster as the original volume¶
When you import a volume from a volume backup, you essentially create a new volume and import the data from the original volume (via a snapshot of the volume in the backup). Hydration of volume data is handled over time by the storage layer.
When importing the volume, you can create the new volume directly, or you can create a PVC and let Kubernetes handle PV and volume binding (via calls to the Robin CSI). Note that taking the direct route only creates the volume, leaving the creation of PVC and PV up to you, along with binding to the volume.
When importing a volume from a volume backup, you need to provide the ID of the volume backup, the name of the the volume you wish to import, and the resource pool (rpool) where storage resources for the new volume will be allocated from. You can also specify various optional storage parameters, such as media type, replication, fault domain, etc., and specify a storage class.
Note
When specifying a storage class, you need to make sure that it doesn’t contain any parameters that conflict with immutable values in the original volume (e.g, fstype, compression, encryption, and blocksize).
6.15.3. Export a volume backup for use on a different cluster¶
Before you can import a volume from a volume backup on a cluster other than the one where the backup was created, you first need to export the backup. The operation generates an encoded token that contains key details of the backup and the repo where it resides. Issue the following command to generate a backup token:
# robin backup export <backupid> --force
Example
# robin backup export 0cae83d46b4411eeb1ecb9f9119206b3 --force
eyJiYWNrdXBfaWQiOiAiMGNhZTgzZDQ2YjQ0MTFlZWIxZWNiOWY5MTE5MjA2YjMiLCAiYmFja3VwX3BhdGgiOiAiZGV2L3RvbS8xNjk2OTE3NzE2L3ZvbHVtZS1zbmFwc2hvdHMvMGM4ZWQ3NjY2YjQ0MTFlZThlMTZhNzZhN2Q2OGM5Y2YvYXBwY29uZmlnL2JhY2t1cF9tZXRhZGF0YS5qc29uIiwgInJlcG9fdHlwZSI6ICJHQ1MiLCAicmVwb19wYXJhbXMiOiB7InJlcG9fdHlwZSI6ICJHQ1MiLCAiY3JlZGVudGlhbHMiOiB7InByb2plY3RfaWQiOiAicm9jay1yYW5nZS0yMDc2MjIifSwgImJ1Y2tldCI6ICJ0ZXN0YnVja2V0LTEwMjkzODQ3NTYiLCAicGF0aCI6ICJkZXYvdG9tLyJ9LCAiYmFja3VwX2tleSI6ICI4NjhmY2M0YTU3YzEwMTUzNDNkMGJjZjAzYWU0YjE4YTI1OWQ2Y2QyMWQ1NjFlMzM1MzRjZTMyNDM1YWJkZDc5In0=
Important
You must copy and store the backup token securely. You will need it when importing the backup on the destination cluster.
Note
The backup token is encoded into a single string in an unencrypted form and it contains details about the backup key and the storage repo. Due to this, you must use the
--forceoption when exporting a backup.If you do not want to include the backup key as part of the backup token, you can use
--exclude-backup-keyand export and import key separately using therobin backup export-keyandrobin backup import-keycommands.
6.15.4. Import a volume backup from a different cluster¶
Before you can import a volume from a backup on a different cluster, you must first import the backup on that cluster. Take the backup_token generated in the previous step and securely transport it to the cluster where you would like to import the backup. Then issue the following command:
# robin backup import <backupid>
Example
# robin backup import eyJiYWNrdXBfaWQiOiAiYTczYjY3ODA2M2E4MTFlZTk2MmE4ZDA3MjUwZmI1NDAiLCAiYmFja3VwX3BhdGgiOiAiZGV2L3RvbS8xNjk1NjQxMzQ5L3ZvbHVtZS1zbmFwc2hvdHMvYTcxZTg5NjI2M2E4MTFlZWFlOTM4N2FlNjdlYjQwOWEvYXBwY29uZmlnL2JhY2t1cF9tZXRhZGF0YS5qc29uIiwgInJlcG9fdHlwZSI6ICJHQ1MiLCAicmVwb19wYXJhbXMiOiB7InJlcG9fdHlwZSI6ICJHQ1MiLCAiY3JlZGVudGlhbHMiOiB7InByb2plY3RfaWQiOiAicm9jay1yYW5nZS0yMDc2MjIifSwgImJ1Y2tldCI6ICJ0ZXN0YnVja2V0LTEwMjkzODQ3NTYiLCAicGF0aCI6ICJkZXYvdG9tLyJ9LCAiYmFja3VwX2tleSI6ICI5N2ViODllYThkMTNlYjNhODc4NzNmMDkyNzRjMGUzOGQ0MTc3NjBjYjEzYmZhOTZlMjY2YzUxYmYwNTczMmNjIn0= --wait
Job: 146 Name: K8sBackupImport State: PROCESSED Error: 0
Job: 146 Name: K8sBackupImport State: PREPARED Error: 0
Job: 146 Name: K8sBackupImport State: COMPLETED Error: 0
Now verify that the backup has been added to the local cluster:
# robin backup list
+----------------------------------+--------------------------+--------+----------+----------------------------+----------+
| Backup ID | Backup Name | Type | Repo | Snapshot Name | State |
+----------------------------------+--------------------------+--------+----------+----------------------------+----------+
| a73b678063a811ee962a8d07250fb540 | volume-backup-1696528646 | VOLUME | testrepo | volume-snapshot-1696528646 | Imported |
+----------------------------------+--------------------------+--------+----------+----------------------------+----------+
Once the backup containing the volume you wish to import has been imported into your cluster, you can proceed with importing the volume.
6.15.5. Import a volume backup with encrypted volumes on a different cluster¶
When importing a backup that contains encrypted volumes, you need access to the backup key generated when the backup was pushed to the external repo. The backup key is necessary for the decryption of volume keys stored in the backup metadata.
If you are importing the backup on the same cluster as the one where the backup originated from (perhaps because the backup had previously been purged from the cluster), then nothing special is required. That’s because the key contained in the backup metadata was encrypted using the master key for the local cluster.
On the other hand, if you are importing the backup into a different cluster, you need to make sure that the backup key is provided when performing the backup import operation. There are a number of ways you can provide the backup key when importing a backup.
By default, the unencrypted backup key is included in the backup token generated when exporting the backup. This ensures that the backup key will be available when importing the backup on the terget cluster.
Alternatively, you can export the backup key on the source cluster and import it on the target cluster. The backup key gets added as a secret and will be available when the backup gets imported. Note that if you import the backup key, you should not include the backup key in the backup token. To do this, just include the –exclude-backup-key command line option.
6.15.6. Import an encrypted volume on another cluster¶
While importing a volume from a backup, you can create the new volume directly, or you can elect to create a PVC.
When creating the volume directly, you can specify the volume name. You cannot, however, provide a name for the bound PVC.
You cannot provide a volume name when you want to create a PVC, however you can provide a desired name for the PVC.
In both cases, you can provide volume-related parameters. For example, replication, faultdomain, access mode, etc.
After the import process is complete, the new volume will contain the same data as the source volume.
Prerequisite
When importing a volume from the backup, the Storageclass parameters must be the same on both source and destination clusters.
You must have the same external repository registered on both source and destination clusters.
When importing a volume from the external repository to a different cluster, the destination cluster must be running Robin CNS v5.4.6.
Log in to the destination cluster to import the backup.
Verify that the same external repository is registered on the destination cluster by running
robin repo listcommand.Note
If it is not registered, you must register the repo before importing the backup. For more information, see Register a repo.
Import the backup token generated on the source cluster by running the following command:
# robin backup import <backup_token>
Example
# robin backup import eyJiYWNrdXBfaWQiOiAiMGNhZTgzZDQ2YjQ0MTFlZWIxZWNiOWY5MTE5MjA2YjMiLCAiYmFja3VwX3BhdGgiOiAiZGV2L3RvbS8xNjk2OTE3NzE2L3ZvbHVtZS1zbmFwc2hvdHMvMGM4ZWQ3NjY2YjQ0MTFlZThlMTZhNzZhN2Q2OGM5Y2YvYXBwY29uZmlnL2JhY2t1cF9tZXRhZGF0YS5qc29uIiwgInJlcG9fdHlwZSI6ICJHQ1MiLCAicmVwb19wYXJhbXMiOiB7InJlcG9fdHlwZSI6ICJHQ1MiLCAiY3JlZGVudGlhbHMiOiB7InByb2plY3RfaWQiOiAicm9jay1yYW5nZS0yMDc2MjIifSwgImJ1Y2tldCI6ICJ0ZXN0YnVja2V0LTEwMjkzODQ3NTYiLCAicGF0aCI6ICJkZXYvdG9tLyJ9LCAiYmFja3VwX2tleSI6ICI4NjhmY2M0YTU3YzEwMTUzNDNkMGJjZjAzYWU0YjE4YTI1OWQ2Y2QyMWQ1NjFlMzM1MzRjZTMyNDM1YWJkZDc5In0= --wait Job: 148 Name: K8sBackupImport State: PROCESSED Error: 0 Job: 148 Name: K8sBackupImport State: PREPARED Error: 0 Job: 148 Name: K8sBackupImport State: COMPLETED Error: 0
Verify the imported backup by running the following command:
# robin backup list
After importing the backup, you need to import the volume by running the following command. For more information see, Import a volume.
Note
When importing the volume from the backup, you can directly create a volume or create PVC. You can provide a volume name when you directly create a volume. However, you cannot provide a name for the volume.
You cannot provide a volume name when you want to create a PVC, but you can provide a desired name for the PVC.
In both cases, you can provide volume-related parameters. For example, replication, faultdomain, access mode, etc.
Run
robin backup listandrobin backup info <backupid>commands to know the backup ID and volume name respectively.To create a volume directly, run the following command:
# robin volume import <backupid> <import_volume> --name <custom_volume name>
Note
You can run the
robin volume listto verify the newly created volume androbin volume info <volume name>to view the details.Example
# robin volume import 0cae83d46b4411eeb1ecb9f9119206b3 pvc-8aa8aefa-b5d6-4b37-888c-3ce4495dff13 default --name imported_82_64 --wait Job: 149 Name: VolumeImport State: PROCESSED Error: 0 Job: 149 Name: VolumeImport State: WAITING Error: 0 Job: 149 Name: VolumeImport State: COMPLETED Error: 0
To create a PVC, run the following command:
# robin volume import <backupid> <import_volume> <namespace> --create-pvc --pvc-name <custom-pvc-name>
Example
# robin volume import 0cae83d46b4411eeb1ecb9f9119206b3 pvc-8aa8aefa-b5d6-4b37-888c-3ce4495dff13 default --create-pvc --pvc-name imported-82-64-pvc --wait Job: 152 Name: VolumeImport State: PROCESSED Error: 0 Job: 152 Name: VolumeImport State: AGENT_WAIT Error: 0 Job: 152 Name: VolumeImport State: COMPLETED Error: 0 Volume 'pvc-a517011f-ea1e-4e01-91b6-38255bc0ac25' is imported
6.16. Auto Volume Cleanup¶
When you remove all nodes from Kubernetes Cluster that serve replicas of a volume, the Auto Volume Cleanup feature automatically deletes the volumes and cleans up the data and stale devices associated with the volumes.
If you want to retain your data before removing the node, you must first manually evacuate all volumes from the drives of that node and then remove the node from the cluster. For more information, see Removing Nodes. Robin recommends evacuating the volumes before removing a node because removing a node entails data loss.
Note
By default, the Auto Volume Cleanup feature is disabled.
6.16.1. Points to consider for the Auto Volume Cleanup feature¶
When you remove a node from a cluster that has a replica for a volume, the status of the volume is changed from sync to the degraded status.
If you remove all nodes from the cluster that have a replica for a volume, the volume is automatically deleted, and the data and the stale devices are cleaned up from the removed nodes.
A node that has no replica of a volume can easily be removed from the cluster.
When you enable this feature, thin clones and snapshots created for these volumes are deleted. The volumesnapshots will also be rendered useless).
Thick clones may or may not be affected, depending on their allocated node.
When you unmount and delete a volume from the cluster, Kubernetes triggers an event for the associated PV. If a PVC exists, it also triggers an event for that.
To enable or disable the Auto Volume Cleanup feature, you need to set the host_vol_drain key in the robin-config ConfigMap to True or False.
6.16.2. Enable Auto Volume Cleanup¶
You can enable the Auto Volume Cleanup feature. If you remove a node from the Kubernetes cluster, this feature automatically deletes the volumes and cleans up data and stale devices associated with the volumes from the removed node.
Prerequisite
Volumes mounted on the removed node must not have any active replicas.
Complete the following steps to enable the Auto Volume Cleanup feature:
Create a
robin-config.yamlfile using the following example:
apiVersion: v1 data: server: | host_vol_drain=True kind: ConfigMap metadata: labels: app.kubernetes.io/instance: robin app.kubernetes.io/managed-by: robin.io app.kubernetes.io/name: robin name: robin-config namespace: robinio
Create the
robin-configConfigMap using therobin-config.yamlfile:
# kubectl create -f robin-config.yaml
Once the robin-config ConfigMap is created, the Auto Volume Cleanup feature will be enabled.
6.16.3. Disable Auto Volume Cleanup¶
You can disable the Auto Volume Cleanup feature at any time.
Complete the following steps to disable the Auto Volume Cleanup feature:
Edit the
robin-configConfigMap:
# kubectl edit configmap -n robinio robin-config
Update the value of the
host_vol_drainkey toFalseand save it:
host_vol_drain=FalseExample
apiVersion: v1 data: server: | host_vol_drain=False kind: ConfigMap metadata: labels: app.kubernetes.io/instance: robin app.kubernetes.io/managed-by: robin.io app.kubernetes.io/name: robin name: robin-config namespace: robinio
Once you save the robin-config ConfigMap, the changes will take effect automatically and the Auto Volume Cleanup feature will be disabled.
6.17. Thin Provisioning¶
Robin supports thin provisioning of the Volumes so that large storage volumes can be created without upfront physical capacity. Thin provisioning is disabled by default and can be enabled on a cluster by updating the storage_provision_factor config attribute to value greater than 1.
However precaution must be taken when thin provisioning is enabled especially if there exists application(s) on the cluster that write data to disk resulting in physical space usage. This is because out of space issues may be encountered by said application(s) due to the aforementioned writes. For example, a drive of size 100GB can host 2 volumes with a total size of 200GB when thin provisioning is enabled but if the combined size of the data written to those volumes goes above 100GB, the volumes would see ERR_NOSPACE errors and filesystem would be moved to read-only state. As a result, thin provsioned clusters must be actively managed.
6.17.1. View current Thin Provisioning factor¶
In order to check the current value for storage_provision_factor config attribute and also the current storage capacity on the cluster, run the following commands. Note that the max amount that can be allocated from the drives is roughly 80% of their capacity given the value of the storage_provision_factor.
# robin config list server storage_provision_factor
Section | Attribute | Value
--------+--------------------------+-------
server | storage_provision_factor | 0.8
# robin drive list
ID | WWN | Host | RPool | Path /dev/disk/by-id | Size(GB) | Movable | Type | Free/Max(GB) | Vols | Role | Status | LastOpr | Write Unit
---+------------------------------------+---------------+---------+----------------------------------------+----------+---------+------+--------------+------+---------+--------+---------+------------
1 | 0x60022480aa4ad2c596745b897d5051 | centos-02-250 | default | scsi-360022480aa4ad2c596745b897d5051 | 50 | N | HDD | 38/38 (100%) | 0/10 | Storage | ONLINE | READY | 512
2 | 0x600224806c9386e9faab2c271b2581 | centos-02-250 | default | scsi-3600224806c9386e9faab2c271b2581 | 100 | N | HDD | 38/38 (100%) | 1/10 | Storage | ONLINE | READY | 4096
Only storage disks are shown. Issue `robin disk list --role all` to view all disks
6.17.2. Update Thin Provisioning factor¶
In order to update the thin provisioning factor and validate that the max amount that can be allocated. from the drives has been adjusted accordingly, run the following commands. Note that the max allocatable amount is roughly 200% of each drive’s respective capacity given the value of the storage_provision_factor.
Note
When the storage_provision_factor attribute has a value of greater than 1, thin provisioning is enabled. In the case that it is lower than 1, thin provisioning is disabled.
# robin config update server storage_provision_factor 2
The 'server' attribute 'storage_provision_factor' has been updated
# robin config list server storage_provision_factor
Section | Attribute | Value
--------+--------------------------+-------
server | storage_provision_factor | 2
# robin drive list
ID | WWN | Host | RPool | Path /dev/disk/by-id | Size(GB) | Movable | Type | Free/Max(GB) | Vols | Role | Status | LastOpr | Write Unit
---+------------------------------------+---------------+---------+----------------------------------------+----------+---------+------+--------------+------+---------+--------+---------+------------
1 | 0x60022480aa4ad2c596745b897d5051 | centos-02-250 | default | scsi-360022480aa4ad2c596745b897d5051 | 50 | N | HDD | 96/96 (100%) | 0/10 | Storage | ONLINE | READY | 512
2 | 0x600224806c9386e9faab2c271b2581 | centos-02-250 | default | scsi-3600224806c9386e9faab2c271b2581 | 100 | N | HDD | 96/96 (100%) | 1/10 | Storage | ONLINE | READY | 4096
Only storage disks are shown. Issue `robin disk list --role all` to view all disks
6.18. Monitoring Robin via Prometheus¶
Robin exposes an endpoint that publishes metrics information in Prometheus format. Users can install Prometheus and configure it to scrape metrics that are relevant to the Robin software. In order to do that the following section needs to be part of Prometheus configuration:
scrape_configs:
- job_name: robin
metrics_path: /metrics
scheme: https
static_configs:
- targets: ['<fill-in-host-ip-of-robin-server>:29446']
In the above configuration snippet you need to fill in the IP address or host name of the node where the Robin master is running. If you are running a Robin high availability (HA) cluster then you can use the virtual IP (VIP) that you set while installing the Robin software.
All metrics published by Robin start with the prefix robin_ and Kubernetes metrics with the prefix kubelet_.
Example
# Robin storage/server metrics
robin_server{instance="centos-60-58"} 1
robin_disk_freeslices{wwn="0x60022480ba5f21af8e40ea473bcb9b8a",devpath="/dev/sda",node_idx="1", node_name="centos-60-58"} 0
robin_disk_freespace{wwn="0x60022480ba5f21af8e40ea473bcb9b8a",devpath="/dev/sda",node_idx="1", node_name="centos-60-58"} 107374182400
robin_disk_freevols{wwn="0x60022480ba5f21af8e40ea473bcb9b8a",devpath="/dev/sda",node_idx="1", node_name="centos-60-58"} 0
robin_disk_maxsegs{wwn="0x60022480ba5f21af8e40ea473bcb9b8a",devpath="/dev/sda",node_idx="1", node_name="centos-60-58"} 0
robin_disk_maxvols{wwn="0x60022480ba5f21af8e40ea473bcb9b8a",devpath="/dev/sda",node_idx="1", node_name="centos-60-58"} 0
6.18.1. Storage Metrics¶
By default, Robin exposes certain storage metrics that can be scraped and manipulated by end users to suit their needs. These metrics cover a wide variety of objects, ranging from device based statistics such as their usage to details on individual application read and write operations on a volume. Each section detailed below is a grouping of related metrics with a description of each metric attached.
6.18.1.1. Node Metrics¶
Metric |
Description |
robin_node_state |
The states of a node. The states can be: 0 = ready, 1 = not ready, 2 = partially ready |
robin_node_maintenance_mode |
The state of node maintenance mode. Value 0 = disabled and 1 = enabled |
6.18.1.2. Disk Metrics¶
Listed below are several disk related metrics that are exposed by Robin:
Metric |
Description |
robin_disk_state |
The state of disks can be any of the following:
|
robin_disk_maintenance_mode |
The disk maintenance mode state. The state can be 0 = enabled 1 = disabled |
robin_disk_freeslices |
The number of free slices on a disk for a new storage request from an application. Each slice is 1GB in size. |
robin_disk_maxsegs |
The number of segments in a disk. Each segment is 32MB in size. |
robin_disk_nsegs |
The number of used segments in a disk. |
robin_disk_rawused |
The used space of a disk, represented in bytes. |
robin_disk_size |
The capacity of a disk, represented in bytes. |
robin_disk_freespace |
The free space on a disk that can be used for allocations, represented in bytes. |
robin_disk_maxvols |
The maximum number of Kubernetes volumes that can be stored in a disk. The default size is 10. |
robin_disk_nvols |
The number of Kubernetes volumes currently stored on a disk. |
robin_disk_freevols |
The number of additional Kubernetes volumes that can be accommodated on a disk. |
robin_disk_maxslices |
The maximum number of slices that can be accommodated on a disk. |
robin_disk_nslices |
The number of used slices of a disk. |
6.18.1.3. Read/Write Metrics¶
Listed below are several read and write based metrics related to volumes that are exposed by Robin:
Metric |
Description |
robin_rdvm_vol_curr_bytes_read |
The number of bytes read from a volume. |
robin_rdvm_vol_curr_bytes_written |
The number of bytes written to a volume. |
robin_rdvm_vol_curr_read_ios |
The number of read requests on a volume. |
robin_rdvm_vol_curr_read_qdepth |
The number of pending reads in the queue for a volume. |
robin_rdvm_vol_curr_read_usecs |
The cumulative time in microseconds taken for all read requests on a volume. |
robin_rdvm_vol_curr_write_ios |
The number of write requests to a volume. |
robin_rdvm_vol_curr_write_qdepth |
The number of pending writes in the queue for a volume. |
robin_rdvm_vol_curr_write_usecs |
The cumulative time in microseconds taken for all write requests on a volume. |
The following table lists several read and write based metrics related to disks that are exposed by Robin:
Metric |
Description |
robin_rdvm_dev_curr_bytes_read |
The number of bytes read from a specified storage device for a specified volume. |
robin_rdvm_dev_curr_bytes_written |
The number of bytes written to a specified storage device for a specified volume. |
robin_rdvm_dev_curr_read_ios |
The number of reads from a specified storage device for a specified volume. |
robin_rdvm_dev_curr_read_qdepth |
The number of pending reads in the queue on a specified storage device for a specified volume. |
robin_rdvm_dev_curr_read_usecs |
The cumulative time in microseconds taken by all read requests on a specified storage device for a specified volume. |
robin_rdvm_dev_curr_write_ios |
The number of writes to the specified storage device for a specified volume. |
robin_rdvm_dev_curr_write_qdepth |
The number of pending writes in the queue on a specified storage device for a specified volume. |
robin_rdvm_dev_curr_write_usecs |
The cumulative time in microseconds taken by all write requests on a specified storage device for a specified volume. |
The following table lists several read and write based metrics related to applications that are exposed by Robin:
Metric |
Description |
robin_rio_vol_curr_bytes_read |
The number of bytes that an application reads from a volume. |
robin_rio_vol_curr_bytes_written |
The number of bytes that an application wrote to a volume. |
robin_rio_vol_curr_read_ios |
The number of the read requests that an application performed on a volume. |
robin_rio_vol_curr_read_qdepth |
The number of the pending read requests from an application on a volume. |
robin_rio_vol_curr_read_usecs |
The cumulative time in microseconds taken for all read requests on a volume. |
robin_rio_vol_curr_write_ios |
The number of writes requested by an application on a volume. |
robin_rio_vol_curr_write_qdepth |
The number of pending writes from an application on a volume. |
robin_rio_vol_curr_write_usecs |
The cumulative time in microseconds taken for all write requests on a volume. |
6.18.1.4. Snapshot Metrics¶
The following table Lists several snapshot related metrics that are exposed by Robin:
Metric |
Description |
robin_snap_dsegs |
The number of on-disk segments written by a snapshot. |
robin_snap_nclones |
The number of clones of a snapshot. |
robin_snap_osegs |
The total number of segments owned by a snapshot. |
robin_snap_state |
The snapshot state of a volume. |
6.18.1.5. Volume Metrics¶
The following table Lists several volume related metrics that are exposed by Robin:
Metric |
Description |
robin_vol_nsnaps |
The number of snapshots of a volume. |
robin_vol_rawused |
The number of used bytes for a volume. |
robin_vol_size |
The allocated size of a volume. |
robin_vol_status |
The state of the volume. Values: 0 = ONLINE, 1 = OFFLINE, 2 = FAULTED, 3 = DEGRADED, 4 = INVALID, 5 = UNKNOWN |
robin_vol_storstatus |
The state of volume’s storage manager. Values: 0 = ONLINE, 1 = OFFLINE, 2 = FAULTED, 3 = DEGRADED, 4 = INVALID, 5 = UNKNOWN |
robin_vol_mount_node_id |
The node ID where the volume is mounted. |
robin_vol_snapshot_space_used |
The space used by volume snapshot in bytes. |
robin_vol_snapshot_space_limit |
The volume snapshot space limit in bytes. |
robin_vol_total_snapshot_count |
The total snapshot count in a volume. |
6.18.1.6. Manager Services Metrics¶
The following table lists different Robin manager services status metrics:
Metric |
Description |
robin_manager_services_robin_server |
The status of Robin server. Value 0 = up and 1 = down |
robin_manager_services_consul_server |
The status of consul server. Value 0 = up and 1 = down |
robin_manager_services_robin_event_server |
The status of Robin event server. Value 0 = up and 1 = down |
robin_manager_services_stormgr_server |
The status of storage manager server. Value 0 = up and 1 = down |
robin_manager_services_pgsql |
The status of PostgresSQL. Value 0 = up and 1 = down. |
robin_manager_services_robin_master |
The status to know if current node is master or not. Value 0 = True |
6.18.1.7. Agent Services Metrics¶
The following table lists different Robin agent services status metrics:
Metric |
Description |
robin_agent_services_robin_agent |
The status of Robin agent service. Value 0 = up and 1 = down |
robin_agent_services_iomgr_service |
The status of Robin agent IO Manager service. Value 0 = up and 1 = down |
robin_agent_services_monitor_server |
The status of Robin agent monitor server. Value 0 = up and 1 = down |
robin_agent_services_consul_client |
The status of Robin agent consul client. Value 0 = up and 1 = down |
6.18.1.8. CSI volume monitoring stats¶
Listed below are CSI volume monitoring related stats that are exposed by Kubernetes:
Metric |
Description |
kubelet_volume_stats_available_bytes |
The number of available bytes in the volume. |
kubelet_volume_stats_capacity_bytes |
The capacity of the volume in bytes. |
kubelet_volume_stats_used_bytes |
The number of used bytes in the volume. |
kubelet_volume_stats_inodes |
The maximum number of inodes in the volume. |
kubelet_volume_stats_inodes_free |
The number of free inodes in the volume. |
kubelet_volume_stats_inodes_used |
The number of used inodes in the volume. |
6.18.1.9. Accessing Storage Metrics¶
All the metrics detailed above can be accessed by running the following command:
# curl -k "https://<ipaddress>:29446/metrics"
The ipaddress parameter showcased above can be one of three options:
The Cluster IP associated with the
robin-masterservice. This can be attained by running the command:kubectl get svc -n robinio robin-master.The IP Address of the node which is currently running as the master manager node.
Example
# curl -k "https://172.19.47.240:29446/metrics"
# Robin storage/server metrics
robin_server{instance="master"} 1
robin_disk_freeslices{wwn="0x60022480db8a635ee6764e60418f9ef7",devpath="/dev/sda",node_idx="1", node_name="telxvm-53-150"} 0
robin_disk_freespace{wwn="0x60022480db8a635ee6764e60418f9ef7",devpath="/dev/sda",node_idx="1", node_name="telxvm-53-150"} 107374182400
robin_disk_freevols{wwn="0x60022480db8a635ee6764e60418f9ef7",devpath="/dev/sda",node_idx="1", node_name="telxvm-53-150"} 10
robin_disk_maxsegs{wwn="0x60022480db8a635ee6764e60418f9ef7",devpath="/dev/sda",node_idx="1", node_name="telxvm-53-150"} 0
robin_disk_maxvols{wwn="0x60022480db8a635ee6764e60418f9ef7",devpath="/dev/sda",node_idx="1", node_name="telxvm-53-150"} 10
robin_disk_maxslices{wwn="0x60022480db8a635ee6764e60418f9ef7",devpath="/dev/sda",node_idx="1", node_name="telxvm-53-150"} 0
robin_disk_nsegs{wwn="0x60022480db8a635ee6764e60418f9ef7",devpath="/dev/sda",node_idx="1", node_name="telxvm-53-150"} 0
robin_disk_nslices{wwn="0x60022480db8a635ee6764e60418f9ef7",devpath="/dev/sda",node_idx="1", node_name="telxvm-53-150"} 0
robin_disk_nvols{wwn="0x60022480db8a635ee6764e60418f9ef7",devpath="/dev/sda",node_idx="1", node_name="telxvm-53-150"} 0
robin_disk_size{wwn="0x60022480db8a635ee6764e60418f9ef7",devpath="/dev/sda",node_idx="1", node_name="telxvm-53-150"} 107374182400
robin_disk_rawused{wwn="0x60022480db8a635ee6764e60418f9ef7",devpath="/dev/sda",node_idx="1", node_name="telxvm-53-150"} 0
robin_disk_freeslices{wwn="0x600224803f4fe2dbd4d0bca550800af9",devpath="/dev/sdb",node_idx="1", node_name="telxvm-53-150"} 0
robin_disk_freespace{wwn="0x600224803f4fe2dbd4d0bca550800af9",devpath="/dev/sdb",node_idx="1", node_name="telxvm-53-150"} 107374182400
robin_disk_freevols{wwn="0x600224803f4fe2dbd4d0bca550800af9",devpath="/dev/sdb",node_idx="1", node_name="telxvm-53-150"} 10
robin_disk_maxsegs{wwn="0x600224803f4fe2dbd4d0bca550800af9",devpath="/dev/sdb",node_idx="1", node_name="telxvm-53-150"} 0
robin_disk_maxvols{wwn="0x600224803f4fe2dbd4d0bca550800af9",devpath="/dev/sdb",node_idx="1", node_name="telxvm-53-150"} 10
robin_disk_maxslices{wwn="0x600224803f4fe2dbd4d0bca550800af9",devpath="/dev/sdb",node_idx="1", node_name="telxvm-53-150"} 0
robin_disk_nsegs{wwn="0x600224803f4fe2dbd4d0bca550800af9",devpath="/dev/sdb",node_idx="1", node_name="telxvm-53-150"} 0
robin_disk_nslices{wwn="0x600224803f4fe2dbd4d0bca550800af9",devpath="/dev/sdb",node_idx="1", node_name="telxvm-53-150"} 0
robin_disk_nvols{wwn="0x600224803f4fe2dbd4d0bca550800af9",devpath="/dev/sdb",node_idx="1", node_name="telxvm-53-150"} 0
robin_disk_size{wwn="0x600224803f4fe2dbd4d0bca550800af9",devpath="/dev/sdb",node_idx="1", node_name="telxvm-53-150"} 107374182400
robin_disk_rawused{wwn="0x600224803f4fe2dbd4d0bca550800af9",devpath="/dev/sdb",node_idx="1", node_name="telxvm-53-150"} 0
robin_disk_freeslices{wwn="0x60022480e965d11b83cd0d074f84fe4f",devpath="/dev/sdc",node_idx="1", node_name="telxvm-53-150"} 0
robin_disk_freespace{wwn="0x60022480e965d11b83cd0d074f84fe4f",devpath="/dev/sdc",node_idx="1", node_name="telxvm-53-150"} 107374182400
robin_disk_freevols{wwn="0x60022480e965d11b83cd0d074f84fe4f",devpath="/dev/sdc",node_idx="1", node_name="telxvm-53-150"} 10
robin_disk_maxsegs{wwn="0x60022480e965d11b83cd0d074f84fe4f",devpath="/dev/sdc",node_idx="1", node_name="telxvm-53-150"} 0
robin_disk_maxvols{wwn="0x60022480e965d11b83cd0d074f84fe4f",devpath="/dev/sdc",node_idx="1", node_name="telxvm-53-150"} 10
robin_disk_maxslices{wwn="0x60022480e965d11b83cd0d074f84fe4f",devpath="/dev/sdc",node_idx="1", node_name="telxvm-53-150"} 0
robin_disk_nsegs{wwn="0x60022480e965d11b83cd0d074f84fe4f",devpath="/dev/sdc",node_idx="1", node_name="telxvm-53-150"} 0
robin_disk_nslices{wwn="0x60022480e965d11b83cd0d074f84fe4f",devpath="/dev/sdc",node_idx="1", node_name="telxvm-53-150"} 0
robin_disk_nvols{wwn="0x60022480e965d11b83cd0d074f84fe4f",devpath="/dev/sdc",node_idx="1", node_name="telxvm-53-150"} 0
robin_disk_size{wwn="0x60022480e965d11b83cd0d074f84fe4f",devpath="/dev/sdc",node_idx="1", node_name="telxvm-53-150"} 107374182400
robin_disk_rawused{wwn="0x60022480e965d11b83cd0d074f84fe4f",devpath="/dev/sdc",node_idx="1", node_name="telxvm-53-150"} 0
robin_disk_freeslices{wwn="0x60022480db8a635ee6764e60418f9ef7-centos-root",devpath="/dev/dm-0",node_idx="1", node_name="telxvm-53-150"} 0
robin_disk_freespace{wwn="0x60022480db8a635ee6764e60418f9ef7-centos-root",devpath="/dev/dm-0",node_idx="1", node_name="telxvm-53-150"} 53687091200
robin_disk_freevols{wwn="0x60022480db8a635ee6764e60418f9ef7-centos-root",devpath="/dev/dm-0",node_idx="1", node_name="telxvm-53-150"} 10
robin_disk_maxsegs{wwn="0x60022480db8a635ee6764e60418f9ef7-centos-root",devpath="/dev/dm-0",node_idx="1", node_name="telxvm-53-150"} 0
robin_disk_maxvols{wwn="0x60022480db8a635ee6764e60418f9ef7-centos-root",devpath="/dev/dm-0",node_idx="1", node_name="telxvm-53-150"} 10
robin_disk_maxslices{wwn="0x60022480db8a635ee6764e60418f9ef7-centos-root",devpath="/dev/dm-0",node_idx="1", node_name="telxvm-53-150"} 0
robin_disk_nsegs{wwn="0x60022480db8a635ee6764e60418f9ef7-centos-root",devpath="/dev/dm-0",node_idx="1", node_name="telxvm-53-150"} 0
robin_disk_nslices{wwn="0x60022480db8a635ee6764e60418f9ef7-centos-root",devpath="/dev/dm-0",node_idx="1", node_name="telxvm-53-150"} 0
robin_disk_nvols{wwn="0x60022480db8a635ee6764e60418f9ef7-centos-root",devpath="/dev/dm-0",node_idx="1", node_name="telxvm-53-150"} 0
robin_disk_size{wwn="0x60022480db8a635ee6764e60418f9ef7-centos-root",devpath="/dev/dm-0",node_idx="1", node_name="telxvm-53-150"} 53687091200
robin_disk_rawused{wwn="0x60022480db8a635ee6764e60418f9ef7-centos-root",devpath="/dev/dm-0",node_idx="1", node_name="telxvm-53-150"} 0
robin_disk_freeslices{wwn="0x60022480db8a635ee6764e60418f9ef7-centos-swap",devpath="/dev/dm-1",node_idx="1", node_name="telxvm-53-150"} 0
robin_disk_freespace{wwn="0x60022480db8a635ee6764e60418f9ef7-centos-swap",devpath="/dev/dm-1",node_idx="1", node_name="telxvm-53-150"} 8254390272
robin_disk_freevols{wwn="0x60022480db8a635ee6764e60418f9ef7-centos-swap",devpath="/dev/dm-1",node_idx="1", node_name="telxvm-53-150"} 10
robin_disk_maxsegs{wwn="0x60022480db8a635ee6764e60418f9ef7-centos-swap",devpath="/dev/dm-1",node_idx="1", node_name="telxvm-53-150"} 0
robin_disk_maxvols{wwn="0x60022480db8a635ee6764e60418f9ef7-centos-swap",devpath="/dev/dm-1",node_idx="1", node_name="telxvm-53-150"} 10
robin_disk_maxslices{wwn="0x60022480db8a635ee6764e60418f9ef7-centos-swap",devpath="/dev/dm-1",node_idx="1", node_name="telxvm-53-150"} 0
robin_disk_nsegs{wwn="0x60022480db8a635ee6764e60418f9ef7-centos-swap",devpath="/dev/dm-1",node_idx="1", node_name="telxvm-53-150"} 0
robin_disk_nslices{wwn="0x60022480db8a635ee6764e60418f9ef7-centos-swap",devpath="/dev/dm-1",node_idx="1", node_name="telxvm-53-150"} 0
robin_disk_nvols{wwn="0x60022480db8a635ee6764e60418f9ef7-centos-swap",devpath="/dev/dm-1",node_idx="1", node_name="telxvm-53-150"} 0
robin_disk_size{wwn="0x60022480db8a635ee6764e60418f9ef7-centos-swap",devpath="/dev/dm-1",node_idx="1", node_name="telxvm-53-150"} 8254390272
robin_disk_rawused{wwn="0x60022480db8a635ee6764e60418f9ef7-centos-swap",devpath="/dev/dm-1",node_idx="1", node_name="telxvm-53-150"} 0
robin_disk_freeslices{wwn="0x60022480db8a635ee6764e60418f9ef7-centos-home",devpath="/dev/dm-2",node_idx="1", node_name="telxvm-53-150"} 0
robin_disk_freespace{wwn="0x60022480db8a635ee6764e60418f9ef7-centos-home",devpath="/dev/dm-2",node_idx="1", node_name="telxvm-53-150"} 44350570496
robin_disk_freevols{wwn="0x60022480db8a635ee6764e60418f9ef7-centos-home",devpath="/dev/dm-2",node_idx="1", node_name="telxvm-53-150"} 10
robin_disk_maxsegs{wwn="0x60022480db8a635ee6764e60418f9ef7-centos-home",devpath="/dev/dm-2",node_idx="1", node_name="telxvm-53-150"} 0
robin_disk_maxvols{wwn="0x60022480db8a635ee6764e60418f9ef7-centos-home",devpath="/dev/dm-2",node_idx="1", node_name="telxvm-53-150"} 10
robin_disk_maxslices{wwn="0x60022480db8a635ee6764e60418f9ef7-centos-home",devpath="/dev/dm-2",node_idx="1", node_name="telxvm-53-150"} 0
robin_disk_nsegs{wwn="0x60022480db8a635ee6764e60418f9ef7-centos-home",devpath="/dev/dm-2",node_idx="1", node_name="telxvm-53-150"} 0
robin_disk_nslices{wwn="0x60022480db8a635ee6764e60418f9ef7-centos-home",devpath="/dev/dm-2",node_idx="1", node_name="telxvm-53-150"} 0
robin_disk_nvols{wwn="0x60022480db8a635ee6764e60418f9ef7-centos-home",devpath="/dev/dm-2",node_idx="1", node_name="telxvm-53-150"} 0
robin_disk_size{wwn="0x60022480db8a635ee6764e60418f9ef7-centos-home",devpath="/dev/dm-2",node_idx="1", node_name="telxvm-53-150"} 44350570496
robin_disk_rawused{wwn="0x60022480db8a635ee6764e60418f9ef7-centos-home",devpath="/dev/dm-2",node_idx="1", node_name="telxvm-53-150"} 0
6.19. Handling Disruptions¶
With Robin, highly available applications can be deployed on Kubernetes as Robin can handle failures of drives, rack or hosts automatically. On a Baremetal setup, volumes can be setup with a replication factor of 2 or 3 to ensure that storage is available even if a drive fails. Users can also choose the fault domain to be ‘host’ to protect against node reboots or lost.
However, in a public cloud environment the cloud disks can be detached from one cloud node and reattached to another one. For example, in AWS an EBS volume can be detached on one EC2 host and reattached to a different EC2 host. Same with GCP where a PD can be moved across GCE nodes. If a cloud node (EC2, GCE, Azure VM) is terminated or rebooted, one would want any cloud drive attached to them (EBS, PD, Block) to be moved to the one or more of the remaining healthy nodes automatically. This is not limited to just cloud disks, but also SAN LUNS that are offered to Robin as disks. The SAN LUNS can also be multi-mounted onto multiple nodes or moved around from node to node. User can still choose to replicate volume on public cloud as it takes sometime to detach and attach drives on cloud platforms.
Just having the storage available during a disruption will not help if Kubernetes can not access it from the Pod. For example a Kubernetes StatefulSet serializes the mounting and unmounting of a volume to protect against possible corruptions. Robin utilizes smart detection techniques to ensure that even if a volume is mounted on multiple nodes, it can differentiate the IOs issued from the previous stale mount and the new mount. With this consistency guarantees, Robin enables the Kubernetes StatefulSet to unmount a volume from a dead node and remount it on a healthy node where the Pod is scheduled to run. Robin actively monitors these events to allow for the fast failover of the Pods without user intervention and consequently enables users to reliably deploy highly available stateful applications on Kubernetes.
6.20. Hardware RAID Controllers¶
6.20.1. Robin Recommendations¶
Robin recommends not using Redundant Array of Independent Disks (RAID) configurations for the physical storage devices that you plan to format with Robin storage. To understand the reasons for this recommendation, review the explanations presented here. However if RAID usage cannot be avoided, Robin recommends the following:
Use the hardware RAID controllers only to set up RAID-1 protection for boot and OS devices. This ensures that the failure of one device does not take the entire node down.
To protect Robin volumes against node and rack failures, you must configure replication across the fault domains. All RAID configurations (RAID-0, RAID-1, RAID-5, and RAID-6) protect data only against device failure on the node. They reduce the storage capacity of the cluster, thus increasing the data storage costs.
Robín’s replication provides maximum protection against all hardware failures including storage devices, storage nodes, and rack failures.
If a RAID controller has a cache, it must have a battery backup. If the cache on the RAID controller does not have a battery backup, Robin recommends disabling the cache as the data on the cache could be lost during a power outage.
6.20.2. Advantages and Disadvantages¶
The following are the advantages and disadvantages of RAID usage with Robin:
RAID-0 - In RAID-0, data is striped across all devices and provides the best performance. However, if one of the physical devices fails, the data on the RAID volume is lost.
RAID-1 - In RAID-1, data is mirrored across two devices. It gives the most redundancy. However, the performance of the volume is determined by the performance of the slowest device in the mirror and the data capacity is reduced by half.
RAID-5 - In RAID-5, data is striped across “n-1” devices. The controller calculates and writes parity data to the nth device. The RAID-5 configuration can tolerate one device failure and provides good performance. However, if one of the devices fails, the admin has to replace the device immediately and start a RAID rebuild to maintain the redundancy. All devices should be of the same size or the size of the smallest device decides the total usable capacity.
RAID-6 - In RAID-6, data is striped across “n-2” devices. The controller calculates and writes parity to two devices. The RAID-6 configuration can tolerate two device failures and provides good performance. However, the disadvantages are the same as RAID-5.
The striped RAID configurations (RAID-0, RAID-5, and RAID-6) provide better IO performance. However, RAID-0 does not give data protection, and RAID-5 and RAID-6 configurations reduce the storage capacity.
All RAID configurations are managed outside of Robin CNS. Therefore, the administrator must manually manage the device failures and it leads to an increase in operations cost.
If you enable hardware RAID, the RAID controller becomes a single point of failure for all the storage devices on the node. If the RAID controller fails, all data on all the storage devices in the node is lost.
Note
For more details on Robin recommendations for RAID controllers, review the points made here.