13. Backup and Restore Applications and Volumes

Robin CNS can:

  • Backup applications and volume data to an external storage repository such as S3 Object Store, Google Cloud Storage, etc.

  • Make applications and volume data portable across Kubernetes clusters spanning on-prem, public and hybrid cloud environments.

Backups containing encrypted volumes

By default, Robin CNS volumes are configured with data-at-rest encryption disabled. No special handling is required when reading or writing data from an unencrypted volume.

A volume key is required when writing data to, and reading data from an encrypted volume. To protect access to encrypted volume data, volume keys are encrypted (using the cluster master key) and persisted as part of the volume record in the cluster’s database. A backup’s metadata contains volume keys for all encrypted volumes in the backup.

The backup metadata also contains a special backup key that is used when encrypting and decrypting volume keys during backup export and import operations. Before storing the volume keys in the backup metadata, they are first decrypted (using the local cluster’s master key) and then re-encrypted using the backup key. The backup key itself is also encrypted (using the local cluster’s master key) before it’s added to the metadata.

Importing an encrypted volume from a backup

The first step when importing an encrypted volume from a backup, is to decrypt the backup key (using the cluster master key) and load it into memory. The volume key is then decrypted (using the backup key), re-encrypted (using the cluster master key), and then stored in the imported volume’s record in the cluster database. This only works on a cluster where the backup originated from. That’s because the backup key stored in the backup’s metadata can only be decrypted using the local cluster’s master key.

Importing an backup with encrypted volumes to a different cluster

Before you can import a volume from a backup on a different cluster, you must first import the backup to the cluster. If the backup contains encrypted volumes, then you also need to provide an unencrypted copy of the backup key during the import operation. When importing the backup, the backup key contained in the backup’s metadata (encrypted using the originating cluster’s master key) is replaced with a copy that was encrypted using the target cluster’s master key.

There are a number of ways you can provide an unencrypted copy of the backup key when importing a volume:

  • By default, the unencrypted backup key is included in the backup token generated when exporting a backup. When you import the backup on a different cluster, the backup key is read in from the token.

  • You can export the backup key on the source cluster and then import it on the target cluster. This adds the backup key on the target cluster as a secret. When importing the backup, the backup key is read in from the secret store. Note that if you import the backup key as a separate operation, it should not be included in the backup token (add the –exclude-backup-key command line option to exclude the backup key).

  • Provide an unencrypted copy of the backup key on the command line when importing the backup key.

  • Provide an unencrypted backup key on the command line when importing the backup.

Note

  • Only backups that contain encrypted volumes will be assigned a backup key.

  • An unencrypted copy of the backup key is only required when importing a backup containing encrypted volumes. Once the import operation has finished, the unencrypted backup key (or the token containing the key) can be discarded.

  • Great care needs to be taken with unencrypted backup keys, as they can be used to decrypt volume keys contained in the backup metadata.

Topics covered in this chapter:

robin backup create

Backup an application snapshot to a repository

robin backup delete

Delete an application or volume backup

robin backup list

List all backups

robin backup info

Show information about a specific backup

robin backup status

Show transfer status of backup to repository

robin backup tenant-share

Share a backup with one or more tenants

robin backup tenant-unshare

Stop sharing a backup with one or more tenants

robin backup user-share

Share a backup with one or more users

robin backup user-unshare

Stop sharing a backup with one or more users

robin backup list-shares

List information about users a backup has been shared with

robin backup export

Generate a token for use when importing a backup into another cluster

robin backup import

Import a backup from another cluster

robin backup list-key

List the backup_key from a backup with encrypted volumes

robin backup export-key

Generate an export token containing the backup key from a backup with encrypted volumes

robin backup import-key

Import the backup key for a backup from another cluster

13.1. Create an application backup

In order to create a backup of an application, issue the following command:

# robin backup create <app_name> <reponame>
                                --snapshotid <snapshotid>
                                --backupname <backupname>
                                --purge-snapshot

app_name

Name of the application being backed up

reponame

Name of the repo where the backup will be pushed

--snapshotid

Unique ID of snapshot to be backed up. If not provided a snapshot will be created implicitly

--backupname

Name to assign to backup

--purge-snapshot

Delete the snapshot pushed to the repo after successfully creating the backup. This option is only valid when a snapshot ID is not provided

Example:

# robin backup create mysql1 app-repofpom --wait
Creating app backup 'mysql1_backup-1581809777'
Job:  213 Name: K8SApplicationBackup State: PROCESSED       Error: 0
Job:  213 Name: K8SApplicationBackup State: WAITING         Error: 0
Job:  213 Name: K8SApplicationBackup State: AGENT_WAIT      Error: 0
Job:  213 Name: K8SApplicationBackup State: COMPLETED       Error: 0

# robin backup list
+----------------------------------+--------------------------+----------+----------------------------+--------+
| Backup ID                        | Backup Name              | Repo     | Snapshot Name              | State  |
+----------------------------------+--------------------------+----------+----------------------------+--------+
| fcc2d538504b11ea821e5d26d9bcf8ac | mysql1_backup-1581809777 | testrepo | mysql1_snapshot-1581809777 | Pushed |
+----------------------------------+--------------------------+----------+----------------------------+--------+

Note

The ability to create a backup is subject to Robin Role Based Access Control (RBAC). By default, all Cluster Users (users having the user role) have permission to create backups, provided they can access the underlying App snapshot. Cluster Administrators (users having the superadmin role) have permission to create backups from all App snapshots. See the section on User Management for details on how Robin RBAC works.

Before creating a backup, the underlying app must be attached to the target repo.

Creates an application backup.

End Point: /api/v3/robin_server/apps

Method: PUT

URL Parameters: None

Data Parameters:

  • action: backup - This mandatory field within the payload specifies the backup operation is to be performed.

  • app_name: <app_name> - This mandatory field within the payload specifies the name of the application to be backed up.

  • repo: <repo_name> - This mandatory field within the payload specifies the name of the repo in which the backup should be stored.

  • backupname: <backup_name> - This mandatory field within the payload specifies the name of the backup to be created.

  • snapshotid: <snapshot_id> - Utilizing this parameter, by specifying a snapshot ID, results in the associated snapshot being backed up.

  • purge_snapshot: true - Utilizing this parameter results in the snapshot being deleted once it has successfully been pushed to the repo. This option is only valid when a snapshot ID is not provided.

Port: RCM Port (default value is 29442)

Headers:

  • Authorization: <auth_token> : Authorization token to identify which user is sending the request. The token can be acquired from the login API.

Success Response Code: 202

Error Response Code: 500 (Internal Server Error), 404 (Not Found Error), 401 (Unauthorized Error), 400 (Invalid Api Usage Error)

Example Response:

Output
{
   "jobid":663,
   "plan":{
      "kind":"k8s",
      "action":"backup",
      "namespace":"t001-u000003",
      "current_user":{
         "user_contexts":[
            "robin",
            "robin-k8s-cluster"
         ],
         "user_capabilities":[
            "AllSuperAdminCapabilities"
         ],
         "tenant":"Administrators",
         "username":"robin",
         "user_permissions":{

         },
         "user_id":3,
         "tenant_id":1,
         "ip_addr":"172.17.0.1",
         "tenants":[
            "Administrators"
         ],
         "session_expires":"2020-09-10T21:28:38",
         "tenant_role":"superadmin",
         "namespace":"t001-u000003",
         "user_context":"robin"
      },
      "backup_name":"demo",
      "opcode":null,
      "repo":"app-repoihos",
      "app_name":"ma-appbd",
      "name":"ma-appbd"
   }
}

13.2. Delete a backup

Run this command to delete a backup:

# robin backup delete <backupid>
                      --repo-purge

backupid

Unique ID of the backup

--repo-purge

In addition to removing the backup from the cluster’s repo catalog, purge the backup from the external storage repo. Note that once the backup is purged from the repo, it cannot be recovered. In addition this option is only valid for backups that are owned/managed by the local cluster.

Example 1: Delete a backup, but leave it on the external repo

Delete the local copy of a backup, but leave it in the external storage repo:

# robin backup delete fcc2d538504b11ea821e5d26d9bcf8ac --wait
Job:  218 Name: StorageRepoPurge     State: VALIDATED       Error: 0
Job:  218 Name: StorageRepoPurge     State: COMPLETED       Error: 0

# robin backup list
+-----------+-------------+------+---------------+-------+
| Backup ID | Backup Name | Repo | Snapshot Name | State |
+-----------+-------------+------+---------------+-------+
+-----------+-------------+------+---------------+-------+

Example 2: Delete a backup and purge it from the external storage repo

# robin backup delete ebc453b45a6611e99876835407ddf3c4 --repo-purge --wait
Job:  218 Name: StorageRepoPurge     State: VALIDATED       Error: 0
Job:  218 Name: StorageRepoPurge     State: COMPLETED       Error: 0

Note

The ability to delete a backup is subject to Robin Role Based Access Control (RBAC). By default, all Cluster Users (users having the user role) have permission to delete from local storage any backups they have created. They do not have permission, however, to purge backups from the repo. Cluster Administrators (users having the superadmin role), on the other hand, have permission to delete all backups as well as purge backups from a repo catalog and the external repo.

All backups that have been pushed to a repo (which is to say, all backups), can be viewed using the robin backup list command. The robin repo contents command also provides information about backups that have been pushed to a specific repo. But this command can only be invoked by Cluster Administrators (users having the superadmin role).

Deletes a backup.

End Point: /api/v3/robin_server/backups

Method: DELETE

URL Parameters: None

Data Parameters:

  • action: purge - This mandatory field within the payload specifies the purge operation is to be performed.

  • backupid: <backupid> - This mandatory field within the payload specifies the ID of the backup to be deleted.

  • repo_purge: true - Utilizing this parameter in the payload results in the backup specified being removed from the external storage repo as well as the local catalog. This option is only valid for backups that are owned/managed by the local cluster.

Port: RCM Port (default value is 29442)

Headers:

  • Authorization: <auth_token> : Authorization token to identify which user is sending the request. The token can be acquired from the login API.

Success Response Code: 202

Error Response Code: 500 (Internal Server Error), 404 (Not Found Error), 401 (Unauthorized Error), 400 (Invalid Api Usage Error)

Example Response:

Output
{
   "jobid":711
}

13.3. List backups

Run this command to list backups:

# robin backup list --app <appname>
                    --app-namespace <app_ns>
                    --wide

--app <appname>

Only show backups for the specified application

--app-namespace <app_ns>

If there are multiple applications of the same name but in different namespaces this option can be used to distinguish them. Note this option is only valid when --app is specified

--wide

Provide additional information about each backup

Example 1: List backups of a specific app

# robin backup list --app mysql1
+----------------------------------+--------------------------+-------+----------------------------+--------+
| Backup ID                        | Backup Name              | Repo  | Snapshot Name              | State  |
+----------------------------------+--------------------------+-------+----------------------------+--------+
| 804232a4124f11eca28da5c0831d395f | mysql1_backup-1631289319 | repo1 | mysql1_snapshot-1631288383 | Pushed |
+----------------------------------+--------------------------+-------+----------------------------+--------+

Example 2: List all backups with extra details

# robin backup list --wide
+----------------------------------+----------------------------+-------+----------------------------+--------+---------+-------------+----------+----------------------+-------------+-----------+
| Backup ID                        | Backup Name                | Repo  | Snapshot Name              | State  | Managed | App Name    | App Type | Owner/Tenant         | Description | Level     |
+----------------------------------+----------------------------+-------+----------------------------+--------+---------+-------------+----------+----------------------+-------------+-----------+
| a3b979f0108a11ec931c0d8a6d76a1ea | u1ns1_nsbackup-1631094817  | repo1 | u1ns1iquap_ss1u1ns1        | Pushed | True    | u1ns1iquap  | flexapp  | u1/t1                |             | namespace |
| 33be96e4108d11ec816715edd7902163 | u11ns1_nsbackup-1631095917 | repo1 | u11ns1plscf_ss1u11ns1      | Pushed | True    | u11ns1plscf | flexapp  | u11/t1               |             | namespace |
| 804232a4124f11eca28da5c0831d395f | mysql1_backup-1631289319   | repo1 | mysql1_snapshot-1631288383 | Pushed | True    | mysql1      | helm     | admin/Administrators |             | app    |
+----------------------------------+----------------------------+-------+----------------------------+--------+---------+-------------+----------+----------------------+-------------+-----------+

Note

The ability to view backups is subject to Robin Role Based Access Control (RBAC). By default, Cluster Users (users having the user role) are able to view all backups they have created, plus any that have been shared with them (see Share a backup with tenants for details). See the section on User Management for details on how Robin RBAC works.

Lists all backups.

End Point: /api/v3/robin_server/backups

Method: GET

URL Parameters:

  • appname=<app_name> : Utilizing this parameter filters the results such that only backups of the specified application are returned.

  • namespace=<app_ns> : Utilizing this parameter filters the results such that only backups of the specified application in the given namespace are returned. Note this option is only valid with the appname parameter is utilized.

Data Parameters: None

Port: RCM Port (default value is 29442)

Headers:

  • Authorization: <auth_token> : Authorization token to identify which user is sending the request. The token can be acquired from the login API.

Success Response Code: 200

Error Response Code: 500 (Internal Server Error), 401 (Unauthorized Error), 400 (Invalid API Usage Error)

Example Response:

Output
{
   "items":[
      {
         "name":"u1ns1_nsbackup-1631094817",
         "id":"a3b979f0108a11ec931c0d8a6d76a1ea",
         "tenant":"t1",
         "username":"u1",
         "snapshot":"u1ns1iquap_ss1u1ns1",
         "app":"u1ns1iquap",
         "appid":24,
         "description":"",
         "kind":"flexapp",
         "repo":"repo1",
         "state":"Pushed",
         "managed":true,
         "share_with_all_tenants":false,
         "tenant_shares":{

         },
         "user_shares":{

         },
         "app_uuid":"a3acf7f6108511ec8bfde90ebdde4d85",
         "namespace_snapshot":true,
         "level":"namespace"
      },
      {
         "name":"u11ns1_nsbackup-1631095917",
         "id":"33be96e4108d11ec816715edd7902163",
         "tenant":"t1",
         "username":"u11",
         "snapshot":"u11ns1plscf_ss1u11ns1",
         "app":"u11ns1plscf",
         "appid":28,
         "description":"",
         "kind":"flexapp",
         "repo":"repo1",
         "state":"Pushed",
         "managed":true,
         "share_with_all_tenants":false,
         "tenant_shares":{

         },
         "user_shares":{

         },
         "app_uuid":"f3417c36108b11ecb02da75c11159457",
         "namespace_snapshot":true,
         "level":"namespace"
      },
      {
         "name":"mysql1_backup-1631289319",
         "id":"804232a4124f11eca28da5c0831d395f",
         "tenant":"Administrators",
         "username":"admin",
         "snapshot":"mysql1_snapshot-1631288383",
         "app":"mysql1",
         "appid":14,
         "description":"",
         "kind":"helm",
         "repo":"repo1",
         "state":"Pushed",
         "managed":true,
         "share_with_all_tenants":false,
         "tenant_shares":{

         },
         "user_shares":{

         },
         "app_uuid":"7269d19a0fc511ec932ba5cfd586197c",
         "namespace_snapshot":false,
         "level":"app"
      }
   ]
}

13.4. Get information about a specific backup

Details of a specific backup can be obtained using the following command:

# robin backup info <backupid>
                    --specs

backupid

Unique ID of the backup (obtained from the output of robin backup list)

--specs

Output the spec file associated with the backup

Example:

# robin backup info 33be96e4108d11ec816715edd7902163

Name                   : u11ns1_nsbackup-1631095917
Id                     : 33be96e4108d11ec816715edd7902163
Snapshot Name          : u11ns1plscf_ss1u11ns1
Application Name       : u11ns1plscf
Application Kind       : flexapp
Snapshot level         : Namespace
State                  : Pushed
Repo                   : repo1
Description            : -
Total Transferred      : 352.0 MB


Trigger Details
-------
Operation              : MANUAL
User                   : u11
Tenant                 : t1

Data Details
-------
Transferred            : 352.0 MB
Skipped                : 0.0 MB
Total                  : 352.0 MB
Start Time             : 08 Sep 2021 10:11:58
End Time               : 08 Sep 2021 10:12:07
Time Taken             : 9s

Note

The ability to view information about a backup is subject to Robin Role Based Access Control (RBAC). By default, Cluster Users (users having the user role) are able to view info for all backups they have created, plus any that have been shared with them (see Share a backup with tenants for details). See the section on User Management for details on how Robin RBAC works.

Information about the local application or volume will only be included in the output if they still exists. This information will never be displayed for backups which are imported from another Robin cluster.

Returns details about a specific backup such as its kind, the objects which are associated with it and its creation time,

End Point: /api/v3/robin_server/k8s_app/<backupid>?action=info&otype=backup

Method: GET

URL Parameters: None

Data Parameters: None

Port: RCM Port (default value is 29442)

Headers:

  • Authorization: <auth_token> : Authorization token to identify which user is sending the request. The token can be acquired from the login API.

Success Response Code: 200

Error Response Code: 500 (Internal Server Error), 404 (Not Found Error), 401 (Unauthorized Error)

Example Response:

Output
{
   "name":"u11ns1_nsbackup-1631095917",
   "id":"33be96e4108d11ec816715edd7902163",
   "tenant":"t1",
   "username":"u11",
   "snapshot":"u11ns1plscf_ss1u11ns1",
   "app":"u11ns1plscf",
   "appid":28,
   "description":"",
   "kind":"flexapp",
   "repo":"repo1",
   "state":"Pushed",
   "managed":true,
   "app_exists":true,
   "snap_exists":true,
   "bytes_count":369098752,
   "share_with_all_tenants":false,
   "tenant_shares":{

   },
   "user_shares":{

   },
   "app_uuid":"f3417c36108b11ecb02da75c11159457",
   "namespace_snapshot":true,
   "level":"namespace",
   "backup_entry":true,
   "spec":"apiVersion: v1\ndata: {run.sh: ''}\nkind: ConfigMap\nmetadata:\n  annotations: {meta.helm.sh\/release-name: mysql1t1u11ns1, meta.helm.sh\/release-namespace: u11ns1}\n  labels: {app: mysql1t1u11ns1, app.kubernetes.io\/managed-by: Helm, chart: mysql-1.6.7,\n    heritage: Helm, release: mysql1t1u11ns1}\n  name: mysql1t1u11ns1-test\n  namespace: u11ns1\n---\napiVersion: v1\ndata: {mysql-password: WVdvNFhrd1ZCOQ==, mysql-root-password: QVNRRVZhd3VQNQ==}\nkind: Secret\nmetadata:\n  annotations: {meta.helm.sh\/release-name: mysql1t1u11ns1, meta.helm.sh\/release-namespace: u11ns1}\n  labels: {app: mysql1t1u11ns1, app.kubernetes.io\/managed-by: Helm, chart: mysql-1.6.7,\n    heritage: Helm, release: mysql1t1u11ns1}\n  name: mysql1t1u11ns1\n  namespace: u11ns1\ntype: Opaque\n---\napiVersion: v1\nkind: PersistentVolumeClaim\nmetadata:\n  annotations: {meta.helm.sh\/release-name: mysql1t1u11ns1, meta.helm.sh\/release-namespace: u11ns1,\n    robin.io\/appid: '1', robin.io\/blocksize: '4096', robin.io\/encryption: none, robin.io\/fstype: ext4,\n    robin.io\/snapname: u11ns1plscf_ss1u11ns1, robin.io\/snapshotid: '1', robin.io\/vol_snapname: ss1u11ns1,\n    robin.io\/volumeid: '19', robin.io\/volumename: pvc-f90e8ae4-56a4-4a7b-aaed-8c7ef337e4e8,\n    robin.io\/zoneid: '1630920119', volume.beta.kubernetes.io\/storage-provisioner: robin}\n  labels: {app: mysql1t1u11ns1, app.kubernetes.io\/managed-by: Helm, chart: mysql-1.6.7,\n    heritage: Helm, release: mysql1t1u11ns1}\n  name: mysql1t1u11ns1\n  namespace: u11ns1\nspec:\n  accessModes: [ReadWriteOnce]\n  resources:\n    requests: {storage: 1Gi}\n  storageClassName: robin\n  volumeMode: Filesystem\n---\napiVersion: v1\nkind: Service\nmetadata:\n  annotations: {meta.helm.sh\/release-name: mysql1t1u11ns1, meta.helm.sh\/release-namespace: u11ns1}\n  labels: {app: mysql1t1u11ns1, app.kubernetes.io\/managed-by: Helm, chart: mysql-1.6.7,\n    heritage: Helm, release: mysql1t1u11ns1}\n  name: mysql1t1u11ns1\n  namespace: u11ns1\nspec:\n  ipFamilies: [IPv4]\n  ipFamilyPolicy: SingleStack\n  ports:\n  - {name: mysql, port: 3306, protocol: TCP, targetPort: mysql}\n  selector: {app: mysql1t1u11ns1}\n  sessionAffinity: None\n  type: ClusterIP\n---\napiVersion: apps\/v1\nkind: Deployment\nmetadata:\n  annotations: {deployment.kubernetes.io\/revision: '1', meta.helm.sh\/release-name: mysql1t1u11ns1,\n    meta.helm.sh\/release-namespace: u11ns1}\n  labels: {app: mysql1t1u11ns1, app.kubernetes.io\/managed-by: Helm, chart: mysql-1.6.7,\n    heritage: Helm, release: mysql1t1u11ns1}\n  name: mysql1t1u11ns1\n  namespace: u11ns1\nspec:\n  progressDeadlineSeconds: 600\n  replicas: 1\n  revisionHistoryLimit: 10\n  selector:\n    matchLabels: {app: mysql1t1u11ns1, release: mysql1t1u11ns1}\n  strategy: {type: Recreate}\n  template:\n    metadata:\n      creationTimestamp: null\n      labels: {app: mysql1t1u11ns1, release: mysql1t1u11ns1}\n    spec:\n      containers:\n      - env:\n        - name: MYSQL_ROOT_PASSWORD\n          valueFrom:\n            secretKeyRef: {key: mysql-root-password, name: mysql1t1u11ns1}\n        - name: MYSQL_PASSWORD\n          valueFrom:\n            secretKeyRef: {key: mysql-password, name: mysql1t1u11ns1, optional: true}\n        - {name: MYSQL_USER}\n        - {name: MYSQL_DATABASE}\n        image: mysql:5.7.30\n        imagePullPolicy: IfNotPresent\n        livenessProbe:\n          exec:\n            command: [sh, -c, 'mysqladmin ping -u root -p${MYSQL_ROOT_PASSWORD}']\n          failureThreshold: 3\n          initialDelaySeconds: 30\n          periodSeconds: 10\n          successThreshold: 1\n          timeoutSeconds: 5\n        name: mysql1t1u11ns1\n        ports:\n        - {containerPort: 3306, name: mysql, protocol: TCP}\n        readinessProbe:\n          exec:\n            command: [sh, -c, 'mysqladmin ping -u root -p${MYSQL_ROOT_PASSWORD}']\n          failureThreshold: 3\n          initialDelaySeconds: 5\n          periodSeconds: 10\n          successThreshold: 1\n          timeoutSeconds: 1\n        resources:\n          requests: {cpu: 100m, memory: 256Mi}\n        securityContext: {privileged: true, runAsUser: 0}\n        terminationMessagePath: \/dev\/termination-log\n        terminationMessagePolicy: File\n        volumeMounts:\n        - {mountPath: \/var\/lib\/mysql, name: data}\n      dnsPolicy: ClusterFirst\n      initContainers:\n      - command: [rm, -fr, \/var\/lib\/mysql\/lost+found]\n        image: busybox:1.32\n        imagePullPolicy: IfNotPresent\n        name: remove-lost-found\n        resources:\n          requests: {cpu: 10m, memory: 10Mi}\n        terminationMessagePath: \/dev\/termination-log\n        terminationMessagePolicy: File\n        volumeMounts:\n        - {mountPath: \/var\/lib\/mysql, name: data}\n      restartPolicy: Always\n      schedulerName: default-scheduler\n      securityContext: {}\n      serviceAccount: default\n      serviceAccountName: default\n      terminationGracePeriodSeconds: 30\n      volumes:\n      - name: data\n        persistentVolumeClaim: {claimName: mysql1t1u11ns1}\n---\napiVersion: rbac.authorization.k8s.io\/v1\nkind: RoleBinding\nmetadata:\n  labels: {app.kubernetes.io\/instance: robin, app.kubernetes.io\/managed-by: robin.io,\n    app.kubernetes.io\/name: robin, robin.io\/domain: ROBIN, robin.io\/role: tenantadmin,\n    robin.io\/tenant: t1, robin.io\/tenant_id: '2', robin.io\/user_id: '4', robin.io\/username: u1}\n  name: rolebinding-u1-edit\n  namespace: u11ns1\nroleRef: {apiGroup: rbac.authorization.k8s.io, kind: ClusterRole, name: edit}\nsubjects:\n- {apiGroup: rbac.authorization.k8s.io, kind: User, name: u1, namespace: u11ns1}\n---\napiVersion: rbac.authorization.k8s.io\/v1\nkind: RoleBinding\nmetadata:\n  labels: {app.kubernetes.io\/instance: robin, app.kubernetes.io\/managed-by: robin.io,\n    app.kubernetes.io\/name: robin, robin.io\/domain: ROBIN, robin.io\/role: user, robin.io\/tenant: t1,\n    robin.io\/tenant_id: '2', robin.io\/user_id: '7', robin.io\/username: u11}\n  name: rolebinding-u11-edit\n  namespace: u11ns1\nroleRef: {apiGroup: rbac.authorization.k8s.io, kind: ClusterRole, name: edit}\nsubjects:\n- {apiGroup: rbac.authorization.k8s.io, kind: User, name: u11, namespace: u11ns1}\n",
   "bytes_skipped":0,
   "bytes_transferred":369098752,
   "op_trigger":"MANUAL",
   "vol_data":true,
   "creation_time":1631095918,
   "creation_end_time":1631095927
}

13.5. Configuring backup schedules and retention policies

This is covered under the Manage Application Backup Schedules section of Applications chapter. See details of the command robin app backup-schedule.

13.6. Monitor status of a backup

The status of transfers of in progress backups to their respective repositories can be viewed using the following command:

# robin backup status --appname <appname>
                      --backupid <backupid>
                      --all

--appname <appname>

Filter by app name

--backupid <backupid>

Filter by backup id

--all

Include all backups in result

Example:

# robin backup status --backupid fe7d6f7a504f11ea92f977d7085a249d
+----------------------------------+-------------------------------------+-----------------+
| Name                             | TransferStatus                      | Schedule/Manual |
+----------------------------------+-------------------------------------+-----------------+
| fe7d6f7a504f11ea92f977d7085a249d | 100% [####################] 736.0MB | MANUAL          |
+----------------------------------+-------------------------------------+-----------------+

Returns the transfer status of backups to a particular repo.

End Point: /api/v3/robin_server/k8s_app/

Method: GET

URL Parameters: None

Data Parameters:

  • action: get_k8s_backup_ops - This mandatory field within the payload specifies that the status of backups should be returned.

  • all: [true|false] - This mandatory field within the payload specifies whether or not the status of all backups (both in progress and completed) should be returned.

  • backupid: <backupid> - Utilizing this parameter within the payload, by specifying a backup ID, results in only the transfer status of the aformentioned backup being returned.

  • appname: <appname> - Utilizing this parameter within the payload, by specifying an application name, results in only the transfer status of the backups associated with the aforementioned application being returned.

Port: RCM Port (default value is 29442)

Headers:

  • Authorization: <auth_token> : Authorization token to identify which user is sending the request. The token can be acquired from the login API.

Success Response Code: 200

Error Response Code: 500 (Internal Server Error), 401 (Unauthorized Error), 400 (Invalid API Usage Error)

Example Response:

Output
{
   "items":{
      "bc8e3a58f3b611eaa93d0b3c9c4e5962":{
         "bytes_skipped":0,
         "snap_name":"mysql-hplt_snapmysql-hplt",
         "bytes_transferred":134217728,
         "state":"Push in progress",
         "op_trigger":"MANUAL",
         "bytes_count":973078528
      },
      "4ba16f22f30911eab74d45afeffd38d2":{
         "bytes_skipped":0,
         "snap_name":"my-apptn_my-s1tc",
         "bytes_transferred":369098752,
         "state":"Pushed",
         "op_trigger":"MANUAL",
         "bytes_count":369098752
      },
      "387a9d46f30911eaae760514a59458bd":{
         "bytes_skipped":0,
         "snap_name":"mysql2_snapmysql2",
         "bytes_transferred":369098752,
         "state":"Pushed",
         "op_trigger":"MANUAL",
         "bytes_count":369098752
      },
      "1c344900f30911eaa97a51af70aa8d37":{
         "bytes_skipped":0,
         "snap_name":"mysql1_snapmysql1",
         "bytes_transferred":369098752,
         "state":"Pushed",
         "op_trigger":"MANUAL",
         "bytes_count":369098752
      },
      "d079f9d0f30a11ea85298d04093a1b78":{
         "bytes_skipped":0,
         "snap_name":"ma-appbd_ma-s1dk",
         "bytes_transferred":100663296,
         "state":"Pushed",
         "op_trigger":"MANUAL",
         "bytes_count":100663296
      },
      "b419d8a2f30611eab3c98b46a91f9934":{
         "bytes_skipped":0,
         "snap_name":"my-appcw_my-s1po",
         "bytes_transferred":369098752,
         "state":"Pushed",
         "op_trigger":"MANUAL",
         "bytes_count":369098752
      }
   }
}

13.7. Share a backup with tenants

Run this command to share a backup with one or more tenants:

# robin backup tenant-share <backupid> [<tenant_list>]
                                        --operations <operations>
                                        --all-tenants

backupid

ID of the backup to share

tenant_list

An optional comma separated list of tenants to share a backup with

--operations <operations>

Comma separated list of operations that can be performed on a backup shared with tenants. To allow all valid operations to be performed, enter ‘ALL_OPERATIONS’ on the command line (default value is ‘view’)

--all-tenants

Share a backup with all tenants

Example 1: Share a backup with a tenant

# robin backup tenant-share fe7d6f7a504f11ea92f977d7085a249d t1 --wait
Job:  229 Name: K8sBackupShare     State: PROCESSED       Error: 0
Job:  229 Name: K8sBackupShare     State: COMPLETED       Error: 0

Example 2: Share a backup with all tenants

# robin backup tenant-share fe7d6f7a504f11ea92f977d7085a249d --all-tenants --wait
Job:  229 Name: K8sBackupShare     State: PROCESSED       Error: 0
Job:  229 Name: K8sBackupShare     State: COMPLETED       Error: 0

Note

The ability share a backup with tenants is subject to Robin Role Based Access Control (RBAC). Only Cluster Administrators (users having the superadmin role) have permission to do so. See the section on User Management for details on how Robin RBAC works.

13.8. Stop sharing a backup with tenants

Stop sharing a backup with one or more tenants:

# robin repo unshare <reponame> [tenant_list]
                                --operations <operations>
                                --all-tenants

backupid

ID of the backup to stop sharing

tenant_list

List of tenants to stop sharing the backup with

--operations <operations>

Comma separated list of operations to stop allowing on a backup shared with specified tenants. If no operations are specified, then stop sharing the backup with all specified tenants.

--all-tenants

Stop sharing the backup with all tenants

Example 1: Stop sharing a backup with a tenant

# robin backup tenant-unshare fe7d6f7a504f11ea92f977d7085a249d t1 --wait
Job:  229 Name: K8sBackupUnshare     State: PROCESSED       Error: 0
Job:  229 Name: K8sBackupUnshare     State: COMPLETED       Error: 0

Example 2: Stop sharing a backup with all tenants

# robin backup tenant-unshare fe7d6f7a504f11ea92f977d7085a249d --all-tenants --wait
Job:  229 Name: K8sBackupUnshare     State: PROCESSED       Error: 0
Job:  229 Name: K8sBackupUnshare     State: COMPLETED       Error: 0

Note

The ability stop sharing a backup with tenants is subject to Robin Role Based Access Control (RBAC). Only Cluster Administrators (users having the superadmin role) have permission to do so. See the section on User Management for details on how Robin RBAC works.

13.9. Share a backup with users

Run this command to share a backup with one or more users:

# robin backup user-share <backupid> <tenant> [<user_list>]
                                               --operations <operations>
                                               --all-tenant-users

backupid

ID of the backup to share

tenant

Tenant user share should be added to

user_list

List of users to share the backup with

--operations <operations>

List of operations the user will be allowed to perform. To allow all valid operations to be performed, specify ‘ALL_OPERATIONS’ on the command line (default value is ‘view’)

--all-tenant-users

Share a backup with all users

Example:

# robin backup user-share fe7d6f7a504f11ea92f977d7085a249d t1 user1 --wait
Job:  236 Name: K8sBackupShare       State: PROCESSED       Error: 0
Job:  236 Name: K8sBackupShare       State: COMPLETED       Error: 0

# robin backup list --full
+----------------------------------+--------------------------+----------+----------------------------+--------+---------+----------+----------+--------------+-------------+
| Backup ID                        | Backup Name              | Repo     | Snapshot Name              | State  | Managed | App Name | App Type | Owner/Tenant | Description |
+----------------------------------+--------------------------+----------+----------------------------+--------+---------+----------+----------+--------------+-------------+
| fe7d6f7a504f11ea92f977d7085a249d | mysql1_backup-1581811498 | testrepo | mysql1_snapshot-1581811498 | Pushed | True    | mysql1   | helm     | admin1/t1    |             |
+----------------------------------+--------------------------+----------+----------------------------+--------+---------+----------+----------+--------------+-------------+
Tenant Shares: ALL_TENANTS
User Shares:
t1:user1: view

Note

The ability share a backup with users is subject to Robin Role Based Access Control (RBAC). By default, only Cluster Administrators (users having the superadmin role) and Tenant Administrators (users having the tenantadmin role) have permission to do so. Tenant Administrators are only allowed to stop sharing a backup with users from their tenant. See the section on User Management for details on how Robin RBAC works.

13.10. Stop sharing a backup with users

Stop sharing a backup with one or more users:

# robin backup user-unshare <backupid> <tenant> [<user_list>]
                                                 --operations <operations>
                                                 --all-tenant-users

backupid

ID of the backup to stop sharing

tenant

Tenant user share should be removed from

user_list

List of users to stop sharing the backup with

--operations <operations>

List of operations to stop allowing users to invoke on the shared backup. To block all valid operations to be performed, specify ‘ALL_OPERATIONS’ on the command line (default value is ‘view’)

--all-tenant-users

Stop sharing a backup with all users (overrides user_list)

Example:

# robin backup user-unshare fe7d6f7a504f11ea92f977d7085a249d t1 user1 --wait
Job:  237 Name: K8sBackupUnshare     State: PROCESSED       Error: 0
Job:  237 Name: K8sBackupUnshare     State: COMPLETED       Error: 0

Note

The ability to stop sharing a backup is subject to Robin Role Based Access Control (RBAC). By default, only Cluster Administrators (users having the superadmin role) and Tenant Administrators (users having the tenantadmin role) have permission to do so. Tenant Administrators are only allowed to stop sharing a backup with users from their tenant. See the section on User Management for details on how Robin RBAC works.

13.11. List shares of backups with tenants and users

List backups that have been shared with one or more tenants or users.

# robin backup list-shares [username]

username

The username of the user to show backup shares for. If username is not supplied, then show backup shares for all users.

Example:

# robin backup list-shares
Tenant Shares:

Backup Id                        | Backup Name                     | Operation | Tenants
---------------------------------+---------------------------------+-----------+---------
8fd19606015011ebaf3abbf769ba4a18 | mysql-t1-u1-1_backup-1601273096 | view      | t2

User Shares:

Backup Id                        | Backup Name                     | Operation | Tenant | Users
---------------------------------+---------------------------------+-----------+--------+------------------
8fd19606015011ebaf3abbf769ba4a18 | mysql-t1-u1-1_backup-1601273096 | view      | t2     | All Tenant Users

Note

The ability to list backup shares is subject to Role Based Access Control (RBAC). Only Cluster Administrators (users having the superadmin role) and Tenant Administrators (users having the tenantadmin role) have permission to do so. Tenant Administrators are only allowed to view user shares for for users from their tenant. See the section on User Management for details on how Robin RBAC works.

13.12. Export a backup

Generate a token for use when importing a backup backup into another cluster:

# robin backup export <backupid>
                      --exclude-backup-key
                      --force

backupid

Application backup ID

--exclude-backup-key

Do not include the backup_key in the backup token. Note that an unencrypted backup_key only gets added when the backup contains encrypted volumes.

--force

Force backup import operation. Use this as part of encrypted volumes backup

Example:

# robin backup list
+----------------------------------+-------------+----------+--------------------+--------+
| Backup ID                        | Backup Name | Repo     | Snapshot Name      | State  |
+----------------------------------+-------------+----------+--------------------+--------+
| 644770427d8911e9b908cbe6a78db423 | mysql1      | testrepo | mysql1_mysql1-snap | Pushed |
+----------------------------------+-------------+----------+--------------------+--------+

# robin backup export 644770427d8911e9b908cbe6a78db423
eyJyZXBvX3R5cGUiOiAiR0NTIiwgImJhY2t1cF9pZCI6ICI2NDQ3NzA0MjdkODkxMWU5YjkwOGNiZTZhNzhkYjQyMyIsICJiYWNrdXBfcGF0aCI6ICJkZXYvdGVzdGluZy9hcHBjb25maWcvMTU1ODU5NDE0Ny9teXNxbDFfMS9teXNxbDFfbXlzcWwxLXNuYXAuanNvbiIsICJyZXBvX3BhcmFtcyI6IHsiYnVja2V0IjogInRlc3RidWNrZXQtMTAyOTM4NDc1NiIsICJyZXBvX3R5cGUiOiAiR0NTIiwgInBhdGgiOiAiZGV2L3Rlc3RpbmcvIiwgImNyZWRlbnRpYWxzIjogeyJwcm9qZWN0X2lkIjogInJvY2stcmFuZ2UtMjA3NjIyIn19fQ==

Note

The ability to export a backup is subject to Robin Role Based Access Control (RBAC). By default, only Cluster Administrators (users having the superadmin role) have permission to do so. See the section on User Management for details on how Robin RBAC works.

Generates a token for use when importing a backup into another cluster:

End Point: /api/v3/robin_server/k8s_app

Method: PUT

URL Parameters: None

Data Parameters:

  • action: export - This mandatory field within the payload specifies that the export operation should be performed

  • bkpid: <backupid> - This mandatory field within the payload specifies the ID of the backup to be exported.

Port: RCM Port (default value is 29442)

Headers:

  • Authorization: <auth_token> : Authorization token to identify which user is sending the request. The token can be acquired from the login API.

Success Response Code: 200

Error Response Code: 500 (Internal Server Error), 401 (Unauthorized Error), 400 (Invalid API Usage Error), 404 (Not Found Error)

Example Response:

Output
{
   "backup_token":"eyJyZXBvX3BhcmFtcyI6IHsiYnVja2V0IjogInJhbWVuZHJhIiwgImNyZWRlbnRpYWxzIjogeyJwcm9qZWN0X2lkIjogInJvY2stcmFuZ2UtMjA3NjIyIn0sICJwYXRoIjogIm15LWFwcGN3LWJ1Y2tldC8iLCAicmVwb190eXBlIjogIkdDUyJ9LCAicmVwb190eXBlIjogIkdDUyIsICJiYWNrdXBfaWQiOiAiMzg3YTlkNDZmMzA5MTFlYWFlNzYwNTE0YTU5NDU4YmQiLCAiYmFja3VwX3BhdGgiOiAibXktYXBwY3ctYnVja2V0LzE1OTUzMzE4NjYvbXlzcWwyXzkvMzg3YTlkNDZmMzA5MTFlYWFlNzYwNTE0YTU5NDU4YmQvYXBwY29uZmlnL2JhY2t1cF9tZXRhZGF0YS5qc29uIn0="
}

13.13. Import a backup

Run this command to import a backup from another cluster:

Note

Importing a backup of a bundle application requires that the same bundle used when creating the original application be present in the destination cluster.

# robin backup import <backup_token>
                      --backup-key <BACKUP_KEY>

backup_token

Token containing details of the backup to import

--backup-key BACKUP_KEY

Backup key for the backup. Note that this is is not a secure option, as the backup key is not encrypted.

Example:

# robin backup import eyJyZXBvX3R5cGUiOiAiR0NTIiwgImJhY2t1cF9pZCI6ICI2NDQ3NzA0MjdkODkxMWU5YjkwOGNiZTZhNzhkYjQyMyIsICJiYWNrdXBfcGF0aCI6ICJkZXYvdGVzdGluZy9hcHBjb25maWcvMTU1ODU5NDE0Ny9teXNxbDFfMS9teXNxbDFfbXlzcWwxLXNuYXAuanNvbiIsICJyZXBvX3BhcmFtcyI6IHsiYnVja2V0IjogInRlc3RidWNrZXQtMTAyOTM4NDc1NiIsICJyZXBvX3R5cGUiOiAiR0NTIiwgInBhdGgiOiAiZGV2L3Rlc3RpbmcvIiwgImNyZWRlbnRpYWxzIjogeyJwcm9qZWN0X2lkIjogInJvY2stcmFuZ2UtMjA3NjIyIn19fQ==

Note

The ability to import a backup is subject to Robin Role Based Access Control (RBAC). By default, only Cluster Administrators (users having the superadmin role) have permission to do so. See the section on User Management for details on how Robin RBAC works.

Import a backup from another cluster.

End Point: /api/v3/robin_server/k8s_app

Method: PUT

URL Parameters: None

Data Parameters:

  • action: import - This mandatory field within the payload specifies that the import operation should be performed

  • backup_token: <backup_token> - This mandatory field within the payload specifies the token containing the details of the backup to import.

Port: RCM Port (default value is 29442)

Headers:

  • Authorization: <auth_token> : Authorization token to identify which user is sending the request. The token can be acquired from the login API.

Success Response Code: 202

Error Response Code: 500 (Internal Server Error), 401 (Unauthorized Error), 400 (Invalid API Usage Error)

Example Response:

Output
{
   "jobid":722
}

13.14. View backup key from backup with encrypted volumes

Run this command to view the backup key from a backup that contains encrypted volumes.

# robin backup list-key <backupid>
                        --force

backupid

Backup ID

--force

Because the returned backup key is unencrypted, the force option is required in order for it to be displayed.

Example:

[root@cscale-82-64 ~]# robin backup list-key 0cae83d46b4411eeb1ecb9f9119206b3 --force
 868fcc4a57c1015343d0bcf03ae4b18a259d6cd21d561e33534ce32435abdd79

13.15. Export backup key from backup with encrypted volumes

Run this command to export the backup key of a a backup that contains encrypted volumes.

# robin backup export-key <backupid>

backupid

Backup ID

Example:

[root@cscale-82-64 ~]# robin backup export-key 0cae83d46b4411eeb1ecb9f9119206b3eyJYWNrdXBfaWQiOiiMGNhZTgzZDQYjQ0MTFlZWIxZWNiOWY5MTE5MjA2YjMiLCAiYmFja3VwX2tleSI6ICI4NjhmY2M0YTU3YzEwMTUzNDNkMGJjZjAzYWU0YjE4YTI1OWQ2Y2QyMWQ1NjFlMzM1MzRjZTMyNDM1YWJkZDc5IiwgInpvbmVpZCI6IDE2OTY5MTc3MTZ9

13.16. Import backup key from backup with encrypted volumes

Run this command to import the backup key from a backup that contains encrypted volumes. The backup key will be added to the secret store, for use when the associated backup is imported.

The backup key can be supplied via backup_key_token (which contains Backup ID and backup key), or it can be supplied on the command line by including the --backup-id and --backup-key command line options.

# robin backup import-key <backup_key_token>
                          --backup-id <BACKUP_ID>
                          --backup-key <BACKUP_KEY>

backup_key_token

Encoded token containing backup ID and backup key from a backup from another cluster.

--backup-id BACKUP_ID

The backup ID

--backup-key BACKUP_KEY

The backup key

Example:

[root@cscale-82-64 ~]# robin backup import-key eyJiYWNrdXBfaWQiOiAiMGNhZTgzZDQ2YjQ0MTFlZWIxZWNiOWY5MTE5MjA2YjMiLCAiYmFja3VwX3BhdGgiOiAiZGV2L3RvbS8xNjk2OTE3NzE2L3ZvbHVtZS1zbmFwc2hvdHMvMGM4ZWQ3NjY2YjQ0MTFlZThlMTZhNzZhN2Q2OGM5Y2YvYXBwY29uZmlnL2JhY2t1cF9tZXRhZGF0YS5qc29uIiwgInJlcG9fdHlwZSI6ICJHQ1MiLCAicmVwb19wYXJhbXMiOiB7InJlcG9fdHlwZSI6ICJHQ1MiLCAiY3JlZGVudGlhbHMiOiB7InByb2plY3RfaWQiOiAicm9jay1yYW5nZS0yMDc2MjIifSwgImJ1Y2tldCI6ICJ0ZXN0YnVja2V0LTEwMjkzODQ3NTYiLCAicGF0aCI6ICJkZXYvdG9tLyJ9LCAiYmFja3VwX2tleSI6ICI4NjhmY2M0YTU3YzEwMTUzNDNkMGJjZjAzYWU0YjE4YTI1OWQ2Y2QyMWQ1NjFlMzM1MzRjZTMyNDM1YWJkZDc5In0=
Backup key for backup with ID '0cae83d46b4411eeb1ecb9f9119206b3' was successfully imported

13.17. Configuring internal repo for Backup

This article describes how to configure an internal repo using minio as a backup repository and secure it with TLS.

  • Ensure that minio is configured to use TLS - https://docs.min.io/docs/how-to-secure-access-to-minio-server-with-tls.html

  • If using a private certificate, create and sign a certificate for the connection. An example of how to do this using OpenSSL can be found here: https://jamielinux.com/docs/openssl-certificate-authority/introduction.html

  • Test that minio TLS configuration is works with the created certificate:

    • cURL the minio server without specifying a certificate - this should fail

    curl https://minio-sever
    
    • cURL the minio server specifying a certificate - this should succeed

    curl --cacert <ca.cert> https://minio-server
    
  • A create a minio object store for the Robin backups - https://docs.min.io/docs/minio-client-complete-guide.html

  • Register the minio backup repository with Robin:

    • Create the credential json file in the below format:

    • Update the minio endpoint and certificate file path . Access key and secret key of minio needs to be configured as the value for “aws_access_key” and “aws_secret_key“.

    {
            "aws_access_key_id" : "minioadmin",
            "aws_secret_access_key" : "minioadmin",
            "end_point": "<mnioserverip/serviceip>:9000",
            "tls": "yes",
            "ca_file": "/root/ca-chain.cert.pem"
    }
    
    • In case you want configure to skip tls validation , configure “validate_certs” parameter as “no” in the credential json and remove the “ca_file” parameter.

    {
              "aws_access_key_id" : "minioadmin",
              "aws_secret_access_key" : "minioadmin",
              "end_point": "<mnioserverip/serviceip>:9000",
              "tls": "yes",
              "validate_certs": "no"
    }
    
    • Register the minio repo to robin. You will have to provide the minio object store path and the path to the credential json while registering the repo.

    # robin repo register miniotest-repo s3://miniotest/test  /root/minio-credential.json readwrite
    
    • Check if the repo has been added to robin

    # robin repo list | grep miniotest-repo
    
  • Test that a backup can be made to the minio repo:

    • Attach minio repo to the app:

    # robin app attach-repo mongoapp mongodbns-backup --wait
    
    • Create a backup from the app

    # robin backup create mongoapp miniotest-repo
    
    • Alternatively we can also backup an existing snapshot to repo

    # robin backup create mongoapp miniotest-repo --snapshotid c9689aca0d0911ebace6af20df9778b7
    
    • List backup and validate the snapshot has been backed up to minio repo

    # robin backup list | grep mongoapp
    
  • Test restore from minio repo,

    • List the backups and note down the backup id.

    # robin backup list | grep mongoapp
    
    • Restore the backup using the backup id.

    robin app restore mongoapp --backupid 89d6cbde0d1111eba8b7b71d742fe09d --wait