Applications

The term “Application” in this document refers to any software that you’d deploy in Docker or LxC containers on the ROBIN Hyperconverged Kubernetes platform. For example, in a stack that includes a WebServer, MiddlewareServer and a DatabaseServer, this document refers to all three as applications. That is to say even complex distributed Big Data and Database software such as Cloudera, Hortonworks, ElasticSearch, Oracle RAC clusters are all refered to as Applications.

The software for an Application itself is packaged into one or more Docker or LxC container images. There are three ways one can deploy these containers on ROBIN Hyperconverged Kubernetes:

  • As straight up Kubernetes YAML files by running kubectl create -f <YAML-FILE>

  • As Helm charts by running helm install <CHART-NAME>

  • As a ROBIN Application bundle by running robin app create <APP-TEMPLATE-FILE>

Kubernetes and Helm documentation can be referred to for using kubectl and helm CLIs respectively. This document only describes deploying applications using the ROBIN Application Bundles.

So, what is a ROBIN application bundle? Why is it needed? How is it different from Helm charts?

ROBIN Application Bundles

The purpose of any application deployment platform is to make it radically simple to bring up and manage the lifecycle of applications. To achieve this radical simplicity, beyond just packaging the application’s code in containers, one also needs a way to describe the infrastructure needs of the application.

_images/apps-bundle-dashboard.png

ROBIN’s Application bundles enable an App-store like experience for deploying any Application, with special focus around the needs of complex distributed data-heavy Big Data, NoSQL and Databases. Applications are deployed as pods on the ROBIN hyperconverged Kubernetes platform. Both Docker and LxC containers are supported. To make 1-Click deployment easy for any application, ROBIN needs to know some basic information about an application; details such as:

  • Services that an application offers. For example, for MongoDB the services would be ConfigServer, QueryRouter, Shards, RepicaSets.

  • The Docker or LxC images for each of these services. They can be vendor published or custom Docker images hosted on DockerHub or any private Docker registry.

  • Optionally any application specific placement constraints can be specified. For example, anti/affinity policies among services.

This information is written down in a simple YAML file, which is called a manifest file. The manifest file, along with an icon for the app and any optional application-specific hooks are packaged into a single tar file and uploaded into the ROBIN platform. This package is called an Application Bundle in ROBIN.

The ROBIN software comes pre-packaged with ready-to use Application Bundles for several applications including:

  • Big Data: HortonWorks, Cloudera, Spark, Elastic Search, ELK, Splunk

  • NoSQL: MongoDB, Cassandra, CouchDB)

  • Databases: Oracle, Oracle RAC, SAP HANA, Microsoft SQLServer, IBM DB2, Postgres, MySQL, MariaDB

  • AI/ML: TensorFlow, Caffe, Theano, R, Jypiter

Users can onboard any application or database that runs on Linux very quickly. Refer to the ROBIN Application Bundle Guide for instructions.

Once the bundles are uploaded ROBIN presents an App-store like view where one can, with 1-Click deploy these applications from the UI, or one single command from the CLI, or a single REST call programmatically.

The highly flexible and customizable ROBIN bundle framework offers the application developers freedom from the underlying infrastructure. You define the application once and deploy it anywhere, on-prem or public clouds like aws, azure and google cloud.

Adding a new application bundle

A new application bundle can be added using robin bundle add command as shown below:

$ robin bundle add [-h] [--image IMAGE] [--integrity_check] name version path

   positional arguments:
     name               File object name
     version            File object version
     path               Pathname of file to upload or URL of the bundle

   optional arguments:
     -h, --help         show this help message and exit
     --image IMAGE      Pathname to the image
     --integrity_check  Send a checksum to ensure integrity of uploaded file.


# Example of adding a bundle

$ robin bundle add elasticsearch 1.0 docker-elasticsearch-6.2.3.tar.gz
File upload: 100% -- (38341/38341)
Job:  169 Name: BundleAdd            State: VALIDATED       Error: 0
Job:  169 Name: BundleAdd            State: COMPLETED       Error: 0

Note

You need to specify a <version> number when adding an application bundle. This number can be same as or different from the version of the application itself. In the above example, you are adding an application bundle, which you have versioned as 1.0, whereas the ElasticSearch software version itself is 6.2.3. Remember that an application bundle is a packaging of the actual application software, an icon, hook scripts and a manifest file. The bundle version number allows you to maintain different versions of all of these components.

Listing available application bundles

Bundles added to ROBIN can be viewed using robin bundle list command. This call is guarded by Role Based Access Control permissions, which means that users will only see the bundles that they have been given view permissions.

$ robin bundle list [-h] [--bundleid BUNDLEID] [--name NAME]
                    [--version VERSION] [--files] [--full] [--json]

optional arguments:
    -h, --help           show this help message and exit
    --bundleid BUNDLEID  Id of bundle to get info for.
    --name NAME          Name of the bundle to list (default is None)
    --version VERSION    version of the bundle to list (default is None)
    --files              List bundle files
    --full               Display additional information about each bundle
    --json               Output in Json

# EXAMPLE:

$ robin bundle list
Bundle Id | Name          | Version | Zone Id    | File Object Id
----------+---------------+---------+------------+----------------
1         | cassandra     | 3.7     | 1517201362 | 1517208926703
2         | elasticsearch | 5.4     | 1517201362 | 1517208949245
3         | hdp           | 2.4.3   | 1517201362 | 1517208985500
4         | myssql        | 3.0     | 1517201362 | 1517209037452
7         | mariadb       | 10.1.23 | 1517201362 | 1517291916828
9         | couchbase     | 4.5     | 1517201362 | 1517292039769
5         | cloudera      | 5.8.2   | 1517201362 | 1517291796554
6         | mongodb       | 3.4.2   | 1517201362 | 1517291861487
8         | couchdb       | 2.0     | 1517201362 | 1517292011644

Deleting an existing application bundle

Removing an existing bundle from the system is done using robin bundle remove command as:

$ robin bundle remove [-h] zoneid bundleid

positional arguments:
  zoneid      Zone Id
  bundleid    Bundle Id

optional arguments:
  -h, --help  show this help message and exit

# EXAMPLE:

$ robin bundle remove 1498721081 6
File object with object_id=1498968125624 successfully deleted.

Helm charts

Services

1-Click application deployment

An application or an entire application stack can be deployed in 1-Click using ROBIN’s UI, CLI or REST API. One might want to customize several parameters when deploying an application. For example, the CPU or Memory to give to certain components, the size and performance characteristics of storage volumes, which services to enable/disable, setting constraints on data locality, service anti/affinity etc. The best way to do this customization is through the ROBIN UI, which automatically renders a very easy, yet flexible, wizard to apply customizations to your application’s deployment. It renders this wizard by parsing the manifest.yaml file in the Application’s bundle.

Customizing your application deployment

A new application is created by logging into the ROBIN UI, navigating to the applications bundle page and clicking on the application bundle. To allow customizing the application, ROBIN renders a wizard by parsing the manifest file for the application bundle. For example, the below screenshot shows how one can customize and deploy a Hortonworks Hadoop cluster by simply specifying the scale, compute needs, storage needs, and optionally any placement restrictions without worrying about the specifics of physical infrastructure on which the cluster would be provisioned. This process is so simple that a fairly large Hadoop cluster can be brought up in less than 30 minutes, which includes allocating the required storage, compute, networking infrastructure and configuring all the required services.

_images/apps-deploy-wizard.png

Provisioning involves four steps, which are automated by ROBIN making them invisible to the end user.

  1. ROBIN selects the best compute nodes to deploy the various service components factoring in system utilization, isolation policies, anti/affinity policies and multi-tenancy quotas. It then generates corresponding Kubernetes pod specifications and publishes them to the Kubernetes API server. The Kubernetes scheduler and Kubelet coordinate to bring up those pods on the nodes specified by ROBIN. The CRI driver that ships with ROBIN Hyperconverged Kubernetes Platform ensures that the containers are brought up with the correct runtime and options (for example, use LxC for Oracle RAC and use Docker graphdriver if the application stores data on the root file system, etc).

  2. Storage volumes are allocated from disks in the ROBIN cluster factoring in media type (SSD, HDD, NVMe), QoS policies, data-locality constraints, data protection requirements (2 or 3-way replication), inline data compression and encryption needs. The CSI driver that ships with ROBIN Hyperconverged Kubernetes Platform ensures that the appropriate PersistentVolumes and PersistentVolumeClaims are created.

  3. The CNI driver that ships with ROBIN Hyperconverged Kubernetes Platform ensures that IP addresses are assigned to each container. The IP addresses are also made persistent, which means that when a pod is relocated from one host to another the IP address of that pod, along with its hostname, are also migrated to the new host.

  4. App-specific hook scripts are executed to customize the Hadoop cluster.

The below figure shows this in action.

_images/apps-deploy-infra.png

Saving application customizations as a template

Template is a way to store the configuration information of an application bundle, and re-use it for deploying similar instances at a later time. For example, in the case of an elasticsearch bundle, one can creates small, medium and large clusters by allocating appropriate resources.

Creating and Saving templates.

Please Navigate to the Bundles Dashboard from the Applications menu. Choose the bundle for which you would like to create a template.

Provide a name for the application. Configure the application by filling in all the mandatory fields if any. Click on the Save Template button and give it a name and save it.

Here is an example using Elasticsearch bundle.

  1. Go to Application > Bundles > Elasticsearch

_images/template-bundle-dashboard.png
  1. Add application name and edit the configuration as necessary and click on Save as Template

_images/save_template.png
  1. In the Save Template pop-up Dialogue, fill the Name Field and Description Field and click Save

_images/save_template_popup.png

Elasticsearch template is created successfully as shown below

_images/template_save_ok.png

Consuming the saved templates

Navigate to the Bundles Dashboard from the Applications menu. Hover over the bundle which you are trying to instantiate. If there is already a template saved the Template link would get highlighted. Clicking on the template link would open a popup window with the template information. Choosing the appropriate template and clicking on Create App button This would take you to the Application provisioning page.

Example 1. Go to Application > Templates > Elasticsearch

_images/use_template.png
  1. Click on the es_template_small as shown below

_images/select_template.png
  1. In the Create App popup window, give App Name and select the resource pool and IP Subnet from the drop down and click on Create App.

_images/template_app_create.png

This will provision the elasticsearch application as defined in the template.

1-Click deploy application from UI

In this section we will show you how to deploy application with 1-Click using Elasticsearch as an example.

  1. Go to Application > Bundles > Elasticsearch

_images/template-bundle-dashboard.png
  1. As shown in below, Add application name(1), Togle the switch to enable/disable services (2), for each service/role change the number of instances (3), cpu and memory(4) and storage requirements(5)

_images/app_create_input.png
  1. Scrolldown to the bottom of the screen and click Provision Application

_images/click_provision.png
  1. You should see a job progress window that shows that the application is being provisioned.

_images/app_provision_job_progress.png

1-Click resource scaling

  1. Go to Applications > Applications. From the applications dashboard click on the application you want to scale the resource.

_images/app_dashboard.png
  1. From the Application details page, click QoS icon as shown below. In this case, let us increase the memory and cpu for the dedicated master nodes.

_images/qos_icon.png
  1. In the pop-up window increase the memory and cpu using the slider and click save.

_images/qos_popup.png
  1. You should see a job progress window that shows that ROBIN is adding additional resources to the containers.

_images/resource_scalling_job_progress.png
  1. Once the job completes, you can see that the containers now showing increased resource.

_images/resource_scalling_succesful.png

1-Click service scaling

  1. Go to Applications > Applications. From the applications dashboard click on the application you want to scale the service/role.

_images/app_dashboard.png
  1. From the Application details page, click on Actions and then Scale Out as shown below. For this example, let us scale out master_eligible_note. As you can see there is currently 1 master_eligible_node on this cluster. Let us scale this to two nodes.

_images/scaleout_menu.png
  1. In the scale out pop-up window use the slider to increase the node by 1 and click Scale Out.

_images/scaleout_popup.png
  1. You should see a job progress window that shows that ROBIN is adding additional containers for the elsticsearch service.

_images/scaleout_job_progress.png
  1. Once the job completes, you can see that the additional container is now showing up in the app info page.

_images/scale_out_ok.png

1-Click application snapshots

  1. Go to Applications > Applications. From the applications dashboard click on the application you want to the snapshot.

_images/app_dashboard.png
  1. From the Application details page, click on MANAGE to go to Snapshot and Clones Dashboard and click on create one now as shown below.

_images/snapshot_manage.png
  1. In the Create Snapshot window provide a name for the snapshot and click create.

_images/create_snapshot.png
  1. You should see a job progress window that shows that ROBIN is taking a snapshot of the application.

_images/snapshot_job_progress.png
  1. Once the job completes, you can see that the snapshot is shown snapshot/clone dashboard page.

_images/snapshot_ok.png

1-Click application clones

  1. Go to Applications > Applications. From the applications dashboard click on the application you want to scale the service/role.

_images/app_dashboard.png
  1. From the Application details page, click on MANAGE to go to Snapshot and Clones Dashboard. Click on actions menu for the snapshot from which you want to create a clone from and select ‘Thin Clone’.

_images/thin_clone_menu.png
  1. In the clone detail page enter the clone name, resource_pool and ip-pool and change cpu and memory based on your requirement. Scroll down to the bottom of the page and click ‘Clone’.

_images/clone_window.png
  1. You should see a job progress window that shows that ROBIN is creating a clone from the snapshot.

_images/clone_job_progress.png
  1. Once the job completes, go to the Application Dashboard as shown in Step 1. You can see that the cloned application is shown in the Dashboard.

_images/clone_ok.png

1-Click application upgrades

ROBIN provides seamless 1-click upgrade for the applications. When a new version of the application bundle is added to ROBIN, all the elligible applications are marked for upgrade. Once the user opens the application from the ROBIN UI it prompts the user that the upgrade is available for this application. Users can complete the upgrade with 1-click or test the upgrade without any impact to the application. ROBIN also supports rolling upgrade if the application supports it. Given below is the example of an upgrade workflow for Elasticsearch application.

  1. Go to Applications > Applications. From the applications dashboard click on the application you want to upgrade.

_images/app_dashboard.png
  1. From the Application details page, click on ‘Update(s) Available’. A popup window will appear which will show what the versions available for upgrade and options to do a Rolling Upgrade and Test the upgrade. Slect the version to upgrade and click upgrade.

_images/app-upgrade.png
  1. You should see a job progress window that shows that ROBIN is upgrading the application.

_images/app-upgrade-job.png

Note

If you have several applications to upgrade, you can use the ROBIN CLI to perform the upgrade on all the applications.

1-Click QoS Controls

  1. Go to Applications > Applications. From the applications dashboard click on the application you want to scale the resource.

_images/app_dashboard.png
  1. From the Application details page, click QoS icon as shown below. In this case, let us increase the memory and cpu for the dedicated master nodes.

_images/qos_icon.png
  1. In the pop-up window adjust the slider to set min/max read and write iops for the volume data1.

_images/qos_popup_iops.png
  1. You should see a job progress window that shows that ROBIN is adding additional resources to the containers.

_images/qos_ok.png

Recipes

Influence placement of pods to meet an application’s requirements

ROBIN provides the user the ability to influence how the application pods and the storage is placed. ROBIN’s powerful placement algorithm offers users several options to provision the application to attain optimal performance. The following are the placement choices available for all the applications out of the box.

  • Round-robin placement of pod across different different nodes, racks and datacenters

  • Prevent placing more than one pod of the same service (ex. Datanode in case of Hadoop) on the same node, rack or datacenter

  • Enforce Storage and Compute for the pod from the same node, rack or datacenter.

  • Enforce Storage and Compute for the pod from the different node, rack and datacenter.

  • Restrict placement of pods on nodes with certain tags

  • Enforce affinity between pods (e.g. Place Namenode pod and zookeeper pod on the same machine)

  • Enforce Anti-affinity between pods (e.g Do not place Namenode pod and Datanode pod on the same machine)

Users can simply turn on or off these placement restrictions from the App Creation menu as shown below. In this example Elasticsearch application is shown.

  1. In the Create Application page, click on the ‘Show advanced configuration options’ as shown below

_images/placement-rules-1.png

Note

Follow the instructions from 1-Click deploy application from UI if you are not familiar with how to create applications on ROBIN.

  1. Simply turn on the applicable placement rules you want to enforce for the application and click Provision.

_images/placement-rules-2.png

Determining on which nodes an application’s pods are running

From the application dashboard click on the application you want to check. In the application INFO tab you can see the physical hosts the pods are running.

_images/container-physical-hosts.png

ROBIN by default shows all the containers in the tile format. You can change to tabular format by clicking the toggle swtich as shown below. This comes in handy when there are more number of pods in the application.

_images/container-physical-hosts-table.png

Relocate an instance/pod from one node to another

If the host becomes unavailable or unreachable in the event of hardware or network failures ROBIN automatically relocates the pods running on that host to other hosts where enough resource is available. ROBIN also allows the user to manually relocate the instance to other hosts with a single click. This feature comes in handy when there are planned maintenance. Follow the instructions below to relocate an instance.

  1. Go to Application Detail page. Click on the drop down menu for the container and click on Relocate as shown below.

_images/instance-relocate.png
  1. Choose the host where you want the instance to relocate to. If you don’t select a host ROBIN will relocate the container to a suitable host honoring the placement rules.

_images/relocate-container.png

Getting console access to a pod

ROBIN provides way to connect to the pod console directly from the UI. It is seamless for both docker and LxC containers. This feature enables true selfservice. Follow the instructions below to connect to the pod console.

  1. Go to Application Detail page. Click on the drop down menu for the container and click on Console as shown below.

_images/container-console.png
  1. ROBIN will open up the console in a different browser tab.

Accessing a pod from a different network

The application pods often need to communticate with applications outside ROBIN or K8S. Chances are the outside application is on a different network than that of ROBIN. You may recall from Add IP-pool section of the document that ROBIN support Calico and OVS drivers for networking. Depending on the ip-pool that was used for the application you may need to follow different steps to access the container.

If the application was created using Calico ip-pool follow the steps below

  1. Go to Application Detail page. Identify the physical host(1) where the pod is running and the physical host port(2) the pod’s service port is mapped to. In the example given below, the elasticsearch pod is running on physical host eqx02-poc01-c03 and the elasticsearch port 9200 is mapped to the physical host port 20000.

_images/calico-port.png
  1. Use <physical-host>:<mapped-port> to access the service. In this case typing eqx02-poc01-c03:20000 on the browser address bar would give you the following result for elasticsearch application.

_images/elastisearch-calico.png

If the application was created using OVS ip-pool follow the steps below

  1. Go to Application Detail page. Identify the ip address(1) of the pod and the port(2) the pod is exposing. In the example given below, the elasticsearch pod’s ip-address is 10.10.1.119 and the elasticsearch isexposing port 9200.

_images/ovs-port.png
  1. Use <container-ip-address>:<service-port> to access the service. In this case typing eqx02-poc01-c03:20000 on the browser address bar would give you the following result for elasticsearch application.

_images/elastisearch-ovs.png

Accessing application logs from a pod

The application logs can be accesses by connecting to pod console. Please follow the steps in section Getting console access to a pod. Most of the time the docker containers write the logs to the stdout. If that is the case you can see the logs by running the following command:

docker logs <docker-container-id>

In order to find the container id of the docker container you need to log into the physical node where the container is running. Refer to the section Determining on which nodes an application’s pods are running if you are not sure how to find the physical node where the container is running. Once you have the physical node follow the steps below

  1. ssh to the physical host:

    $ ssh <host-name>
    
  2. Run the command below to find the docker container id:

    $ docker ps -a |grep esapp-ovs-master-eligible-node
    
    7194bbcadf1c        docker.elastic.co/elasticsearch/elasticsearch@sha256:de7061f7fa78aa78de4b38287fb1f0a21689b685cf1fb6694195088b5c307880   "/robin/robin_entr..."   49 minutes ago      Up 49 minutes                           k8s_esapp-ovs-master-eligible-node-01_ip-10-10-1-119_t001-u000002_c6d7f2ea-d8dc-11e8-b14a-002590042b86_0
    

Note

docker ps -a is the docker command to list all the docker containers running in the physical host. We are then using ‘grep’ command to find the container id of the pod.

  1. Run the below command to see the docker logs:

    $ docker logs 7194bbcadf1c
    docker logs 7194bbcadf1c
    /usr/share/elasticsearch/data
    cluster.name: esapp-ovs
    node.name: vnode119-poc01.robinsystems.com
    node.master: true
    node.data: true
    node.ingest: false
    network.bind_host: 0.0.0.0
    network.publish_host: 10.10.1.119
    discovery.zen.ping.unicast.hosts: 10.10.1.119,10.10.1.115,10.10.1.88
    discovery.zen.minimum_master_nodes: 2
    http.cors.enabled: true
    http.cors.allow-origin: "*"
    xpack.license.self_generated.type: basic
    node.attr.host_name: eqx02-poc01-c03.robinsystems.com
    node.attr.rack_id: default
    cluster.routing.allocation.awareness.attributes: rack_id,host_name
    path.data: /usr/share/elasticsearch/data
    [2018-10-26T05:05:48,791][INFO ][o.e.n.Node               ] [vnode119-poc01.robinsystems.com] initializing ...
    

How to connect to ROBIN pod

All the ROBIN services are running in a special pod called as robinds. This pod runs in all the physical hosts managed by ROBIN. Depending on the role (Compute, Storage, Manager) of the node, applicable ROBIN services run in the pod.

To connect to the ROBIN container, follow either of the below steps.

Connect using rbash. ROBIN utility rbash is shipped with the ROBIN software. Run the below command to source robinrc and use the rbash command to connect to the container:

$ source /urs/local/bin/robinrc
$ rbash robin

You can also connect to the container using docker command.

Find the container id using the below command:

$ docker ps -a |grep rcm
6413e96a646a        fb55ff6220f6                                                                                 "/opt/robin/curren..."   2 days ago          Up 2 days                               k8s_robinrcm_robin-ds-jg8fx_kube-system_f30d3c60-d721-11e8-b14a-002590042b86_0

Then, connect to the docker container using the container id using the below command:

$ docker exec -it  6413e96a646a bash

Accessing hook logs of a pod

To access hook logs log into the ROBIN pod as described in How to connect to ROBIN pod from physical node that is acting as ROBIN Master and follow the below steps.

  1. Run the below command to find the job of the task. In this case the job id is 83:

    $ robin job log
    83       | ApplicationCreate  | Adding application 'esapp-ovs'                                                                                 | COMPLETED        | 25 Oct 22:05:20 | 0:00:46 | robin  |
    |->84    | RoleCreate         | Provisioning containers for role 'master_eligible_node'                                                        | COMPLETED        | 25 Oct 22:05:24 | 0:00:23 | robin  |
    |  |->85 | VnodeAdd           | Adding vnode 'esapp-ovs.master_eligible_node.01' on eqx02-poc01-c03.robinsystems.com                           | COMPLETED        | 25 Oct 22:05:24 | 0:00:22 | robin  |
    |  |->86 | VnodeAdd           | Adding vnode 'esapp-ovs.master_eligible_node.02' on eqx02-poc01-c01.robinsystems.com                           | COMPLETED        | 25 Oct 22:05:24 | 0:00:23 | robin  |
    |  |->87 | VnodeAdd           | Adding vnode 'esapp-ovs.master_eligible_node.03' on eqx02-poc01-c02.robinsystems.com                           | COMPLETED        | 25 Oct 22:05:24 | 0:00:20 | robin  |
    
  2. The hook logs pertaining to this job can be found by going to the directory as shown below:

    $ cd /var/log/robin/server/83
    $ ls
    job.log  postcreate.err  postcreate.log  validate.err  validate.log
    

What happens when a pod is stopped?

ROBIN releases all the compute resource allocated to the pod, when a pod is stopped. ROBIN maintains all the metadata about the pod in ROBIN’s internal database. The persistant storage volumes and the data are maintained as is. So the next time when pod comes back up the storage volume is attached to the pod.

Determining how many and what resources an application is consuming

The top section of the application’s detail page shows all the compute resources memory, cpu, gpu and the number of containers the application is consuming.

_images/application-compute-resources.png

Click on the storage tab, to find out the storage resources consumed by the application.

_images/application-storage-resources.png

Getting performance metrics of an application

To view the realtime perform metrics of an application go to Application detail page and click on Perfomance tab as shown below.

_images/application-performance.png

What to do when an application or pod is in ADMIN_WAIT state?

When a pod goes to ADMIN_WAIT state, it usually means that there is an issue with the pod. It could be storage related issues, or application related issues which may have brought the container down. Follow the below steps to troubleshoot the issue.

  1. Run the below command to check if all the containers of the application are in ONLINE state:

    $ robin instance list --app esapp-ovs
    ID | Owner/Tenant         | ResPool  | Container                         | Hostname                        | IP Addresses   | State   | Status | Host            | Engine | Cores | GPUs | Mem     | Ctime
    ---+----------------------+----------+-----------------------------------+---------------------------------+----------------+---------+--------+-----------------+--------+-------+------+---------+----------------------
    8  | robin/Administrators | dev-pool | esapp-ovs.master_eligible_node.01 | vnode119-poc01.robinsystems.com | 10.10.1.119/24 | STARTED | ONLINE | eqx02-poc01-c03 | docker | 4     | 0    | 12.0 GB | 25 Oct 2018 22:05:47
    9  | robin/Administrators | dev-pool | esapp-ovs.master_eligible_node.02 | vnode115-poc01.robinsystems.com | 10.10.1.115/24 | STARTED | ONLINE | eqx02-poc01-c01 | docker | 4     | 0    | 12.0 GB | 25 Oct 2018 22:05:47
    10 | robin/Administrators | dev-pool | esapp-ovs.master_eligible_node.03 | vnode88-poc01.robinsystems.com  | 10.10.1.88/24  | FAULTED | OFFLINE | eqx02-poc01-c02 | docker | 4     | 0    | 12.0 GB | 25 Oct 2018 22:05:47
    
  2. Log into the host where the FAULTED container is running and check the application logs as shown in Accessing application logs from a pod. If the issue cannot be found proceed to the next step.

  3. Now connect to the ‘robinds’ pod (As you may recall from the previous sections robinds is a special pod that runs all the ROBIN services). Refer to section How to connect to ROBIN pod and check the ROBIN logs for any errors.

  4. Fix the underlying issue and re-start the pod.

  5. If any of the logs indicate storage related issues then refer to Troubleshooting storage issues for further troubleshooting.

Encrypt and/or compress data of an application

ROBIN provides the ability encrypt and/or compress the data of an application at the volume level. This needs to be set during the application creation. It cannot be changed after the application is provisioned. ROBIN supports LZ4 compression and CHACHA20, AES128 and AES256 encryptions. Follow the steps below to enable encryption and compression for the application.

_images/encryption-compression.png

Influencing configuration of an application’s storage volumes

While provisioning an application several configurations can set for the storage volumes.

  • Media Type - HDD, SSD, NVMe

  • Replication - App Protected, 2 Copies, 3 Copies

  • Fault Domain - Disk, Host, Datacenter

  • Block Size - 512, 4k

  • Compression - LZ4

  • Encryption - CHACHA20, AES128 and AES256

  • Layout
    • Pack:

    • Round-Robin:

  • Workload
    • Throughput: The volume has high throughput requirements. Typically these are volumes that are used for large stream of IOs such as data volumes. The number of such volumes per device is configurable. By default Robin allocates no more than 2 such volumes per device.

    • Latency: The volume has low latency requirements. Typically these are volumes that are used for heart beats. The amount of data written is small, but the IO has to complete within strict time limits. These volumes will not be allocated on the same device as a “Throughput” volume. The number of such volumes per device is configurable. By default ROBIN allocates no more than 2 such volumes per device.

    • Dedicated: The entire device is dedicated to a volume. This ensures that the volume gets all the IO bandwidth offered by the device. This can result in under utilized storage capacity if the volume size is less than the device size.

    • Ordinary: All other volumes that are not classified as any of the above.

All these options can be configured from the application creation page as shown below

_images/storage-configs.png

Setting up multiple network cards for an application

The application is automatically configured with private and public networks, when you are using calico based ip-pool. The internal communication happens through the 192.x.x.x subnet while the external communication happens through the physical host network by leveraging port mapping.

When you are using OVS and if the application requires multiple network cards (for example, Oracle RAC database requires two network cards one for internal network and the other for external network) it can be configured using ROBIN bundles.

Determining the number of clones for an application?

Follow the steps below to find out how many snapshots and clones an application has.

  1. Go to Application details page and click on Manage as shown below. This page lists all the snapshots and the clones of the application.

_images/clone-dashboard.png

What happens when an application’s snapshot is deleted?

When an application’s snspshot is deleted ROBIN removes all the meta about the snapshot. The snapshot which is successor to the snapshot being deleted will own all the data. If there are stale block they will be inventually garbage collected. If the snapshot being deleted is the latest one then the deletion will result in free space. If the snapshot is not the latest one then deleting a snapshot may or may not free up the space.

Note

ROBIN doesn’t allow deleting a snapshot of the application if there are any clones created from the snapshot.

Relocating an application’s data from one disk to another

  1. Run the command below to find all the volumes in the disk:

    $ robin drive info 0x500a0751096700e9
     Drive: 0x500a0751096700e9
       Role: Storage
       Type: SSD
       Make: Crucial/Micron MX1/2/300
       Model: Micron_M500_MTFDDAK960MAV
       Zone: 1540338880
       Path: /dev/disk/by-id/ata-Micron_M500_MTFDDAK960MAV_1327096700E9
       Node: eqx02-poc01-c01.robinsystems.com
       Protected: No
       Reattachable: No
       State: READY
       Status: ONLINE
       Maintenance: OFF
       Allocations:
         Capacity: 894.25 GB
         Allocated: 20.0 GB
         Allocation Factor: 0.02
         Stormgr AllocScore: 98
         Volumes: 2
         App          |                                                                              Volume | Allocated | Volume Size
         -------------+-------------------------------------------------------------------------------------+-----------+-------------
         esapp-calico | esapp-calico.master_eligible_node.03.root_fs.1.b5d4a60b-22fb-4c1d-9f75-db6a90c52ef4 |   12.0 GB | 10.0 GB
         esapp-ovs    |    esapp-ovs.master_eligible_node.02.root_fs.1.ba676664-b712-4d9f-8087-ee81679ec9a3 |   12.0 GB | 10.0 GB
    
       Physical Usage:
         Usage: 1.22 GB
         Free: 882.69 GB
       Tags: {}
       Maximum volumes per disk: 10
       Maximum latency sensitive volumes per disk: 2
       Maximum throughput intensive volumes per disk: 1
    
  2. For each volume run the following command to relocate the volume to another disk:

    $ robin drive evacuate 0x500a0751096700e9 --volume esapp-calico.master_eligible_node.03.root_fs.1.b5d4a60b-22fb-4c1d-9f75-db6a90c52ef4
    Submitted job '109'. Use 'robin job wait 109' to track the progress