18. Virtual Machines Support On Robin CNP

Robin CNP supports running Linux Kernel-based Virtual Machines (KVM) on your clusters. It supports virtual machines (VMs) using the Linux KVM Application. Robin CNP exposes the KVM parameters as Robin bundles for creating VMs on CNP.

If your cluster is running Robin CNP version 5.3.3 and above, you can run VMs on your Robin clusters.

You can register any qcow2 format KVM image with a File Collection on Robin CNP clusters and create a Robin bundle using the registered KVM image. For more information about Robin Bundles, see Using the created Robin bundle, you can create VMs and manage them using Robin CNP. You can deploy virtual machines just like applications using Robin CNP.

Note

To create VMs and manage them on Robin CNP, you should have knowledge about Robin Bundles and KVM. For more information about Robin Bundles, see here.

Using Robin CNP, you can perform the following VM related operations:

  • Deploy a VM (Using robin app create command)

  • Start a VM

  • Restart a VM

  • Stop a VM

  • Relocate a VM

  • Delete a VM

18.1. Points to Consider for using KVM on Robin

The following are the important points you should consider when using VMs on Robin CNP:

  • Supported Networking for VMs:

    • OVS Networking - Version 5.3.3 onwards

    • Calico Networking (IPv4 only) - Version 5.3.7 onwards

  • You can take snapshots, clone, backup, and restore of the VMs in CNP.

  • A virtual machine can run any Operating System that is compatible with qcow2 and the underpinning KVM version.

18.2. High-Level Steps to Create and Manage VMs

The following are the high-level steps for creating and managing VMs on Robin CNP.

  1. Check prerequisites.

  2. Verify CNP configuration.

  3. Download required qcow2 image from a reliable source.

  4. Register the downloaded qcow2 image with Robin CNP.

  5. Customize the manifest.yaml file for creating KVM Robin Bundle.

  6. Create a KVM Robin Bundle tar file.

  7. Add the KVM Robin Bundle.

  8. Create a VM using the KVM Bundle.

  9. Lifecycle management of a virtual machine.

18.3. Prerequisites

18.3.1. Software Prerequisites

  • The Robin cluster on which the VM is to be deployed is installed with Robin Cloud Native Platform v5.3.3 or above.

In order to verify this, run the following command:

# robin version

18.3.2. Hardware Prerequisites

  • The following must be enabled in the BIOS settings of the bare metal server:

    • Intel Processors - VD-T

    • AMD Processors - SVM

    • BIOS VT-D (Intel)

    • SVM (AMD)

In order to verify these settings have been appropriately set, run the following command:

# virt-host-validate

18.3.3. Image Prerequisites

  • The VM image must be in qcow2 format.

  • The VM image should have the cloud init service. If the image does have cloud init service, the networking for the VM must be configured manually.

  • The VM image must mount when booting.

  • The Robin file collection on which the VM image is to be hosted must be larger than the size of the compressed aforementioned image.

In order to verify the size of the available file collection(s), run the following command:

# robin collection list

18.3.4. Robin Install Prerequisites

The following Robin installation pre-checks must pass:

  • KVM Virtualization

  • KVM vhost-net

  • KVM Networking

  • KVM PCI Device Passthrough (For SRIOV support)

In order to verify these pre-checks have passed, run the following command:

# ./robin-install.sh precheck

18.3.5. Networking Prerequisites

  • The Robin cluster is configured with a public OVS based IP pool from which the VM IPs are assigned from.

  • VM IP addresses must be in the same L3 subnet as the hosts. For example this means if the host IP is 101.1.1.10/24 then the IP-Pool must also have a netmask of 24 bits.

  • VM IP addresses must be in a routable LAN from the hosts on which they are deployed.

  • The VM image must be configured with static IP addresses.

18.3.6. Storage Prerequisites

  • The storage capacity of the nodes within the cluster must be greater than or equal to the planned size of the file system(s) of the VMs as defined in the manifest file for the respective Robin Bundle.

  • The specified capacity of the rootfs volume of the KVM based application to be deployed must be greater than the size of the associated VM image.

18.4. Register a VM Image

In order to create a KVM application, the appropriate qcow VM image must first be uploaded to a file collection and registered with Robin. The image can either be present locally or available for download via a registry already known to Robin. An example command for registering a pre-downloaded image is shown below. More details on the robin image add command can be found here.

Example

# robin image add centos7 2009 QCOW ~/centos-kvm/CentOS-7-x86_64-GenericCloud-2009.qcow2
File upload: 100% -- (1391329280/1391329280)
Job:   84 Name: ImageAdd             State: VALIDATED       Error: 0
Job:   84 Name: ImageAdd             State: COMPLETED       Error: 0

18.5. Create a KVM Robin Bundle

A KVM Robin Bundle is a tar file containing the following components: a manifest file, a directory for icons and a directory for scripts. The following section focuses on the manifest file as it is a mandatory file and is considered the blueprint for the application. The other components are optional and can be populated at the user’s descretion. More details on the aforementioned components and how to customize them can be found here.

Note

Given that a KVM bundles are treated similarly to other application bundles, the process of creating and registering them is the same. These steps are detailed here whilst the sections detailed below focus on customizing manifest file attributes which differentiate KVM based bundles from other application bundles.

18.5.1. Sample Manifest file

The following is an example of a manifest file for a KVM application running the CentOS operating system.

name: CentOS
version: "7"
icon: icon.png
snapshot: enabled
clone: enabled
roles: [server]

server:
   name: CentOS
   multinode: true
   scaleout: enabled
   scaledown_mem: enabled
   addvolume: enabled
   scaledown_mem: enabled
   compute:
      memory: 8G
      hugepages_1gi: 0G
      hugepages_2mi: 0G
      # hugepages_locked: "yes"
      # hugepages_noshared: "yes"
      # memballoon: "model=none,stats.period=10"
      cpu:
       reserve: true
       cores: 8
       # rt_cores: 1-7
       # rt_sched: "fifo"
       # rt_priority: 1
       # emulatorpin: "outside_cpuset"
       # cache_ways: 4
       # kvmopts: "mode=host-model,match=exact"
       # kvmopts: "mode=host-passthrough,require=tsc-deadline,check=none"
   devices:
   - name: fec
      type: pci
      class: "0x120000"
      vendor: "0x8086"
      count: 0
   image:
      name: centos7
      version: "2009"
      engine: kvm
      ostype: linux
      osvariant: rhel7
      graphics: "none"
      imgsize: "8G"
   storage:
   - type: root_fs
      media: ssd
      bus: scsi
      path: /
      size: 20G
   - type: data
      media: ssd
      bus: scsi
      count: 1
      path: /dev/data
      fstype: raw
      size: 10G
   - type: logs
      media: ssd
      bus: scsi
      count: 1
      path: /dev/log
      fstype: raw
      size: 10G
   # livenessProbe:
   #   tcpSocket:
   #     port: 22
   #   initialDelaySeconds: 180
   #   periodSeconds: 10
   env:
   # KVM specific env variables
   - CLOUD_INIT:
      type: boolean
      value: true
   - CLOUD_INIT_ADMIN_USER: robin
   - CLOUD_INIT_ADMIN_PASSWORD:
      type: password
      value: "robin123"
   - CLOUD_INIT_USERDATA:
      type: yaml
      value: |
         # cloud-config
         password: "{{ROLES.server.ENV.CLOUD_INIT_ADMIN_PASSWORD}}"
         chpasswd:
         list: |
            root: "{{ROLES.server.ENV.CLOUD_INIT_ADMIN_PASSWORD}}"
         expire: False
   - QEMU_GUEST_AGENT: qemu-guest-agent-2.12.0-3.el7.x86_64.rpm

18.5.2. Customizing the Manifest file

The following are the Role level sections that need to be edited for VM deployment. More details on all customisable sections and/or attributes within the manifest file can be found here.

Image

Within each role an image section must be specified. For VM deployment this section is associated to the KVM image that was uploaded to a file collection earlier. Below are the attributes that need to be customized for the respective image.

  • name - This name must match the image name supplied when the image was registered.

  • version - This version must match the image version supplied when the image was registered.

  • engine - Engine can be docker/kvm/lxc. Docker for docker containers, KVM for VM’s and LXC for Linux containers.

  • ostype - The guest configuration for a type of operating system (Example: Linux, Windows). This will attempt to pick the most suitable ACPI and APIC settings, optimally supported mouse drivers, Virtio, and generally accommodate other operating system quirks..

  • osvariant - It is the kind of GuestOS. It is an optional specification. However, it provides some default parameters for each virtual machine that can help improve performance for a specific operating system or distribution.

  • graphics - Name of the graphics device to allow for graphical interaction with the guest OS. Specify ‘none’ to indicate no such device.

  • imagesize - Specify the size of the image.

Note

Details for all of the above attributes apart from name and version with regards to valid values that can be specified for each of them, can be found in the official KVM documentation.

Devices

  • name - Name of the device.

  • type - PCI is the default type of device.

  • class - FPGA class.

  • vendor- Vendor name of the PCI card.

  • count - Number of devices to be allocated

Storage

Within each role a storage section must be specified. Below are the key attributes that need to be edited for storage space availability within the VM.

  • type - The value specified for this attribute indicates the volume type (e.g. data / commitlog / root_fs / etc.). There can be multiple volume types per role depending on the number of volumes however they all need to be unique within the role.

  • size - The size of volume created for the specified volume type within the role.

Note

Non root_fs partitions are created with a raw file system type and so will need to be formatted to the required file system type from within the VM.

Environment Variables

Within each role an env section can be used to specify environment variables to be exposed within the VM. Below are environment variables that Robin recommends be exposed for VM configuration.

  • CLOUD_INIT - This variable should be specified with its type set to boolean and the value as True for images that support the use of Cloud Init configurations.

  • CLOUD_INIT_ADMIN_USER - This variable should be set to the name of admin user that should be created as part of the VM deployment.

  • CLOUD_INIT_ADMIN_PASSWORD - This variable should be specified with its type set to password and the value being the password for the aforementioned admin user.

  • KVM_SKIP_NETWORKING - This variable should be specified with its type set to boolean and the value as False so that networking is automatically managed by Robin via the Cloud Init config. Set it to true if the networking is configured manually outside of the config specified.

Note

For more details on Cloud Init configurations and the possible values that can be specified, review the documentation here.

18.6. Create a Virtual Machine

After the respective VM images and associated KVM based application bundles are registered, a VM can be created using the robin app create from-bundle command. More details about this command can be found here. An example usage of the command is shown below.

Example

# robin app create from-bundle demo-vm 1 --rpool default --ip-pool public --wait
Job: 6365 Name: ApplicationCreate    State:VALIDATED    Error: 0
Job: 6365 Name: ApplicationCreate    State:PROCESSED    Error: 0
Job: 6365 Name: ApplicationCreate    State:WAITING      Error: 0
Job: 6365 Name: ApplicationCreate    State:COMPLETED    Error: 0

18.7. Access a Virtual Machine

To access a VM, the IP Address of the aforementioned machine needs to be determined first. In order to do so use the robin instance list command. More details about this command can be found here. An example usage of the command is shown below.

Example

# robin instance list
ID | Owner/Tenant         | ResPool | Container         | Hostname                 | IP Addresses   | Status | Autopilot | LastOpr | Host   | Engine | Cores | GPUs | Mem (GB) | Ctime
---+----------------------+---------+-------------------+--------------------------+----------------+--------+-----------+---------+--------+--------+-------+------+----------+----------------------
1  | robin/Administrators | default | demo-vm.server.01 | vnode74.robinsystems.com | 10.7.201.74/16 | ONLINE | Enabled   | STARTED | demo15 | kvm    | 8     | 0    | 8.0      | 26 Jan 2021 15:58:46

After the obtaining the IP Address for the relevant VM container, any supported remote access method, such as SSH, can be used to access the aforementioned machine alongside the credentials specified in the respective application bundle’s manifest file. An example is shown below.

Example

# ssh vmuser@10.7.201.74
vmuser@10.7.201.74's password:
Last login: Thu Jan 28 06:04:40 2021 from 10.10.100.116

18.8. Lifecycle Management of a Virtual Machine

18.8.1. Start a Virtual Machine

In order to start a previously stopped VM, use the robin app start command. More details about this command can be found here. An example usage of this command is shown below. If the application name is not known, the robin app list command can be used. Additional details for this command can be found here.

Note

If an application consists of multiple VM containers, and only individual containers need to be started use the respective robin instance start and robin instance list commands. More details for these commands can be found here.

Example

# robin app start demo-vm --wait
Job:  6589 Name: ApplicationStart          State: PROCESSED       Error: 0
Job:  6589 Name: ApplicationStart          State: AGENT_WAIT      Error: 0
Job:  6589 Name: ApplicationStart          State: COMPLETED       Error: 0

After a VM is started, it might be the case that the VM’s IP Address is not pingable even though the VM container is available and the IP Address can be used to gain access to the console. This can result in the above operation failing. One reason for this could be that the networking for the VM is customised directly in the manifest file for the associated application bundle. To avoid this issue enable networking using cloud-init in order to ensure the VM is always pingable. This can be achieved by modifying the VM image to be inline with cloud-init specifications.

Note

To disable ping checks entirely, set the attribute ping_check to have a value of false within the manifest file of the respective application bundle under the role level image section. This results in the platform not performing any ping checks on the VM’s IP Addresses as part of a health check during any lifecycle management operations.

18.8.2. Stop a Virtual Machine

In order to stop a running VM, use the robin app stop command. More details about this command can be found here. An example usage of this command is shown below. If the application name is not known, the robin app list command can be used. Additional details for this command can be found here.

Note

If an application consists of multiple VM containers, and only individual containers need to be stopped use the respective robin instance stop and robin instance list commands. More details for these commands can be found here.

Example

# robin app stop demo-vm --wait
Job:  6589 Name: ApplicationStop          State: PROCESSED       Error: 0
Job:  6589 Name: ApplicationStop          State: AGENT_WAIT      Error: 0
Job:  6589 Name: ApplicationStop          State: COMPLETED       Error: 0

18.8.3. Shutdown a Virtual Machine from Console

In order to shutdown a VM via the console, first access the console using the steps provided here. Next run the following command in the console:

# shutdown -h

When a VM is shutdown via console, the respective VM container is automatically restarted unless the Autopilot feature for the concerned application is disabled. More details on how to disable Autopilot via the robin ap disable command, can be found here.

18.8.4. Restart a Virtual Machine

In order to restart a VM, use the robin app restart command. More details about this command can be found here. An example usage of this command is shown below. If the application name is not known, the robin app list command can be used. Additional details for this command can be found here.

Note

If an application consists of multiple VM containers, and only individual containers need to be restarted use the respective robin instance restart and robin instance list commands. More details for these commands can be found here.

Example

# robin app restart demo-vm --wait
Job:  234 Name: ApplicationStart          State: VALIDATED       Error: 0
Job:  234 Name: ApplicationStart          State: PREPARED        Error: 0
Job:  234 Name: ApplicationStart          State: AGENT_WAIT      Error: 0
Job:  234 Name: ApplicationStart          State: COMPLETED       Error: 0

Note

When an VM container is restarted, the VM itself is rebooted.

18.8.5. Delete a Virtual Machine

In order to delete a VM, use the robin app delete command. More details about this command can be found here. An example usage of this command is shown below. If the application name is not known, the robin app list command can be used. Additional details for this command can be found here.

Note

If an application consists of multiple VM containers, and only individual containers need to be restarted use the respective robin instance delete and robin instance list commands however this is not recommended. More details for these commands can be found here.

Example

# robin app delete demo-vm --wait
Job:  238 Name: ApplicationDelete          State: VALIDATED       Error: 0
Job:  238 Name: ApplicationDelete          State: PREPARED        Error: 0
Job:  238 Name: ApplicationDelete          State: AGENT_WAIT      Error: 0
Job:  238 Name: ApplicationDelete          State: COMPLETED       Error: 0