************************************** Virtual Machines Support On Robin CNP ************************************** Robin CNP supports running Linux Kernel-based Virtual Machines (KVM) on your clusters. It supports virtual machines (VMs) using the Linux KVM Application. Robin CNP exposes the KVM parameters as Robin bundles for creating VMs on CNP. If your cluster is running Robin CNP version 5.3.3 and above, you can run VMs on your Robin clusters. You can register any ``qcow2`` format KVM image with a File Collection on Robin CNP clusters and create a Robin bundle using the registered KVM image. For more information about Robin Bundles, see Using the created Robin bundle, you can create VMs and manage them using Robin CNP. You can deploy virtual machines just like applications using Robin CNP. .. Note:: To create VMs and manage them on Robin CNP, you should have knowledge about Robin Bundles and KVM. For more information about Robin Bundles, see `here `_. Using Robin CNP, you can perform the following VM related operations: - Deploy a VM (Using robin app create command) - Start a VM - Restart a VM - Stop a VM - Relocate a VM - Delete a VM Points to Consider for using KVM on Robin ========================================== The following are the important points you should consider when using VMs on Robin CNP: * Supported Networking for VMs: - OVS Networking - Version 5.3.3 onwards - Calico Networking (IPv4 only) - Version 5.3.7 onwards * You can take snapshots, clone, backup, and restore of the VMs in CNP. * A virtual machine can run any Operating System that is compatible with ``qcow2`` and the underpinning KVM version. High-Level Steps to Create and Manage VMs ========================================= The following are the high-level steps for creating and managing VMs on Robin CNP.  1. Check prerequisites. 2. Verify CNP configuration. 3. Download required ``qcow2`` image from a reliable source. 4. Register the downloaded ``qcow2`` image with Robin CNP. 5. Customize the ``manifest.yaml`` file for creating KVM Robin Bundle. 6. Create a KVM Robin Bundle tar file. 7. Add the KVM Robin Bundle. 8. Create a VM using the KVM Bundle. 9. Lifecycle management of a virtual machine. Prerequisites ============== Software Prerequisites ^^^^^^^^^^^^^^^^^^^^^^ - The Robin cluster on which the VM is to be deployed is installed with Robin Cloud Native Platform v5.3.3 or above. In order to verify this, run the following command: .. code-block:: text # robin version Hardware Prerequisites ^^^^^^^^^^^^^^^^^^^^^^ - The following must be enabled in the BIOS settings of the bare metal server: - Intel Processors - VD-T - AMD Processors - SVM - BIOS VT-D (Intel) - SVM (AMD) In order to verify these settings have been appropriately set, run the following command: .. code-block:: text # virt-host-validate Image Prerequisites ^^^^^^^^^^^^^^^^^^^ - The VM image must be in ``qcow2`` format. - The VM image should have the ``cloud init`` service. If the image does have ``cloud init`` service, the networking for the VM must be configured manually. - The VM image must mount when booting. - The Robin file collection on which the VM image is to be hosted must be larger than the size of the compressed aforementioned image. In order to verify the size of the available file collection(s), run the following command: .. code-block:: text # robin collection list Robin Install Prerequisites ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ The following Robin installation pre-checks must pass: - KVM Virtualization - KVM vhost-net - KVM Networking - KVM PCI Device Passthrough (For SRIOV support) In order to verify these pre-checks have passed, run the following command: .. code-block:: text # ./robin-install.sh precheck Networking Prerequisites ^^^^^^^^^^^^^^^^^^^^^^^^^ - The Robin cluster is configured with a public OVS based IP pool from which the VM IPs are assigned from. - VM IP addresses must be in the same L3 subnet as the hosts. For example this means if the host IP is 101.1.1.10/24 then the IP-Pool must also have a netmask of 24 bits. - VM IP addresses must be in a routable LAN from the hosts on which they are deployed. - The VM image must be configured with static IP addresses. Storage Prerequisites ^^^^^^^^^^^^^^^^^^^^^ - The storage capacity of the nodes within the cluster must be greater than or equal to the planned size of the file system(s) of the VMs as defined in the manifest file for the respective Robin Bundle. - The specified capacity of the rootfs volume of the KVM based application to be deployed must be greater than the size of the associated VM image. Register a VM Image ===================== In order to create a KVM application, the appropriate ``qcow`` VM image must first be uploaded to a file collection and registered with Robin. The image can either be present locally or available for download via a registry already known to Robin. An example command for registering a pre-downloaded image is shown below. More details on the ``robin image add`` command can be found `here `_. **Example** .. code-block:: text # robin image add centos7 2009 QCOW ~/centos-kvm/CentOS-7-x86_64-GenericCloud-2009.qcow2 File upload: 100% -- (1391329280/1391329280) Job: 84 Name: ImageAdd State: VALIDATED Error: 0 Job: 84 Name: ImageAdd State: COMPLETED Error: 0 Create a KVM Robin Bundle ========================== A KVM Robin Bundle is a tar file containing the following components: a manifest file, a directory for icons and a directory for scripts. The following section focuses on the manifest file as it is a mandatory file and is considered the blueprint for the application. The other components are optional and can be populated at the user's descretion. More details on the aforementioned components and how to customize them can be found `here `_. .. Note:: Given that a KVM bundles are treated similarly to other application bundles, the process of creating and registering them is the same. These steps are detailed `here `_ whilst the sections detailed below focus on customizing manifest file attributes which differentiate KVM based bundles from other application bundles. Sample Manifest file ^^^^^^^^^^^^^^^^^^^^^ The following is an example of a manifest file for a KVM application running the CentOS operating system. .. code-block:: yaml name: CentOS version: "7" icon: icon.png snapshot: enabled clone: enabled roles: [server] server: name: CentOS multinode: true scaleout: enabled scaledown_mem: enabled addvolume: enabled scaledown_mem: enabled compute: memory: 8G hugepages_1gi: 0G hugepages_2mi: 0G # hugepages_locked: "yes" # hugepages_noshared: "yes" # memballoon: "model=none,stats.period=10" cpu: reserve: true cores: 8 # rt_cores: 1-7 # rt_sched: "fifo" # rt_priority: 1 # emulatorpin: "outside_cpuset" # cache_ways: 4 # kvmopts: "mode=host-model,match=exact" # kvmopts: "mode=host-passthrough,require=tsc-deadline,check=none" devices: - name: fec type: pci class: "0x120000" vendor: "0x8086" count: 0 image: name: centos7 version: "2009" engine: kvm ostype: linux osvariant: rhel7 graphics: "none" imgsize: "8G" storage: - type: root_fs media: ssd bus: scsi path: / size: 20G - type: data media: ssd bus: scsi count: 1 path: /dev/data fstype: raw size: 10G - type: logs media: ssd bus: scsi count: 1 path: /dev/log fstype: raw size: 10G # livenessProbe: # tcpSocket: # port: 22 # initialDelaySeconds: 180 # periodSeconds: 10 env: # KVM specific env variables - CLOUD_INIT: type: boolean value: true - CLOUD_INIT_ADMIN_USER: robin - CLOUD_INIT_ADMIN_PASSWORD: type: password value: "robin123" - CLOUD_INIT_USERDATA: type: yaml value: | # cloud-config password: "{{ROLES.server.ENV.CLOUD_INIT_ADMIN_PASSWORD}}" chpasswd: list: | root: "{{ROLES.server.ENV.CLOUD_INIT_ADMIN_PASSWORD}}" expire: False - QEMU_GUEST_AGENT: qemu-guest-agent-2.12.0-3.el7.x86_64.rpm Customizing the Manifest file ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ The following are the Role level sections that need to be edited for VM deployment. More details on all customisable sections and/or attributes within the manifest file can be found `here `_. **Image** Within each role an ``image`` section must be specified. For VM deployment this section is associated to the KVM image that was uploaded to a file collection earlier. Below are the attributes that need to be customized for the respective image. - ``name`` - This name must match the image name supplied when the image was registered. - ``version`` - This version must match the image version supplied when the image was registered. - ``engine`` - Engine can be docker/kvm/lxc. Docker for docker containers, KVM for VM’s and LXC for Linux containers. - ``ostype`` - The guest configuration for a type of operating system (Example: Linux, Windows). This will attempt to pick the most suitable ACPI and APIC settings, optimally supported mouse drivers, Virtio, and generally accommodate other operating system quirks.. - ``osvariant`` - It is the kind of GuestOS. It is an optional specification. However, it provides some default parameters for each virtual machine that can help improve performance for a specific operating system or distribution. - ``graphics`` - Name of the graphics device to allow for graphical interaction with the guest OS. Specify 'none' to indicate no such device. - ``imagesize`` - Specify the size of the image. .. Note:: Details for all of the above attributes apart from ``name`` and ``version`` with regards to valid values that can be specified for each of them, can be found in the official KVM documentation. **Devices** - ``name`` - Name of the device. - ``type`` - PCI is the default type of device. - ``class`` - FPGA class. - ``vendor``- Vendor name of the PCI card. - ``count`` - Number of devices to be allocated **Storage** Within each role a ``storage`` section must be specified. Below are the key attributes that need to be edited for storage space availability within the VM. - ``type`` - The value specified for this attribute indicates the volume type (e.g. data / commitlog / root_fs / etc.). There can be multiple volume types per role depending on the number of volumes however they all need to be unique within the role. - ``size`` - The size of volume created for the specified volume type within the role. .. Note:: Non ``root_fs`` partitions are created with a ``raw`` file system type and so will need to be formatted to the required file system type from within the VM. **Environment Variables** Within each role an ``env`` section can be used to specify environment variables to be exposed within the VM. Below are environment variables that Robin recommends be exposed for VM configuration. - ``CLOUD_INIT`` - This variable should be specified with its type set to boolean and the value as True for images that support the use of Cloud Init configurations. - ``CLOUD_INIT_ADMIN_USER`` - This variable should be set to the name of admin user that should be created as part of the VM deployment. - ``CLOUD_INIT_ADMIN_PASSWORD`` - This variable should be specified with its type set to password and the value being the password for the aforementioned admin user. - ``KVM_SKIP_NETWORKING`` - This variable should be specified with its type set to boolean and the value as False so that networking is automatically managed by Robin via the Cloud Init config. Set it to true if the networking is configured manually outside of the config specified. .. Note:: For more details on Cloud Init configurations and the possible values that can be specified, review the documentation `here `_. Create a Virtual Machine ========================= After the respective VM images and associated KVM based application bundles are registered, a VM can be created using the ``robin app create from-bundle`` command. More details about this command can be found `here `_. An example usage of the command is shown below. **Example** .. code-block:: text # robin app create from-bundle demo-vm 1 --rpool default --ip-pool public --wait Job: 6365 Name: ApplicationCreate State:VALIDATED Error: 0 Job: 6365 Name: ApplicationCreate State:PROCESSED Error: 0 Job: 6365 Name: ApplicationCreate State:WAITING Error: 0 Job: 6365 Name: ApplicationCreate State:COMPLETED Error: 0 Access a Virtual Machine =========================== To access a VM, the IP Address of the aforementioned machine needs to be determined first. In order to do so use the ``robin instance list`` command. More details about this command can be found `here `_. An example usage of the command is shown below. **Example** .. code-block:: text # robin instance list ID | Owner/Tenant | ResPool | Container | Hostname | IP Addresses | Status | Autopilot | LastOpr | Host | Engine | Cores | GPUs | Mem (GB) | Ctime ---+----------------------+---------+-------------------+--------------------------+----------------+--------+-----------+---------+--------+--------+-------+------+----------+---------------------- 1 | robin/Administrators | default | demo-vm.server.01 | vnode74.robinsystems.com | 10.7.201.74/16 | ONLINE | Enabled | STARTED | demo15 | kvm | 8 | 0 | 8.0 | 26 Jan 2021 15:58:46 After the obtaining the IP Address for the relevant VM container, any supported remote access method, such as SSH, can be used to access the aforementioned machine alongside the credentials specified in the respective application bundle's manifest file. An example is shown below. **Example** .. code-block:: text # ssh vmuser@10.7.201.74 vmuser@10.7.201.74's password: Last login: Thu Jan 28 06:04:40 2021 from 10.10.100.116 Lifecycle Management of a Virtual Machine ========================================== Start a Virtual Machine ^^^^^^^^^^^^^^^^^^^^^^^^ In order to start a previously stopped VM, use the ``robin app start`` command. More details about this command can be found `here `_. An example usage of this command is shown below. If the application name is not known, the ``robin app list`` command can be used. Additional details for this command can be found `here `_. .. Note:: If an application consists of multiple VM containers, and only individual containers need to be started use the respective ``robin instance start`` and ``robin instance list`` commands. More details for these commands can be found `here `_. **Example** .. code-block:: text # robin app start demo-vm --wait Job: 6589 Name: ApplicationStart State: PROCESSED Error: 0 Job: 6589 Name: ApplicationStart State: AGENT_WAIT Error: 0 Job: 6589 Name: ApplicationStart State: COMPLETED Error: 0 After a VM is started, it might be the case that the VM's IP Address is not pingable even though the VM container is available and the IP Address can be used to gain access to the console. This can result in the above operation failing. One reason for this could be that the networking for the VM is customised directly in the manifest file for the associated application bundle. To avoid this issue enable networking using ``cloud-init`` in order to ensure the VM is always pingable. This can be achieved by modifying the VM image to be inline with ``cloud-init`` specifications. .. Note:: To disable ping checks entirely, set the attribute ``ping_check`` to have a value of false within the manifest file of the respective application bundle under the role level ``image`` section. This results in the platform not performing any ping checks on the VM's IP Addresses as part of a health check during any lifecycle management operations. Stop a Virtual Machine ^^^^^^^^^^^^^^^^^^^^^^^ In order to stop a running VM, use the ``robin app stop`` command. More details about this command can be found `here `_. An example usage of this command is shown below. If the application name is not known, the ``robin app list`` command can be used. Additional details for this command can be found `here `_. .. Note:: If an application consists of multiple VM containers, and only individual containers need to be stopped use the respective ``robin instance stop`` and ``robin instance list`` commands. More details for these commands can be found `here `_. **Example** .. code-block:: text # robin app stop demo-vm --wait Job: 6589 Name: ApplicationStop State: PROCESSED Error: 0 Job: 6589 Name: ApplicationStop State: AGENT_WAIT Error: 0 Job: 6589 Name: ApplicationStop State: COMPLETED Error: 0 Shutdown a Virtual Machine from Console ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ In order to shutdown a VM via the console, first access the console using the steps provided here. Next run the following command in the console: .. code-block:: text # shutdown -h When a VM is shutdown via console, the respective VM container is automatically restarted unless the Autopilot feature for the concerned application is disabled. More details on how to disable Autopilot via the ``robin ap disable`` command, can be found `here `_. Restart a Virtual Machine ^^^^^^^^^^^^^^^^^^^^^^^^^^ In order to restart a VM, use the ``robin app restart`` command. More details about this command can be found `here `_. An example usage of this command is shown below. If the application name is not known, the ``robin app list`` command can be used. Additional details for this command can be found `here `_. .. Note:: If an application consists of multiple VM containers, and only individual containers need to be restarted use the respective ``robin instance restart`` and ``robin instance list`` commands. More details for these commands can be found `here `_. **Example** .. code-block:: text # robin app restart demo-vm --wait Job: 234 Name: ApplicationStart State: VALIDATED Error: 0 Job: 234 Name: ApplicationStart State: PREPARED Error: 0 Job: 234 Name: ApplicationStart State: AGENT_WAIT Error: 0 Job: 234 Name: ApplicationStart State: COMPLETED Error: 0 .. Note:: When an VM container is restarted, the VM itself is rebooted. Delete a Virtual Machine ^^^^^^^^^^^^^^^^^^^^^^^^^ In order to delete a VM, use the ``robin app delete`` command. More details about this command can be found `here `_. An example usage of this command is shown below. If the application name is not known, the ``robin app list`` command can be used. Additional details for this command can be found `here `_. .. Note:: If an application consists of multiple VM containers, and only individual containers need to be restarted use the respective ``robin instance delete`` and ``robin instance list`` commands however this is **not recommended.** More details for these commands can be found `here `_. **Example** .. code-block:: text # robin app delete demo-vm --wait Job: 238 Name: ApplicationDelete State: VALIDATED Error: 0 Job: 238 Name: ApplicationDelete State: PREPARED Error: 0 Job: 238 Name: ApplicationDelete State: AGENT_WAIT Error: 0 Job: 238 Name: ApplicationDelete State: COMPLETED Error: 0