*********************************************** Virtual Machine Support *********************************************** From version 5.3.3 and above, Robin CNP supports Linux Kernel-based Virtual Machines (KVM) and consequently allows for virtual machines to be deployed within clusters. All necessary KVM parameters can be specified within a Robin bundle and the respective ``qcow2`` images can also be uploaded to a File Collection within the cluster in order to spawn virtual machines with the desired customization. Since virtual machines are deployed via Robin Bundles, they are treated just like any other application on the platform with the full range of lifecycle management operations available for them. .. Note:: In order to create and manage virtual machines on CNP, knowledge of Robin Bundles and KVM are needed. More information on Robin Bundles can be found `here `__. In addition, below are some points that should be considered before deploying virtual machines: * OVS Networking for VMs from CNP version 5.3.3 onwards whereas Calico Networking is only supported on IPv4 clusters and from version 5.3.7 onwards. * All recovery workflows such as snapshots, clones, backups, and restorations of VMs are supported. * All virtual machines running an operating system that is compatible with ``qcow2`` and the underpinning KVM version (1.5.3) are supported. .. Note:: If your Robin cluster setup has a combination of physical and virtual machines, you must ensure that the KVM VM is deployed on a physical node of the cluster. You can use the Pod affinity policy or Pod placement policy and ensure the KVM VM is deployed on a physical node of the cluster. High-Level Steps to Create and Manage VMs ========================================== The following are the high-level steps for creating and managing VMs on Robin CNP.  1. Check prerequisites. 2. Verify CNP configuration. 3. Download required ``qcow2`` or ``ISO`` image from a reliable source. 4. Register the downloaded ``qcow2`` or ``ISO`` image with Robin CNP. 5. Customize the ``manifest.yaml`` file for creating KVM Robin Bundle. 6. Create a KVM Robin Bundle tar file. 7. Add the KVM Robin Bundle. 8. Create a VM using the KVM Bundle. 9. Lifecycle management of a virtual machine. Prerequisites ============== Software Prerequisites ----------------------- - The Robin cluster on which the VM is to be deployed is installed with Robin Cloud Native Platform v5.3.3 or above. In order to verify this, run the following command: .. code-block:: text # robin version Hardware Prerequisites ----------------------- - The following must be enabled in the BIOS settings of the bare metal server: - Intel Processors - VD-T - AMD Processors - SVM - BIOS VT-D (Intel) - SVM (AMD) In order to verify these settings have been appropriately set, run the following command: .. code-block:: text # virt-host-validate Image Prerequisites -------------------- - The VM image must be in ``qcow2`` or ``ISO`` format. - The VM image should have the ``cloud init`` service. If the image does have ``cloud init`` service, the networking for the VM must be configured manually. - The VM image must mount when booting. - The Robin file collection on which the VM image is to be hosted must be larger than the size of the compressed aforementioned image. In order to verify the size of the available file collection(s), run the following command: .. code-block:: text # robin collection list Robin Install Prerequisites ---------------------------- The following Robin installation pre-checks must pass: - KVM Virtualization - KVM vhost-net - KVM Networking - KVM PCI Device Passthrough (For SRIOV support) In order to verify these pre-checks have passed, run the following command: .. code-block:: text # ./robin-install.sh precheck Networking Prerequisites ------------------------- - The Robin cluster is configured with a public OVS based IP pool from which the VM IPs are assigned from. - VM IP addresses must be in the same L3 subnet as the hosts. For example this means if the host IP is 101.1.1.10/24 then the IP-Pool must also have a netmask of 24 bits. - VM IP addresses must be in a routable LAN from the hosts on which they are deployed. - The VM image must be configured with static IP addresses. Storage Prerequisites ---------------------- - The storage capacity of the nodes within the cluster must be greater than or equal to the planned size of the file system(s) of the VMs as defined in the manifest file for the respective Robin Bundle. - The specified capacity of the rootfs volume of the KVM based application to be deployed must be greater than the size of the associated VM image. Register a VM Image --------------------- In order to create a KVM application, the appropriate ``qcow`` or ``ISO`` VM image must first be uploaded to a file collection and registered with Robin. The image can either be present locally or available for download via a registry already known to Robin. An example command for registering a pre-downloaded image is shown below. More details on the ``robin image add`` command can be found `here `__. **Example for qcow image** .. code-block:: text # robin image add centos7 2009 QCOW ~/centos-kvm/CentOS-7-x86_64-GenericCloud-2009.qcow2 File upload: 100% -- (1391329280/1391329280) Job: 84 Name: ImageAdd State: VALIDATED Error: 0 Job: 84 Name: ImageAdd State: COMPLETED Error: 0 **Example for ISO image** .. code-block:: text # wget https://mirror.umd.edu/centos/7/isos/x86_64/CentOS-7-x86_64-DVD-2009.iso --2022-12-07 12:32:14-- https://mirror.umd.edu/centos/7/isos/x86_64/CentOS-7-x86_64-DVD-2009.iso Resolving mirror.umd.edu (mirror.umd.edu)... 128.8.207.141 Connecting to mirror.umd.edu (mirror.umd.edu)|128.8.207.141|:443... connected. HTTP request sent, awaiting response... 200 OK Length: 4712300544 (4.4G) [application/octet-stream] Saving to: ‘CentOS-7-x86_64-DVD-2009.iso’ CentOS-7-x86_64-DVD-2009.iso 100%[============================================================================================>] 4.39G 34.3MB/s in 2m 39s 2022-12-07 12:34:55 (28.3 MB/s) - ‘CentOS-7-x86_64-DVD-2009.iso’ saved [4712300544/4712300544] # robin image add centos7 2009 ISO CentOS-7-x86_64-DVD-2009.iso File uploaded: Calculating hash. This could take a while depending on image size Job: 87 Name: ImageAdd State: PROCESSED Error: 0 Job: 87 Name: ImageAdd State: COMPLETED Error: 0 # robin bundle add centos kvm http://builds.robinsystems.com:8080/job/robin-bundles/job/master/lastSuccessfulBuild/artifact/BUNDLES/kvm-centos-7-745_master.tar.gz --wait File uploaded: Calculating hash. This could take a while depending on image size Job: 88 Name: BundleAdd State: PROCESSED Error: 0 Job: 88 Name: BundleAdd State: COMPLETED Error: 0 Create a KVM Robin Bundle ========================== A KVM Robin Bundle is a tar file containing the following components: a manifest file, a directory for icons and a directory for scripts. The following section focuses on the manifest file as it is a mandatory file and is considered the blueprint for the application. The other components are optional and can be populated at the user's descretion. More details on the aforementioned components and how to customize them can be found `here `__. .. Note:: Given that a KVM bundles are treated similarly to other application bundles, the process of creating and registering them is the same. These steps are detailed `here `__ whilst the sections detailed below focus on customizing manifest file attributes which differentiate KVM based bundles from other application bundles. Sample Manifest file --------------------- The following is an example of a manifest file for a KVM application running the CentOS operating system. .. code-block:: yaml name: CentOS version: "7" icon: icon.png snapshot: enabled clone: enabled roles: [server] server: name: CentOS multinode: true scaleout: enabled scaledown_mem: enabled addvolume: enabled scaledown_mem: enabled compute: memory: 8G hugepages_1gi: 0G hugepages_2mi: 0G # hugepages_locked: "yes" # hugepages_noshared: "yes" # memballoon: "model=none,stats.period=10" cpu: reserve: true cores: 8 # rt_cores: 1-7 # rt_sched: "fifo" # rt_priority: 1 # emulatorpin: "outside_cpuset" # cache_ways: 4 # kvmopts: "mode=host-model,match=exact" # kvmopts: "mode=host-passthrough,require=tsc-deadline,check=none" devices: - name: fec type: pci class: "0x120000" vendor: "0x8086" count: 0 image: name: centos7 version: "2009" engine: kvm ostype: linux osvariant: rhel7 graphics: "none" imgsize: "8G" storage: - type: root_fs media: ssd bus: scsi path: / size: 20G - type: data media: ssd bus: scsi count: 1 path: /dev/data fstype: raw size: 10G - type: logs media: ssd bus: scsi count: 1 path: /dev/log fstype: raw size: 10G # livenessProbe: # tcpSocket: # port: 22 # initialDelaySeconds: 180 # periodSeconds: 10 env: # KVM specific env variables - CLOUD_INIT: type: boolean value: true - CLOUD_INIT_ADMIN_USER: robin - CLOUD_INIT_ADMIN_PASSWORD: type: password value: "robin123" - CLOUD_INIT_USERDATA: type: yaml value: | # cloud-config password: "{{ROLES.server.ENV.CLOUD_INIT_ADMIN_PASSWORD}}" chpasswd: list: | root: "{{ROLES.server.ENV.CLOUD_INIT_ADMIN_PASSWORD}}" expire: False - QEMU_GUEST_AGENT: qemu-guest-agent-2.12.0-3.el7.x86_64.rpm Customizing the Manifest file ------------------------------ The following are the Role level sections that need to be edited for VM deployment. More details on all customisable sections and/or attributes within the manifest file can be found `here `__. **Image** Within each role an ``image`` section must be specified. For VM deployment this section is associated to the KVM image that was uploaded to a file collection earlier. Below are the attributes that need to be customized for the respective image. - ``name`` - This name must match the image name supplied when the image was registered. - ``version`` - This version must match the image version supplied when the image was registered. - ``engine`` - Engine can be docker/kvm/lxc. Docker for docker containers, KVM for VM’s and LXC for Linux containers. - ``ostype`` - The guest configuration for a type of operating system (Example: Linux, Windows). This will attempt to pick the most suitable ACPI and APIC settings, optimally supported mouse drivers, Virtio, and generally accommodate other operating system quirks.. - ``osvariant`` - It is the kind of GuestOS. It is an optional specification. However, it provides some default parameters for each virtual machine that can help improve performance for a specific operating system or distribution. - ``graphics`` - Name of the graphics device to allow for graphical interaction with the guest OS. Specify 'none' to indicate no such device. - ``imagesize`` - Specify the size of the image. .. Note:: Details for all of the above attributes apart from ``name`` and ``version`` with regards to valid values that can be specified for each of them, can be found in the official KVM documentation. **Devices** - ``name`` - Name of the device. - ``type`` - PCI is the default type of device. - ``class`` - FPGA class. - ``vendor``- Vendor name of the PCI card. - ``count`` - Number of devices to be allocated **Storage** Within each role a ``storage`` section must be specified. Below are the key attributes that need to be edited for storage space availability within the VM. - ``type`` - The value specified for this attribute indicates the volume type (e.g. data / commitlog / root_fs / etc.). There can be multiple volume types per role depending on the number of volumes however they all need to be unique within the role. - ``size`` - The size of volume created for the specified volume type within the role. .. Note:: Non ``root_fs`` partitions are created with a ``raw`` file system type and so will need to be formatted to the required file system type from within the VM. **Environment Variables** Within each role an ``env`` section can be used to specify environment variables to be exposed within the VM. Below are environment variables that Robin recommends be exposed for VM configuration. - ``CLOUD_INIT`` - This variable should be specified with its type set to boolean and the value as True for images that support the use of Cloud Init configurations. - ``CLOUD_INIT_ADMIN_USER`` - This variable should be set to the name of admin user that should be created as part of the VM deployment. - ``CLOUD_INIT_ADMIN_PASSWORD`` - This variable should be specified with its type set to password and the value being the password for the aforementioned admin user. - ``KVM_SKIP_NETWORKING`` - This variable should be specified with its type set to boolean and the value as False so that networking is automatically managed by Robin via the Cloud Init config. Set it to true if the networking is configured manually outside of the config specified. .. Note:: For more details on Cloud Init configurations and the possible values that can be specified, review the documentation `here `__. Create a Virtual Machine ========================= After the respective VM images and associated KVM based application bundles are registered, a VM can be created using the ``robin app create from-bundle`` command. More details about this command can be found `here `__. An example usage of the command is shown below. **Example** .. code-block:: text # robin app create from-bundle demo-vm 1 --rpool default --ip-pool public --wait Job: 6365 Name: ApplicationCreate State:VALIDATED Error: 0 Job: 6365 Name: ApplicationCreate State:PROCESSED Error: 0 Job: 6365 Name: ApplicationCreate State:WAITING Error: 0 Job: 6365 Name: ApplicationCreate State:COMPLETED Error: 0 Create a VM Using Cisco DCNM E1000 Virtual Interface ----------------------------------------------------- Robin CNP supports Cisco Data Center Network Manager (DCNM) E1000 virtual network interface for KVMs. You can deploy the Cisco DCNM E1000 interface when creating a KVM on a Robin CNP cluster. .. Note:: The Cisco DCNM E1000 Virtual Interface is supported only on KVMs with OVS. You can deploy a KVM bundle to use an OVS IP Pool with the model as E1000 only using an ``input.yaml`` file. **Sample input.yaml file** .. code-block:: yaml :emphasize-lines: 5 roles: - name: server ippools: - ippool: ovs-3 model: e1000 - ippool: sec-3 primary: ovs-3 **Example** .. code-block:: text # robin app create from-bundle --rpool default --namespace z1 sec-pool 1 /root/e1000_input.yaml --wait Job: 86 Name: ApplicationCreate State: PROCESSED Error: 0 Job: 86 Name: ApplicationCreate State: WAITING Error: 0 Job: 86 Name: ApplicationCreate State: COMPLETED Error: 0 Create a VM with a custom UUID ------------------------------- Robin CNP provides you an option to provide a custom Universally Unique Identifier (UUID) manually for a KVM-based application via the ``input.yaml`` at runtime when creating the application. This feature enables the use of application licenses which are tied to the UUID of the VM. .. Note:: Only standard UUIDs can be specified and their uniqueness must be verified externally ensuring that it hasnt been used previously for another VM. **Sample input.yaml file** The following is the sample input.yaml file with the UUID: .. code-block:: yaml :emphasize-lines: 4 appname: "kvm-centos-8-1" roles: - name: server uuid: ["7dc29533-b055-4dd7-a9ef-5799db2c098a"] ippools: - ippool: ovs-mgmt-0 **Example** .. code-block:: text # robin app create from-bundle --rpool default --namespace default ovs 1 /root/uuid_input.yaml --wait Job: 86 Name: ApplicationCreate State: PROCESSED Error: 0 Job: 86 Name: ApplicationCreate State: WAITING Error: 0 Job: 86 Name: ApplicationCreate State: COMPLETED Error: 0 .. Note:: You can verify the provided UUID is present by running this command on the host: ``virsh dumpxml | grep uuid``. In addition, the following command can also be run within the VM to verify the UUID is properly stamped: ``sudo dmidecode -t system | grep UUID`` Create a VM with static MAC address ------------------------------------- Robin CNP provides you an option to specify a static MAC address for a KVM-based application through the ``input.yaml`` at runtime when creating the application. This feature enables you to use the VM licenses, which are tied to the MAC addresses of the VMs. For more information, see `Static MAC address `__. The following key is used to specify the static MAC address for the KVM-based application in the ``input.yaml``: - static_macs .. Note:: For KVM-based applications, only MACVLAN, OVS, and SR-IOV with VFIO-PCI drivers are supported. **Sample input.yaml file** .. code-block:: yaml :emphasize-lines: 6,9 roles: - name: server multinode_value: 2 ippools: - ippool: bond-1 static_macs: ["fa:16:3e:11:12:11", "fa:16:3e:11:12:12", "fa:16:3e:11:12:13", "fa:16:3e:11:12:14"] static_ips: ["fd74:ca9b:3a09:8829:0010:0009:0001:0001", "fd74:ca9b:3a09:8829:0010:0009:0001:0002"] - ippool: bond-2 static_macs: ["fa:16:3e:11:13:11", "fa:16:3e:11:13:12", "fa:16:3e:11:13:13", "fa:16:3e:11:13:14"] static_ips: ["fd74:ca9b:3a09:8830:0010:0009:0001:0001", "fd74:ca9b:3a09:8830:0010:0009:0001:0002"] **Example** .. code-block:: text # robin app create from-bundle --rpool default --namespace default ovs 2 /root/staticmac_input.yaml --wait Job: 86 Name: ApplicationCreate State: PROCESSED Error: 0 Job: 86 Name: ApplicationCreate State: WAITING Error: 0 Job: 86 Name: ApplicationCreate State: COMPLETED Error: 0 .. Note:: To specify the static MAC address for KVM-based applications, you must pass the **IP-Pool** name in the ``input.yaml``, not in the ``--ip-pools`` option of the ``robin app create`` command. Access a Virtual Machine ========================== After creating a VM, you can access it. Access a Linux Virtual Machine ------------------------------- To access a VM, the IP Address of the aforementioned machine needs to be determined first. In order to do so use the ``robin instance list`` command. More details about this command can be found `here `__. An example usage of the command is shown below. **Example** .. code-block:: text # robin instance list ID | Owner/Tenant | ResPool | Container | Hostname | IP Addresses | Status | Autopilot | LastOpr | Host | Engine | Cores | GPUs | Mem (GB) | Ctime ---+----------------------+---------+-------------------+--------------------------+----------------+--------+-----------+---------+--------+--------+-------+------+----------+---------------------- 1 | robin/Administrators | default | demo-vm.server.01 | vnode74.robinsystems.com | 10.7.201.74/16 | ONLINE | Enabled | STARTED | demo15 | kvm | 8 | 0 | 8.0 | 26 Jan 2021 15:58:46 After the obtaining the IP Address for the relevant VM container, any supported remote access method, such as SSH, can be used to access the aforementioned machine alongside the credentials specified in the respective application bundle's manifest file. An example is shown below. .. Note:: Tenant admins and tenant users can access the KVM application console from the Robin UI. **Example** .. code-block:: text # ssh vmuser@10.7.201.74 vmuser@10.7.201.74's password: Last login: Thu Jan 28 06:04:40 2021 from 10.10.100.116 Access a Windows Virtual Machine using VDI -------------------------------------------- Robin CNP enables you to access a Windows VM using the VDI (Virtual desktop infrastructure). For VDI, Robin CNP uses Guacamole application to access GUI of a Windows VM. **Prerequisite** * Set up VDI and install Guacamole application. **Set up VDI** You must set up VDI (virtual desktop infrastructure) to access a Windows VM using GUI. When you run the ``robin-script.sh setup-vdi`` script to setup VDI, Robin CNP installs Guacamole application. .. Note:: This VDI setup is a one-time process per cluster. Once you setup VDI, you can access Windows VMs created before or after setting up VDI. Guacamole is an HTML5 web application that provides access to desktop environments using remote desktop protocols such as virtual network computing (VNC) and remote desktop protocol (RDP). For more information, see `Apache Guacamole `__. When you execute the ``robin-script.sh setup-vdi`` script, Robin CNP deploys installs Guacamole application during VDI setup. After deploying installing the Guacamole application, Robin CNP makes the VNC and RDP connections automatically with Guacamole application to access the Windows VM. Accessing a VM is managed through Role-Based Access Control (RBAC). Fore more information about how RBAC works in Robin CNP, see `Robin user management `__. Complete the following steps to setup VDI: 1. Extract the gorobintar file: .. code-block:: text # tar -xvf If the gorobintar file is already extracted, follow the step 2. 2. Change your current working directory to the directory where you have extracted the GoRobin tar file in step 1: .. code-block:: text # cd 3. To setup VDI, run the ``robin-script.sh setup-vdi`` script: .. code-block:: text # ./robin-script.sh setup-vdi --vdi-tenant-admin= --vdi-tenant-password= --vdi-guacadmin-password= --vdi-tenant-namespace= --vdi-tenant= --vdi-branding-title= **Example** .. code-block:: text # ./robin-script.sh setup-vdi --vdi-tenant-admin=admin --vdi-tenant-password=Robin123 --vdi-guacadmin-password=guacadmin --vdi-tenant-namespace=vdi-administrators --vdi-tenant=Administrators --vdi-branding-title=VM-PORTAL Robin CNP Setup-vdi Extracting Payload : DONE Setting up VDI Namespace vdi-administrators : DONE Setting up VDI Server in Administrators : DONE Setting up VDI Broker in Administrators : DONE Updating ROBIN UI for Administrators : DONE Setting up VDI Admin User in Administrators : DONE Successfully setup VDI in Administrators Complete the following steps to access a Windows VM using VDI: 1. Log in to Robin CNP UI using your username and password. 2. On the **dashboard** page, on the left hand side, click **Applications** > **Applications**. 3. On the **Applications** page, select the Windows VM you want to access. Optionally, you can also search the Windows VM using the search box. 4. On the **INFO** page, select dropdown icon next to the Windows VM name, and then click **RDP Console**. A new window appears with the login UI. Log in to the UI using the username and password of VDI tenant created during VDI setup. Lifecycle Management of a Virtual Machine =========================================== Start a Virtual Machine ------------------------- In order to start a previously stopped VM, use the ``robin app start`` command. More details about this command can be found `here `__. An example usage of this command is shown below. If the application name is not known, the ``robin app list`` command can be used. Additional details for this command can be found `here `__. .. Note:: If an application consists of multiple VM containers, and only individual containers need to be started use the respective ``robin instance start`` and ``robin instance list`` commands. More details for these commands can be found `here `__. **Example** .. code-block:: text # robin app start demo-vm --wait Job: 6589 Name: ApplicationStart State: PROCESSED Error: 0 Job: 6589 Name: ApplicationStart State: AGENT_WAIT Error: 0 Job: 6589 Name: ApplicationStart State: COMPLETED Error: 0 After a VM is started, it might be the case that the VM's IP Address is not pingable even though the VM container is available and the IP Address can be used to gain access to the console. This can result in the above operation failing. One reason for this could be that the networking for the VM is customised directly in the manifest file for the associated application bundle. To avoid this issue enable networking using ``cloud-init`` in order to ensure the VM is always pingable. This can be achieved by modifying the VM image to be inline with ``cloud-init`` specifications. .. Note:: To disable ping checks entirely, set the attribute ``ping_check`` to have a value of false within the manifest file of the respective application bundle under the role level ``image`` section. This results in the platform not performing any ping checks on the VM's IP Addresses as part of a health check during any lifecycle management operations. Stop a Virtual Machine ----------------------- In order to stop a running VM, use the ``robin app stop`` command. More details about this command can be found `here `__. An example usage of this command is shown below. If the application name is not known, the ``robin app list`` command can be used. Additional details for this command can be found `here `__. .. Note:: If an application consists of multiple VM containers, and only individual containers need to be stopped use the respective ``robin instance stop`` and ``robin instance list`` commands. More details for these commands can be found `here `__. **Example** .. code-block:: text # robin app stop demo-vm --wait Job: 6589 Name: ApplicationStop State: PROCESSED Error: 0 Job: 6589 Name: ApplicationStop State: AGENT_WAIT Error: 0 Job: 6589 Name: ApplicationStop State: COMPLETED Error: 0 Shutdown a Virtual Machine from Console ---------------------------------------- In order to shutdown a VM via the console, first access the console using the steps provided here. Next run the following command in the console: .. code-block:: text # shutdown -h When a VM is shutdown via console, the respective VM container is automatically restarted unless the Autopilot feature for the concerned application is disabled. More details on how to disable Autopilot via the ``robin ap disable`` command, can be found `here `__. Restart a Virtual Machine --------------------------- In order to restart a VM, use the ``robin app restart`` command. More details about this command can be found `here `__. An example usage of this command is shown below. If the application name is not known, the ``robin app list`` command can be used. Additional details for this command can be found `here `__. .. Note:: If an application consists of multiple VM containers, and only individual containers need to be restarted use the respective ``robin instance restart`` and ``robin instance list`` commands. More details for these commands can be found `here `__. **Example** .. code-block:: text # robin app restart demo-vm --wait Job: 234 Name: ApplicationStart State: VALIDATED Error: 0 Job: 234 Name: ApplicationStart State: PREPARED Error: 0 Job: 234 Name: ApplicationStart State: AGENT_WAIT Error: 0 Job: 234 Name: ApplicationStart State: COMPLETED Error: 0 .. Note:: When an VM container is restarted, the VM itself is rebooted. Delete a Virtual Machine ------------------------- In order to delete a VM, use the ``robin app delete`` command. More details about this command can be found `here `__. An example usage of this command is shown below. If the application name is not known, the ``robin app list`` command can be used. Additional details for this command can be found `here `__. .. Note:: If an application consists of multiple VM containers, and only individual containers need to be restarted use the respective ``robin instance delete`` and ``robin instance list`` commands however this is **not recommended.** More details for these commands can be found `here `__. **Example** .. code-block:: text # robin app delete demo-vm --wait Job: 238 Name: ApplicationDelete State: VALIDATED Error: 0 Job: 238 Name: ApplicationDelete State: PREPARED Error: 0 Job: 238 Name: ApplicationDelete State: AGENT_WAIT Error: 0 Job: 238 Name: ApplicationDelete State: COMPLETED Error: 0 View Virtual Machine Images ---------------------------- In order to view a list of registered VM images, run the ``robin image list`` command. More details about this command can be found `here `__. An example usage of the command is shown below. **Example** .. code-block:: text # robin image list Image Id | Owner/Tenant | Name | Version | Format | Content Id | File Object Id | File Name ---------+----------------------+--------+---------+--------+----------------------------------+----------------+----------------------------------------- 1 | robin/Administrators | centos | 2003 | QCOW | ef7f109590094e37c54aead73f3cdedc | 1611689015480 | CentOS-7-x86_64-GenericCloud-2003.qcow2 2 | robin/Administrators | centos | 1907 | QCOW | 160aa274e7a69f2edb50e2d89f54270b | 1611689714495 | CentOS-7-x86_64-GenericCloud-1907.qcow2 Delete Virtual Machine Bundle ------------------------------- In order to delete a VM bundle, use the ``robin bundle delete`` command as a VM bundle is treated the same to any other bundle. More details about this command can be found `here `__. An example usage of this command is shown below. If the necessary parameters, namely bundle ID and zone ID, are not known the ``robin bundle list`` command can be used. Additional details for this command can be found `here `__. .. Note:: Before a VM bundle can be deleted, all applications consisting of VM's deployed from the respective bundle must be deleted via the ``robin app delete`` command. Details of this command can be found `here `__. **Example** .. code-block:: text # robin bundle remove 1599768542 2 --wait Are you sure you want to delete [y/n] ? y Job: 437 Name: BundleRemove State: VALIDATED Error: 0 Job: 437 Name: BundleRemove State: PREPARED Error: 0 Job: 437 Name: BundleRemove State: FINALIZED Error: 0 Job: 437 Name: BundleRemove State: COMPLETED Error: 0 Delete Virtual Machine Images ------------------------------- In order to delete a VM image, use the ``robin image delete`` command. More details about this command can be found `here `__. An example usage of this command is shown below. If the necessary parameter, namely the image ID, is not known the ``robin image list`` command can be used. Additional details for this command can be found `here `__. .. Note:: Before a VM image can be deleted, all bundles (and consequently all applications) that refer to the respective image must be deleted via the ``robin bundle delete`` command. Details of this command can be found `here `__. **Example** .. code-block:: text # robin image delete 1 Are you sure you want to delete [y/n] ? y Job: 897 Name: ImageRemove State: VALIDATED Error: 0 Job: 897 Name: ImageRemove State: COMPLETED Error: 0