************************************************************************************************* Upgradeable Bundles ************************************************************************************************* Robin natively supports the upgrade of bundle based applications and depending on the application, can perform the update in a rolling or a simple collective manner. In order to ensure application resiliency in case of an upgrade failure, Robin takes a snapshot of the application before the process begins which consequently allows the user to rollback their application to a healthy state if update procedure goes awry. Moreover Robin provides the following upgrade related functionalities: • An "upgrade available" button on the application page within the Robin UI if a bundle update is available to inform the user. • Ability to select a particular target bundle if multiple version updates are available. • Ability to perform a rolling upgrade or just test the upgrade without impacting the application. • Ability to track upgrade progress. • Ability to pause/resume rolling upgrades. ---------------------------------------------------------------------------------------------- Manifest Attributes ---------------------------------------------------------------------------------------------- Detailed below are the attributes that need to be added to a new bundle such that its predeccesors can be upgraded to it. Not all of the options are needed, but it is recommended to go through each one and determine whether or not it should be used as their usage depends very heavily on the application/workload that is to be upgraded. ====================== ======================================== Attribute Description ====================== ======================================== upgrade_from List of versions from which an upgrade is supported (can be regular expressions). Note this attribute is mandatory for upgradeable bundles. rolling_upgrade Boolean value indicating if rolling upgrades are supported for the role. Default is false if it is not specified. upgrade_order The order in which the roles should be upgraded. preupgrade hook Hook script which will be run before upgrade starts. Can be set to run at an application or Vnode level. postupgrade hook Hook script which will be run after the upgrade completes. Can be set to run at an application or Vnode level. upgradeorder hook Hook script which will determine the order in which instances, of a particular role, will be upgraded. Can be set to run at an application level. ====================== ======================================== ---------------------------------------------------------------------------------------------- Technical Considerations ---------------------------------------------------------------------------------------------- In order to achieve the upgrade of a docker container, the following steps need to be performed: - Stop the docker container - Unmount the respective volumes of the container - Deploy a new docker container using a modified docker image - Mount the aformentioned volumes at the right paths within the new container - Start the container Given this procedure, there are several considerations that need to be accounted for when building an upgradeable bundle. These are detailed below alongside the solution Robin provides to tackle the challenge. **Scenario**: A container is utilizing a ``rootfs`` volume wherein which it is maintaining a configuration file. The upgrade for this container will depend on how the Docker entrypoint script is written and how the container utilizes the configuarition file. The various cases are detailed below: *The Docker entrypoint is generating the config file dynamically.* In this case, since the docker entrypoint is creating the config file on the fly there is no issue with the upgrade process. This is because the modified image to be used for the new container will include any neccesary changes to the entrypoint and thus will generate a config file that is compatible with the new image. *The config file is part of the image and updated later by the user/application.* In this case, after the upgrade completes, since the old ``rootfs`` volume will be mounted at the same path within th econtainer, the outdated config file will be present instead of the one packaged within the modified image. This may result in some issues as the original config file may not be compatible with the upgraded image. To solve this issue, Robin allows for ``preupgrade`` vnodehook script to be run before the procedure such that the config file is updated to be compatible with the new image. *The config file is part of the image and updated later by the vnode/application hooks.* In this case, since the hook scripts are updating the configuration file there is no issue with the upgrade process. This is because the hook scripts within the modified image will ensure the configuration file via validation and/or modification that the file is compatible with the newer image. ---------------------------------------------------------------------------------------------- Example ---------------------------------------------------------------------------------------------- Detailed below is an example of a manifest file for an ElasticSearch application that can be upgraded from any 5.x version or 6.6.1 to 6.6.2. Note the highlighted upgrade specific tags for reference. .. code-block:: yaml name: elasticsearch version: 6.6.2 description: elasticsearch icon: elastic.png roles: [master_node, data_node] snapshot: "enabled" serialize: true clone: "enabled" clonemode: unfenced upgrade_order: [master_node, data_node] # Upgrade specific attribute master_node: name: "Master Only Node" description: "Master Only Node" serialize: true multinode: true candisable: true scaleout: "enabled" rolling_upgrade: true # Upgrade specific attribute can_replace_storage: true image: name: elasticsearch version: "6.6.2" engine: docker entrypoint: docker-entrypoint.sh upgrade_from: [5*, 6.6.1] # Upgrade specific attribute compute: memory: 12288M cpu: reserve: false cores: 4 storage: - type: data1 media: hdd path: /usr/share/elasticsearch/data size: 200G env: MASTER_NODE: type: boolean value: true label: "Master Node" DATA_NODE: type: boolean value: false label: "Data Node" INGEST_NODE: type: boolean value: false label: "Ingest Node" ES_JAVA_OPTS: "-Xms6g -Xmx6g" CLUSTERNAME: "{{APP_NAME}}" IP_ADDRESS: "{{IP_ADDRESS}}" UNICAST_HOST: "{% for r in app['roles'] if ( 'master_node' in r['name']) %}{% for v in r['vnodes'] %}{{v['network'][0]['allocated_ip']}}{% if not loop.last %},{% endif %}{% endfor %}{% if not loop.last %},{% endif %}{% endfor %}" DATA_DIRS: "{% for v in vnode['storage'] if 'data' in v['type'] %}{{v['path']}}{% if not loop.last %} {% endif %}{% endfor %}" vnodehooks: preupgrade: "python preupgrade.py" # Upgrade specific attribute postupgrade: "python postupgrade.py" # Upgrade specific attribute data_node: name: data_node multinode: true candisable: true scaleout: "enabled" rolling_upgrade: true # Upgrade specific attribute can_replace_storage: true image: name: elasticsearch version: "6.6.2" engine: docker entrypoint: docker-entrypoint.sh upgrade_from: [5*, 6.6.1] # Upgrade specific attribute compute: memory: 12288M cpu: reserve: false cores: 4 storage: - type: data1 media: hdd path: /usr/share/elasticsearch/data size: 200G env: ES_JAVA_OPTS: "-Xms6g -Xmx6g" CLUSTERNAME: "{{APP_NAME}}" IP_ADDRESS: "{{IP_ADDRESS}}" UNICAST_HOST: "{% for r in app['roles'] if ( 'master_node' in r['name']) %}{% for v in r['vnodes'] %}{{v['network'][0]['allocated_ip']}}{% if not loop.last %},{% endif %}{% endfor %}{% if not loop.last %},{% endif %}{% endfor %}" MASTER_NODE: type: boolean value: false label: "Master Node" DATA_NODE: type: boolean value: true label: "Data Node" INGEST_NODE: type: boolean value: false label: "Ingest Node" DATA_DIRS: "{% for v in vnode['storage'] if 'data' in v['type'] %}{{v['path']}}{% if not loop.last %} {% endif %}{% endfor %}" vnodehooks: preupgrade: "python preupgrade.py" # Upgrade specific attribute postupgrade: "python postupgrade.py" # Upgrade specific attribute apphooks: health: "python3.4 health.py" validate: "python3.4 validate_app.py" postcreate: "python3.4 cluster_status.py" poststart: "python3.4 cluster_status.py" postclone: "python3.4 cluster_status.py" postrollback: "python3.4 cluster_status.py" preupgrade: "python3.4 cluster_status.py" # Upgrade specific attribute