************************************************************************************************* Influencing Workflow Behavior ************************************************************************************************* During an application's lifecycle several house keeping tasks may need to be completed to ensure the seamless deployment and maintenance of the application. Depending on the application, tasks may need to be completed before or the life cycle operation to be performed. Moreover the aforementioned tasks may need to be executed in different contexts since it might need to performed at an application level or with a finer grain of control at the container level. In order to achieve seamless application deployment and maintenance Robin supports several application lifecycle operations and provides a framework to execute application/Vnode level hooks before or after the respective operations. In addition, Robin also provides the capability to run custom entrypoint scripts for Docker based applications. Moreover in certain scenarios, Kubernetes objects that are not currently supported in the Robin framework but can be spawned natively are needed for a successful application deployment. To resolve this, Robin allows one to specify a set of YAMLs for Kubernetes object creation before and after the instantiation of the main pods/containers. ---------------------------------------------------------------------------------------------- Vnode Hook Scripts ---------------------------------------------------------------------------------------------- Vnode hook scripts are run for each individual container. For each Vnode lifecycle operation, there is an accompanying script that can be run either before or after the operation is executed. The following interpreters are supported for Vnode hooks: - bash - sh - Python .. Note:: It is highly recommended that Vnode hook scripts are idempotent i.e. the consquences of their execution should remain constant regardless of how many times they run. The list of Vnode hooks that can be run are as follows: ============= ======================================== Hook Name Description ============= ======================================== precreate Script that is executed before an Vnode is created during an ApplicationCreate job postcreate Script that is executed after creating a Vnode during an ApplicationCreate job prestart Script that is executed before starting a Vnode during ApplicationStart job poststart Script that is executed after starting a Vnode during ApplicationStart job prestop Script that is executed before stopping a Vnode during ApplicationStop job poststop Script that is executed after stopping a Vnode during ApplicationStop job pregrow Script that is executed before scaling out during ApplicationScale job. Will be run on the new Vnode. postgrow Script that is executed after scaling out during ApplicationScale job. Will be run on the new Vnode. preclone Script that is executed before cloning a Vnode during ApplicationClone job. postclone Script that is executed after cloning a Vnode during ApplicationClone job. presnapshot Script that is executed before taking a snapshot of a Vnode during ApplicationSnapshot job. postsnapshot Script that is executed after taking a snapshot of a Vnode during ApplicationSnapshot job. prerollback Script that is executed before rolling back a Vnode during ApplicationRollback job. postrollback Script that is executed after rolling back a Vnode during ApplicationRollback job. predestroy Script that is executed before deleting a Vnode during ApplicationDelete job. postdestroy Script that is executed after deleting a Vnode during ApplicationDelete job. ============= ======================================== .. Note:: Vnode hook scripts have access to all ENV variables defined in the manifest for the Vnode in the form of key value pairs alongside two additional keys, hostname and ROBINHOST, whose values are the container hostname and physical hostname of the node on which it is placed respectively. As a result, any custom arguments that need to be passed to the scripts should be done so as key-value pairs and parsed appropriately in the script. ---------------------------------------------------------------------------------------------- Application Hook Scripts ---------------------------------------------------------------------------------------------- As the name implies, application hook scripts are run at the application level and thus can impact any containers that are part of the application. Application hooks are executed once all application containers are successfully spawned. For each application lifecycle operation, there is an accompanying script that can be run either before or after the operation is executed. In addition, there are two additional scripts which can be run to extract information and derive the health status of an application. These scripts are detailed in the table below. The following interpreters are supported for application hooks: • bash • sh • perl • Python • Ruby • ansible-playbook .. Note:: It is highly recommended that Vnode hook scripts are idempotent i.e. the consquences of their execution should remain constant regardless of how many times they run. The list of application hooks that can be run are as follows: ============= ======================================== Hook Name Description ============= ======================================== info Information hooks are executed to provide additional details about an application. validate Validate hooks are executed to ensure the deployment inputs are appropriate before creating an application. health Health hooks are executed to assess the health of an application. The hook runs every 90 seconds and is expected to return 0 or 1. precreate Script that is executed before creating an application. Typically, any validation and/or prechecks that are needed for the deployment are included in this script. postcreate Script that is executed after an application is successfully created. Typically, any initial setup tasks can be included in this hook. prestart Script that is executed before an application is started. poststart Script that is executed after an application is started. prestop Script that is executed before an application is stopped. Typcially, any commands needed gracefully stop the application are included in this hook. poststop Script that is executed after an application is stopped. pregrow Script that is executed before an application is scaled out. postgrow Script that is executed after an application is scaled out. Typically, any commands to update the application configuration to account for the new instance are included in this hook. preclone Script that is executed before an application is cloned. postclone Script that is executed after an application is cloned. It will be invoked on the cloned application. presnapshot Script that is executed before a snapshot of an application is taken. Typically, any commands to Quiesce the application are included in this hook. postsnapshot Script that is executed after a snapshot of an application has been taken. Typically, any commands to un-Quiesce the application are included in this hook. prerollback Script that is executed before an application is rolled back to a snapshot. Typically, any commands to gracefully stop an application (if required) are included in this hook. postrollback Script that is executed after an application has been rolled back to a snapshot. predestroy Script that is executed before deleting an application. postdestroy Script that is executed after deleting an application. ============= ======================================== .. note:: The only argument application hooks can process is a path to a JSON file containing the application defintion and details. As a result, the script will have to parse this information in order to utilize it. In addition hooks that run for operations such as cloning and scaling require both the current and original application configurations. Thus the path to both JSON files are passed to the respective scripts. ---------------------------------------------------------------------------------------------- Order of execution ---------------------------------------------------------------------------------------------- All specified application level hooks will be run once per lifecycle operation regardless of it was triggered via an event or manually by a user. In addition all specified Vnode level hooks will also be run once for each container associated with an application everytime a lifecycle operation occurs. For manifests in which there is a combination of application and vnode level hooks specified for a particular operation, the following order will be adhered to if the respective scripts exist: the application pre-operation hook will run, followed by the pre and post-operation Vnode level hooks and finally the application post-operation hook will run. .. Note:: For every lifecycle operation only the pre/post hook scripts for that particular operation will run regardless of its a Vnode or application level hook script. ---------------------------------------------------------------------------------------------- Creating additional Kubernetes objects ---------------------------------------------------------------------------------------------- Robin bundles allow users to deploy Kubernetes native objects such as daemonsets, cronjobs, configmaps, secrets etc. as pre/post actions while executing bundle application hooks. Users can configure Kubernetes object YAML files to their liking and include them within the ``scripts`` folder. In addition, depending on the directory they are placed in these objects will be spawned before and and/or after the application create stage. Since these objects are considered to be part of the Robin application, when the associated application is deleted the aforementioned objects will be cleaned up automatically. In order indicate that additional Kubenetes objects need to be created a directory named ``k8s`` should be present within``scripts`` folder of the bundle. For Kubenetes objects that need to be created pre-application deployment, the respective YAML definitions need to be placed within another sub-directory called ``pre`` within the ``k8s`` folder. Similarly, for Kubenetes objects that need to be created post-application deployment, their respective YAML definitions should be under the ``post`` directory within the ``k8s`` folder. An example of the bundle directory structure with Kubernetes configuration YAMLs is shown below: .. code-block:: yaml - manifest.yaml - scripts |--k8s |-- pre |-- test-daemonset-preapp.yaml |-- test-configmap-preapp.yaml |-- .jinjaignore |-- post |-- test-daemonset-postapp.yaml |-- test-configmap-postapp.yaml .. Note:: Kubenetes objects created as part of the application deployment will be spawned in the same namespace as the associated application. In addition users are able to use Robin defined macros as a part of specified YAML definitions in order to access run time information. An example of their usage is shown below: .. code-block:: yaml kind: ConfigMap metadata: name: "{{APP_NAME}}" namespace: "{{APP_NS}}" data: port_config: 2222 .. code-block:: yaml kind: Secret metadata: name: "{{APP_NAME}}-post-secret" namespace: "{{APP_NS}}" type: Opaque data: username: YWRtaW4= Users will also have the option to skip Robin Jinja macro evaluation for YAMLs under the ``pre`` and ``post`` directories within the ``k8s`` folder by including a ``.jinjaignore`` file as showcased in the directory structure above. In this file, users can list the YAMLs for which macro evaluation should be skipped. An example of a ``.jinjaignore`` file is shown below: .. code-block:: text ### cat .jinjaignore test-daemonset-preapp.yaml test-configmap-preapp.yaml In order to skip Jinja macro evaluation for all YAML files under the ``pre`` and ``post`` directories, a single wildcard character can be specified within the ``.jinjaignore`` file as shown below: .. code-block:: text ### cat .jinjaignore * .. ---------------------------------------------------------------------------------------------- Example of bundle with vnode hooks ---------------------------------------------------------------------------------------------- Given below is the manifest file for Postgres bundle which uses vnode hooks. .. code-block:: yaml name: postgres version: "9.5" icon: logo.png snapshot: enabled clone: enabled roles: [postgres] postgres: name: postgres multinode: true description: Runs a PostgreSQL (object-relational database management system) image: name: postgres version: 9.5 engine: docker storage: - type: data media: ssd path: /var/lib/postgresql/data/pgdata count: 1 fixed: true size: 10G env: POSTGRES_PASSWORD: type: password value: "postgres" POSTGRES_DB: postgres POSTGRES_USER: postgres PGDATA: "{{ROBIN_STORAGE.data.DIRS}}" #POSTGRES_INITDB_ARGS: compute: memory: 1024M cpu: reserve: false cores: 1 vnodehooks: postcreate: bash check_pgsql poststart: bash check_pgsql postgrow: bash check_pgsql postclone: bash check_pgsql postrollback: bash check_pgsql In this example the bash script check_pgsql is executed after create, start, scale, clone and rollback operations. The content of the check_pgsql is as follows. .. code-block:: shell #!/bin/bash source $(dirname $0)/vnode_validate sleep 20 cmd="docker exec $CONTAINERNAME " if [[ -n "$PG_DBNAME" ]]; then cmd+="psql -U $PG_USER -lqt | cut -d \| -f 1 | grep -qw $PG_DBNAME" else cmd+="psql -U $PG_USER -lqt | cut -d \| -f 1 | grep -qw postgres" fi while [[ 1 ]]; do state=`docker inspect --format={{.State.Status}} $CONTAINERNAME` if [[ "$state" != *"running"* ]]; then echo "Docker instance $CONTAINERNAME is not running" exit 1 fi out=`eval $cmd` cmd_status=$? echo "Executing command" echo -e "Output\n------------------\n$out" if [[ $cmd_status -eq 0 ]]; then #if [[ "$out" == *"$PG_DBNAME"* ]]; then #exit 0 #fi exit 0 fi sleep 10 done ---------------------------------------------------------------------------------------------- Example of bundle with app hooks ---------------------------------------------------------------------------------------------- .. code-block:: yaml name: WordPress version: "1.0" roles: [mysql, wordpress] serialize: true icon: wordpress-logo-stacked-rgb.png clonemode: unfenced snapshot: enabled clone: enabled mysql: name: mysql image: name: mysql version: "5.7" engine: docker storage: - media: ssd path: "/var/lib/mysql" type: data env: MYSQL_ROOT_PASSWORD: type: password value: "robin123" MYSQL_DATABASE: "wordpress" MYSQL_USER: "robin" MYSQL_PASSWORD: type: password value: "robin123" vnodehooks: poststart: "bash check_mysql" postcreate: "bash check_mysql" postrollback: "bash check_mysql" postclone: "bash check_mysql" wordpress: name: wordpress image: name: robinsys/wordpress version: "1.0" engine: docker depends: [mysql] storage: - media: ssd type: data path: "/var/www/html" env: WORDPRESS_DB_HOST: "{{ROLES[0].VNODES[0].IP_ADDRESS}}:3306" WORDPRESS_DB_USER: "{{ROLES[0].VNODES[0].ENV.MYSQL_USER}}" WORDPRESS_DB_PASSWORD: type: password value: "{{ROLES[0].VNODES[0].ENV.MYSQL_PASSWORD}}" WORDPRESS_DB_NAME: "{{ROLES[0].VNODES[0].ENV.MYSQL_DATABASE}}" service_ports: [80] vnodehooks: poststart: "bash updt_mysql MYSQL_HOSTNAME={{ROLES[0].VNODES[0].HOSTNAME}} WORDPRESS_PH={{ROLES[1].VNODES[0].ALLOCATED_HOST_PUBLIC_HOSTNAME}} WORDPRESS_PIP={{ROLES[1].VNODES[0].ALLOCATED_HOST_PUBLIC_IP}} WORDPRESS_SERVICE_PORT={{ROLES[1].VNODES[0].MAPPED_PORT.80}} WORDPRESS_ELASTIC_IP={{ROLES[1].VNODES[0].ELASTIC_IP}}" postclone: "bash updt_mysql MYSQL_HOSTNAME={{ROLES[0].VNODES[0].HOSTNAME}} WORDPRESS_PH={{ROLES[1].VNODES[0].ALLOCATED_HOST_PUBLIC_HOSTNAME}} WORDPRESS_PIP={{ROLES[1].VNODES[0].ALLOCATED_HOST_PUBLIC_IP}} WORDPRESS_SERVICE_PORT={{ROLES[1].VNODES[0].MAPPED_PORT.80}} WORDPRESS_ELASTIC_IP={{ROLES[1].VNODES[0].ELASTIC_IP}}" apphooks: info: "python3.4 app_info.py" In this example the python script app_info.py is executed as info hook. The content of app_info.py is as follows. .. code-block:: python import sys import json import socket def get_app_info(app_config): app_info = {} for r in app_config.get('roles', []): for v in r.get('vnodes', []): if v.get('role') == "wordpress": if v.get('hostname'): service_url = None for port in [80]: try: s = socket.socket(socket.AF_INET, socket.SOCK_STREAM) s.settimeout(1) s.connect((v.get('hostname'), port)) service_url = "http://{}/wp-admin".format(v.get('hostname')) break except: continue if service_url: app_info['service_urls'] = [{"name": "Wordpress", "url": service_url}] return json.dumps(app_info) if __name__ == '__main__': json_file = sys.argv[1] with open(json_file) as f: app_config = json.load(f) print(get_app_info(app_config)) ---------------------------------------------------------------------------------------------- Snapshot Hooks ---------------------------------------------------------------------------------------------- Here is the manifest file for Mariadb. This manifest uses both pre and post snapshot hooks to quiesce and unquiesce the Mariadb database before and after taking the snapshot. .. code-block:: yaml name: Maria DB based on Docker description: My organization's MariaDB application website: http://www.mycompany.com icon: Mariadb.png snapshot: enabled clone: enabled clonemode: unfenced roles: [mariadb] mariadb: name: mariadb description: Runs mariadb database server commandline: "-W" image: name: mariadb engine: docker version: "10.3.5" storage: - type: data media: ssd path: "/var/lib/mysql" count: 1 fixedcount: true env: MYSQL_ROOT_PASSWORD: type: password value: "robin123" MYSQL_DATABASE: robin MYSQL_USER: robin MYSQL_PASSWORD: type: password value: "robin123" vnodehooks: precreate: bash vnode_sample opr=precreate postcreate: bash check_mariadb prestart: bash vnode_sample opr=prestart poststart: bash check_mariadb prestop: bash vnode_sample opr=prestop poststop: bash vnode_sample opr=poststop pregrow: bash vnode_sample opr=pregrow postgrow: bash check_mariadb preclone: bash vnode_sample opr=preclone postclone: bash check_mariadb presnapshot: bash quiesce.sh postsnapshot: bash unquiesce.sh prerollback: bash vnode_sample opr=prerollback postrollback: bash check_mariadb predestroy: bash vnode_sample opr=predestroy postdestroy: bash vnode_sample opr=postdestroy apphooks: validate: python3.4 validate_template.py precreate: bash app_sample precreate postcreate: bash app_sample postcreate prestart: bash app_sample prestart poststart: bash app_sample poststart prestop: bash app_sample prestop poststop: bash app_sample poststop pregrow: bash app_sample pregrow postgrow: bash app_sample postgrow preclone: bash app_sample preclone postclone: bash app_sample postclone presnapshot: bash app_sample presnapshot postsnapshot: bash app_sample postsnapshot prerollback: bash app_sample prerollback postrollback: bash app_sample postrollback Here is the content for quiesce.sh .. code-block:: bash #!/bin/bash source $(dirname $0)/vnode_validate sleep 2 cmd="docker exec $CONTAINERNAME " cmd+="mysql -u root -p$ROOT_PASSWD $DB_NAME -e \"FLUSH TABLES with READ LOCK\"" out=`eval $cmd` cmd_status=$? echo "Executing command" echo -e "Output\n------------------\n$out" if [[ $cmd_status -eq 0 ]]; then exit 0 fi exit 1 Here is the content for unquiesce.sh .. code-block:: bash #!/bin/bash source $(dirname $0)/vnode_validate sleep 2 cmd="docker exec $CONTAINERNAME " cmd+="mysql -u root -p$ROOT_PASSWD $DB_NAME -e \"UNLOCK TABLES\"" out=`eval $cmd` cmd_status=$? echo "Executing command" echo -e "Output\n------------------\n$out" if [[ $cmd_status -eq 0 ]]; then exit 0 fi exit 1 ---------------------------------------------------------------------------------------------- Clone Hooks ---------------------------------------------------------------------------------------------- The below mongodb bundle uses clone hook. .. code-block:: yaml name: MongoDB version: 1.0 description: MongoDB icon: icon.png roles: [mongodb] snapshot: enabled clone: enabled clonemode: unfenced mongodb: name: mongo multinode: true multinode_value: 3 scaleout: enabled version: "1.0" rolling_upgrade: true can_replace_storage: true image: name: mongo version: 3.4 engine: docker entrypoint: shardsvr_entrypoint.sh compute: memory: 8G cpu: reserve: false cores: 4 storage: - type: data media: ssd path: /data/db fstype: ext4 size: 50G env: REPLICASET: "rs0" TLS_PEM_KEY: stash/mongodb.pem CLUSTER_ROLE: "shardsvr" ARBITER: type: number value: 0 min: 0 max: 1 label: "ARBITER" ARBITERS: "{% for r in app['roles'] if role['name'] == r['name'] %}{% for v in r['vnodes'] %}{{v['hostname']}}:{{v['env']['allocated']['ARBITER']}}{% if not loop.last %},{% endif %}{% endfor %}{% if not loop.last %},{% endif %}{% endfor %}" vnodehooks: postcreate: "python3.4 create.py HOSTNAME={{HOSTNAME}} MEMBERS={{HOSTNAMES}}" postgrow: "python3.4 scaleout.py HOSTNAME={{HOSTNAME}} MEMBERS={{HOSTNAMES}}" postclone: "python3.4 clone.py HOSTNAME={{HOSTNAME}} MEMBERS={{HOSTNAMES}}" preupgrade: "python3.4 preupgrade.py HOSTNAME={{HOSTNAME}}" apphooks: info: "python3.4 mongoinfo.py" upgradeorder: "python3.4 app_upgradeorder.py" presnapshot: "python3.4 app_presnap.py" clone.py .. code-block:: python #!/usr/bin/env python from mongosetup import MongoCluster from process_args import process_args if __name__ == '__main__': curr_host, members, replicaset, clusterrole, query_router, tls, arbiters, container_name = process_args() # Run the replica creation in the last node. print(members[-1], curr_host, clusterrole, query_router) rs = MongoCluster(curr_host, members, replicaset, role=clusterrole, query_router=query_router, tls=tls, arbiters=arbiters, container_name=container_name) if members[-1] == curr_host: print(rs) rs.reconfig() rs.run('db.isMaster()') else: # Just wait till the mongod process boot straps rs.wait_for_mongod() ---------------------------------------------------------------------------------------------- Upgrade Hooks ---------------------------------------------------------------------------------------------- Here is an example of an upgrade hook used in mongodb bundle. .. code-block:: yaml name: MongoDB version: 1.0 description: MongoDB icon: icon.png roles: [mongodb] snapshot: enabled clone: enabled clonemode: unfenced mongodb: name: mongo multinode: true multinode_value: 3 scaleout: enabled version: "1.0" rolling_upgrade: true can_replace_storage: true image: name: mongo version: 3.4 engine: docker entrypoint: shardsvr_entrypoint.sh compute: memory: 8G cpu: reserve: false cores: 4 storage: - type: data media: ssd path: /data/db fstype: ext4 size: 50G env: REPLICASET: "rs0" TLS_PEM_KEY: stash/mongodb.pem CLUSTER_ROLE: "shardsvr" ARBITER: type: number value: 0 min: 0 max: 1 label: "ARBITER" ARBITERS: "{% for r in app['roles'] if role['name'] == r['name'] %}{% for v in r['vnodes'] %}{{v['hostname']}}:{{v['env']['allocated']['ARBITER']}}{% if not loop.last %},{% endif %}{% endfor %}{% if not loop.last %},{% endif %}{% endfor %}" vnodehooks: postcreate: "python3.4 create.py HOSTNAME={{HOSTNAME}} MEMBERS={{HOSTNAMES}}" postgrow: "python3.4 scaleout.py HOSTNAME={{HOSTNAME}} MEMBERS={{HOSTNAMES}}" postclone: "python3.4 clone.py HOSTNAME={{HOSTNAME}} MEMBERS={{HOSTNAMES}}" preupgrade: "python3.4 preupgrade.py HOSTNAME={{HOSTNAME}}" apphooks: info: "python3.4 mongoinfo.py" upgradeorder: "python3.4 app_upgradeorder.py" presnapshot: "python3.4 app_presnap.py" preupgrade.py .. code-block:: python #!/usr/bin/python3.4 from upgradehelper import UpgradeHelper from process_args import process_args import logging import sys logger = logging.getLogger("robin.upgrade") if __name__ == '__main__': curr_host, members, replicaset, clusterrole, query_router, tls, arbiters, container_name = process_args() upgrade_helper = UpgradeHelper(hostname=curr_host, tls=tls) logger.info("Trying to step down {} as secondary if its primary".format(curr_host)) if upgrade_helper.is_stable_state(): upgrade_helper.step_down() else: logger.error("Replica set is not in stable state, hence failing upgrade") sys.exit(1) ---------------------------------------------------------------------------------------------- Validation Hook ---------------------------------------------------------------------------------------------- Give below is the manifest file for elasticsearch which uses validate hook to validate the deployment inputs. .. code-block:: yaml name: elasticsearch version: 6.2.3 description: elasticsearch icon: elastic.png roles: [master_eligible_node] snapshot: "enabled" serialize: true clone: "enabled" clonemode: unfenced upgrade_order: [master_eligible_node] master_eligible_node: name: master_eligible_node multinode: true candisable: true scaleout: "enabled" addvolume: "enabled" rolling_upgrade: true can_replace_storage: true image: name: docker.elastic.co/elasticsearch/elasticsearch version: "6.2.3" engine: docker entrypoint: docker-entrypoint.sh upgrade_from: [5] compute: memory: 12288M cpu: reserve: false cores: 4 storage: - type: data1 media: hdd path: /usr/share/elasticsearch/data size: 200G env: ES_JAVA_OPTS: "-Xms6g -Xmx6g" CLUSTERNAME: "{{APP_NAME}}" IP_ADDRESS: "{{IP_ADDRESS}}" UNICAST_HOST: "{% for r in app['roles'] if ( 'dedicated_master_node' in r['name'] or 'master_eligible_node' in r['name']) %}{% for v in r['vnodes'] %}{{v['network'][0]['allocated_ip']}}{% if not loop.last %},{% endif %}{% endfor %}{% if not loop.last %},{% endif %}{% endfor %}" MASTER_NODE: type: boolean value: true label: "Master Node" DATA_NODE: type: boolean value: true label: "Data Node" INGEST_NODE: type: boolean value: false label: "Ingest Node" DATA_DIRS: "{% for v in vnode['storage'] if 'data' in v['type'] %}{{v['path']}}{% if not loop.last %} {% endif %}{% endfor %}" service_ports: [9200] vnodehooks: preupgrade: "python3.4 preupgrade.py" postupgrade: "python3.4 postupgrade.py" apphooks: info: "python3 app_info.py" health: "python3.4 health.py" validate: "python3.4 validate_app.py" postcreate: "python3.4 cluster_status.py" poststart: "python3.4 cluster_status.py" postclone: "python3.4 cluster_status.py" postrollback: "python3.4 cluster_status.py" preupgrade: "python3.4 cluster_status.py" postupgrade: "python3.4 cluster_status.py" validate_app.py .. code-block:: python #!/usr/bin/python3.4 import sys import os import time import json def validate(appconfig): roles = appconfig.get('roles', []) elk_role = None UNICAST_HOST = None BOX_TYPE = None CLUSTERNAME = None IP_ADDRESS = None ES_JAVA_OPTS = None master_nodes = 0 for role in roles: if role.get('name', None) in ('master_eligible_node'): elk_role = role vnodes = elk_role.get('vnodes', []) if role.get('name', None) in ('master_eligible_node'): master_nodes = master_nodes + len (vnodes) elk_node = vnodes[0] elk_env = elk_node.get('env', {}) ES_JAVA_OPTS = elk_env.get('ES_JAVA_OPTS', None) CLUSTERNAME = elk_env.get('CLUSTERNAME', None) UNICAST_HOST = elk_env.get('UNICAST_HOST', None) IP_ADDRESS = elk_env.get('IP_ADDRESS', None) if not ES_JAVA_OPTS: raise Exception("ES_JAVA_OPTS not provided") if not UNICAST_HOST : raise Exception("UNICAST_HOST not provided") if not CLUSTERNAME: raise Exception("CLUSTERNAME not provided") if not IP_ADDRESS: raise Exception("IP_ADDRESS not provided") for vnode in vnodes: dir_check = False storage_def = vnode.get('storage', []) for item in storage_def: partitions = item.get('partitions', []) if partitions: for part in partitions: fstype = part.get('fstype', 'ext4') if fstype == "raw": continue if part.get('path', '/')[:29] == '/usr/share/elasticsearch/data': dir_check = True else: fstype = item.get('fstype', 'ext4') if fstype == "raw": continue if item.get('path', '/')[:29] == '/usr/share/elasticsearch/data': dir_check = True if not dir_check: raise Exception("Data directory mount point not '/usr/share/elasticsearch/data'") if master_nodes < 2: raise Exception("Minimum master nodes should be 2. Add additional nodes to master_eligible_node") try: configfile = sys.argv[1] with open(configfile) as fp: config = json.loads(fp.read()) validate(config) except Exception as ex: print("{}".format(ex.args[0]), file=sys.stderr) sys.exit(1) ---------------------------------------------------------------------------------------------- Info Hook ---------------------------------------------------------------------------------------------- .. code-block:: yaml name: WordPress version: "1.0" roles: [mysql, wordpress] serialize: true icon: wordpress-logo-stacked-rgb.png clonemode: unfenced snapshot: enabled clone: enabled mysql: name: mysql image: name: mysql version: "5.7" engine: docker storage: - media: ssd path: "/var/lib/mysql" type: data env: MYSQL_ROOT_PASSWORD: type: password value: "robin123" MYSQL_DATABASE: "wordpress" MYSQL_USER: "robin" MYSQL_PASSWORD: type: password value: "robin123" vnodehooks: poststart: "bash check_mysql" postcreate: "bash check_mysql" postrollback: "bash check_mysql" postclone: "bash check_mysql" wordpress: name: wordpress image: name: robinsys/wordpress version: "1.0" engine: docker depends: [mysql] storage: - media: ssd type: data path: "/var/www/html" env: WORDPRESS_DB_HOST: "{{ROLES[0].VNODES[0].IP_ADDRESS}}:3306" WORDPRESS_DB_USER: "{{ROLES[0].VNODES[0].ENV.MYSQL_USER}}" WORDPRESS_DB_PASSWORD: type: password value: "{{ROLES[0].VNODES[0].ENV.MYSQL_PASSWORD}}" WORDPRESS_DB_NAME: "{{ROLES[0].VNODES[0].ENV.MYSQL_DATABASE}}" service_ports: [80] vnodehooks: poststart: "bash updt_mysql MYSQL_HOSTNAME={{ROLES[0].VNODES[0].HOSTNAME}} WORDPRESS_PH={{ROLES[1].VNODES[0].ALLOCATED_HOST_PUBLIC_HOSTNAME}} WORDPRESS_PIP={{ROLES[1].VNODES[0].ALLOCATED_HOST_PUBLIC_IP}} WORDPRESS_SERVICE_PORT={{ROLES[1].VNODES[0].MAPPED_PORT.80}} WORDPRESS_ELASTIC_IP={{ROLES[1].VNODES[0].ELASTIC_IP}}" postclone: "bash updt_mysql MYSQL_HOSTNAME={{ROLES[0].VNODES[0].HOSTNAME}} WORDPRESS_PH={{ROLES[1].VNODES[0].ALLOCATED_HOST_PUBLIC_HOSTNAME}} WORDPRESS_PIP={{ROLES[1].VNODES[0].ALLOCATED_HOST_PUBLIC_IP}} WORDPRESS_SERVICE_PORT={{ROLES[1].VNODES[0].MAPPED_PORT.80}} WORDPRESS_ELASTIC_IP={{ROLES[1].VNODES[0].ELASTIC_IP}}" apphooks: info: "python3.4 app_info.py" In this example the python script app_info.py is executed as info hook. The content of app_info.py is as follows. .. code-block:: python import sys import json import socket def get_app_info(app_config): app_info = {} for r in app_config.get('roles', []): for v in r.get('vnodes', []): if v.get('role') == "wordpress": if v.get('hostname'): service_url = None for port in [80]: try: s = socket.socket(socket.AF_INET, socket.SOCK_STREAM) s.settimeout(1) s.connect((v.get('hostname'), port)) service_url = "http://{}/wp-admin".format(v.get('hostname')) break except: continue if service_url: app_info['service_urls'] = [{"name": "Wordpress", "url": service_url}] return json.dumps(app_info) if __name__ == '__main__': json_file = sys.argv[1] with open(json_file) as f: app_config = json.load(f) print(get_app_info(app_config)) ---------------------------------------------------------------------------------------------- Health Hook ---------------------------------------------------------------------------------------------- Health hooks are used to let Robin know the health of the application. Given below is the manifest file for Elasticsearch that uses health hook. .. code-block:: yaml name: elasticsearch version: 6.2.3 description: elasticsearch icon: elastic.png roles: [master_eligible_node] snapshot: "enabled" serialize: true clone: "enabled" clonemode: unfenced upgrade_order: [master_eligible_node] master_eligible_node: name: master_eligible_node multinode: true candisable: true scaleout: "enabled" addvolume: "enabled" rolling_upgrade: true can_replace_storage: true image: name: docker.elastic.co/elasticsearch/elasticsearch version: "6.2.3" engine: docker entrypoint: docker-entrypoint.sh upgrade_from: [5] compute: memory: 12288M cpu: reserve: false cores: 4 storage: - type: data1 media: hdd path: /usr/share/elasticsearch/data size: 200G env: ES_JAVA_OPTS: "-Xms6g -Xmx6g" CLUSTERNAME: "{{APP_NAME}}" IP_ADDRESS: "{{IP_ADDRESS}}" UNICAST_HOST: "{% for r in app['roles'] if ( 'dedicated_master_node' in r['name'] or 'master_eligible_node' in r['name']) %}{% for v in r['vnodes'] %}{{v['network'][0]['allocated_ip']}}{% if not loop.last %},{% endif %}{% endfor %}{% if not loop.last %},{% endif %}{% endfor %}" MASTER_NODE: type: boolean value: true label: "Master Node" DATA_NODE: type: boolean value: true label: "Data Node" INGEST_NODE: type: boolean value: false label: "Ingest Node" DATA_DIRS: "{% for v in vnode['storage'] if 'data' in v['type'] %}{{v['path']}}{% if not loop.last %} {% endif %}{% endfor %}" service_ports: [9200] vnodehooks: preupgrade: "python3.4 preupgrade.py" postupgrade: "python3.4 postupgrade.py" apphooks: info: "python3 app_info.py" health: "python3.4 health.py" validate: "python3.4 validate_app.py" postcreate: "python3.4 cluster_status.py" poststart: "python3.4 cluster_status.py" postclone: "python3.4 cluster_status.py" postrollback: "python3.4 cluster_status.py" preupgrade: "python3.4 cluster_status.py" postupgrade: "python3.4 cluster_status.py" health.py .. code-block:: python #!/usr/bin/python3.4 import requests import sys import json from eslib import get_app_info def main(): json_file = sys.argv[1] with open(json_file) as f: app_config = json.load(f) print(get_app_info(app_config)) return get_app_info(app_config) if __name__ == "__main__": main() get_app_info .. code-block:: python def get_app_info(app_config): es_url = None is_cluster_healthy = 1 for r in app_config.get('roles', []): if r.get('name') in ("es_node","es_master","es_slave","dedicated_master_node", "master_eligible_node", "data_node", "hot_node", "warm_node", "ingest_node", "coordination_node"): for v in r.get('vnodes', []): if v.get('hostname'): es_url = "http://%s:9200/_cluster/health?pretty" % (v.get('hostname')) response = requests.get(es_url) resp_data = response.json() if ( resp_data['status'] == 'green' or resp_data['status'] == 'yellow') : is_cluster_healthy = 0 break if is_cluster_healthy == 0: break return is_cluster_healthy ---------------------------------------------------------------------------------------------- Scale Hook ---------------------------------------------------------------------------------------------- .. code-block:: yaml name: MongoDB version: 1.0 description: MongoDB icon: icon.png roles: [mongodb] snapshot: enabled clone: enabled clonemode: unfenced mongodb: name: mongo multinode: true multinode_value: 3 scaleout: enabled version: "1.0" rolling_upgrade: true can_replace_storage: true image: name: mongo version: 3.4 engine: docker entrypoint: shardsvr_entrypoint.sh compute: memory: 8G cpu: reserve: false cores: 4 storage: - type: data media: ssd path: /data/db fstype: ext4 size: 50G env: REPLICASET: "rs0" TLS_PEM_KEY: stash/mongodb.pem CLUSTER_ROLE: "shardsvr" ARBITER: type: number value: 0 min: 0 max: 1 label: "ARBITER" ARBITERS: "{% for r in app['roles'] if role['name'] == r['name'] %}{% for v in r['vnodes'] %}{{v['hostname']}}:{{v['env']['allocated']['ARBITER']}}{% if not loop.last %},{% endif %}{% endfor %}{% if not loop.last %},{% endif %}{% endfor %}" vnodehooks: postcreate: "python3.4 create.py HOSTNAME={{HOSTNAME}} MEMBERS={{HOSTNAMES}}" postgrow: "python3.4 scaleout.py HOSTNAME={{HOSTNAME}} MEMBERS={{HOSTNAMES}}" postclone: "python3.4 clone.py HOSTNAME={{HOSTNAME}} MEMBERS={{HOSTNAMES}}" preupgrade: "python3.4 preupgrade.py HOSTNAME={{HOSTNAME}}" apphooks: info: "python3.4 mongoinfo.py" upgradeorder: "python3.4 app_upgradeorder.py" presnapshot: "python3.4 app_presnap.py" scaleout.py .. code-block:: python #!/usr/bin/env python from mongosetup import MongoCluster from process_args import process_args if __name__ == '__main__': curr_host, members, replicaset, clusterrole, query_router, tls, arbiters, container_name = process_args() rs = MongoCluster(curr_host, members, replicaset, role=clusterrole, query_router=query_router, tls=tls, arbiters=arbiters, container_name=container_name) if members[-1] == curr_host: print(rs) rs.scaleout() rs.run('db.isMaster()') else: # Just wait till the mongod process boot straps rs.wait_for_mongod()