Docker集中化web界面管理平台-Shipyard部署记录

Docker图形页面管理工具基本常用的有三种: DOCKER UI,Shipyard,Portainer。对比后发现,Shipyard最强大,其次是Portainer,最后是Docker ui。之前介绍了DOcker的web管理工具DockerUI,下面介绍下Docker的另一个web界面管理工具Shipyard的使用。Shipyard(github)是建立在docker集群管理工具Citadel之上的可以管理容器、主机等资源的web图形化工具,包括core和extension两个版本,core即shipyard主要是把多个 Docker host上的 containers 统一管理(支持跨越多个host),extension即shipyard-extensions添加了应用路由和负载均衡、集中化日志、部署等;Shipyard是在Docker Swarm实现对容器、镜像、docker集群、仓库、节点进行管理的web系统。

DockerUI功能:通过Web浏览器的命令行来管理的任务。
DockerUI的优点
1)可以对运行着的容器进行批量操作
2)在容器网络中-会显示容器与容器间的网络关系
3)在Volumes中显示了所有挂载目录

DockerUI一个致命的缺点:不支持多主机.试想一下,如果有N台docker主机时-我就需要一台台的用dockerui进行管理,想想都觉得麻烦。就单台主机而言,dockerui是一款不错的管理工具。

Shipyard功能:是一个集成管理docker容器、镜像、Registries的系统,它可以简化对横跨多个主机的Docker容器集群进行管理. 通过Web用户界面,你可以大致浏览相关信息,比如你的容器在使用多少处理器和内存资源、在运行哪些容器,还可以检查所有集群上的事件日志。
其特性主要包括:
1)支持节点动态集群,可扩展节点的规模(swarm、etcd方案)
2)支持镜像管理、容器管理、节点管理等功能
3)可视化的容器管理和监控管理
4)在线容console终端

Shipyard几个概念
1)engine
一个shipyard管理的docker集群可以包含一个或多个engine(引擎),一个engine就是监听tcp端口的docker daemon。shipyard管理docker daemon、images、containers完全基于Docker API,不需要做其他的修改。另外,shipyard可以对每个engine做资源限制,包括CPU和内存;因为TCP监听相比Unix socket方式会有一定的安全隐患,所以shipyard还支持通过SSL证书与docker后台进程安全通信。
2)rethinkdb
RethinkDB是一个shipyard项目的一个docker镜像,用来存放账号(account)、引擎(engine)、服务密钥(service key)、扩展元数据(extension metadata)等信息,但不会存储任何有关容器或镜像的内容。

Shipyard生态
shipyard是由shipyard控制器以及周围生态系统构成,以下按照deploy启动顺序进行介绍(下面几个就是shipyard使用脚本安装后,启动的几个容器名)
1)RethinkDB
deploy首先启动的就是RethinkDB容器,shipyard采用RethinkDB作为数据库来保存用户等信息
2)Discovery
为了使用Swarm,我们需要一个外部的密钥值存储群容器,shipyard默认是采用了etcd。
3)shipyard_certs
证书管理容器,实现证书验证功能
4)Proxy
默认情况下,Docker引擎只监听Socket,我们可以重新配置引擎使用TLS或者使用一个代理容器,转发请求从TCP到Docker监听的UNIX Socket。
5)Swarm Manager
Swarm管理器
6)Swarm Agent
Swarm代理,运行在每个节点上。
7)Controller
shipyard控制器,Remote API的实现和web的实现。

Centos7.5系统下Shipyard(中文版)部署记录

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 [root@docker-218 ~] # cat /etc/redhat-release CentOS Linux release 7.5.1804 (Core)   [root@docker-218 ~] # ifconfig|grep 172.16.60.218 inet 172.16.60.218 netmask 255.255.255.0 broadcast 172.16.60.255   关闭主机防火墙 [root@docker-218 ~] # systemctl disable firewalld.service [root@docker-218 ~] # systemctl stop firewalld.service [root@docker-218 ~] # iptables -F [root@docker-218 ~] # firewall-cmd --state not running   安装docker [root@docker-218 ~] # yum install docker   修改docker配置文件,添加下面一行,进行docker加速设置 [root@node-1 ~] # vim /etc/sysconfig/docker                //在文件底部添加下面一行, (这里就是直接写: xxx.mirror.aliyuncs.com) ADD_REGISTRY= ‘--add-registry xxx.mirror.aliyuncs.com‘   启动docker服务 [root@docker-218 ~] # systemctl start docker   下载相关镜像(这些镜像如果不提前下载,则在下面一键安装部署时会自动下载,不过要等待一段时间. 所以最好提前下载,一键部署时就很快了) [root@docker-218 ~] # docker pull rethinkdb [root@docker-218 ~] # docker pull microbox/etcd [root@docker-218 ~] # docker pull shipyard/docker-proxy [root@docker-218 ~] # docker pull swarm [root@docker-218 ~] # docker pull dockerclub/shipyard   下载官方一键部署脚本 https: //shipyard-project .com /deploy             这个地址已经失效 最新下载地址: https: //pan .baidu.com /s/1ATM32S7tLA35Q-xK7-TgzQ     提取密码: kgqi   接着执行一键部署 替换Controller为中文版 [root@docker-213 ~] # chmod 755 shipyard-deploy [root@docker-213 ~] # sh shipyard-deploy Deploying Shipyard   -> Starting Database   -> Starting Discovery   -> Starting Cert Volume   -> Starting Proxy   -> Starting Swarm Manager   -> Starting Swarm Agent   -> Starting Controller Waiting  for  Shipyard on 172.16.60.213:8080   Shipyard available at http: //172 .16.60.213:8080 Username: admin Password: shipyard   部署后,可以看到相应的shipyard容器已经创建好了 [root@docker-218 ~] # docker ps CONTAINER ID        IMAGE                          COMMAND                  CREATED             STATUS              PORTS                                            NAMES 0cc242b4d90b        dockerclub /shipyard :latest      "/bin/controller -..."    19 seconds ago      Up 15 seconds       0.0.0.0:8080->8080 /tcp                            shipyard-controller ce08a7f0f62f        swarm:latest                    "/swarm j --addr 1..."    20 seconds ago      Up 19 seconds       2375 /tcp                                          shipyard-swarm-agent 9d2dd2bd5bff        swarm:latest                    "/swarm m --replic..."    20 seconds ago      Up 19 seconds       2375 /tcp                                          shipyard-swarm-manager 3435b5e2d13a        shipyard /docker-proxy :latest    "/usr/local/bin/run"      21 seconds ago      Up 20 seconds       0.0.0.0:2375->2375 /tcp                            shipyard-proxy 315ca39f00dd        alpine                          "sh"                      21 seconds ago      Up 21 seconds                                                        shipyard-certs 564f25ac8130        microbox /etcd :latest            "/bin/etcd -addr 1..."    22 seconds ago      Up 21 seconds       0.0.0.0:4001->4001 /tcp , 0.0.0.0:7001->7001 /tcp    shipyard-discovery bff634944376        rethinkdb                       "rethinkdb --bind all"    22 seconds ago      Up 22 seconds       8080 /tcp , 28015 /tcp , 29015 /tcp                    shipyard-rethinkdb   最后访问http: //172 .16.60.218:8080,使用admin /shipyard 用户名和密码登录即可.  (注意:一键部署之后,需要稍等一会儿,8080端口才能起来)   =================================================================================== 如果想要修改web访问端口,则操作如下: [root@docker-218 ~] # cat shipyard-deploy |grep 8080      echo  "  PORT: specify the listen port for the controller (default: 8080)" SHIPYARD_PORT=${PORT:-8080}   比如将脚本中默认的8080端口改为80端口 [root@docker-218 ~] # sed -i ‘s/8080/80/g‘ shipyard-deploy   然后重新部署即可

shipyard的web界面展示如下

删除Shipyard环境操作

1 2 3 4 5 6 7 8 9 10 11 12 13 [root@docker-218 ~] # cat shipyard-deploy |ACTION=remove bash             Removing Shipyard   -> Removing Database   -> Removing Discovery   -> Removing Cert Volume   -> Removing Proxy   -> Removing Swarm Agent   -> Removing Swarm Manager   -> Removing Controller Done   [root@docker-218 ~] # docker ps CONTAINER ID        IMAGE               COMMAND             CREATED             STATUS              PORTS               NAMES

Shipyard添加其他节点主机(centos7.X系统的主机)的操作记录

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 比如将172.16.60.220机器节点加入到上面的shipyard管理中.   则需要在被加入节点机器上如下操作: 1) 在被加入的节点上按照上面的最新地址下载shipyard-deploy 2) 执行下面操作 (注意下面etcd地址要写成shipyard部署机的ip地址. 首次执行时会自动下载镜像,需要等待一段时间) [root@docker-220 ~] # cat shipyard-deploy| ACTION=node DISCOVERY=etcd://172.16.60.218:4001 bash ...... Node added to Swarm: 172.16.60.220   [root@docker-220 ~] # docker ps CONTAINER ID        IMAGE                          COMMAND                  CREATED             STATUS              PORTS                    NAMES 65b7783d7e1e        swarm:latest                    "/swarm j --addr 1..."    12 seconds ago      Up 10 seconds       2375 /tcp                  shipyard-swarm-agent 1b50c974b6e7        swarm:latest                    "/swarm m --replic..."    14 seconds ago      Up 12 seconds       2375 /tcp                  shipyard-swarm-manager 2514cf31947b        shipyard /docker-proxy :latest    "/usr/local/bin/run"      17 seconds ago      Up 15 seconds       0.0.0.0:2375->2375 /tcp    shipyard-proxy bc9c79df1707        alpine                          "sh"                      19 seconds ago      Up 16 seconds         注意,添加节点时,上面一键脚本需要运行在被添加的节点主机上,而不是shipyard部署节点的机器上.   然后登录http: //172 .16.60.218:8080,在 "容器管理" - "主机节点" 中看到添加的节点信息了.   删除节点机 [root@docker-220 ~] # cat shipyard-deploy |ACTION=remove bash -s Removing Shipyard   -> Removing Database   -> Removing Discovery   -> Removing Cert Volume   -> Removing Proxy   -> Removing Swarm Agent   -> Removing Swarm Manager   -> Removing Controller Done [root@docker-220 ~] # docker ps CONTAINER ID        IMAGE               COMMAND             CREATED             STATUS              PORTS               NAMES 

需要注意:如果添加CentOS6.x系统的节要先升级kernel和docker

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 1) 升级内核 [root@docker-6 ~] # cat /etc/redhat-release CentOS release 6.9 (Final)   [root@docker-6 ~] # uname -r 2.6.32-696.el6.x86_64   可以参考http: //elrepo .org /tiki/tiki-index .php    根据系统版本选择el源   先导入公钥 [root@docker-6 ~] # rpm --import https://www.elrepo.org/RPM-GPG-KEY-elrepo.org [root@docker-6 ~] # rpm -Uvh http://www.elrepo.org/elrepo-release-6-8.el6.elrepo.noarch.rpm   升级内核 [root@docker-6 ~] # yum --enablerepo=elrepo-kernel install kernel-lt -y 或者安装 [root@docker-6 ~] # yum --enablerepo=elrepo-kernel install kernel-ml -y   修改Grub.conf [root@docker-6 ~] # vim /etc/grub.conf ..... default=0                      // 将此项的值由1改为0   重启系统 [root@docker-6 ~] # init 6   重启后再次查看内核,发现已经升级了 [root@docker-6 ~] # uname -r                4.4.162-1.el6.elrepo.x86_64   2) 升级docker (如果之前部署了docker环境的话) 下面的docker二进制文件,百度云盘下载地址: https: //pan .baidu.com /s/1AWBbF-CXfXFkIIYs5DTCmA 提取密码: a4r6 [root@docker-6 ~] # wget https://get.docker.com/builds/Linux/x86_64/docker-1.10.3 [root@docker-6 ~] # chmod 755 docker-1.9.1                //这是下载的docker二进制文件   停止docker服务 [root@docker-6 ~] # service docker stop   替换docker二进制文件 [root@docker-6 ~] # cp /usr/bin/docker /usr/bin/docker_bak [root@docker-6 ~] # cp docker-1.9.1 /usr/bin/docker   启动docker服务 [root@docker-6 ~] # service docker start   下载shipyard一键部署脚本 [root@docker-6 ~] # chmod 755 shipyard-deploy [root@docker-220 ~] # cat shipyard-deploy| ACTION=node DISCOVERY=etcd://172.16.60.218:4001 bash

Nginx做域名转发:  在nginx配置文件http块里添加如下代码:

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 upstream shipyard {       server 172.16.60.218:8080; } server {      listen          80;      server_name     kevin.shipyard.com;         location / {          proxy_pass http: //shipyard ;          proxy_http_version 1.1;          proxy_set_header Upgrade $http_upgrade;          proxy_set_header Connection  "upgrade" ;      } }     以下三行代码为nginx做WebSockets跳转否则,在执行shell时会报400错误 proxy_http_version 1.1; proxy_set_header Upgrade $http_upgrade; proxy_set_header Connection  "upgrade" ;

利用shipyard部署swarm集群及管理容器, 则swarm-manage为shipyard部署机, 其他swarm-node节点则为添加到shipyard管理中的节点.

 

 

注:

GitHub地址:https://github.com/shipyard/shipyard-project.com

#!/bin/bash

if [ "$1" != "" ] && [ "$1" = "-h" ]; then
    echo "Shipyard Deploy uses the following environment variables:"
    echo "  ACTION: this is the action to use (deploy, upgrade, node, remove)"
    echo "  DISCOVERY: discovery system used by Swarm (only if using ‘node‘ action)"
    echo "  IMAGE: this overrides the default Shipyard image"
    echo "  PREFIX: prefix for container names"
    echo "  SHIPYARD_ARGS: these are passed to the Shipyard controller container as controller args"
    echo "  TLS_CERT_PATH: path to certs to enable TLS for Shipyard"
    echo "  PORT: specify the listen port for the controller (default: 8080)"
    echo "  IP: specify the address at which the controller or node will be available (default: eth0 ip)"
    echo "  PROXY_PORT: port to run docker proxy (default: 2375)"
    exit 1
fi

if [ -z "`which docker`" ]; then
    echo "You must have the Docker CLI installed on your \$PATH"
    echo "  See http://docs.docker.com for details"
    exit 1
fi

ACTION=${ACTION:-deploy}
IMAGE=${IMAGE:-shipyard/shipyard:latest}
PREFIX=${PREFIX:-shipyard}
SHIPYARD_ARGS=${SHIPYARD_ARGS:-""}
TLS_CERT_PATH=${TLS_CERT_PATH:-}
CERT_PATH="/etc/shipyard"
PROXY_PORT=${PROXY_PORT:-2375}
SWARM_PORT=3375
SHIPYARD_PROTOCOL=http
SHIPYARD_PORT=${PORT:-8080}
SHIPYARD_IP=${IP}
DISCOVERY_BACKEND=etcd
DISCOVERY_PORT=4001
DISCOVERY_PEER_PORT=7001
ENABLE_TLS=0
CERT_FINGERPRINT=""
LOCAL_CA_CERT=""
LOCAL_SSL_CERT=""
LOCAL_SSL_KEY=""
LOCAL_SSL_CLIENT_CERT=""
LOCAL_SSL_CLIENT_KEY=""
SSL_CA_CERT=""
SSL_CERT=""
SSL_KEY=""
SSL_CLIENT_CERT=""
SSL_CLIENT_KEY=""

show_cert_help() {
    echo "To use TLS in Shipyard, you must have existing certificates."
    echo "The certs must be named ca.pem, server.pem, server-key.pem, cert.pem and key.pem"
    echo "If you need to generate certificates, see https://github.com/ehazlett/certm for examples."
}

check_certs() {
    if [ -z "$TLS_CERT_PATH" ]; then
        return
    fi

    if [ ! -e $TLS_CERT_PATH ]; then
        echo "Error: unable to find certificates in $TLS_CERT_PATH"
        show_cert_help
        exit 1
    fi

    if [ "$PROXY_PORT" = "2375" ]; then
        PROXY_PORT=2376
    fi
    SWARM_PORT=3376
    SHIPYARD_PROTOCOL=https
    LOCAL_SSL_CA_CERT="$TLS_CERT_PATH/ca.pem"
    LOCAL_SSL_CERT="$TLS_CERT_PATH/server.pem"
    LOCAL_SSL_KEY="$TLS_CERT_PATH/server-key.pem"
    LOCAL_SSL_CLIENT_CERT="$TLS_CERT_PATH/cert.pem"
    LOCAL_SSL_CLIENT_KEY="$TLS_CERT_PATH/key.pem"
    SSL_CA_CERT="$CERT_PATH/ca.pem"
    SSL_CERT="$CERT_PATH/server.pem"
    SSL_KEY="$CERT_PATH/server-key.pem"
    SSL_CLIENT_CERT="$CERT_PATH/cert.pem"
    SSL_CLIENT_KEY="$CERT_PATH/key.pem"
    CERT_FINGERPRINT=$(openssl x509 -noout -in $LOCAL_SSL_CERT -fingerprint -sha256 | awk -F= ‘{print $2;}‘)

    if [ ! -e $LOCAL_SSL_CA_CERT ] || [ ! -e $LOCAL_SSL_CERT ] || [ ! -e $LOCAL_SSL_KEY ] || [ ! -e $LOCAL_SSL_CLIENT_CERT ] || [ ! -e $LOCAL_SSL_CLIENT_KEY ]; then
        echo "Error: unable to find certificates"
        show_cert_help
        exit 1
    fi

    ENABLE_TLS=1
}

# container functions
start_certs() {
    ID=$(docker run         -ti         -d         --restart=always         --name $PREFIX-certs         -v $CERT_PATH         alpine         sh)
    if [ $ENABLE_TLS = 1 ]; then
        docker cp $LOCAL_SSL_CA_CERT $PREFIX-certs:$SSL_CA_CERT
        docker cp $LOCAL_SSL_CERT $PREFIX-certs:$SSL_CERT
        docker cp $LOCAL_SSL_KEY $PREFIX-certs:$SSL_KEY
        docker cp $LOCAL_SSL_CLIENT_CERT $PREFIX-certs:$SSL_CLIENT_CERT
        docker cp $LOCAL_SSL_CLIENT_KEY $PREFIX-certs:$SSL_CLIENT_KEY
    fi
}

remove_certs() {
    docker rm -fv $PREFIX-certs > /dev/null 2>&1
}

get_ip() {
    if [ -z "$SHIPYARD_IP" ]; then
        SHIPYARD_IP=`docker run --rm --net=host alpine ip route get 8.8.8.8 | awk ‘{ print $7;  }‘`
    fi
}

start_discovery() {
    get_ip

    ID=$(docker run         -ti         -d         -p 4001:4001         -p 7001:7001         --restart=always         --name $PREFIX-discovery         microbox/etcd:latest -addr $SHIPYARD_IP:$DISCOVERY_PORT -peer-addr $SHIPYARD_IP:$DISCOVERY_PEER_PORT)
}

remove_discovery() {
    docker rm -fv $PREFIX-discovery > /dev/null 2>&1
}

start_rethinkdb() {
    ID=$(docker run         -ti         -d         --restart=always         --name $PREFIX-rethinkdb         rethinkdb)
}

remove_rethinkdb() {
    docker rm -fv $PREFIX-rethinkdb > /dev/null 2>&1
}

start_proxy() {
    TLS_OPTS=""
    if [ $ENABLE_TLS = 1 ]; then
        TLS_OPTS="-e SSL_CA=$SSL_CA_CERT -e SSL_CERT=$SSL_CERT -e SSL_KEY=$SSL_KEY -e SSL_SKIP_VERIFY=1"
    fi
    # Note: we add SSL_SKIP_VERIFY=1 to skip verification of the client
    # certificate in the proxy image.  this will pass it to swarm that
    # does verify.  this helps with performance and avoids certificate issues
    # when running through the proxy.  ultimately if the cert is invalid
    # swarm will fail to return.
    ID=$(docker run         -ti         -d         -p $PROXY_PORT:$PROXY_PORT         --hostname=$HOSTNAME         --restart=always         --name $PREFIX-proxy         -v /var/run/docker.sock:/var/run/docker.sock         -e PORT=$PROXY_PORT         --volumes-from=$PREFIX-certs $TLS_OPTS        shipyard/docker-proxy:latest)
}

remove_proxy() {
    docker rm -fv $PREFIX-proxy > /dev/null 2>&1
}

start_swarm_manager() {
    get_ip

    TLS_OPTS=""
    if [ $ENABLE_TLS = 1 ]; then
        TLS_OPTS="--tlsverify --tlscacert=$SSL_CA_CERT --tlscert=$SSL_CERT --tlskey=$SSL_KEY"
    fi

    EXTRA_RUN_OPTS=""

    if [ -z "$DISCOVERY" ]; then
        DISCOVERY="$DISCOVERY_BACKEND://discovery:$DISCOVERY_PORT"
        EXTRA_RUN_OPTS="--link $PREFIX-discovery:discovery"
    fi
    ID=$(docker run         -ti         -d         --restart=always         --name $PREFIX-swarm-manager         --volumes-from=$PREFIX-certs $EXTRA_RUN_OPTS         swarm:latest         m --replication --addr $SHIPYARD_IP:$SWARM_PORT --host tcp://0.0.0.0:$SWARM_PORT $TLS_OPTS $DISCOVERY)
}

remove_swarm_manager() {
    docker rm -fv $PREFIX-swarm-manager > /dev/null 2>&1
}

start_swarm_agent() {
    get_ip

    if [ -z "$DISCOVERY" ]; then
        DISCOVERY="$DISCOVERY_BACKEND://discovery:$DISCOVERY_PORT"
        EXTRA_RUN_OPTS="--link $PREFIX-discovery:discovery"
    fi
    ID=$(docker run         -ti         -d         --restart=always         --name $PREFIX-swarm-agent $EXTRA_RUN_OPTS         swarm:latest         j --addr $SHIPYARD_IP:$PROXY_PORT $DISCOVERY)
}

remove_swarm_agent() {
    docker rm -fv $PREFIX-swarm-agent > /dev/null 2>&1
}

start_controller() {
    #-v $CERT_PATH:/etc/docker:ro     TLS_OPTS=""
    if [ $ENABLE_TLS = 1 ]; then
        TLS_OPTS="--tls-ca-cert $SSL_CA_CERT --tls-cert=$SSL_CERT --tls-key=$SSL_KEY --shipyard-tls-ca-cert=$SSL_CA_CERT --shipyard-tls-cert=$SSL_CERT --shipyard-tls-key=$SSL_KEY"
    fi

    ID=$(docker run         -ti         -d         --restart=always         --name $PREFIX-controller         --link $PREFIX-rethinkdb:rethinkdb         --link $PREFIX-swarm-manager:swarm         -p $SHIPYARD_PORT:$SHIPYARD_PORT         --volumes-from=$PREFIX-certs         $IMAGE         --debug         server         --listen :$SHIPYARD_PORT         -d tcp://swarm:$SWARM_PORT $TLS_OPTS $SHIPYARD_ARGS)
}

wait_for_available() {
    set +e 
    IP=$1
    PORT=$2
    echo Waiting for Shipyard on $IP:$PORT

    docker pull ehazlett/curl > /dev/null 2>&1

    TLS_OPTS=""
    if [ $ENABLE_TLS = 1 ]; then
        TLS_OPTS="-k"
    fi

    until $(docker run --rm ehazlett/curl --output /dev/null --connect-timeout 1 --silent --head --fail $TLS_OPTS $SHIPYARD_PROTOCOL://$IP:$PORT/ > /dev/null 2>&1); do
        printf ‘.‘
        sleep 1 
    done
    printf ‘\n‘
}

remove_controller() {
    docker rm -fv $PREFIX-controller > /dev/null 2>&1
}

if [ "$ACTION" = "deploy" ]; then
    set -e

    check_certs

    get_ip 

    echo "Deploying Shipyard"
    echo " -> Starting Database"
    start_rethinkdb
    echo " -> Starting Discovery"
    start_discovery
    echo " -> Starting Cert Volume"
    start_certs
    echo " -> Starting Proxy"
    start_proxy
    echo " -> Starting Swarm Manager"
    start_swarm_manager
    echo " -> Starting Swarm Agent"
    start_swarm_agent
    echo " -> Starting Controller"
    start_controller

    wait_for_available $SHIPYARD_IP $SHIPYARD_PORT

    echo "Shipyard available at $SHIPYARD_PROTOCOL://$SHIPYARD_IP:$SHIPYARD_PORT"
    if [ $ENABLE_TLS = 1 ] && [ ! -z "$CERT_FINGERPRINT" ]; then
        echo "SSL SHA-256 Fingerprint: $CERT_FINGERPRINT"
    fi
    echo "Username: admin Password: shipyard"

elif [ "$ACTION" = "node" ]; then
    set -e

    if [ -z "$DISCOVERY" ]; then
        echo "You must set the DISCOVERY environment variable"
        echo "with the discovery system used with Swarm"
        exit 1
    fi

    check_certs

    echo "Adding Node"
    echo " -> Starting Cert Volume"
    start_certs
    echo " -> Starting Proxy"
    start_proxy
    echo " -> Starting Swarm Manager"
    start_swarm_manager $DISCOVERY
    echo " -> Starting Swarm Agent"
    start_swarm_agent

    echo "Node added to Swarm: $SHIPYARD_IP"
    
elif [ "$ACTION" = "upgrade" ]; then
    set -e

    check_certs

    get_ip

    echo "Upgrading Shipyard"
    echo " -> Pulling $IMAGE"
    docker pull $IMAGE

    echo " -> Upgrading Controller"
    remove_controller
    start_controller

    wait_for_available $SHIPYARD_IP $SHIPYARD_PORT

    echo "Shipyard controller updated"

elif [ "$ACTION" = "remove" ]; then
    # ignore errors
    set +e

    echo "Removing Shipyard"
    echo " -> Removing Database"
    remove_rethinkdb
    echo " -> Removing Discovery"
    remove_discovery
    echo " -> Removing Cert Volume"
    remove_certs
    echo " -> Removing Proxy"
    remove_proxy
    echo " -> Removing Swarm Agent"
    remove_swarm_agent
    echo " -> Removing Swarm Manager"
    remove_swarm_manager
    echo " -> Removing Controller"
    remove_controller

    echo "Done"
else
    echo "Unknown action $ACTION"
    exit 1
fi