Docker Machine
Introduction
In December 2014 Docker Inc. announced Docker Machine, Swarm and Compose for orchestrating distributed applications. Docker Machine eliminates the need to create virtual machines and install Docker before starting Docker containers, and handles provisioning and installation of all components for you.
It works on multiple cloud providers, and recently support for Openstack with Identity v3 was added to Git.
Let's take a quick look at deploying a cluster!
Preparations
Before we set out on our adventure, we need to have our house in order.
Software Installation
As of release 0.3.0 of Docker Machine you can use the official binaries available on github.
After you have fetched a suitable executable, move it somewhere into you $PATH and rename it to "docker-machine". You should also have Docker installed locally on your machine.
Environment
To make things simpler, you should import your Zetta.IO access credentials into your environment from your .openrc file.
On Linux and Mac OS X you source the file into your current shell:
source Standard.openrc
On Windows you add them to your running environment by by accessing "Control Panel" » "System and Security" » "System" » "Advanced" tab » "Environment Variables...". Alternatively you can just set them manually in your CMD or PowerShell.
In addition to this you should have a working copy of the Openstack Client. As an alternative you can also use the Dashboard, but this is not covered in this article. Our client machine is running Ubuntu Linux 14.04 with version 1.0.2 of Openstack Client and version 1.5.0 of Docker.
Security Groups
Docker Machine and Docker needs direct access to all instances on the Docker and Swarm API ports over TLS in addition to direct SSH access. The default policy on Zetta.IO allows SSH access, but not access to the API ports.
We'll set up a separate Security Group to open up access from the world:
$ openstack security group create DockerAPI $ openstack security group rule create --proto tcp --dst-port 2376 DockerAPI $ openstack security group rule create --proto tcp --dst-port 3376 DockerAPI
Flavors and Images
We need to know the IDs of the instance flavor and base image to use. This can be done via the Dashboard or with the Openstack Client:
$ openstack flavor list +----+------------+-------+------+-----------+------+-------+-------------+-----------+-------------+ | ID | Name | RAM | Disk | Ephemeral | Swap | VCPUs | RXTX Factor | Is Public | Extra Specs | +----+------------+-------+------+-----------+------+-------+-------------+-----------+-------------+ | 11 | t2.micro | 1024 | 0 | 0 | | 1 | 1.0 | True | | | 12 | t2.small | 2048 | 0 | 0 | | 2 | 2.0 | True | | | 6 | m3.medium | 4096 | 14 | 0 | | 1 | 1.0 | True | | | 7 | m3.large | 8192 | 28 | 0 | | 2 | 1.0 | True | | | 8 | m3.xlarge | 16384 | 56 | 0 | | 4 | 1.0 | True | | | 9 | m3.2xlarge | 32768 | 112 | 0 | | 8 | 1.0 | True | | +----+------------+-------+------+-----------+------+-------+-------------+-----------+-------------+
We decide on the m3.medium flavor size, and note that the ID of this is "6".
$ openstack image list +--------------------------------------+---------------------------------------------------------------------------+ | ID | Name | +--------------------------------------+---------------------------------------------------------------------------+ | 6ada08d7-3217-469e-977c-ea97c6d399a0 | Fedora 20 [Local Storage] | | f51d126b-c929-44e4-ab3e-4c32225145f4 | Fedora 20 [Volume Storage] | | d0a89aa8-9644-408d-a023-4dcc1148ca01 | Ubuntu 14.04 (AMD64) [Local Storage] | | e12ff7e3-9638-4b27-b050-616880d832af | Ubuntu 14.04 (AMD64) [Volume Storage] | | 90ecf7e2-0645-4429-b7fb-695ee5f183f7 | Ubuntu 12.04 (AMD64) [Volume Storage] | | 1f290dac-7ebd-497c-a0fe-96ccaeb17254 | Ubuntu 12.04 (AMD64) [Local Storage] | +--------------------------------------+---------------------------------------------------------------------------+
We decide on the "Ubuntu 14.04 (AMD64) [Local Storage]" image, and note that the ID of this is "d0a89aa8-9644-408d-a023-4dcc1148ca01".
Workarounds
The current version of Docker Machine is not using the correct environment variables. While we are waiting for this to be fixed, let's do a quick workaround:
$ export OS_TENANT_NAME=$OS_PROJECT_NAME $ export OS_DOMAIN_ID=$OS_PROJECT_DOMAIN_ID
Please note that when OS_DOMAIN_ID is specified, Openstack Client will try to scope to a domain instead of a project/tenant. If you want to use Openstack Client in the same session after setting this, you have to unset OS_DOMAIN_ID first.
The current version of Docker Machine does not reuse or deallocate floating IP addresses. You have to release these back into the Public pool manually using the Openstack Client or Dashboard, or you will quickly use up your quota.
Creating a Docker instance
Now that everything is in order, we can finally start our first machine. We've split up the command line for easier reading:
docker-machine create \ --driver openstack \ --openstack-flavor-id 6 \ --openstack-image-id d0a89aa8-9644-408d-a023-4dcc1148ca01 \ --openstack-floatingip-pool Public \ --openstack-ssh-user ubuntu \ --openstack-sec-groups default,DockerAPI \ zettaio-docker-01
Let's examine the options we've used.
--driver openstack loads the generic Openstack driver.
--openstack-flavor-id 6 tells the driver what flavor to use. We noticed that "6" was the ID of the flavor "m3.medium" above.
--openstack-image-id d0a89aa8-9644-408d-a023-4dcc1148ca01 is the image we want to use. This could be any other image supported by Docker Machine.
--openstack-floatingip-pool Public is what pool to allocate a floating IP from. If this is not defined, Docker Machine assumes that there is no need for this, and will try to access the instance using it's local IPv4 address, something that will not work over the internet. "Public" is the shared pool on Zetta.IO.
--openstack-ssh-user ubuntu is the default user created by cloud-init during image creation. For Fedora it usually is "fedora", but since we are using Ubuntu the user is called "ubuntu". By default Docker Machine has this set to "root".
--openstack-sec-groups default,DockerAPI is a comma separated list of what security groups should be activated on the instance. This should reference groups we have defined earlier.
Lastly we set the name for the instance, here we have used the name "zettaio-docker-01".
Let's run it:
$ docker-machine create \ --driver openstack \ --openstack-flavor-id 6 \ --openstack-image-id d0a89aa8-9644-408d-a023-4dcc1148ca01 \ --openstack-floatingip-pool Public \ --openstack-ssh-user ubuntu \ --openstack-sec-groups default,DockerAPI \ zettaio-docker-01 INFO[0002] Creating machine... sudo: unable to resolve host zettaio-docker-01 sudo: unable to resolve host zettaio-docker-01 sudo: unable to resolve host zettaio-docker-01 INFO[0174] "zettaio-docker-01" has been created and is now the active machine. INFO[0174] To point your Docker client at it, run this in your shell: eval "$(docker-machine env zettaio-docker-01)"
Awesome, we're in business.
The "sudo: unable to resolve host zettaio-docker-01" error from above is due to the default cloud-init configuration on the official Ubuntu images missing the "manage_etc_hosts" option.
The process took about 2-3 minutes, but we have noticed that there is a bug in the Docker 1.5.0 packages that can make the process hang. If you have problems, use the "--debug" option before the "create" command to get a full output.
Let's see the state of our instance:
$ docker-machine ls NAME ACTIVE DRIVER STATE URL SWARM zettaio-docker-01 * openstack Running tcp://185.56.186.167:2376
And let's run a container on it:
$ eval "$(docker-machine env zettaio-docker-01)" $ docker run busybox echo hello world Unable to find image 'busybox:latest' locally latest: Pulling from busybox 511136ea3c5a: Pull complete df7546f9f060: Pull complete ea13149945cb: Pull complete 4986bf8c1536: Already exists Digest: sha256:2ae4190f4401ca2501f9b6855be21287f43590054598b134e8ff70b11b1e4327 Status: Downloaded newer image for busybox:latest hello world
Now you have a simple docker instance ready to deploy containers on, and you can create more instances to scale up.
To delete the instance, you use the "rm" command:
$ docker-machine rm zettaio-docker-01 INFO[0000] Deleting OpenStack instance... INFO[0001] The machine was successfully removed.
As noted above, this command does not deallocate the Floating IP from your pool.
If you have failed instances that can not be removed, you can wipe them from the local Docker Machine configuration by using the "-f" flag to the "rm" command. Please note that this will ignore the driver, and not delete the machine in Openstack.
Let's create a cluster!
One of the newest features in Docker is the Docker Swarm clustering system. Docker Machine has native support for this, and makes it really easy to set up.
First we need to create a swarm token. You can do this using a local Docker instance or using the instance we created earlier.
$ docker run swarm create Unable to find image 'swarm:latest' locally 511136ea3c5a: Pull complete ae115241d78a: Pull complete f49087514537: Pull complete fff73787bd9f: Pull complete 97c8f6e912d7: Pull complete 33f9d1e808cf: Pull complete 62860d7acc87: Pull complete bf8b6923851d: Pull complete Status: Downloaded newer image for swarm:latest 395777e61973d481e26d5520e451835a
The last line of output is your token. Write it down, it is used for cluster autodiscovery.
Swarm Master
We need to create an instance that we use as the Swarm Master. This instance is also used for deploying containers, so resources will not be wasted.
$ docker-machine create \ --driver openstack \ --openstack-flavor-id 6 \ --openstack-image-id "d0a89aa8-9644-408d-a023-4dcc1148ca01" \ --openstack-floatingip-pool Public \ --openstack-ssh-user ubuntu \ --openstack-sec-groups default,DockerAPI \ --swarm \ --swarm-master \ --swarm-discovery token://<TOKEN FROM ABOVE> \ swarm-master
The Swarm Master node should now be up and running.
Swarm Nodes
Let's create two more nodes, so we have a 3 node cluster:
$ docker-machine create \ --driver openstack \ --openstack-flavor-id 6 \ --openstack-image-id "d0a89aa8-9644-408d-a023-4dcc1148ca01" \ --openstack-floatingip-pool Public \ --openstack-ssh-user ubuntu \ --openstack-sec-groups default,DockerAPI \ --swarm \ --swarm-discovery token://<TOKEN FROM ABOVE> \ swarm-node-00 $ docker-machine create \ --driver openstack \ --openstack-flavor-id 6 \ --openstack-image-id "d0a89aa8-9644-408d-a023-4dcc1148ca01" \ --openstack-floatingip-pool Public \ --openstack-ssh-user ubuntu \ --openstack-sec-groups default,DockerAPI \ --swarm \ --swarm-discovery token://<TOKEN FROM ABOVE> \ swarm-node-01
Now we can enable the Swarm environment:
$ eval $(docker-machine env --swarm swarm-master)
Let's look at the whole cluster:
$ docker info Containers: 4 Nodes: 3 swarm-master: 185.56.186.163:2376 | Containers: 2 | Reserved CPUs: 0 / 1 | Reserved Memory: 0 B / 3.861 GiB swarm-node-00: 185.56.186.164:2376 | Containers: 1 | Reserved CPUs: 0 / 1 | Reserved Memory: 0 B / 3.861 GiB swarm-node-01: 185.56.186.165:2376 | Containers: 1 | Reserved CPUs: 0 / 1 | Reserved Memory: 0 B / 3.861 GiB
To test out the cluster, we will start a few Redis containers, and set the affinity of these to run on separate hosts:
$ docker run -d --name redis_1 -e ‘affinity:container!=redis_*’ redis 73e8a011e2af05161573119b20436c4f6efee58f608bc01b20b95000427b712e $ docker run -d --name redis_2 -e ‘affinity:container!=redis_*’ redis e31377cc35c586cb902a7421ae1c161c8ddd7729503c7ead27f132779109c02c
Let's see if they started up on separate nodes:
$ docker ps CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES e31377cc35c5 redis:3 "/entrypoint.sh redi About a minute ago Up About a minute 6379/tcp swarm-node-01/redis_2 73e8a011e2af redis:3 "/entrypoint.sh redi 4 minutes ago Up 4 minutes 6379/tcp swarm-master/redis_1
Job well done.
Resources
For more information, please consult the following documentation: