The key thing we need to make this work is the ability to keep the load balancer up-to-date with the containers that can handle web requests. Fortunately I came across docker-loadbalancer which gave me a solution:
- Use Registrator to spot when Docker starts and stops containers, and write them to Consul
- Use consul-template to rewrite nginx's config file when new web services are added to or removed from the Consul store.
Here's a hugely helpful post from Joseph Miller on how registrator & consul work together.
I created an unbelievably small Flask web server that simply shows the host name, which is the container ID you're running in. I used this as the high-priority task in Microscaling-in-a-Box, to randomly vary the number of running instances of this server container. The other task is just a busybox to represent low priority work.
Here's a short video showing it running.
If you'd like to run this yourself, all the instructions are on the GitHub page.
Next steps include distributing the containers across a cluster rather than a single machine (we've done this before with simple sleeper containers, without the load balancing aspect). And of course as part of the Force12 product we're working on mapping performance metrics to the number of containers to run.