Monday, 25 April 2016

3 Ways DevOps Is Helping Destroy The World

What?!! No really. Here's my doomsday rationale
  1.  A scary proportion of the world’s energy is already spent powering data centres (~2%)
  2. Cloud data centre capacity is growing fast but is inefficiently used (average ~10% resource utilisation).
  3. Infrastructure as code makes it easy to overprovision in the cloud. If machine creation can be scripted it can be reproduced. You can create 100 machines as easily as 1.
So, in 5 years’ time the 2% of energy currently being used in data centres could be 4% or higher. That’s a lot of power stations. And a lot of overprovisioning.

Why do we Overprovision in the Cloud?
Because we can.

DevOps has given us the power and it’s not like we have a lot of choice. Autoscaling is not magic - you can’t scale up in real time because it takes at least a few minutes to bring up a new VM. So, if you cannot predict the future (and if you can, your remuneration package must be  good) then to handle unexpected demand you have to overprovision and keep that extra capacity sitting around hot-but-idle. And then thank goodness you have that choice. In the old days we just fell over ;-)

Cloud+devops gives us the option of overprovisioning to avoid failure. So we do overprovision and that’s not a crazy judgement. It you fall over your company might fail. Overprovisioning is just money.

What does philosophy have to say? 
My favourite 18th century German philosopher Kant would say “what if everyone overprovisioned their infrastructure?”

The answer is higher energy usage in a world where energy generation is mostly CO2-producing fossil fuels. Hmm. Kant would say not ideal. It’s a shame to be sitting in a cloud of CO2, but it’s particularly galling if that’s just to keep data centre capacity idle.

What can we do?
Hope for AWS to save us! 

Or, alternatively....

VMs and cloud infrastructure don’t help with server utilisation. It feels like they should but the data suggest that in practise they don’t. They probably make it worse. However, there are lots of new technologies coming along that do help: containers, microscaling, orchestrators & serverless architectures (potentially). 

Just look at Google, they use all of these technologies to achieve server utilisation of around 70%, which is >5 times what the rest of us manage. If we were all achieving that then maybe devops wouldn't help destroy the world after all. 

Tuesday, 12 April 2016

Microscaling with an NSQ Queue

Today we're launching a release of Microscaling-in-a-Box that scales to maintain the length of an NSQ queue. With this, we're moving out of the realms of the theoretical to something that is genuinely useful for a real world scenario.

Microscaling makes a system's architecture dynamic – able to respond to real demand in real time. With Microscaling-in-a-Box you can now scale the number of containers that process jobs on a queue, so that you always have an appropriate number to cope with the rate that jobs are arriving on the queue. 

What's in this release?

This release has everything you need to run Microscaling on a single machine on which you have installed Docker.  You'll get the following containers: 
  • an example message producer, which simulates external demand and adds items to the queue
  • an NSQ Server which maintains the queue
  • a Microscaling agent
  • an example message consumer, which takes items off the queue
  • an example low priority background process (defaults to a plain busybox-style image)

Microscaling-in-a-Box produces a Docker Compose file for you with all these containers. You can substitute your own containers for background and consumer via the UI (and if you want to change the producer container, it's easy to edit the Docker Compose file accordingly). 

The Microscaling agent uses control theory to change the mix of consumer and background containers on the machine, with the goal of maintaining a target NSQ queue length that you specify (default 50 messages).

What can you use this release for?

If you have two microservices communicating using NSQ you can use this release to try regulating them to process the queue smoothly. This is still early code so we would recommend caution - it's may not be wise to roll it straight out to production just yet.   

What will you see?

In this video you can see Microscaling controlling consumer ("Priority1") and background ("Priority2") containers to maintain a queue length of 50.

What's the architectural approach?

The approach of separating microservices with a queue is a very simple, effective and common architecture that "decouples" the two microservices so they operate independently.

  • The producer does some work and puts all the information about what it wants to happen next in a message 
  • It puts the message on a queue
  • The consumer picks up the message from the queue and processes it in its own time.
  • The queue is responsible for making sure the message is reliably stored until it's processed. 
  • The queue itself manages message ordering.
This is a good architecture because if the consumer gets busy and cannot process the messages quickly enough that doesn't hold up the producer - the queue just gets longer. 

If the queue starts to get too long you currently have three options
  • Just let the queue get longer and hope that eventually the consumer catches up
  • Notice the long queue and manually spin up additional consumer instances 
  • Set up autoscaling to automatically provision additional cloud VM resources for the consumer when the queue gets long. The problem is that VMs take several minutes to spin up, so it can be hard to set the autoscaling triggers correctly without excessive over-provisioning


Microscaling gives you another option. There may be something running on your system that isn’t time-critical to your business (let’s call this Batch1) or there may even just be unallocated resources in your infrastructure “just in case”.

Microscaling can spot your queue is getting too long and can, in real time
  • scale up consumer instances on any unallocated resources
  • scale down Batch1 instances to free up space, then scale up consumer instances on the newly freed space
  • notice when the queue is back under control and scale down the consumer instances and restart any stopped Batch1 instances
If Microscaling notices that you still can’t keep up it will (in a future release) trigger autoscaling if required, but by reacting more quickly Microscaling can usually just smooth out the performance of your microservices and stop big queue backlogs from developing. Like flying a plane, it’s better to make small adjustments as you go along rather than big changes after a problem has happened, which can easily make you crash!

Of course, this all relies on your microservices being scalable and replaceable (cattle not pets).

The future

NSQ is just one sort of queue - the same approach can be taken with any queue mechanism like RabbitMQ, or queues on AWS or Azure. And of course there are other metrics we can monitor to keep track of how well a given microservice is performing. We'll be adding support for more metrics and target types - but if you can't wait we welcome contributions (here's the agent code on GitHub).  

We're also currently scheduling containers simply through the Docker API, so this should work for Docker Swarm as well as Docker on a single box. We've previously done demos on other schedulers and we'll be adding support for this into Microscaling too.  Again, we'd love your input on this, whether it's feedback on our roadmap or contributions in code. 

There's also work to be done on supporting more different containers within the same deployment, and on tuning the control parameters.

Stay abreast of all the latest developments by following us on Twitter or starring us on GitHub: 

Thursday, 7 April 2016

Force12 is changing its name to Microscaling Systems!

As we've been demoing the Microscaling concept and talking about it at conferences, we've realised it has a lot of potential as a product. It also works well with the microservices approach many organisations are adopting. 

We're a literal bunch, and wanted a name that makes what we do clearer. We're building a product for microscaling systems - so that’s what we should be called. 

Therefore is becoming Microscaling Systems. That’s μS for short.

You'll find us at, and @microscaling on Twitter

Friday, 1 April 2016

Microscaling at //build/

Here's the video from the Microsoft //build/ event about Azure Container Service - featuring microscaling on Mesos/Marathon, starting at 52:00!