Tuesday, 12 April 2016

Microscaling with an NSQ Queue

Today we're launching a release of Microscaling-in-a-Box that scales to maintain the length of an NSQ queue. With this, we're moving out of the realms of the theoretical to something that is genuinely useful for a real world scenario.

Microscaling makes a system's architecture dynamic – able to respond to real demand in real time. With Microscaling-in-a-Box you can now scale the number of containers that process jobs on a queue, so that you always have an appropriate number to cope with the rate that jobs are arriving on the queue. 

What's in this release?

This release has everything you need to run Microscaling on a single machine on which you have installed Docker.  You'll get the following containers: 
  • an example message producer, which simulates external demand and adds items to the queue
  • an NSQ Server which maintains the queue
  • a Microscaling agent
  • an example message consumer, which takes items off the queue
  • an example low priority background process (defaults to a plain busybox-style image)

Microscaling-in-a-Box produces a Docker Compose file for you with all these containers. You can substitute your own containers for background and consumer via the UI (and if you want to change the producer container, it's easy to edit the Docker Compose file accordingly). 

The Microscaling agent uses control theory to change the mix of consumer and background containers on the machine, with the goal of maintaining a target NSQ queue length that you specify (default 50 messages).

What can you use this release for?

If you have two microservices communicating using NSQ you can use this release to try regulating them to process the queue smoothly. This is still early code so we would recommend caution - it's may not be wise to roll it straight out to production just yet.   

What will you see?

In this video you can see Microscaling controlling consumer ("Priority1") and background ("Priority2") containers to maintain a queue length of 50.

What's the architectural approach?

The approach of separating microservices with a queue is a very simple, effective and common architecture that "decouples" the two microservices so they operate independently.

  • The producer does some work and puts all the information about what it wants to happen next in a message 
  • It puts the message on a queue
  • The consumer picks up the message from the queue and processes it in its own time.
  • The queue is responsible for making sure the message is reliably stored until it's processed. 
  • The queue itself manages message ordering.
This is a good architecture because if the consumer gets busy and cannot process the messages quickly enough that doesn't hold up the producer - the queue just gets longer. 

If the queue starts to get too long you currently have three options
  • Just let the queue get longer and hope that eventually the consumer catches up
  • Notice the long queue and manually spin up additional consumer instances 
  • Set up autoscaling to automatically provision additional cloud VM resources for the consumer when the queue gets long. The problem is that VMs take several minutes to spin up, so it can be hard to set the autoscaling triggers correctly without excessive over-provisioning


Microscaling gives you another option. There may be something running on your system that isn’t time-critical to your business (let’s call this Batch1) or there may even just be unallocated resources in your infrastructure “just in case”.

Microscaling can spot your queue is getting too long and can, in real time
  • scale up consumer instances on any unallocated resources
  • scale down Batch1 instances to free up space, then scale up consumer instances on the newly freed space
  • notice when the queue is back under control and scale down the consumer instances and restart any stopped Batch1 instances
If Microscaling notices that you still can’t keep up it will (in a future release) trigger autoscaling if required, but by reacting more quickly Microscaling can usually just smooth out the performance of your microservices and stop big queue backlogs from developing. Like flying a plane, it’s better to make small adjustments as you go along rather than big changes after a problem has happened, which can easily make you crash!

Of course, this all relies on your microservices being scalable and replaceable (cattle not pets).

The future

NSQ is just one sort of queue - the same approach can be taken with any queue mechanism like RabbitMQ, or queues on AWS or Azure. And of course there are other metrics we can monitor to keep track of how well a given microservice is performing. We'll be adding support for more metrics and target types - but if you can't wait we welcome contributions (here's the agent code on GitHub).  

We're also currently scheduling containers simply through the Docker API, so this should work for Docker Swarm as well as Docker on a single box. We've previously done demos on other schedulers and we'll be adding support for this into Microscaling too.  Again, we'd love your input on this, whether it's feedback on our roadmap or contributions in code. 

There's also work to be done on supporting more different containers within the same deployment, and on tuning the control parameters.

Stay abreast of all the latest developments by following us on Twitter or starring us on GitHub: 

No comments:

Post a Comment