To keep things simple we haven't set up service discovery on the Mesos cluster. Instead we're using an Azure Storage Queue that all the containers can access.
For the demo we'll be creating 4 Marathon Apps that run as Docker containers.
- Producer - containers add items to the queue. We start 3 producers on startup and you can scale these manually using the Marathon UI.
- Consumer - containers remove items from the queue and are scaled by the Microscaling Engine.
- Remainder - any spare capacity is used for this background task. You can change the Docker image to use your own using the Marathon UI.
- Microscaling - the engine scales the demo and sends data to our Microscaling-in-a-Box site.
Azure Storage Account
You'll need an Azure Storage Account and access keys to create the queue.
- If you don’t currently have an Azure subscription you can get a free-trial.
- Sign in to the Azure Portal.
- Navigate to New -> Data + Storage -> Storage Account
- Create a storage account with these settings.
- Name - this must be globally unique across all Azure Storage Accounts.
- Replication - make sure you choose Locally-redundant storage for the queue.
- Resource Group - create a new resource group for the queue.
- Once the storage account has been created navigate to Settings -> Access Keys and make a note of your access keys.
Launch ACS Cluster
- See this tutorial for launching an ACS cluster.
- In Basics screen set the following
- USERNAME - e.g. "azureuser". You'll need this for connecting to the cluster via SSH.
- SSH PUBLIC KEY - also needed for SSH access.
- RESOURCE GROUP - choose a name for your cluster.
- In ACS Settings screen set the following
- AGENT COUNT - increase from 1 to 2.
- DNS PREFIX - choose a name for your cluster.
Open SSH tunnel
- To access the Marathon REST API and the DC/OS web UIs you need to open an SSH tunnel on port 80 to the master node.
Create Marathon Apps
- You're now ready to install the Marathon demo apps. Please sign up for a Microscaling-in-a-Box account if you don't already have one.
- On the Start page choose Mesos / Marathon.
- The Install page links to this post so you can continue to the Run page.
- On the Run page you'll see the commands for installing the Marathon Apps. The marathon-install command needs Ruby but has no other dependencies.
- For MSS_MARATHON_API use http://localhost/marathon if you're using ACS.
- You'll need to also set AZURE_STORAGE_ACCOUNT_NAME and AZURE_STORAGE_ACCOUNT_KEY with the access keys you created earlier.
View The Demo
- Once Marathon has launched the apps the results will appear in the Microscaling-in-a-Box site. You'll see the Microscaling Engine scaling the consumer and remainder containers to maintain the target queue length.
Uninstall the Marathon Apps
- You can use the marathon-uninstall command to remove the demo apps from your cluster.
Delete the Azure Resources
Important: You'll be charged for running the Azure VMs and any data stored on the Azure Queue. So its important to delete these when you don't need them anymore.
- Sign in to the Azure Portal.
- Select Resource Groups from the left hand menu.
- Find and delete the 2 Resource Groups you created for the ACS cluster and the Azure Queue.
As well as supporting Marathon and Azure Storage Queues we also support the Docker API and NSQ. We'll be adding support for more queues and container orchestrators.
Stay abreast of all the latest developments by following us on Twitter or starring us on GitHub: