Size Matters… A LOT!

A few months ago I wrote about resource contention and how the size of data matters on the cloud. Today I decided to share a true story that happened to a friend of mine.

Things that work during development and quality assurance can yield unexpected results at scale. It’s no surprise that we often say that Microsoft Azure is there to help your applications survive instant success. Going from 1 to 100 000 users overnight is a problem that we all dream to have!

Let’s look at the following line item from my friend’s bill.

Last month, the bandwidth generated by his apps totaled 783 Gigabytes. To some this number looks really small, to others this number looks astronomical. So let’s put some perspective on this story.

The apps that consumed this data, were mobile apps. Think about it for a second, that’s a lot of mobile devices! The data in question was Json. That’s not what you expected right? So how did so much Json go over the wire? Well whenever an app started, it would download a Json file from Azure Blob Storage that was then used to locate resources to display. Each Json file wasn’t that big, but the sheer success of his apps generated a staggering amount of traffic.

To be honest, this is the kind of surprise we all dream of! We all want our apps to go viral, but are we ready?

In this specific situation, there are a couple of things that can be done to reduce the app’s monthly bandwidth requirements.

  1. Setting Cache-Control on blobs can help reduce bandwidth
  2. Using ETags to identify whether the blob on the server has changed since it was last downloaded.
  3. Use a different serialization protocol. Protobuf could reduce the overall bandwidth consumption generated by his apps.
  4. Use Json minification to reduce the size of each blob. Remember blank spaces eat precious bytes.
  5. Partition the data into smaller blobs, my guess is that most users will not stray far from the first pages.
  6. If we can’t partition the blobs we might want to consider the use of compression.


Trackbacks and Pingbacks:

  1. End of Month Research Roundup – April 2014 | endjin blog - May 1, 2014

    […] Size Matters… A LOT! (Alexandre Brisebois) […]


Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

This site uses Akismet to reduce spam. Learn how your comment data is processed.