Size Matters… A LOT!
A few months ago I wrote about resource contention and how the size of data matters on the cloud. Today I decided to share a true story that happened to a friend of mine.
Things that work during development and quality assurance can yield unexpected results at scale. It’s no surprise that we often say that Microsoft Azure is there to help your applications survive instant success. Going from 1 to 100 000 users overnight is a problem that we all dream to have!
Let’s look at the following line item from my friend’s bill.
Last month, the bandwidth generated by his apps totaled 783 Gigabytes. To some this number looks really small, to others this number looks astronomical. So let’s put some perspective on this story.
The apps that consumed this data, were mobile apps. Think about it for a second, that’s a lot of mobile devices! The data in question was Json. That’s not what you expected right? So how did so much Json go over the wire? Well whenever an app started, it would download a Json file from Azure Blob Storage that was then used to locate resources to display. Each Json file wasn’t that big, but the sheer success of his apps generated a staggering amount of traffic.
To be honest, this is the kind of surprise we all dream of! We all want our apps to go viral, but are we ready?
In this specific situation, there are a couple of things that can be done to reduce the app’s monthly bandwidth requirements.
- Setting Cache-Control on blobs can help reduce bandwidth
- Using ETags to identify whether the blob on the server has changed since it was last downloaded.
- Use a different serialization protocol. Protobuf could reduce the overall bandwidth consumption generated by his apps.
- Use Json minification to reduce the size of each blob. Remember blank spaces eat precious bytes.
- Partition the data into smaller blobs, my guess is that most users will not stray far from the first pages.
- If we can’t partition the blobs we might want to consider the use of compression.
Alexandre Brisebois ☁ 