Benefit of Using Directories in Windows Azure Blob Storage Service

June 11, 2013 — 1 Comment

I’ve been playing around with Windows Azure Blob Storage Service for a while now and I have noticed that using directories greatly facilitates debugging and maintenance.

Imagine a Container that contains 400 000 block blobs. The Windows Azure tools in Visual Studio 2012 allow you to list 100 000 blobs. When this limit is reached you are notified and there is nothing you can do to continue listing. At that moment you can use other tools like CloudBerry Explorer and Azure Management Studio. Both tools will allow you to browse the full list of blobs.

Listing 100 000 takes a considerable amount of time. Visual Studio lists blobs as a flat list, which isn’t very useful when you surpass 1000 blobs. CloudBerry Explorer and Azure Management Studio allow you to browse by directory.

The following blob contains information about the location of Microsoft Canada.

quebec-montreal-microsoft-canada-corporation.json

By its name, we know that this will be listed in the container root. Directories are built into Windows Azure Blob Storage Service. Enabling directory listing for this blob, we need to tweak its name.

quebec/montreal/microsoft-canada-corporation.json

Using the ‘/‘ in the blob name, we are creating a hierarchy that can be used to retrieve blobs by browsing through directories.

Listing the contents of the container using CloudBerry Explorer or Azure Management Studio will list the root directory from the blob name. In this case it will list "quebec". Listing the contents of this directory will result in a list of directories containing "montreal". The microsoft-canada-corporation.json blob will be accessible from this last directory.

Listing using directories

 

Listing without directories


As shown above, the directory tree is part of the blob name.

Using directories doesn’t change anything when you are accessing blobs directly. But if you need to browse or filter blobs then directories are the way to go.

Save yourself time by storing

  • Blobs by categories and dates
  • Batches of data in the same directory

By employing this practice, I have been able to save myself countless hours. I no longer have to wait monstrous amounts of time for blobs to be listed when I’m performing manual maintenance checks. I’m now able to browse through my data in a much more structure and efficient manner.

Directories Windows Azure Blob Storage Service are like directories on your hard drive, if they are used properly, they help organize your data. When you have too many files in a directory, listing the contents of the directory will take a reasonable amount of time. In the end, keeping reasonably sized directories will help maintain acceptable performance for code that is required to list contents of directories in order to perform its work.

The more blobs, the more you have to think about building an efficient hierarchy.

Trackbacks and Pingbacks:

  1. Dew Drop – June 12, 2013 (#1,566) | Alvin Ashcraft's Morning Dew - June 12, 2013

    […] Benefit of Using Directories in Windows Azure Blob Storage Service (Alexandre Brisebois) […]

    Like

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s