Using Time-based Partition Keys in #Azure Table Storage

In a previous post about storing Azure Storage Table entities in descending order I combined a time-based key with a guid in order to create a unique key. This is practical when you need to use combined keys for the Row Keys or Partition Key. But it’s not practical for logs.

A better solution for logs, is to generate a Partition Key based on time. This allows you to query for logs by time periods. There are many ways to generate time-based partitions, so I will cover the two that I use the most.

Query Level Caching For Entity Framework And Windows Azure SQL Database Including Transient Fault Handling

Caching with Entity Framework has always been a challenge. Out of the box Entity Framework doesn’t have second level caching support. Even though, many open source solutions like the Scalable Object Persistence (SOP) Framework exist, I decided to implement a query level cache that uses the Transient Fault Handling Application Block to execute retry policies to transparently handle transient faults.

Keep in mind that transient faults are normal and that its not a question of if they will occur, it’s really a question of when they will occur. SQL Database instances are continuously being shifted around to prevent the service’s performance from degrading.

The Database fluent API I created executes reusable queries. Reusable queries greatly simplify the application’s design by encapsulating query logic in named concepts. They are like Stored Procedures in SQL Server, where team members can discover functionality by reading file names from the query folders. More details about reusable queries can be found in the reference section at the bottom of this post.

The code from this Post is part of the Brisebois.WindowsAzure NuGet Package

To install Brisebois.WindowsAzure, run the following command in the Package Manager Console

PM> Install-Package Brisebois.WindowsAzure

Get more details about the Nuget Package.

Querying Over Windows Azure Table Storage Service

Windows Azure Table Storage Service can be queried using various approaches. I use the Windows Azure Storage NuGet package and reusable queries.

A Query on Windows Azure Table Storage Service usually completes in 200 milliseconds. On busy systems this is an eternity! To help my services perform better, and to reduce costs of operation, I built the TableStorageReader to executes queries with cache, in turn reducing the number of transactions and reducing the latencies created by communications over the network.

The reader executes reusable queries. Reusable queries greatly simplify the application’s design by encapsulating query logic in named concepts. They are like Stored Procedures in SQL Server, where team members can discover functionality by reading file names from the query folders. More details about reusable queries in the reference section at the bottom of this post.

The TableStorageReader is a fluent api that allows you to create tables, query tables and apply cache over queries.

The code from this Post is part of the Brisebois.WindowsAzure NuGet Package

To install Brisebois.WindowsAzure, run the following command in the Package Manager Console

PM> Install-Package Brisebois.WindowsAzure

Get more details about the Nuget Package.

Test Driving Windows Azure Media Services

Recently Microsoft released Windows Azure Media Services preview, which essentially is a Platform as a Service that empowers developers to build workflows for the creation, management, and distribution of media.

This post is a retrospective of my first experience with this new service.

Build Reusable, Testable Queries Part 2

Query encapsulation can become quite empowering. For instance, query objects can be decorated, extended and reused. They allow us to implement concepts like targeted caching or user defined queries. They even allow us to execute the same query on two different data sources.

In part 2 of this series, we will be looking at how we can leverage the IModelQuery interface to enable us to reuse queries.

Build Reusable, Testable Queries Part 1

Queries are complicated to test and nearly impossible to reuse. In systems of all sizes they get duplicated,  modified and augmented to the point where we are afraid to modify them. They are essentially questions about data that generate information upon which we can act. They should be reusable. They should be predictable and above all, they should be testable.