Using StartsWith to Filter on RowKeys

There are many scenarios where filtering on partial RowKeys makes sense. One of these scenarios is Azure Diagnostics Log analysis where events are partitioned by time based PartitionKeys and by compound RowKeys. This allows us to filter and find information effectively.

Event RowKeys are composed of deployment IDs, role names, instance names, categories and other information:

8637d014bcf94452a1e48f393a11674b___Brisebois.WorkerRole___Brisebois.WorkerRole_IN_0___0000000001652031520___WADLogsLocalQuery

Querying WADLogsTable Effectively

The following example, shows how to target a specific table partition and filter events based on a StartsWith pattern.

var storageAccount = Microsoft.WindowsAzure.Storage.CloudStorageAccount.Parse(connectionString);
var client = storageAccount.CreateCloudTableClient();

var table = client.GetTableReference("WADLogsTable");

// Querying Windows Azure Diagnostics by Partition for a partial RowKey
var query = new FindWithinPartitionStartsWithByRowKey("0635204061600000000", "8637d014bcf94452a");
var result = query.Execute(table);

Using Time-based Partition Keys in #Azure Table Storage

In a previous post about storing Azure Storage Table entities in descending order I combined a time-based key with a guid in order to create a unique key. This is practical when you need to use combined keys for the Row Keys or Partition Key. But it’s not practical for logs.

A better solution for logs, is to generate a Partition Key based on time. This allows you to query for logs by time periods. There are many ways to generate time-based partitions, so I will cover the two that I use the most.