The Challenge

As developers, we deal with lots of complexity, and this is a good thing. It forces us to be creative, and sometimes to go beyond our known universe to overcome challenges.

Microsoft Azure is designed to help us make the right choices. It imposes performance targets through a multitude of mechanisms like throttling and quotas. One of which, I’m sure you have come to know, is that we cannot scale a Cloud Service to zero instances. Let’s stop for a moment and think about this limitation for a second. How would you creatively overcome this challenge?

Moving a Service to Microsoft Azure

Having written about lifting and shifting a Console Application, I decided to mirror the scenario and describe how a similar method can be used to lift and shift a Windows Service to Microsoft Azure.

Every time I dig a little deeper into Azure, I’m amazed at how much there is to know. Having done a few projects with Cloud Services in the past, I thought it would be interesting to see if it was possible to lift and shift a Windows Service into an Azure Worker Role.

Lift and Shift: The action of moving a workload to a new environment, without altering the application’s code.

Opening Ports on Cloud Services

There are scenarios that warrants us to open ports of the Windows Firewall. Imagine an application that coordinates work across many compute nodes. The workload coordinator needs to know and manage each compute node.

The diagram above, depicts a Virtual Machine that has direct access to instances of a Cloud Service without going through a Load Balancer. Both the Virtual Machine and Cloud Service are deployed to a Virtual Network on Microsoft Azure. Using Internal Endpoints in this scenario, would not yield the desired configuration. Endpoints are defined at the Cloud Service boundary and are IP addresses belong to the Data Center’s Network Address Space. Therefore, In order to allow the Virtual Machine to communicate over specific ports to individual Cloud Service instances, we need to use PowerShell and a Startup Task to configure the Windows Firewall.

Configuring Internal Endpoints

There are many scenarios where Internal Endpoints make sense. Imagine a web application that needs to communicate with a middle tier. The communication between both Cloud Services does not need to leave Microsoft Azure networks. The diagram above depicts a scenario where a Web Role has access to a middle tier Cloud Service without going through the public Internet.

Role instances running in a Cloud Service on Microsoft Azure communicate through internal and external connections that vary depending on the type of communication that is needed. An internal endpoint can connect by using a protocol of HTTP, TCP or UDP.

The configuration of an Internal Endpoint is done through the Service Definition. Below is the template used to describe what can be done in terms of Endpoint configurations. 

Configuring Elevated Privileges

There are various scenarios that require us to execute applications with elevated privileges. It’s common for this requirement to surface during lift and shift efforts.

This configuration is done in the Service Definition file. It requires us to specify the Runtime Execution Context to elevated.

Runtime Configuration Template

This template uses the ProgramEntryPoint as an EntryPoint. Roles also support NetFxEntryPoint.

<Runtime executionContext="[limited|elevated]">
   <Environment>
     <Variable name="<variable-name>" value="<variable-value>">
      <RoleInstanceValue xpath="<xpath-to-role-environment-settings>"/>
    </Variable>
  </Environment>
  <EntryPoint>
     <ProgramEntryPoint commandLine="worker.cmd" setReadyOnProcessStart="true" />
   </EntryPoint>
</Runtime>

Scaling Azure Cloud Services

When we build applications on the cloud, we usually favor scaling-out over scaling-up. Consequently, scaling in and out is supported out of the box on Microsoft Azure. On rare occasions, there is a business need to scale-up during peak hours and to scale-down for the quiet hours. This blog post will show you how to achieve this with the help of Microsoft Azure Automation and with PowerShell.

Keep in mind, that scaling up and down requires us to be creative.

Moving to Microsoft Azure

Every time I dig a little deeper into Azure, I’m amazed at how much there is to know. Having done a few projects with Cloud Services in the past, I thought it would be interesting to see if it was possible to lift and shift a Console Application into an Azure Worker Role.

Lift and Shift: The action of moving a workload to a new environment, without altering the application’s code.

Don’t Frown on CSVs

In Microsoft Azure (Azure) CSV and Avro can help you deal with unpredictable amounts of data.

CSV files are surprisingly compact. They compresses really well and allows us to work with datasets that do not fit in RAM. This low-tech solution is often overlooked and frowned upon by developers who don’t get the opportunity to work with very large datasets.

Root cause analysis scenarios have led me to comb through several days’ worth of logs. More often than not, this represents gigabytes worth of data. Exporting application logs to a CSV files, I was able to parse and analyze them with minimal resources.

With these two options available to us, why should we consider using the CSVs? Well, Avro is still fairly new and unsupported by most systems. CSVs can be imported into Databases, Azure Table Storage, Hadoop (HDInsight), ERPs… And a slew of other systems with minimal effort. Heck, you can even open CSV files in Microsoft Excel!

Are Cloud Services Still Relevant?

Recently, Windows Azure Webjobs were introduced to Windows Azure Web Sites. Webjobs are background tasks that come in 3 different flavors (Manual, Triggered and Continuous). These jobs can be used to execute tasks such as database maintenance, RSS aggregation and queue processing. As new features rollout for Windows Azure Web Sites, it’s reasonable to ask whether Cloud Services are still relevant.

I see Cloud Services as a flexible scale unit, where I can spread compute intensive workloads over an echo-system of Role instances. As always, with great power (flexibility) comes great responsibility (we need to care about details).

Fixing the Microsoft.ApplicationServer.Caching.ConfigStoreException

Today I came across the ConfigStoreException. My Worker Roles were constantly being recycled by Windows Azure and I wasn’t able to use the remote debugger to get at any information. (Using Windows Azure SDK 2.2)

To be fair, the Worker Roles ran fine on my local Windows Azure Emulator. As you can imagine, I was completely baffled by all this… So I decided to check if Windows Azure Diagnostics was able to log anything. Fortunately, I was able to get two exceptions. (The exceptions were found in the WADWindowsEventLogsTable located in the Windows Azure Storage Account used for your Windows Azure Diagnostics)