Temporarily Taking a Cloud Service Role Instances Off of the Load Balancer

By default, public endpoints are load balanced and there are times when we need to take a Role instance off the load balancer. Fortunately, the Windows Azure team has made this possible.

In order to relieve Role instance from too much stress cause by too many requests. You can throttle inbound requests by telling the load balancer that the Role is busy. This can allow the Role to stabilize and get back to normal.

Another wacky scenario could be that you have an active Role that holds a lot of state and that you wanted to create a failover Role instance that would span update and fault domains. To accomplish this, you could use Blob leasing to control which instance is actively receiving inbound requests from the load balancer.

I recently tried toying with the second scenario, but haven’t found it to be as responsive as I would have liked. I will be looking for more patterns and reasons to use this nifty feature!

How does it work?

The Load Balancer polls the Role instance every couple of seconds to know whether it’s busy. If the role is busy, it will stop sending it requests. Once the Load Balancer finds that the Role is available, it will start sending it requests.

In order to communicate the Role’s status to the Load Balancer, you will need to subscribe to the StatusCheck event on the RoleEnvironment. When the event handler is called, you will be able to asses the state of your Role instance and decide whether to call the SetBusy method found on the RoleInstanceStatusCheckedEventArgs. As mentioned above, this will stop inbound requests until the status is checked.

The Code

public class WorkerRole : RoleEntryPoint
{
    public override bool OnStart()
    {
        RoleEnvironment.StatusCheck += RoleEnvironmentStatusCheck;

        return base.OnStart();
    }

    // Use the busy object to indicate that the status of the role instance must be Busy
    private volatile bool busy = true;

    private void RoleEnvironmentStatusCheck(object sender, RoleInstanceStatusCheckEventArgs e)
    {
        if (this.busy)
        {
            // Sets the status of the role instance to Busy for a short interval.
            // If you want the role instance to remain busy, add code to
            // continue to call the SetBusy method
            e.SetBusy();
        }
    }

    public override void Run()
    {
        Trace.TraceInformation("Worker entry point called", "Information");

        while (true)
        {
            Thread.Sleep(10000);
        }
    }

    public override void OnStop()
    {
        base.OnStop();
    }
}

References

• RoleEnvironment.StatusCheck Event
• RoleEnvironment Events