Comparing HTTP and HTTPS URIs

Working with URIs is a regular exercise when we crawl the web. Consequently, comparing HTTP and HTTPS URIs quickly becomes an interesting challenge. Fortunately, .NET has us covered with the Uri.Compare(Uri, Uri, UriComponents, UriFormat, StringComparison) Method

[TestMethod]
public void CompareHttpAndHttpsURIsTest()
{
    var http = new Uri("http://www.microsoft.com");
    var https = new Uri("https://www.microsoft.com");

    var result = Uri.Compare(http,
        https,
        UriComponents.Host | UriComponents.PathAndQuery,
        UriFormat.SafeUnescaped, StringComparison.OrdinalIgnoreCase);

    Assert.IsTrue(result == 0);
}

The next challenge, will probably be around comparing query strings. Because this gets complicated quickly, I’ll leave this one for another time.

Scaling an FTP Ingestion Service

Making an FTP Ingestion Service Highly Available (HA) can be a challenge. On Azure, we can take advantage of the Microsoft Azure Traffic Manager to direct users to the closest FTP Server. In this specific scenario, we assume that all FTP Servers are configured the same way and that users only have write access. When a document is uploaded over FTP, it is moved to a Microsoft Azure Storage Account that is used as persistent storage.

Using PowerShell to Authenticate Against OAuth

From development to deployment, PowerShell is becoming the ‘go to’ automation technology on Microsoft Azure. So, I decided to use PowerShell to perform automated tests against a Web API (a.k.a REST service). These tests are built to run during the execution of a Continuous Release cycle and confirm that the API is responding as expected.

Getting Acquainted With #Azure Service Bus Event Hubs

The Microsoft Azure ecosystem just keeps growing. This week Microsoft unveiled a very welcomed addition to the Microsoft Azure Service Bus. Event Hubs join ranks with Queues, Topics and Relays to offer options adapted to your needs.

Contrasting available Service Bus Flavors?

• Relays – are used to bridge communications over the cloud in a secure and transparent manner.
• Queues – are pipes that allow for many publishes and many consumers to communicate over a single channel. This is great for Competing Consumers and for Queue-based Load Leveling.
• Topics – are pipes that allow fan out scenarios, where each consumer gets his own copy of the inbound queue. It also has some handy features like filters. Use this flavor to implement Pipes and Filters.
• Event Hubs – are a bit more complex. Event Hubs enable the collection of event streams at high throughput, from a diverse set of devices and services. In other words, they help us deal with the 3 Vs.
  • Volume (amount of data)
  • Velocity (speed of data in and out)
  • Variety (range of data types and sources).

Microsoft Azure Service Bus Event Hubs

Event Hub join ranks with Queues, Topics and Relays to offer options adapted to your needs. They province the mechanisms necessary to collection of event streams at high throughput, from a diverse set of devices and services. They are composed of a Published Policy, of Consumer Groups and of Partitions.

Event Hubs support the following scenarios:

• Collecting event streams at high throughput from devices/services for use in real-time and batch processing.
• Connecting millions of devices from diverse platforms for sending data (with individual authentication and flow control).
• Process event streams per device “in order” using several backend services (publish/subscribe).

Considerations Prior to Creating an Event Hub

You must put some effort in capacity planning before you create an Event Hub. In order to make the right decisions let’s go over a couple details about Event Hubs.

Defining an HTTP Transient Error Detection Strategy For REST Calls

public class HttpTransientErrorDetectionStrategy
    : ITransientErrorDetectionStrategy
{
    private readonly List<HttpStatusCode> statusCodes = 
        new List<HttpStatusCode>
        {
            HttpStatusCode.GatewayTimeout,
            HttpStatusCode.RequestTimeout,
            HttpStatusCode.ServiceUnavailable,
        };

    public HttpTransientErrorDetectionStrategy(bool isNotFoundAsTransient = false)
    {
        if (isNotFoundAsTransient)
            statusCodes.Add(HttpStatusCode.NotFound);
    }

    public bool IsTransient(Exception ex)
    {
        var we = ex as WebException;
        if (we == null)
            return false;

        var response = we.Response as HttpWebResponse;

        var isTransient = response != null 
                                   && statusCodes.Contains(response.StatusCode);
        return isTransient;
    }
}

Characteristics of Good REST Services

REST (Representational state transfer) Services play an important role in today’s interconnected systems. By exploiting HTTP, a well-known protocol, they enable applications of all types to easily exploit these service.

Continue Reading...