0727_WEB_a_Growth_t618

Getting Acquainted With #Azure Service Bus Event Hubs

The Microsoft Azure ecosystem just keeps growing. This week Microsoft unveiled a very welcomed addition to the Microsoft Azure Service Bus. Event Hubs join ranks with Queues, Topics and Relays to offer options adapted to your needs.

Contrasting available Service Bus Flavors?

  • Relays – are used to bridge communications over the cloud in a secure and transparent manner.
  • Queues – are pipes that allow for many publishes and many consumers to communicate over a single channel. This is great for Competing Consumers and for Queue-based Load Leveling.
  • Topics – are pipes that allow fan out scenarios, where each consumer gets his own copy of the inbound queue. It also has some handy features like filters. Use this flavor to implement Pipes and Filters.
  • Event Hubs – are a bit more complex. Event Hubs enable the collection of event streams at high throughput, from a diverse set of devices and services. In other words, they help us deal with the 3 Vs.
    • Volume (amount of data)
    • Velocity (speed of data in and out)
    • Variety (range of data types and sources).

Microsoft Azure Service Bus Event Hubs

Event Hub join ranks with Queues, Topics and Relays to offer options adapted to your needs. They province the mechanisms necessary to collection of event streams at high throughput, from a diverse set of devices and services. They are composed of a Published Policy, of Consumer Groups and of Partitions.

Event Hubs support the following scenarios:

  • Collecting event streams at high throughput from devices/services for use in real-time and batch processing.
  • Connecting millions of devices from diverse platforms for sending data (with individual authentication and flow control).
  • Process event streams per device “in order” using several backend services (publish/subscribe).

Considerations Prior to Creating an Event Hub

You must put some effort in capacity planning before you create an Event Hub. In order to make the right decisions let’s go over a couple details about Event Hubs. Continue Reading…

microsoft-mvp

Microsoft Azure MVP (2014)

Being recognized for something I’m so passionate about is amazing! The past year has truly been rewarding. It was a year of change, opportunity and challenges.

I’m honored to receive the 2014 Microsoft Azure MVP award.

I joined the Microsoft Azure MVPs last July and was welcomed by a group of remarkable individuals. Inspired by our energetic community, I strive to learn and share about what it means to build applications for the cloud.

2013 was a year of change for the Microsoft Azure community. Services, Partners and Technologies have made their way to the cloud. Consumers have a rich ecosystem of services with which to compose their solutions. These are very exciting times for developers who tinker with the cloud. Competition, creativity and business opportunities create the perfect setting for a fast pace evolution. I really think that we’re only seeing the tip of the iceberg, 2014 is going inspiring!

I’d like to thank everyone who contributed over the past year. I’m very grateful for your questions, insights and opinions. The adventure has been very interesting and I truly hope that you will continue to be part of it.

Thank you for this exceptional award.

The Microsoft Most Valuable Professional (MVP) Award is our way of saying thank you to exceptional, independent community leaders who share their passion, technical expertise, and real-world knowledge of Microsoft products with others. It is part of Microsoft’s commitment to supporting and enriching technical communities. Even before the rises of the Internet and social media, people have come together to willingly offer their ideas and best practices in technical communities.

Microsoft MVP web site

clouds

Microsoft Azure is a vast ecosystem of services that keeps on growing. As a new comer to this ecosystem, you may feel overwhelmed. But don’t worry, I wrote this post to answer questions like “where should I start?” and “What resources are available to me?”

Where Should I Start?

This is often the hardest thing to identity when we start tinkering with Azure. I started exploring Azure by reading free e-books that you can download from the Patterns & Practices website.

The first books I read were

Then I read Cloud Design Patterns. The Patterns & Practices team has also written a book about CQRS Journey, which can greatly influence your overall design.

If you’re looking to buy a book, I highly recommend Cloud Architecture Patterns: Using Microsoft Azure, a book written by Bill Wilder. This should probably be the first book you read. It’s packed with tons of valuable information that applies to most cloud vendors. Continue Reading…

nasa-mission-black-holes-nustar-crab-nebul

#Azure Storage Tables – DateTime.MinValue is not Within the Supported DateTime Range

Azure Table Storage is a NoSQL storage key/value based part of Microsoft Azure Storage services. It’s really good at absorbing massive amounts of data. It’s really good at massive parallel operations work on small amounts of data. But it’s horrible when it comes time to extract large amounts of data in a serial manner. We’ll get to that topic in a future post.

DateTime is a type that is used to represent time in .NET. We use it very liberally without much afterthought. But when we start playing with Azure Table Storage, we have to think about the supported DateTime range. At this time, it’s also important to note that the local Azure Storage Emulators run on various database flavors like SQL Server and LocalDB. These databases do not impose the same limitations for DateTime values. Therefore, this bug will only show up on Microsoft Azure.

In order to limit headaches, I’m providing the following table to help identify what is supported and what isn’t.

Common Language Runtime type Details
byte[] An array of bytes up to 64 KB in size.
bool A Boolean value.
DateTime A 64-bit value expressed as Coordinated Universal Time (UTC). The supported DateTime range begins from 12:00 midnight, January 1, 1601 A.D. (C.E.), UTC. The range ends at December 31, 9999.
double A 64-bit floating point value.
Guid A 128-bit globally unique identifier.
Int32 or int A 32-bit integer.
Int64 or long A 64-bit integer.
String A UTF-16-encoded value. String values may be up to 64 KB in size.

Find out More about Azure Storage

time distortion

Using Time-based Partition Keys in #Azure Table Storage

In a previous post about storing Azure Storage Table entities in descending order I combined a time-based key with a guid in order to create a unique key. This is practical when you need to use combined keys for the Row Keys or Partition Key. But it’s not practical for logs.

A better solution for logs, is to generate a Partition Key based on time. This allows you to query for logs by time periods. There are many ways to generate time-based partitions, so I will cover the two that I use the most. Continue Reading…

flying blind

Lessons Learned – On #Azure, Log Everything!


Log everything, I mean it! If it wasn’t logged, it never happened.

So you’re probably thinking “doesn’t logging tax an application’s performance?” Absolutely, some logging frameworks can even bring down your application under load. So you really need to be careful about what you log. Therefore, we should log everything that can help us figure out what went horribly wrong. I used the past tense, because on the cloud it’s normal to fail. Never build an application without thinking of Mean Time to Recovery. In other words, how will you recover and how long will it take. Sometimes, when you start on a fresh project, you can only plan for the obvious cases. That means that we have a lot to learn from a brand new application. So in order to facilitate this learning process, I urge you to log anything that can help the DevOps that will get a phone call late at night. Who knows… you might be the one supporting this application.

Take a moment and think about your system. What would you need to know, to be able to identify what went wrong?

Running applications on the cloud without meaningful logs is like flying an airplane without windows or instruments. We know we’re going somewhere… well we think we’re going somewhere. But really where are we going? Is the engine on? Are we climbing or descending? How high are we?

Are these questions making you uneasy?

Let’s think about our applications, do we really know what’s going on? Sure we have performance counters around CPU, Network, Memory and Disk utilization. But what kind of information does that really provide about our application? Knowing that a Virtual Machine (VM) is running, is like being on an airplane without knowing where we’re going. Having meaningful logs provides us with the insights required to know where we’re going. Continue Reading…

Speaker Idol

Speaker Idol 2014 (Lundi 16 Juin 2014)

Grand retour du concours Speaker Idol version 2014 avec de nombreux prix pour les participants!!!  Pour le dernier meeting de l’année nous vous invitons à venir présenter votre techno préférée, votre librairie fétiche, votre projet open source innovateur ou tout autre sujet touchant le développement logiciel.  En fait, le choix de la techno n’est pas si important, ce qui l’est c’est de gagner de l’expérience pour présenter un sujet technique.  C’est un “soft skill” qui est primordial dans le développement de votre carrière.  En effet, vous aurez à faire des présentations à des clients ou à vos patrons.  Une présentation bâclée ou mal présentée peux être un frein à un projet, un vente ou même une promotion.  Rappelez-vous que ce qui sera jugé sera votre présentation et non la techno que vous présenter.

Le gagnant/la gagnante aura la possibilité de faire une présentation complète d’une heure la prochaine saison.

Nous vous suggérons de visionner une formation gratuite de Plurasight qui se nomme “Get Involved” par Scott Hanselman et Jeff Attwood: http://getinvolved.hanselman.com

Ainsi que:

Les juges pour la soirée:

  • Etienne Tremblay
  • Richard Dinel
  • Chantal Bossé

Continue Reading…

Microsoft Azure

Samedi le 7 juin 2014, nous avons eu le plaisir d’organiser le Azure Camp Redux dans les bureaux de Microsoft à Montréal. La journée fut un success et les questions étaient au rendez-vous. Visitez le site de la Communauté .NET Montréal pour voir les commentaires de la journée.

Grâce à la complicité de Microsoft qui commandite cet événement, il n’y a pas de frais pour y assister.  Il n’est pas nécessaire d’être membre de la Communauté .NET Montréal et d’avoir payé le membership annuel quoique nous aimerions bien que vous le soyez, après tout, ce n’est que $25 pour une année chargée de meetings.  Allez donc!  Devenez membre en payant votre membership en ligne :-)

Azure Camp – Redux (Samedi 7 Juin 2014)

Le Azure Camp est une journée où vous pouvez vous familiariser avec le Cloud et la plateforme Microsoft Azure.  En avant-midi, vous assisterez à 3 présentations et en après-midi, vous mettrez le tout en pratique durant la période de hacking.

À qui s’adresse cette journée?  À tout développeur Web (PHP, Java, Ruby, Python, Node.js, .NET) désirant tirer avantage du Cloud et faire ses premiers pas avec Azure.  Oui oui, Azure ce n’est pas que pour les développeurs .NET!  Notez que c’est une journée d’introduction destinée aux devs qui n’ont pas encore vu de présentation ou testé Azure.

Rami Sayar de Microsoft sera sur place afin de répondre à vos questions.

Les présentations seront en français avec du matériel anglais.

Continue Reading…

coffee_cup

What fuels your development efforts?

I recently came to the aggravating conclusion that coffee just doesn’t do it for me. Coffee gives me a quick boost and the crash that follows is impressive!

The day after a coffee boost usually results in me being more tired. Hence the start of a vicious cycle. Coffee to start me up and hide the fact that I’m getting very tired.

This realization got me thinking about other tricks that we use as developers to fuel our development efforts.

Some of us are heavy energy drink users, some drink tea and others binge on sugar. Let’s not forget other substances like alcohol that can stimulate creativity or slow us down enough to concentrate on what we’re doing.

Throughout my career I’ve tried a couple of these and I’m curious to hear about your personal experiences. Continue Reading…

picking a direction

How do you build features on #Azure?

For the past few months my team and I have been using git-flow to build a new REST API for an existing cloud native application. The challenge was impressive, but this is how we did it.

Our team is spread across many geographical locations. We use a mix & match of best practices to help us communicate. For example, we use Skype, it’s on all our devices (phones, tables, laptops and desktops…) and makes collaboration easy! Our code lives in Git and is continuously built and released to our integration environment for validation.

Our team isn’t unique. Many of my colleagues work in distributed teams. Surprisingly, we all share the same common challenges. Time zones, cultural differences and language barriers are now part of each decision we make. Fortunately we all speak a common language, C#.

With this added complexity, how did we go to production on time? We implemented a process that we continuously tweak and adjust to meet our goals. Continue Reading…