Excluding Compiler Generated Code from the NDepend Dependency Matrix

Using the Dependency Matrix generated by NDepend I recently noticed that it included compiler generated code which isn’t of interest to me at this point in time. I was looking for a way to exclude it from the solutions’ Dependency Matrix. Fortunately, Patrick from NDepend team provided me with a great answer on Stackoverflow.

What is NDepend?

NDepend is a Visual Studio tool to manage complex .NET code and achieve high Code Quality. With NDepend, software quality can be measured using Code Metrics, visualized using Graphs and Treemaps, and enforced using standard and custom Rules.
Hence the software design becomes concrete, code reviews are effective, large refactoring are easy and evolution is mastered.

Windows Azure SQL Data Sync – Is This Service Right For Your Database?

I recently tried to use Windows Azure SQL Data Sync to replicate the data from my production database to a staging SQL Database (located in the same datacenter as the production SQL Database) instance and also to an
on-premise SQL Server Database instance.

Every 24 hours, the application inserts 10 to 40 million rows and removes all stale data from the tables. As you can imagine, this generates quite a large change-set for Windows Azure SQL Data Sync. In summary, the volume of data is quite important and it is short-lived.

Windows Azure Blob Storage Service – Migrating Blobs Between Accounts

I recently created a Windows Azure Storage Account without an Affinity Group and was faced with the task of migrating the data to a newly created Windows Azure Storage Account which belongs to the Affinity Group used by the project’s Cloud Services.

By physically grouping Cloud and Storage services together, Affinity Groups contribute to the overall performance of your application by reducing network latency. Consequently it reduces the amount of time a CPU remains idle waiting for files to process. It may not seem important at first, but it makes a world of difference for compute intensive worker roles who depend on Windows Azure Storage for IO.

Migrating data can be achieved in a couple of ways

• Download the data and upload the data to the new Storage Account. (This is by far the option that takes the longest time)
• Use tools like Azure Management Studio (Some of these solutions cost money)
• You can make your own tool

Windows Azure SQL Data Sync – Optimizing The Initial Sync Process For Large Data Sets

Today I setup SQL Data Sync to synchronize
30 million records between 3 database instances.
 
A master database (HUB) needed to be replicated to a staging database hosted in Windows Azure SQL Database and to a second instance hosted on-premises in SQL Server 2008 R2.

Needless to say that I rapidly hit a bottleneck… The initial replication took forever, so I started looking for ways to get it to complete in a timely manner.

The first alternative I came up with, only works when you’re replicating databases within the Windows Azure SQL Database Service. Using the Create Database as Copy of statement as described in "Creating a Staging Database Instance From a Production Instance on Windows Azure SQL Database". I created a copy of my Hub database and proceeded to creating a sync relationship between the Hub database and the newly created copy.

The second alternative also relies on creating a copy of the Hub database. The main difference is that the copy can be created on-premises or in a other datacenters. Using the export functionality, create a backpac of your Hub database and restore (import) it in the environment of your choice. Then create a sync relationship between both database instances.

The prescribed best practice is to start with data in only one database. If you have data in multiple databases,
SQL Data Sync treats each row as a conflict that needs resolution. This causes the initial synchronization to go very slow – taking several days to several months, depending on the database size.

Following this advice, it took close to 5 hours to complete the first synchronization and it took close to 20 hours to complete the synchronization with the on-premise databases. Originally, this is why I looked into making a copy of my database before creating the sync group. 

Creating a copy of my database took close to 4 hours and the initial synchronization took close to 10 hours. The process of synchronizing two exact copies of my database took me about 14 hours instead of the 5 hours it took me when I started off with an empty database. 

References

• Avoid a slow and costly initial synchronization
• SQL Data Sync (Preview) Best Practices

Windows Azure SQL Data Sync – The Maximum Limit For Index or Statistics Key Column List is 16

I recently used SQL Data Sync to sync a Windows Azure SQL Database with an on-premises database instance hosted in SQL Server 2008 R2. When SQL Data Sync tried to create the schema it threw the following exception.

Database provisioning failed with the exception "SqlException Error Code: -2146232060 – SqlError Number:1904, Message: The index ‘XI_IndexName’ on table ‘dbo.TableName’ has 21 column names in index key list. The maximum limit for index or statistics key column list is 16. " For more information, provide tracing ID ‘802aeb36-fa15-****-****-b8acc99125d3’ to customer support.

Confused as to where this exception came from, looked around and found a few posts (listed below) explaining restrictions for creating Indexes. 

Then I realized that my schema has already been created on Windows Azure SQL Database, so I tried publishing the  schema from my Database Project to my on-premises database instance and for some reason, the error was not raised.

Then I reinitiated the data sync process and was able to complete the initial synchronization.

My guess is that by publishing the schema myself, I was able to relieve SQL Data Sync from trying to create the schema and allowed it to directly start synchronizing the data.

 

More Posts About This Limitation

• Creating Indexes (Database Engine)
• SQL Azure Data Sync error – maximum limit for index key column list is 16?
• Any way around unique index 16 column max
• What are the maximum number of Columns per index key,foreign key,primary key in SQL Server
• SQL SERVER – Maximum Columns per Primary Key – Fix : Error : Msg 1904, Level 16, The index on table has column names in index key list. The maximum limit for index or statistics key column list is 16

XMAS Came Early For All Windows Azure Developers!

I’m really excited about today’s (April 30th 2013) Windows Azure SKD version 2.0 release. I take this as an early XMAS gift for all Windows Azure Developers and above all its got tons of awesome goodness!

The first thing I noticed when I upgraded my projects to use the new SDK, was the speed of the Windows Azure Emulator, it just doesn’t compare. What a time saver!

To upgrade your project download the new Windows Azure SDK 2.0 for Visual Studio 2012. Once you have finished installing the project, you will need to update the Windows Azure Configuration Manager through Nuget. Then right click on your cloud project and navigate to the Application tab. Click on the Update button. If for some reason your cloud services will cycle continuously, it might be because the Microsoft.WindowsAzure.ServiceRuntime reference in your projects haven’t been updated,. You can do so manually by referencing the DLL typically found in
Program Files\Microsoft SDKs\Windows Azure\.NET SDK\v2.0\

Consider Using UTC (Coordinated Universal Time) When Designing REST APIs

With an ecosystem of devices that is constantly evolving, it’s hard to predict who will consume your REST APIs and how they will consume them. Devices as we know them are changing shape, consequently applications are constantly adapting to new platforms.

Lets face it, mobile apps are the norm and shouldn’t be ignored. They travel and shouldn’t be considered as sedentary.

Imagine a scenario where you are on a business trip and try to fetch today’s news from a different continent. Your device formulates a URI with a date and calls a service… no results…
That’s weird! Its 10 PM and I should get results for today’s news. This is probably the moment where you realize that you aren’t in your normal time zone… your phone has adapted to your new geographic location but the APIs the app is calling has not!

The Unity Application Block And The ServicerLocator : Reducing Risks While Introducing an IoC in a Brownfield Project

In my previous post Building Windows Azure Services Without Compromising Testability I demonstrated how the Unity Application Block can be used to build a testable Windows Azure Service. Since then I started to play with the ServiceLocator in order to introduce a new testable code base in a brownfield environment.

As many of colleagues would agree, the ServiceLocator is often considered to be an anti-pattern, but there are some scenarios where it shines. Introducing an Inversion of Control (IoC) container in a brownfield project is a scenario where the ServiceLocator can be leveraged in order to reduce risks.

When we start working on a brownfield project, the outcome of our modifications is usually quite unpredictable. Rewriting the entire code base is costly and very risky. The lack of documentation makes it hard for a team to rewrite modules without forgetting features and accounting for undocumented behavior. The ServiceLocator can be used to introduce the IoC container without significantly altering the structure of the application. We can start introducing the new code base in very targeted areas. Consequently, allowing us to bring the application under test. Furthermore it allows us to refactor and reorganize the architecture of the application from within. As the project evolves its important to keep in mind that as we overhaul the brownfield project, the ultimate goal is to factor out the ServiceLocator in order to keep the IoC Container confined to the application’s bootstrap. This of course is not something that can be achieved instantly, it takes time and planning.

MSDN Licensing and Windows Azure: What You Should Know

Many  are wondering about using MSDN licenses within Windows Azure. It has come to my attention that the software is not licensed for use on 3rd party shared servers (“cloud” environments) or in production environments.

Where the Software Can be Installed and Run

The licensed user can install and use the software on any number of devices. The software can be installed and used on your devices at work, at home, at school, and even on devices at a customer’s office or on dedicated hardware hosted by a 3rd party. However, the software is not licensed for use on 3rd party shared servers (“cloud” environments) or in production environments.

Which is from page 12 of Visual Studio 2012 and MSDN Licensing Whitepaper – January-2013, downloadable here.

Furthermore, client OS like Windows 8 and Windows 7 are not to be used in Windows Azure Virtual Machines. As far as I know, Windows Azure supports Server OS versions. The following are the supported Windows Server distributions as of April 2013:

• Windows Server 2012 (en_us) – All editions – x64
• Windows Server 2008 R2 (en_us) – All editions SP1 – x64

Please read the following regarding licensing on Windows Azure

All Microsoft software installed in the Windows Azure Virtual Machine environment must be properly licensed. Windows Azure Virtual Machines include by default a license for use of Windows Server in the Windows Azure environment. Certain Windows Azure Virtual Machine offerings may also include additional Microsoft software on a per-hour or evaluation basis. Licenses for other software must be obtained separately. For information about Microsoft’s License Mobility program see:
http://www.microsoft.com/licensing/software-assurance/license-mobility.aspx

References

• Visual Studio 2012 and MSDN Licensing White Paper
• Create a Virtual Machine Running Windows Server
• Creating and Uploading a Virtual Hard Disk that Contains the Windows Server Operating System
• Microsoft server software support for Windows Azure Virtual Machines
• Windows Azure Cloud Services, Virtual Machines, and Virtual Network SLA

Are You Getting a 403 (Forbidden) From Windows Azure Storage Services?

If your development computer isn’t set to the correct date / time and you are working with Windows Azure you might end up like this guy! I know I did!

Recently I tried to shift back the clock on my developer computer so that I could test a caching mechanism.

After setting the clock back one day, I was unable to connect to Windows Azure services like Blob Storage Service, Queue Storage Service and Table Storage Service. I constantly got HTTP Status Code 403 Forbidden.

I logged into the Windows Azure Management Portal and saw that everything was as it should. I checked the Windows Azure Service Dashboard to see if any services were down… I even check that my storage keys had not been regenerated.

I fired up fiddler and got nothing more than I already knew… then it came to me, whenever http calls are made, the client sends a timestamp to the server and whenever the client and the server are out of sync, calls are often refused. So I set my clock back to the right time and the problem went away.

All in all, this was my fault, I tried to fool the system and I got caught.

Everything boils down to the fact that the Authentication for the Windows Azure Storage Services accept request timestamps who’s date times that are within 15 minutes of the services’ current time.

All authenticated requests must include the Coordinated Universal Time (UTC) timestamp for the request. You can specify the timestamp either in the x-ms-date header, or in the standard HTTP/HTTPS Date header. If both headers are specified on the request, the value of x-ms-date is used as the request’s time of creation.

The storage services ensure that a request is no older than 15 minutes by the time it reaches the service. This guards against certain security attacks, including replay attacks. When this check fails, the server returns response code 403 (Forbidden).