Welcome to EMC Consulting Blogs

Back in May I presented to the UKAzureNet group about how we at EMC Consulting had developed the See The Difference website using many technologies including Windows Azure.

Part of this session included some information on how we had consumed data we were storing in Windows Azure Table Storage. In this post, I am going to describe the patterns I designed for this in much more detail.

The challenge of consuming data from Table Storage

The Windows Azure SDK provides an API out of the box to consume table storage data. This API is built on top of WCF Data Services (formerly ADO.NET Data Services), and thus uses OData messaging to execute queries against table storage.

Table storage uses an implementation of OData with the following key characteristics:

• It  adds table storage specific authorization HTTP headers to each message which map to your Windows Azure account.
• $expand does not work, because table storage provides a simpler, schemaless entity-attribute-value model.
• Entity group transactions only work against a single table in a single partition.
• $orderby does not work, because table storage cannot reorder data (data is ordered on its partition and row keys only).

These constraints of table storage provide many challenges as to how you design your table storage implementations. Choice of partition keys, row keys and the granularity of each table is key to designing effect table storage. Other people have already blogged heavily on how to solve such issues.

When you have successfully designed your table storage model, the next challenge is how to consume this model in an effective manner. Yes the Windows Azure SDK provides you with an API, but you have to figure out the following:

• How can I make this testable using BDD specs?
• How do I handle exceptions and diagnostics?
• How do I make my data access code maintainable?
• How do I encapsulate my table storage access in a way that calling code is ignorant of the implementation?
• How do I work with transactions across entities?
• How can I effectively implement retries and compensation for failure?

Faced with all these challenges, I ended up architecting a Repository-Mapper-Command pattern, which I will describe below.

An overview of the Repository-Mapper-Command pattern

The Repository-Mapper-Command pattern is, as the name suggests an implementation of three well known design patterns chained together:

From a calling clients perspective, the repository is called to either retrieve or persist domain object(s). The repository will call mapper(s) classes to translate between domain objects and table storage entities. The repository will call command objects to execute commands against the table storage context. The command objects use table storage entities as currency and not the domain objects.

For the rest of this article, a Comment domain object will be the subject of this pattern. A comment domain object represents a single comment stored in table storage, which requires retrieval and persistence using this pattern.

An important aside on the role of data service context and transactions

Before describing each part of this pattern in more detail, it is worth recapping on the role of DataServiceContext in WCF Data Services (used by TableServiceContext).

The best way to think of DataServiceContext is to consider it as a constructor of an OData message or messages. For example, if you read an entity, update this entity and insert another entity in a single batch operation (by using the MergeOption on SaveChanges), this will execute as a single request-reply message. This message is the boundary of what can be done in a single transaction.

An instance of a DataServiceContext object is not thread safe. The ramifications of this is that each transaction (unit of work) should be executed using a new instance of the context. It also means that the contents of an acidic transaction needs to be represented in a set of domain objects using composition. For example, if I needed to transactionally insert a customer entity and the customer’s address within the same transaction, the domain model must represent this in a single object graph.

One final gotcha when using Table Storage entity group transactions; the build in retry policy mechanism build into the table storage client does not work when using entity group transactions; I had to implement my own retry mechanism within a repository and this was a key motivator in implementing the command pattern.

These facts are fundamental to the design of a repository that uses WCF Data Services such as table storage repositories.

The Repository

The repository provides an API that works using domain objects, ignorant of the data access and storage technologies used. The interfaces enable the repository to work with an IoC container such as Castle Windsor, which enables repositories to be stubbed out for testing purposes.

The following diagram illustrates the repository classes and interfaces (click to open it in the original size):

The table below describes the responsibilities of each class and interface in the repository:

The core currency of the repository API is a domain object. The repository exposes methods such as Single to retrieve a single domain object.

A transaction cannot flow outside of a repository, or outside of a method on a repository. This is good practice regardless of the type of repository, but it is imperative given the nature of the DataServiceContext class. Hence, if a calling client calls persist against two repositories, this will occur as two separate ACID transactions. Failure must be handled using a compensating transaction to leave data in a consistent state.

Each method on a repository must instantiate its own service context for thread safety. The creation of a service context with the configuration is handled by TableStorageRepositoryBase. The service context instantiated is of type ApplicationDataServiceContext, which provides strongly typed access (via IQueryable<T>) to each table service entity.

A single repository method will execute one or more commands against a single instance of the service context. Hence, a transaction can flow between commands within a single repository method.

In order for a command to be executed, the repository must map any domain objects into table storage entity objects using a mapper. Likewise the result of a read command requires a mapper to map the table storage entity back to a domain object.

The pattern implements a Persist method, rather than add and update. This means the repository is responsible for deciding whether to add or update the domain object(s), based in whether the object has an entity key set or not (when the entity key is null). For example the persist method will execute an AddToObject when the entity key is null.

The repository catches TableStorageCommandException objects, and actions the exception accordingly, including logging the exception against Windows Azure diagnostics via the TableStorageRepositoryBase.LogCommandException method.

The Mapper

The follow diagram illustrates the mapper classes and interfaces, including the related domain objects and table storage entities (click to open it in the original size):

The table below describes the responsibilities of each class and interface in the mapper:

The mapper classes above around mapping between the Comment domain object and the CommentRow table service entity object. The mappers are required to translate between the domain objects surfaced by the repository and the table storage entities executed by commands.

The Command

The command encapsulates a single operation against table storage and a table service entity (such as a LINQ query). The repository will chain together commands against a single ApplicationDataServiceContext, promoting reuse and improving the maintainability of your code. The command also provides a natural place to handle OData errors thrown as InvalidOperationException objects.

The follow diagram illustrates the Command classes and interfaces (click to open it in the original size):

The table below describes the responsibilities of each class and interface in the command pattern:

Each command represents a single unit of work executed against a shared service context. Commands do not use the domain objects; they are coupled to the table storage implementation details. Each command has an interface so that interactions can be tested in specifications.

By convention, each command exposes an Execute method, with at least one argument for ApplicationDataServiceContext. If an exception occurs, the TableStorageCommandBase class contains common exception parsing code to translate the OData error into a strongly typed exception (TableStorageCommandException).

Summary

Use of the Repository-Mapper-Command pattern for Windows Azure table storage enables you to implement storage access with:

• Testable BDD specs.
• Diagnostics logging and exception handling.
• Reuse LINQ operations improving maintainability.
• Provide an API that encapsulates all storage implementation details.
• Works with entity group transactions.
• Enables the implementation of retries and compensation where needed.
• Centralizes object dependencies via IoC.

Useful Resources

• Windows Azure table storage concepts.
• Programming Table Storage Whitepaper
• Walkthrough Windows Azure table storage

There are a few events coming up over the next couple of weeks that I’m speaking at (all in London) with a colleague of mine James Broome about the Windows Azure platform:

• Microsoft BizSpark Camp March 29th, targeted at startup businesses
• Microsoft Architect Insight Conference 2010 April 1st, targeted at architects
• UK AzureNet User Group, April 15th, targeted at developers

Hope to see you there!

Background

Since ASP.NET 2.0, the framework has provided a localization model based on localizing text into RESX files. RESX files are compiled and deployed as part of the website, and referenced through declarative mark-up on your pages, cleanly separating your localized resources from your code.

Using RESX files to store the resources is highly performant and scalable, because resources are deployed locally with the website, meaning the resources will scale with the number of web servers you run. There are however two main downsides to this approach:

• Changes to resource text requires the resource assemblies to be deployed to the web servers in your farm.
• Resource changes need to be done at design time and cannot be achieved at run time.

These constraints to resources files have led many people to store resources in SQL Server, enabling your to centrally manage and maintain resources at run time without any deployment headaches. This model works really well, and does not require any code changes within your website, because the .NET framework uses a provider based model for swapping out RESX storage for a custom provider (using the globalization configuration section).

For all the benefits of the SQL Server resource provider approach, it does present one major drawback; SQL Server represents a single instance which must be scaled up rather than out to cater for demand.

Windows Azure Table storage provides a new storage option for resources. Table storage is non relational (as are resources), providing highly performant storage that scales out on demand. Therefore for resources, table storage provides all the benefits of a SQL Server provider coupled with the scale out capability of RESX files. With resources being stored in the cloud, the ASP.NET website can reside on premise or in a web role in Windows Azure.

The Solution

The Azure Table Storage Resource Provider source code can be found at:

http://localization.codeplex.com

The provider implements the standard .NET interfaces for building a resource provider. The entire set of classes used by the solution are shown in the diagram below:

The provider has been built using the Windows Azure SDK July CTP. In order for you to develop with it, you must install the SDK and have the StorageClient assembly. This assembly encapsulates data access to table storage via ADO.NET Data Services.

Configuration

There are two items you need to configure in order to use the provider:

• You need to ensure your ASP.NET website config file includes the <globalization /> configuration element as demonstrated in the sample web sitein the solution.
• You need to change your Azure configuration file to use your own Azure Table storage account (you can use the development fabric during development).

Table Storage Structure

The provider expects two tables to exist in your account:

• GlobalResources, to store global resources
• LocalResources, to store local resources

These two tables have the same structure:

The partition key is the culture name concatenated with the resource set name, encoding to be URI safe. For example:

en-gbab2fpb61gesb2fsb61mplelocb61lresources.b61spx

is the partition key for the culture en-GB and the (local) resource set /Pages/SampleLocalResources.aspx. Note that it is stored all in lower case, because access to the partition is case sensitive, so the provider uses a lower case key at all times.

The table row key is the resource key of the resource. For example, headerliteral.text is the resource key for a literal control with the ID of HeaderLiteral. This resource key identifies the text property of the control. Again, note the provider uses the key stored in lower case.

The other attributes each table requires are:

• CultureName, which contain the name of the culture.
• ResourceSetName, which contains the name of the resource set.
• Data, which contains the actual localized data.

The provider needs to have an invariant set of resources for each localized resource set you implement. This is used as the fall back culture when you do not have a localized resource set for your culture.

Introduction

The data access wars are over; long live the ORM wars. The new battles are fought with even more fanatical zealous behaviour than in previous conflicts. Rather than just debating the pros and cons of how best to execute SQL and wire it into our objects, there is now a whole new dimension to developer’s polarized opinions; what is the best ORM to use, if you believe you should use one at all that is.

The objective of any ORM is to bridge the object-relational impedance mismatch that occurs between a normalized database model (optimized for data storage, consistency and performance), and an object model (optimized for writing maintainable domain logic). Without an ORM, a developer needs to bridge this mismatch by writing tedious code that is hard to test and maintain. Thus ORMs should improve productivity, and the trade off is one of loss of control over the exact SQL executed against your database, and the performance overhead of running the ORMs code.

What makes the ORM wars so intensely fought is that there is little or no objective comparisons of the available options, and many people do not have equal experience in all the options either, meaning they find it hard to come to an objective view. This post attempts to provide an objective guide to choosing a technology to access data and return the data in the desired entity model of the developers choice.

This post compares the most common set of options facing a .NET developer at the present time, although other ORMs are available. The options considered here are:

• Writing traditional ADO.NET data access code and manually mapping the data to your object model.
• Using LINQ to SQL to surface data and LINQ to Objects to map this data to your object model.
• Using NHibernate as an ORM to access data and map this information to your object model.
• Using ADO.NET Entity Framework as an ORM to access data and map this information to your object model.

Assessment Criteria

In order to provide an objective assessment of each option, we need to have specific criteria which to measure. The criteria used in this post are as follows:

• Productivity and Maintainability
• Control over SQL
• Operational Support and Maturity
• Mapping Capabilities
• Database Vendor Support
• Modelling Approach
• Coherence with the rest of the .NET Framework
• Testability
• Futures

Each technology choice is measured out of ten for each area of assessment. No overall “score” is given to each technology option, because that is missing the point of this work; really it depends on the design goals of your project as to what the best fit is.

Productivity and Maintainability

The business objective of an ORM is to provide a more productive method of populating an object model with data than traditional data access code, which is prone to human error (due to repetition). This section looks at how productive the development phase is for each option, looking at tooling, engineering practices and maintenance after release.

ADO.NET Data Access Code

A typical traditional data access layer in a .NET application will use ADO.NET Connection, Command, Parameter and DataReader objects to execute stored procedures (containing SQL) against a database connection. Parameters are passed into a stored procedure via a Command object, and after the procedure is executed, data is read by using a DataReader, or reading the output Parameter objects of the command. Normally, a developer will map the contents of a DataReader object to their own Entity object in code.

From a productivity perspective, the creation of Command and Parameter objects to execute against a stored procedure is prone to human error during development, because of the monotony of the task, and the lack of any compile time safety beyond the syntax of the ADO.NET code. The work of mapping a DataReader to an Entity is also tedious, although the advent of C# 3.0 and LINQ to Objects has made this task less onerous.

From a maintainability perspective, such a data access pattern does have one benefit over ORM approaches; SQL encapsulated in stored procedures can be more easily patched in a production system without a redeployment of the application.

Score: 2 /10

LINQ to SQL

LINQ to SQL provides code generation of data access objects using a command line tool called sqlmetal, which is integrated into Visual Studio 2008. It auto generates objects which represent to table structure of the database and are accessed using an auto generated DataContext. This provides a very quick start up time of developers, and the visual tooling is Visual Studio is very good.

A common misconception with the data objects that LINQ to SQL generates is that they replace the entities in your traditional architecture. They do not, because LINQ to SQL is not a fully fledged ORM. You will not be able to model the generated data objects as you would like using the designer. The best approach to using LINQ to SQL is to create your entities yourself and map the data objects to your custom entities using LINQ to Objects. This means that there is still be an amount of tedious mapping code to maintain, but the hand crafted SQL stored procedures have gone completely and you do have compile time safety over your data objects and mapping code.

The biggest issue that LINQ to SQL has both for development and maintenance is what happens when you make changes to the data model during development. Whilst none of the options here have a magic answer to this problem, but really the only answer LINQ to SQL has is to regenerate the entire data model. This becomes very onerous if you have customized the auto generated data objects in some say (like changed the names of properties or data objects using the designer). You can circumvent this weakness by not changing what is generated, but that does beg the question of the point of the designer in the first place.

Score: 6 / 10

NHibernate

NHibernate enables you to map your own entities to the database model using XML mapping files. Normally one mapping file is created per entitiy. Each mapping file contains meta data about the database model the entity maps to and how this data is mapped to the entity. Execution of these maps to populate objects with data from a database is handled using a Session object, which manages connections to database and generation of SQL against the database.

Like LINQ to SQL, the fact that NHibernate generates the SQL to execute against the database removes one productivity issue. However, each mapping file must be maintained by hand, and tooling support is thin on the ground, not really getting any better than XML intellisense. This additional control does not suffer from LINQ to SQL’s code regeneration issues (because nothing is generated), but is does beg the question “haven’t I just solved one maintenance problem but created a new one?”. Additionally, querying a repository using the session is achieved using a loosely typed syntax (string based), which is prone to error unlike a LINQ implementation like LINQ to SQL or LINQ to Entities (entity framework). LINQ support is however under development by the open source community.

Post production, any changes to the underlying database schema will require the deployment of the NHibernate mapping file (usually embedded within the data access layer assembly).

Score: 4 / 10

ADO.NET Entity Framework

The ADO.NET Entity Framework is an ORM that maps the structural model of a database schema to a conceptual (object) model. The framework uses the Entity Data Model (EDM) to achieve this. The EDM is broken down into three sections: the structural model (SSDL), the conceptual model (CSDL) and the mapping model (MSL), which are stored as XML normally embedded within an assembly. The entity framework is similar to NHibernate in that ORM mappings are stored as XML, but differs in the separation the EDM forces between the three models.

Like LINQ to SQL, the ADO.NET Entity Framework provides code generation for its EDM via a command line tool called EdmGen, which has integrated designer support in Visual Studio 2008 SP1. Because this design time support is based on changing the conceptual model of the EDM, the entity framework does not suffer from the same shortcomings as LINQ to SQL, in that it is a fully fledged ORM that will can (on the whole) produce the desired object model from the designer.

Better still is the tooling for the entity framework. This is where it plays an ace card; compile time support for the EDM comes for free, and what a free lunch it is. The entity framework refuses to build if there is an issue with the EDM that will cause it to fail under any condition. By this I mean any issue. So, the EDM will fail to build if there is an issue with your database schema, or if your conceptual model will not work under all CRUD conditions. Surely, this is the real problem an ORM was supposed to fix, and of these options, the entity framework is the only option that really helps you.

However, the ADO.NET Entity Framework still suffers from difficulties when in comes to changing the database model post production release. Depending on the nature of these changes will depend on how much of the EDM needs to be reworked; if the change is an addition to the structural model, it is possible to use EdmGen to regenerate the SSDL while keeping the conceptual model (you will need to do some extra mapping).

Score: 9 / 10

Control over SQL

If database performance matters (I mean it really matters to your scenario), then you may need to strictly control the SQL that is executed against your database. For these scenarios, ADO.NET Data Code cannot be beaten, because you can do anything you like. Most of the time however, all three of the other options will suffice. All the other options also provide support for stored procedures should it be necessary to execute some SQL that the technology cannot generate.

ADO.NET Data Access Code Score: 10 / 10

All The Rest Score: 6 / 10

Operational Support and Maturity

Its all too easy for developers to focus on a solution from a purely development perspective, but the reality is a truly useful application will remain in production far longer than its development phase. In many organizations the development team are not responsible for keeping an application running, largely because this is not cost effective. Hence, a support team is normally responsible for keeping the application running, and their skill set differs from that of the development team, in that they have less application specific knowledge, but more platform specific knowledge.

Operational support teams are largely concerned with procedures to get a system back online should an application fail. When these procedures do not suffice, the team will escalate the issue to the development team and ultimately the vendor’s support team.

For many businesses, this is a critical reason why they have chosen Microsoft technologies over other vendors, or open source offerings; operational support like the fact that they can fall back on Microsoft support should they need to. Having said that, NHibernate has a very active open source community, and if all else fails, you have the source code which you can fix.

Another important angle to operational support is maturity; the wisdom goes that the more mature a technology is, the less likely it is to have bugs. Of the technologies compared here, the most mature is ADO.NET Data Access code, then NHibernate, then LINQ to SQL, and finally ADO.NET Entity Framework.

ADO.NET Data Access Code Score: 10 / 10

LINQ to SQL Score: 8 / 10

NHibernate Score : 6 /10

ADO.NET Entity Framework Score : 6 / 10

Mapping Capabilities

The mapping capabilities of an ORM can prove a crucial deciding factor when choosing an ORM to use for data access. After all, the whole point of an ORM is to bridge the object relational impedance mismatch between your desired object model and the best relational database model.

Of the options considered here, only two can be considered as ORMs in the true sense (NHibernate and ADO.NET Entity Framework). Hence only these two can be measured in this area.

Firstly, both ORMs deal with associations between objects perfectly. In other words, both options model one to one, one to many and many to many associations without issue.

NHibernate provides more options for modelling inheritance than ADO.NET Entity Framework. The entity framework provides support of table based inheritance. This means that each sub classed object maps to a separate table, each row representing the sub-classed object. NHibernate also supports table based inheritance, but it additionally provides other options for modelling inheritance.

NHibernate Score : 10 / 10

ADO.NET Entity Framework Score : 8 / 10 

Database Vendor Support

Many businesses have a chosen database vendor, and thus choosing a data access option will require that you can easily utilize the chosen database vendor. Common databases include SQL Server, Oracle, DB2 and MySQL.

ADO.NET Data Access Code

Out of the box ADO.NET has support for both SQL Server and Oracle, and other third party providers can be purchased for support for most common database vendors. This is the most supported method of accessing data from .NET code, an in some more exotic database it may be the only viable option.

Score : 10 /10

LINQ to SQL

As the name suggests, LINQ to SQL is for use only with SQL Server. If this is your database, LINQ to SQL is not a problem. If you are using anything else, LINQ to SQL is not an option.

Score : 1 /10

NHibernate

NHibernate has support for SQL Server and Oracle out of the box, and it is possible to roll your own support of other database vendors.

Score : 8 /10

ADO.NET Entity Framework

Out of the box the ADO.NET entity framework only has support for SQL Server, but unlike LINQ to SQL, other database vendors are not ruled out, as the architecture supports using a custom data access provider. Custom providers can either be built or purchased, and third party support is growing by the day for support of databases such as Oracle and MySQL. For example, DevArt have providers that support multiple third party databases here. There are also samples available from Microsoft on rolling your own here.

Score : 6 /10

Modelling Approach

There are two initial approaches to designing a solution with a database and an object model: either design the database model first, or the object model first (although clearly these two activities should never be executed entirely in isolation). There is no right or wrong order to these tasks, but each of the technology options provide varying support of both approaches.

ADO.NET Data Access Code

This option does not stop you from approaching a solution’s design from either standpoint, but then neither does it really provide any support for either option. Here, you can do what you like, including doing some very bad practices indeed. The only modelling you will have is the class designer and your database vendor tools.

Score : 3 /10

LINQ to SQL

LINQ to SQL only really models the data layer, which means you are free to do the object and database modelling either way round. But like the straight data access option it’s not really giving you much value here.

Score : 3 /10

NHibernate

NHibernate enables you to model your database and entities in either order, and for many in the object model first camp, this option feels like the most natural choice of the four here. But again, modelling support only comes in the form of not hindering you, rather than helping you

Score : 6 /10

ADO.NET Entity Framework

If there is one area that will rule the entity framework out for many developers, this is it. In reality, the entity framework currently only supports database model first development. If you can work this way, the modelling support is superb, but this does limit how the entity framework can be employed, particularly if you do not have control of an existing database’s model; this part of the entity framework is very unforgiving, yet also most giving.

Score : 3 /10

Coherence with the rest of the .NET Framework

One often overlooked point when considering a data access option is how coherent the technology is with the rest of the .NET Framework. By coherent I mean that the API feels like a natural extension to the rest of the framework. This is important, because it reduces the learning curve for developers new to the technology, and can also help future proof it with advances in the core framework. It was for all these reasons that I have always disliked many aspects of the Enterprise Library; because it felt unnatural to program against, and at its worst hid many core aspects of framework development.

In these respects, NHibernate is at a distinct disadvantage in that all the other options are the .NET framework. From a developers perspective its API shares no coherence with either the underlying ADO.NET paradigm, or the IQueryable (LINQ) paradigm. This makes learning NHibernate a uphill task for .NET developers skilled in either of these areas.

Of all the options, ADO.NET Entity Framework scores best, because it supports both the ADO.NET paradigm and the IQueryable paradigm, meaning that developers who have invested time in these areas can adopt the entity framework more easily. Another important area to consider here is the support ADO.NET Entity Framework has from other parts of the .NET Framework. Again the ADO.NET Entity Framework wins here because it has a lot of support from both the ADO.NET Data Services team and the ASP.NET Dynamic Data team. Once you have an entity model, you can create a data service or scaffold a website fairly easily, giving you significant quick wins (LINQ to SQL does not have this level of support).

ADO.NET Data Access Code Score : 5 / 10

LINQ to SQL Score : 6 / 10 

NHibernate Score : 1 / 10

ADO.NET Entity Framework Score : 9 / 10 

Testability

Making software easily testable by creating unit tests is an important engineering practice in ensuring quality in your solution. More specifically, the ability to test the interactions between the layers of your architecture is as important to testing for results of executing code in a specific layer of your application.

ADO.NET Data Access Code and NHibernate

Both of these options enable you to create unit tests and mock layers of the architecture, because you are complete control over all the code you test. This is because both options support the use of POCO classes for the creation of business objects and entities. Additionally, both options provide interfaces for you to mock out enabling you to test interactions.

Score : 10 /10

LINQ to SQL and ADO.NET Entity Framework

Both of these options generate out classes for you to work with that represent your database (for LINQ to SQL) or your entities (Entity Framework). These objects are not POCO objects, which forces a dependency of framework assemblies in all parts of your application that consume these objects. For ADO.NET Entity Framework, this problem is more serious, because entities proliferate around your architecture more than data layer objects. Additionally, these generated objects do not expose separate interfaces, meaning that you cannot use mocking practices against these objects to test interactions easily. The counter argument here is that you do not need to test these interactions, because the code is generated anyway.

Score : 1 /10

Futures

No one wants to invest money or time in a technology that has no future, or will be quickly superseded. It is a key factor for development teams who consider that their application will evolve through several releases over a period of time.

ADO.NET Data Access Code

You can be sure that your investment here will not be superseded, but neither will it be greatly improved upon in future. Its mature, but it is also feature light.

Score : 3 /10

LINQ to SQL

Microsoft are not developing LINQ to SQL features, because it is now considered to be superseded by the entity framework. Of all the options here, this one has the most bleak future (although Microsoft will obviously continue to support the technology).

Score : 1 /10

NHibernate

NHibernate continues to have a active ALT.NET community, and features continue to evolve. LINQ support is due soon, which will level the playing field with the Entity Framework in terms of support for ADO.NET Data Services and ASP.NET Dynamic Data. The only doubt here is how a maturing entity framework will affect NHibernate support in the future.

Score : 7 /10

ADO.NET Entity Framework

Many of the entity framework’s short comings are promised to be fixed in the .NET 4 timeframe. In particular, it will (apparently) support POCO objects, object model first design (including database generation) and better testability. What is for certain is that Microsoft are backing this horse, which should not be overlooked.

Score : 10 /10

Summary

Wow… that was a long blog post! But in short:

You might choose to use ADO.NET Data Access Code when:

• You really really need to worry about performance
• You’ve got a database not supported by any other means
• You have not need for an entity model in your application

You might choose to use LINQ to SQL when:

• You want to create a prototype quickly
• You haven’t got .NET 3.5 SP1 installed so you can’t use the Entity Framework
• You only ever want to work with SQL Server

You might choose to NHibernate when:

• You like modelling an object model before the database model
• You don’t have control over the database model
• Operational support have no issues with using a third party framework
• You want good testability
• The project is sufficiently complex that you would benefit from the investment creating maps manually
• You are not concerned with the lack of ADO.NET Data Services or ASP.NET Dynamic Data Support

You might choose to use the ADO.NET Entity Framework when:

• You have control over the database model and are happy modelling this first
• You want compile time safety over your entire ORM
• You want productivity without compromising your architecture (very much!)
• You want ADO.NET Data Services or ASP.NET Dynamic Data support
• You trust the generated non-POCO entities
• You want the most future proof option

This is a blog I've been meaning to write for over a year now, but have just not had the time. Its as much a "note to self" as a blog for external consumption.

WCF provides the ability to generate a client proxy that communicates with a service, such as a SOAP service conforming to basic profile. Generating a proxy can be done by either using the SvcUtil.exe command from the Visual Studio command prompt, or by using the "Add Service Reference" feature of Visual Studio 2008 and pointing Visual Studio at the WSDL of your service. This action will generate a client proxy class and the required WCF client configuration file. You can write code against this client proxy class and armed with the client configuration, WCF will handle the communication to the service endpoint and the (de)serialization of messages into CLR types.

When a client proxy class is instantiated, WCF will build a channel stack based on the binding settings contained in the client configuration. This binding includes the address of the service, the transport to use, the encoding of the message and any service policies that apply to the service endpoint. The channel stack is the runtime representation of the way in which the WCF client will communicate with the service. For example, an average SOAP based service will communicate using an HTTP transport using a text encoding, and the service may apply additional WS-* policies such as WS-Security to implement service authentication.

Much of what I have just written is common knowledge, but much less written about the performance implications of how best to use a generated WCF client proxy. There are a few really important points to note when using these generated classes, and how this affects the performance of your application:

Consider caching the proxy object

When you instantiate a proxy class, WCF examines the binding configuration and uses reflection to generate the channel stack at runtime. This channel is fairly intensive to create, and thus creating the client proxy class every time you make a call to a service operation is excessively expensive. Thus, you should consider caching the proxy object, or hold the proxy object in a static variable. For a web site the most logical place to cache the proxy is using System.Web.Caching. You need to be aware that you can only reuse the same proxy when the channel stack is exactly the same, and this means running in the same security context (for example you might need to cache by username if you are using Windows impersonation).

Consider calling request-reply service operations asynchronously

For a request-reply calls to a service you should consider calling the asynchronous BeginMyOperation() method rather than the synchronous MyOperation(). This is because when a synchronous call to a service operation is made, the thread is blocked until the reply is received. This causes the CPU to work much harder than if you use asynchronous calls and deal with the response in a callback.

Always call the Close() method on the proxy object

The WCF client proxy contains a method named Close() which you should call after you have completed communications with the service endpoint. This method closes communications over the channel stack and disposes of any unmanaged resources. Failure to call this method will seriously degrade the performance of your client application, as communications are not properly closed and unmanaged resources may consume excess memory.

Do not instantiate the client in a Using statement, or call the Dispose() method

Calling the dispose method can lead to some communication exceptions being hidden (because the unmanaged resources are disposed too quickly). It is also unnecessary to call Dispose() when you call the Close() method, because this method handles disposal of resources for you.

It's been a long time since I last wrote a blog post; around six months to be exact. It's not that I have not been busy writing though. In April I began embarking on one of the hardest things I've ever done; writing a book with my co-author John Shaw.

And now for a brief advertisement; the book is called "Pro ADO.NET Data Services : Working with RESTful Data", and is going to be published by Apress in December. Feel free to order itJ. In fact, here's a pretty picture link…

Colleagues of mine said to me that I should blog my experiences of writing it. Howard suggested to me that I seek comments from the community about what content to include much like Ayende Rahien has done for his up and coming book. I have resisted, not only because I am exhausted from writing, but also because I think I've had a deep sense of insecurity about the whole business.

I went into the process thinking it would be like writing a big blog; a marathon blog. But in reality, it's a much harder process than that. Every line of text gets scrutinized… and rightly so, for there is no room for mistakes in a book. But the real insecurity comes from realizing you're not a seasoned writer; you're an IT professional writing a book in your spare time. Starting a chapter from a blank page is a really daunting prospect; where do you start, and how do you structure the text?

I have found it very hard deciding what content to include and what level of detail to go into. John and I decided at the beginning of the process that we wanted to write a "real world" book, where we painted ADO.NET Data Services in the context of the enterprise, because we were sick of reading about how to apply the latest and greatest technology with no mention of using it with surrounding technologies. The result is that the book covers a really diverse set of technologies you may find in your enterprise, from WCF to BizTalk and from ASP.NET AJAX to Silverlight, we have stitched together a picture of how many development landscapes look and how these technologies apply to ADO.NET Data Services. This broad array of technologies made the writing process even harder; just how do you cover ASP.NET and AJAX solutions in one chapter and make the text useful and relevant without writing a whole book about the subject?

Like software development, it is difficult to accept when you are done writing a book, because you can always add a little bit more. This problem is exacerbated when you are writing about something not yet released, as was the case when we began the project. I had to keep asking myself whether the delivery was fit for purpose, and I'm pleased to say that I'm really happy with what we've covered in the book.

But I could always write more about ADO.NET Data Services, in the next two weeks I am going to write a couple of blog entries covering additional material on more exotic subjects, such as using ADO.NET Data Services with Oracle.

It just so happens that the completion of my chapters in the book also coincides with the start of PDC 2008 in Los Angeles, where I am currently writing this post from. I think PDC this year marks the start of a new chapter in Microsoft technologies. It's certainly the most important event since WCF and WPF were announced several years ago and I now have the time to start thinking in earnest about the impact of .NET 4.0, Oslo and Dublin. After the concluding chapter of my book covers the future of services and cloud computing, it feels like the perfect place to be right now.

I never really got that excited about Silverlight 1.0, mainly because whilst it had a great core graphics engine and did video streaming very well, it's feature set was just not rich enough to make it really useful to applications that demanded deep functionality.

One of the features that was clearly lacking was connectivity; you could not directly consume services; the main workaround was to use the connectivity features of ASP.net AJAX to consume services from Javascript. Thankfully, this problem has been rectified in Silverlight 2.0. Alongside other new features in Silverlight 2.0, I am now much more interested in what I can do with the technology.

When calling any service from within Silverlight 2.0, you have to call the service asynchronously (like AJAX). The reason this is important is because of the nature of the browser itself, which ultimately is the host you are running in.

There are two sets of services you need to consider consuming from Silverlight: services that are self describing (such as SOAP / WSDL and RSS / ATOM) and services that are not (such as REST and POX). Silverlight 2.0 can consume all of these types of services, but it does so in distinctly different ways.

The easiest services to consume in Silverlight 2.0 are SOAP based services (something sure to infuriate all the RESTafarians out there!). They are consumed in Silverlight using a WCF service proxy configured to the BasicHttpBinding (BasicProfile 1.1). They are easiest to consume, because it is possible to auto generate a service proxy from the WSDL. Unlike AJAX, XML is also the preferred encoding over JSON, because the Silverlight runtime has good XML performance in managed code unlike Javascript. When returning XML, Silverlight gives you two main approaches to parsing the asynchronous result (returned via an event handler): either using LinqToXML or using the XmlSerializer to deserialize the result into a predefined type. SOAP faults, however are not supported, because of security restrictions on the browser, which could cause some issues to existing services that use the fault mechanism to return exceptions.

Other services (such as REST) are a little harder to consume, and the one thing that did surprise me was that there is no support for the auto proxy generation used by ASP.net AJAX against the WebHttpBinding. You have to construct a Uri string manually and call the service either by using the WebClient class in the case of HTTP GET resuests (REST), or by using the HttpWebRequest class for other HTTP verbs. If the service uses JSON encoding, parsing the response can be achieved in one of two ways: either through WCF's DataContractJSONSerializer (similar in concept to the XmlSerializer), or by using LinqToJSON, which you can find here:

http://www.codeplex.com/Json

Whilst XML seems to be the first citizen of Silverlight, it is important that JSON encoding is supported easily, as many services you write may need to be consumed from both an AJAX enabled page and a Silverlight control. In these circumstances, you would definitely opt to use JSON, because the performance of XML in Javascript is poor, and the payload size is also bulkier.

RSS and ATOM feeds are consumed using a mixture of the two methods above; you need to manually construct a Uri and pass it into a WebClient which calls the feed, but Silverlight does provide you with a SyndicationFeed class to parse the response.

One final point to make is around security. As with AJAX, there is a difference between private services that are written and consumed only by your application and public services that are consumed by multiple applications across the internet.

For privates services, Silverlight uses the ASP.net forms authentication mechanism (based around an authenticated cookie) to secure your services from other clients. This means that there is no additional code (on top of your ASP.net code plus config) to write in order to secure your own services.

For public services, the same cross domain security issues apply as they do in the AJAX world. Silverlight supports its own ClientAccessPolicy.xml format as well as Flash's CrossDomain.xml file format to resolve this problem. So long as the service you call has one of these two files in its root, Silverlight will be able to call the services cross domain.

So I just got back from TechEd Barcelona, and I had a great conference in no small part due to the people I went with: Merrick Chaffer, James Dawson and Paul McMillan. This is a selection of some of Conchango's finest from the world of .Net development, infrastructure and SQL Server. One of the reasons the conference was so much more enjoyable in the company of my colleagues was that I got as much out of the conversations we had outside of the sessions as I did from what I learnt in the sessions and labs.

Of all the subjects covered at this year's TechEd, none had more coverage than LINQ, and between us, none had more debate than the impact LINQ has on our worlds. This is a conversation I can see playing out across the Developer / DBA divide the world over, as the majority of solutions developed require some sort of data access to a database. David Chappell made a comment in his REST vs SOAP session (which was probably the best session I went to) that struck a real chord with me; he said "people are only passionate when something is in doubt" Conversations of this nature can tend to take a slightly religious standpoint, so I'll try to bear in mind David's comments.

So with LINQ to SQL (and LINQ to Entities) the most hotly debated point was around the auto generation of LINQ's dynamic SQL vs. using stored procedures. In the old days, one point favoring of using stored procedures was the performance gain from a cached execution plan. But since SQL Server 2005, dynamic SQL is also cached, so this reason for using stored procedures has largely gone away. One benefit dynamic SQL has over stored procedures is that you only select what you need in all cases. In a typical scenario with stored procedures and a DAL populating an object model, stored procedures will tend to be reused even when all of data is not used by the calling DAL method. There ways around this, but it largely involves an ever increasing list of stored procedures (GetObjectByX), make the solution less manageable.

What became clear in our debates on this topic is that really all depends on who you trust more: a developer writing a stored procedure or the LINQ to SQL engine? I began the week at TechEd a skeptic of LINQ to SQL, but the more I looked at what the engine produced, the more I began to trust it. Sure it won't do a better job than a highly trained SQL expert (like Paul), but is it good enough and better than your average developer? From what I have tried I would say the answer is yes. And the bottom line is, when you need that extra performance out of a finely tuned stored procedure, you can still use them.