Welcome to EMC Consulting Blogs

Earlier this week The Register posted an article “'Full' SQL Server planned for Microsoft's Azure cloud” which included a few unsurprising comments about the feedback cycle that SQL Data Services (SDS), which is still in beta, is going through.  An interesting read, but not much news – and besides, The Register, could be called the IT Tabloid and articles should be consumed in that context.  All of a sudden the choice quotes were translated into “A Mid-Course Correction for SQL Data Services” followed by Mary J Foley’s interpretation, “SQL Data Services to get an overhaul” – and once Mary J spins it it becomes the truth.

I am not privy to any inside information but neither have I read anything that makes me think that SDS is fundamentally changing.  Let’s back up a bit and look at two things:

1. The relational model does not scale and cannot work in a massively distributed parallel environment – the underlying basis of the relational model makes this impossible.  Regardless of how SDS looks to the end-user or developer it cannot, as a cloud-based data storage mechanism, be truly relational.
2. Microsoft has a very successful relational database in SQL Server.  Although technically the ‘S’ (SQL) in SDS is incorrect, you can’t blame the marketers who want to use the brand, market position and penetration of SQL Server to push SDS.  If SDS was called ‘Azure Data Services’ there would be less confusion, but also less interest and excitement.  I have seen Microsoft evangelist types referring, in a matter of fact manner, to SDS as the ‘Relational Database in the Cloud’ – it may be wrong but it has a nice ring to it.

Adding to the confusion is that SDS (an Azure) is built on SQL 2008 as the and “SDS uses ADO.NET to communicate with the underlying SQL Server-based platform”, which people seem to have interpreted to mean that SQL Server features will gradually be surfaced.  All that this means, as far as I can see, is that SQL Server has been used for storage – and nothing else.  SQL Server stores data pretty well, but that does not mean that your entities are actually stored as tables.  I recently worked on a project where SQL 2008 and the new filestream feature was used to store data.  The system pretty much has a proprietary data format that is super-optimised for storing the requirement and although some meta-data is stored and SQL does a good job of persisting the data in a transactional manner, ultimately the data is binary – the relational model could not cope with the requirements.  I imagine that SQL Server in SDS is used in a similar fashion and there is no way to surface the underlying SQL Server functionality, apart from being able to see data about the platform, fabric or whatever else is under the hood.

I am sure that Microsoft has, since the announcement of Azure and SDS last year, been inundated with questions as to how to port applications to Azure – most of it easy enough until you get to the database.  Existing databases of Microsoft’s SQL Server customers cannot fit into the Azure model (or any cloud model for that matter) and customers think that Microsoft has some ‘splaining to do.  I can picture the arguments in Redmond between the SDS engineers and the marketers as to who is not going to explain away this confusion.  So, for the benefit of the Azure and SQL Community, and in the absence of a Microsoft explanation (so far) let me give an interpretation of SDS and how it will evolve to fit the incumbent database practices.  As a disclaimer, please bear in mind that I have no more information than you and definitely less than my hoity-toity MVP colleagues and the likes of Mary J Foley – this is my personal opinion.

1. SDS will remain a non-relational cloud-based database technically similar to BigTable, SimpleDB and CouchDB.
2. As a massively distributed database certain assumptions, based on what traditional RDBMS’s enforce, will have to be rethought (read as ‘thrown out the window’).  Pat Helland could be roped in to talk more about this – his forklift example (slide 38 and the video is here), which I have seen in a MS slide deck, is a good one.
3. The ‘relational’ features that will be added over time are what the market perceives to be relational features.  Most of the market thinks that the ‘relation’ in relational database refers to ‘relationships’ and therefore 'foreign keys’ and therefore ‘joins’.  So the query interface will add more support for joins (left, inner, outer) and they will have the relational stamp – they are probably better described as set features but nobody is asking for set features.
4. SDS will be able to be integrated into existing applications by (probably in the order of release)
1. Being supported by ADO.NET Data Services.  If you built your applications with the RESTFul tier it can be replaced easily.
2. Being supported by the Entity Framework.  If you drank the EF Kool-Aid you will be able to port easily.
3. Providing an ADO.NET Provider.  Tough to build but doable.
4. Providing a translation layer for T-SQL.  May not be easy and won’t be feature complete, but they can give it a go
5. But that just covers language features.  From a data control an administration point of view SDS will:
1. Provide tools to move data seamlessly between SDS and the on-premise database.  For a clue of how this will work look at the SDS Powershell Client developed by Jamie Thomson.
2. Provide better front-end administration tools
3. Provide tools to ‘relationalise’ (transform) and move the entire SDS authority/container onto a SQL Server database

So if there are any ‘softies out there who want to share more info, either comment or send me an email.  It seems however that we have to wait until MIX09 – apparently there are some big announcements afoot for SDS.

Simon Munro @simonmunro