Welcome to EMC Consulting Blogs Sign in | Join | Help

Memoirs of a Plumber

Moans, Groans and Scary-tales...

An Epitaph for a Stored Procedure (no, not really)

Simon Evans some time ago questioned if given the parametric nature of queries generated by ORMs, stored procedures still have a place in developers toolbox? At the end of the day why bother if ORMs seem these days seem to do everything for free? It looks to me that statement that "stored procedures are dead" (although Simon has never said that) has gotten a life of its own and some people made it into a mantra of sorts... Argument, which Simon quotes, that ORM generated parameterised queries are as performant as stored procedures is absolutely valid and from this point of view stored procedures are redundant: the days when we had to implement every single CRUD operation as a sproc are gone. People seem to forget however that using sprocs for performance was not the only reason to use them.

The primary reasons why stored procedures are still worthwhile to be considered is the principle of isolation: personally I strongly believe that as much as we can, we should avoid dependencies between business and database layers. Dave Hanson with whom I work on a project for a large executive recruitment agency posted his doubts on his blog and his views are very close to mine, these days we often find ourselves on the receiving end of these dependencies when the DBA decides to change schema of the database and we have to refactor large portions of program logic to deal with the change. Having said that dependencies are not the only problem with ORM based persistence layer.

First major problem which is a "feature" of all ORMs is the problem of lost updates, as their default mode of operation goes along the lines of:

image

As any SQL aficionado will immediately recognise, this approach is a potential disaster in a multi-user system. If two of these batches were to execute concurrently, there is a fairly good chance that one would override the changes made by the other and instead of final value of initial balance + 20 we would end up with much less. Surely, this problem can be overcome by implementing optimistic locking, running the entire procedure within a transaction with sufficient isolation level etc but why bother when the problem can be solved easily with running a piece of code along the lines of:

image

Not only is the second approach simpler, it also works well in a multi-user environment and minimises lock contention.

Yet another reason why stored procedures are helpful is simply the power of SQL which is extremely efficient tool when it comes to handling large sets of data: when writing a stored procedure a developer has at his disposal all the weird and wonderful inventions of SQL including aggregate functions, non-equi joins, common table expressions etc. and the list goes on. On a recent POC for easyjet.com I used SQL tricks which would be simply awkward if not impossible to implement as operations on table based entities. Strangely enough results of those queries were successfully mapped to a set of WCF data contracts using LINQ to SQL mapping attributes because if anything is certain, it is the fact that manually mapping datasets to objects is a thing of the past (unless of course you are hardly pressed for performance which is another story).

Technorati Tags: ,,
Published 10 November 2008 22:59 by marcin.kaluza
Filed under: , ,

Comments

 

simon.evans said:

Hi Marcin

A couple of points I'd like to make:

1. I'd like to reiterate that I have never stated that "stored procedures are dead"... In fact, I firmly do not believe this. The problem is that people like to simplify design decisions as being "good" or "bad" and "right" or "wrong", because they want a technology to solve their problem. In reality, stored procs and ORMs are just tools in the toolbox for developers. The skill is in knowiong when to best apply them. This is why we should not become prescriptive about applying either ORMs or Stored Procs as the "right" way of building all solutions.

2. I don't think using an ORM blurs or removes the seperation between data access and business logic. There are several approaches to using an ORM and one valid way is to view an ORM as a replacement data access layer and fully encapsulate it in this layer. In this pattern, you have complete isolation, but have removed stored procs for the situation. However you design a system, what is important is that you have clear seperation of business logic from data access.

3. If your DBA decides to change the DB schema, you have to make changes to your code base either with or without an ORM. This may well include changing your ADO.NET code. I don't see what stored procs give you here.

4. Using an ORM doesn't mean you can't handle SQL in stored procs. For example, your update code could go into an entity framework function call, which would call a stored proc.

November 12, 2008 14:28
 

marcin.kaluza said:

Hi Simon,

Ad 1) I hope you did not get an impression that I tried to make you responsible for this unfortunate statement as I could not state more clearly that you never said that. And I fully agree wit the rest of this comment.

Ad 2) Let me just say that we differ in this point. I find the idea of replacing hand crafted mapping from "objects to datasets" with ORM only to later manually map "objects to objects" somewhat unattractive.

Ad 3) I will demonstrate exactly how to do this in a future post

Ad 4) That's true, but at the same time most (fortunatelly not all) ORMs will not allow you to use sprocs for entity CRUD.

November 12, 2008 19:25
 

Anthony.Steele said:

Hi

Making a database independent from the business layer is something that you say is good. See Simon's first comment on this - when is it actually an important goal? The majority of applications that I have worked on involve only one application using the database that has been created specifically for it, so schema independence in such systems is not important.

As in Simon's third point, when a change in such a system is requested, typically some new data will have to be both persisted, and manipulated onscreen. Even if you can de-couple the DB schema from the codebase, you wouldn't want to. Both will have to be updated to support the new functionality however you go about it.

For lost updates, I would hope that the ORM would supports transactions. And stored procedures.

I do believe that stored procedures have a role to play, for complex or precise SQL queries as you suggest. That is, in my opinion, an 80/20 thing - in most cases you won't need it and an ORM will be quicker and simpler. But I'd hate to be without stored procedures entirely.

November 16, 2008 22:26
 

marcin.kaluza said:

Hi Anthony,

What the user wants to see is very rarely a single entity (mapped 1:1 to a database table) and for this very reason it often makes sense to build entities which are isolated from physical storage (i.e. entities which can be hydrated from a complex query). When this happens and we need means to persist them back to the database, we need to customize CRD operations and this is where stored procedures come handy. You may argue that it is 20% of the cases (I would say even less) but these rare cases can often "make or break" the product.

FYI it is not possible to avoid lost updates when using ORM "out of the box" (just select, inserts and updates). For this you need transaction running in repeatable read isolation level and even if you do this it will fail badly when it comes to update in multiuser environment. Which deserves another post ...

I do not want you or anyone else to get a wrong impression that I am against ORM in any shape or form as personally I am rather fond of NHibernate for complex scenarios and LINQ for simpler ones: they definitely save development time. This saving however comes at a cost which is often forgotten/ignored when people jump on the next shiny bandwagon.

November 17, 2008 21:45
 

Anthony.Steele said:

Marcin

With "Making a database independent from the business layer" we are really just using different definitions.  Should it be made "be made independent" so that entities in code do dot map 1:1 to fields on the database, and be retrieved in other ways? We agree that that's necessary and good.

Can the database layer be "made independent" so that it can be changed for new functionality without changing the entities? My point is that's often not the kind of change that's needed to add functionality.

I agree that the transactions that are frequent and can lock other users out should definitely not be done in relatively long-lived operations where code will begin a transaction, calculate values, sends updates and then commits. But as you say, that case where stored procedures are needed is less than 20%.

November 19, 2008 09:51
Anonymous comments are disabled
Powered by Community Server (Personal Edition), by Telligent Systems