Welcome to EMC Consulting Blogs Sign in | Join | Help

SSIS Junkie

SSIS: Dataflow expressions

Did you know that it is possible to set expressions on components within the dataflow?

Expressions on data-flow components was virtually the very last feature that was added to SSIS 2005 prior to RTM. All components CAN support expressions on their custom properties, the component developer decides whether or not it WILL by setting IDTSCustomProperty90.ExpressionType which is set to one of the values of the DTSCustomPropertyExpressionType enumeration.

Not all components allow you to set their properties using expressions. If they do, then those properties will show up in the properties pane of the dataflow task containing the component. The property is defined using a form of fully-qualified path such as [ComponentName].[PropertyName] or [ComponentName].[ComponentOutput].[ColumnName].[PropertyName].

Here is a little oddity of this feature. Each column in the output of a Derived Column component has a property called [FriendlyExpression] which is the expression that gets applied on that column at execution time. The [FriendlyExpression] property can be set using an expression and herein lies the oddity. The result of the expression has to be, in itself, a valid expression. I can't think of anywhere else in SSIS where this happens.

Expressions on components can be really useful. The best use that I know of is that the SQL command used in a Datareader component can be changed at execution time. The component itself doesn't provide the ability to change the SQL command from a variable like the OLE DB Source component does so putting an expression on the component is the only way to accomplish it.

Any questions on component expressions?

-Jamie

Published 09 October 2007 05:12 by jamie.thomson

Comments

 

Pythian Group Blog » Blog Archive » Log Buffer #66: a Carnival of the Vanities for DBAs said:

October 12, 2007 17:38
 

Philippe Cand said:

Jamie,

I am a little unsure about your comment

"

Expressions on components can be really useful. The best use that I know of is that the SQL command used in a Datareader component can be changed at execution time. The component itself doesn't provide the ability to change the SQL command from a variable like the OLE DB Source component does so putting an expression on the component is the only way to accomplish it.

"

I am actually building SQL commands on the fly for DataReaders data Flows using the expression for each datapull on the control flow  by combining strings with variables at runtime.

Can you please clarify this? is that has something to do with the limitation on variables values set at runtime AFTER pre-validation?

Cheers,

Philippe

October 16, 2007 02:25
 

jamie.thomson said:

Philippe,

I'm not quite sure what you're asking. You are currently doing exactly what I'm suggesting.

-Jamie

October 16, 2007 14:09
 

Matt Reimer said:

Hi

I am attempting to use the sqlcommand property with a Datareader to implement a query for an incremental load of a table.  I read the last log id tranferred and add it to the WHERE clause at runtime.  This works fine for most of the tables but for one monster table (large number of columns not rows) the expression is too large for the expression editor. (over 4000 char)  Is it possible to bypass the expression and set [ComponentName].[PropertyName] in a script task?

October 17, 2007 05:19
 

jamie.thomson said:

Matt,

Yeah, unfortunately this is a huge limitation of expressions and there is no way around it for your scenario. That I know of anyway.

Wish I had a better answer for you.

-Jamie

October 17, 2007 16:42
 

Jim said:

One thing about using expressions, the property that you define with the expression looks just like it was hard-coded in the component. This is particularly deceiving when you've used an expression for a SQL statement as you can open and even edit the SQL, but it doesn't save the edit because the expression overrides it.

So, as a trick, I've taken to putting the comment "-- NOTE: This is defined by an expression" as the first line in any of my SQL. Now, when I open the component, I have a reminder of where to look to edit the code. It's proven very helpful ...

October 26, 2007 14:31
 

JoeSalvatore said:

I think Jim's commenting the first line of SQL to indicate that it is defined by an expression is great. I've been caught many times trying to figure out why a given SQL change was not "taking" only to realize it was being overwritten (or is that overshadowed) by an expression. Thanks, Jim!

November 6, 2007 03:56
New Comments to this post are disabled

This Blog

Syndication

Powered by Community Server (Personal Edition), by Telligent Systems