Welcome to EMC Consulting Blogs Sign in | Join | Help

SSIS Junkie

Once upon a time this blog was a hive of activity. Now however its pretty lifeless as you can probably tell so if are pining for more of the same you can find me over at http://sqlblog.com/blogs/jamie_thomson. I look forward to seeing you there!

SSIS: Datetime variables don't always do what you expect

I recently came across some slightly strange behaviour on one of my SSIS packages and I thought it was worth posting up here to make people aware of it.

I was using the "Execute SQL Task" to insert values into a SQL Server datetime field. The values were stored in a SSIS datetime variable and I was using a parameterised query to insert them. The problem was that the day and the month were getting reversed so for example 11th March was getting inserted as 3rd November...12th March as 3rd December etc...

It turned out that the problem was outside of SSIS. The datetime value in SSIS is effectively passed to SQL Server as (e.g.) "12/03/2005 12:34:56" and it is the default language setting of the user under which the connection is being made that determines whether that string is interpreted as 12th March (i.e. the correct date) or 3rd December (the incorrect date).

I tested under 2 conditions, once with a default language of 'british' and the other with a default language of 'us_english'. In the case of the former my date was inserted correctly, and vice versa in the case of the latter.

Allan Mitchell helped me to work out what was going on here and in his words, SQL Server is effectively saying to itself :

Jamie is american.  He said "05/03/2005".  I am English.  I read his
date in my format like this "03/05/2005"

which is a simplified summing up of the problem here.

[Note that changing the regional date and time settings on the box had no influence on this behaviour.]

I'm hoping something can be done to make sure this isn't an issue however because the issue is outside of SSIS I doubt that anything CAN be done. Its just something we're going to have to live with I guess! As long as people are aware of it. That's the important thing.

If you want to see what the default langauge is for each user then from within SQL Server Management Studio execute:

select * from sys.syslogins

And if you want to change a user's default language you will need something like the following:

EXEC sp_defaultlanguage 'int\jamie.thomson', 'british'

or

EXEC sp_defaultlanguage 'int\jamie.thomson', 'us_english'

-Jamie

============================================================================

UPDATE 13th March 2005: Mark Durley from the SSIS dev team is currently looking into this issue so watch this space for updates.

In the meantime, following Paul Shotts' comment to this post (see comments section below) I looked into alternative methods of using an SSIS datetime where the the user's default language wouldn't cause any ambiguity. The solution is to use the SSIS DATEPART function to build a string in the form YYYY-MM-DD HH:MI:SS which contains no ambiguity about which number represents the date and which number represents the Month. This is done using expressions. In my Execute SQL Task I have an expression on the SQLStatementSource property as follows:

"insert into DatesDemo values ( '" +
(DT_STR, 4, 1252) DATEPART("yyyy", @[System::StartTime]) + "-" +
(DT_STR, 4, 1252) DATEPART("mm", @[System::StartTime]) + "-" +
(DT_STR, 4, 1252) DATEPART("dd", @[System::StartTime]) + " " +
(DT_STR, 4, 1252) DATEPART("hh", @[System::StartTime]) + ":" +
(DT_STR, 4, 1252) DATEPART("mi", @[System::StartTime]) + ":" +
(DT_STR, 4, 1252) DATEPART("ss", @[System::StartTime]) +
"')"

This works fine and always inserts the correct date. Hopefully Mark will come up with a fix that means I don't have to do this because its an overhead that I don't want to and shouldn't need to put up with.

Published 12 March 2005 16:38 by jamie.thomson

Comments

 

jamie.thomson said:

Could the datetime value not be passed in a more international standard format like 'YYYY-MM-DD HH:MM:SS'?
Otherwise an alternative is always to use a convert function in parameterised SQL statements.
March 12, 2005 17:02
 

jamie.thomson said:

My final workaround was to get the date into an unambigous format as a string (dd mmm yyyy) and then treat it as a string and NOT a datetime.. not very elegant and potential prone to problems but potentially less fiddly then having to concatenate together the UTC style format each time...
March 14, 2005 08:18
 

jamie.thomson said:

I use allways YYYYMMDD without any seperator. I works allways.
Willfried
March 14, 2005 10:34
 

jamie.thomson said:

Willfried,
In my case I need time precision as well. I don't think YYYYMMDDHHMISS will work although I will test it later and find out.
Either way, I still have to build up a string so its a moot point really.

Thanks for your comment.

-Jamie
March 14, 2005 10:57
 

jamie.thomson said:

Having just had a similar problem, "Invalid character value for cast specification." when inserting a DateTime variable into a SQL table, it turned out the issue was not my date formats, but the parameter mapping Data Type. Using DATE works, but it took me a while as I went through the other date named types - DBDATE, DBTIME and DBTIMESTAMP.
March 20, 2005 00:33
 

TrackBack said:

April 26, 2005 22:15
 

Joe Salvatore said:

Interesting read! I've a follow up on using datetime within SSIS expressions. Carefully consider your approach when converting from Datetime to String: Option 1
(DT_STR, 4, 1252) DATEPART("yyyy", @[System::StartTime]) + "-" +
(DT_STR, 4, 1252) DATEPART("mm", @[System::StartTime]) + "-" +
(DT_STR, 4, 1252) DATEPART("dd", @[System::StartTime]) + " " +
(DT_STR, 4, 1252) DATEPART("hh", @[System::StartTime]) + ":" +
(DT_STR, 4, 1252) DATEPART("mi", @[System::StartTime]) + ":" +
(DT_STR, 4, 1252) DATEPART("ss", @[System::StartTime])

Option 2
SUBSTRING((DT_STR, 30, 1252)GETDATE(), 1, 19 )

Option 3
SUBSTRING((DT_STR, 30, 1252)@[System::StartTime], 1, 19 )


Option 1 returns 2006-1-5 9:6:5
Option 2 returns 2006-01-05 09:06:05
Option 3 returns 1/5/2006 9:06:05 AM
January 5, 2006 15:32
 

Marcus Lee said:

You can just change the date format will do:

SET DATEFORMAT dmy;

declare @date datetime

set @date =  '12/03/2005'

print @date

http://msdn2.microsoft.com/en-us/library/ms189491.aspx

December 3, 2007 01:58
 

Marcus Lee said:

Oh...sorry guys, seems like it doesn't work in SSIS.

December 3, 2007 02:23
 

SSISfriedmybrain said:

um you guys may have worked this out already as this is a pretty old thread.

my solution was to pass in the dates as 'date' variables under 'Parameter mapping' and then use the '?' parameter in your SqlStatement Expression.

simple example:

"SELECT <somecolumnname> from <sometable>

WHERE (MyDate >= ?)"

Hope this makes sense

February 28, 2008 05:13
 

Tim Toennies said:

Additionally if you're tempted to use this generated expression as a filename you have to replace "/" & ":" with another character as neither are permitted in the name of a file.  Using Joe's Option 2 as a starting point you may want to try this:

REPLACE(SUBSTRING((DT_STR, 30, 1252)GETDATE(), 1, 19 ),":","-")

March 12, 2008 15:51
 

jbanko said:

trying to build a filename from todays date and can't figure out how to change GETDATE() to a string 'YYYYMMDD'

It can't be this hard..........

thx

joeb

June 16, 2008 18:45
 

jbanko said:

Oh yeah,

this is in expression builder. Want the filename to be

'YYYYMMDD_835.txt'

for the destination property in a file system task

thx

joeb

June 16, 2008 18:53
 

jamie.thomson said:

June 17, 2008 09:29
 

Adam Tappis said:

Just had some fun with this when implementing custom logging. Basically I'm constructing an XML string and including in there both @[System::StartTime] and GETDATE() and I get the following:

   <StartTime>11/07/2008 18:06:45</StartTime>

   <EndTime>2008-07-11 18:06:53.452000000</EndTime>

As both are of different types i.e. one is a DB datetime and the other is a .NET DateTime, I thought I should first normalise them to the same type so they convert consistently. So I added another type conversion to my expression e.g.

(DT_WSTR, 50) (DT_DBTIMESTAMP) @[System::StartTime]

(DT_WSTR, 50) (DT_DBTIMESTAMP) GETDATE()

* I tend to use DT_WSTR as I don't like explicitly defining the codepage

Now I get:

   <StartTime>2008-07-11 18:06:45</StartTime>

   <EndTime>2008-07-11 18:06:53.452000000</EndTime>

As I'm not interested too much in the milliseconds I use SUBSTRING(,1,19) and hey presto, they look the same:

   <StartTime>2008-07-11 18:06:45</StartTime>

   <EndTime>2008-07-11 18:06:53</EndTime>

Hope this helps.

Adam Tappis.

July 11, 2008 19:46
 

jamie.thomson said:

Adam,

I really appreciate you taking the time to share this. Thank you very much.

-Jamie

July 11, 2008 20:05
 

Chris said:

Found this on my quest for the perfect formatting, and similar to Adam's formatting, I have found that for my use, the following works best for my email notifications

using the following casts...

(DT_WSTR,30)(DT_DATE)@[System::StartTime]  

(DT_WSTR, 30) (DT_DATE) GETDATE()

generates something like...

Start Time: 8/13/2008 11:14:06 AM

End Time: 8/13/2008 11:14:07 AM

which ends up being very easy to read.

August 13, 2008 17:19
 

srinivas said:

I have a problem with my SSIS package.

I am trying to convert a unicode String (sometimes empty or null) to Datetime while importing an excel file into my SQL server database.

I tried derived column and date conversion transformations but in vain.

Any help is appreciated. Thanks in advance.

Sri

August 27, 2008 20:58
 

Nicolas said:

hi,

i find the validation of dates in SSIS to be very poor.

so i am trying to find a way validating dates in a script task but i got no success.

maybe one you the gurus here could try and see where the error lies.

regards

Nicolas

===================================================

Imports System

Imports System.Data

Imports System.Math

Imports Microsoft.SqlServer.Dts.Pipeline.Wrapper

Imports Microsoft.SqlServer.Dts.Runtime.Wrapper

Imports System.Text.RegularExpressions

Public Class ScriptMain

   Inherits UserComponent

   Public Overrides Sub Input0_ProcessInputRow(ByVal Row As Input0Buffer)

       validateStr2DateIsoFromEE(Row.myDt, _

       Row.dtValidated, Row.dtValidated_IsNull)

   End Sub

   Sub validateStr2DateIsoFromEE( ByVal aStr As String, _

       ByRef outDate As Date, ByRef outDate_isNull As Boolean )

       Dim provider As IFormatProvider

       Dim dt As Date

       Dim pattern As String = "\b(\d{8})\b"

       Dim rgx As Regex = New Regex(pattern)

       Dim m As Match = rgx.Match(aStr)

       If m.Success Then

           If Date.TryParseExact(aStr, "yyyyMMdd", Nothing, Globalization.DateTimeStyles.None, dt) Then

               outMsg = "valid date"

               outDate = dt

           Else

               outMsg = "only numbers - but not valid date"

               outDate_isNull = True

           End If

       Else

           outMsg = "other chars besides numbers; or a length different then 8 chars"

           outDate_isNull = True

       End If

msgbox("the validation was: "+ outMsg )

   End Sub

End Class

===================================================

October 15, 2008 15:12
New Comments to this post are disabled

This Blog

Syndication

Powered by Community Server (Personal Edition), by Telligent Systems