Welcome to EMC Consulting Blogs Sign in | Join | Help

Guy Sturgess' Blog

  • MCTS for SQL Server 2008 - Business Intelligence Development and Maintenance Part II

    Well here we are, I promised in my last post that I would let you know how my studying was going on and also share with everyone anything I find of interest in the course of my studies.

    I have now done the first 4 chapters of the book which are all focused on SSIS before moving onto both the areas of SSAS and SSRS required by the syllabus.  My overall impression of the book is very good it is split down into a series of lessons within each chapter each with a time estimate (Which I think are a little short myself) followed by an exercise or two to allow you to practice what you have learned.

    I am not going to replay the entire book here but it has highlighted a few good points which are well worth I think sharing.  The first is a couple of short comings with the book, well not so much short comings but more a limitation of the book being a printed object.

    The first of these is the link produced in the book to the AdventureWorks2008 database installs which the book is based on.  Unfortunately Microsoft have seen it fit to move the database install and it can now be found at http://msftdbprodsamples.codeplex.com/releases/view/37109 don't be tempted to use the R2 one as I have done a comparison and they are suitably different which I think may catch you out later.

    The only other issue I have had is with the Exercise 1 in Chapter 3 which makes use of a pre-written SSIS package that imports a series of Excel spreadsheets into a Table in SQL.  The first issue is that the table they are imported into doesn't actually exist. Maybe this is an omission in the book or the fact that it used to exist in the sample database but no longer does.  After a bit of digging around in the SSIS package I managed to produce my own table which seems to work fine.

    The script for which is shown below:-

    USE [AdventureWorks2008]

    GO

    SET ANSI_NULLS ON

    GO

    SET QUOTED_IDENTIFIER ON

    GO

    CREATE TABLE [dbo].[stg_Customer](

    [FirstName] [nvarchar](255) NOT NULL,

    [LastName] [nvarchar](255) NOT NULL,

    [FullName] [nvarchar](511) NOT NULL

    [Address1] [nvarchar](255) NOT NULL,

    [City] [nvarchar](255) NOT NULL,

    [State] [nvarchar](255) NOT NULL,

    [Phone] [nvarchar](255) NOT NULL,

    [Age] [int] NULL,

    [FileName] [nvarchar](255) NOT NULL,

    [LoadDateTime] [datetime] NOT NULL,

    ) ON [PRIMARY]

    GO

    This seems to fix the missing table issue.  You will also find that when you run it using the sample Excel spreadsheets you get an error in the Data Flow shape, more precisely with the Data Conversion.  There is a couple of ways to fix this but the simplest is to edit the shape and select Configure Error Output and set both to ignore failure. 

    (See Image1 In Attached Document)

    This should fix it; if you want to go the step further you could add additional code to replace the resulting Null column. I used the existing Derived Column shape and added an extra derivation to cope with the Nulls. ISNULL(New_Age)?0:New_Age

    (See Image2  In Attached Document)

    That's it for issues I have found so far, I will of course mention any more I find as I find them in later posts.

    So now onto the items I think the book does really well.  The first is it explains a very good pattern for use with configuration settings allowing you to store your entire configuration details in SQL server and then reference them indirectly.  I have read a few times of this technique but the book lays it out in real simple terms.

    If you want to know more about this take a look at Jamie Thompson's very good blog post on the subject: - http://consultingblogs.emc.com/jamiethomson/archive/2005/11/02/SSIS_3A00_-Indirect-configurations-ROCK_2100_.aspx as he explains it a lot better than I ever could.

    The other thing I think the book does well is relate the ideas it is describing to real world situations, the book is full of real world examples from the authors along with at the end of each chapter a case study which you get the chance to solve before reading the answer provided by the Authors.

    A good example of this is the suggested UPSERT pattern which draws in data from a source file and current database table and then uses a Merge shape (Left Outer Join) in the Data flow to generate two flows, one of matched records and one of new records, the new records are directly inserted into the database and the matched records are passed in this instance to a staging table to be used as a data source for a subsequent update statement. 

    (See Image3 In Attached Document)

    As I progress through the book I will blog further and try and share anything I find with others.  Needless to say I am learning a lot here and sharing what I think would be of interest to others.

     

  • MCTS for SQL Server 2008 - Business Intelligence Development and Maintenance Part 1

    So after a little while away from blogging I am back and I thought this time would start by covering some of my experiences preparing for the MCTS exam 70-448 which radically covers SSIS, SSRS and SSAS.  Thats enough acronyms for now so on with the post.

    The first is where to start with the prep for the exam.  I have been working now with SSIS and SSRS and a little SSAS over the last few months so I thought I would already know a lot of the items required for the exam.  How wrong can you be.  I took a couple of online free test that are linked off http://www.braindumps.com/70-448.htm and soon discovered that the exam covers a lot of the rarely used features of tools.

     I suppose this should be no surprise as I have already done 2 MCTS exams for BizTalk 2006 and WCF and both I found to a certain extend grassed over the basics and concentrated on certain features such as the rules engine in BizTalk.  I am now in the position of were do I go next.

    A quick Google brought up a book by Microsoft press so 10 minutes and £30 I am now the over of the official self paced training kit for the exam.  A brief skim through the book shows it covers everything I need and comes with both practice exams and also 180 day version of SQL Server 2008 Enterprise edition.  Having and MSDN subscription I thought I would ignore this version and download and install the full Developer edition.  

    Download not a problem,just over 3gb so not to bad but now here is my first problem and I suppose the main reason for this post.  I tried to install SQL server on my laptop and came to the page which checks for all required pre-requistes and it stated that I have everything installed but I needed to re-boot my laptop.  Bit odd but hey-ho I reboot the machine and run the install again and to my surprise it stated again that I needed to re-boot my laptop.

    So time to start digging around in Google.  I managed to find a few articles that suggested a couple of things to try such as reinstalling Visual Studio 2008 and then trying again and none of the worked until finally I found an article that gave me the answer. 

    There is an issue within windows (XP at least) that means if certain operations in an install fail items are left lying around.  One of which is a registry entry used for file re-names on startup.  Normally part of an uninstall process.  SQL Server uses the fact that this registry key is entry to determine if the machine needs a reboot.

    The answer is then open REGEDIT, browse to the key HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\Session Manager\PendingFileRenameOperations you will see a list of registry entries, a quick check of each one will confirm if the file still exists or not and if not simple delete the list of keys.

    Re-run the install and away you go.  It then said to download the latest version of the AdventureWorks2008 database.  The link in the book is no longer valid but a quick Google (What did we do before Google???) and the link was found:-

    http://msftdbprodsamples.codeplex.com/releases/view/37109

    A tiny download and quick install and I was away.

    I will keep you updated on my progress over the coming weeks and with my exam booked for mid May time is going fast.

      

  • Custom Documentum 6.5 connector for FAST ESP

    I have spent the last couple of weeks getting my head around the FAST ESP product and particular around the process of creating custom content connectors to import documents into to FAST.  I have actually developed 2 Custom connectors the first is a BizTalk adapter to take any document handles by BizTalk and make it search-able.

    The second is a custom connector for Documentum which is what I am going to talk about today.  Both FAST and Documentum come with a number of APIs that allow for the easy export and import of documents and related actions.  The way I will approach this is to talk about both products on there own and then discuss the choices I made and why and finally a high level view of how the connector has been implemented.

    As I work for EMC I suppose I should really start with Documentum, to be more precise Documentum 6.5.  Out of the box Documentum provides two methods for accessing the repository and extracting and importing both documents and users.  The first is a series of DLL's called the Documentum Foundation Classes (DFC).  These are a set of .Net classes that allow you to interface directly with a Documentum repository but as they are classes then must be local to the Documentum install.

    The second option is a web service and a pre generated set of proxy classes called Documentum Foundation Services (DFS). DFS has been around for a little while now in fact since Documentum 6.0 and replaced the old web service framework that  previous versions of Documentum provided.  It exposes through the client side proxy an array of functionality.  

    Along with the client side pre-generated proxies is a very good SDK with a wide range of examples covering most things you would like to do.  Another nice feature is that all traffic between the client and the service is handled by WCF services with either HTTP binding for cross server or named pipe for local traffic all done using MTOM encoding to reduce message size.

    So now onto FAST ESP like Documentum fast offers two flavors of connect management.  This time though rather than being two separate approaches to the same problem they are developed in two separate languages and support different functionality.  The first is a .Net API called the ContentAPI which allows an HTTP connection to be built to a FAST ESP content distributor and documents be added, deleted or updated.

    The second item provided by FAST is the Content Connector Toolkit (CCTK) which is a Java API what as well as allowing for the management of content in the same way as the ContentAPI it also allows for the maintenance of the Security Access Module (SAM) user database, I couldn't get a clear answer from FAST on whether a new .Net API will be developed to support SAM connectivity but I am sure now they are owned by Microsoft it won't be far away and I will keep you informed.

    So thats the two products covered an the tools offered, now to which ones I chose and why.  Continuing the theme I will start with Documentum.  I looked at both the DFC and DFS offerings and in terms of functionality offered there is only minor differences but I felt that DFS with its web service connection offered a far more flexible approach to software deployment so thats the one I chose.

    Onto FAST, this is a slightly easier choice mainly as I was planning to develop the connector in .Net the ContentAPI seemed the logical choice, I could of looked at wrapping the CCTK with something like JNBridge or IKVM and in fact I have done that as I needed to look at updating SAM but for the initial content feed the ContentAPI gave me exactly what I wanted in an easy to code against format.

    As stated above one of the key choices here was software deployment, when you look at the size of the network traffic generated by both APIs you can see that the DFS API document size is a lot smaller due to the encoding used so if network permissions allow I would recommend placing the software closer in the network to the FAST content distributor than the Documentum service.  Although over a few documents this is not such an issue when running large upload the performance degradation must be taken into account.

    Now onto the implementation itself, the key here is to extract documents from a Documentum repository and post them to FAST ESP to make them searchable.  The first stage of this is as you may expect extracting the documents and deciding what is to be sent to FAST.

    There are two approaches here and both have there merits and issues, the first option is to execute a DQL query selecting all the documents either from a cabinet, folder or even a type from Documentum and then comparing the last changed date against a repository of id's and last changed dates held in a SQL database and from this extracting a list of deltas to be applied.  This is in fact how the FAST connector for Documentum 5.3 actually works but rather than using SQL it actually stores it in a proprietary binary format on the local disk which is very fast and I understand follows the same indexing approach as the FAST server itself.

    The second approach and the one I chose was to run a DQL statement again against a cabinet, which returns all the documents updated in a specified period, in my case all documents updated in the last X days but you could change that to simply use a date range, time range in fact even just documents added.  With DQL it is fairly easy to return just what you want.

    Now we have a list of the documents in question it is a simple case of iterating through all the documents in the list and using a method provided in the SDK performing a get on the documents with body contents.  From this you end up with a data object that not only contains all the meta data about the document such as the title, folder location and date and time information but also the body of the document itself which which the returned an object allows the body to be displayed as either a string or a byte array which goes quite nicely with the concept in fast that the body of the document is best submitted as a byte array.

    It is from here a simple case of using a feeder generated using a factory method on the ContentAPI to build a FAST document and submit it to a FAST content distributor.  Of course in the process of all this I have made configurable all the parameters for the content distributor, the DQL to be run and the Documentum store to use, which must meet the same format requirements as the one provided in the DFS sample application.

    So that it a simple feeder that searches for a list of documents and then submits them to FAST for processing and indexing.  Just a few other points before I finish.  I have begun playing with the CCTK wrapping it so I can call it from .Net and I will update this blog post later with how that goes and a small note on how I have structured the solution.  I have created a number of projects, one that contains the FAST implementation, one that contains the Documentum implementation and then two separate projects one that implements it all as a console app and one that implements it as a windows service.    

    I have heard in the last few days that FAST now produce a connector for Documentum 6.5 so when I get a copy I will pit it against my own and see how mine performs.  Could give some interesting results as mine will probably be simpler but with limited functionality against theirs which has a vast array of functionality out of the box.

    I hope you find this post interesting and if you have any further comments or question please contact me directly. 



  • How to develop in Python , Part VI 'Exception handling and file access'

    Subtitle: "My philosophy, like color television, is all there in black and white"

    So here we are at the sixth and final part of my guide on how to develop in Python and as promised in the last post I am going to cover exception handling as you know we all right perfect code and also file access.  But to start with lets look at exceptions.

     To start with lets write a program that will always file so open Eclipse and add a source file, I am sure you can do this without instruction by now.  The name of the source file at this point is not important so just pick a name.  I called mine error but the name is not important.

    Now in the source file add the following line of code and run the code and see what happens:-

    print raw_input("Enter a number) - 1 

    If you entered a number you should get an error that looks something like the one below:

    Traceback (most recent call last):

        File "/home/guy/error.py", line 2, in print raw_input("Enter a number") - 1

    Typeerror: unsupported operand type(s) for -: 'str' and 'int'

    As you can see this is a fairly short error message but this is mainly due to this being a simple error in the base method.  If it was further down a call stack you would get an entire stack trace.  So what is it trying to tell us, well the key is the last two lines really.

    The second to last line actually shows the point at which the error occurred, so we know where it happened but what happened.  This is where the last line comes in which actually shows the error.  In this case it is a type error caused by the fast that the raw_input function always returns a string and then we are trying to do maths on a string field.

    It is simple enough to fix by changing the line to be:

    print input("Enter a number") - 1

    As you can see it all works as expected, what happens though if you enter say a letter rather than a number you get a different error thrown and we have all been there in the past with a piece of code that works fine till you give it to a tester and he always find the one thing you overlooked.

    Python provides a method of handling exceptions so you can allow for retry or display something to the user.  The concept will be familiar to all .Net Dev's who are used to the try catch format although as you would expect the syntax is slightly different.

    Lets take the line above and wrap it up to report an error if it occurs:

    #Open the try catch block, notice the :

    try: 

         print input("Enter a number") - 1

    except:

         print "I said enter a number......"

    This is a basic error handling scenario but shows that with a little thought you can create a simple section of code that retries if an error occurs, I put together the sample below to show a basic model that should allow for most cases, enter the code and see how you get on:

    loop = 1

    #Start the loop

    while loop == 1

         try:

              value = input("Enter a number, (0 exits)") - 1

              print value

         exception:

              print "Invalid entry"

         continue

         if (value == 1):

              loop = 0

     Well thats it a basic program but it shows the basic principals, so lets take it a stage further and look at adding multiple exception handlers.  In Python you have the ability to trap different types of errors, some examples of this are the Typeerror we have already seen, Nameerror and Syntaxerror and lots more.

    Handling these differently allows you to handle them differently.  Best way to show this is with a code snippet:-

    #Start the try block

    try:

         #Your code here

    exception Typeerror:

         #Do something

    exception Nameerror:

         #Do something else

    exception Syntaxerror:

         #Do something else

    exception:

        #Final catch all

     

    The final part of this is the ability if you want to raise your own exceptions and then catch and handle them how you want.  The first part of this is to define your exception which is basically just a simple class with a __init__ section with a list of parameters and maybe some other methods to display certain error message formats.

    Then to actually generate the exception you execute a command in the format:

    raise myExceptionClass(Parameters)

    and catch it as normal:

    exception myExceptionClass

    One last nice feature of the try syntax is the ability to add a finally section which is always run even if an exception is thrown, you can only have one though. This can be used to do things such as close readers and validate system state. So the complete format structure is:-

    try:

    exception:

    finally:

    Thats it for exception handling, lets move onto the final topic I am going to include which is file access, something I have been playing with recently so I thought I would include it here.

    Lets start by opening a file, so create a new source file again the name is not important and also a text file called test.txt in the root of the C: drive which contains any text you would like.  Maybe just copy and paste this post in as it will give you a large block to work with.  Now in the source file add the following line

    openfile = open('C:\test.txt','r')

    openfile.read()

    Thats all very good  but what is the open doing well it is opening the file test.txt for read access 'r' this can be changed to 'w' write, 'u' update etc.

    It is all very well but what has it actuality read, you can direct the output of the read to the display using the print command.  Add the line below to the end of the file: 

    print openfile.read()

    Did it fail, when you ran it well it should have.  This is a demonstration of pointers in python, each file reader uses a pointer as to where it is currently in the file so the first read() reads to the end of the file and then the second one fails as the pointer is already at the end of the file.  You can reposition the pointer using the seek function.  Add the following line of code just prior to the print:-

    openfile.seek(0,0)

    This moves the pointer to the start of the file and is made of up two parameters the first is the amount to move either forwards or backwards from a given a point and the second is where to base the movement from 0 means the start of the file, 1 is the current cursor position and 2 is the end of the file.

    You have a number of other functions available to you in python, tell() will tell you the current cursor position, readline() will read from the current position to the next carriage return and one powerful one which is readlines() which will read from the current cursor position and forward and will return a list object of all the lines in the file split by carriage returns.  write() which as you may expect writes to a file.

    The final function of note is close() function which releases resources and should always be called to close a file, maybe through a try, finally block.

    The last item on file access is pickles, no I haven't finally lost it...  Python provides a method to save an entire object to a file as a binary representation which can later be restored if needed.  I have never used this myself but I am sure someone can give me a good use for it?

    So how do you do it well the annotated code below shows the pickling and unpickling of a list:

    #Import the pickle module

    import pickle

    #Create a list

    list = ['Monday', 'Tuesday', 'Wednesday', 'Thursday','Friday']

    #Open a file to take the result

    file = open('pickle.txt','w')

    #Pickle the list

    pickle.dump(list,file)

    #close the file

    file.close()

    #so lets get it back

    uploadFile = open(pickle.txt,'r')

    #Unpickle the object

    readList = pickle.load(uploadFile)

    #Close the file

    uploadFile.close()

    #Just to prove it works

    print readList

    Well that it the end of the last lesson. I hope some of you from this will at least have some basic grasp of python and been keen to learn more.  I have only covered what I think are the main points here but if anyone thinks I have missed anything key then let me know and I will be happy to consider adding it.

    What now? well if you want any more info there is loads out there on the web but a good place to start is http://www.python.org/ which is the official website for the Python programming language and a wealth of useful information.

    Thanks for taking the time to get this far and happy coding.... 




     

  • How to develop in Python, Part V 'Classes and importing modules'

    Subtitle: 'We are the Knights who say..... "Ni"! '

    Here we are, on the penultimate topic for this blog series post on developing in Python. As promised I am going to cover classes and importing modules today as these are closely related topics.

    So firstly what is a class,  A class is the core of any modern Object Oriented Programming language such as Python, most of the time, you will be writing classes. If you have a background in C or C++, C# the list goes on, many class aspects are similar, but some are different. For example, Python doesn't permit functions to exist outside the class declaration like C++;.

    But the concept of data encapsulation is the same in all the modern OOP languages, so a class in Python contains data (fields for example) and operations that manipulate the data (functions). For example, a class that describes an employee would include fields like firstName, lastName, dateOfBirth, basicSalary and department, and methods that operate on the data, such as CalculateSalary() which calculates the employee's basic salary plus the overtime, for example, and another method like RaiseBasicSalary() that raises the basic salary of the employee.

    The use of class allows an increased amount of code reuse for little extra effort and Python like may other languages can support inheritance and polymorphism.

    Taking all the above how do we define a class in Python? Again as in the previous tutorials I am going to use a section of annotated code to try and explain the basic concepts of a class. So start a new source file in eclipse which I am sure if you are this far through you can do by now and lets get coding:

    #Firstly we have to say we are defining a class, note the : at the end

    class windowGlass:

    #Next we have to define the initializer

          def __init__(self,x,y)

              self.x = x

              self.y=y

              tint = clear

              type=standard

         #Now lets define some functions against the class

         def area(self):

              return self.x * self.y

         def tint(self, text):

              self.tint = text

         def type(self,text):

              self.type = text

         def fullDecription(self):

              return "Window is ",self.x, " by ", self.y," is ",self.type," with a glass type of",self.type

    What have we created here then, well basically this is a class that describes a window glass with 4 fields as properties of the window, the x and y lengths of the window glass, the tint and the glass type.  The class also contains a number of functions that can be executed against the it to return things such as the area of the glass or a full description made up of all the properties.

    There is one special function called __init__ which is actually called when an instance of the class is instantiated. In our case this sets a couple of defaults on the class for the type and tint and takes in the lengths of x and y.  

    You will also note that all the functions take in a parameter of self, this is how a class is referred to from within itself and it handled automatically by the interpreter but it has to be there for all functions to work.

    So how do we use this class we have created, as stated above you need to create and instance of a class before you can use it, so lets use the class you have just entered and try a few things.  Just type in this code:-

    #Create an instance of the class

    plainGlass  = windowGlass(25,25)

    Now you have your instance you can use this as a simple collection of variables and functions. Type the code below and then run the code to see what happens:-

    #Print the area 

    print plainGlass.area()

    #Print description

    print plainGlass.fullDescription()

    #Update the type and re-print

    plainGlass.type('Frosted')

    print plainGlass.fullDescription()

    Thats it you now have a working class and as you can see this allows for a high level of code reuse and of course fits with the object oriented model of the world.

    So if you have once class as our windowGlass you can then inherit this class and create another class which has all the functionality of the base class plus any extra that is required. Imagine we move into the stained glass business.  At the end of the day a stained glass window is a glass window with a bit of color so we want all the functionality of the base class plus a bit more. 

    Add the following code section just below the class you have already entered:

    #Inherit from the base class

    class stainedWindow(windowGlass):

    #Define a new initialization to override the base one

          def __init___(self,x,y):

              self.x= x

              self.y = y

              stainedPattern = "None"

    #Add a new method to set the stained pattern

         def stainedPattern(self,text):

              self.stainedPattern = text

    #Override the description method so it includes the new text

          def fullDecription(self):

              return "Window is ",self.x, " by ", self.y," is ",self.type," with a glass type of",self.type, " with a pattern of ",         self.stainedPattern

    OK, now we have our class if you add the code below you will see all the methods from the base class are available plus new overridden methods

    #Create an instance of the class

    glass  = stainedGlass(25,25)

    #Print the area 

    print glass .area()

    #Print description

    print glass.fullDescription()

    #Update the type and re-print

    glass.type('Frosted')

    glass.stainedPattern('Sunrise')

    print glass.fullDescription()

    As you can see classes have a wide array of uses and now we can define them lets look at the best way to maximize re-use.  That is to say creating a module with the class in it and then importing it where it is needed.

    So how do you create a module, well it couldn't be easier so lets create one.  To you current solution file in Eclipse add a new source file called moduleTest.py and cut and paste both the class definitions you have just created into that file and then save it.  Thats it, you have created a module.

    Now to use it in your main source file, it is simply a case of adding an import statement to the top of the source file, in fact it can be anywhere in the source file but lets keep with standards and put our at the top.  You may already see a couple of import lines at the top,if so add this line afterwards, if not just add it to the top:-

    import moduleTest

    and to get all our code working again it is a simple case of locating the two class instantiation lines and adding the module name to them as shown in the example below:-

    glass  = moduleTest.stainedGlass(25,25)

    Once you have changed both lines you should be able to re-run your program with no issues.

    There are a couple of small other points worth noting, you can for example just import certain items from a module if you so wish using the following code as an example, this has the advantage that you can then from the moduleTest. from all your following code:-

    from moduleTest import stainedGlass

    Thats it for today we are almost there, you will hopefully by now be well on your way to developing in Python and for my last post in this series I will cover error handling and file access.  I know we never read from text files or ever have code that generates errors but just in case I think I should cover it.......

     

     

  • How to develop in Python,part IV 'Functions, Tuples, Lists and Dictionaries'

    Subtitle: "This is an ex parrot"

    So here we are at the fourth part of the blog series I am writing on how to develop in Python, I will assume you have already all the other posts in the series so I will dive straight into the meat of this post.

    As stated in the last post I am planning to cover Functions, Tuples, Lists and Dictionaries, this may seem like a bit of an odd mix but what I am hoping to do use functions with complex data structures to pass lists around and also how iterations over complex data structures can be performed.

    Lets get started then with functions, Python provides out of the box a number of pre-defined functions that can be used in your own program along with the ability like any other programming language to define your own.  A good example of a pre-built function is the raw_input() function which allows a user to enter a string and then it is returned to the calling program.

    How do you define your own functions? This is actually quite straight forward and follows the format:-

    def function_name(parameter1,parameter2,....):

         {Your code here}

         return {value to be returned to the main program

    The important points to note here are the colon at the end of the def line like a lot of the python syntax this is very important as it marks the end of the function definition and for those of you from a .Net background or for a matter of fact a lot of programming languages you don't define the return type or the type of any of the parameters. You can in fact have as many parameters as you like passed to the function but it will only return one value, which may sound limiting but with the use of list, dictionaries and tuples this can be got round.

    Enough of the theory lets open eclipse and create a new source file as covered in the previous posts and create our own function that very simply takes in two values and multiplies them and returns a result. The first part is within the source file is to define the actual function:-

    def Multiply(value1,value2):

         return value1 * value2

    see how simple that was so OK it was not that complicated a task, lets no use this in a program so just below the code you have just entered enter the following code:-

    loop = 1

    while loop == 1:

         valueEntered1 = input("Multiply this:")

         valueEntered2=input("By this:")

         return= Multiply(valueEntered1,valueEntered2)

        print valueEntered1,"*",valueEntered2,"=",result

     

    If you run this you will see it will ask for two values and show the result and repeat infinitum, it looks a bit clunky at the moment but could be tidied up by a tweak to the multiply function and the way it is called:

    def Multiply(value1,value2):

         print valueEntered1,"*",valueEntered2,"=",value1*value2   

    loop = 1

    while loop == 1:

         Multiply(input("Multiply this:"),input("By this:"))

    As you can see by nesting functions you can get some real power out of this and the new code base is a lot cleaner. Well thats functions covered so I will now go on and cover the data structures part starting with Tuples.

    Tuples are a nice concept and are basically an imputable list of values that means once they are defined that values can't be changed.  They are good for static lists of data such as days of the week, months of the year you get the idea. 

    How are they defined? like all other thing sin Python syntax is key here, using days of the week as an example you would define it as:

    days = ('Monday','Tuesday','Wednesday','Thursday','Friday','Saturday','Sunday)

    Internal Python takes the entry and turns them into an indexed list with the first items having an index of 0, the second 1 and so on so our example internally would look like:

    Index       Value

    0             Monday

    1             Tuesday

    2             Wednesday

    3             Thursdays

    4             Friday

    5             Saturday

    6             Sunday  

    Lists are similar to Tuples but with one key difference in that they are mutable and are therefore much better suited to everyday uses as generally we want to change data, the main difference is how it is initially defined.  Lets take a basic example and have a little play with some of the functions of lists.

    The best way to approach this is to write a simple program that plays around with a very simple list. Create a new source file and enter the code below.  In a change from the norm in this one I have used the comment (#) notation to explain what is going on:

    #Start by defining the List, note the [] brackets this time  

    planets=["Mars","Earth","Venus","Jupiter"]

    #Print the third planet in our case Venus, note the zero base

    print planets[2]     

    #Print a range of value

    print planets[0:2]

    #Now we need to add a planet, lets add Saturn to the end of the list using the append function which adds to the end of the #list

    planets.append("Saturn")

    #The strange thing with lists in Python is that you don't use the same syntax to delete an entry, lets say we have had #enough of Martian attacks and delete the planet.

    del planets[0]

    #And just to prove the list has been shuffled up

    print planets[0]

    If you run the example you will now understand the basics of lists, there are other functions you can use but I will leave you to find those yourself.

    The final item for today is dictionaries which are basically a list of key value pairs which allow you to hold a value and then look it up by referencing a key, the example I am going to use if a simple translation dictionary but in reality you could have the value set to anything from a simple string to even a list or another dictionary.

    A dictionary is defined in a similar way to a List or Tuple with the main difference being the use of {} brackets and the syntax for the key value pair.  Open a new source file and code again I have used the comment notation to describe what is going on:

    #Create a new dictionary

    translator = {'Monday':'Lundi','Tuesday':'Mardi','Wednesday':'Mercredi','Thursday':'Jeudi','Friday':'Vendredi'}'

    #As you can see I missed the weekend off my list so lets add Saturday

    translator['Saturday']='Samedi'

    #Now lets print the lists first all the keys then all the values

    print translator.keys()

    print translator.values()

    #You can then access individual values using the key

    print translator['Saturday']

    #Like a list you can also delete entries

    del translator['Saturday']

    #And just to prove it

    print translator.keys()

    print translator.values()

    #You can also convert either the keys or values to lists

    keys = translator.keys()

    values = translator.values()

    #With that done I can show you the sort function on the list

    print keys

    keys.sort()

    print keys 

     As stated above there are lots of other functions you can use such as the len function to count the entries in a list or dictionary.

    So that completes todays post which I think is more that long enough so next time I will cover Classes and Importing modules as they are closely linked and then that just leaves File IO and of course error handling.

    Happy coding......

  • How to develop in Python, Part III 'Variables, Loops and Conditions'

    Subtitle: Nobody expects the Spanish Inquisition!

    OK, now to carry on with my learnings so far of Python, for those who have been following the series you will know that this time I am planning to cover variables, loops and conditions which all fall very nicely together.

    Variables as you are more than probably already aware store a value that can then be either used in future calculations, viewed or even updated later. So how does Python implement this?  The first major difference from most other programming languages is that variables when defined do not have a type specified but rather the type is inferred by the interpreter from the first usage.

    So lets look at that a little closer, if we create a new project called VariablesAndLoops in the same way as we did in the last post and then add a module to this called PyDev Module to this called Variables.  The add the following lines to the module:-

    #Declare a numeric variable

    v=1

    #Print the value of v 

    print "V is equal to ",v

    #Increment v by 1

    v=v+1

    #Print the value of v 

    print "V is equal to ",v

    #Now do something strange

    v="I am a tree"

    #Print the value of v

    print "V is equal to ",v

    Run this and see what happens, you would maybe expect this to error when you try and assign a string to a numeric by Python quite happily takes this line and stores the value.  The reason for this is that a variable in Python is not strongly typed like it is in say C#.Net rather it is treated as purely an object which can hold a value be that numeric or string.

    This does mean though that you could get yourself into a mess that the you won't find out till run time as the IDE will happily let you add the line v=v+1 to the end of the code but when you run this as it is interpreted it will run fine till this point and then throw a type error.

    We have already talked about concatenation but here is a point worth noting, if you use the format in the code above it will always add a space for you before the concatenated value but if you changed the code to read print "V is equal to " + v no space will be added.   Just something to watch.

    Thats variables now we have variable we can move onto loops, in Python the main loop types are while and for loops. Which work in slightly different ways so lets start with while loops.  The code sample below which can be entered after your variables code shows a simple while loop and is by far the best way to explain how they work:-

    loopcount = 0

    while loopcount < 10:

         print "This is the",loopcount,"time round"

         loopcount=loopcount+1

    print "Loop Stopped"

    To most of you the format should be somewhat familiar with a while statement that says do this while the loopcount is less that 10 and then each time printing a statement and then incrementing the count.  The important points to note are the ':' at the end of the while line which marks the end of the condition statement and also the indenting as all text that is indented is included in the loop so the last statement as it is not indented will only be run once.

    To complete this small portion just a quick summary of the expression that are valid in Python

    Expression          Function

    <                        Less than

    <=                      Less than or equal to

    >                        More than

    >=                      More than or equal to

    !=                       Not equal to

    <>                      Not equal to (Alternative)

    ==                      Equal to

    All the above will be familiar to most developers.

    Now onto for loops which allow you to iterate over a list, tuple or dictionary all of which I will cover later but for this section I am going to use the simplest list I can think of which is a string, i.e. a list of single characters.  Lets try a simple example which again can be entered into your variable module you have already created:-

    textstring = "Mary had a little lamb"

    for letter in textstring:

         print letter

    print "Thats is the entire string", textstring

    So what have we done here?  we create a simple string variable that contains a sentence and then the for loop is setup so that it will loop through every letter in the string and print it out, finally to prove the string has not been changed it is printed in full at the end.  

    I know this is not a very good example but the power of for loops will come when I start to talk about tuples, lists and dictionaries later.

    The last thing for today is the if statement which Python like every other programming language has.  Again this is very similar in format to most other languages with a couple of minor styling differences. The best way to show this is to produce a short section of code and get you to add this to your already growing module.

    If you enter and run this code and run it you will hopefully see how an if statement works:-

    value = raw_input("Please enter A,B or C.......: ")

    if value == "A":

         print "You entered value A"

    elif value == "B":

         print "You entered value B"

    else:

         print "You didn't enter A or B"  

    When you run this you will be asked to enter a letter which will be stored in a value and then evaluated against the if statement that follows.  The important points to note are the ':' at the end of each statement line and the indentation as that it works in the same way as a loop using indentation to control what is within its boundary of execution.  Other than that it is a standard if, else if, else type structure using the conditions already defined.

    Next time I am going to tackle functions, tuples, lists and dictionaries building on the looping we have already covered and making it actually useful.

  • How to develop in Python, part II 'The Hello World Times'

    Subtitle:"Nudge, nudge, wink, wink, know what I mean"

    OK so I assume by the fact that you are reading part II that the first part has really grabbed your interest or you just like my Monty Python references which I can assure you will now be kept to an absolute minimum as it is time to start going some real Dev work.

    In the first part of this series I talked about how to setup development environment using Eclipse with the Dev plug-in so without further ado lets create our first working Python code which in the great traditions of programming will be a "Hello World" program.

    The first part of this is to create a project within which to perform the work, those familiar with visual studio will see a lot of similarities here but a few key differences exist.  Start by opening eclipse and selecting File, New, PyDev Project. On this screen you will enter the project name, select the grammar version and the Python interpreter to use.

    Set the project name to "HelloWorld", Set the grammar version to 2.3 and the interpreter the second value in drop down list which will be something\python.exe. If you only have one option then select the link below the box, followed by new.

    Give it a name such as Python and then locate the Python.exe file you installed as part of part 1, if you accepted the defaults this will be in the path C:\Python23\Python.exe. Then it is just a matter of accepting all the defaults and and once finished clicking OK to close the open window.

    So you have now added the interpreter so select from the drop down list and select finish to create the actual project.  What you are then presented with is the new project on the left in the PyDev Package Explorer which contains a folder called src and a link to the interpreter.  The src folder is important as this is where all the development will be done.

    As we are ready to begin lets try a simple program, right click the source folder and select New>PyDev Module and a window will appear on the right hand side of the screen for you to enter the name for your file.  I would suggest Hello World, and select a template of Module:Main and click finish.

    You will now have a development window open on the right with a date at the top along with your name and and an opening line of code "if __name__ == '__main__':" which I will explain in more detail later when we cover If statements. Lets start with some simple stuff.  Enter the following lines below the If noting each line will need to be indented by a single Tab as Python works on indentation rather than bracketing to group code within a context:

    print "Hello World"

    print "Hello","World" 

    Now select the green run arrow at the top of the it will ask you to save the changes so accept that and then it will run your code.  You will see a console window at the bottom of the screen and the "Hello World" text will appear twice as the ',' is interpreted as a concatenation character.

    Thats your first Python program but before I finish I just want to finish off with some more print statements that show the basic maths function of Python. (# Marks a comment which is not executed)

    #Addition

    print 1+1

    #Subtraction

    print 5-2

    #Multiplication

    print 5*5

    #Division

    print 25/4

    #Remainder

    print 25%4

    #Exponent

    print 2**4

    That covers the standard math functions and like most other programming languages Python follows the BODMAS (http://www.easymaths.com/What_on_earth_is_Bodmas.htm) order when performing calculations on a longer statement.

    Well thats it for this part, in the next part I am planning to cover variables loops and conditions. 

  • How to develop in Python, the series begins

    Subtitle - He is not the messiah, he is a very naughty boy

    Let me start by providing some background on the reason for this post and the rest of the ones in the series.

    Imagine yourself if you will as a .Net dev not unlike myself sat on a course learning all about how to implement search solutions using FAST ESP (http://www.microsoft.com/enterprisesearch/en/us/Fast.aspx)   and when the subject of custom document processing stages is raised the instructor turns to the class and says now lets look at some Python.

    As you might expect my first thoughts where maybe we would see the 'Dead Parrot Sketch' or maybe an exert from 'Life of Brian' to explain some pertinent point about search technology but no he is talking about a programming language which I must admit I had heard of once but never developed any code in.

    I am not going to go into here the history of the language or extol its benefits and virtues instead of you would like to find out this information then please go to http://www.python.org/ which has a wealth of background and information on the language.  What I am proposing to do is provide over a series of posts my experiences of learning a new programming language which I have drawn out of various tutorials and reference articles.    

     So lets begin with the basics, before you can start developing in any language the first thing you need is a development environment and this was my first challenge, Python is an interpreted language for which you can download various versions of the interpreter from http://www.python.org/ . As FAST ESP uses version 2.3.5 that is the version I went for.

    The next step is of course to decide on the development environment (IDE), coming from a .Net environment I am used to using Visual Studio so the thought of using something like notepad to do my development in fills me with horror so after a little digging around and reading a few other peoples comments I decided to go with an IDE that will be familiar to Java developers Eclipse which can be used with a plug-in called PyDev to allow a fully functioning development environment.

    So lets go through the sequence of getting a development environment up and running:-

     1) Goto http://www.python.org/ and download the relevant version of the language and then double click the EXE to install

    2) Goto http://java.sun.com/ and use the search box to locate the latest JRE download and install this 

    3) Goto http://pydev.org/download.html and follow the link to download a version of Eclipse (I actually used the C/C++ version as I don't care about the language and this is the smallest one)

    4) Extract the downloaded Eclipse version to a folder on a Drive, the location is not important but as it doesn't run an install to create menu options etc I would recommend maybe the root of a main drive

    5) Open Eclipse and accept all defaults when asked

    6) Add Python to the list of update sites (Windows,Preferences,Install/Update,Available Software Sites,Add)

    7) On the Add Site pop-up enter the name as Python and the location as http://pydev.org/updates and select OK

    8) Install the PyDev components by going to Help, Install New Software selecting Python in the Work With list

    9) In the bottom half of the screen two options appear, select both and then click Finish and except all defaults.

    Thats it, I known it was a bit of a slog but you now have a fully functioning IDE for developing your Python code.

    In the next series of posts I am planning to cover of the basics of Python as a language and to maybe perform some comparisons to C#.Net where I feel is appropriate.

     

  • Oracle 10g StartUp Error in Win2k3 - RemoteOperationException: ERROR: Failed to create command process

    This is my first Blog post so be gentle.... 

    We have been having an issue starting Oracle instances in a Windows 2k3 enviroment from the provided management console, you can of course just re-boot the box and they will normally restart fine but this as you can understand is not a totally practical option.  We have found that we can open the console fine and view the status of the current running Oracle instances but if they are stopped and we attempt to start them the error RemoteOperationException: ERROR: Failed to create command process is being thrown.

    After a bit of digging around I managed to locate a solution which I though may be of use to other people and should work for any windows OS although the instructions are aimed at Win2k3 the same principals should apply to other windows enviroments, it would just be a case of translating the instructions.

    Start/Administrative Tools/Local Security Setting

    Double click on the 'Log on as a batch job' 

    Add either an AD or local account in there that you can use when starting the Oracle instance

    Thats it that simple, no restarts required so follow the steps and off you go.   

    You can find these steps on various other sites but another one doesn't hurt.

      

Powered by Community Server (Personal Edition), by Telligent Systems