Rich Newman

August 24, 2013

Connecting to Tibco EMS from a C# Client

Filed under: .net, c#, EMS, Spring.Net, Tibco — Tags: , , , , — richnewman @ 1:04 pm

Introduction

This is a slightly esoteric post, but hopefully it will be of use somewhere.  Below is some reasonably simple code that shows the basic use cases for connecting to a Tibco EMS Server topic from a C# client using Tibco’s TIBCO.EMS.dll.

It actually points at a locally installed instance of EMS.  That is, the server code is running on the workstation.  If you have the Windows EMS installation you can just install it in the default path and the instructions below should work.  Alternatively it’s relatively simple to change the server name from ‘localhost’ and put a username and password in the code to connect to a secure EMS Server.

How to Use the Code

  • Get hold of the Windows Tibco EMS install, and install it on your workstation on the default path.  Note that Tibco do NOT provide this for free: you need to have paid them for a licence.  If you work for a large organization you will probably find you already have a licence.
  • Create a C# console application called ‘TestEMS’, and reference TIBCO.EMS.dll (the C# EMS assembly) from the install.
  • Paste in the code below.
  • Run the code.  You should see a console window showing that a message has been sent by a publisher and received by a subscriber.

If you want to connect to an existing EMS server in your organization all you need is the TIBCO.EMS.dll assembly, not a full client install. You need to change the appropriate parts of the code below to point to the server: change ‘localhost’ to the server name, and add a username and password if they are needed.  Obviously you don’t need to start the server locally if you do this.

What the Sample Does

As you can see, the code starts the local EMS server, then creates a topic publisher and a subscriber to the same topic.  It then sends a ‘Hello World’ message using the publisher.  The subscriber’s message handler prints out the message to the console when it receives it.

The sample code is slightly more sophisticated than the most basic use case.  It also shows how to set a property on a message, and then to how filter the subscription using a message selector based on the property.  The message is sent with an additional property of ‘Owner’ with value ‘Rich Newman’.  The subscriber is only listening for messages with an Owner property that contains the string ‘Rich Newman’.  You can see this if you change the name of the owner in the message that’s sent: the message listener will not get the message.

Code

using System;
using System.Diagnostics;
using System.Threading;
using TIBCO.EMS;

namespace TestEMS
{
    class Program
    {
        static void Main(string[] args)
        {
            Console.WriteLine("Test started");
            new Program().Run();
            Console.ReadLine();
        }

        private void Run()
        {
            StartEMSServer();
            CreateEMSServerTopicPublisher();
            CreateClientTopicSubscriber("Owner LIKE '%Rich Newman%'"); // Pass "" for no message selector
            EMSServerPublishThisMessage("Hello World""Owner""Rich Newman");
        }

        #region EMS Server
        private const string tibcoEMSPath = @"C:\tibco\ems\5.0\bin\";
        private readonly string tibcoEMSExecutable = tibcoEMSPath + "tibemsd.exe";
        private Process tibcoEMSProcess;
        public void StartEMSServer()
        {
            tibcoEMSProcess = new Process();
            ProcessStartInfo processStartInfo = new ProcessStartInfo(tibcoEMSExecutable);
            tibcoEMSProcess.StartInfo = processStartInfo;
            processStartInfo.WorkingDirectory = tibcoEMSPath;
            bool started = tibcoEMSProcess.Start();
            Thread.Sleep(500);
        }

        TopicConnection publisherConnection;
        TopicSession publisherSession;
        TopicPublisher emsServerPublisher;
        private void CreateEMSServerTopicPublisher()
        {
            TopicConnectionFactory factory = new TIBCO.EMS.TopicConnectionFactory("localhost");
            publisherConnection = factory.CreateTopicConnection(""""); // Username, password
            publisherSession = publisherConnection.CreateTopicSession(falseSession.AUTO_ACKNOWLEDGE);
            Topic generalTopic = publisherSession.CreateTopic("GeneralTopic");
            emsServerPublisher = publisherSession.CreatePublisher(generalTopic);

            publisherConnection.Start();
        }

        internal void EMSServerPublishThisMessage(string messagestring propertyNamestring propertyValue)
        {
            TextMessage textMessage = publisherSession.CreateTextMessage();
            textMessage.Text = message;
            textMessage.SetStringProperty(propertyNamepropertyValue);
            emsServerPublisher.Publish(textMessage);
            Console.WriteLine("EMS Publisher published message: " + message);
        }

        #endregion

        #region EMS Client
        TopicConnection subscriberConnection;
        TopicSession subscriberSession;
        private void CreateClientTopicSubscriber(string messageSelector)
        {
            TopicConnectionFactory factory = new TIBCO.EMS.TopicConnectionFactory("localhost");
            subscriberConnection = factory.CreateTopicConnection("""");  // Username, password
            subscriberConnection.Start();
            subscriberSession = subscriberConnection.CreateTopicSession(falseSession.AUTO_ACKNOWLEDGE);
            Topic clientTopic = subscriberSession.CreateTopic("GeneralTopic");
            TopicSubscriber clientTopicSubscriber = subscriberSession.CreateSubscriber(clientTopicmessageSelectortrue);
            clientTopicSubscriber.MessageHandler += new EMSMessageHandler(test_MessageHandler);
        }

        void test_MessageHandler(object senderEMSMessageEventArgs args)
        {
            Console.WriteLine("EMS Client received message: " + args.Message.ToString());
        }

        #endregion
    }
}

December 3, 2012

Asynchronous Programming in .Net: Async and Await for Beginners

Introduction

There are several ways of doing asynchronous programming in .Net.  Visual Studio 2012 introduces a new approach using the ‘await’ and ‘async’ keywords.  These tell the compiler to construct task continuations in quite an unusual way.

I found them quite difficult to understand using the Microsoft documentation, which annoyingly keeps saying how easy they are.

This series of articles is intended to give a quick recap of some previous approaches to asynchronous programming to give us some context, and then to give a quick and hopefully easy introduction to the new keywords

Example

By far the easiest way to get to grips with the new keywords is by seeing an example.  For this initially I am going to use a very basic example: you click a button on a screen, it runs a long-running method, and displays the results of the method on the screen.

Since this article is about asynchronous programming we will want the long-running method to run asynchronously on a background thread.  This means we need to marshal the results back on to the user interface thread to display them.

In the real world the method could be running a report, or calling a web service.  Here we will just use the method below, which sleeps to simulate the long-running process:

        private string LongRunningMethod(string message)
        {
            Thread.Sleep(2000);
            return "Hello " + message;
        }

The method will be called asynchronously from a button click method, with the results assigned to the content of a label.

Coding the Example with Previous Asynchronous C# Approaches

There are at least five standard ways of coding the example above in .Net currently.  This has got so confusing that Microsoft have started giving the various patterns acronyms, such as the ‘EAP‘ and the ‘APM‘.   I’m not going to talk about those as they are effectively deprecated.  However it’s worth having a quick look at how to do our example using some of the other approaches.

Coding the Example by Starting our Own Thread

This simple example is fairly easy to code by just explicitly starting a new thread and then using Invoke or BeginInvoke to get the results back onto the UI thread.  This should be familiar to you:

        private void Button_Click_1(object sender, RoutedEventArgs e)
        {
            new Thread(() => { 
                string result = LongRunningMethod("World");
                Dispatcher.BeginInvoke((Action)(() => Label1.Content = result)); 
            }).Start();
            Label1.Content = "Working...";
        }

We start a new thread and hand it the code we want to run.  This calls the long-running method and then uses Dispatcher.BeginInvoke to call back onto the user interface thread with the result and update our label.

Note that immediately after we start the new thread we set the content of our label to ‘Working…’.  This is to show that the button click method continues immediately on the user interface thread after the new thread is started.

The result is that when we click the button our label says ‘Working…’ almost immediately, and then shows ‘Hello World’ when the long-running method returns.  The user interface will remain responsive whilst the long-running thread is running.

Coding the Example Using the Task Parallel Library (TPL)

More instructive is to revisit how we would do this with tasks using the Task Parallel Library.  We would typically use a task continuation as below.

        private void Button_Click_2(object sender, RoutedEventArgs e)
        {
            Task.Run<string>(() => LongRunningMethod("World"))
                .ContinueWith(ant => Label2.Content = ant.Result, 
                              TaskScheduler.FromCurrentSynchronizationContext());
            Label2.Content = "Working...";
        }

Here we’ve started a task on a background thread using Task.Run.  This is a new construct in .Net 4.5.  However, it is nothing more complicated than Task.Factory.StartNew with preset parameters.  The parameters are the ones you usually want to use.  In particular Task.Run uses the default Task Scheduler and so avoids one of the hidden problems with StartNew.

The task calls the long-running method, and does so on a threadpool thread.  When it is done a continuation runs using ContinueWith.  We want this to run on the user interface thread so it can update our label.  So we specify that it should use the task scheduler in the current synchronization context, which is the user interface thread when the task is set up.

Again we update the label after the task call to show that it returns immediately.  If we run this we’ll see a ‘Working…’ message and then ‘Hello World’ when the long-running method returns.

Coding the Example Using Async and Await

Code

Below is the full code for the async/await implementation of the example above.  We will go through this in detail.

       private void Button_Click_3(object sender, RoutedEventArgs e)
        {
            CallLongRunningMethod();
            Label3.Content = "Working...";        
        }

        private async void CallLongRunningMethod()
        {
            string result = await LongRunningMethodAsync("World");
            Label3.Content = result;
        }

        private Task<string> LongRunningMethodAsync(string message)
        {
            return Task.Run<string>(() => LongRunningMethod(message));
        }

        private string LongRunningMethod(string message)
        {
            Thread.Sleep(2000);
            return "Hello " + message;
        }

Asynchronous Methods

The first thing to realize about the async and await keywords is that by themselves they never start a thread.  They are a way of controlling continuations, not a way of starting asynchronous code.

As a result the usual pattern is to create an asynchronous method that can be used with async/await, or to use an asynchronous method that is already in the framework.  For these purposes a number of new asynchronous methods have been added to the framework.

To be useful to async/await the asynchronous method has to return a task.  The asynchronous method has to start the task it returns as well, something that maybe isn’t so obvious.

So in our example we need to make our synchronous long-running method into an asynchronous method.  The method will start a task to run the long-running method and return it.  The usual approach is to wrap the method in a new method.   It is usual to give the method the same name but append ‘Async’.  Below is the code to do this for the method in our example:

        private Task<string> LongRunningMethodAsync(string message)
        {
            return Task.Run<string>(() => LongRunningMethod(message));
        }

Note that we could use this method directly in our example without async/await.  We could call it and use ‘ContinueWith’ on the return value to effect our continuation in exactly the same way as in the Task Parallel Library code above.  This is true of the new async methods in the framework as well.

Async/Await and Method Scope

Async and await are a smart way of controlling continuations through method scope.  They are used as a pair in a method as shown below:

        private async void CallLongRunningMethod()
        {
            string result = await LongRunningMethodAsync("World");
            Label3.Content = result;
        }

Here async is simply used to tell the compiler that this is an asynchronous method that will have an await in it.  It’s the await itself that’s interesting.

The first line in the method calls LongRunningMethodAsync, clearly.  Remember that LongRunningMethodAsync is returning a long-running task that is running on another thread.  LongRunningMethodAsync starts the task and then returns reasonably quickly.

The await keyword ensures that the remainder of the method does not execute until the long-running task is complete.  It sets up a continuation for the remainder of the method. Once the long-running method is complete the label content will update: note that this happens on the same thread that CallLongRunningMethod is already running on, in this case the user interface thread.

However, the await keyword does not block the thread completely.  Instead control is returned to the calling method on the same thread.  That is, the method that called CallLongRunningMethod will execute at the point after the call was made.

The code that calls LongRunningMethod is below:

        private void Button_Click_3(object sender, RoutedEventArgs e)
        {
            CallLongRunningMethod();
            Label3.Content = "Working...";        
        }

So the end result of this is exactly the same as before.  When the button is clicked the label has content ‘Working…’ almost immediately, and then shows ‘Hello World’ when the long-running task completes.

Return Type

One other thing to note is that LongRunningMethodAsync returns a Task<string>, that is, a Task that returns a string.  However the line below assigns the result of the task to the string variable called ‘result’, not the task itself.

string result = await LongRunningMethodAsync("World");

The await keyword ‘unwraps’ the task.  We could have attempted to access the Result property of the task (string result = LongRunningMethodAsync(“World”).Result.  This would have worked but would have simply blocked the user interface thread until the method completed, which is not what we’re trying to do.

I’ll discuss this further below.

Recap

To recap, the button click calls CallLongRunningMethod, which in turn calls LongRunningMethodAsync, which sets up and runs our long-running task.  When the task is set up (not when it’s completed) control returns to CallLongRunningMethod, where the await keyword passes control back to the button click method.

So almost immediately the label content will be set to “Working…”, and the button click method will exit, leaving the user interface responsive.

When the task is complete the remainder of CallLongRunningMethod executes as a continuation on the user interface thread, and sets the label to “Hello World”.

Async and Await are a Pair

Async and await are always a pair: you can’t use await in a method unless the method is marked async, and if you mark a method async without await in it then you get a compiler warning.  You can of course have multiple awaits in one method as long as it is marked async.

Aside: Using Anonymous Methods with Async/Await

If you compare the code for the Task Parallel Library (TPL) example with the async/await example you’ll see that we’ve had to introduce two new methods for async/await: for this simple example the TPL code is shorter and arguably easier to understand.  However, it is possible to shorten the async/await code using anonymous methods, as below. This shows how we can use anonymous method syntax with async/await, although I think this code is borderline incomprehensible:

        private void Button_Click_4(object sender, RoutedEventArgs e)
        {
            new Action(async () =>
            {
                string result = await Task.Run<string>(() => LongRunningMethod("World"));
                Label4.Content = result;
            }).Invoke();
            Label4.Content = "Working...";
        }

Using the Call Stack to Control Continuations

Overview of Return Values from Methods Marked as Async

There’s one other fundamental aspect of async/await that we have not yet looked at.  In the example above our method marked with the async keyword did not return anything.  However, we can make all our async methods return values wrapped in a task, which means they in turn can be awaited on further up the call stack.  In general this is considered good practice: it means we can control the flow of our continuations more easily.

The compiler makes it easy for us to return a value wrapped in a task from an async method.  In a method marked async the ‘return’ statement works differently from usual.  The compiler doesn’t simply return the value passed with the statement, but instead wraps it in a task and returns that instead.

Example of Return Values from Methods Marked as Async

Again this is easiest to see with our example.  Our method marked as async was CallLongRunningMethod, and this can be altered to return the string result to the calling method as below:

        private async Task<string> CallLongRunningMethodReturn()
        {
            string result = await LongRunningMethodAsync("World");
            return result;
        }

We are returning a string (‘return result’), but the method signature shows the return type as Task<string>.  Personally I think this is a little confusing, but as discussed it means the calling method can await on this method.  Now we can change the calling method as below:

        private async void Button_Click_5(object sender, RoutedEventArgs e)
        {
            Label5.Content = "Working...";
            string result = await CallLongRunningMethodReturn();
            Label5.Content = result;
        }

We can await the method lower down the call stack because it now returns a task we can await on.  What this means in practice is that the code sets up the task and sets it running and then we can await the results from the task when it is complete anywhere in the call stack.  This gives us a lot of flexibility as methods at various points in the stack can carry on executing until they need the results of the call.

As discussed above when we await on a method returning type Task<string> we can just assign the result to a string as shown.  This is clearly related to the ability to just return a string from the method: these are syntactic conveniences to avoid the programmer having to deal directly with the tasks in async/await.

Note that we have to mark our method as ‘async’ in the method signature (‘private async void Button_Click_5′) because it now has an await in it, and they always go together.

What the Code Does

The code above has exactly the same result as the other examples: the label shows ‘Working…’ until the long-running method returns when it shows ‘Hello World’.  When the button is clicked it sets up the task to run the long-running method and then awaits its completion both in CallLongRunningMethodReturn and Button_Click_5.  There is one slight difference in that the click event is awaiting: previously it exited.  However, if you run the examples you’ll see that the user interface remains responsive whilst the task is running.

What’s The Point?

If you’ve followed all the examples so far you may be wondering what the point is of the new keywords.  For this simple example the Task Parallel Library syntax is shorter, cleaner and probably easier to understand than the async/await syntax.  At first sight async/await are a little confusing.

The answer is that for basic examples async/await don’t seem to me to be adding a lot of value, but as soon as you try to do more complex continuations they come into their own.  For example it’s possible to set up multiple tasks in a loop and write very simple code to deal with what happens when they complete, something that is tricky with tasks.  I suggest you look at the examples in the Microsoft documentation which do show the power of the new keywords.

Code

The full code for these examples is available to download.

Conclusion

This article has only covered the basics of the async/await keywords, although I think it’s addressed all the things that were confusing me when trying to learn about them from the Microsoft documentation.  There are some obvious things it hasn’t covered such as cancelling tasks, exception handling, unwrapping tasks (and why you might need to do that) and how to deal with the reentrancy problems that arise.  All of these are covered reasonably well in the documentation.

Personally I think async and await are far from intuitive: the compiler is performing some magic of a kind we don’t usually see in C#.  The result is that we are yielding control in the middle of a method to the calling method until some other task is complete.  Of course we can do similar things with regular task continuations, but the syntax makes regular continuations look slightly less magical.

However, async/await are a powerful way of controlling multithreaded code once you understand what they are doing.  They can make fairly complex threading look simple.

November 21, 2012

Why Starting a New Task in the Task Parallel Library (TPL) Doesn’t Always Start a New Thread

Filed under: .net, c#, Task Parallel Library, threading, TPL — Tags: , , , , — richnewman @ 4:56 am

The Task Parallel Library in .Net is a wonderful resource.  However, one thing that is a little confusing is that it is actually possible to start a task that runs on the same thread as the current code.  That is, we can do Task.StartNew and it still runs on the same thread.  This isn’t the behaviour we expect.

This can happen particularly when we are writing .Net user interface applications where we marshal data on to the UI thread.

Consider the sample code below, which runs in the button click event of a simple WPF application.  This is a not unusual pattern for a user interface application: we start a Task on a new thread and then perform a continuation on the user interface thread using the current synchronization context, which is the user interface thread synchronization context (TaskScheduler.FromCurrentSynchronizationContext()).  We would typically do this so that we could write the results of the Task into the user interface.  There are several articles on the internet that describe this.

private void button1_Click(object sender, RoutedEventArgs e)
{
    Debug.WriteLine("UI thread: " + Thread.CurrentThread.ManagedThreadId);
    Task.Factory.StartNew(() => 
        Debug.WriteLine("Task 1 thread: " + Thread.CurrentThread.ManagedThreadId))
    .ContinueWith(ant =>
    {
        Debug.WriteLine("Continuation on UI thread: " + Thread.CurrentThread.ManagedThreadId);
        Task.Factory.StartNew(() 
            => Debug.WriteLine("Task 2 thread is UI thread: " + Thread.CurrentThread.ManagedThreadId));
    }, TaskScheduler.FromCurrentSynchronizationContext());
}

However, in the code above we then start a new Task from the continuation (task 2).  This runs on the user interface thread by default.  This is almost certainly not what we are trying to do.  We probably want to run something on a background thread, which is what usually happens when we start a Task.  As a result it’s slightly hazardous behaviour; we can easily end up blocking the user interface thread because we think the code is running on a background thread.

The output from a couple of button clicks using this code is as below:

UI thread: 8
Task 1 thread: 9
Continuation on UI thread: 8
Task 2 thread is UI thread: 8
UI thread: 8
Task 1 thread: 10
Continuation on UI thread: 8
Task 2 thread is UI thread: 8

The reason for this behaviour is that the user interface thread synchronization context is still in force when we start our second task.

Fortunately, in this case at least there is a simple solution, even if it isn’t so obvious.  We can tell the new Task to run explicitly using the default synchronization context instead of the user interface synchronization context when we start it.  The simplest syntax for this changes the code as below.  Notice it passes TaskScheduler.Default as an argument to Task.Start.  The terminology here for a user interface application is a little confusing.  The ‘current synchronization context’ is accessed using TaskScheduler.FromCurrentSynchronizationContext, which we typically do from the user interface thread.  Then the ‘current’ context that results in delegates being queued back to the user interface thread.  The ‘default synchronization context’ is the context that results in delegates being queued to the thread pool.

private void button1_Click(object sender, RoutedEventArgs e)
{
    Debug.WriteLine("UI thread: " + Thread.CurrentThread.ManagedThreadId);
    Task.Factory.StartNew(() => 
        Debug.WriteLine("Task 1 thread: " + Thread.CurrentThread.ManagedThreadId))
    .ContinueWith(ant =>
    {
        Debug.WriteLine("Continuation on UI thread: " + Thread.CurrentThread.ManagedThreadId);
        Task task2 = new Task(() 
            => Debug.WriteLine("Task 2 thread: " + Thread.CurrentThread.ManagedThreadId));
        task2.Start(TaskScheduler.Default);
    }, TaskScheduler.FromCurrentSynchronizationContext());
}

The output from this is now as we would expect:

UI thread: 9
Task 1 thread: 10
Continuation on UI thread: 9
Task 2 thread: 11
UI thread: 9
Task 1 thread: 11
Continuation on UI thread: 9
Task 2 thread: 10

For more detailed discussion on this see:

http://stackoverflow.com/questions/6800705/why-is-taskscheduler-current-the-default-taskscheduler

http://stackoverflow.com/questions/12245935/is-task-factory-startnew-guaranteed-to-use-another-thread-than-the-calling-thr

February 7, 2012

Delegate Syntax in C# for Beginners

Filed under: .net, beginners guide, c#, code syntax, delegate — Tags: , , , , — richnewman @ 3:48 am

Introduction

I have been programming with C# since it came out but I still find the delegate syntax confusing.  This is at least partially because Microsoft have changed the recommended syntax regularly over the years.  This article is a quick recap of the various syntaxes.  It also looks at some of the issues with using them in practice.  It’s worth knowing about all the various syntaxes as you will almost certainly see all of them used.

This article is just a recap: it assumes that you know what a delegate is and why you’d want to use one.

.Net and Visual Studio Versions

The first thing to note is that you can use any of these syntaxes as long as you are using Visual Studio 2008 or later and targeting .Net 2.0 or later.

Named methods were available in .Net 1.0, anonymous methods were introduced in .Net 2.0, and lambda expressions were introduced in .Net 3.0.  However, like much of .Net 3.0, which is based on the .Net 2.0 assemblies, lambda expressions will compile to .Net 2.0 assuming you have the appropriate version of Visual Studio.

Note also that lambda expressions can do (almost) everything anonymous methods can do, and effectively supersede them as the preferred way of writing inline delegate code.

Code

A listing of the code for this article is availableThe complete working program is also available.

The Delegate

For all of these examples we need a delegate definition.  We’ll use the one below initially.

        private delegate void TestDel(string s);

Named Methods

Named methods are perhaps the easiest delegate syntax to understand intuitively.  A delegate is a typesafe method pointer.  So we define a method:

        private void Test(string s)
        {
            Console.WriteLine(s);
        }

Now we create an instance of our method pointer (the delegate above) and point it at our method.  Then we can call our method by invoking the delegate.  The code below prints out ‘Hello World 1′.  This is easy enough, but all a little cumbersome.

            TestDel td = new TestDel(Test);
            td("Hello World 1");

There’s one slight simplification we can use.  Instead of having to explicitly instantiate our delegate with the new keyword we can simply point the delegate directly at the method, as shown below.  This syntax does exactly the same thing as the syntax above, only (maybe) it’s slightly clearer.

            TestDel td2 = Test;
            td2("Hello World 2");

There is an MSDN page on named methods.

Anonymous Methods

The anonymous method syntax was introduced to avoid the need to create a separate method.  We just create the method in the same place we create the delegate.  We use the ‘delegate’ keyword as below.

            TestDel td3 = 
                delegate(string s)
                {
                    Console.WriteLine(s);
                };
            td3("Hello World 3");

Now when we invoke td3 (in the last line) the code between the curly braces executes.

One advantage of this syntax is that we can capture a local variable in the calling method without explicitly passing it into our new method.  We can form a closure.  Since in this example we don’t need to pass our string in as a parameter we use a different delegate:

        private delegate void TestDelNoParams();

We can use this as below.  Note that the message variable is not explicitly passed into our new method, but can nevertheless be used.

            string message = "Hello World 4";
            TestDelNoParams td4 = 
                delegate()
                {
                    Console.WriteLine(message);
                };
            td4();

There is an MSDN page on anonymous methods.

Lambda Expressions

Lambda expressions were primarily introduced to support Linq, but they can be used with delegates in a very similar way to anonymous methods.

There are two basic sorts of lambda expressions.  The first type is an expression lambda.  This can only have one statement (an expression) in its method.  The syntax is below.

            TestDel td5 =  s => Console.WriteLine(s);
            td5("Hello World 5");

The second type is a statement lambda: this can have multiple statements in its method as below.

            string message2 = "Hello World 8";
            TestDel td6 =
                s => 
                { 
                    Console.WriteLine(s); 
                    Console.WriteLine("Hello World 7");
                    Console.WriteLine(message2);
                };
            td6("Hello World 6");

Note that this example also shows a local variable being captured (a closure being created).  We can also capture variables with expression lambdas.

There is an MSDN page on lambda expressions.

Return Values

Nearly all of the examples above can be extended in a simple way to return a value.  The exception is expression lambda which cannot return a value. Doing this is usually an obvious change: we change our delegate signature so that the method it points to returns a value, and then we simply change the method definition to return a value as usual.  For example the statement lambda example above becomes as below.  The invocation of tdr6 now returns “Hello ” + message2, which we write to the console after the invocation returns:

            string message2 = "World 8";
            TestDelReturn tdr6 =
                s =>
                {
                    Console.WriteLine(s);
                    Console.WriteLine("Hello World 7");
                    return "Hello " + message2;
                };
            Console.WriteLine(tdr6("Hello World 6"));

The full list of all the examples above modified to return a value can be seen in the code listing in the method ExamplesWithReturnValues.

Events

All of these syntaxes can be used to set up a method to be called when an event fires.  To add a delegate instance to an event we used the ‘+=’ syntax of course.  Suppose we define an event of type TestDel:

        private event TestDel TestDelEventHandler;

We can add a delegate instance to this event using any of the syntaxes in an obvious way.  For example, to use a statement lambda the syntax is below.  This looks a little odd, but certainly makes it easier to set up and understand event handling code.

            TestDelEventHandler += s => { Console.WriteLine(s); };
            TestDelEventHandler("Hello World 24");

Examples of setting up events using any of the syntaxes above can be found in the code listing.

Passing Delegates into Methods as Parameters: Basic Case

Similarly all of the syntaxes can be used to pass a delegate into a method, which again gives some odd-looking syntax.  Suppose we have a method as below that takes a delegate as a parameter.

        private void CallTestDel(TestDel testDel)
        {
            testDel("Hello World 30");
        }

Then all of the syntaxes below are valid:

            CallTestDel(new TestDel(Test));  // Named method
            CallTestDel(Test);               // Simplified named method
            CallTestDel(delegate(string s) { Console.WriteLine(s); });  // Anonymous method
            CallTestDel(s => Console.WriteLine(s));  // Expression lambda
            CallTestDel(s => { Console.WriteLine(s); Console.WriteLine("Hello World 32"); });  // Statement lambda

Passing Delegates into Methods as Parameters: When You Actually Need a Type of ‘Delegate’

Now suppose we have a method as below that expects a parameter of type Delegate.

        private void CallDelegate(Delegate del)
        {
            del.DynamicInvoke(new object[] { "Hello World 31" });
        }

The Delegate class is the base class for all delegates, so we can pass any delegate into CallDelegate.  However, because the base Delegate class doesn’t know the method signature of the delegate we can’t call Invoke with the correct parameters on the Delegate instance.  Instead we call DynamicInvoke with an object[] array of parameters as shown.

Note that there are some methods that take Delegate as a parameter in the framework (e.g. BeginInvoke on a WPF Dispatcher object).

There’s a slightly unobvious change to the ‘Basic Case’ syntax above if we want to call this method using the anonymous method or lambda expression syntax.  The code below for calling CallDelegate with an expression lambda does NOT work.

            CallDelegate(s => Console.WriteLine(s));  // Expression lambda

The reason is that the compiler needs to create a delegate of an appropriate type, cast it to the base Delegate type, and pass it into the method.  However, it has no idea what type of delegate to create.

To fix this we need to tell the compiler what type of delegate to create (TestDel in this example).  We can do this with the usual casting syntax (and a few more parentheses) as shown below.

            CallDelegate((TestDel)(s => Console.WriteLine(s)));  // Expression lambda

This looks a little strange as we don’t normally need a cast when assigning a derived type to a base type, and in any case we’re apparently casting to a different type to the type the method call needs.  However, this syntax is simply to tell the compiler what type of delegate to create in the first place: the cast to the base type is still implicit.

We need to do this for any of the syntaxes apart from the very basic named method syntax (where we’re explicitly creating the correct delegate):

            CallDelegate(new TestDel(Test));  // Named method
            CallDelegate((TestDel)Test);      // Simplified named method
            CallDelegate((TestDel)delegate(string s) { Console.WriteLine(s); });  // Anonymous method
            CallDelegate((TestDel)(s => Console.WriteLine(s)));  // Expression lambda
            CallDelegate((TestDel)(s => { Console.WriteLine(s); Console.WriteLine("Hello World 32"); }));  // Statement lambda

Actions/Funcs

There is one further simplification that we can use in the examples in this article.  Instead of defining our own delegates (TestDel etc.) we can use the more generic Action and Func delegates provided in the framework.  So, for example, everywhere we use TestDel, which takes a string and returns void, we could use Action<string> instead, since it has the same signature.

August 6, 2011

Closures in C#

Filed under: .net, c#, closure — Tags: , , — richnewman @ 3:08 am

Introduction

There seems to be some confusion about closures in C#: people are mystified as to what they are and there’s even an implication that they don’t work the way you’d expect.

As this short article will explain they are actually quite simple, and do work the way you’d expect if you’re an object oriented programmer.  The article will also briefly look at why there is confusion surrounding them, and discuss whether they are an appropriate tool in an object oriented program.

Closures

Wikipedia defines a closure as ‘a first-class function with free variables that can be bound in the lexical environment’.  What that means is that a ‘closure’ is a function that can access variables from the environment where it is declared without them being explicitly passed in as parameters.  In object-oriented programming a method in a class is a closure of sorts: it can access fields of the class directly without needing them to be passed in.  However, more usually in C# ‘closure’ refers to a function declared as an anonymous delegate that uses variables that are not explicitly passed into it, but are available at the point it is created.

Basic Example

Consider the code below:

        internal void MyFunction()
         {
             int x = 1;
             Action action = () => { x++; Console.WriteLine(x); };
             action();              // Outputs '2'
         }

Here we define an anonymous delegate (a function) ‘action’, and call it.  This increments x and outputs it to the console, even though x isn’t passed into it.  x is simply declared in the ‘lexical environment’ (the calling method).

‘action’ is a closure, and we can say it is ‘closed over’ x, and that x is an ‘upvalue’.  Note that it’s only a closure because of the way it uses x.  Not all anonymous delegates are closures.

By the way, in the Java community anonymous functions frequently are referred to as ‘closures’.  (The Java community has been debating whether ‘closures’ should be added to the language for some time.)

Where’s the Confusion?

The example above is pretty clear and simple.  So how has it caused confusion?

The answer is that the value of x in MyFunction after the ‘action’ call is now 2.  Furthermore, x is completely shared between MyFunction and the action delegate: any code that changes the value in one changes the value in the other.

Consider the code below:

        internal void MyFunction()
         {
             int x = 1;
             Action action = () => { x++; Console.WriteLine(x); };
             action();              // Outputs '2'
             x++;
             action();              // Outputs '4'
         }

Here we call ‘action’, increment x in the calling method (MyFunction), and then call ‘action’ again.  Overall we started with x at 1, and incremented it 3 times, twice in our ‘action’ delegate and once in the calling method.  So it’s no surprise that the shared variable ends up with a value of 4.

This shows that we can change our ‘upvalue’ in the calling method and it is then reflected in our next call to the function: x is genuinely shared in this example.

Whilst this is a little odd (see below) it’s perfectly logical: x is shared between the calling method and any calls to our ‘action’ function.  There’s only one version of x.

This isn’t the way closures work in most functional programming languages (not that you could easily implement the example above, since all variables are immutable in functional languages).  The concept of closures has come from functional languages, so many people are surprised to find them working this way in C#.  There is more on this later.

Closures and Scope

This becomes even odder if we allow the local variable x to go out of scope (which would usually lead to it being destroyed), but retain a reference to the delegate that uses it:

        internal void Run()
        {
             Action action = MyFunction();
             action();             // Outputs '5'
        }

        internal Action MyFunction()
        {
             int x = 1;
             Action action = () => { x++; Console.WriteLine(x); };
             action();              // Outputs '2'
             x++;
             action();              // Outputs '4'
             return action;
        }

Here our Run method retrieves the action delegate from MyFunction and calls it.  When it calls it the value of x is 4 (from the activity in MyFunction), so it increments that and outputs 5.  At this point MyFunction is out of scope so the local variable x would normally have been destroyed.

Again, logically this is what we’d expect, but it looks strange.

This also gives an indication that closures are hard to implement.

A Little Odd?

For an object-oriented programmer this is a little odd.  This is because we’re not used to being able to share local variables with a separate method in this way.

Normally if we want a method to act on a local variable we have to pass it in as a parameter.  If the local variable is a value type as above it gets copied, and changing it in the method will not affect its value in the calling method.  So if we wanted to use it in the calling method we’d have to pass it back explicitly as a return value or an output parameter.

Of course, there are good reasons why we don’t usually allow a method to access any variable in a calling method (quite apart from the practicalities of actually being able to do it with methods other than anonymous functions).  These are to do with encapsulation and ensuring we can maintain state in a way that’s easy to deal with.  We only really allow data to be shared at a class level, or globally if we’re using static variables, although in general we try to keep them to a minimum.

So in some ways closures of this kind break our usual object-oriented encapsulation.  My feeling is that they should be used sparingly in regular object-oriented code as a result.

Other writers have gone further than this, because if you don’t understand that an upvalue is fully shared between the anonymous function and the calling code you can get unexpected behaviour.  See, for example, this article ‘Closures in C# Can Be Evil’.

Conclusion

The concepts behind closures in C# are actually fairly straightforward.  However, if we use them it’s important we understand them and the effects on scope, or we may get behaviour we don’t expect.

June 12, 2009

Extending Classes: Extension Methods or Inheritance?

Filed under: .net, c#, dotnet — Tags: — richnewman @ 12:09 am

Introduction

Extension methods were introduced in C# 3.0 as a way of extending a class without necessarily having access to the original source code of the class.

As discussed in the C# Programming Guide on MSDN, extension methods were primarily introduced to the language to allow LINQ to add standard query operators such as GroupBy and OrderBy to any class that implements IEnumerable<T>.  This is very neat syntactically.

This article will examine how extension methods can be created and used.  It will then show one example where inheritance is probably a better approach to class extension.

Basic Usage of Extension Methods

To illustrate the basic approach to using extension methods as usual a code example is available.

This contains three projects:

  1. A ‘Core’ class library intended to represent code developed by a core group of developers and used as a library by other developers.
  2. A ‘DerivedExtended’ library that is intended to represent code extending the Core library, developed by a second group of developers.
  3. A ‘Client’ library that uses the DerivedExtended library.

Initially the Core library contains just one class with just one public property:

namespace Core
{
    public class MyCoreClass
    {
        public int CoreValue { get; set; }
    }
}

How Extension Methods Work: Writing an Extension Method

We can extend our core class in our DerivedExtended library using extension methods.  To do this clearly DerivedExtended has to reference Core.  The syntax is simple to set up an extension method:

using Core;
 
namespace DerivedExtended
{
    public static class Extended
    {
        public static void SetCoreValue(this MyCoreClass myCoreClass)
        {
            myCoreClass.CoreValue = 1;
        }
    }
}

To make this work both the class and the method in it are declared static, and we use the ‘this’ keyword on the myCoreClass parameter.  The name of the static class (‘Extended’) is not relevant for the purposes of the extension method (it’s not used to invoke it in any way).

Our extension method can only access the public interface of the class it is extending.  That is in our case it can access the CoreValue property because it is public: if it were private the extension method would not be able to set it.

How Extension Methods Work: Calling an Extension Method

We now set up client code to use this:

using System;
using DerivedExtended;
 
namespace Client
{
    class Program
    {
        static void Main(string[] args)
        {
            // Instantiate the core class
            Core.MyCoreClass myCoreClass = new Core.MyCoreClass();
 
            // Call extension method
            myCoreClass.SetCoreValue();
 
            // Get property value from core class and display it
            Console.WriteLine(myCoreClass.CoreValue);
 
            Console.ReadLine();
        }
    }
}

Note the syntax for calling the SetCoreValue extension method: to the client code it looks exactly like a normal instance method on the myCoreClass instance.  Note also that we have to use ‘using DerivedExtended’ directive for the compiler to find the relevant extension method.

Clearly the code above prints ‘1’ to the console window: the extension method has set the property value as you would expect.

Why This is Fragile

Now imagine our core developers decide that they need a method to set the core value, and that the method should be setting the value to 100.  If they are library developers they may know nothing about the client code and DerivedExtended library written by the second development team.  So they make the simple change below:

namespace Core
{
    public class MyCoreClass
    {
        public int CoreValue { get; set; }
 
        public void SetCoreValue()
        {
            myCoreClass.CoreValue = 100;
        }
    }
}

Now if we run our client code we see that the behaviour has been changed: it prints out 100.  However nothing has ostensibly been broken: our DerivedExtended developers could easily rebuild against the new Core library, not spot anything had changed, and ship the code with changed behaviour.  Of course in my simple example it doesn’t look like the effects of this are likely to be catastrophic.  However because of the extension method the DerivedExtended developers now need to be much more careful when the implementation of the Core library changes.

Extension Methods or Inheritance?

Of course, in object-oriented languages we already have a means of extending a class where we don’t necessarily have access to or want to change the original source code: it’s called inheritance.

In C# and other object-oriented languages there is extensive support for allowing a base class writer to control how their class will be extended by inheritance, such as declaring methods as virtual, declaring member variables as protected and so on.  A number of our core design patterns are based around designing classes that can be extended in this way (e.g. Template Method).

Why This Isn’t a Problem with Inheritance

Now consider the same extension as above being made with inheritance.  Firstly we revert our Core class to simply have the CoreValue property:

namespace Core
{
    public class MyCoreClass
    {
        public int CoreValue { get; set; }
    }
}

Then we construct a Derived class that contains a SetCoreValue method that again sets the value to 1.  Thus we extend our code in a new library in the traditional way:

namespace DerivedExtended
{
    public class Derived : Core.MyCoreClass
    {
        public void SetCoreValue()
        {
            base.CoreValue = 1;
        }
    }
}

We can now change our client code to use this extension:

    class Program
    {
        static void Main(string[] args)
        {
            // Now consider case where we extend by inheritance
 
            // Instantiate the Derived class
            Derived derived = new Derived();
 
            // Call the derived method
            derived.SetCoreValue();
 
            // Get property value from derived class and display it
            Console.WriteLine(derived.CoreValue);
 
            Console.ReadLine();
        }
    }

}

And clearly here the code will print out ‘1’ again.

Now, as before, assume our Core library developers add a SetCoreValue method to the Core class, unaware that there already is one in the Derived class:

    public class MyCoreClass
    {
        public int CoreValue { get; set; }
 
        public void SetCoreValue()
        {
            CoreValue = 100;
        }
    }

This doesn’t change the behaviour of the existing client code, which still prints out ‘1’.  However, we get a compiler warning:

‘DerivedExtended.Derived.SetCoreValue()’ hides inherited member ‘Core.MyCoreClass.SetCoreValue()’. Use the new keyword if hiding was intended.

So not only has the code not been broken, but our DerivedExtended developers will get a warning that there is a problem when they come to recompile their code against the new library.

Note also that any code the Core developers write calling SetCoreValue on the Core class will correctly call the Core class version (they don’t know about the Derived class version so can’t want to call it).

This behaviour is deliberate: by default a method in a derived class hides rather than overrides a method with the same signature in a base class if there has been no explicit use of the ‘new’ or ‘virtual/overrides’ keywords.  This is precisely because of the circumstances with regard to extension described here.

Summary

So in summary, in the circumstances described where a development team is extending classes in a library it seems better to do this by inheritance than by extension methods.  Obviously it isn’t always possible to extend a class by inheritance and we may need to use other techniques, but in general extension methods seem fragile to me.

To an extent the same argument applies to any class in our code, whether in a library or not.  Suppose we use extension methods and then later on we alter the implementation of the class we are extending.  We can break our extension methods without necessarily noticing we have done so.  Of course this is true of any code that uses the public interface of our class if its behaviour changes, so is less valid as an argument against extension methods.

MSDN

Microsoft itself agrees with the central argument in this article.  In the page on extension methods in the C# programming guide it says:

General Guidelines

In general, we recommend that you implement extension methods sparingly and only when you have to. Whenever possible, client code that must extend an existing type should do so by creating a new type derived from the existing type. For more information, see Inheritance (C# Programming Guide).

When using an extension method to extend a type whose source code you cannot change, you run the risk that a change in the implementation of the type will cause your extension method to break.

Conclusion

Extension methods can be fragile.  My recommendation is that you don’t write your own extension methods, for the reasons explained above.

March 2, 2008

Model-View-Presenter using the Smart Client Software Factory (Introduction To CAB/SCSF Part 25)

Introduction

Part 23 and part 24 of this series of articles described the Model-View-Presenter pattern.

This article explains how the Smart Client Software Factory supports this pattern by generating appropriate classes.

Guidance Automation Packages in the Smart Client Software Factory

We saw how we could use the Smart Client Application Guidance Automation Package to set up a Smart Client Application in part 18. We can also set up a Model-View-Presenter pattern in a Smart Client application using another of the Guidance Automation Packages.

This will only work in an existing Smart Client Application.

Running the Model-View-Presenter Package

To use the Guidance Automation Package we right-click in Solution Explorer on a project or folder where we want to run the package. It is intended that we do this in the Views folder in a business module. On the right-click menu we select ‘Smart Client Factory/Add View (with presenter)’. We get a configuration screen that lets us name our view, and also lets us put the classes that get created into a folder. For the purposes of this example we name our view ‘Test’, and check the checkbox that says we do want to create a folder for the view.

When we click ‘Finish’ we get three classes and a TestView folder as below:

mvpsolutionexplorer.jpg

Classes Created

  1. TestView
    This is (obviously) our View class. It is intended that this contain the auto-generated code to display the View. As discussed in the previous articles any complex view logic will not go into this class, but will go into the Presenter.
  2. TestViewPresenter
    This is our Presenter class. As discussed in previous articles this should contain logic to deal with user events. It should also contain any complex view logic, and should directly update the View with the results of an view logic calculations. It has access to the View class via an interface.
  3. ITestView
    This is the interface that the View implements. The Presenter can only update the View through this interface.

Diagram

In terms of the diagrams shown in parts 23 and 24 this looks as below. Remember that we may or may not have arrows between the Model and the View depending on whether we are using the active View or passive View version of Model-View-Presenter:

mvpdiagram2.jpg

Where’s the Model?

The Guidance Automation package does not set up a Model class for us. As we have seen, the Model has no direct references to a View/Presenter pair (it raises events), and there may be multiple View/Presenter pairs for one Model. Further the Model would not usually be in the same folder, or even in the same component, as our View and Presenter.

For these reasons we are expected to set up our Model classes separately by hand.

Note that the Presenter (and the View as well if we are using the active View pattern) will have a direct reference to the Model. We will have to add these references manually.

Active and Passive View: a Quick Recap

Remember that in Model-View-Presenter the Presenter updates the View via an interface. We can set this up so only the Presenter is allowed to update the View. This is the ‘passive View’ pattern. We can also set this up so that the Presenter can update the View in complex cases, but the View can also update itself (in response to an event or user request) in simple cases. This is the ‘active View’ pattern.

Active and Passive View: Which Should We Use?

The pattern described in the SCSF documentation is the passive View: the documentation implies that all updates to the View should be done by the Presenter.

However there is nothing to stop us using the active View pattern with the classes generated by the Guidance Automation Package. We can add code to update the View wherever we like. In fact I would recommend using active View in simple cases: passive View should only be used where we are putting too much logic into the View class.

Should We Use Model-View-Presenter for Every Screen? A Personal View

Let me also reiterate a point made in part 24. It’s easy to get obsessive about the use of patterns and use them everywhere without thinking. My personal opinion is that we should only use the full Model-View-Presenter pattern where we have a complex screen that will benefit from the separation of the View and Presenter classes. For very basic screens the pattern is really too complex to give us benefit. In simple cases I think it is fine to put event handling and screen update logic directly behind the screen.

Note that I don’t think this applies to the use of the Model. We should always separate out the business logic from our screens into separate classes (this is what Martin Fowler calls ‘Separated Presentation’). However, we frequently have screens that don’t show any business logic or business data, so we may not need a Model class either.

For example an About screen that just shows the system name and version won’t need separate View and Presenter classes, and probably won’t need anything in a Model class either.

Equally a screen that shows a read-only grid of currencies used in a trading system probably doesn’t need separate View and Presenter classes. In this case the currencies themselves should be in a Model class so that other screens can access them.

Implementation Details: What We’d Expect

If we examine the diagram above, we expect the Presenter to have a data member with type of our ITestView interface that it will use to access the View. We expect the View to implement the ITestView interface to allow this. We further expect the View to have a direct reference to the Presenter class (a data member), which it will use to invoke code relating to user events. We’d probably expect both the View and the Presenter classes to be created the first time the View is needed.

Implementation Details: the Base Presenter Class

The actual details of the implementation of the Presenter are a little unusual.

If we look at the code generated by the Guidance Automation Package we see that the TestViewPresenter above has been given its core functionality by inheriting from an abstract Presenter<TView> class. Remember that the generic ‘TView’ simply lets us provide a type whenever we use the Presenter class. Here we inherit from Presenter, and provide the type when we inherit:

    public partial class TestViewPresenter : Presenter<ITestView>
    {

This allows the base Presenter class to have a data member of type ITestView (which is what we expect), rather than it being directly in the TestViewPresenter class. Note that the base Presenter is in the Infrastructure.Interface project (which is one of the reasons why we have to use this pattern in a Smart Client application).

The base Presenter class exposes our ITestView data member publicly, contains a reference to our WorkItem, and has disposal code and a CloseView method. It also has virtual OnViewReady and OnViewSet methods. These get called when you’d expect from the name and let us respond at the appropriate times by overriding the methods in our TestViewPresenter class.

All the above functionality in the base Presenter class means that the derived TestViewPresenter class is basically empty when it is created. It is up to us to put logic in there to handle user events and complex view logic.

The TestView class is a normal user control. It implements ITestView and contains a reference to the TestViewPresenter as we’d expect. It also calls OnViewReady as appropriate (in the OnLoad event of the user control). Again other than this TestView is also basically empty.

Conclusion

This article has shown us how to set up Model-View-Presenter classes using the Smart Client Software Factory, and discussed some issues surrounding it.

February 23, 2008

Table of Contents: Introduction to using Financial Products Markup Language (FpML) with Microsoft .NET Tools

Part 1 Introduction

An overview of how we can use Visual Studio to examine the FpML XSDs. Shows how to create a Visual Studio project containing the FpML schemas, and how to use that to navigate through them. Also shows how to validate the XML examples that are provided in the FpML download, both using Visual Studio and in C# code.

Part 2 How FpML Validates A Basic Trade Document

Looks at the structure of an FpML trade document in some detail. Explains how the FpML schemas fit together to validate such a document, and examines the use of base and extension types, and of substitution groups, in the XSDs.

Part 3 Generating .Net Classes From The FpML XSD Schema

Shows how to generate .Net classes in C# from the XML schema documents using Xsd.exe.  Explains how to load data into these classes from an FpML document using C# code, and to save the document back out again from the classes.

Part 4 Problems With Using Xsd.exe To Generate .Net Classes From The FpML XSD Schema

Goes through some of the problems with the C# code generated in part 3, and discusses how to fix them.

January 28, 2008

Problems with Using xsd.exe to Generate .NET Classes from the FpML XSD Schema (Introduction to using FpML with .NET Tools Part 4)

Introduction

Part 3 of this series of articles showed how we can generate .NET classes from the FpML XSD schema using xsd.exe. It showed how we can then use standard .NET serialization syntax to populate the classes from FpML documents, and vice versa.

However, as mentioned in part 3, xsd.exe generates buggy code when used with the FpML XSDs. This article will go through some of the problems with this code and describe how to fix them.

Corrected .NET Classes

Corrected generated classes, which are the end result of the work in this article, are available. This code has been corrected such that it appears to work in most circumstances. However we cannot be certain that it is free of all bugs.

The corrected code is based on FpML 4.2. The main aim of this article is to give you a starting point if you are trying to generate .NET code from other versions of the specification.

Please note also that this article was written using the xsd.exe supplied with Visual Studio 2005 Service Pack 1. Different versions of Visual Studio, or Visual Studio 2005 without the service pack applied, may give different results. See the discussion in the Comments for this article for more details.

Generating the Code

Part 3 of this series of articles described how to use xsd.exe to generate C# code from the FpML schemas. This gave us a file called fpml-main-4-2_xmldsig-core-schema.cs, which contains a Document class which should be the root class for our serialization. Unfortunately if we attempt to create an XmlSerializer object using this code we get exceptions.

Problem 1: Substitution Groups and Extension

The most fundamental problem with our generated code is that xsd.exe has got confused about the substitution groups and associated extension that were discussed in some detail in part 2 of this series of articles.

In particular it has not decorated the Product property of our Trade class correctly to allow it to deal with all the possible products correctly.

In part 2 we saw that the product element in the FpML is replaced with various individual product types using the substitutionGroup syntax. The product element is a sub-element of the trade element in the XML. As a result in our C# code we have a Trade class which contains a data member of type Product and a public property that gets and sets this. Excerpts from our generated code are as below:

[System.CodeDom.Compiler.GeneratedCodeAttribute("xsd", "2.0.50727.42")]
[System.SerializableAttribute()]
[System.Diagnostics.DebuggerStepThroughAttribute()]
[System.ComponentModel.DesignerCategoryAttribute("code")]
[System.Xml.Serialization.XmlTypeAttribute(Namespace="http://www.fpml.org/2005/FpML-4-2")]
public partial class Trade {
 
    private Product productField;
 
    public Product product {
        get {
            return this.productField;
        }
        set {
            this.productField = value;
        }
    }

The problem with this code is that, as we saw in part 2, we need to be able to put a Swap object (for example) in to our Product data member. We have a Swap class in our code, and this inherits Product, as we’d expect. So we can store a Swap object in our Product data member.

However, .NET doesn’t know how to serialize or deserialize a swap element in the XML into our product property without being told. In particular, if the deserializer finds an element called ‘swap’ where it’s expecting ‘product’ it doesn’t know that it should deserialize into a Swap object. As we saw in part 2, in ird_ex01_vanilla_swap.xml there is a ‘swap’ element where you might expect a ‘product’ element.

This is very easily fixed in this case. We simply decorate the Product property with an XmlElementAttribute that tells the serializer what to do:

    [System.Xml.Serialization.XmlElementAttribute("swap", typeof(Swap))]
    public Product product {
        get {
            return this.productField;
        }
        set {
            this.productField = value;
        }
    }

This does solve the problem for swap. However, there are multiple other product types that need the XmlElementAttribute added to Product to get the serialization to work (there are over twenty in fact). Also it turns out that this problem isn’t limited to Product. There are other elements that use the substitution group and extension property in this way and have the same problem

At first glance it appears there is no easy solution to this: we are going to have to go through all the possible places where there might be errors and correct the code by hand. However, there’s a much easier solution.

Problem 1 Solution

For simple examples of this problem xsd.exe will correctly generate the required XmlElementAttribute. It’s not immediately obvious why it fails to do so with the full FpML schemas.

It turns out that the reason xsd.exe gets this wrong is because the FpML schemas are spread across multiple files. If we create one big xsd file containing all of the FpML schema files and then run xsd.exe on this the problem goes away. I’ll leave you to draw your own conclusions about the quality of the xsd.exe code.

So to fix the problem we can cut and paste all of the FpML xsd files into one file, removing the include statements that become redundant. An example of this is available. We then use xsd.exe as described in part 3 to create our C# classes.

Problem 2: RoutingIds

If we fix problem 1 as above, we still get an error when we try to create our XmlSerializer object. The error message says ‘Connot convert type ‘RoutingId[]’ to ‘RoutingId’’.

This exception arises because xsd.exe has got an XmlArrayItemAttribute wrong in class RoutingIdsAndExplicitDetails. The generated code for the routingIds property in this class is as below:

    [System.Xml.Serialization.XmlArrayItemAttribute("routingId", typeof(RoutingId), IsNullable=false)]
    public RoutingId[][] routingIds {
        get {
            return this.routingIdsField;
        }
        set {
            this.routingIdsField = value;
        }
    }

The XmlArrayItemAttribute says that the property relates to an array of type RoutingId. However, the property (correctly) is of type RoutingId[][] which is an array of arrays of type RoutingId. So the XmlArrayItemAttribute should be changed as below:

    [System.Xml.Serialization.XmlArrayItemAttribute("routingId", typeof(RoutingId[]), IsNullable=false)]
    public RoutingId[][] routingIds {
        get {
            return this.routingIdsField;
        }
        set {
            this.routingIdsField = value;
        }
    }

Testing the Generated Classes

The two fixes above appear to make the generated classes work correctly. With these changes we can deserialize our ird_ex01_vanilla_swap.xml FpML document into the classes. We can then serialize it back into XML, and we end up with the same document we started. We saw this in one of our code examples from part 3. It’s not easy to test this code is working in all cases however. One approach is to take all the sample FpML files provided with the FpML download and attempt to deserialize them and reserialize them. The code is working if the final document is the same as the original one.

Testing Program

A testing program that does this is available. This contains the code we have seen before (in part 3) for serialization and deserialization.

It also contains a basic class for comparing the original and final FpML documents and outputting any differences. It does this simply by iterating through the lines in the two files and comparing them. This may not be the best way to do this as it is quite difficult with XML.

It is difficult because there can be valid differences between the files that are hard to deal with. For example we have a ‘difference’ where the original line is:

<hourMinuteTime>09:00:00</hourMinuteTime>

After deserialization and reserialization this becomes:

<hourMinuteTime>09:00:00.0000000+00:00</hourMinuteTime>

These are clearly the same thing but our file comparer has to be able to deal with it. It does so in a very basic way by hard-coding such differences to be ignored in method ‘CompareLines’.

Extent of Testing of Generated Code

Because of the difficulties described above the testing program has only been used to test that the generated code works with the interest rate derivatives and credit derivatives sample files.

Usefulness of Generated Classes

In the last two articles we have demonstrated that we can generate C# classes based on the FpML XSD specification, and with a little work can deserialize FpML documents into this object model, manipulate the objects, and serialize back into FpML.

However, my personal opinion is that we need to think carefully as to whether we want to use these classes. FpML has a very hierarchical structure, and as a result in the generated code we have very many classes interacting to represent even a simple trade. Our object model is not very easy to understand or use as a result.

For example, suppose we want to change the notional on the fixed leg of our interest rate swap (ird_ex01_vanilla_swap.xml) once we have it in the object model. Starting with the top-level Document object that we have deserialized, the syntax is as below:

        private void ChangeNotional(Document document)
        {
            DataDocument dataDocument = (DataDocument)document;
            Trade trade = (Trade)dataDocument.Items[0];
            Swap swap = (Swap)trade.Item;
            InterestRateStream interestRateStream0 = swap.swapStream[0];
            InterestRateStream interestRateStream1 = swap.swapStream[1];
            Calculation calculation1 = (Calculation)interestRateStream1.calculationPeriodAmount.Item;
            Notional notional = (Notional)calculation1.Item;
            notional.notionalStepSchedule.initialValue = 1000000;
        }

In fact, I don’t think this routine is complex enough, since it should really check which of interestRateStream0 and interestRateStream1 is the fixed leg.

A code example incorporating this code is available.

It’s hard to argue that this code is straightforward: for instance we have a number of ‘Item’ properties referenced that can be of various types. We have to know which type we want. In addition we have the issue that we are not entirely sure that the code generated by xsd.exe is free of bugs, even after the work we have done to patch it up.

As a result in a current project I am working on we have decided not to use the classes generated by xsd.exe, but instead to deserialize into flatter structures of our own design.

Conclusion

This article has shown that it is possible to fix the code generated by xsd.exe from the FpML schemas such that we can deserialize/serialize FpML documents into/out of the object model. We have also shown that it is difficult to test that this will work correctly in all cases, and that the resulting object model is not all that easy to use.

Licensing of FpML Specifications

The FpML Specifications of this document are subject to the FpML Public License (the “License”); you may not use the FpML Specifications except in compliance with the License. You may obtain a copy of the License at http://www.FpML.org.
The FpML Specifications distributed under the License are distributed on an “AS IS” basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See the License for the specific language governing rights and limitations under the License.
The Licensor of the FpML Specifications is the International Swaps and Derivatives Association, Inc. All Rights Reserved.

http://www.fpml.org/documents/license.html

November 24, 2007

Workspace Types (Introduction to the CAB/SCSF Part 17)

Filed under: .net, c#, CAB, Composite Application Block, Workspace — richnewman @ 10:25 pm

Introduction

Part 16 of this series of articles explained in general terms why Workspaces are useful. It also examined the methods that are available on a Workspace via the IWorkspace interface.

There are five Workspace types provided with the Composite Application Block framework: DeckWorkspace, ZoneWorkspace, TabWorkspace, MdiWorkspace and WindowWorkspace. This article looks in a little more detail at these various Workspace types and associated SmartPartInfo types, and gives some code examples.

The ToolBox

Some Workspaces can be added to the Visual Studio toolbox as shown below. To do this you (as usual) right-click and select ‘Choose Items…’. Then click the ‘Browse…’ button, and browse to your Microsoft.Practices.CompositeUI.WinForms.dll library. If you click ‘OK’ to this the Workspaces shown should be added:

smartparttoolbox.jpg

DeckWorkspace

The DeckWorkspace is the first of the Workspace types provided with the CAB that we shall look at.

This is ‘deck’ as in ‘deck of cards’. When we show a SmartPart in a DeckWorkspace it fills the area of the Workspace completely. If we show another SmartPart it replaces the original SmartPart in the view completely. However, the old SmartPart is still there in the deck, immediately behind the original SmartPart. If we add a third SmartPart it gets displayed at the front of the deck, but the other two SmartParts are still there in order. If you close the third SmartPart the second one will get displayed.

A code example of use of a DeckWorkspace is available. The shell form for this CAB application has a DeckWorkspace on it. The project also contains two SmartParts, with red and blue backgrounds. A series of buttons allow the user to call methods on the DeckWorkspace, providing either the red or the blue SmartPart as an argument. The methods available are the IWorkspace methods discussed in part 16: Show, Hide, Activate, Close.

deckworkspace.jpg

This behaves as described in part 16. If we try to Activate, Close or Hide a SmartPart that hasn’t been Shown we get an exception. However if we Show the red SmartPart, then Show the blue SmartPart, then Activate the red SmartPart the red SmartPart that was originally shown gets shown again.

As shown above, the form also has a ‘Details’ button. This shows the current ActiveSmartPart, and the Items and Workspaces collections on the RootWorkItem. As a result it will show you which SmartParts are loaded at any given time.

To use a DeckWorkspace you can simply drag one onto a form and start coding. There is no specific SmartPartInfo class for the DeckWorkspace .

The DeckWorkspace is a useful type if you are building an Outlook-style interface. It can act as the main display area in the application, and gives us the behaviour we desire (which was outlined in part 16).

ZoneWorkspace

The ZoneWorkspace allows the user to define ‘zones’ or areas within the Workspace where SmartParts can be shown. As mentioned briefly in part 16 a SmartPartInfo object can be used to define which zone a SmartPart will be shown in .

The easiest way to set up a ZoneWorkspace is firstly to drag one onto your form or user control from the ToolBox. Then drag ordinary Windows Forms panels onto the ZoneWorkspace and position them to define your zones. If you look in their properties you will see that these panels have a ZoneName property where you should give your zones sensible names.

To show a SmartPart in a specific zone you need to tell it the zone via a SmartPartInfo object. The easiest way to do this is to drag a ZoneSmartPartInfo item from the Toolbox shown above onto your form or user control. You can make your ZoneSmartPartInfo reference a specific zone by setting ITS ZoneName property to the ZoneName of the zone. You can then show SmartParts in the zone by calling the Show method and passing the ZoneSmartPartInfo object as the second parameter.

A code example of this is available. Once again this has two SmartParts, red and blue. It also has two zones, left and right, as shown. Further it has two visual ZoneSmartPartInfo objects as discussed above, one of which references the left zone and one of which references the right zone. Which one of these is used when you click on the buttons is controlled by the radio buttons shown below.

The buttons are as above, apart from the Apply buttons. The Apply buttons call ApplySmartPartInfo on the ZoneWorkspace with the appropriate ZoneSmartPartInfo object depending on which of the radio buttons is selected. They can thus be used to change which zone a SmartPart is displayed in: if the red SmartPart is shown in the left zone (as shown below) we can check the ‘Right zone’ radio button and hit ‘Apply’ for the red SmartPart, which will move it to the right zone.

zoneworkspace.jpg

One thing to note about the ZoneWorkspace is that calling Show with a SmartPart does not necessarily bring that SmartPart to the front. The Show method adds the SmartPart to the appropriate collections, gives the SmartPart the focus and make the SmartPart into the ActiveSmartPart. However in spite of this the SmartPart can still be hiding behind another SmartPart.

For this reason DeckWorkspaces are often put into zones of ZoneWorkspaces, since a SmartPart will be brought to the front when Show is called on a DeckWorkspace.

TabWorkspace

TabWorkspace has the standard Windows Forms TabControl as a base class, and acts like a CAB version of that control. Every SmartPart that you add to the TabWorkspace gets shown in its own tab.

There’s a TabSmartPartInfo class we can use to set details on our SmartParts and tabs. This has a Title property which can be used to set the title of the TabPage the SmartPart is being displayed in. It also has a Position property. This can be set to TabPosition.Beginning or TabPosition.End, and affects where the new tab appears in relation to existing tabs.

There’s a code example that shows this, with the same buttons and red and blue SmartParts as in the previous examples. This has no tabs showing at start up. When we click the Show button for a SmartPart it calls the Show method for the TabWorkspace, passing the appropriate SmartPart as a parameter.

The Show method creates a new tab with the SmartPart displayed on it.

tabworkspace.jpg

In addition the example has visual TabSmartPartInfo components that are passed to the Show methods, one for the red SmartPart and one for the blue. These set the titles on the tab pages, and control where they are added.

This example also has ‘Apply’ buttons that let us change the titles of the tab pages from a value entered in a TextBox. The buttons do this by setting the Title property on the appropriate TabSmartPartInfo and then calling the ApplySmartPartInfo method.

The SmartPartInfo you give to the Show method implements ISmartPartInfo and has no additional members.

MdiWorkspace

The MdiWorkspace allows each SmartPart to be displayed in a separate child window inside a parent Form. This allows us to build MDI applications using the Composite Application Block.

One thing to note about the MdiWorkspace is that, as with other Workspaces, our child SmartParts have to be User Controls. We can’t display child Forms inside an MdiWorkspace with the code as it stands. This is in spite of the fact that our children look and behave like Forms: they have title bars and maximize and minimize buttons, for example.

MdiWorkspaces can use the WindowSmartPartInfo type to give additional information about SmartParts that are being displayed. As usual, this has a Title property. It also has properties to determine whether our MDI child is modal, whether it displays minimize, maximize and/or control boxes, and its location.

Once again a code example is available, and this behaves in the same way as the examples above. This includes allowing us to change the titles by entering some text in a TextBox and clicking ‘Apply’, which calls ApplySmartPartInfo as for the TabWorkspace above.

mdiworkspace.jpg

WindowWorkspace

A WindowWorkspace lets us display our SmartParts in floating windows, each one in a separate window. Once again it uses the WindowSmartPartInfo type to give additional information about SmartParts that are being displayed.

A code example is available. This is extremely similar to the MdiWorkspace example above, except that obviously it deals with a WindowWorkspace and not an MdiWorkspace.

windowworkspace.jpg

Conclusion

In this article we have reviewed the various types of Workspace that are available in the CAB framework, and have given some code examples.

So far in this series of articles we have only looked at the Composite Application Block. Part 18 of the series will rectify this by taking a first look at the Smart Client Software Factory.

Older Posts »

The Shocking Blue Green Theme. Create a free website or blog at WordPress.com.

Follow

Get every new post delivered to your Inbox.

Join 82 other followers