Rich Newman

August 24, 2013

Connecting to Tibco EMS from a C# Client

Filed under: .net, c#, EMS, Spring.Net, Tibco — Tags: , , , , — richnewman @ 1:04 pm

Introduction

This is a slightly esoteric post, but hopefully it will be of use somewhere.  Below is some reasonably simple code that shows the basic use cases for connecting to a Tibco EMS Server topic from a C# client using Tibco’s TIBCO.EMS.dll.

It actually points at a locally installed instance of EMS.  That is, the server code is running on the workstation.  If you have the Windows EMS installation you can just install it in the default path and the instructions below should work.  Alternatively it’s relatively simple to change the server name from ‘localhost’ and put a username and password in the code to connect to a secure EMS Server.

How to Use the Code

  • Get hold of the Windows Tibco EMS install, and install it on your workstation on the default path.  Note that Tibco do NOT provide this for free: you need to have paid them for a licence.  If you work for a large organization you will probably find you already have a licence.
  • Create a C# console application called ‘TestEMS’, and reference TIBCO.EMS.dll (the C# EMS assembly) from the install.
  • Paste in the code below.
  • Run the code.  You should see a console window showing that a message has been sent by a publisher and received by a subscriber.

If you want to connect to an existing EMS server in your organization all you need is the TIBCO.EMS.dll assembly, not a full client install. You need to change the appropriate parts of the code below to point to the server: change ‘localhost’ to the server name, and add a username and password if they are needed.  Obviously you don’t need to start the server locally if you do this.

What the Sample Does

As you can see, the code starts the local EMS server, then creates a topic publisher and a subscriber to the same topic.  It then sends a ‘Hello World’ message using the publisher.  The subscriber’s message handler prints out the message to the console when it receives it.

The sample code is slightly more sophisticated than the most basic use case.  It also shows how to set a property on a message, and then to how filter the subscription using a message selector based on the property.  The message is sent with an additional property of ‘Owner’ with value ‘Rich Newman’.  The subscriber is only listening for messages with an Owner property that contains the string ‘Rich Newman’.  You can see this if you change the name of the owner in the message that’s sent: the message listener will not get the message.

Code

using System;
using System.Diagnostics;
using System.Threading;
using TIBCO.EMS;

namespace TestEMS
{
    class Program
    {
        static void Main(string[] args)
        {
            Console.WriteLine("Test started");
            new Program().Run();
            Console.ReadLine();
        }

        private void Run()
        {
            StartEMSServer();
            CreateEMSServerTopicPublisher();
            CreateClientTopicSubscriber("Owner LIKE '%Rich Newman%'"); // Pass "" for no message selector
            EMSServerPublishThisMessage("Hello World""Owner""Rich Newman");
        }

        #region EMS Server
        private const string tibcoEMSPath = @"C:\tibco\ems\5.0\bin\";
        private readonly string tibcoEMSExecutable = tibcoEMSPath + "tibemsd.exe";
        private Process tibcoEMSProcess;
        public void StartEMSServer()
        {
            tibcoEMSProcess = new Process();
            ProcessStartInfo processStartInfo = new ProcessStartInfo(tibcoEMSExecutable);
            tibcoEMSProcess.StartInfo = processStartInfo;
            processStartInfo.WorkingDirectory = tibcoEMSPath;
            bool started = tibcoEMSProcess.Start();
            Thread.Sleep(500);
        }

        TopicConnection publisherConnection;
        TopicSession publisherSession;
        TopicPublisher emsServerPublisher;
        private void CreateEMSServerTopicPublisher()
        {
            TopicConnectionFactory factory = new TIBCO.EMS.TopicConnectionFactory("localhost");
            publisherConnection = factory.CreateTopicConnection(""""); // Username, password
            publisherSession = publisherConnection.CreateTopicSession(falseSession.AUTO_ACKNOWLEDGE);
            Topic generalTopic = publisherSession.CreateTopic("GeneralTopic");
            emsServerPublisher = publisherSession.CreatePublisher(generalTopic);

            publisherConnection.Start();
        }

        internal void EMSServerPublishThisMessage(string messagestring propertyNamestring propertyValue)
        {
            TextMessage textMessage = publisherSession.CreateTextMessage();
            textMessage.Text = message;
            textMessage.SetStringProperty(propertyNamepropertyValue);
            emsServerPublisher.Publish(textMessage);
            Console.WriteLine("EMS Publisher published message: " + message);
        }

        #endregion

        #region EMS Client
        TopicConnection subscriberConnection;
        TopicSession subscriberSession;
        private void CreateClientTopicSubscriber(string messageSelector)
        {
            TopicConnectionFactory factory = new TIBCO.EMS.TopicConnectionFactory("localhost");
            subscriberConnection = factory.CreateTopicConnection("""");  // Username, password
            subscriberConnection.Start();
            subscriberSession = subscriberConnection.CreateTopicSession(falseSession.AUTO_ACKNOWLEDGE);
            Topic clientTopic = subscriberSession.CreateTopic("GeneralTopic");
            TopicSubscriber clientTopicSubscriber = subscriberSession.CreateSubscriber(clientTopicmessageSelectortrue);
            clientTopicSubscriber.MessageHandler += new EMSMessageHandler(test_MessageHandler);
        }

        void test_MessageHandler(object senderEMSMessageEventArgs args)
        {
            Console.WriteLine("EMS Client received message: " + args.Message.ToString());
        }

        #endregion
    }
}

December 3, 2012

Asynchronous Programming in .Net: Async and Await for Beginners

Introduction

There are several ways of doing asynchronous programming in .Net.  Visual Studio 2012 introduces a new approach using the ‘await’ and ‘async’ keywords.  These tell the compiler to construct task continuations in quite an unusual way.

I found them quite difficult to understand using the Microsoft documentation, which annoyingly keeps saying how easy they are.

This series of articles is intended to give a quick recap of some previous approaches to asynchronous programming to give us some context, and then to give a quick and hopefully easy introduction to the new keywords

Example

By far the easiest way to get to grips with the new keywords is by seeing an example.  For this initially I am going to use a very basic example: you click a button on a screen, it runs a long-running method, and displays the results of the method on the screen.

Since this article is about asynchronous programming we will want the long-running method to run asynchronously on a background thread.  This means we need to marshal the results back on to the user interface thread to display them.

In the real world the method could be running a report, or calling a web service.  Here we will just use the method below, which sleeps to simulate the long-running process:

        private string LongRunningMethod(string message)
        {
            Thread.Sleep(2000);
            return "Hello " + message;
        }

The method will be called asynchronously from a button click method, with the results assigned to the content of a label.

Coding the Example with Previous Asynchronous C# Approaches

There are at least five standard ways of coding the example above in .Net currently.  This has got so confusing that Microsoft have started giving the various patterns acronyms, such as the ‘EAP‘ and the ‘APM‘.   I’m not going to talk about those as they are effectively deprecated.  However it’s worth having a quick look at how to do our example using some of the other approaches.

Coding the Example by Starting our Own Thread

This simple example is fairly easy to code by just explicitly starting a new thread and then using Invoke or BeginInvoke to get the results back onto the UI thread.  This should be familiar to you:

        private void Button_Click_1(object sender, RoutedEventArgs e)
        {
            new Thread(() => { 
                string result = LongRunningMethod("World");
                Dispatcher.BeginInvoke((Action)(() => Label1.Content = result)); 
            }).Start();
            Label1.Content = "Working...";
        }

We start a new thread and hand it the code we want to run.  This calls the long-running method and then uses Dispatcher.BeginInvoke to call back onto the user interface thread with the result and update our label.

Note that immediately after we start the new thread we set the content of our label to ‘Working…’.  This is to show that the button click method continues immediately on the user interface thread after the new thread is started.

The result is that when we click the button our label says ‘Working…’ almost immediately, and then shows ‘Hello World’ when the long-running method returns.  The user interface will remain responsive whilst the long-running thread is running.

Coding the Example Using the Task Parallel Library (TPL)

More instructive is to revisit how we would do this with tasks using the Task Parallel Library.  We would typically use a task continuation as below.

        private void Button_Click_2(object sender, RoutedEventArgs e)
        {
            Task.Run<string>(() => LongRunningMethod("World"))
                .ContinueWith(ant => Label2.Content = ant.Result, 
                              TaskScheduler.FromCurrentSynchronizationContext());
            Label2.Content = "Working...";
        }

Here we’ve started a task on a background thread using Task.Run.  This is a new construct in .Net 4.5.  However, it is nothing more complicated than Task.Factory.StartNew with preset parameters.  The parameters are the ones you usually want to use.  In particular Task.Run uses the default Task Scheduler and so avoids one of the hidden problems with StartNew.

The task calls the long-running method, and does so on a threadpool thread.  When it is done a continuation runs using ContinueWith.  We want this to run on the user interface thread so it can update our label.  So we specify that it should use the task scheduler in the current synchronization context, which is the user interface thread when the task is set up.

Again we update the label after the task call to show that it returns immediately.  If we run this we’ll see a ‘Working…’ message and then ‘Hello World’ when the long-running method returns.

Coding the Example Using Async and Await

Code

Below is the full code for the async/await implementation of the example above.  We will go through this in detail.

       private void Button_Click_3(object sender, RoutedEventArgs e)
        {
            CallLongRunningMethod();
            Label3.Content = "Working...";        
        }

        private async void CallLongRunningMethod()
        {
            string result = await LongRunningMethodAsync("World");
            Label3.Content = result;
        }

        private Task<string> LongRunningMethodAsync(string message)
        {
            return Task.Run<string>(() => LongRunningMethod(message));
        }

        private string LongRunningMethod(string message)
        {
            Thread.Sleep(2000);
            return "Hello " + message;
        }

Asynchronous Methods

The first thing to realize about the async and await keywords is that by themselves they never start a thread.  They are a way of controlling continuations, not a way of starting asynchronous code.

As a result the usual pattern is to create an asynchronous method that can be used with async/await, or to use an asynchronous method that is already in the framework.  For these purposes a number of new asynchronous methods have been added to the framework.

To be useful to async/await the asynchronous method has to return a task.  The asynchronous method has to start the task it returns as well, something that maybe isn’t so obvious.

So in our example we need to make our synchronous long-running method into an asynchronous method.  The method will start a task to run the long-running method and return it.  The usual approach is to wrap the method in a new method.   It is usual to give the method the same name but append ‘Async’.  Below is the code to do this for the method in our example:

        private Task<string> LongRunningMethodAsync(string message)
        {
            return Task.Run<string>(() => LongRunningMethod(message));
        }

Note that we could use this method directly in our example without async/await.  We could call it and use ‘ContinueWith’ on the return value to effect our continuation in exactly the same way as in the Task Parallel Library code above.  This is true of the new async methods in the framework as well.

Async/Await and Method Scope

Async and await are a smart way of controlling continuations through method scope.  They are used as a pair in a method as shown below:

        private async void CallLongRunningMethod()
        {
            string result = await LongRunningMethodAsync("World");
            Label3.Content = result;
        }

Here async is simply used to tell the compiler that this is an asynchronous method that will have an await in it.  It’s the await itself that’s interesting.

The first line in the method calls LongRunningMethodAsync, clearly.  Remember that LongRunningMethodAsync is returning a long-running task that is running on another thread.  LongRunningMethodAsync starts the task and then returns reasonably quickly.

The await keyword ensures that the remainder of the method does not execute until the long-running task is complete.  It sets up a continuation for the remainder of the method. Once the long-running method is complete the label content will update: note that this happens on the same thread that CallLongRunningMethod is already running on, in this case the user interface thread.

However, the await keyword does not block the thread completely.  Instead control is returned to the calling method on the same thread.  That is, the method that called CallLongRunningMethod will execute at the point after the call was made.

The code that calls LongRunningMethod is below:

        private void Button_Click_3(object sender, RoutedEventArgs e)
        {
            CallLongRunningMethod();
            Label3.Content = "Working...";        
        }

So the end result of this is exactly the same as before.  When the button is clicked the label has content ‘Working…’ almost immediately, and then shows ‘Hello World’ when the long-running task completes.

Return Type

One other thing to note is that LongRunningMethodAsync returns a Task<string>, that is, a Task that returns a string.  However the line below assigns the result of the task to the string variable called ‘result’, not the task itself.

string result = await LongRunningMethodAsync("World");

The await keyword ‘unwraps’ the task.  We could have attempted to access the Result property of the task (string result = LongRunningMethodAsync(“World”).Result.  This would have worked but would have simply blocked the user interface thread until the method completed, which is not what we’re trying to do.

I’ll discuss this further below.

Recap

To recap, the button click calls CallLongRunningMethod, which in turn calls LongRunningMethodAsync, which sets up and runs our long-running task.  When the task is set up (not when it’s completed) control returns to CallLongRunningMethod, where the await keyword passes control back to the button click method.

So almost immediately the label content will be set to “Working…”, and the button click method will exit, leaving the user interface responsive.

When the task is complete the remainder of CallLongRunningMethod executes as a continuation on the user interface thread, and sets the label to “Hello World”.

Async and Await are a Pair

Async and await are always a pair: you can’t use await in a method unless the method is marked async, and if you mark a method async without await in it then you get a compiler warning.  You can of course have multiple awaits in one method as long as it is marked async.

Aside: Using Anonymous Methods with Async/Await

If you compare the code for the Task Parallel Library (TPL) example with the async/await example you’ll see that we’ve had to introduce two new methods for async/await: for this simple example the TPL code is shorter and arguably easier to understand.  However, it is possible to shorten the async/await code using anonymous methods, as below. This shows how we can use anonymous method syntax with async/await, although I think this code is borderline incomprehensible:

        private void Button_Click_4(object sender, RoutedEventArgs e)
        {
            new Action(async () =>
            {
                string result = await Task.Run<string>(() => LongRunningMethod("World"));
                Label4.Content = result;
            }).Invoke();
            Label4.Content = "Working...";
        }

Using the Call Stack to Control Continuations

Overview of Return Values from Methods Marked as Async

There’s one other fundamental aspect of async/await that we have not yet looked at.  In the example above our method marked with the async keyword did not return anything.  However, we can make all our async methods return values wrapped in a task, which means they in turn can be awaited on further up the call stack.  In general this is considered good practice: it means we can control the flow of our continuations more easily.

The compiler makes it easy for us to return a value wrapped in a task from an async method.  In a method marked async the ‘return’ statement works differently from usual.  The compiler doesn’t simply return the value passed with the statement, but instead wraps it in a task and returns that instead.

Example of Return Values from Methods Marked as Async

Again this is easiest to see with our example.  Our method marked as async was CallLongRunningMethod, and this can be altered to return the string result to the calling method as below:

        private async Task<string> CallLongRunningMethodReturn()
        {
            string result = await LongRunningMethodAsync("World");
            return result;
        }

We are returning a string (‘return result’), but the method signature shows the return type as Task<string>.  Personally I think this is a little confusing, but as discussed it means the calling method can await on this method.  Now we can change the calling method as below:

        private async void Button_Click_5(object sender, RoutedEventArgs e)
        {
            Label5.Content = "Working...";
            string result = await CallLongRunningMethodReturn();
            Label5.Content = result;
        }

We can await the method lower down the call stack because it now returns a task we can await on.  What this means in practice is that the code sets up the task and sets it running and then we can await the results from the task when it is complete anywhere in the call stack.  This gives us a lot of flexibility as methods at various points in the stack can carry on executing until they need the results of the call.

As discussed above when we await on a method returning type Task<string> we can just assign the result to a string as shown.  This is clearly related to the ability to just return a string from the method: these are syntactic conveniences to avoid the programmer having to deal directly with the tasks in async/await.

Note that we have to mark our method as ‘async’ in the method signature (‘private async void Button_Click_5′) because it now has an await in it, and they always go together.

What the Code Does

The code above has exactly the same result as the other examples: the label shows ‘Working…’ until the long-running method returns when it shows ‘Hello World’.  When the button is clicked it sets up the task to run the long-running method and then awaits its completion both in CallLongRunningMethodReturn and Button_Click_5.  There is one slight difference in that the click event is awaiting: previously it exited.  However, if you run the examples you’ll see that the user interface remains responsive whilst the task is running.

What’s The Point?

If you’ve followed all the examples so far you may be wondering what the point is of the new keywords.  For this simple example the Task Parallel Library syntax is shorter, cleaner and probably easier to understand than the async/await syntax.  At first sight async/await are a little confusing.

The answer is that for basic examples async/await don’t seem to me to be adding a lot of value, but as soon as you try to do more complex continuations they come into their own.  For example it’s possible to set up multiple tasks in a loop and write very simple code to deal with what happens when they complete, something that is tricky with tasks.  I suggest you look at the examples in the Microsoft documentation which do show the power of the new keywords.

Code

The full code for these examples is available to download.

Conclusion

This article has only covered the basics of the async/await keywords, although I think it’s addressed all the things that were confusing me when trying to learn about them from the Microsoft documentation.  There are some obvious things it hasn’t covered such as cancelling tasks, exception handling, unwrapping tasks (and why you might need to do that) and how to deal with the reentrancy problems that arise.  All of these are covered reasonably well in the documentation.

Personally I think async and await are far from intuitive: the compiler is performing some magic of a kind we don’t usually see in C#.  The result is that we are yielding control in the middle of a method to the calling method until some other task is complete.  Of course we can do similar things with regular task continuations, but the syntax makes regular continuations look slightly less magical.

However, async/await are a powerful way of controlling multithreaded code once you understand what they are doing.  They can make fairly complex threading look simple.

November 21, 2012

Why Starting a New Task in the Task Parallel Library (TPL) Doesn’t Always Start a New Thread

Filed under: .net, c#, Task Parallel Library, threading, TPL — Tags: , , , , — richnewman @ 4:56 am

The Task Parallel Library in .Net is a wonderful resource.  However, one thing that is a little confusing is that it is actually possible to start a task that runs on the same thread as the current code.  That is, we can do Task.StartNew and it still runs on the same thread.  This isn’t the behaviour we expect.

This can happen particularly when we are writing .Net user interface applications where we marshal data on to the UI thread.

Consider the sample code below, which runs in the button click event of a simple WPF application.  This is a not unusual pattern for a user interface application: we start a Task on a new thread and then perform a continuation on the user interface thread using the current synchronization context, which is the user interface thread synchronization context (TaskScheduler.FromCurrentSynchronizationContext()).  We would typically do this so that we could write the results of the Task into the user interface.  There are several articles on the internet that describe this.

private void button1_Click(object sender, RoutedEventArgs e)
{
    Debug.WriteLine("UI thread: " + Thread.CurrentThread.ManagedThreadId);
    Task.Factory.StartNew(() => 
        Debug.WriteLine("Task 1 thread: " + Thread.CurrentThread.ManagedThreadId))
    .ContinueWith(ant =>
    {
        Debug.WriteLine("Continuation on UI thread: " + Thread.CurrentThread.ManagedThreadId);
        Task.Factory.StartNew(() 
            => Debug.WriteLine("Task 2 thread is UI thread: " + Thread.CurrentThread.ManagedThreadId));
    }, TaskScheduler.FromCurrentSynchronizationContext());
}

However, in the code above we then start a new Task from the continuation (task 2).  This runs on the user interface thread by default.  This is almost certainly not what we are trying to do.  We probably want to run something on a background thread, which is what usually happens when we start a Task.  As a result it’s slightly hazardous behaviour; we can easily end up blocking the user interface thread because we think the code is running on a background thread.

The output from a couple of button clicks using this code is as below:

UI thread: 8
Task 1 thread: 9
Continuation on UI thread: 8
Task 2 thread is UI thread: 8
UI thread: 8
Task 1 thread: 10
Continuation on UI thread: 8
Task 2 thread is UI thread: 8

The reason for this behaviour is that the user interface thread synchronization context is still in force when we start our second task.

Fortunately, in this case at least there is a simple solution, even if it isn’t so obvious.  We can tell the new Task to run explicitly using the default synchronization context instead of the user interface synchronization context when we start it.  The simplest syntax for this changes the code as below.  Notice it passes TaskScheduler.Default as an argument to Task.Start.  The terminology here for a user interface application is a little confusing.  The ‘current synchronization context’ is accessed using TaskScheduler.FromCurrentSynchronizationContext, which we typically do from the user interface thread.  Then the ‘current’ context that results in delegates being queued back to the user interface thread.  The ‘default synchronization context’ is the context that results in delegates being queued to the thread pool.

private void button1_Click(object sender, RoutedEventArgs e)
{
    Debug.WriteLine("UI thread: " + Thread.CurrentThread.ManagedThreadId);
    Task.Factory.StartNew(() => 
        Debug.WriteLine("Task 1 thread: " + Thread.CurrentThread.ManagedThreadId))
    .ContinueWith(ant =>
    {
        Debug.WriteLine("Continuation on UI thread: " + Thread.CurrentThread.ManagedThreadId);
        Task task2 = new Task(() 
            => Debug.WriteLine("Task 2 thread: " + Thread.CurrentThread.ManagedThreadId));
        task2.Start(TaskScheduler.Default);
    }, TaskScheduler.FromCurrentSynchronizationContext());
}

The output from this is now as we would expect:

UI thread: 9
Task 1 thread: 10
Continuation on UI thread: 9
Task 2 thread: 11
UI thread: 9
Task 1 thread: 11
Continuation on UI thread: 9
Task 2 thread: 10

For more detailed discussion on this see:

http://stackoverflow.com/questions/6800705/why-is-taskscheduler-current-the-default-taskscheduler

http://stackoverflow.com/questions/12245935/is-task-factory-startnew-guaranteed-to-use-another-thread-than-the-calling-thr

November 7, 2012

CPDOs for Beginners

Introduction

CPDOs (constant proportion debt obligations) are in the news currently.  In spite of a judge describing a CPDO issued by ABN Amro as ‘grotesquely complicated’ the basic concept behind the instrument is pretty straightforward.  This article describes the strategy behind a CPDO at a high level.

Overview of a CPDO

A CPDO is a financial instrument issued by a bank that a sophisticated investor can invest in.  To the investor CPDOs behave like bonds that pay a higher rate of interest than similar instruments issued by the bank.  So the investor gives the bank some money for a certain period, usually several years.  The bank pays a high rate of interest throughout the period and, in theory, gives the money back at the end.

To achieve this higher rate of interest the bank effectively speculates with the money they are given.  They speculate in quite a distinctive way, however.

The Basic Strategy – Sell CDS Protection to Generate Income

The first thing the bank does is to invest the money the investor has given them in something that will pay them the normal rate of interest.  If that was all they did obviously they would not be able to pay the high rate of interest to the investor.

So the bank needs a way of generating extra money.  To do this they use CDS.  I have written an earlier article on the exact mechanics of CDS, but you don’t need to know the details to understand CPDO.  What you do need to know is that CDS are like insurance contracts: if you sell protection on a CDS you receive periodic payments in return for a small chance that you will have to pay back a much larger sum.  You pay back the larger sum if the a specific company gets into financial difficulty: this is known as a ‘credit event’ or ‘default’.

Note that the banks actually use CDS indexes in CPDOs, which are CDS on a basket of companies rather than one company.  However the idea is the same.

The basic strategy is that at the start of the period the bank sells enough CDS protection to easily generate enough money to pay the high rate of interest to the investor for the period of the investment, assuming there is no need to pay anything back because of defaults.  The bank works out how much protection to sell using a set of rules that are defined in the CPDO documentation.

Periodic Rebalancing

Once they have done that they leave everything alone for a while.  After a set period of time they look at how the CPDO is doing.  At this point they may change the amount of CDS protection they are selling.  This is known as ‘rebalancing’.

The CPDO may be doing well: none of the CDS may have had credit events, for example.  In this case the bank might reduce the amount of CDS protection they are selling.

Conversely some of the CDS in the CPDO have suffered defaults.  It may be that the amount of CDS that have been sold will no longer be enough to generate the money needed to repay the investor.  In this case the bank might increase the amount of CDS protection they are selling.

The bank will do this rebalancing periodically throughout the life of the CPDO.  It is usually done every six months to coincide with the dates that the CDS indexes are updated.

Rules

Note that all of this is done according to the set of rules that are defined in advance: it’s not a judgment call.  The rules can look fairly complex.  However all they really do is describe a way of calculating ‘leverage’, which is the value of the money to be received from the risky CDS contracts outstanding versus the amount of cash they need to generate, maybe multiplied by a fixed factor. The bank will try to keep the leverage constant at every rebalancing: if the amount of cash to be generated has increased (because there have been defaults) then the value of the money from the CDS contracts needs to be increased, so we enter into more contracts.

Possible Results of the Strategy

The aim, of course, is to generate plenty of money through this speculation in CDS.  The ideal is to generate so much money that it can all be invested in relatively riskless instruments and still pay for the cashflows on the CPDO.  At that point you don’t need to speculate any more: you can ‘cash in’ the CPDO.

An alternative is that you lose so much money from paying out on the CDS that you get to the point where it’s clear you won’t be able to pay the interest rate and principal on the CPDO.  The rules governing the CPDO usually define this ‘cash out’ point: when it is reached the structure will be unwound and the bank will pay what money is left back to the investor.

A final alternative is that the ‘cash in’ or ‘cash out’ points are never reached, but and the CPDO just expires but without sufficient cash to repay the investor in full.

Martingale

Those of you who are familiar with gambling will recognize this as a simple martingale strategy.  If you lose you increase the amount you are betting.  This is in the hope that you will win next time and get all your money back.  Of course if you lose again you can increase the amount you are betting again, but you run the risk of losing substantial amounts of money.

Effects of Market Moves

There are a few other points to notice about this product:

  • The best case for the investor is to get the high rate of interest on their investment and all of their money back.  The CPDO is not speculative in the sense that the investor can make better returns if the market moves favourably.
  • The investor is selling protection in the credit markets.  The CPDO will lose money when companies get into financial difficulty.  If many get into financial difficulty simultaneously the CPDO may well suffer large losses.  This means the investor will not get much of their money back.  Of course, that’s exactly what happened with many of these instruments.
  • It’s possible to structure a CPDO such that in normal market conditions there is a good chance that the CPDO will cash in.  If the bank is simply trying to get a little bit of extra interest by speculating in CDS over a long period of time then they might usually win their bet.
  • I’ll leave it to the reader to decide whether this is really a product that anyone should have been ‘investing’ in, AAA-rated or not.

Conclusion

This article has given a very high-level overview of CPDOs.  It has inevitably glossed over some details, but hopefully it explains the basic idea.

CPDOs were invented in the credit boom, and when the crash came they lost many people a lot of money.  I doubt we shall see them again any time soon except in lawsuits.

September 8, 2012

Beginner’s Guide to Techniques for Refreshing Web Pages: Ajax, Comet, HTML5

Filed under: ajax, beginners guide, html, introduction, tutorial, web — Tags: , , , , — richnewman @ 6:07 pm

Introduction

This article briefly discusses the technologies used in modern browsers to display web pages, and goes into a little more detail about the user experience on those web pages, in particular how we can get part of a web page to refresh automatically when data changes on a web server.

Browsers and HTML

I’m sure anyone who’s reading this page is aware that the web is based on a request and response process that returns web pages of data.  If I click on a link my browser makes a request for the web page specified in the link, the request gets routed to the appropriate server, and the server responds with a page of HTML which my browser displays.

HTML (hypertext markup language), of course, is a simple markup language that tells a browser where to put text and images on a web page using tags (e.g. <header>).  The request format is a text URL (uniform resource locator) of the kind you see all the time in your browser’s navigation bar.  Furthermore, the returned text can contain additional links that the browser will show underlined and that I can click on.

Anyone who uses the internet understands this, but the success of the web is at least in part due to the simplicity of that model.  The HTML is just text with a few special tags in angle brackets, and all a browser has to do is know how to send a request, handle the response, and then draw a screen using the HTML that’s returned.  Similarly all a web server has to do is know how to receive a request, get the right HTML text to send back, and send it.  Often web servers will simply store the text in text files on their hard drive, and just load and send the right one in response to a request depending on the text of the request.

At root it’s unbelievably simple; just look what it’s turned into.

Other Technologies Used In Web Browsers

Of course modern browsers aren’t as simple as described above and there are a number of other technologies that they understand and developers can use.

Firstly, developers want to write code, so there’s also a programming language embedded into every modern browser.  This is Javascript.

Javascript allows programmers to write little bits of code that can run when events happen in the browser.  The Javascript can manipulate what’s displayed in the browser programmatically, or can perform other actions.

For the Javascript to change what’s displayed it needs to manipulate the HTML.  Obviously this can be done by simply changing the text.  However, there’s a programmatic representation of a web page that Javascript can use to manipulate elements within it.  This format is the Document Object Model or DOM.

Another baseline technology for what gets displayed to the client is Cascading Style Sheets (CSS).  These allow a common look and feel to be applied to a group of web pages without the need for detailed coding in each page.

Drawbacks of the Basic HTML Request/Response Page-Based Model

HTML + Javascript + CSS allows us to create quite sophisticated web pages.  However, there’s one big drawback with the model as described above: to display new data we have to click on a link to request a whole new page and wait whilst it loads.

For a more sophisticated user experience there are a few things we might like to have:

  1. The ability to refresh part of a web page without reloading the entire page.  Initially this could be initiated by the user clicking a button, but we want just the relevant data to update, not the entire page.
  2. The ability to do this refresh whilst allowing the user to continue to interact with the rest of the page.  That is, the request shouldn’t block the user, it should be asynchronous.
  3. The ability to update the page when data changes on the server without the user having to refresh in any way.

1.  Refreshing Part of a Web Page

The first problem that developers tried to solve was updating part of a web page in place without reloading the entire page.  There are several ways of doing this.

IFrames

However, there are some simple approaches that predate Ajax.  One is to use IFrames.  These are HTML elements within a page that can issue their own request/responses to a web site and render the results independently of the rest of the page.  They have a SRC property that can be set to a URL.  If you set the src to a different or the same URL (say on a button click) the new data will appear without a full page reload.

Many developers don’t like IFrames.  Search engines can get confused by them.  They may show scrollbars if your content doesn’t fit correctly.  Worse if your user has scrolled to the bottom of a page and then you load a new shorter page in the same frame they may be off the bottom of it.  Because of restrictions in HTML IFrames can usually only issue requests to the same site as the main site of the page they are on.  All of this means people have looked for better solutions.

Script Injection

Another approach to refreshing part of a web page is client-side script injection.  This takes advantage of the fact that Javascript code in a web page can be retrieved from a server using a URL via a src tag.

The basic approach is the same as for IFrames: we can set or reset the src of the script code, and the browser will retrieve the script from the URL and execute it.  If we send back valid Javascript that updates part of our web page, or calls a function that does, then we don’t have to refresh the entire page.

One advantage of this approach is that script tags can issue requests to any URL, not just the same site as the page they are on.  One disadvantage of this approach is that it can lead to security vulnerabilities in the code.

JSONP

JSONP is just a way of using client-side script injection across domains to get data from a different website: we request the script from the different server and it returns it as the parameters of a Javascript function which immediately executes and uses the payload.

2.  Refreshing Part of a Web Page Asynchronously

Ajax (Asynchronous Javascript and XML) is probably the primary technology for this.  Ajax is actually a label applied to a way of using many of the technologies described above to allow web pages to be displayed and then be updated asynchronously without reloading the entire page.

The main distinguishing feature of Ajax is that it uses a relatively new request response mechanism.  This is called XMLHttpRequest.  When a browser makes a request using XMLHttpRequest it provides the name of a Javascript function that will be called back by the server.  This function will have access to the data sent back from the server.

The original call to the server will not block and will not reload the page.  The user can carry on interacting with the page as usual, even if the call to the server takes some time.

It is up to the callback function to make whatever changes it needs to make to the web page using the usual Javascript techniques described above.  This will typically involve updating just a part of the screen.

One thing to note here is that the data returned is just text.  It doesn’t have to be XML, in spite of the names (XMLHttpRequest, AjaX).

3.  Updating a Page Automatically when Data Changes on the Server

Ajax as described so far updates a page in place, but only in response to a request from the web page.  This means that the user has to click a button or something similar for the page to update.

Obviously there are situations where data is changing and we would like it to update on our web page without the need for the user to manually refresh.

There are quite a few ways of doing this, some of them direct extensions to the Ajax model described above:

Polling

Javascript allows us to fire events that run code in the browser at set intervals.  So a very simple approach is to automatically request a refresh of the part of the screen we are interested in updating periodically.  We can do this using the Ajax techniques above, so that the rest of the screen remains responsive.

The drawback to this is we may make requests when no data has changed, putting unnecessary load on our servers.  Also our data on the client may well be out of date at any given time if we are between polling requests.

We really want a way for our server to send data only when it’s changed, and at the moment it has changed.

Long Polling

Another approach is long polling.  Here the browser fires off a request with a long timeout and sets up a handler for the results using Ajax as before.  However, the server itself doesn’t respond until it has data that has changed, and then it sends the data in response to the original request.  The browser handles the request and immediately sets up another long timeout request for future updates.

The disadvantage of this approach is that the server has to keep the connection (a network socket) open until it has data.  In general it will have as many connections as it has clients waiting for data.  This obviously puts load on the server and the number of sockets that the server can possibly use becomes a limiting factor.  Also this is clearly a more complex solution to implement than normal (short) polling.

Streaming

In streaming the client makes a request and the server responds with an open response that keeps the communication channel open and allows subsequent responses to be sent to the client.  The server may eventually time out the connection, or may keep it open indefinitely.  If the connection times out the client will have to make another request to refresh the data.  So this approach is like long polling with the client needing to make fewer requests.

One drawback of this approach is that many proxy servers buffer http responses until they are complete: that is, they won’t send on the message until they have the completion.  This means the client won’t get timely updates.  Another obvious drawback is that this is a fairly complex way of keeping data up to date.

With all of these approaches the call backs from the server tend to tie up one http communication channel.  As a result many approaches to solving the problem use (at least) two channels: one for polling or streaming to  update the data in place, and one for regular requests from the client to the server.

A number of commercial frameworks have been built using these techniques.

Comet

Comet is a name that’s been applied to the techniques described above to update a web page in place automatically when data changes on the server using a longlasting HTTP connection.

HTML 5 Web Sockets

HTML 5 web sockets are the new way to do bidirectional communication between a web page and a server.  They don’t use the old HTTP request/response at all, but instead set up one dedicated channel for communication between client and server.  This is fast, and the messages involve very little redundant header information, unlike conventional HTTP requests.

The main drawback of this new technology currently is that many browsers do not support it.  For example, it doesn’t work in the last version of Internet Explorer, IE9, although it works in IE10.

References

http://aspalliance.com/1391_Four_Ways_to_Locally_Update_a_Web_Page.8

http://www.xfront.com/REST-Web-Services.html

http://www.websocket.org/quantum.html

http://www.w3schools.com/jsref/tryit.asp?filename=tryjsref_button_disabled

September 6, 2012

Credit Default Swap Index Trading: Bid/Offer, Rolls, Roll Conventions

Introduction

I apologize in advance for the rather esoteric subject here, but I have in the past done some work with our CDS index trading desk.  The roll dates for the US indexes are once again approaching.  Immediately after the roll date we trade both the old index, the new index, and the roll between them.  The terminology, and what we are buying and selling, confuses us poor developers no end.  It is immensely confusing because of the price/spread conventions.

At the last roll I wrote some notes on this, which I’m publishing here to help anyone else struggling with the market.

This article focuses on the US CDS index trading market, and uses the Markit CDX indexes as examples.  This isn’t a general introduction to those markets, rather it’s a discussion of the mechanics of the market and what the bid/offer spreads actually mean.

I have written a general introduction to the credit default swap market.  Markit themselves have guides to the index markets (see the Primers section).

CDS Index Pricing

Some indexes trade on spread (e.g. CDX IG), some on price (e.g. CDX HY).  The series of articles on credit default swaps include a description of what these terms mean.

Indexes That Trade On Spread

In many ways spread is easier to think about.  In March 2012 some real prices quoted by a dealer were 90.0/90.5 for the CDX NA IG 18 index, 84.25/85.0 for the CDX NA IG 17, and 5.50/5.69 for roll.  I explain what these numbers actually mean below.

The Index Bid/Offer When Trading on Spread

The 18 price of 90.0/90.5 means that as a customer we pay the equivalent of a running premium of 90.5 to buy protection against defaults in the index, we receive 90.0 to sell protection.  Of course we don’t actually pay that premium, we pay a fixed premium of 100 and an upfront fee that adjusts for the difference to the quoted spread of 90.5.

As a customer we always buy at the offer, and sell at the bid.  Here we are buying protection at the offer (90.5).  So for a customer to ‘buy’ here means to buy protection.

The Roll When Trading On Spread

A roll trade is designed to swap a position in the old index into a position in the new index.  So you will sell protection in the old index and buy it in the new index, or vice versa.

The roll was trading at 5.50/5.69.  When you ‘buy the roll’ you, as usual, buy at the offer (at 5.69).  By convention for securities trading on spread you are buying protection on the new series (the 18), selling it on the old series (the 17).  I’m swapping a ‘long’ (bought protection) position in the 17 for a long position in the 18.

However, for reasons we shall see later it’s easier to think about bid and offer here.  If I buy the roll I’m trading at the offer price (5.69): I’m lifting the offer.  When I do that I’m also trading at the offer price for the 18, but trading at the bid price (hitting the bid) for the 17.

To calculate what trades you enter into at what prices if you trade the roll, you start with the 18 and work out the 17 from the roll spread.  You subtract the roll spread from the 18 price.  Note that this applies to indexes that trade on spread only: see below.

In this case if I buy the roll, I’m buying the 18 at the 18 offer (90.5) and calculate the 17 bid as 90.5-5.69 = 84.81.  Note that this is a better price for me than the actual outright 17 bid (84.25): I effectively receive premium at 84.81 instead of 84.25.

Note also that I haven’t crossed the spread on the 17: I’m trading at an implied 17 bid of 84.81 but this is still less than the offer of 85.00.  For obvious reasons this is an important check for a trader.

If I sell the roll at 5.50, I’m selling the 18 (at 90.0) and the implied 17 offer is 90.0-5.5 = 84.5, again a better price to me than the outright price of 85.0.

Indexes That Trade On Price

The CDX HY index trades on price.  In March 2012 some real prices quoted by a dealer were 97.56/97.75 for the CDX NA HY 18 index, 98.75/98.94  for the CDX NA HY 17 index and 1.19/1.31 for the roll.

The Index Bid/Offer When Trading On Price

Here buy protection/sell protection is reversed from above because of the way the index is quoted.  The 18 price of 97.56/97.75 means that as a customer I still ‘buy’ at the offer (97.75).  However, now this number is used to calculate my upfront fee directly, rather than referring to a notional spread over the life of the trade.  As discussed in my earlier article, the calculation is 100-price = points, and points is the percentage we apply to the notional to calculate the fee.

For the dealer to make money the customer has to pay more to buy protection than they would receive to sell protection.  Here if I ‘buy’ at 97.75 the associated fee is 2.25% of the notional, if I ‘sell’ at 97.56 the associated fee is 2.44% of notional.

What this means is that I’m actually buying protection at the bid (97.56) and selling protection at the offer (97.75), which is the reverse of trading on spread.

So for the HY index we are quoting ‘like a bond’.  If I buy the 18s at the offer (97.75) this is like entering a long bond position (buying a bond), going short protection.

To recap, the percentage fee I actually pay is 100 minus the quoted spread, so for the 18s this is 2.44/2.25.  So I receive 2.25 if I sell protection, pay 2.44 if I buy protection (percent of the notional).

The Roll When Trading On Price

The roll was trading at 1.19/1.31.

If I lift the offer on the roll (1.31) then I’m lifting the offer on the 17 and hitting the bid on the 18 (97.56).  Note that’s the reverse of what I did above for the IG.  This is just the convention: I explain why it’s the convention below.

However this is the SAME trade as lifting the offer on the roll for the IG in terms of buying/selling protection: I’ve sold protection on the 17, bought protection on the 18.

As for the IG we derive the 17 price for the roll from the 18 price and the roll spread.  Here we add.  Again I explain why this is the convention below.

So if I lift the offer on the roll I’ll calculate the implied 17 offer price by adding the roll offer price to the 18 bid price.

That is, the implied 17 = 97.56 + 1.31 = 98.87.  So we’ve sold protection on the 17 at 98.87, which means we receive more upfront fee than if we’d just lifted the offer on the outright 17 (98.94): remember we actually receive 100 minus the value here.  So the customer again gets a better deal by trading the roll than by trading the two outrights.

Note also we haven’t crossed the spread for the 17 (98.75 < 98.87 < 98.94).

Summary of Calculations When You Trade a Roll

Trading on Spread

New index 90.0/90.5

Roll 5.50/5.69

Implies

Customer buys roll (lifts offer), old index bid price = 90.5-5.69 = 84.81 (old bid = new offer – roll offer)

Customer sells roll (hits bid), old index offer price  = 90.0-5.50 = 84.5 (old offer = new bid – roll bid)

Trading on Price

New index 97.56/97.75

Roll 1.19/1.31

Implies

Customer buys roll (lifts offer), old index offer price = 97.56+1.31 = 98.87 (old offer = new bid + roll offer)

Customer sells roll (hits bid), old index bid price = 97.75+1.19 = 98.95 (old bid = new offer + roll bid)

Explanations for the Roll Conventions

The first thing to note is that in general it costs more to buy protection and you receive more if you sell protection in the newer index.  This is because the protection is for six months longer.  You may also have already had defaults in the old index so fewer names are being covered.  However, the index constituents are revised to include more liquid and better-quality credits and this can offset this effect to some extent.

Roll Spread Convention

This means that if you are trading on spread (new index spread- old index spread) is positive, because the new spread is larger, regardless of buy protection/sell protection.   We want to quote a positive spread, so we make that the calculation for our roll spread.

If you are trading on price then things are reversed:  (old index price – new index price) will be positive, so we make that our roll spread.  This is because the upfront fee is larger for the new index (it costs more to buy protection), so the actual price we quote is smaller for the new.

Hence the difference in the roll calculations: if trading on spread to get the old index spread we subtract the roll spread from the new index spread, if trading on price to get the old index price we add the roll spread to the new index price.

Bid/Offer Convention

The bid/offer conventions now follow from the above.  For both price conventions we will always have one trade at the bid and one at the offer in a roll.  We are always buying one index and selling the other.  There are alternative ways of quoting to those described above.  However, if we quote with the conventions described above we always have offer greater than bid, and are always buying protection on the newer index if we buy the roll, so it makes sense to do it this way.

Conclusion

I warned you it was confusing: hopefully this article at least gives some idea of what’s going on.

August 25, 2012

15-Minute Beginner’s Guide to Windows 8

Filed under: beginners guide, introduction, metro, Windows, Windows 8 — Tags: , , , , — richnewman @ 5:40 pm

Introduction

Windows 8 has been released to manufacturing, and is available to developers on MSDN.  It’s quite disorientating for people who’ve worked with Windows for a while.  I’ve been playing with it and wrote some notes for myself, so I thought I’d turn them into a quick guide to navigating your way around it.

I’m going to assume you’re experienced with previous versions of Windows, you’ve managed to get Windows 8 installed, and have got past the logon screen to the start screen.  I’m also going to assume you’re a developer and therefore don’t like to reach for the mouse too much whilst working: there will be a lot of shortcut keys in this.  I’m also assuming you don’t have a touch screen.

Philosophy

The first thing to realize is that Windows 8 is intended to be both a desktop operating system (OS) and a tablet operating system.  This is logical: Microsoft need a version of Windows that can run on low-powered tablets, so they either had to write a new OS or make Windows itself capable of doing it.  They went for the latter.

However, desktop and tablet operating systems are inevitably slightly different.  Windows 8  on a PC is effectively a desktop operating system with a tablet operating system embedded in it.

Windows 8 Style User Interface (previously ‘Metro’)

The tablet part of the new OS has a new tiled user interface design, currently called the ‘Windows 8 style user interface’.  It also has tablet-style apps that run full screen.  There’s a store for the apps: it looks like Microsoft is going to pursue the proprietary locked-in approach to tablet software that other companies are using.  Apple fanboys might want to think about the effects of Apple’s approach on the industry.

Of course Windows 8 still has a full old Windows 7 style desktop within it, including all of the old desktop applications that don’t have to run full screen.

Start Screen

The start screen is the one you see in all the screenshots.

Think of the start screen as a fullscreen and more sophisticated version of the start menu in Windows 7.  It even starts in a similar way: you go to far bottom left of the screen and click.

Obviously you can click on any of the tiles to launch the new apps.  You can also click on tiles for old desktop apps, although you may need to set them up.  You can also navigate and launch apps by using the arrow keys and Enter.

You can get back to the start screen once you’ve launched an app if  you hit the Windows key, or, as already mentioned, if you move your mouse to the far bottom left and click.  Hitting the Windows key again will take you back to where you were.

Rearranging the Start Screen

You can drag tiles around on the start screen to rearrange them.  You can move the mouse to the far left or far right to scroll.  If you right-click the background to the start screen an option for ‘All Apps’ appears, and you can right-click one of these to add it to the main start screen.

You can zoom out by clicking the little minus sign in the bottom right of the screen.  This is useful if you’ve set up a lot of tiles.  It allows you to move groups of tiles around by dragging, and to name them, by right-clicking.

Search

You can just start typing the name of your application with the start screen visible.  It will immediately show a search screen and filter down to your application in a few keystrokes, after which you can just hit ‘enter’.  Again this is very similar to starting an application with the keyboard in Windows 7 via the Start menu, except it’s faster and far more powerful.

You can also bring up the search screen from the charms menu (see below), or with Windows+q.

You can search for files or settings; just use the options underneath the search box. You can also search WITHIN an app in the same way.  For example, the way to search in the Wikipedia app is to use the Windows 8 search menu: there’s no visible search functionality in the apps own screens.

You can add applications to the start screen from the search screen as well.  Find the item you want to add with the search features and right-click it.

Windows 7 Style Desktop

You can get to the old style desktop by clicking on the desktop tile on the start screen, or by hitting Windows+d.  If the desktop is already running there are other ways of getting to it: more on this later.

The only really noticeable change in the new desktop is that the Start button has disappeared, to be replaced by the start screen as discussed above.  There are a few minor improvements to the desktop as well: for example, Task Manager is far more powerful, and if you do a large file copy you get a little chart of the speed over time in the copy dialog.  Also Windows Explorer now has a ribbon interface.

Missing from the Windows 8 desktop are the gadgets that you could set up on the desktop, and the Aero glass look for the title bars of the windows.  They aren’t available.  Window title bars don’t even have a gradient, they are just solid blocks of colour.  This is the new ‘chromeless’ look: it also affects things like scrollbars and buttons.

‘Charms’ Menu

Windows+c, or move mouse to bottom right or top left brings up the so-called ‘charms’ menu from anywhere.  It slides out from the right side of the screen.  This has icons for Search, Share, Start, Devices and Settings.  Search and Start bring up the relevant screens discussed above.

Settings

The Settings icon on the charms menu lets you access the full tablet settings screen by clicking on ‘Change PC Settings’ at the very bottom.  Here you can do things like change the picture on the lock screen or change the background to the Start screen (under ‘Personalize’), or change your password (under ‘Users’).

Tablet Apps

The tablet apps need work, although some of them are already pretty good.  Obviously you visit the Store app from the start screen to browse and install additional apps.  Here’s the default weather app, which has a lot more detail if you scroll to the right:

Many of the apps have menus in them.  To bring these up right-click on the background, or use Windows+z.

You can close an app with Alt + F4 or by moving the mouse to the top middle of the screen and dragging all the way to the bottom.  You can also right-click in the left-hand slideout menu mentioned above.

You usually have to scroll in an app by moving to the bottom of the screen and using the scrollbar that appears.  There don’t appear to be any mouse gestures to scroll.

Moving Between Apps

If you move the mouse to the bottom left and then move up, or to the top left and then move down, then a slideout menu appears on the left side of the screen with all the tablet apps previewed apart from the one you are currently in.  You can click on one to go to it.  This menu treats the entire desktop as one tablet app.

You can also bring up the slideout menu and tab between apps with Windows+Tab.  This is actually a bit annoying as it doesn’t include the current app so you can’t change your mind and stay where you are.

You can move to the last tablet app you were in by moving the mouse to the top left and clicking.

You can move between all open applications, desktop plus tablet, with Alt-Tab.

Tiling Tablet Apps

You can’t actually fully tile tablet apps, but you can show a main app and have a second one in a sidebar at the left- or right-hand side of the screen.  This is called the ‘Snap’ feature.  The sidebar will stay there as you show different apps in the main window area, including if you bring up the desktop.

By default this only works on fairly high resolution screens, 1366×768 or higher, which means it won’t work on most laptops or corporate desktops unfortunately.

To set this up bring up the lefthand slideout menu (bottom left and move up with the mouse), leftclick the open app that you want in a sidebar, and drag it into the sidebar position.

You can make the desktop itself into a sidebar, in which case it shows the open desktop applications.  You can also drag the sidebar divider to the right or left, which will close the sidebar or make it the main app.

Browsers

There are actually two versions of Internet Explorer 10 in Windows 8: the tablet app version and the version that runs on the desktop.  The app version has a less easy interface, but more significantly it will only run Adobe Flash on certain websites that Microsoft has vetted as safe.  The desktop version has no such restrictions.

Old Windows Keys Combinations

Most of the useful old Windows keys combinations still work from anywhere, including in tablet apps.  So Windows+e will bring up a Windows Explorer window on the desktop from anywhere, Windows+m will go to the desktop and minimize all applications.

Start Screen Right Click Menu

If you move your mouse to the bottom left to bring up the start screen icon and then right-click instead of left-clicking you get a handy power user menu for desktop functionality.  This works from anywhere.  The menu includes options to go directly to the Explorer, Task Manager, Event Viewer, Control Panel, Search, Desktop or an admin Command Prompt.

June 26, 2012

Floating Point Arithmetic

Filed under: .net — richnewman @ 2:35 pm

We had a discussion about floating point arithmetic on 32-bit processors in our team this morning: here’s a brief note I made on the subject a long time ago.

double a = 95.0;
double b = 94.0001;
double c = a – b;

In .Net this give c = 0.99989999999999668. It should be 0.9999 obviously. This is a classic floating point problem: big numbers and small numbers don’t mix well for addition and subtraction. You get the same problem in Java. This is because there’s an IEEE standard for how 32-bit doubles (and other floating point types) should work, which is used in both languages.

The real problem is there’s no easy way of dealing with the fact that the real numbers are infinite and computers just aren’t. More to the point, for efficiency we really want to put some ceiling on our level of accuracy, and in particular we really only want to use 32 bits on a 32-bit processor.

There’s a lot of detail on this on the web, some of it extremely tedious with geeks showing off how mathematical they are, some of it containing sensible advice:

http://docs.oracle.com/cd/E19957-01/806-3568/ncg_goldberg.html

http://www.ibm.com/developerworks/java/library/j-math2/index.html

In both .Net and Java there are specific types that are designed to be slow but allow arbitrary accuracy: these are Decimal (.Net) and BigDecimal (Java). Actually Decimal isn’t designed for arbitrary accuracy, it’s 128 bits, but it solves the problem above.

In Excel of course you can’t use these. Excel cheats a little bit in that if you only display 4 DPs (which it does by some default magic) it rounds to that so it appears you get 0.9999. In fact if you set it to show 15DP you get the same answer as above.

Microsoft gives some sensible non-geeky advice about dealing with this in Excel: round. You can even round to what’s displayed because that’s designed to work and there’s a specific option to do it (although it warns that, of course, you’re losing quite a bit of accuracy if you do this everywhere in a complex sheet). In the .Net example at the top of the page you get the right answer if you do Math.Round(c, 14). Another technique used in Excel is to test for a floating point value to within a small epsilon (i.e. don’t test for 0, test for 0 plus or minus epsilon).

http://support.microsoft.com/kb/214118

February 17, 2012

Creating Code Listings in WordPress using Visual Studio

Filed under: code, Visual Studio, Wordpress — Tags: , , — richnewman @ 12:19 am

In WordPress it has historically been quite hard to paste code listings into the editors and get them to look good.  There are some plugins that try to do it, but none of them seem to work very well with code copied out of Visual Studio.

One option for creating code listings from Visual Studio is to paste an HTML version of the code into the HTML editor in WordPress.  In the past I’ve done this by pasting the code from Visual Studio into Word, saving as HTML, and then running a little bespoke program to tidy up the mess that Word makes of the resulting HTML.  All that was a pain.

Things are much easier in versions of Visual Studio from 2010 onwards, where you can use options in Microsoft’s Productivity Power Tools.  This is a Visual Studio extension (accessible via Tools/Extension and Updates…). In all versions it has an option to copy code as HTML, amongst other things.

Unfortunately with Visual Studio 2010 and 2012 there’s no easy way to paste as HTML directly into the HTML editor in WordPress.  If you try you just get the text.

The trick here is to first paste into something that expects an HTML listing and will show all the tags as a result.  You can do this by adding an HTML page to your Visual Studio project and pasting into there.  The full steps to do this in any version of Visual Studio from 2010 onwards are below.

However, note that there’s an even easier way to do this in Visual Studio 2013 only (at the time of writing).  If you install the Productivity Power Tools 2013 in Visual Studio 2013 it puts an option on the edit menu ‘Copy Html Markup’.  If you select some code and then use this option you can paste directly into the HTML editor of your post in WordPress, and if you then switch to the visual editor you should see that the code is copied correctly.

The steps to get a code listing from Visual Studio 2010 or 2012 into WordPress in the same format as it is in Visual Studio are:

  1. Install the Productivity Power Tools in Visual Studio using Tools/Extension Manager.
  2. Add an HTMLPage item to your Visual Studio project
  3. Select the code you want to copy and hit Ctrl-C
  4. Go to your HTMLPage, right-click, Paste Alternate (not Paste: there should be a second paste option on the context menu).  This should show you the text with the HTML tags.
  5. Ctrl-A, Ctrl-C to select and copy the listing with the tags.
  6. Go to the HTML editor (‘Text’ tab at top) of your post in WordPress and paste the code in there at the appropriate point.

Below is an example of the results of this with a short method.  First what the end result looks like:

        static void MyMethod(int x, string y = "Hello"string z = "World") 
        {
            Console.WriteLine(y + " " + z + ": " + x);
        }

Then what the underlying HTML looks like.  You can tweak the generation of this in the Productivity Power Tools options but going with the defaults works fine for me:

<pre style="font-family:Consolas;font-size:13;color:black;background:white;">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">static</span>&nbsp;<span style="color:blue;">void</span>&nbsp;MyMethod(<span style="color:blue;">int</span>&nbsp;x,&nbsp;<span style="color:blue;">string</span>&nbsp;y&nbsp;=&nbsp;<span style="color:#a31515;">&quot;Hello&quot;</span>,&nbsp;<span style="color:blue;">string</span>&nbsp;z&nbsp;=&nbsp;<span style="color:#a31515;">&quot;World&quot;</span>)&nbsp;
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;{
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2b91af;">Console</span>.WriteLine(y&nbsp;+&nbsp;<span style="color:#a31515;">&quot;&nbsp;&quot;</span>&nbsp;+&nbsp;z&nbsp;+&nbsp;<span style="color:#a31515;">&quot;:&nbsp;&quot;</span>&nbsp;+&nbsp;x);
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;}</pre>

February 7, 2012

Delegate Syntax in C# for Beginners

Filed under: .net, beginners guide, c#, code syntax, delegate — Tags: , , , , — richnewman @ 3:48 am

Introduction

I have been programming with C# since it came out but I still find the delegate syntax confusing.  This is at least partially because Microsoft have changed the recommended syntax regularly over the years.  This article is a quick recap of the various syntaxes.  It also looks at some of the issues with using them in practice.  It’s worth knowing about all the various syntaxes as you will almost certainly see all of them used.

This article is just a recap: it assumes that you know what a delegate is and why you’d want to use one.

.Net and Visual Studio Versions

The first thing to note is that you can use any of these syntaxes as long as you are using Visual Studio 2008 or later and targeting .Net 2.0 or later.

Named methods were available in .Net 1.0, anonymous methods were introduced in .Net 2.0, and lambda expressions were introduced in .Net 3.0.  However, like much of .Net 3.0, which is based on the .Net 2.0 assemblies, lambda expressions will compile to .Net 2.0 assuming you have the appropriate version of Visual Studio.

Note also that lambda expressions can do (almost) everything anonymous methods can do, and effectively supersede them as the preferred way of writing inline delegate code.

Code

A listing of the code for this article is availableThe complete working program is also available.

The Delegate

For all of these examples we need a delegate definition.  We’ll use the one below initially.

        private delegate void TestDel(string s);

Named Methods

Named methods are perhaps the easiest delegate syntax to understand intuitively.  A delegate is a typesafe method pointer.  So we define a method:

        private void Test(string s)
        {
            Console.WriteLine(s);
        }

Now we create an instance of our method pointer (the delegate above) and point it at our method.  Then we can call our method by invoking the delegate.  The code below prints out ‘Hello World 1′.  This is easy enough, but all a little cumbersome.

            TestDel td = new TestDel(Test);
            td("Hello World 1");

There’s one slight simplification we can use.  Instead of having to explicitly instantiate our delegate with the new keyword we can simply point the delegate directly at the method, as shown below.  This syntax does exactly the same thing as the syntax above, only (maybe) it’s slightly clearer.

            TestDel td2 = Test;
            td2("Hello World 2");

There is an MSDN page on named methods.

Anonymous Methods

The anonymous method syntax was introduced to avoid the need to create a separate method.  We just create the method in the same place we create the delegate.  We use the ‘delegate’ keyword as below.

            TestDel td3 = 
                delegate(string s)
                {
                    Console.WriteLine(s);
                };
            td3("Hello World 3");

Now when we invoke td3 (in the last line) the code between the curly braces executes.

One advantage of this syntax is that we can capture a local variable in the calling method without explicitly passing it into our new method.  We can form a closure.  Since in this example we don’t need to pass our string in as a parameter we use a different delegate:

        private delegate void TestDelNoParams();

We can use this as below.  Note that the message variable is not explicitly passed into our new method, but can nevertheless be used.

            string message = "Hello World 4";
            TestDelNoParams td4 = 
                delegate()
                {
                    Console.WriteLine(message);
                };
            td4();

There is an MSDN page on anonymous methods.

Lambda Expressions

Lambda expressions were primarily introduced to support Linq, but they can be used with delegates in a very similar way to anonymous methods.

There are two basic sorts of lambda expressions.  The first type is an expression lambda.  This can only have one statement (an expression) in its method.  The syntax is below.

            TestDel td5 =  s => Console.WriteLine(s);
            td5("Hello World 5");

The second type is a statement lambda: this can have multiple statements in its method as below.

            string message2 = "Hello World 8";
            TestDel td6 =
                s => 
                { 
                    Console.WriteLine(s); 
                    Console.WriteLine("Hello World 7");
                    Console.WriteLine(message2);
                };
            td6("Hello World 6");

Note that this example also shows a local variable being captured (a closure being created).  We can also capture variables with expression lambdas.

There is an MSDN page on lambda expressions.

Return Values

Nearly all of the examples above can be extended in a simple way to return a value.  The exception is expression lambda which cannot return a value. Doing this is usually an obvious change: we change our delegate signature so that the method it points to returns a value, and then we simply change the method definition to return a value as usual.  For example the statement lambda example above becomes as below.  The invocation of tdr6 now returns “Hello ” + message2, which we write to the console after the invocation returns:

            string message2 = "World 8";
            TestDelReturn tdr6 =
                s =>
                {
                    Console.WriteLine(s);
                    Console.WriteLine("Hello World 7");
                    return "Hello " + message2;
                };
            Console.WriteLine(tdr6("Hello World 6"));

The full list of all the examples above modified to return a value can be seen in the code listing in the method ExamplesWithReturnValues.

Events

All of these syntaxes can be used to set up a method to be called when an event fires.  To add a delegate instance to an event we used the ‘+=’ syntax of course.  Suppose we define an event of type TestDel:

        private event TestDel TestDelEventHandler;

We can add a delegate instance to this event using any of the syntaxes in an obvious way.  For example, to use a statement lambda the syntax is below.  This looks a little odd, but certainly makes it easier to set up and understand event handling code.

            TestDelEventHandler += s => { Console.WriteLine(s); };
            TestDelEventHandler("Hello World 24");

Examples of setting up events using any of the syntaxes above can be found in the code listing.

Passing Delegates into Methods as Parameters: Basic Case

Similarly all of the syntaxes can be used to pass a delegate into a method, which again gives some odd-looking syntax.  Suppose we have a method as below that takes a delegate as a parameter.

        private void CallTestDel(TestDel testDel)
        {
            testDel("Hello World 30");
        }

Then all of the syntaxes below are valid:

            CallTestDel(new TestDel(Test));  // Named method
            CallTestDel(Test);               // Simplified named method
            CallTestDel(delegate(string s) { Console.WriteLine(s); });  // Anonymous method
            CallTestDel(s => Console.WriteLine(s));  // Expression lambda
            CallTestDel(s => { Console.WriteLine(s); Console.WriteLine("Hello World 32"); });  // Statement lambda

Passing Delegates into Methods as Parameters: When You Actually Need a Type of ‘Delegate’

Now suppose we have a method as below that expects a parameter of type Delegate.

        private void CallDelegate(Delegate del)
        {
            del.DynamicInvoke(new object[] { "Hello World 31" });
        }

The Delegate class is the base class for all delegates, so we can pass any delegate into CallDelegate.  However, because the base Delegate class doesn’t know the method signature of the delegate we can’t call Invoke with the correct parameters on the Delegate instance.  Instead we call DynamicInvoke with an object[] array of parameters as shown.

Note that there are some methods that take Delegate as a parameter in the framework (e.g. BeginInvoke on a WPF Dispatcher object).

There’s a slightly unobvious change to the ‘Basic Case’ syntax above if we want to call this method using the anonymous method or lambda expression syntax.  The code below for calling CallDelegate with an expression lambda does NOT work.

            CallDelegate(s => Console.WriteLine(s));  // Expression lambda

The reason is that the compiler needs to create a delegate of an appropriate type, cast it to the base Delegate type, and pass it into the method.  However, it has no idea what type of delegate to create.

To fix this we need to tell the compiler what type of delegate to create (TestDel in this example).  We can do this with the usual casting syntax (and a few more parentheses) as shown below.

            CallDelegate((TestDel)(s => Console.WriteLine(s)));  // Expression lambda

This looks a little strange as we don’t normally need a cast when assigning a derived type to a base type, and in any case we’re apparently casting to a different type to the type the method call needs.  However, this syntax is simply to tell the compiler what type of delegate to create in the first place: the cast to the base type is still implicit.

We need to do this for any of the syntaxes apart from the very basic named method syntax (where we’re explicitly creating the correct delegate):

            CallDelegate(new TestDel(Test));  // Named method
            CallDelegate((TestDel)Test);      // Simplified named method
            CallDelegate((TestDel)delegate(string s) { Console.WriteLine(s); });  // Anonymous method
            CallDelegate((TestDel)(s => Console.WriteLine(s)));  // Expression lambda
            CallDelegate((TestDel)(s => { Console.WriteLine(s); Console.WriteLine("Hello World 32"); }));  // Statement lambda

Actions/Funcs

There is one further simplification that we can use in the examples in this article.  Instead of defining our own delegates (TestDel etc.) we can use the more generic Action and Func delegates provided in the framework.  So, for example, everywhere we use TestDel, which takes a string and returns void, we could use Action<string> instead, since it has the same signature.

Older Posts »

The Shocking Blue Green Theme. Create a free website or blog at WordPress.com.

Follow

Get every new post delivered to your Inbox.

Join 82 other followers