Rich Newman

February 17, 2012

Creating Code Listings in WordPress using Visual Studio

Filed under: code, Visual Studio, Wordpress — Tags: , , — richnewman @ 12:19 am

In WordPress it has historically been quite hard to paste code listings into the editors and get them to look good.  There are some plugins that try to do it, but none of them seem to work very well with code copied out of Visual Studio.

One option for creating code listings from Visual Studio is to paste an HTML version of the code into the HTML editor in WordPress.  In the past I’ve done this by pasting the code from Visual Studio into Word, saving as HTML, and then running a little bespoke program to tidy up the mess that Word makes of the resulting HTML.  All that was a pain.

Things are much easier in versions of Visual Studio after 2010.  In Visual Studio 2010 you can use an option in Microsoft’s Productivity Power Tools.  This is a Visual Studio extension (accessible via Tools/Extension Manager). It has an option to directly copy the code as HTML, amongst other things.  In versions of Visual Studio later than 2010 this option to copy the code as HTML is included in Visual Studio.

Unfortunately there’s no easy way to paste as HTML directly into the HTML editor in WordPress.  If you try you just get the text.

The trick here is to first paste into something that expects an HTML listing and will show all the tags as a result.  You can do this by adding an HTML page to your Visual Studio project and pasting into there.

So the steps to get a code listing from Visual Studio into WordPress in the same format as it is in Visual Studio are:

  1. In Visual Studio 2010, install the Productivity Power Tools in Visual Studio using Tools/Extension Manager.  In Visual Studio 2012 or later there’s no need to do this step.
  2. Add an HTMLPage item to your Visual Studio project
  3. Select the code you want to copy and hit Ctrl-C
  4. Go to your HTMLPage, right-click, Paste Alternate (not Paste: there should be a second paste option on the context menu).  This should show you the text with the HTML tags.
  5. Ctrl-A, Ctrl-C to select and copy the listing with the tags.
  6. Go to the HTML editor (‘Text’ tab at top) of your post in WordPress and paste the code in there at the appropriate point.

Below is an example of the results of this with a short method.  First what the end result looks like:

        static void MyMethod(int x, string y = "Hello"string z = "World") 
        {
            Console.WriteLine(y + " " + z + ": " + x);
        }

Then what the underlying HTML looks like.  You can tweak the generation of this in the Productivity Power Tools options but going with the defaults works fine for me:

<pre style="font-family:Consolas;font-size:13;color:black;background:white;">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">static</span>&nbsp;<span style="color:blue;">void</span>&nbsp;MyMethod(<span style="color:blue;">int</span>&nbsp;x,&nbsp;<span style="color:blue;">string</span>&nbsp;y&nbsp;=&nbsp;<span style="color:#a31515;">&quot;Hello&quot;</span>,&nbsp;<span style="color:blue;">string</span>&nbsp;z&nbsp;=&nbsp;<span style="color:#a31515;">&quot;World&quot;</span>)&nbsp;
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;{
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2b91af;">Console</span>.WriteLine(y&nbsp;+&nbsp;<span style="color:#a31515;">&quot;&nbsp;&quot;</span>&nbsp;+&nbsp;z&nbsp;+&nbsp;<span style="color:#a31515;">&quot;:&nbsp;&quot;</span>&nbsp;+&nbsp;x);
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;}</pre>

June 6, 2009

Running Visual Studio as an Administrator under Windows Vista

Filed under: .net, Visual Studio — Tags: , , , , , — richnewman @ 11:36 pm

Introduction

This article discusses why we should run Visual Studio as an administrator, and examines the easiest way to do that under Windows Vista.

Visual Studio and Administrator Rights

As .Net developers we need to be able to run Visual Studio as a Windows administrator.

Visual Studio 2005 actually warns you at start up if you are not running it as an administrator under Vista, and Microsoft recommends that you always run it with administrator privileges.  In fact MSDN gives a list of scenarios where Visual Studio 2005 will have ‘issues’ if it is not running with these privileges:

http://msdn.microsoft.com/en-us/vstudio/aa972193.aspx

For other versions of Visual Studio and operating systems there are fewer issues about not running Visual Studio as an administrator.  However there are a number of scenarios where you need administrative privileges.  These include needing to install or reinstall anything, or to do any kind of Office add-in work, or to use ActiveX controls, or even just to register a library for COM Interop.

Running Visual Studio as an Administrator

Under earlier versions of Windows developers usually just made themselves local administrators on their development computers to ensure Visual Studio had sufficient privileges.   Under Vista however, this is not enough (it is still necessary to be an administrator, but not sufficient).

This is because if you run an application under Vista by default it does not have administrative privileges unless specifically given them.  This is true even if the user running the application has administrative privileges.

Granting Visual Studio Administrative Privileges

There are (at least) three ways of starting Visual Studio with administrative privileges in Vista.

Note that for all three you need access to an administrator account.  It’s easiest if you are logged in as an administrator, in which case you will simply get a User Account Control warning saying ‘A program needs your permission to continue’ (and you can just click ‘OK’).  If you are not logged in as an administrator the User Account Control will ask you for an administrator’s ID and password before it will launch the program.

The three ways of running Visual Studio with administrative privileges are:

  1. Right-click on the shortcut to Visual Studio and select ‘Run as administrator’ from the context menu.
  2. Type ‘devenv’ into the Start Search box (at the bottom of the Vista Start menu: just hit the Windows key and you are in it).  Then hit Control-Shift-Enter, rather than just Enter.  Control-Shift-Enter tells Windows to start the program with administrative privileges.  This one is for those of you who don’t like reaching for a mouse.
  3. Right-click a shortcut to Visual Studio, select the Compatibility tab, and check the ‘Run this program as an administrator’ checkbox at the bottom.

Always Running Visual Studio as an Administrator

The last of the items above (item 3) is the one to use if you always want to run Visual Studio as an administrator.  Once the checkbox is checked in future you will be able to click on any shortcut to Visual Studio and it will run with elevated privileges.

Note that setting the ‘Run this program as an administrator’ property on a shortcut actually sets it on the underlying executable itself (in this case devenv.exe).  It isn’t possible as far as I can see to have one shortcut that will run the program as an administrator, and another one that will run it with normal privileges.

User Account Control

If you always run Visual Studio as an administrator you are going to get the User Access Control warning every time you start it, even if you are logged in as an administrator to Windows.  Obviously you can just click ‘OK’ to dismiss this warning, but it may tempt you to turn User Access Control off.

Note that this is true only if Vista’s User Account Control (UAC) is turned on.  Many developers turn UAC off, and in this case Vista behaves in the same way as earlier versions of Windows with regard to starting Visual Studio: if you are logged in as an administrator then Visual Studio will by default run with administrative privileges.

The Administrator Account

Vista also has an account called ‘Administrator’ which behaves differently from other administrator accounts.  In fact it behaves like administrator accounts in earlier versions of Windows, in that all programs launched when using it run with administrator privileges by default.  There’s no need to specifically set up the program as described above.

As a developer your really shouldn’t need to use this account: you can develop with administrator privileges using the techniques described in this article.

However you may have occasions when you aren’t sure whether a program is failing because of some coding error or simply because a process is being launched with insufficient privileges.  In these cases it may be useful to use the Administrator account temporarily to simply rule out a problem with privileges.  Note that if you work for a large organization they are almost certainly not going to let you near this account, however: this is really only useful for those developing at home.

Using the Administrator Account

To enable the Administrator account start a command prompt with administrator privileges as described above (type ‘cmd’ in the Start Search box and hit Control-Shift-Enter).  Then enter:

net user Administrator /active:yes

This has a blank password by default.  To set a password use:

net user Administrator {password}

You can now log off and log on as the Administrator.  Once you are done with any testing you should disable this account again as below

net user Administrator /active:no

Note that disabling the account does not clear the password.  However if you forget it you can always set it again as above when you come to use the account again (provided you have access to at least one account with administrator privileges).

Aside: Administrator Account on Windows XP

The Administrator account discussed above is the main default administrator account for a computer.  Previous versions of Windows had this account as well, although other accounts with administrator rights behaved in the same way.

In particular it existed under XP.  Not only that it was enabled by default, and had a blank password.  If you’ve got an XP computer at home try hitting Ctrl-Alt-Del twice on the screen that shows your accounts, and then enter ‘Administrator’ and a blank password.  If that logs you on with admin rights you may want to consider disabling the account as described above.

March 2, 2008

Model-View-Presenter using the Smart Client Software Factory (Introduction To CAB/SCSF Part 25)

Introduction

Part 23 and part 24 of this series of articles described the Model-View-Presenter pattern.

This article explains how the Smart Client Software Factory supports this pattern by generating appropriate classes.

Guidance Automation Packages in the Smart Client Software Factory

We saw how we could use the Smart Client Application Guidance Automation Package to set up a Smart Client Application in part 18. We can also set up a Model-View-Presenter pattern in a Smart Client application using another of the Guidance Automation Packages.

This will only work in an existing Smart Client Application.

Running the Model-View-Presenter Package

To use the Guidance Automation Package we right-click in Solution Explorer on a project or folder where we want to run the package. It is intended that we do this in the Views folder in a business module. On the right-click menu we select ‘Smart Client Factory/Add View (with presenter)’. We get a configuration screen that lets us name our view, and also lets us put the classes that get created into a folder. For the purposes of this example we name our view ‘Test’, and check the checkbox that says we do want to create a folder for the view.

When we click ‘Finish’ we get three classes and a TestView folder as below:

mvpsolutionexplorer.jpg

Classes Created

  1. TestView
    This is (obviously) our View class. It is intended that this contain the auto-generated code to display the View. As discussed in the previous articles any complex view logic will not go into this class, but will go into the Presenter.
  2. TestViewPresenter
    This is our Presenter class. As discussed in previous articles this should contain logic to deal with user events. It should also contain any complex view logic, and should directly update the View with the results of an view logic calculations. It has access to the View class via an interface.
  3. ITestView
    This is the interface that the View implements. The Presenter can only update the View through this interface.

Diagram

In terms of the diagrams shown in parts 23 and 24 this looks as below. Remember that we may or may not have arrows between the Model and the View depending on whether we are using the active View or passive View version of Model-View-Presenter:

mvpdiagram2.jpg

Where’s the Model?

The Guidance Automation package does not set up a Model class for us. As we have seen, the Model has no direct references to a View/Presenter pair (it raises events), and there may be multiple View/Presenter pairs for one Model. Further the Model would not usually be in the same folder, or even in the same component, as our View and Presenter.

For these reasons we are expected to set up our Model classes separately by hand.

Note that the Presenter (and the View as well if we are using the active View pattern) will have a direct reference to the Model. We will have to add these references manually.

Active and Passive View: a Quick Recap

Remember that in Model-View-Presenter the Presenter updates the View via an interface. We can set this up so only the Presenter is allowed to update the View. This is the ‘passive View’ pattern. We can also set this up so that the Presenter can update the View in complex cases, but the View can also update itself (in response to an event or user request) in simple cases. This is the ‘active View’ pattern.

Active and Passive View: Which Should We Use?

The pattern described in the SCSF documentation is the passive View: the documentation implies that all updates to the View should be done by the Presenter.

However there is nothing to stop us using the active View pattern with the classes generated by the Guidance Automation Package. We can add code to update the View wherever we like. In fact I would recommend using active View in simple cases: passive View should only be used where we are putting too much logic into the View class.

Should We Use Model-View-Presenter for Every Screen? A Personal View

Let me also reiterate a point made in part 24. It’s easy to get obsessive about the use of patterns and use them everywhere without thinking. My personal opinion is that we should only use the full Model-View-Presenter pattern where we have a complex screen that will benefit from the separation of the View and Presenter classes. For very basic screens the pattern is really too complex to give us benefit. In simple cases I think it is fine to put event handling and screen update logic directly behind the screen.

Note that I don’t think this applies to the use of the Model. We should always separate out the business logic from our screens into separate classes (this is what Martin Fowler calls ‘Separated Presentation’). However, we frequently have screens that don’t show any business logic or business data, so we may not need a Model class either.

For example an About screen that just shows the system name and version won’t need separate View and Presenter classes, and probably won’t need anything in a Model class either.

Equally a screen that shows a read-only grid of currencies used in a trading system probably doesn’t need separate View and Presenter classes. In this case the currencies themselves should be in a Model class so that other screens can access them.

Implementation Details: What We’d Expect

If we examine the diagram above, we expect the Presenter to have a data member with type of our ITestView interface that it will use to access the View. We expect the View to implement the ITestView interface to allow this. We further expect the View to have a direct reference to the Presenter class (a data member), which it will use to invoke code relating to user events. We’d probably expect both the View and the Presenter classes to be created the first time the View is needed.

Implementation Details: the Base Presenter Class

The actual details of the implementation of the Presenter are a little unusual.

If we look at the code generated by the Guidance Automation Package we see that the TestViewPresenter above has been given its core functionality by inheriting from an abstract Presenter<TView> class. Remember that the generic ‘TView’ simply lets us provide a type whenever we use the Presenter class. Here we inherit from Presenter, and provide the type when we inherit:

    public partial class TestViewPresenter : Presenter<ITestView>
    {

This allows the base Presenter class to have a data member of type ITestView (which is what we expect), rather than it being directly in the TestViewPresenter class. Note that the base Presenter is in the Infrastructure.Interface project (which is one of the reasons why we have to use this pattern in a Smart Client application).

The base Presenter class exposes our ITestView data member publicly, contains a reference to our WorkItem, and has disposal code and a CloseView method. It also has virtual OnViewReady and OnViewSet methods. These get called when you’d expect from the name and let us respond at the appropriate times by overriding the methods in our TestViewPresenter class.

All the above functionality in the base Presenter class means that the derived TestViewPresenter class is basically empty when it is created. It is up to us to put logic in there to handle user events and complex view logic.

The TestView class is a normal user control. It implements ITestView and contains a reference to the TestViewPresenter as we’d expect. It also calls OnViewReady as appropriate (in the OnLoad event of the user control). Again other than this TestView is also basically empty.

Conclusion

This article has shown us how to set up Model-View-Presenter classes using the Smart Client Software Factory, and discussed some issues surrounding it.

February 23, 2008

Table of Contents: Introduction to using Financial Products Markup Language (FpML) with Microsoft .NET Tools

Part 1 Introduction

An overview of how we can use Visual Studio to examine the FpML XSDs. Shows how to create a Visual Studio project containing the FpML schemas, and how to use that to navigate through them. Also shows how to validate the XML examples that are provided in the FpML download, both using Visual Studio and in C# code.

Part 2 How FpML Validates A Basic Trade Document

Looks at the structure of an FpML trade document in some detail. Explains how the FpML schemas fit together to validate such a document, and examines the use of base and extension types, and of substitution groups, in the XSDs.

Part 3 Generating .Net Classes From The FpML XSD Schema

Shows how to generate .Net classes in C# from the XML schema documents using Xsd.exe.  Explains how to load data into these classes from an FpML document using C# code, and to save the document back out again from the classes.

Part 4 Problems With Using Xsd.exe To Generate .Net Classes From The FpML XSD Schema

Goes through some of the problems with the C# code generated in part 3, and discusses how to fix them.

January 20, 2008

How FpML Validates a Basic Trade Document (Introduction to using FpML with .NET Tools Part 2)

Introduction

Part 1 of this series of articles gave a general introduction to using Microsoft .NET tools to investigate FpML XSD schemas.

This article will focus more closely on the FpML itself, by looking at the structure of an FpML trade XML document in some detail. It will explain how the FpML schemas fit together to validate such a document. In particular it will examine the use of base and extension types, and of substitution groups, in the XSDs.

As mentioned in part 1, FpML is complex and there’s little documentation. This article will attempt to redress this by explaining how it works assuming only that you know the basics of how an XSD works.

These articles assume you understand the basics of XML Schema Definition language. If you don’t there are a number of good tutorials available on the internet. For example there’s quite a good one at http://www.w3.org/TR/xmlschema-0/. These articles will reference the tutorial for the less obvious XSD structures.

The fpml-main Schema

In part one of this series of articles we showed that the file ird_ex01_vanilla_swap.xml is a valid document in accordance with the FpML schemas. ird_ex01_vanilla_swap.xml is an example file provided with the FpML download that represents a simple interest rate swap. However, at first glance it’s not entirely clear how the large number of XSD files that define the FpML schema fit together to validate this XML document.

The first thing to notice is that the XML document references the main root schema fpml-main-4-2.xsd in namespace http://www.fpml.org/2005/FpML-4-2. FpML puts all its definitions in one namespace, which for the version we are using is http://www.fpml.org/2005/FpML-4-2.

If we look at file fpml-main-4-2.xsd and collapse it to an appropriate level we see something as below. I’ve moved the schema attributes onto multiple lines for clarity:

fpml-mainxsd.jpg

As we can see, this XSD defines a number of namespaces, sets our targetNamespace for the schema, and includes a number of other XSDs from the set of FpML XSDs. It defines one complex type, ValuationDocument. Remember that complex types only have to appear in the XML if they are referenced as the type of an element: ValuationDocument does not have to appear in any XML just by virtue of being in this document.

The schema finally defines the one mandatory top-level element in this document, which is an element called ‘FpML’ which is of type ‘Document’. If we expand the FpML element definition we find it just contains an annotation (comment). So in effect all this schema is saying is that our root element for an FpML document must be called ‘FpML’ and it must be a Document.

Schemas Needed to Validate an Interest Rate Swap

It’s also worth noting here that we don’t need all the XSDs that are imported here to validate the interest rate swap successfully. In fact we can comment them all out apart from fpml-ird-4-2.xsd, provided we add an import for fpml-doc-4-2.xsd (which is included in fpml-valuation-4-2.xsd). We need to comment out the ValuationDocument complex type as well if we do this.

Note you can use the usual comment out/uncomment buttons in Visual Studio to comment XML (and hence XSD) code.

So the ird_ex01_vanilla_swap.xml file will validate with fpml-main changed as below. This makes it clear that we are only using a subset of our XSD files to validate an interest swap, which is what you’d expect. Note that fpml-ird-4-2.xsd itself includes a number of other FpML schemas, so we are not JUST validating with the two included below:

fpml-mainxsdcommented.jpg


Document Type

We can now navigate to the Document complex type, which is in file fpml-doc-4-2.xsd. Remember that to do this in Visual Studio all we need do is right-click “Document” and select “Go to definition” (or hit F12). We see that Document is defined as ‘abstract’. This is equivalent to a class being abstract in C#. It means we can’t use Document directly as the type of an element, but have to derive from it first. Document also contains an attributeGroup that references StandardAttributes.atts. This has one mandatory attribute ‘version’ (which applies to the FpML element), and can have value “4-0”, “4-1”, or “4-2”.

So now we need to know what complex types derive from the Document type. The easiest way to find this out using Visual Studio is to search our project for “Document” (in quotes), since any extension has to use the extension syntax which looks like:

<xsd:extension base="Document">

If we do this we find there are two types that extend Document: “DataDocument” and “Message”. Obviously the ‘Message’ type of document is used for messaging. Here we are only looking at trade representations in XML, which use the DataDocument type of Document.

The XML Document

If we now look at our instance XML document we see that, as expected, it starts with an FpML tag, which has a version attribute. It then uses the xsi:type attribute to specify that we are using a DataDocument.

Most FpML trade documents will have the general structure below, as we shall see. That is, the root FpML element will contain a trade element and a number of party elements. The trade element will contain a tradeHeader element, and an element that represents a specific product, in this case swap. For other product types the swap element will be substituted directly with some other product element.

vanillaswapxml.jpg


DataDocument

We saw above that our FpML element is of type DataDocument. The DataDocument definition is in fpml-doc-4-2.xsd, and a summarized version is below. If we look at the full version we can see there is a comment that says that the data document is intended to ‘contain sets of data without expressing any processing intention’:

datadocumentxsd.jpg

As expected this extends Document. It adds a sequence, starting with a Validation.model group. The element in this group is optional, so it doesn’t don’t appear in our XML at all.

The next item in our schema is a choice between a sequence of trade and portfolio elements, or of an event element. Our XML uses the trade and portfolio choice. Note that both of these are optional (minOccurs = 0), and our XML has no portfolio element. We examine the Trade element in more detail below.

Finally in our sequence we can have multiple (or zero) elements called ‘party’. These are intended to identify any parties to the trade (see the comments in the full version of the XSD). They are of type Party: this has an ‘id’ attribute, a required ‘partyId’ element, and optional partyName and account elements. If you look again at our XML you can see that we have two ‘party’ elements defined, with just the id attribute and mandatory partyId provided.

Trade

So in an FpML document we have a root element called ‘FpML’ that has to be of type Document. Document is abstract, and one extension is DataDocument, which can contain an element called ‘trade’ of type Trade.

The XSD definition of the Trade complex type is below:

tradexsd.jpg

The first thing to notice is that everything is optional (minOccurs=0) apart from the tradeHeader element, and the referenced product element. These two elements are the only ones that appear in our sample XML (although indirectly in the case of product, as we shall see below).

TradeHeader

By now you’ve probably got the idea of how this works and how to use Visual Studio to navigate through the schema and find out what needs to be where in the XML. As a result we won’t go into how the tradeHeader element is constructed, which is straightforward. The comment on the TradeHeader complex type says ‘a type defining trade related information which is not product specific’, which is self-explanatory.

It’s interesting to note that the ONLY information relating to a trade that is not ‘product specific’ here is the trade date and at least one trade ID (along with whose ID it is). If you are constructing a base trade class for a hierarchy of trade types this is really the only data you can meaningfully put in it.

Product and Swap

The product element is a little more complex. If we look again at our sample XML we see that we have a ‘swap’ element in the XML where we expect the ‘product’ element to be.

If we go to the definition of the referenced product element we find that it is of complex type Product, is abstract, and is defined in fpml-shared-4-2.xsd.

However, in spite of the product element (and its associated type Product) being abstract the schemas are not using straightforward extension here. Hence there is no ‘xsi:type’ attribute in our XML instance document. Instead we are using substitution groups.

If we locate the definition of ‘swap’ in fpml-ird-4-2.xsd we see that it is an element of type Swap with substitutionGroup of product. This means it can be substituted directly for the product element in our final XML (as can any other element tagged in this way):

  <xsd:element name="swap" type="Swap" substitutionGroup="product">
    <xsd:annotation>
      <xsd:documentation xml:lang="en">A swap product definition.</xsd:documentation>
    </xsd:annotation>
  </xsd:element>

The substitution group syntax on its own means that element swap and type Swap need have no relation to type Product. Swap could be a completely new type with no reference to the base Product type. We can substitute whatever we like for the product element.

However, in the FpML schemas complex type Swap also extends type Product, which means this element has to take account of the definition of the Product complex type.

So, to summarize, we have a ‘swap’ element in our XML document where there is a ‘product’ element in the XSD. We can put the swap element in there because of the substitution group syntax: we just substitute the swap element for the product element. However, the swap element is also of type Swap which extends type Product, so it must take account of the Product definition as well.

This is certainly difficult to understand.

Product itself is abstract, has an id attribute and contains a named group called ‘Product.model’. However, all of this is optional and does not consequently appear in our sample instance XML.

Difficulties with Substitution Groups for product in the FpML

As explained above, the use of both substitution groups and extension to replace product elements in the FpML schemas is confusing.

In my opinion even if we ignore the use of extension the use of substitution groups is confusing for developers. Most XSD constructs map to object oriented concepts. However, because we are using substitution groups here we are effectively saying that absolutely any type can be a member of our parent trade type at this point in the schema. It’s like including a generic object data member in a class that can be used to contain ANY other class. In OO terms we have no idea what interface this object will implement. It’s hard to represent this in UML as well: see http://www.xmlmodeling.com/documentation/specs/substitutionGroup. Finally, as we shall see in part 3, the Microsoft tool for creating classes from schemas (xsd.exe) struggles with the combination of extension and substitution groups used in the FpML schemas.

Swap

The Swap complex type is again fairly straightforward. Everything is optional except for at least one element called swapStream, of type InterestRateStream. Clearly these represent the swap legs for our interest rate swap, and consequently we have two of them in our sample instance XML. As mentioned above you’ve probably got the idea of how this all fits together by now, and how to investigate further, so we’ll leave the schemas at this point.

Conclusion

This article has attempted to show how validation using the FpML schemas works, and to give an idea of the structure of a trade XML document in FpML.

Part three of this series of articles will examine how we can generate .NET classes from these schemas using xsd.exe, and will look at some of the problems associated with doing this.

Licensing of FpML Specifications

The FpML Specifications of this document are subject to the FpML Public License (the “License”); you may not use the FpML Specifications except in compliance with the License. You may obtain a copy of the License at http://www.FpML.org.
The FpML Specifications distributed under the License are distributed on an “AS IS” basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See the License for the specific language governing rights and limitations under the License.
The Licensor of the FpML Specifications is the International Swaps and Derivatives Association, Inc. All Rights Reserved.

http://www.fpml.org/documents/license.html

December 29, 2007

Introduction to using Financial Products Markup Language (FpML) with Microsoft .NET Tools (Part 1)

Introduction

This short series of articles will attempt to show how Microsoft’s standard .NET tools (Visual Studio and xsd.exe) can be used to investigate and use Financial products Markup Language (FpML).

This first article gives an overview of how we can use Visual Studio to examine the FpML XSDs, and use it to validate the XML examples that are provided in the FpML download.

Part 2 of the series will examine how the FpML XSDs fit together for basic trade types.

These articles assume you understand the basics of XML Schema Definition language. If you don’t there are a number of good tutorials available on the internet. For example there’s quite a good one at http://www.w3.org/TR/xmlschema-0/. These articles will reference the tutorial for the less obvious XSD structures.

FpML

Financial products Markup Language is intended as an industry-standard method of representing financial products using XML. Details can be found at www.fpml.org.

The XML representations of financial products are defined using XML Schema Definition (XSD). These can be downloaded from the site.

FpML is Complex

The FpML XSDs are intended to define XML for a very wide range of financial products. They are also intended to be easily extensible for any new products. To achieve this products are built up from small building blocks into a complex structure. This means the XML is very hierarchical (it has many layers of tags).

An alternative approach would be to represent financial products in as flat a structure as possible. By ‘flat’ we mean a simple series of name-value pairs (with associated types for the values), and no hierarchy. Even quite complex derivatives products can be represented by a flat schema (obviously such things as basket products are an exception to this). A flat structure is easier to understand, and can be easier to handle in code. It also leads to less verbose XML, and hence smaller message sizes.

However, this is not the approach FpML takes for good reasons: whilst it’s easier to deal with a flat structure, it’s hard to extend as you cannot easily re-use blocks of tags in a new structure.

As well as defining formats for a wide range of trade types, the FpML XSDs define message formats for standard trade events (e.g. confirmations, credit event notifications). They also define standard enumerations for such things as day count conventions.

As a result all this FpML is inevitably complex and difficult to understand. Furthermore there’s little free documentation available.

Visual Studio Support for XSDs

Visual Studio (combined with C#) has good support for XML and XSD. As you may know, we can load XSDs into it and it will automatically check that they are internally consistent. We can then load XML into it and validate that XML against the XSDs on the fly at design-time.

As we shall show the .NET languages also have support for validating XML using XSDs at run-time.

In addition there is an ‘XML Schema Definition Tool’ called xsd.exe that ships with Visual Studio. We can use this to automatically generate classes in a .NET language of our choice based on the XSDs. We can then load the XML data into those classes (at run-time obviously). If we don’t like the classes it’s very easy to manually tweak them to do what we want. In fact, as we shall see, xsd.exe is far from perfect and we may well need to tweak the classes to get them to work properly.

We can even usually use xsd.exe to generate DataSets from the XSDs and populate those from the XML. Of course, DataSets give us some disadvantages (e.g. speed) over plain old classes, but they allow us to track changes, and easily bind our data into grids for display and editing.

This series of articles will take us through all of the scenarios outlined above. However, as we shall see, the complexity of the FpML schemas (and some Microsoft bugs) can make handling FpML in Visual Studio challenging.

Initially we will work with Visual Studio 2005 (and .NET 2.0). Later on we’ll have a look at changes in Visual Studio 2008.

Getting Started with FpML and Visual Studio

The first thing to do if you want to learn about FpML is to download the schemas. This can be done from www.fpml.org, although you will need to register with the site. For this article I have used the FpML 4.2 Recommendation in Zip format, which you can download via the ‘Specifications’ tab.

Once you have downloaded and extracted this, create a console application project in Visual Studio. Add ALL of the XSDs found in the xml folder from your download to this project (using the Project/Add Existing Item… menu).

XSD

We are going to start with interest rate swaps. The details of these are defined in file fpml-ird-4-2.xsd. Right-click this file in Solution Explorer and select View Code. DON’T just ‘Open’ it as the default is to open with a DataSet Editor, which gives a graphical view. Note that you can change the default if you right-click/Open With. If you select the XML Editor as the default you will get the code view when you double-click in future.

You should now have the raw XSD in the document window. It’s a bit daunting, so right-click in the document window, and select Outlining/Toggle All Outlining. This collapses the XSD entirely. If you now expand it once (by clicking the ‘+’ sign) you will see that the XSD includes some other schemas, then defines some complex types, then some elements and some groups. One of the elements is called ‘swap’, which is encouraging. This is of type ‘Swap’ which we can see defined as a complex type. However it’s not clear how we get from this to XML for an interest rate swap.

Note here that Visual Studio validated the schema when you displayed it. You can see this if you deliberately break the schema. Try changing the type of the swap element mentioned above to ‘notSwap’. If you now look at the Warnings in the Error List (menu View/Error List) you’ll see Visual Studio is telling you that ‘Type ‘http://www.fpml.org/2005/FpML-4-2:notSwap&#8217; is not declared.’ It’s also telling you the line number and the file, and you can as usual double-click the error to go to it.

notswap.jpg

Note also that Visual Studio only validates the schema when the schema file is open in the document window (try saving it and closing it with the error still in there: the warning disappears). Visual Studio will quite happily validate all of the FpML XSDs if you open them individually however.

XML

Now we will add an XML file representing an interest rate swap to our project, and get the XSDs to validate it. Our original download contains XML examples in the subfolders of the xml folder referred to above. In particular, there is a folder called interest-rate-derivatives that contains a file called ird_ex01_vanilla_swap.xml. This XML file references the XSD files we have already examined using the schemaLocation attribute. However the schemaLocation attribute says that the schemas are in a folder at a level above the XML file.

So create a folder in your solution (right-click the solution in Solution Explorer and select Add…/New Folder). Then right-click the folder, choose Add/Existing Item…, and add the ird_ex01_vanilla_swap.xml file into this folder.

Now if you open the file in Visual Studio it should validate correctly straight away: there should be no errors or warnings in the Error List. Again you can see this is working if you manually edit the XML: try changing the partyReference element near the top to ‘notpartyReference’. You will get a warning the notpartyReference is an invalid child element and ‘partyReference’ is expected.

notpartyreference.jpg

Visual Studio and XML: Navigation and Outlining

We have already seen that we can use Visual Studio’s outlining (collapsible regions) in the usual way with XSD files, and the same is true of XML files.

We can also use Visual Studio’s standard ‘Go To Definition’ functionality with both XSD and XML files. Put your cursor inside partyReference in the XML and hit F12 (or right-click, Go To Definition). Visual Studio will show the place in the XSD where partyReference is defined (which is inside file fpml-doc-4-2.xsd).

This shows that partyReference is an element in complex type TradeIdentifier. It is itself of type ‘PartyReference’. If you right-click ‘PartyReference’ and Go To Definition in the same way you can navigate within the XSD: you can see that PartyReference is an extension of base type ‘Reference’, and if you go to the definition of that you can see that it is an abstract complex type with one attribute called ‘href’ of type IDREF.

You can then navigate back through the items you’ve visited as usual in Visual Studio using the Navigate Backward toolbar button, or hitting Control + – (minus key).

It’s worth noting that the normal toolbar Comment/Uncomment buttons work fine with XML and XSD, including commenting sensibly across several lines: you don’t have to type ‘<!–‘ and ‘–>’. Also there’s good intellisense support if you are entering XML or XSD code, including code snippets on the context menu.

Finally if you load all of your XSDs into a project as described above you can (obviously) use Visual Studio’s normal Find in Files functionality (Ctrl-Shift-F) to locate items anywhere in the XSDs.

Validation in Code

We have seen how we can use Visual Studio to validate that a schema is correctly formed, and to validate an XML instance document against a schema. We can also do this validation in C# code.

Validation of a Schema in Code

To validate a schema we can load it into an XmlSchemaSet and compile it as below. Put this code in the Main method of the C# console application created above to see this working:

            XmlSchemaSet xmlSchemaSet = new XmlSchemaSet();
            xmlSchemaSet.Add("http://www.fpml.org/2005/FpML-4-2", @"..\..\fpml-main-4-2.xsd");
            xmlSchemaSet.Compile();

If the schemas are not properly formed you will get an error when you attempt to compile them.

Validation of XML Against a Schema in Code

To validate an XML instance document against the schema we use an XmlReader in association with a ValidationEventHandler as below. The code below goes in Main:

            XmlReaderSettings xmlReaderSettings = new XmlReaderSettings();
            xmlReaderSettings.Schemas.Add(xmlSchemaSet);
            xmlReaderSettings.ValidationType = ValidationType.Schema;
            xmlReaderSettings.ValidationEventHandler += new ValidationEventHandler(xmlReaderSettings_ValidationEventHandler);
 
            XmlReader xmlReader = XmlReader.Create(@"..\..\XML\ird_ex01_vanilla_swap.xml", xmlReaderSettings);
            while (xmlReader.Read()) { };
 
            Console.WriteLine("Validation complete");
 
            Console.ReadLine();

Additionally we need a ValidationEventHandler method: this will be called if there are any errors:

        static void xmlReaderSettings_ValidationEventHandler(object sender, ValidationEventArgs e)
        {
            if (e.Severity == XmlSeverityType.Warning)
            {
                Console.Write("WARNING: ");
                Console.WriteLine(e.Message);
            }
            else if (e.Severity == XmlSeverityType.Error)
            {
                Console.Write("ERROR: ");
                Console.WriteLine(e.Message);
            }
        }

The code for this example is available. To test these try deliberately breaking both the schemas and the XML and ensure you get appropriate error messages. Note that this code is based on Microsoft’s own help examples at http://msdn2.microsoft.com/en-us/library/3740e0b5(VS.80).aspx.

Note on Visual Studio 2008

The above discussion used Visual Studio 2005. Visual Studio 2008 behaves in the same way except:

  1. Visual Studio 2008 defaults to the XML Editor as the editor for XSD files, which is more sensible than the default DataSet Editor in Visual Studio 2005.
  2. Visual Studio 2008 automatically references xmlsig.xsd in Microsoft’s own schemas folder (normally c:\Program Files\Microsoft Visual Studio 9.0\xml\Schemas). xmlsig.xsd is the standard XML Signature namespace. However, the FpML files are referencing their own version of this schema, which is called xmldsig-core-schema.xsd. As a result if you open fpml-main-4-2.xsd in the designer, for instance, you get warnings about types in this schema being defined twice.To fix this bring up the Properties window for the fpml-main-4-2.xsd file, and click in the Schemas property. Click the button that appears. The screen that comes up lets you select the schemas that are automatically used for the file. Find xmlsig.xsd and in the ‘Use’ column set it to ‘Do not use this schema’ (which gives it a red cross as below). This will get rid of the warnings.

dontuseschema.jpg

Conclusion

This article has looked at some of Visual Studio’s functionality for handling XSDs and the associated FpML. Part 2 of this series will examine how the FpML fits together for a basic trade XML document.

Licensing of FpML Specifications

The FpML Specifications of this document are subject to the FpML Public License (the “License”); you may not use the FpML Specifications except in compliance with the License. You may obtain a copy of the License at http://www.FpML.org.
The FpML Specifications distributed under the License are distributed on an “AS IS” basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See the License for the specific language governing rights and limitations under the License.
The Licensor of the FpML Specifications is the International Swaps and Derivatives Association, Inc. All Rights Reserved.

http://www.fpml.org/documents/license.html

The Shocking Blue Green Theme. Blog at WordPress.com.

Follow

Get every new post delivered to your Inbox.

Join 80 other followers