View Driven Data Design; don’t let your view shape your data

Recently I have found myself coming across a couple of view to data scenarios where you can see that the view has driven the data model. This makes my spidey sense tingle as this strikes me as too tight coupling between data and view.

So what am I actually talking about? I’ve called it “View Driven Data Design”.

So an example to make things clearer; any developer who has been in the business long enough will have had the opportunity to work with a UI which involves category and sub category dropdowns / pickers. These values are usually driven by a data source (database, xml definition etc.) which are dynamic. What do I mean by dynamic? I’m referring to none hard coded values which require recompilation to be available. Either through an administration function, a nightly batch job updating the data or through a configuration file. These can have some cascading / restriction functionality built in to improve usability but are dynamic. These values can be stored by id – usually Foreign Keyed to the source – to allow for normalisation etc. This is fine for this scenario however there are times when this approach is overkill and can cause future issues and frustrations.

Imagine a scenario where you have an single data item (entity) which is modelled as a single class definition. An instance of this class requires classification. Unlike our earlier example the “category” is driven by a list of hard coded enum values. The UI designer has specified that on a specific value of category there will be another option to further categorise the entity. This is also driven by a distinct number of values in another enum. Further more there is a specific correlation between category and sub category options. So like before these values can be stored on two properties of the entity. What’s wrong with this I hear you ask? On the face of it nothing is wrong. It is however when you come to use these categorisations in other parts of the system when the problems creep in.

So you’ve created the instances in your CRUD section of the application and it’s all running fine however you now have another requirement to use them in a different form and you require to run some logic depending on classification – this is where the issue raises it’s ugly head.

When trying to classify an instance of the entity you now need to check different combinations of two properties. This involves the same business logic being put in, potentially, multiple places in the code base which is bad. If there are other values to check as well this adds unnecessary noise and complexity to the business logic. So how do we make the developers life easier and keep the UI designer happy at the same time?

It all starts at the data model level and answering “what value(s) do we actually have to store and how can this improve code clarity?”.

Start off by looking to see if the distinct list of defined combinations between the two enum sets can be merged into one set. If there are two many options then leaving them split will probably need to remain, however if the sub categories only apply to a small number of main categories then making one set may be on. This will need to be determined on an implementation case by case basis.

Right we have one property on the entity and the UI designer still wants their category and sub category dropdowns with visibility rules etc. This is where we need to loosely couple the data model away from the view. This is the exact reason why we have the concept of a view model. Using the entity data model we can define and populate a specific model to drive the view. In this case this will probably include like for like property mapping where nothing special needs to happen (eg. simple text values such as names) but for the single category enum it will need to get mapped to the two different options and can then drive the visibility rules etc. On the save action a conversion can map from the two options back into one ready to save.

This allows for the UI to be driven with the user experience the designer/analyst required while also making further business logic simpler and clearer to understand for the developer; both to write originally but also for future code maintenance.

So in the future when your designer wants fancy dropdowns and categorisations take a moment to ask does it need to be stored that way? Your future self will thank you instead of taking your name in vein.

Hello World with KnockoutJs–Code Restructure

Recently I have been reading a number of articles and watching training videos from Pluralsight about Javascript and KnockoutJs. After reading the “Hello world, with Knockout JS and ASP.NET MVC 4!” by Amar Nityananda I wanted to see if I could apply the techniques of what I’d learnt to a relatively simple example. After reading the original article I couldn’t help but think “this Js could be better structured, I think I could do this” … so this is what I came up with.

To get an understanding of the problem please read the original article first.

I’ve only concentrated on the client side JavaScript as this was the challenge …

// setup the namespace
var my = my || {};

$(function () {

    my.Customer = function () {
        var self = this;
        self.id = ko.observable();
        self.displayName = ko.observable();
        self.age = ko.observable();
        self.comments = ko.observable();
    };

    my.vm = function () {
        var 
            // the storing array
            customers = ko.observableArray([]),
            // the current editing customer
            customer = ko.observable(new my.Customer()),
            // visible flag
            hasCustomers = ko.observable(false),
           
            // get the customers from the server
            getCustomers = function() {
                // clear out the client side and reset the visibility
                this.customers([]);
                hasCustomers(false);
               
                // get the customers from the server
                $.getJSON("/api/customer/", function(data) {
                    $.each(data, function(key, val) {
                        var item = new my.Customer()
                            .id(val.id)
                            .displayName(val.displayName)
                            .age(val.age)
                            .comments(val.comments);
                        customers.push(item);
                    });
                    hasCustomers(customers().length > 0);
                });
            },
            // add the customer from the entry boxes
            addCustomer = function() {
                $.ajax({
                    url: "/api/customer/",
                    type: 'post',
                    data: ko.toJSON(customer()),
                    contentType: 'application/json',
                    success: function (result) {
     // add it to the client side listing, this will add the id to the record;
     // not great for multi user systems as one client may miss an updated record
     // from another user
                        customers.push(result);
                        customer(new my.Customer());
                    }
                });
            };

        return {
            customers: customers,
            customer: customer,
            getCustomers: getCustomers,
            addCustomer: addCustomer,
            hasCustomers: hasCustomers
        };
    } ();

    ko.applyBindings(my.vm);
});

The main points I was trying to cover are:

• Code organisation through namespaces – I don’t want any potential naming conflicts. This isn’t a big issue in a small application but once they start growing it’s good to try and avoid putting definitions into the global namespace.
• Expose the details of the view model through the Revealing Module pattern – this keeps details clean as well as improved data binding clarity.

I haven’t gone into too much detail as the rest of the project infrastructure is very similar to the original article and can be found there. I wanted to post this as an example of how to make the Js structured.

Happy to discuss further; contactable in the comments or via twitter.

Updated Edit

After a comment from ranga I have read the link posted and update the code (see below). I have moved the view model and the Customer declaration outside of the on document ready event. I have also change hasCustomers to be a computed value. The interesting read about the performance hit about the events firing on array manipulation was really interesting – worth a read.

// setup the namespace
var my = my || {};

my.Customer = function () {
    var self = this;
    self.id = ko.observable();
    self.displayName = ko.observable();
    self.age = ko.observable();
    self.comments = ko.observable();
};

my.vm = function () {
    var
    // the storing array
    customers = ko.observableArray([]),
    // the current editing customer
    customer = ko.observable(new my.Customer()),
    // visible flag
    hasCustomers = ko.computed(function () {
        return customers().length > 0;
    }),

    // get the customers from the server
    getCustomers = function () {
        // clear out the client side
        this.customers([]);

        // get the customers from the server
        $.getJSON("/api/customer/", function (data) {
            var items = new Array();
            $.each(data, function (key, val) {
                var item = new my.Customer()
                    .id(val.id)
                    .displayName(val.displayName)
                    .age(val.age)
                    .comments(val.comments);
                items.push(item);
            });
            customers(items);
        });
    },

    // add the customer from the entry boxes
    addCustomer = function () {
        $.ajax({
            url: "/api/customer/",
            type: 'post',
            data: ko.toJSON(customer()),
            contentType: 'application/json',
            success: function (result) {
                // add it to the client side listing, this will add the id to the record;
                // not great for multi user systems as one client may miss an updated record
                // from another user
                customers.push(result);
                customer(new my.Customer());
            }
        });
    };

    return {
        customers: customers,
        customer: customer,
        getCustomers: getCustomers,
        addCustomer: addCustomer,
        hasCustomers: hasCustomers
    };
} ();

$(function () {
    ko.applyBindings(my.vm);
});

Keep the feedback coming, I’m always up for learning and improving.

Software Bucket List–Nov 2012

After reading Rob Maher’s blog post about the concept of software bucket list it got me thinking.

Let’s take a step back for a minute; what’s a bucket list? A bucket list is a list of all the things you want to do before you pass away. So a software bucket list for a developer is a list of software you’d ideally like to write before you retire. As Rob says it can be anything, not learn this or learn that etc. but what would you like to do?

So this brings me back to me, his blog post got me thinking. The whole idea of the list is that it will evolve over time but as of November 2012 this is my list:

• Integrated disc golf membership system with result submission with associated automated renewals etc.
• Work for and write software for a Formula 1 team
• Multi player strategy re-write (modernise) of an old game we use to play in the early 00’s – Europa
• Build <insert description here> to enable working from home for my own company – idea still hasn’t been thought of yet.

The last one doesn’t quite fit but it’s more a target to keep me inspired about moving forward.

Anyway, this is just a start and now it’s a seed that’s planted I’m sure I’ll come up with more but for now what’s yours?

Debugging WCF messages before serialization

Over the past couple of weeks I’ve been doing a lot of work with WCF to do with the messages being sent between client and server and securely signing them. Signing the message isn’t the subject of this post however developing the following technique to see the message before it’s sent did lead me onto the solution for debugging messages being sent between client and server.

This post is to show how you can intercept the message, update it once its been serialized into xml (or just look at the xml) but before it’s been sent to the server. The basis of this all down the the IClientMessageInspector and adding this to your client calling code.

Let’s take a look at the message inspector and then I’ll talk through the additional classes which go with the inspector and then finally I’ll show you how to use the message inspector with a WCF client proxy generated class; in both code and through configuration.

IClientMessageInspector

public class DebugMessageInspector : IClientMessageInspector
{
    public object BeforeSendRequest(ref Message request, IClientChannel channel)
    {
    }

    public void AfterReceiveReply(ref Message reply, object correlationState)
    {
    }
}

Above is the generated method signatures of which the IClientMessageInspector expects to be implemented. The naming of the methods describes exactly the point in which they are called; BeforeSendRequest is as the request is sent to the server, AfterReceiveReply is as the response is returned. On the way out this is executed after the message has been created from the strongly typed objects and on the way in it executes before any serialization is performed if using strongly typed proxy and class definitions generated from a WSDL.

Inside each of the methods you can call the following code to allow inspection of the message.

private Message Intercept(Message message)
{
    // read the message into an XmlDocument as then you can work with the contents
    // Message is a forward reading class only so once read that's it.
    MemoryStream ms = new MemoryStream();
    XmlWriter writer = XmlWriter.Create(ms);
    message.WriteMessage(writer);
    writer.Flush();
    ms.Position = 0;
    XmlDocument xmlDoc = new XmlDocument();
    xmlDoc.PreserveWhitespace = true;
    xmlDoc.Load(ms);

    // read the contents of the message here and update as required; eg sign the message

    // as the message is forward reading then we need to recreate it before moving on
    ms = new MemoryStream();
    xmlDoc.Save(ms);
    ms.Position = 0;
    XmlReader reader = XmlReader.Create(ms);
    Message newMessage = Message.CreateMessage(reader, int.MaxValue, message.Version);
    newMessage.Properties.CopyProperties(message.Properties);
    message = newMessage;
    return message;
}

The above is a very simple way of reading the message into a raw xml format and then converting it back into a Message for further processing from the framework.

So how do you hook the message inspector to the client proxy call?

You need an IEndPointBehavior.

IEndpointBehavior definition

The new end point behavior allows you to add, amongst other things, message inspectors to the current ClientRuntime of the client definition using the behavior.

Implementing the interface gives you the following structure:

public class DebugMessageBehavior : IEndpointBehavior
{
    public void AddBindingParameters(ServiceEndpoint endpoint, BindingParameterCollection bindingParameters)
    {
    }

    public void ApplyClientBehavior(ServiceEndpoint endpoint, ClientRuntime clientRuntime)
    {
        // add the inspector to the client runtime
        clientRuntime.MessageInspectors.Add(new DebugMessageInspector());
    }

    public void ApplyDispatchBehavior(ServiceEndpoint endpoint, EndpointDispatcher endpointDispatcher)
    {
    }

    public void Validate(ServiceEndpoint endpoint)
    {
    }
}

As you can see a number of extension hook points are available, but for this post we’ll concentrate on the “ApplyClientBehavior” method. In this method it gives you access to the MessageInspectors collection on the ClientRuntime which is a SynchronizedCollection on which any IClientMessageInspector implemented class instances can be added to.

This is it for the behavior definition. Lets see how we use it with a WCF client proxy class.

How do we use the IClientMessageInspector?

There are two ways in which the inspector can be used with a client proxy class. It can either be added in via code or through configuration. There are pros and cons for both routes. If done through code it will always be added, however a con is it will always be added. If however it is done through configuration it can be enabled/disabled without recompiling the code. This can be especially  handy if the functionality is only for debugging and you don’t want it to be used during Production or it’s used to turn on/off certain functionality which is only applicable under certain circumstances.

Let’s take a look at using the message inspector first with code and then through configuration.

Using the IClientMessageInspector in code

Through code (example code can be found on GitHub) you add it to the Endpoint Behavior collection directly. This can be done either straight after the proxy definition or at any point before any method is actually executed.

var client = new ServiceReference1.Service1Client();            client.Endpoint.Behaviors.Add(new DebugMessageBehavior());

As you can see from the above it is instantiated calling a default constructor. At this point you can pass in constructor parameters however this will cause you issues if you want to move it to be bound in configuration later.

And that’s it. Straight forward; simple.

Using the IClientMessageInspector through configuration

Adding the behavior through configuration takes a bit more work but as stated earlier being able to attach the functionality purely through configuration without changing code is worth the extra effort. To attach through configuration we need to define a new BehaviorExtensionElement which describes our new behavior.

public class DebugMessageBehaviourElement : BehaviorExtensionElement
{
    protected override object CreateBehavior()
    {
        return new DebugMessageBehavior();
    }

    public override Type BehaviorType
    {
        get
        {
            return typeof(DebugMessageBehavior);
        }
    }
}

Once this has been created and the solution built the binding configuration needs to updated as below:

  <behaviors>
    <endpointBehaviors>
      <behavior name="debugInspectorBehavior">
        <debugInspector/>
      </behavior>
    </endpointBehaviors>
  </behaviors>
  <client>
    <endpoint address="http://localhost:60428/Service1.svc" binding="basicHttpBinding"
        bindingConfiguration="BasicHttpBinding_IService1" contract="ServiceReference1.IService1"
        name="BasicHttpBinding_IService1" behaviorConfiguration="debugInspectorBehavior" />
  </client>
  <extensions>
    <behaviorExtensions>
      <add name="debugInspector" type="Client.DebugMessageBehaviourElement, Client"/>
    </behaviorExtensions>
  </extensions>

This is all in the system.serviceModel node in your web/app config file. The important parts are highlighted in bold.

First off you have to define a behavior extension pointing at your newly created behavior element class. This is a reflection string so the fully qualified type name needs to be followed by the name of the assembly that contains it.

Secondly you create an endpoint behavior which now uses the newly created extension. At this point the intellisense in Visual Studio will complain however it’s fine.

Third and finally you specify the newly created endpoint behavior as the behaviorConfiguration for your client end point.

Result

Either way, on the way out you can see the whole message is:

<?xml version="1.0" encoding="utf-8"?>
<s:Envelope xmlns:s="http://schemas.xmlsoap.org/soap/envelope/">
    <s:Header>
        <Action s:mustUnderstand="1" xmlns="http://schemas.microsoft.com/ws/2005/05/addressing/none">http://tempuri.org/IService1/GetData</Action>
    </s:Header>
    <s:Body>
        <GetData xmlns="http://tempuri.org/">
            <value>7</value>
        </GetData>
    </s:Body>
</s:Envelope>

And on the way back:

<?xml version="1.0" encoding="utf-8"?>
<s:Envelope xmlns:s="http://schemas.xmlsoap.org/soap/envelope/">
    <s:Body>
        <GetDataResponse xmlns="http://tempuri.org/">
            <GetDataResult>You entered: 7</GetDataResult>
        </GetDataResponse>
    </s:Body>
</s:Envelope>

Conclusion

Message inspectors are a very powerful tool when working with WCF and integration services. They can be used for debugging purposes as shown or for performing other actions on the message before it gets sent or on its return such as securely signing the message with a certificate.

The demo code I’ve used for this post can be found on Github; but remember it comes with a “Works on my machine” seal so use at own risk.

Opening Pluralsight Single Page Application course code in VS2010

John Papa published a very good course on Pluralsight the other day about how to create a Single Page Application (SPA) and due to the popularity of the course Pluralsight opened it up for a short time only for free (thanks!). This allowed me to see exactly the type of quality I would receive when my training budget at work comes through soon (fingers crossed for October).

After watching some of the videos it was really interesting but I wasn’t quite getting how it all fitted together in the solution so was very pleased to find that the free sign up included being able to download the course material including the final solution. After downloading the zip file, expanding the contents and locating the .sln file I double clicked it and waited for Visual Studio 2010 to fire it up.

On Visual Studio 2010 opening the solution I was presented with a very sad looking solution explorer.

This is due to the fact that the course was written in Visual Studio 2012 and hence .net 4.5 was used for the projects. This got me thinking that it shouldn’t cause a problem as all the technologies which were being used are also available for .net 4 so I got about trying to get it to work.

Step 1; getting the projects to load

This in itself wasn’t that tricky. Firing up the .csproj files in notepad and locating the TargetFrameworkVersion I changed it to “v4.0”, saved and repeated it for all of the projects in the solution.

<TargetFrameworkVersion>v4.5</TargetFrameworkVersion>

To

<TargetFrameworkVersion>v4.0</TargetFrameworkVersion>

Once all the projects had been updated I reopened the solution in Visual Studio 2010 and got a prompt about configuring IISExpress. CodeCamper is configured to use IISExpress however if you don’t have it installed you will need to install it or configure the solution to work on a local IIS instance.

On clicking “Yes” Visual Studio attempted to load up the projects and Resharper did it’s usual solution scan and I was presented with a much happier looking solution explorer.

Step2; Lets try and build

On building the solution I got the following warnings however it did build.

warning CS1684: Reference to type 'System.Data.Spatial.DbGeometry' claims it is defined in 'c:\Program Files (x86)\Reference Assemblies\Microsoft\Framework\.NETFramework\v4.0\System.Data.Entity.dll', but it could not be found
warning CS1684: Reference to type 'System.Data.Spatial.DbGeography' claims it is defined in 'c:\Program Files (x86)\Reference Assemblies\Microsoft\Framework\.NETFramework\v4.0\System.Data.Entity.dll', but it could not be found

After a “successful” build I fired up the IISExpress site and went to start browsing the site to start seeing how it fitted together. However this was not the case and I got the following error.

Step3; changing 4.5 to 4

The changing of required framework version had not finished at the csproj level so I went through the web.config of the asp.net mvc project and changed the targetFramework attribute to “4.0”.

I also removed targetframework=4.5 from the httpruntime element below. Removed the machine key below that and hit refresh.

By now I was pretty sure that dependencies and reference errors are likely to be due to the framework version. With this I did a little Google and found that the version number had been bumped up to 4 for System.Net.Http in .net 4.5 however for .net 4.0 this needs to continue to reference version 2. With this in mine I updated the web.config from:

<dependentAssembly>
  <assemblyIdentity name="System.Net.Http" publicKeyToken="b03f5f7f11d50a3a" culture="neutral" />
  <bindingRedirect oldVersion="0.0.0.0-4.0.0.0" newVersion="4.0.0.0" />
</dependentAssembly>

to

<dependentAssembly>
  <assemblyIdentity name="System.Net.Http" publicKeyToken="b03f5f7f11d50a3a" culture="neutral" />
  <bindingRedirect oldVersion="0.0.0.0-4.0.0.0" newVersion="2.0.0.0" />
</dependentAssembly>

and hit refresh.

At this point I decided I would ignore the AntiXss encoder reference for now and come back to it later. I’m not going to publish the site anywhere so not worried about cross site scripting. However if you are going to put a site up into the wild then you will need to stop this kind of attack. Discussion for this is out of scope for this post.

I hit refresh and was presented with the Code Camper blue screen as below.

However no data.

step 4; getting data

At this point I thought I’m pretty close and pretty sure this will have something to do with the connection string. So after checking the connection strings configured in the web.config and noticing which one was currently active I checked to make sure the CodeCamper.sdf was present and ran.

Nothing; so changed the CodeCamperDatabaseInitializer below to always drop and recreate the DB to make sure the DB was fresh with the seeded data (remember to remove this and put it back to one of the other options otherwise it will delete everything all the time).

public class CodeCamperDatabaseInitializer :
    //CreateDatabaseIfNotExists<CodeCamperDbContext> // when model is stable
    //DropCreateDatabaseIfModelChanges<CodeCamperDbContext> // when iterating
    DropCreateDatabaseAlways<CodeCamperDbContext>

Nothing. At this point I decided to remove the sdf file and update the connection string to use to be the SQLExpress one. I knew I had that installed and running so would be perfectly fine for study purposes.

Time to try running it again to see which error would occur next. The next error was inside the NInject Dependency Scope implementation, in the NinjectDependencyScope.GetService(Type serviceType) method …

… with the wonderfully descriptive error message of

“Method not found: ‘System.Delegate System.Reflection.MethodInfo.CreateDelegate(System.Type)”.

This was due to a version miss match of Ninject; uninstalling the Ninject nuget package and reinstalling it to get the correct version down seemed to resolve this.

Another rebuild and run. This time the EntityFramework was not happy with the current build. After the last issue with Ninject I thought I’d try the same routine so I uninstalled the EntityFramework nuget package and reinstalled it. It seemed to be originally wanting .net 4.5 but it’s happy with 4. This needed to be done in both the CodeCamper.Data and CodeCamper.Web projects.

At this point I hit f5 and crossed my fingers. I was greeted with a screen full of green friendly messages about getting data from the asp.net web api and a smiling photo of the mighty ScottGu; finally I’d got it working!

Conclusion

There is one out standing issue which was not resolved and that is around the AntiXss functionality but the rest seems to work as expected.

Thanks again to John Papa and Pluralsight for creating and publishing this course and I can’t wait to get my full subscription to start enjoying the other courses.