TDD and the Red-Green-Refactor approach

We’re all familiar with the term test-driven development now, but it seems a lot of developers still aren’t using it in their normal daily routine. TDD has been in circulation since 2003 or earlier, and has probably been mainstream since 2005. But I’ve worked at companies and spoken to developers at other firms where they admit that they still don’t really follow the test-first approach (and was guilty of it myself when I first started out with unit-testing).

Test-driven (or test-first) development is a whole new mindset and a discipline which you need to practice quite a lot before it becomes second-nature. At first it seems cumbersome and slow, especially when time is precious and the clock is ticking. So the temptation is that you write the implementation first and then write your tests afterwards and make sure they pass. No one will ever know, right? And surely it took the same amount of time as if you’d written it test first?  Or possibly even quicker because you’re already familiar with the code that’s being tested?  Well apart from breaking every rule in the TDD handbook, maybe you’d be surprised to find that it probably took longer than if you’d written it test first.

To explain, here’s a summary of TDD red-green-refactor development (by the way if you’re not from a testing background, red and green refer to a test framework GUI runner such as NUnit where tests that pass are displayed green and tests that fail are shown in red).

1. Red: Before you write a single line of code, you need to be familiar with the requirement you are about to satisfy. If not, there’s no point starting – how will you ever get the desired end result?  The days of hacking out code in the vain hope that by the laws of probability, you might eventually get it right are well and truly over. But in an ideal world you will hopefully have a user story you are familiar with and a particular task you’re supposed to be working on. So once you have a clear idea of what it is you are trying to achieve, write your test first and run it to see that it does actually go red (you may need to stub out the class and method under test before it will compile, but this is OK). You have now achieved two things: you have an automated test which will ensure the integrity of your software over time, and it also describes the client requirement. As such, it documents the expected behaviour of the software which will be invaluable for  maintenance over the coming weeks/months/years of the software’s lifetime. By writing the test, it proves (assuming you’ve got it right of course) that you understand the business requirement.  And if not, it proves that you didn’t understand it!

2. Green: Now you have a test in place, you can write the code to make the test pass but only write enough code just to make it pass and nothing more. Time = money and no client has ever been happy when software is delivered late. If the test passes, then that requirement has been satisfied and you can move onto the next task (or next user story). Don’t waste time messing around with code if your tests are all green. In the long run this tactic will save you a lot of time and is key to working in an Agile manner.

3. Refactor: Finally (and only if you are running ahead of schedule and have time spare) you can refactor code where the tests are already green. And then – most importantly – re-run your tests to prove you haven’t broken anything. But remember, you’re not doing this for the client’s benefit. You are doing it for your benefit but at the client’s expense. If you go back to a client and tell them that you’re behind schedule because you spent two days refactoring code that was already working, they’ll probably fire you on the spot! OK, maybe not anything quite that drastic but you certainly won’t make yourself very popular.

As developers we always need to justify how our precious (and expensive) time has been spent, so spend it wisely. We are all providing a service to our clients and this means that doing what’s best for the client takes priority over what’s best for us. And delivering working software on time is what they really want, not software that’s beautifully refactored but cost twice as much to build (and ultimately still does the same job it did before the refactor). I’m not suggesting that you start writing unmanageable code which just happens to make your tests pass. But you need to find the right balance between correctly functioning software which may not be very pretty but gets the job done, and correctly functioning software which has had hours of refactoring work spent on it while other areas of the system remain unfinished or have defects. So hopefully you will start writing your tests first and keep your clients happy.

Posted in Agile, TDD

Generating WCF service proxy classes using SvcUtil

There are various ways to make calls to a WCF service from client code but this particular example uses the Microsoft svcutil.exe proxy generation tool. This is most appropriate when you want to consume a third party service and don’t have access to the service contract type declarations, but you can obtain a service metadata WSDL document from the service. To check if service metadata exchange (mex) is enabled, browse to the service and put a ?wsdl query string parameter at the end of the URL path. If it returns a WSDL XML document in the response, then metadata is enabled and you can use SvcUtil.

However, if you do have direct access to the service contract declarations (e.g. via a shared assembly) then I recommend using the WCF ChannelFactory<T> class instead of using svcutil.exe because it makes life a lot easier (see my post on how to use ChannelFactory to create re-usable proxy classes).

The following example is based on a WCF web application project using HTTP binding, but it would work just as well with a self-hosted service using another transport such as TCP.

Example service
I’ll start with a simple WCF service contract with a single operation called RegisterUser. The ServiceKnownType attribute is used to tell the WCF data contract serialiser which concrete type (in this case UserRegistration) implements the IUserRegistration interface. I haven’t shown the service implementation here, but you can just assume that it exists. Generating service proxies is all about the service contract (interface) – not the service implementation.

The UserRegistration type is a simple DTO (data transfer object) used to transfer data between a client application and the service. The DataContract and DataMember attributes which decorate the class enable the WCF data contract serialiser to convert it to and from XML.

Generating the service proxy
So now we have a service and data contract class but how can the client access them? One option is to use the .NET service model metadata tool svcutil which will generate the necessary proxy code and service configuration for us. Among the many command line switches that are available for svcutil, you can specify a namespace for the auto-generated proxy types. This is a great help in differentiating between types which are defined in the service layer when you are using them in client code. One important point about svcutil is that it should be run using the Visual Studio command prompt. Your service also needs to be hosted on IIS, IIS Express (Microsoft WebMatrix) or Cassini (Visual Studio development web server) before you can generate proxies. To ensure the service is hosted properly, try browsing to the service URL before attempting to generate the proxy.

Below is an example svcutil command line. I recommend using a fixed port number for WCF web application service host projects during development, so that you can add svcutil command lines to a batch file which can be re-run each time a service contract changes. To set a fixed port, go to service host project Properties > Web > Specific port, and enter a port number of your choice which obviously doesn’t conflict with another port in use in your solution (or within the system you are working on).

svcutil.exe http://<service url>?wsdl /t:code /n:*,Example.Client.ServiceProxies

Note the command line specifies a namespace Example.Client.ServiceProxies via the /n switch. There are additional command line switches to specify the output directory path, config file name, etc, but I have tried to keep this example as simple as possible for now. I recommend you look at Microsoft’s documentation for svcutil on the MSDN website and try out the other command line switches which are available. However the ones I’ve found most useful are:

/n:*,<namespace>  – Namespace which will apply to all generated code (and also appears in the service contract config). If you don’t specify a namespace on the command line, all generated code will belong to the global namespace which trust me, will come back and bite you when your client types and service types all become indistinguishable. Especially in your integration test projects where you will end up in project reference and type name conflict hell. The * parameter means all types will be included under the same namespace.

/d:<directory>  – Destination file path for the generated output. This is really useful for outputting the proxy code and config files to a specific location in your solution folder structure (e.g. a folder in your client project). However if several projects in a solution need the same proxy files such as a client app and a test project – which is a common situation – then direct the output to a shared folder and add the files to different projects as a link (Add > Existing Item > Add As Link).  This will prevent multiple copies of the same file being scattered throughout your solution.

/config:<config file name>  – Specific name for the generated .config file. By default it’s called output.config but obviously if you are generating proxies for more than one service in the same location, the output.config files will overwrite each other which is not very helpful. If I want separate config files, I usually specify the same name as the service, e.g. AccountService.config.

/mergeConfig  –  Rather than having a separate config file for each service, you might want all config output to a single file. This is what the /mergeConfig switch does. You can still use this together with the /config switch to specify the name of a single config ouput file for all services.  Or you can specify the name of an existing config file you want the WCF configuration to be written to (e.g. the Web.config of a web application).

Automating proxy generation
Every time a service contract or data contract interface changes, you will need to regenerate your proxy files. This can get pretty boring after a while and you don’t want to keep typing out command lines by hand because it’s slow and error-prone. So create a batch file containing the svcutil command line for each service (either all in the same batch file or separately – it’s not important) and add to your solution. I usually create a solution folder called Scripts and add all my batch files to that. Simple automation like this will save a lot of wasted time and mistakes in the long run and it also means the batch file can be added to your source version control system which is important. Every time you find yourself typing out the same thing again and again, automate it and get it under source control.

Adding svcutil output to your client project
When you run the svcutil command, it generates two files. The first is a code (.cs) file containing proxy classes for the service and any types decorated with the [DataContract] attribute which will be automatically serialised. The second is a config file containing the WCF service endpoint configuration you will need to communicate with your service. Simply add the code file to your client project in the same way you’d add any other class  file. I usually add a new folder to my client projects called something like ‘ServiceProxies’ and add the autogenerated proxy code files to this folder to stop them cluttering up the project (if your client app is using several services, then you will have separate proxy files for each one so keeping them grouped together in folder like this is a good idea).

Be aware that whenever you add a WCF service proxy file to your client project for the first time, you will also need to reference the System.Runtime.Serialization and System.ServiceModel assemblies before the client project will compile. I think the names of the assemblies make it pretty self-explanatory what their purpose is. System.ServiceModel is the core service library containing WCF types and System.Runtime.Serialization is required for data contract serialisation and deserialisation over the wire.

Next, you need to add the auto-generated service configuration to your client’s app.config or web.config file. If the client project is not a web application and doesn’t currently have a config file, then simply add an application configuration file to it. Regardless of whether it’s an app.config or web.config, the process is still the same. Copy the entire <system.serviceModel> section from the auto-generated service config and insert it into the <configuration> section of your client’s .config file. It doesn’t matter where it goes but I usually keep all my service config below existing configuration sections to stop it turning into an unreadable mess (which XML config can do very quickly).

Calling the service
The auto-generated proxy code and service configuration will both use the namespace you specified on the svcutil command line. So in this example, the UserRegistration data contract class will appear as Example.Client.ServiceProxies.UserRegistration in the client code even though its real namespace in the service layer is Example.Services.DTO.UserRegistration. This prevents any confusion, making it crystal clear to everyone that this type is a proxy of a type actually defined in the service layer.

Now to call the service from the client app, simply use the auto-generated service proxy class which will have the same name as your service but with a ‘Client’ suffix. So in this case it’s called AccountServiceClient.

When used in combination with the service configuration (which contains the address and binding responsible for creating the channel stack) the proxy will automatically marshall the call across to the hosted WCF service. As you can see, once the proxy classes and configuration are in place, the client code needed to call a service is very trivial.  This is a far cry from the difficulties of older messaging technologies and shows just how much WCF has made things easier from a developer’s perspective.

I describe how to solve compiler errors when sharing data contracts between multiple services in this separate post.

Posted in Automation, C#, WCF

Keeping domain models and view models separate with ASP.NET MVC and WCF

The location, role and responsibility of objects within a software system is a common topic in the forums with plenty of disagreement about what is and isn’t correct. But first, let’s just define some commonly found objects and their roles:

Domain repository entity: Object which is an in-memory representation of a record persisted in some sort of backing data store. In most business scenarios, a repository entity will be mapped to a physical database table via an ORM tool such as NHibernate or the Microsoft Entity Framework. 

Domain model entity: An object which lives in the service layer and typically contains both logic and/or data. This is where your business rules should be defined. Often, a domain model entity and a domain repository entity will be the same thing. In other words, an object which contains business logic and is also mapped to a data store (e.g. an Order domain entity may be mapped to an Order database table, but also contains business logic to calculate the total cost of the order, including tax). This arrangement allows data to be processed by business code and persisted via the same object.

Data contract: This is a DTO (data transfer object) which is decorated with the WCF DataContract attribute. This is essentially a ‘dumb’ object containing just data and no logic, and is used to pass data across service boundaries via serialisation. Data contracts will usually by mapped to domain models to copy data from one to the other.

View model: Another type of DTO but limited to just the MVC UI layer. View model objects are used to pass data between MVC controller action methods and the views which display and capture model data.

Passing data between services and the web UI
So with these definitions in mind, it is OK (and necessary) for data contracts which are defined in your WCF service layer to be passed to and from your MVC web application. However for this to be possible, the MVC web app must have access to the data contract type declarations. The way you achieve this is to always have your WCF service implementation and WCF service & data contracts in separate assemblies. Then from your MVC web app, you can reference the project/assembly containing the service & data contracts (but do not reference the service implementation assembly).  Now your MVC web app can happily use data contracts defined in the service layer, and will still compile.

However it’s not OK for domain entities to be passed across the same service boundaries because then you blur the line between what should be a simple data transfer object and something containing logic which should never be directly exposed to your MVC views and controllers. Ignore this at your peril otherwise you will end up with business logic contaminating your UI layer which will lead to no end of problems in the future and result in a lot of refactoring and maintenance.

But how do you keep them apart?  Well from a coding point of view, one approach is that only classes that have absolutely no business logic should be decorated with DataContract and DataMember attributes. This prevents serialisation of domain entities meaning that they can’t accidentally or intentionally be used as a parameter in service contract. Another more basic check is to make sure that the web UI project never references an assembly containing domain entities (sounds obvious but I’ve seen it happen). This will keep them safely isolated from the UI. However from a physical point of view, the simple answer is that you can’t absolutely guarantee this won’t happen. It only takes one developer to unwittingly do the wrong thing and the rot starts to set in.

So if you can’t physically keep them apart, what can you do to contain the problem?  Well you have to rely on some fundamental development techniques which have been around for years: discipline, team communication and code reviews. OK I’m not going to win any awards for innovation here, but building a reliable system is more than just writing a lot of tests and then assuming everything’s OK. You have to enforce design rules and best practices which the development team sticks to, and the appropriate use of domain entities, view models, data contracts, etc. is all part of that.

Passing data within the UI
Any application with an MVC web client and WCF services will reach a point where types defined in the service layer and types defined in the web UI meet, and in most cases that will happen in your MVC controllers. But for most scenarios, that’s the only time they will share the same space. It will also help if you give data contracts and view models different names. View model names typically should reflect the view they relate to, and I usually add a ‘Model’ suffix to them for clarity (but it’s down to personal preference how you do it). However if you are using the SvcUtil tool to generate service proxies, I recommend you specify a namespace so that it’s clear which models are defined in the service layer (see this post for generating service proxies).

In a typical case where you need to get data from a view model object into a data contract object so that it can be sent to a service, all you have to do is map the properties between the two via an object initialiser, in a constructor, or using a mapping tool such as AutoMapper, although AutoMapper can be quite hungy on memory resources so be aware of this when you decide what to use. Writing your own mapping code is trivial so why use anything else?

There’s a bone of contention about whether data contracts defined in the service layer should be used as view models. Personally I don’t have a problem with it and actually prefer to have my view model types declared as data contracts in the service layer rather than in the MVC app. This is so I only have to define data annotation validation rules once rather than defining them against data contracts in the service layer, and then again for  view models in the MVC application. This makes unit/integration testing easier and reduces the chance of a property not being validated properly. As a result, data contracts declared in the service layer are passed directly into my views (so there are very few actual view models declared in the MVC app).

However you may not like this and prefer to have separate, dedicated view models because it avoids the situation where views end up being strongly-typed to classes defined in a different layer of your architecture. How you do it is entirely down to person choice.

Posted in Architecture, ASP.NET MVC, WCF

NuGet update fails in VS2010

This is an annoying issue I discovered today. I was trying to download a package but it failed because it wasn’t compatible with my current version of NuGet. So I tried to update the NuGet Package Manager but that also failed. To make things worse still, the NuGet Package Manager uninstall button was disabled so now I was at a complete dead end.

The solution is to run VS2010 as Administrator (right-click the menu item and select Run as administrator)

Go to Tools > Extension Manager, and NuGet Package Manager uninstall button should now be enabled.

Uninstall NuGet and restart VS2010.  Then from the Online Gallery, install the latest version of NuGet Package Manager.  This time it should be successful.

Posted in VS2010 | Leave a comment

Unit-testing Asynchronous Events

Once in the realm of asynchronous calls, we have no guarantee of which thread or precisely when an event handler might be called, which makes testing asynchronous events that much more difficult. As a result it makes the test shown below (taken from the previous post unit-testing synchronous events) completely redundant because we rely on the assertion being called after (and only after) the event has fired, which this does not do.

In a synchronous event situation, this test works because the thread which calls basket.Add is the same thread which executes the lambda expression, therefore it doesn’t return from the Add method until after local variable eventFired is set to true. However there are no such guarantees with an asynchronous event.

Because an asynchronous event handler will execute on a different thread to the one which initiated the call (that caused the event to fire) we need some mechanism to stop the original thread from performing any assertions until after the event has fired. There is more than one way to achieve this, but for this example the mechanism comes in the form of an EventWaitHandle which allows one thread to ‘signal’ another thread that something has happened. In this particular case I’m using the EventWaitHandle sub-class ManualResetEvent, which as its name suggests remains signalled until it is manually set back to non-signalled.

The example below tests a class called BackgroundTask (not shown) which performs an operation asynchronously on a separate worker thread and then fires the TaskComplete event once the task has finished. Because it’s asynchronous, the call to task.DoCalculationAsync returns almost immediately which is the issue we need to solve.

Rather than the lambda expression setting a boolean value, this time it sets the ManualResetEvent to signalled via a call to its Set method. Meanwhile the original thread which called task.DoCalculationAsync will block on the call to WaitOne until (a) the wait handle becomes signalled, or (b) the 30 second timeout expires. If the wait handle is signalled within the 30 second time limit, then eventFired will be set to true but if the timeout expires, it will be set to false and the test will fail.

Timeouts and continuous integration
The previous section raises the obvious issue of what is an acceptable timeout limit in your tests. I included an arbitrary timeout value in the above example just to show the possible options, but if you are testing a background task which might vary in how long it takes to complete, then use a larger value (e.g. 5 minutes) or don’t specify one at all which means it will never timeout. However just bear in mind that this might cause big delays or a complete stand-still on your CI server, so having a ‘fail-safe’ timeout is probably a good idea. The time it takes for tests to complete on your CI environment will vary due to the workload on the server and this should be taken into consideration. A test on your development machine might take twice as long to run on the CI because there might be several build agents running on the same box / virtual.

Posted in C#, Multi-threading, NUnit, TDD | Leave a comment

Unit-testing Synchronous Events

It’s easy to forget (or choose not to) test events when you’re writing test-driven code, but they mustn’t be ignored because client code which subscribes to your events will depend on them firing under the correct circumstances. And without tests to prove this happens, there is no guarantee that they are doing their job properly, which will have a serious impact on all of the subscribers.

What might deter developers is the pub/sub nature of events, but this is no barrier to testing their behaviour. Assuming the event is triggered by a call to a synchronous method, then the result will be a call by the same thread (therefore on the same call stack) to one or more event handler delegates – so there are no asynchronous timing or thread issues to be worry about. Handling asynchronous events is slightly more difficult which I discuss at the bottom of this article.

Here is an example based on a retail website’s shopping basket (cart) which fires an event whenever an item is added or removed from the basket. To start with, here’s an NUnit test to check that an event fires whenever an item is added to the basket. It requires a field to tell us if the event has fired, and an event handler which sets the field to true. The event handler basket_ItemCountChanged is then hooked up to the ItemCountChanged event.

The test currently doesn’t compile because the types/methods/properties marked in red don’t exist yet, so here is the code to make it pass. There is a simple product interface and concrete class which is what gets added to the basket.

And here is the shopping basket itself.  It’s generic so that it can store any type, has an internal collection of items of type T, and fires the ItemCountChanged event whenever an item is added to the basket. Note: to keep the example simple, I have excluded code to make shared data access thread-safe.

Refactoring the test
The test now passes, however it’s a bit messy. So we can simplify it by dispensing with the _eventFired field and basket_ItemCountChanged event handler and replacing them with a local variable and an anonymous delegate in the form of a simple lambda expression. Not only does this make the test completely self-contained, but it’s a lot neater and more readable now.

Finally, re-run the test which goes green.

In the next post I discuss testing asynchronous events.

Posted in C#, NUnit, TDD | Leave a comment

Implementing IDisposable

There’s a bit more to implementing the IDisposable interface than just one Dispose method. For non-deterministic finalization to work properly you need to implement the disposable pattern shown below, making sure to distinguish between whether cleanup is being done via the public Dispose method, or via the finalizer (destructor) during garbage collection.

The private Dispose method actually does all of the work while the _disposed field ensures cleanup only happens once. The disposeManaged parameter indicates whether disposal is being triggered by a call to the public Dispose method by some client code (for example, when the object goes out of scope of a using statement) or if the finalizer has been called by the .NET runtime garbage collector. If the public Dispose method is called, then finalization is prevented by a call to the static method GC.SuppressFinalize. Again, this is to  prevent cleanup happening twice.

One really important point to note is that any derived classes must not implement their own finalizers.

Making it reusable
Just like with the IEquatable<T> interface, this has suddenly added a lot of noise to our class which doesn’t even do anything yet, decreasing readability and maintainability. Time for a reusable base class.

Now all you need to do in a derived class is implement the DisposeManaged and DisposeUnmanaged methods (and avoid adding a finalizer).  DisposeManaged and DisposeUnmanaged are called automatically so do not call them directly.  However if there is more than one level of inheritance, remember to call the base implementations of DisposeManaged and DisposeUnmanaged like so:

Posted in C# | Leave a comment