Program Specification Through the Mechanism of Automated Tests

My previous findings regarding the testing of public methods versus private methods opened the door to allow me to better understand that the mindset of a developer performing unit “testing” is different from the mindset of a developer “specifying program behaviour through the mechanism of automated test methods”. The difference is subtle but important.

In the concept and terminology of Behaviour-Driven Development, Design, Analysis and Programming, the resulting program code is simply an implementation of previously specified behaviours. Those said behaviours are initially defined at the business level via the process of requirements gathering and specification, and can be expressed in the business’s language as a set of scenarios and stories. These requirements can then be further analysed and broken down into logical and physical program specifications.

Regardless of whether a test-first approach to development is taken, by the end of the construction task, there should be a set of unit test cases (automatic and/or manual) that were successfully executed and that covers all of the logic specified in the program specification. In other words, by the end of the construction task, the developer has verified that the program behaves as specified.

Typically a program’s specification is written in a document somewhere which is divorced from the location of the actual code -  so great discipline and effort is required in order to keep the specification and code in synch over time. Depending on the company and financial constraints of the project, this may not be feasible.

However, what if we could remove this boundary between the document and the code?

Automated test cases serve a number of useful purposes, including:

- Behaviour verification;

- Acting as a “safety net” for refactoring code;

- Regression testing;

- Professional evidence that one’s own code actually does what one has said it does; and

- Acting as sample code for other developers so they can see how to use one’s code.

What if the automated test assets were the actual living program specification that is verified through the mechanism of test methods?

This concept has been recognised before, and the libraries JBehave, RSpec, NBehave and NSpec have been the result. While I honour the efforts that have gone into those libraries and the ground-breaking concepts, I do not necessarily like the usage of them.

In fact, all I want is to be able to express my specifications in my MSTest classes without the need for a different test runner or the need for another testing framework. In addition, I want other team members to easily grasp the concept and quickly up-skill without too much disruption.

While a DSL might be more ideal under these circumstances, I write my automated test assets in the C# language. Working within that constraint, I want to express the program specification in test code. With special note to the agreeable BDD concept that “Test method names should be sentences” I devised the following structure for expressing program specifications as automated test methods.

namespace |MyCompany|.|MyAssembly|.BehaviourTests. |MyClassUnderSpecification|Spec

{

public class |MyPublicMethodUnderSpecification|Spec

{

[TestClass]

public class Scenario|WithDescriptiveName|

{

[TestMethod]

public void When|MoreScenarioDetails|Should|Blah|

{

// Test code here

}

[TestMethod]

public void When|AnotherSubScenario|Should|Blah|

{

// Test code here

}

}

}

}

This structure allows for the naming of the public method that is under specification (the parent class name), a general scenario with a description of the context or given setup required (the nested MSTest class name), followed by the test method names which further narrow the scenario and provide descriptive details of the expected behaviour.

The test names look neat in the Test View in Visual Studio (when the Full Classname column is visible), and are readable just like their corresponding stories.

The full class name in this instance would look like:

|MyCompany|.|MyAssembly|.BehaviourTests. |MyClassUnderSpecification|Spec. |MyPublicMethodUnderSpecification|Spec +Scenario|WithDescriptiveName|

With automated spec/test assets/classes structured in this fashion, instead of developers performing unit “testing”, developers can instead embrace the mindset of “specifying program behaviour through the mechanism of automated test methods”. In turn, developers should be more focused on their tasks and it should also lead to higher quality code bases.

Oh, and I haven’t even mentioned how much easier it would be for a developer new to a project to learn and understand the code base by looking at the test view and reading all the scenarios that have been specified and coded…

Showdown: Testing Private Methods Versus Public Methods

The concept of testing private methods has been debated by many people, and up until now I have been happy to sit on the fence.

However, recently I have been looking at simplifying and optimising automated testing code. As part of that process, I created a small sample application with one public service operation method that calls a number of private methods.

In order to answer the question about whether there are worthwhile advantages to testing private methods, I decided to do a shoot-out of automated testing code. Here was the line up:

1) All the test code required to thoroughly test the sample application by testing both private and public methods; and

2) All the test code required to thoroughly test the sample application by testing just the public methods.

The result was conclusive: in order to thoroughly test the sample application, less test cases were required and significantly less automated testing code was required when just explicitly testing the public methods (and implicitly testing the logic in private methods).

Here is a summary of my findings.

- Less test cases were required because test cases for the logic contained within the private methods overlapped with test cases that were still required in order to thoroughly test the public methods;

- The private methods were still thoroughly tested implicitly through the test cases for the public methods; and

- The resulting test code was less brittle so developers can more freely refactor the application code without easily breaking the test code.

Don’t believe me? Perform a similar exercise for yourself!

TeamReview 2010 Installation Problem

The other day, a work colleague asked for my assistance regarding a problem installing TeamReview 1.1.2 (for Visual Studio 2010 Beta 2).

The error related to “TeamReview.VSNetAddIn.Addin”, with:

System.InvalidOperationException: Method failed with unexpected error code 3.

This error message is not very helpful, so I decided to use my friends Reflector and LinqPad to help me look for the offending code and then quickly execute and replicate the exception.

To cut a long story short, I discovered that the exception occurs under Windows 7 when the following Visual Studio add-in folder does not exist:

C:\Users\[Your User Name]\AppData\Roaming\Microsoft\MSEnvShared\Addins”

This equates to:

%APPDATA%\Microsoft\MSEnvShared\AddIns

The problem also happens on Windows XP:

C:\Documents and Settings\[Your User Name]\Application Data\Microsoft\MSEnvShared\AddIns

In order to resolve the issue, manually create the folders through Windows Explorer or run the following command:

mkdir %APPDATA%\Microsoft\MSEnvShared\AddIns

Another work colleague has now raised this issue on the TeamReview CodePlex site.

Desktop Virtual Machine Software Comparison

A few years ago I was using Microsoft Virtual PC at home, and I quickly transitioned to VMware Server and VMware Player due to my desire to use USB devices inside the VM.

Not fully satisfied, I tried other virtual machine software and subjectively determined that Sun VirtualBox ran my virtual machines faster and provided the USB support that I required. I have been a happy user of VirtualBox since then, and I have had no need to look elsewhere… until VirutalBox 3.0 started to support multiple CPUs.

The promise of multiple CPUs being used within a VM is a tempting feature that once embedded in one’s consciousness, becomes a mandatory requirement. For this reason, Microsoft Virtual PC (even the Windows 7 version) is unacceptable.

I have been experiencing stability issues in VirtualBox for many months now. In fact, I cannot use the multiple CPU feature in VirtualBox because after approximately 5 or 10 minutes of usage the graphics display corrupts and the virtual machine hangs. In addition, support for Direct X and graphics acceleration is experimental, and Windows Aero does not work within the VM. Multiple monitors are only partially supported through the ‘headless’ mode and using Remote Desktop with the /span option.

However on the positive side, VirtualBox supports Virtual PC, VMware and open hard disk formats, in addition to a seamless window mode, USB, and snapshots.

My recent investigations into virtual machine software have surprised me – and I must applaud VMware.

Both VMware Player 3 and Workstation 7 support multiple CPUs, multiple monitors, Windows Aero, DirectX and graphics acceleration, USB, Unity / seamless / integrated mode, and the ability to create virtual machines (previously Player did not have this ability).

VMware Workstation in addition provides snapshots and record/replay functionality – among other things.

Virtual PC does not support multiple CPUs, nor multiple monitors, nor graphics acceleration, nor Windows Aero, but it does now support USB (due to the concept of Windows 7 XP mode) and desktop integration.

So in conclusion, I have discarded VirtualBox and now prefer VMware Player.

Perhaps VirtualBox 4.0 will be stable with multiple CPUs, work fully with multiple monitors, support Windows Aero, and have non-experimental support for DirectX and graphics acceleration? (It would just be nice to have the option to use snapshots – which VMware Player doesn’t and probably will never have)…

Converting Between Virtual Disk Formats

In addition to its own native VDI virtual disk file format, Sun’s VirtualBox supports both Virtual PC’s VHD virtual disk file format and WMWare’s VMDK virtual disk file formats, AND has the ability to convert your virtual disks between these formats.

What they don’t tell you however, is that the virtual disk must be fully removed from the VirtualBox Virtual Media Manager before the command will run successfully (if you have been using the virtual disk previously in a virtual machine in VirtualBox).

For example, the command to convert a VMDK to a VHD is:

"c:\program files\sun\VirtualBox\VBoxManage.exe" clonehd "my.vhd" "my.vmdk" –format vmdk

Below is the documentation from section 8.16 VBoxManage clonehd:

VBoxManage clonehd         <uuid>|<filename> <outputfile>
                           [--format VDI|VMDK|VHD|RAW|<other>]
                           [--variant Standard,Fixed,Split2G,Stream,ESX]
                           [--type normal|writethrough|immutable]
                           [--remember]

where the parameters mean:

format

Allow to choose a file format for the output file different from the file format of the input file.

variant

Allow to choose a file format variant for the output file. It is a comma-separated list of variant flags. Not all combinations are supported, and specifying inconsistent flags will result in an error message.

type

Only honored if –remember is also specified. Defines what kind of hard disk type this image should be.

remember

Keep the destination image registered after it was successfully written.

Lessons In Domain-Driven Design – Basic Overview

Recently I wanted to gain a clear, basic overview of Domain-Driven Design, and I stumbled upon Domain-Driven Design Pattern Summaries document from http://domaindrivendesign.org/resources/what_is_ddd which is an excerpt of the book Domain-Driven Design, Addison-Wesley, by Eric Evans, 2004. The summaries document is licensed under the Creative Commons Deed: Attribution 2.0.

I also watched a presentation by Eric Evans entitled “What I learned about DDD since the book”.

After studying the material, I drew a simple diagram to illustrate the Domain Model, in a way that made sense to me.

A Domain Model is a sphere of knowledge.

A Value Object contains a number of attributes that describes an object, but does not provide a mechanism to uniquely identify it.

An Entity is essentially an object that has a specific unique identity.

An Aggregate is a composition of a number of Entities and Value Objects, with one specific Entity being the root/parent/master.

A Repository encapsulates the data access functionality of Aggregates and can emulate access to a collection of objects. The role of the Repository is to provide an object-oriented view of the persistence layer and translate the object-oriented Domain Model to the physical data store.

A Domain Service is used for processes or transformations in the domain that are not the responsibility of an Entity or Value Object.

A Domain Event is some event that occurs that is important to be noted by a Domain Expert. An event happens and the effect is that state is changed.

Lessons In TFS Work Item Customisation – System.State Is Special

I was customising some TFS 2008 Work Items, and I wanted to automate it so that the System.State value change to a new state value when a custom field was changed to a specific value.

Quite simply, you can’t do this. The System.State field cannot have a value copied into it, or have other actions done to it.

If you are serious about automating like this, the only workaround is to create a custom program that listens for a particular event or custom action, and then use the rich TFS API and programmatically change the value of System.State.

The same applies to System.Reason.

Lessons In MSBuild – Targets Are Called Only Once

By design, targets in a MSBuild process are only called once. This is an important concept to understand, as it is unusual for the typical developer.

I had a situation where I wanted to call the same target from a few places in my MSBuild project – but I wanted to pass in different property values. I didn’t want to repeat myself (DRY principle).

If <CallTarget …> is used then the target is only called once within that process.

I found a sneaky way to work around this by examining the file C:\Program Files\MSBuild\Microsoft\VisualStudio\TeamBuild\ Microsoft.TeamFoundation.Build.targets

Instead of using <CallTarget …>, use <MSBuild …> which spawns off a new MSBuild process, thus allowing the target to be called each time.

<MSBuild Projects="$(MSBuildProjectFile)" Targets="MyTarget" Properties="MyProperty1=blah;MyProperty2=$(MyVariable)\blah2" />

Lessons In MSBuild – Applying the Single Responsibility Principle to MSBuild Projects

As an experienced developer, I promote the use of good practices in object oriented designs.

I was modifying a number of MSBuild and TFS Build project files for several related environments and I noticed that they both duplicated information and each had more than one responsibility – such as specifying the solution to build and compilation settings such as code analysis, configuring testing, and then perhaps performing some custom operations.

I wondered if I could apply the Single Responsibility Principle to the build project files.

Indeed, you can split your project files into multiple files, and from the main project you can import the other file by using the following syntax.

<Import Project="MyCommonTargets.proj" />

nVidia Display Driver Prevents USB Wake Up

I didn’t know that a video display driver could prevent my USB mouse and keyboard from waking up after the computer has been sleeping… but it is true.

After installing the new July 2009 version of the nVidia GeForce 7900 GS display driver, when I turned on my Vista 64-bit machine after it had been sleeping, the windows logon screen would display and then quickly flick black a couple of times. And then… then the mouse and keyboard did not even have power from the USB.

Not a very funny joke nVidia.