Unit tests and Use Cases

A common challenge when writing unit tests for a large code base is keeping track of what you are testing. Without a good process for categorizing what the individual tests are actually testing, it can quickly become unmanageable. This is especially true for projects with several hundred tests.

The way I have decided to tackle this challenge in ExpressUnit is through use cases. For those of you unfamiliar with the concept of use cases, here’s the Wikipedia definition:

“A use case in software engineering and systems engineering is a description of a system’s behavior as it responds to a request that originates from outside of that system. In other words, a use case describes “who” can do “what” with the system in question. The use case technique is used to capture a system’s behavioral requirements by detailing scenario-driven threads through the functional requirements.”

Source: http://en.wikipedia.org/wiki/Use_case

Since use cases are a description of how the system should behave, I find them to be a very natural grouping for unit/integration tests. The mechanism for grouping tests by use cases in ExpressUnit is actually quite simple. I have simply extended the test attributes (UnitTest/IntegrationTest) to take in an optional use case parameter. The new syntax works as follows:

 [UnitTest(UseCase="Confirm.Equals will return true if objects are equal")]

public void ConfirmDoubleObjectsAreEqualTests()

{

    Confirm.Equals(1.0, 1.0);

}

The above example is from ExpressUnit’s own Confirm test suite where all Confirm.Equals tests have been linked to the use case: “Confirm.Equals will return true if objects are equal”.

Version 3.2 of ExpressUnit contains a new UI window for displaying test results broken down by use cases. (See Figure1). You can navigate to this window by clicking the Use Case tab after running all your tests.

Figure1

Continuous integration: ExpressUnit and Hudson

To manage large software projects with many developers and a large code base, it’s very useful to automate builds and test runs through continuous integration. The most common setup is to have a continuous integration server build your source code and run all your tests on code check-in. This gives you almost immediate feedback on whether or not your latest code check-in broke the build by either causing a compiler error or test failure. There are several alternatives out there for continuous integration. I have chosen to integrate ExpressUnit with Hudson which is an open source continuous integration platform. The nice thing about Hudson is that it’s very easy to configure. The interface is extremely user friendly and lets you handle everything through a simple web based UI. I won’t go into much detail on how to set up Hudson as there are plenty of good tutorials available on the web. Instead I will list some of the necessary steps to integrate ExpressUnit with Hudson

1)Add your source code management tool through the Hudson web UI

2) Set up MsBuild as your source code builder (point to your solution file)

3) In the Windows batch command window make a call to “ExpressUnitGui.exe runtests” which triggers ExpressUnit to run and close after all tests have been run (No user interaction required)

4)Select to publish JUnit test result report. ExpressUnit will, when running in command line mode, generate an xml file with test results which will be picked up by Hudson and displayed in the UI. The generated xml file is compliant with the JUnit test result format required by Hudson, so Hudson will be able to display information about your test runs. You will be able to see statistics of test runs over time as well as information about why particular tests failed.

I am actually using ExpressUnit as my testing framework while building ExpressUnit. The following screen shot shows my ExpressUnit project on Hudson

Figure1

On the left you can see the build history, and over on the right is the test trend graphed over time.

Please drop me a line if you need more detailed instructions on how to get ExpressUnit up and running with Hudson

Is pure TDD practical?

 

As a software developer I have always considered unit testing an important asset to programming. Most developers who do unit testing will naturally develop their own style and technique for writing tests. However many have started to embrace a new paradigm called Test Driven Development (TDD).

TDD mandates that the unit test must be written before the actual implementation of the code you are testing. I like this idea in principle, but I argue that it might not always be practical. Instead I practice what I call ATDD – Almost Test Driven Development. The difference is that I implement my code before writing the unit test, but I always envision what the test will look like before writing any code. When creating the test, I may also debug into my new code to capture output that I can use in the Assert statement of my test. This clearly violates the TDD paradigm since I already have an implementation to test. However, I feel this is more practical since it may save me a lot of typing if the result value of my code is complex and hard to produce manually. This is especially true if the output is a long complex string / xml. It is of course very important to be very careful when manually inspecting the result of the method.

Another problem I have with the pure TDD approach is compilation overhead. In large .Net solutions it is not always practical to compile and re-run your tests for every little change you make to a method while it is being created. I therefore believe the most practical approach is to implement your code first, but do so with the unit test in mind.

Bulk Insert Sql Server

In one of my recent projects I needed to import data from tab delimited files into an SQL Server database. I decided to go with the Bulk Insert option. Bulk Insert is a neat little T-SQL function that lets you import tabular data from a structured file to a corresponding database table.

Here’s an example of how to use Bulk Insert

BULK INSERT Customer
FROM ‘c:\import\customer.txt’
WITH
(
  KEEPNULLS,
  FIELDTERMINATOR =’\t’,
  ROWTERMINATOR =’ \n’

)

The above code samples imports tabular customer data from customer.txt, and inserts it into the Customer table.  It’s important that the column in both the file and the table are in the same order.

My project called for a tab delimited file, but you can use whatever delimiter you like. Other typical formats you might see are comma separated files (csv) and | separated files.

One thing to keep in mind: Don’t represent null values using some variation of a “null” – string since this will be seen as a regular string, which will fail if you try to insert it into a non char column (or a char column too small to hold your null string). You should instead leave it as an empty value in your delimited file.

More information and documentation can be found here:

http://msdn.microsoft.com/en-us/library/ms188365.aspx

 

Unit Testing in Visual Studio Express 2008

In an earlier blog post I described how to use Nunit for unit testing in Visual Studio Express. One short coming of this strategy is that you can’t easily debug your tests. This is because Visual Studio Express doesn’t seem to let you attach external processes. (We would need to attach the external Nunit process to step into the code).

To get around this rather annoying problem, I have created my own testing framework; ExpressUnit. The current version of ExpressUnit is now 3.x. The latest version offers two new benefits over previous versions:

1. The new UI interface has been created using WPF
2. ExpressUnit 3.x supports continuous integration with Hudson which means you can use ExpressUnit to run your  tests on your build server. In my current development environment I have configured ExpressUnit to run all tests on every code check-in. My continuous Integration environment is currently set up using Subversion, Hudson and ExpressUnit.
   

Instead of relying on an external application to run the tests, ExpressUnit will run from within your Visual Studio Solution. The idea is that you download ExpressUnit in the form of a Visual Studio project and add it to your solution.

The added project becomes your unit test project where you may add your test classes with the belonging unit tests. No configuration is required.

To add tests, simply add a test class with public test methods. The class and methods must be decorated with the [TestClass] and [UnitTest]/[ItegrationTest ] attributes respectively. The unit test attribute should be used for unit tests and the Integration test attribute is meant for integration tests. ExpressUnit will let you filter test runs on these attributes.

It’s usually my preference to add all my tests to the ExpressUnit project, but ExpressUnit allows for tests to be defined in any project in your solution (as long as the ExpressUnit project has a reference to it).

Anyway, that’s enough background information! To make this as useful as possible I will demonstrate how to use ExpressUnit through an example.

Step1:

Download ExpressUnitGui from

http://code.google.com/p/expressunit/

Step2:

Add the ExpressUnitGui project to your already existing Visual Studio 2008 solution.

After completing this step, your solution will look similar to Figure1

Figure1

Step3:

Start writing your first ExpressUnit test.

As mentioned earlier, you must start by adding a test class with at least one test method to the ExpressUnitGui project. The class may look something like the following:

[TestClass]

public class SampleTest

{

[UnitTest]

public void AddTest()

{

MathClass math = new MathClass();

Confirm.Equals(4,math.Add(1,3));

}

}

Step4:

Run your test!

To run your tests, select ExpressUnitGui as your startup project and press F5

The UI window will load with a simple interface to execute one or more of your tests. (Figure2). You may run entire test suits or individual test by right clicking the appropriate tree node in the tree view, but to run all tests at once, all you have to do is click the green “Play” button. The test results are shown in the right pane, and as you can see in Figure2, each test result offers an expand button to view more details about failing tests.

There is also an option to run all tests automatically on start-up of ExpressUnit. This feature is controlled by the runTestsOnStartup flag in the application’s App.Config (Default value is false)

Figure2

API

The following is an overview of the API offered by ExpressUnit

Confirm methods

The confirm class contains several static methods used to confirm the outcome of your tests

Confirm.Equals(object obj1, object obj2):

This method compares two objects for equality by using the object’s own Equals implementation.

Confirm.Equals will throw EqualsException if the two objects are not equal.

Confirm.Different(object obj1, object obj2):

This method is the inverse of Confirm.Equals, and is used to check if two objects are unequal.

UnEqualityException is thrown if the two objects are equal.

Confirm.IsGreater(long/double val1, long/double val2)

This method is used to test if val1 is numerical bigger than val2

Confirm.SameCollections(IEnumerable param1, IEnumerable param2)

This method will test if two objects of type IEnumerable are the “same”, and the definition of “same” is that all contained objects are equal.(based on the contained object’s own Equal implementation. For example, it may be used to compare the content of two arrays or collections.

EqualityException is thrown if the two instances are not the “same”.

Confirm.ExceptionThrown(Type expectedExceptionType,TargetMethod target)

This method will check if the target method throws the expected exception. The target method is passed in as a delegate of type TargetMethod.

Attributes

The following are attributes defined by ExpressUnit

[TestClass]

This attribute defines a class as a test class.

[UnitTest]

This attribute defines a method as a unit test

[IntegrationTest]

This attribute defines a method as an integration test

[Ignore]

This attribute will cause a test to be ignored when running your tests

[ExceptionThrown(Type exceptionType)]

This attribute can be put on a test to indicate that an exception of type exceptionType is expected to be thrown. The test will fail if the exception is not thrown.

Conclusion

If this this article has made you curious about ExpressUnit, you may download it from

http://code.google.com/p/expressunit/

Don’t hesitate to drop me a line if you have comments or would like to see additional functionality in ExpressUnit.

Unit testing in Visual Studio Express II

In a previous blog entry, I talked about how you can add unit testing capability to projects in Visual Studio Express by using Nunit.

Instead of using the Nunit Gui client, you can automate the test run by using a post build event and the console version of the Nunit client.

To automatically run your tests when building: Add the following line to your Visual Studio post build event:

“[Local Nunit path]\nunit-console.exe” $(TargetDir)MyTests.dll /xml $(TargetDir)results.xml

This will start Nunit (behind the scenes), and run all tests defined in MyTests.dll. The test results are written to results.xml.
It is worth noting that the build will fail if any of the unit tests fail.

Linq

Version 3.5 of the .Net framework introduced Linq as new way of dealing with object models.

I haven’t had time to play with Linq extensively, but my impression so far is good.

Linq uses a model where content from your data source is mapped into objects, or entities if you like that notation better.

A typically Linq scenario is to build an object model around a relational database where each table will be represented by a typed object in your object model. The Linq framework will aid you in generating the actual entity classes and all CRUD statements. The framework will also maintain/expose a data context which serves both as your connection to the database, and your channel for invoking actions on your data source. The data context will do all the book-keeping of object state (insert, update, delete) as well. The only requirement is that you keep the context “alive” until all changes have been submitted to the database.

Linq can also work with other data sources such as Xml, but if you ask me; its real strength comes from the ability to query typed objects in collections. Any collection implementing IEnumerable<T> can be queried using Linq. This can save you a lot of “foreaching” when trying to filter a collection based on specific parameters.

The syntax of a Linq query resembles Sql syntax, which makes it an easy transition if you have some experience writing basic Sql.

Unit testing in Visual Studio Express

Visual Studio Express is a scaled down version of Visual Studio. The product is available for free, and can be downloaded from Microsoft. However, the product has one serious short coming; it doesn’t support unit testing!

It is unfortunate that Microsoft has excluded unit testing from Visual Studio Express, but there is still hope for those of you who want to do Test Driven Development. The solution is to add a third party unit testing framework. I have looked into Nunit, which can be downloaded for free from http://www.nunit.org/index.php

To write unit tests in Visual Studio Express using Nunit, follow the following steps.

1. Download Nunit from http://www.nunit.org/index.php

2. Create a class library project in Visual Studio Express

3. Add a reference to nunit.framework.dll (This file was downloaded as part of step 1)

4. Write your test code.

The test code follows the following format:

using NUnit.Framework;

namespace UnitTests

{

[TestFixture()]

public class UnitTest

{

       [Test]

public void TestAdd()

{

     int a = 4 + 4;

     Assert.AreEqual(8, a);

}

}

Start the Nunit GUI application, and open the class project which was created as part of step 2).

Your tests can be run from the Nunit GUI application, as shown in the following screenshot.

 An alternative to NUnit can be found here:

http://torgeirhelgevold.wordpress.com/2008/05/31/unit-testing-in-visual-studio-express-2008/

Removing data dependency in unit tests

A common problem when writing unit tests is data dependency. Typically, when you are writing tests for a method, the same method is making a call to another method which makes a call to some external data source. Code listing 1 illustrates this: The SellStock method depends on the value returned from StockService.

Code Listing 1

public class Trader

{

private int limit = 50;

public bool SellStock(stringstockName)

{

     int stockValue = StockService.GetStockValue(stockName);

     if(stockValue > limit)

     {

          return true;

     }

     return false;

}

}

I feel this dependency is undesirable in the context of a unit test, and the following is a technique I commonly use to remove this dependency:

Instead of calling StockService.GetStockValue directly, I move the call into a protected virtual method:

Code Listing 2

public class Trader

{

private int limit = 50;

public bool SellStock(stringstockName)

{

     intstockValue = GetStockValue(stockName);

     if(stockValue > limit)

     {

          return true;

     }

     return false;

}

protected virtual intGetStockValue(stringstockName)

{

     return StockService.GetStockValue(stockName);

}

}

The idea is that the test class can inherit the Trader class, and override GetStockVaule to return a predefined value.

Code listing3 illustrates this:

Code Listing 3

public class TraderTest : Trader

{

private int limit = 50;

private int stockServiceResult = 40;

public boolSellStock(stringstockName)

{

     int stockValue = GetStockValue(stockName);

     if(stockValue > limit)

     {

          return true;

     }

     return false;

}

 protected override int GetStockValue(stringstockName)

{

     return stockServiceResult;

}

 

[TestMethod]

public void SellStockTest()

{

     Assert.AreEqual(false,this.SellStock(“test stock”));

}

}

 

The result is that we can test the SellStock method without having to call the StockService. I personally use this technique extensively when writing unit tests, and feel it’s a really good way to remove data dependencies.

 

Antivirus SDKs

A while ago, I was faced with the challenge of doing a programmatic virus scan from a custom application.

I decided that the best approach would be to integrate with an antivirus SDK.

After doing some research, I found two antivirus SDKs which were both available as free trial versions. The first one was made by a company called Sophos. Downloads and instructions for how to integrate with the product can be found at www.sophos.com.

The other product I looked into was an open source alternative called Clam. The Clam engine itself is free, but I was not able to find a free integration tool for the engine. The integration tool I ended up trying is called Metadefender, which can be downloaded as a free trial version from www.metadefender.com. The commercial version of Metadefender is not free, but the license is not very expensive.

Both products were pretty easy to integrate into my application, and it seems that my application could use either one.