Archive for the ‘Testing’ Category
Sunday, November 15th, 2009



At the recent SharePoint Conference in Vegas I took a few minutes out to talk to Gil Zilberfeld of Typemock about what 21apps is doing with SharePoint and what I see as the next steps in the community regarding SharePoint development.

Looking at what areas I see as being a focus in the SharePoint development space,  how I will continue to push TDD but will also, now that people are starting to talk about good SharePoint development practices, start to look at the wide picture – looking at how we complement the Unit Tests with integration tests,  looking at ways to automate use acceptance tests and generally looking at ways to make the code better so that testers can focus on the scenarios and complex tests rather than dealing with the ’stupid bugs’.

Original post on Typemock Blog

Thursday, June 25th, 2009



Following on from my rant response yesterday about the more negative views from the SharePoint community my reading this morning was so much more positive.

Unit Testing Workflows

In my response to a question from Aaron Weiker I said that I would like to see much more guidance and investigation into the way we approach Unit Testing workflows in SharePoint,  so it was great to find Richard Fennell is doing a lot of work in this area.  I have been working with Typemock Isolator, CThru and SilverUnit testing recently (post coming) and was very interested to see that Richard had picked up this whilst looking for a possible solution to the challenges of Unit Testing SharePoint workflows; something I have to confess I had not even considered.

I’m looking forward to seeing where Richard gets with this and also the use of Fit/Fitness, although I’m personally not a big fan of the way tables are used to define the tests.

Integration Testing SharePoint from MSTest

Having been a bit behind with my reviews of the latest Codeplex P&P project I missed the discussion with Francis on how they found a use for Typemock Isolator to do Integration tests.

The code is really very very simple,  many will be glad to hear, and deals with the places where your code makes a call to SPContext.Current or SPFarm.Local.

The simple ideas really are the best ones.

As you can see,  a few small steps for testing but big steps for SharePoint testing and a demonstration that in general the SharePoint community is committed to improving the way solutions are developed.

Tuesday, June 23rd, 2009



Just read a post by Sahil Malik where he gives his reasons for saying

TDD + SharePoint 2007? Well – screw it! Not worth it IMO.

The reason for his statement is that he sees the actual C# code developed as being a very small % of the total SharePoint project and the effort to do this as not worth the return when you consider the project costs overall.

The problem is, like so many people, he is getting TDD mixed up with doing good development.  He dismisses TDD because it is harder to do in SharePoint because of the API.  It’s not TDD that makes it harder,  it being able to isolate the API and design testable code that is harder.

If you listened to the SharePoint Podshow I did with Eric Shupps we were in agreement that what actually mattered was that you developed good quality code,  that you had the processes in place to enable code reviews, unit testing, integration testing, and the like, that all go together to ensure the code you do develop for your SharePoint solutions is Enterprise ready.

I hold my hands up, I do talks on TDD and SharePoint.  I don’t do this because I think that everyone should do this on day one,  I do it because people have used SharePoint as an excuse to not doing quality development.  The comment ‘we can’t do unit testing because we are using SharePoint’ has been common among development teams,  other just don’t do unit testing anyway so it never even cropped up.   So if I can demonstrate the you can go all the way and actually code your solution using a technique like TDD then this argument no longer holds water.

What the question should be

Quality Development and SharePoint, is it worth doing?

And the answer to that is a big Hell Yes!

And if your happen to use TDD to write some of that quality code, then that’s good too.

The key thing to remember is that you need to make a judgement call about the value; in Sahil’s defence he did mention this.  What I can’t take is that he adopts the ’screw it’ stance before he even starts. This to me is a much of a problem as the people at the other end of the spectrum that think you can ONLY code using TDD.

Tuesday, May 19th, 2009



One of my favourite tools Typemock has just been released in a fantastic ASP.Net bundle and have once again offered this up to the blogging community to get for free.

Unit Testing ASP.NET? ASP.NET unit testing has never been this easy.
Typemock is launching a new product for ASP.NET developers – the ASP.NET Bundle – and for the launch will be giving out FREE licenses to bloggers and their readers.
The ASP.NET Bundle is the ultimate ASP.NET unit testing solution, and offers both Typemock Isolator, a unit test tool and Ivonna, the Isolator add-on for ASP.NET unit testing, for a bargain price.
Typemock Isolator is a leading .NET unit testing tool (C# and VB.NET) for many ‘hard to test’ technologies such as SharePoint, ASP.NET, MVC, WCF, WPF, Silverlight and more. Note that for unit testing Silverlight there is an open source Isolator add-on called SilverUnit.
The first 60 bloggers who will blog this text in their blog and tell us about it, will get a Free Isolator ASP.NET Bundle license (Typemock Isolator + Ivonna). If you post this in an ASP.NET dedicated blog, you’ll get a license automatically (even if more than 60 submit) during the first week of this announcement.
Also 8 bloggers will get an additional 2 licenses (each) to give away to their readers / friends.
Go ahead, click the following link for more information on how to get your free license.

I’ve not had the chance to try out the Ivonna or SilverUnit add-ons to the framework yet, but will be looking at these in the near future and they can help you test more of your SharePoint solutions.

Thursday, April 23rd, 2009



Great to see that more people are publishing their experiences with unit testing SharePoint.   Here are some of examples I have found that are really worth looking at:

Richard Fennell – Testing SharePoint Workflows using TypeMock Isolator (part 1, part 2 and part 3) – Richard is also going to be talking at a few NextGen meetings on the subject.

SharePoint Dev Wiki is really starting to get some good content not just on unit testing but on the whole development field.

Jeremy Thake (creator of SharePoint Dev Wiki) did a great usergroup meeting web cast on SharePoint Development with Unit Testing

I’ve previously mentioned the Patterns and Practices work and they have started to do some great Channel 9 videos which really do make this so much more accessible.  Brilliant Job!

If you have any links or posts on the subject please share by adding a link in the comments, and if you’ve done something new and exciting help the SharePoint Dev Wiki by posting details.

Looks like my predictions are starting to come true,  people really are doing SharePoint development better :)

Thursday, March 5th, 2009



A short and to the point post by Dror Helper at Typemock looks at industry findings on the cost benefits of doing TDD.

The Cost of Test Driven Development

..the research proved two points:

  1. Using TDD reduce the amount of bugs in the code significantly
  2. Using TDD takes more time then not using TDD

This is in line with what was expected, the question is does the extra time doing TDD reduce the quality or is it the unit testing.  Could doing Test Last unit testing give similar results?

There is also the unquantifiable part to the numbers in the quality of the developers in each team.  This has a significant impact on the code quality.  Using processes like TDD help to raise the bar for all of the developers and the cost benefit will improve over time.  It would be a very interesting exercise to see how these numbers work out in a typical SharePoint project.

On the same lines Phil Haack (great name) has a post on Bug Driven Development where he quotes Robert Glass and Steve McConnell on the costs not doing some form of automated testing which reinforces the value in getting the right processes in place.

Thursday, February 19th, 2009



My friend and fellow SharePoint MVP Eric Shupps has been promising to enter the conversation, started a number of years ago by Spence Harbar, and provide his wisdom and views on why he thinks TDD in SharePoint is just wrong!  It has been debated over email and over a SharePint for many many months.   Finally Eric has put pen to paper (or fingers to keyboard) and joined the conversation.

SPTDD: SharePoint and Test Driven Development, Part One

I now know why it took Eric so long to get his post out,  it’s taken me an hour to even read it.   As with any good debate Eric managed to get me pondering his views, what I see as misconceptions and ideas and I will be coming back with a number of posts in response.  Sorry I just can’t write that much in one blog post :)

I will start by countering a specific part of the of Erics post for the moment;   TDD has it’s origins in Extreme Programming but it is not intrinsically linked to pair programming.  I think Eric has been mislead by some more zealot supported of TDD on this.  He did however miss one of the main benefits of pair programming in his complete dismissal of the approach which is Shared Knowledge.   Later in the post he talks about lack of knowledge, poor quality developers, and also the ability to be able to actually code against SharePoint before you try something so difficult as unit testing.   Pair programming can help in this regard – but it’s something you really need to have a desire to make work.   I have used it,  it’s not for everyone and you will always have personality clashes!  I have found it to be very useful in specific areas and in small chunks.  If you do get a few really good pair programmers I think you can see improved productivity – the reality however is most people never give it long enough to see the benefits.

Monday, January 19th, 2009



This post is part of a series on Test Driven Development – Using Dependency Injection

We introduced some terms previously to help describe some of the things we need to do in order to make our code more testable and started the refactoring of our solution in Step 1. In this step we are going to complete the refactor and implement Dependency Injection in the RandomMessage class.

There are a couple of ways to pass in the object we want to work with; we will use the Constructor as this is probably the most common and easiest to use.  We have a few things that we need to do to complete the refactoring

1. Updated the tests to pass a RandomNumber object into the constructor of GetMessage

2. Update the GetMessage class to support the passing in of the RandomNumber object

3. Create a fake of the RandomNumber class that implements the IRandomNumber interface so that we can test our code

4. Confirm we have fixed our test and all tests pass.

Because we previously refactored out test code to have a setup method we only have to make one change to use the planned constructor.

[SetUp]
public void SetUp()
{
      message = new ObjectModel.RandomMessage(new RandomNumber());
}

We then update our production code to support this new Constructor, it should look like this with the constructor highlighted.  The GetRandomMessage method has also been updated to use the _randomNumber variable.

    public class RandomMessage
    {
        IRandomNumber _randomNumber;
        public RandomMessage(IRandomNumber randomNumber)
        {
            _randomNumber = randomNumber;
        }
        public string GetRandomMessage(string WebUrl)
        {
            string message = "";
            try
            {
                using (SPSite site = new SPSite(WebUrl))
                {
                    using (SPWeb web = site.OpenWeb())
                    {
                        SPList messages = web.Lists["Messages"];
                        int rndMessage = _randomNumber.GetRandomNumber(messages.ItemCount);
                        message = messages.Items[rndMessage].Title;
                    }
                }
            }
            catch (UriFormatException)
            {
                message = "Error: Please provide a site url for the message list.";
            }
            catch (ArgumentException)
            {
                message = "Error: Messages list could not be found";
            }
            catch (FileNotFoundException)
            {
                message = "Error: Site could not be found";
            }
            return message;
        }
    }

To get the solution to compile you can update the WebPart class with the similar change to the one in the tests.

Running the tests you should see your previous tests continue to work, showing once more the value of the tests – we refactor with confidence, we know that when done our production code still does what the developer wanted it to do.. it continues to meet the specification.

Meets the specification,  what am I taking about!  Doesn’t the code pass the tests?

I’ll leave this question with you for now,  I will be talking about this a lot over the coming months..

Back to our Red bar which should look something like this

image

The observant among you will have noticed that the error has changed.  We are no longer getting a warning from Isolator about unsupported objects.  What we now have is a genuinely failing test.   There is a chance that your test may have passed, its a slim chance, as our test was asserting that we returned the 100th message from the list.  If your call to the GetRandomNumber returned 99 it will match.

Obviously this is not acceptable, and is the reason we introduced Dependency Injection.   Rather than faking the return from GetRandomNumber using Isolator (mocking) we are going to pass in our object that implements the same IRandomNumber interface.  If you remember back to when we Introduced some terminology we talked about loose coupling.   Our GetRandomMessage class is now loosely coupled to the RandomNumber class so we can create our own, based on a common Interface, and use this instead.

We create our own test class that allows us to define the values returned from the GetRandomNumber method.  You could put this in it’s own .cs file depending on how you like to structure your projects – personally I keep this in the same file as the tests that use it.

Continuing the principle of only coding what we need to our new fake class should look something like this

    public class FakeRandomNumber : ObjectModel.IRandomNumber
    {
        #region IRandomNumber Members
        public int GetRandomNumber(int max)
        {
            return 99;
        }
        #endregion
    }

We have hard coded the return value of 99 as this is what we want to fake – to return the last item in the list.  If we added more tests that need different values we may extend this with a simple property that allows us to set the return value within the test.

Update the test to use the FakeRandomNumber we just created, we will need to define our own instance of the RandomMessage class.

//Act
ObjectModel.RandomMessage message = new ObjectModel.RandomMessage(new FakeRandomNumber());
string rndMessage = message.GetRandomMessage("http://validsiteurl");

We have our green bar,  and everything passes.

image

In the final part of the series we will review the changes we have made and how TDD really does help you design better code.

TDD SharePoint – Using Dependency Injection Summary

Wednesday, January 7th, 2009



In our previous white paper Unit Testing SharePoint – Getting into the Object Model we introduced the concept of mocking, looked at the new Typemock Isolator AAA API to demonstrate testing against SPSite, SPWeb, SPList and SPListItem objects and reinforced the benefit of using TDD.

In order to keep the white paper easy to follow we used the Isolator features to fake the call to our GetRandomNumber() method as part of the test GetRandomMessage_MultipleMessages_ReturnMessage.

[Test]
public void GetRandomMessage_MultpleMessages_ReturnMessage()
{
    //Arrange
    SPSite site = Isolate.Fake.Instance<SPSite>(Members.ReturnRecursiveFakes);
    Isolate.Swap.NextInstance<SPSite>().With(site);
    Isolate.WhenCalled(() => site.OpenWeb().Lists["Messages"].ItemCount).WillReturn(1000);
    Isolate.WhenCalled(() => message.GetRandomNumber(1000)).WillReturn(99);
    Isolate.WhenCalled(() => site.OpenWeb().Lists["Messages"].Items[99].Title).WillReturn("Message One Hundred");
    //Act
    string rndMessage = message.GetRandomMessage("http://validsiteurl");
    //Assert
    Assert.AreEqual("Message One Hundred", rndMessage);
}

This makes for a simple test and helped us focus on the objective of testing the SharePoint object model, however this test does not work if we use Isolator for SharePoint which can only fake SharePoint objects. With this version you will get an error advising of the need to upgrade to the commercial version.

UnitTest.ObjectModel.Test.RandomMessage.GetRandomMessage_MultpleMessages_ReturnMessage:
TypeMock.TypeMockException :
*** Can not fake non-SharePoint type UnitTest.ObjectModel.RandomMessage with current license.
In order to fake non SharePoint types please upgrade to Typemock Isolator Commercial license
See more information at http://www.typemock.com/sharepointpage.php.

There are a few solutions to the problem

  1. Upgrade to the Full version of Typemock Isolator
  2. Introduce another mocking framework like NMock to mock this method
  3. Redesign you application to be more testable and make this test possible without using Typemock Isolator

NMock is an open source mocking framework that provides similar functionality to Typemock however NMock (or any other mocking framework) is unable to fake SharePoint objects as they make use of sealed classes.  We would recommend that you don’t attempt to work with two different mocking frameworks at the same time as this will make you testing much more difficult to follow and maintain.

There has been a lot of debate around Typemock being too powerful and if the value of designing testable code is as important now that Typemock can solve these issues.

We’re not going to argue the case for or against here; We’ll work on the assumption that you have Isolator for SharePoint and will redesign for testability.

Our refactoring of the project has been broken down into 3 steps proceeded by an introduction to some common vocabulary will be published over the next few days.

Tuesday, November 25th, 2008



In our white paper Unit Testing SharePoint Solutions – Getting into the Object Model we needed to use Natural Mocks to allow the testing exceptions that are thrown as part of the the SPSite constructor.  Our test code looked like this and although not complicated it introduced another concept that could complicate things for newbies more than necessary.

[Test]
public void GetRandomMessage_SiteNotFound_ReturnErrorMessage()
{
    //Arrange
    using (RecordExpectations expections = RecorderManager.StartRecording())
    {
        expections.ExpectAndThrow(new SPSite("http://invalid_url"),  new FileNotFoundException());
    }

    //Act
    string rndMessage = message.GetRandomMessage("http://invalid_url");

    //Assert
    Assert.That(rndMessage.StartsWith("Error"));
    Assert.That(rndMessage.Contains("Site could not be found"));
}

 

With the release of V5.1.2 of Isolator for SharePoint a new API has been added to the AAA syntax to support just this scenario.

Isolate.Swap.NextInstance<T>().ConstructorWillThrow();

Our tests will now look like this:

 

[Test]
public void GetRandomMessage_SiteNotFound_ReturnErrorMessage()
{
    //Arrange
    Isolate.Swap.NextInstance<SPSite>().ConstructorWillThrow(new FileNotFoundException());

    //Act
    string rndMessage = message.GetRandomMessage("http://invalid_url");

    //Assert
    Assert.That(rndMessage.StartsWith("Error"));
    Assert.That(rndMessage.Contains("Site could not be found"));
}

You will agree this is a much easier API and helps to lower the barrier to Test Driven SharePoint development a little bit more.   We will be updating our white paper to reflect this change and updating out code over the next few days.

Update 25 Nov 08:

You will also need to add the [ClearMocks] directive to the start of you Test Fixture to ensure Isolator clears up after each test.

[TestFixture]
[ClearMocks]
public class RandomMessage

Also one of the tests will now fail if you are using the new Isolator for SharePoint with the following error:

TypeMock.TypeMockException:

*** Can not fake non-SharePoint type UnitTest.ObjectModel.RandomMessage with current license. In order to fake non SharePoint types please upgrade to Typemock Isolator Commercial license

See more information at http://www.typemock.com/sharepointpage.php.

I will be following up with the Guys at Typemock to see how much room there will be to support non SharePoint objects in the SharePoint Edition for just these scenarios.