« VS2010 - New SharePoint Features
SPDisposeCheck - the tool every SharePoint developer needs »

Wednesday, November 12th, 2008

Unit Testing SharePoint - Getting into the Object Model

Posted in Agile  |  Comments


The white paper and associated code can be found here Unit Testing SharePoint Solutions - Getting into the Object Model

The first document in the Beginners Guide to Test Driven Web Part Development Series provided an introduction to unit testing, a look at how TDD influences your design and makes your code testable. That document deliberately avoided testing the SharePoint Object Model as in order to do this we need to introduce the concept of mocking.

This second document in the series will cover the concept of mocking, specifically looking at the new Typemock Isolator (V5.1.1 or later) AAA API to demonstrate testing against SPSite, SPWeb, SPList and SPListItem objects.

Some of the key things you should get from this white paper are

  • How to unit test the object model
  • These unit tests are much less brittle using the new Isolator AAA API
  • You do not need a specific SharePoint configuration on your development machine, we are not Integration testing
  • Reinforce TDD approach to SharePoint development

The white paper is part of the Beginners Series but introduces some fairly advanced concepts, as with SharePoint the learning curve can be fairly steep.  You can work through this white paper on it’s own but I recommend looking at the first part as this goes into more detail about TDD.

The white paper and associated code can be found here Unit Testing SharePoint Solutions - Getting into the Object Model

We welcome you comments and suggestion for improving the current white papers and also ideas for the future.

This entry was posted on Wednesday, November 12th, 2008 at 10:54 am and is filed under Agile. You can follow any responses to this entry through the RSS 2.0 feed. You can leave a response, or trackback from your own site.

Viewing 12 Comments

    • ^
    • v
    Hi,
    I’m reading your paper, and I found in page 19 your method GetRandomMessage. A first point there is that your Test Method is not testing the functionality of the method (it need to get back a text from a List Item), but it is testing if the site exists. That is not what you call “Unit Test”, or, at least, that is only a very, very partial Unit Test, and your assertion that “we are happy that we have covered the code sufficiently” is pure and simple not true. Then you add just a new catch statement, and, of course, your Unit Test will pass again, because you are NOT testing the method, you are testing if the site exists.
    That takes me to the second point: if you make a Unit Test method that really tests the functionality, and you are using the AAA API of TypeMocks, you need to tie it to the working-code: you need to make fake SPSite, SPWeb and SPList objects in the test-code. If you change your working code so that it uses a WebService, for example, to retrieve the message, your Unit Test will NOT work anymore, although the working-code is correct and working.
    Regards,
    Gustavo Velez [MVP MOSS]
    • ^
    • v
    Gustavo, great feedback - I'm not sure I agree with it entirly though.
    Your first comment about not testing the value that comes back from a List Item? That was the whole point of the first tests on page 16,17,18 using the Islolate.WhenCalled method to fake the actual call to SharePoint SPList[].Items[].Title. The additional tests regarding the exceptions on sites were to demonstrate how to test all of your code branches.
    My comments on 'covering the code sufficently' was deliberate, people have to make pragmatic judgements on when they feel they have done enough tests, and you will see from the last statement before the conclusion that we had in fact identified another test.

    Regarding the use of AAA API you are correct, my test code does have some brittleness to it, I would love to find a solution that negates this? However the AAA API makes the code far less brittle than it was previously. Your comment about using a WebService as an example of a code refactor that would break my tests. If the solution architecture was refactored in such a way I would expect some refactoring to take place, perhaps I would just mock out the WebService call? If I created the web service perhaps my tests would be valid but within the context of the WebService.

    If this were not a beginners guide I may have looked to introduce a ServiceRepository pattern to seperate out the tests against the SharePoint object model as was done in the Patterns and Practices example. But that would have confused the message.

    Andrew
    • ^
    • v
    Hi Andrew,
    Thanks for your clarifications. I’m glad that we agree in the identification of the weak points of refactoring, Unit Test with TypeMocks new API and SharePoint. For me, Unit Test gives me more than the certainty that the code is running good today; it gives me the guarantee that it will stay ok in six months, when we change the code… In six months, after the changes, I will run the tests again and be sure that nothing is broken. If I need to change the test code in parallel with the changes in the working code, the guarantee is gone…
    Regards your comment about the code in pages 16, 17 and 18, the code that you show there is not testing SharePoint code, you are testing your Welcome class that manipulates just a string; the techniques to make Unit Test for “normal” software, inclusive to make mocks and stubbers, are not new and very well known. The problems begin when we try to make Unit Test for servers with sealed classes and classes without public constructors, as SharePoint; that is also not a new problem, I’m writing about the theme since more than a year (“Mocks, Mocking, Mockers y SharePoint, http://geeks.ms/blogs/gvelez/archive/2007/08.aspx).
    Actually, my point is very simple: I am very, very afraid of BAD Unit Test… much more that of NO Unit Test at all. If I can make a bad parable, it is the same that we see if we go to an airport to catch a fly: you need to put your aftershave in a little transparent bag, and everybody has the beautiful feeling that we are safe because, of course, everybody can see the aftershave of everybody… If we develop Unit Test that doesn’t test the really working code, we will have the feeling that we are very smart and safe, because we are making Unit Test, and everybody knows that if you make Unit Test, you are at the safe side… but it is wrong, it is very wrong… The next bomb will be for sure not in an aftershave bottle, and the next bug will be for sure not in the catch statement …
    That is a point especially important for people that are beginning with Unit Test; you introduce your paper as “a beginners guide”, thus you need to show to the “beginner” how they can develop robust, reliable test code…
    Greetings,
    Gustavo Velez [MVP MOSS]
    • ^
    • v
    Gustavo,

    I think you are commenting on the first document in the series - the Welcome class was in Part 1.

    Please see Part 2 which includes the testing of the SharePoint Object model here: http://www.21apps.com/tdd-getting-into-sharepoi...

    This is a beginners guide to Testing SharePoint, I perhaps made an assumption wrongly that the reader had done sonme Unit Testing, I would welcome any suggestions on how to convey the the need to develop robust, reliable test code.

    Andrew
    • ^
    • v
    Gustavo,

    Commenting on the article http://geeks.ms/blogs/gvelez/archive/2007/08.aspx. Your lately and in the future testing are not Unit tests, these are Integration tests! You have a dependency on the SharePoint layer, database, web.config configuration, security etc - the list could go on in complex solutions. If developers are to adopt unit testing, to always get the latest code and re-run the tests before checking in then the unit tests need to run quicky, any interaction with SharePoint and it's database will slow this down, but will also turn the tests into Integration tests.

    On that subject I have absolutely no problem with leveraging tools like NUnit to do integration testing, but these run out of band on the build server. Lets not confuse Unit (Developer) testing with Integration testing.

    Andrew
    • ^
    • v
    Hi Andrew,
    My apologies for the mistake about the document: you are right; I was looking to the wrong file. And indeed, in the pages that you mention you are using SharePoint code, thus I retract my comment concerning that.
    About your second comment, well, formally speaking you are right, future testing it is not really Unit Test, it is more Regression Test (I like to see Integration Test more as how code behave in relation with other parts of the project, not in relation with the host [SharePoint in our case]). As I say in the article, I like to have different thinks in a Unit Test for SharePoint:
    - It needs to be repeatable: each developer in the team need to use the same test-set with the same results, also in the future
    - It needs to have no references to the actual state of SharePoint: otherwise it cannot be repeatable. Mocking is the only solution, I think. Your point about Integration Test is correct, Integration Test to the actual state of SharePoint is nonsense
    - It needs to be fast (as you mentioned): but in a bigger meaning, it needs to be faster to run, but it needs to be faster to code as well… the task of the developer is to write functional code; if you need to write 2 lines of test-code for each line of working-code, we are doing something wrong…
    By the way, I was wondering why do you not use the Test Tools of Visual Studio? (you use NUnit, I see it in your document)… have you found something that run better in NUnit than in VS? I’m using the latest version of TypeMocks with VS 2008 and its default Test Tools without any nuisance...
    Greetings
    Gustavo Velez [MVP MOSS]
    • ^
    • v
    Gustavo,
    I've updated the post as after noting you comment it was not obvious which white paper to download.
    No real reason for using NUnit over the test tools in Visual Studio just that's what I'm used to.

    I think we are really on the same page and look forward to talking to you more on the SPTypeMock project.
    • ^
    • v
    Hi Andrew,
    Yes, I agree, we are in the same wave… finally, the purpose is to make possible Unit Test for SharePoint, and you, the folk at TypeMocks and other couple of persons are making it achievable. Together with Carlos Segura (http://www.ideseg.com), other MVP MOSS, are we working very close with Gil Zilberfeld, Avi Kaye and Roy Osherove of TypeMocks to test the new versions, and trying to pull out the best of the tool… by the way, the things that they are making and that are now in the pipe-line are very exciting, in special for SharePoint, don’t think you so?…
    Gustavo Velez [MVP MOSS]
    • ^
    • v
    Thanks for this article it's great!

    I'm having a bit of a funny happening in my code and I was wondering if it's me or the the API.

    I've set up the following:
    // ----------------------------------------------------------
    SPList list = Isolate.Fake.Instance(Members.ReturnRecursiveFakes);
    SPListItem item = Isolate.Fake.Instance(Members.ReturnRecursiveFakes);

    Isolate.Swap.NextInstance().With(list);
    Isolate.Swap.NextInstance().With(item);

    Isolate.WhenCalled(() => list.GetItemById(1)).WillReturn(item);
    Isolate.WhenCalled(() => item["Customer Name"]).WillReturn("Bob Jones");
    Isolate.WhenCalled(() => item["NumberOfItems"]).WillReturn(2);
    // ----------------------------------------------------------
    However when I call item["Customer Name"] the mock object returns '2' instead of 'Bob Jones' it appears to be ignoring the field name set in the item[] and just returns the last value that was set with 'WhenCalled()'.

    Have I set this up incorrectly?
    Thanks
    • ^
    • v
    @Garth,

    This is indeed a missing feature in Typemock Isolator - we don't handle conditional expectations correctly with our AAA API. This feature is supported by our older APIs (see the discussion on the Isolator forums here: http://www.typemock.com/community/viewtopic.php...).

    We are working on extending the AAA API with more features like this one, so you can pretty safely assume you'll get it in an upcoming version.

    Doron (developer, Typemock)
    • ^
    • v
    Cool. Thanks for the response. I'm using Natural Mocks for this instance and it's working.

    Shotto
    • ^
    • v
    Here's some sample code of the Natural Mocks I'm using now (nothing worse than a posting that says I've got it working an no sample code ;) )

    /////////////////////////////////////////////////
    SPList list = RecorderManager.CreateMockedObject();
    SPListItem item = RecorderManager.CreateMockedObject();
    using (RecordExpectations recorder = RecorderManager.StartRecording())
    {
    recorder.ExpectAndReturn(list.GetItemById(1), item);

    recorder.ExpectAndReturn(item["ID"], 1);
    recorder.ExpectAndReturn(item["Customer Name"], "Bob Jones");
    recorder.ExpectAndReturn(item["NumberOfItems"], 2);

    }
 

Trackbacks

(Trackback URL)

close Reblog this comment
Powered by Disqus · Learn more
blog comments powered by Disqus