As part of the 21SCRUM development project I implemented build verification testing using Visual Studio 2010. 21SCRUM has a great suit of unit tests for the core logic, it has some integration tests (although I would like more) but I found a lot of the coding was very targeted at the User Interface. So much so that no real changes to the core logic have been made since the beta release.
The team did a number of Spike tests 1 to see if it was possible to do unit testing of the JavaScript code. Unfortunately nothing that was looked at really provided the confidence or ease of use needed to become a part of the development process.
I have flirted with automated UI testing for a long time, looking at WinRunner, Segue, QA Runner and various other tools along the way. I actually remember doing UI testing against a FoxPro application that used OCR technology to verify the text on the screen. The problem has always been that the tests are brittle, any slight changes to the UI and they break and it becomes a case of diminishing returns. However having failed to find a good solution to unit testing the UI code I decided to give the new Visual Studio 2010 tools a look.
My first impressions were very good, I actually changed the session I was doing at the SUGUK event in Wolverhampton due to being so impressed by the tools. This post is going to be one of a number I do covering lessons learned as the team develop our test suite using these tools for real on 21SCRUM.
The first lesson and one that you need to do before you start any UI testing is start from a known place.
Start from a known place
I quickly discovered that getting good, repeatable and maintainable automated UI tests to work over time you really have to make sure you always start at the same place. This means no additional browser windows open, no test web parts added to your pages, no left over’s from the last test or development.
One of the reasons I love Unit Testing is the isolation for any real environment or the need to go through a lot of setup and tear down steps.
In order to do this properly and repeatedly you need to have a script that will refresh your test location.
The script used for 21SCRUM is simple and does the following:
- - Deletes the test site collection
- - Create the test site collection based on required site template
- - Uploads the 21SCRUM sandboxed solution to the Solution Gallery
- - Activate the 21SCRUM solution
These steps ensure that when the tests are run they always start from a clean site collection. The script has been extended to provided sub sites and additional site collections but the changes are incremental and added as the tests demand.
Powershell
Like all good developers Powershell is the scripting language of choice. Actually kicked off with a good old fashioned .bat file << interested to know if there are any alternatives to this?
The batch file RefreshSite.Bat (powershell.exe is in my standard path)
powershell -File RefreshSite.ps1
The powershell script does the work
//Load the SharePoint Commandlets
Add-PSSnapin Microsoft.SharePoint.Powershell
//Delete the site collection – don’t prompt the user to confirm
Remove-SPSite -Identity “http://aberdovey.com” -Confirm:$false
//Create a new site collection based on the Team Site Template
Get-SPWebTemplate | Where{ $_.Title -eq “Team Site” } | ForEach-Object{ New-SPSite http://aberdovey.com –OwnerAlias DOMAIN\USER -Name “Aberdovey” -Template $_ }
//Add the solution to the solution Gallery – it must have the full path name and not a relative path
Add-SPUserSolution -LiteralPath C:\<full path to the location of the WSP>\21Scrum.Solution.wsp -Site http://aberdovey.com -Confirm:$false
// Activate the solution – and yes Microsoft have done it again with the naming! Install means Activate
Install-SPUserSolution -Identity 21Scrum.Solution -Site http://aberdovey.com -Confirm:$false
This simple command allows the refreshing of the environment to be run quickly and ensures the tests always start from a known place.
1 Spike Tests – short time-boxed pieces of work looking to prove an approach or providing additional knowledge to make estimates more accurate.