Automated UI Testing Tip – Automating deployment to Solutions Gallery

As part of the 21SCRUM development project I implemented build verification testing using Visual Studio 2010.  21SCRUM has a great suit of unit tests for the core logic,  it has some integration tests (although I would like more) but I found a lot of the coding was very targeted at the User Interface.  So much so that no real changes to the core logic have been made since the beta release.

The team did a number of Spike tests 1 to see if it was possible to do unit testing of the JavaScript code.  Unfortunately nothing that was looked at really provided the confidence or ease of use needed to become a part of the development process.

I have flirted with automated UI testing for a long time,  looking at WinRunner, Segue, QA Runner and various other tools along the way.  I actually remember doing UI testing against a FoxPro application that used OCR technology to verify the text on the screen.  The problem has always been that the tests are brittle, any slight changes to the UI and they break and it becomes a case of diminishing returns.  However having failed to find a good solution to unit testing the UI code I decided to give the new Visual Studio 2010 tools a look.

My first impressions were very good,  I actually changed the session I was doing at the SUGUK event in Wolverhampton due to being so impressed by the tools.  This post is going to be one of a number I do covering lessons learned as the team develop our test suite using these tools for real on 21SCRUM.  

The first lesson and one that you need to do before you start any UI testing is start from a known place.

Start from a known place

I quickly discovered that getting good, repeatable and maintainable automated UI tests to work over time you really have to make sure you always start at the same place.  This means no additional browser windows open,  no test web parts added to your pages, no left over’s from the last test or development.   

One of the reasons I love Unit Testing is the isolation for any real environment or the need to go through a lot of setup and tear down steps.

In order to do this properly and repeatedly you need to have a script that will refresh your test location.

The script used for 21SCRUM is simple and does the following:

  • - Deletes the test site collection
  • - Create the test site collection based on required site template
  • - Uploads the 21SCRUM sandboxed solution to the Solution Gallery
  • - Activate the 21SCRUM solution

These steps ensure that when the tests are run they always start from a clean site collection.   The script has been extended to provided sub sites and additional site collections but the changes are incremental and added as the tests demand.

Powershell

Like all good developers Powershell is the scripting language of choice.  Actually kicked off with a good old fashioned .bat file <<  interested to know if there are any alternatives to this?

The batch file RefreshSite.Bat  (powershell.exe is in my standard path)

powershell -File RefreshSite.ps1

The powershell script does the work

//Load the SharePoint Commandlets

Add-PSSnapin Microsoft.SharePoint.Powershell

//Delete the site collection – don’t prompt the user to confirm

Remove-SPSite -Identity “http://aberdovey.com” -Confirm:$false

//Create a new site collection based on the Team Site Template

Get-SPWebTemplate | Where{ $_.Title -eq “Team Site” } | ForEach-Object{ New-SPSite http://aberdovey.com –OwnerAlias DOMAIN\USER -Name “Aberdovey” -Template $_ }

//Add the solution to the solution Gallery – it must have the full path name and not a relative path

Add-SPUserSolution -LiteralPath C:\<full path to the location of the WSP>\21Scrum.Solution.wsp -Site http://aberdovey.com -Confirm:$false

// Activate the solution – and yes Microsoft have done it again with the naming!  Install means Activate

Install-SPUserSolution -Identity 21Scrum.Solution -Site http://aberdovey.com -Confirm:$false

 

This simple command allows the refreshing of the environment to be run quickly and ensures the tests always start from a known place.

 

1 Spike Tests – short time-boxed pieces of work looking to prove an approach or providing additional knowledge to make estimates more accurate.

This entry was posted in Development, SharePoint and tagged , , . Bookmark the permalink.
  • Chetan

    Automated Coded UI Testing Browser Compatibility issues and Unexpected Close problem in visual studio 2010

    Hello All,

    I am facing problem to test one of my website through Coded UI Testing. I am using MS Studio 2010. And I found problem during testing through Different Browser. Also sometimes facing the problem of Fatal Error and Unexpected Error need to restart studio 2010. Can you please help me out how to resolve this issue.

    I get this error frequently : fatal error : debugging information corrupt; recompile module” need to restart visual studio.

    Also any one knows how to do i create test for all browser like Firefox and Internet explorer

    Also Sometimes during automated Test. Its keep running which is never ending. Any one knows why this problem appears during testing ( Coded UI Testing for Web Site).

    I am using Visual Studio 2010. and testing one website through Automated Coded UI test and having some problems
    as per below

    Issue No : 1 How can i record the event for FireFox through Coded UI test. Cause When i try to record the Events from Firefox its record successfully. but when i tried to run that Automated Test its Giving Fatal Error : Debugging information Currupt. Recompile module” and giving message to restart the Visual studio 2010.

    Issue no 2 : Its working fine with Internet Explorer. but sometimes its keep running which is never end.

    Issue no 3 : Is there any way i create all test for Internet Explored and it will work with Firefox? Like i record events through Internet explorer for my website and after that any small change or function that will work with Firefox too.

    If any one solve it i will rewarded him 50 to 100 USD.

    contact : chetansabhaya@gmail.com

    Thanks

  • http://www.21apps.com AndrewWoody

    In answer to 1) – There is no option available to record the tests in Firefox. You have to use IE for web based recording.

    In answer to 2) there is a Beta plugin for Firefox that will allow the playback of scripts, recorded in IE, in Firefox. http://visualstudiogallery.msdn.microsoft.com/e

  • Chetansabhaya1

    Hello Andrew,

    Thank for your reply. But some times its unexectedly close the visual studio 2010. and say restart to visual studio during testing. Also sometimes it keep going in loop which never ends.

    Let me konw if you know something on this

    Thanks
    Chetan

  • http://www.facebook.com/ian.cosgrove Ian Cosgrove

    Wow a batch file, really? How is it being run? Can’t the same thing be accomplished with a task from the windows task scheduler?

  • http://www.21apps.com AndrewWoody

    Ian,

    Yheh a batch file – reality is that I want this to run as part of the Team City build process just like it run’s all the unit tests.

    At this point I don’t have this working, it’s on the todo list of process improvements though.

    Andrew