Agile Sharepoint development by 21apps and Andrew Woodward MVP

In this article and screen cast we will discuss the idea behind the CI process and focus on the technical tricks that we need to make in order to get this working for a SharePoint 2010 project using Team City and Subversion.

One of the first things that you will want to do for any development project is setup your Continuous Integrationprocess. The value of having a server automatically monitor your source code repository for changes and validating they work is immense

Its worth mentioning that doing Continuous Integration is not something you will want to leave until later in the project.  Waiting means the solution gets bigger and more complex which in turn make the setup more comples.   My recommendation:  Get this configured early, during Sprint 0  (i.e. before you write any real production code) if at all possible.

The basic CI Process

The diagram below shows the normal process that occurs.  

(1) a developer checks in some code changes to the source code repository.

(2) The CI server monitors the source code repository for any changes

When changes are detected the code is retrieved and the automated build is started.  This build will normally include running Unit, Integration and Build Verification tests.

(3) The success or failure of the automated build is reported back to the Team (including the developer).

In some teams breaking the build means the developer has a forfeit of some sort.  The idea is to encourage team work but also to discourage broken builds.

 

When the build breaks!

A term I read recently, ‘Stop-the-Line’,  is used in Toyota manufacturing process where any problem results in the production line being stopped until the cause of the problem is found, resolved and ideally prevented from re-occurring.    

This should be the same when a build breaks; the whole Team should stop and help get the build working again.  It is not just a problem for the developer who last checked in – the whole team need the process and need the builds to be working.

Setting Up in Sprint 0

The actual process described above is a little into the development cycle, when the build is in place and when the team are working on production code.   Before this can happen there needs to be a process put in place.  I refer to this initial as getting the strawman solution done.

definition: Strawman

In general, a strawman is an object, document, person, or argument that temporarily stands in for and is intended to be “knocked down” by something more substantial.    

source: Whatis.com

The aim is to take some code, which will be replaced hence the Strawman term through the CI process and get the team build notifications working.   You should aim to prove the successful builds but also what happens when the build fails.  Did the team get notified?  and did they react?

The Strawman Exercise

21apps is a small and geographically dispersed team,  we make extensive use of cloud based services for things like source control as they prove to be cost effective and flexible and our strawman  process tends to follow these steps:

1) Create a repository for the project  (Hosted Subversion)

2) Create the strawman project  (Simple ‘hello world’ web part)

3) Create a simple build script  – small steps mean a successful compile is our objective

4) Create a automated build configuration on our CI server – we use Team City

5) Prove the successful and failed builds

In this post we are focusing on step (4) setting up Team City as our CI server. 

To make this easier I have provided a quick video walkthrough on what we have done for the early builds of Project Aberdovey- the process will be very similar for your SharePoint 2010 solutions.

 

SharePoint 2010 – CI with TeamCity from Andrew Woodward on Vimeo.

 

Video Spotlights

Key areas that are highlighted in the video that you will enable you to build SharePoint 2010 projects in Team City.

If you are using MSBuild ensure you set the MSBuild version to Microsoft.Net Framework 3.5 and x64 run platform.

 

Force Team City to use MSBuild 4.0 as this is needed by adding the Environment Variable

Name: MSBuild

Reference Syntax: %env.MSBuild%

Value: %system.DotNetFramework4.0_x86_Path%

Firstly what is a SharePoint SPI?

No it’s not another new version, it’s a generic term that is used to describe a SharePoint Project Item within the context of a Visual Studio project.

Visual Studio 2010 has introduced a lot of great SharePoint specific features making it significantly easier to get started with SharePoint development. This post however is not about the tools, it is intended to point out one little bit of the hidden functionality that unfortunately didn’t get completed in time.

The Visual Studio tools hide away so of the configuration options in .spdata files. This is normally great as you can change these via the UI tools and everything works. However I discovered a small issue during a recent refactoring of a project.

When you create a new SharePoint SPI (for example a web part) that needs to add a reference to the assembly it updates the .spdata file with information (as below)

<?xml version="1.0" encoding="utf-8"?>
<ProjectItem Type="Microsoft.VisualStudio.SharePoint.WebPart"
   DefaultFile="MyWebpart.cs"
   SupportedTrustLevels="All"
   SupportedDeploymentScopes="Site"
   xmlns="http://schemas.microsoft.com/VisualStudio/2010/SharePointTools/SharePointProjectItemModel">
  <Files>
    <ProjectItemFile Source="Elements.xml" Target="MyWebpart\" Type="ElementManifest" />
    <ProjectItemFile Source="MyWebpart.webpart" Target="MyWebpart\" Type="ElementFile" />
  </Files>
  <SafeControls>
    <SafeControl Name="MyWebpart"
       Assembly="$SharePoint.Project.AssemblyFullName$"
       Namespace="_21apps.Sample.SPSolution" TypeName="*" IsSafe="true" />
  </SafeControls>
</ProjectItem>

 

In here you will see references to the ProjectItemFiles (.weppart and element manifest) and also an entry for the SafeControls.

If you look closely at the SafeControl entry you will see the Assembly is using a Token

Assembly=”$SharePoint.Project.AssemblyFullName$”

This is great as it allows the SafeControl to pickup the actual assembly name at compile time. The problem comes with the next part of the entry

Namespace=”_21apps.Sample.SPSolution”

As you can see this is not token based, which means that if you later decide to rename you namespace, and this is case sensitive, the SharePoint safe control entry will no longer be valid and you will get an error like the one below when you deploy the solution.

A Web Part or Web Form Control on this Web Part Page cannot be displayed or imported because it is not registered on this site as safe.

For more info on what this actually means have a look at Maurice Prather’s post from back in 2005, the information is still applicable today.

Unfortunately this namespace is unlikely to be made into a Token any time soon. Perhaps one for the community to pickup in the meantime. For now it is case of being aware that you need to correct this manually if you do change your namespace.

Being mainly focused on development aspects of SharePoint 2010 I haven’t spent as much time as I perhaps should have looking at the new features that have been added to make Administrators lives easier.  Having just upgraded one on my development machines from SharePoint 2010 Foundation to SharePoint 2010 Server for Internet Sites,  which on a single box installation with AD and SQL local went without a hitch, I thought it worth looking at what the Health Analyzer was telling me.

As you can see I have a few issues:

Security

The security Issue I can live with and I suspect everyone running the Beta has the same as using full admin accounts is the only way to get SharePoint working fully at this stage.

Performance

This is a dev box, so I am ok with SQL running on the same machine,  interesting that this is the only performance issue reported as the machine is a Hyper-V VM with 4 processors allocated and only 4GB RAM – but it seems to be running fine.

Configuration

3 items here – the second is about Email not configured – which is correct.  And again I have used built in accounts for the service identities as this was a single box install – I may fix these up at some point.

The 3rd item Missing server side dependencies caught my eye,  and I have to say the dialog that was displayed brought a smile to my face.

The server was reporting that I had referenced a web part TestJQuery.WebPart1.WebPart1 in two places and that the files to support these web parts were not on the system meaning the user will get an error.

This is awesome!

I know why these are here,  it was a spike test that I had done previously and had obviously forgot to clear it down.  What it does show is that the administrators (and developers on your own environments) can now proactively check for missing DLLs.

This was not one of the features I asked for in SharePoint 2010 – but is one that I am very glad exists. 

Availability

How many times have you been called in to find the logs have maxed out the drive space and SharePoint has crashed?   Here the health analyzer is telling me I may have an issue with space.    I’m ok with this,  I have a 40GB drive with 11GB free – I like to keep the drives around 40GB as I move then to my laptop and boot to VHD and this needs the full drive size as free space in order to boot – so I can let this one go as well.

The only thing left to say this is:

Thank you SharePoint Product Team.

I think this was a best practice in SharePoint 2007, but was one that you could get away with.  You have internal and external URLs that will be used to access the same web application.   Internally for historical or vanity reasons this is set as something like http://companyname, externally you provide access, often via a proxy like ISA Server, via https://extranet.companyname.com – and you do SSL termination at the ISA server as well.

You create the web application using the internal name,  get everything setup and then do the following to provide external access

• - Add an Alternative Access Mapping for https://extranet.companyname.com in the Internet Zone
• - Add another internal URL to the Internet Zone http://extranet.companyname.com
• - In IIS add a new binding to the web application so it picks up http://extranet.companyname.com
• - Publish the SharePoint site via ISA, with SSL access externally and no SSL internally

This scenario would work well,  you would cut down on the web.configs to be maintained – assuming you didn’t use features to do this for you, and you would have a slightly lower overhead on the server – which was often needed in the 32bit setups.

Problems in 2010  (and for some in 2007)

With the introduction of the client API, and a few other services, Microsoft is now making use of the WCF for its web services.   These offer lots and lots of advantages,  but have one limitation (at least in 3.5) that makes the above scenario fail. 

You cannot have multiple bindings on any IIS website

If you do set your environment up this way and you are using the client API you will hit problems.  Assuming you have coded you client requests with a fail delegate (like this code snippet)

  1: var lists;

  2: var context;

  3:

  4: function sample() {

  5:     context = new SP.ClientContext.get_current();

  6:

  7:     this.site = context.get_web();

  8:

  9:     context.load(this.site);

 10:

 11:     lists = site.get_lists();

 12:     context.load(lists);

 13:

 14:     context.executeQueryAsync(onPass, onFail);

 15: }

 16:

 17: function onFail(sender, args) {

 18:     alert("Fail:" + args.get_message() + '\n' + args.get_stackTrace());

 19: }

 20:

 21: function onPass(sender, args) {

 22:     onPass(lists.getByTitle('List'));

 23: }

when the async mehtod is called it will fail and the onFail method will show a 500 error.  One of the best things is the logging in 2010 has been improved so a quick look on the server and you should see something like the following in the event log:

Log Name:      Application

Source:        System.ServiceModel 3.0.0.0

Date:          31/12/2009 16:22:06

Event ID:      3

Task Category: WebHost

Level:         Error

Keywords:      Classic

User:          domain\sp-admin

Computer:      computername

Description:

WebHost failed to process a request.
 Sender Information: System.ServiceModel.ServiceHostingEnvironment+HostingManager/2389992
 Exception: System.ServiceModel.ServiceActivationException: The service '/_vti_bin/client.svc'
cannot be activated due to an exception during compilation.
The exception message is: This collection already contains an address with scheme http.
There can be at most one address per scheme in this collection.

Parameter name: item. ---> System.ArgumentException:
This collection already contains an address with scheme http.
There can be at most one address per scheme in this collection.

Parameter name: item

   at System.ServiceModel.UriSchemeKeyedCollection.InsertItem(Int32 index, Uri item)

   at System.Collections.Generic.SynchronizedCollection`1.Add(T item)

   at System.ServiceModel.UriSchemeKeyedCollection..ctor(Uri[] addresses)

   at System.ServiceModel.ServiceHost..ctor(Type serviceType, Uri[] baseAddresses)

   at System.ServiceModel.Activation.ServiceHostFactory.CreateServiceHost(Type
serviceType, Uri[] baseAddresses)

   at System.ServiceModel.Activation.ServiceHostFactory.CreateServiceHost(
String constructorString, Uri[] baseAddresses)

   at System.ServiceModel.ServiceHostingEnvironment.HostingManager.CreateService(
String normalizedVirtualPath)

   at System.ServiceModel.ServiceHostingEnvironment.HostingManager.ActivateService(
String normalizedVirtualPath)

   at System.ServiceModel.ServiceHostingEnvironment.HostingManager.EnsureServiceAvailable(
String normalizedVirtualPath)

   --- End of inner exception stack trace ---

   at System.ServiceModel.ServiceHostingEnvironment.HostingManager.EnsureServiceAvailable(
String normalizedVirtualPath)

   at System.ServiceModel.ServiceHostingEnvironment.EnsureServiceAvailableFast(String relativeVirtualPath)
 Process Name: w3wp
 Process ID: 3924

Event Xml:

<Event xmlns="http://schemas.microsoft.com/win/2004/08/events/event">

  <System>

    <Provider Name="System.ServiceModel 3.0.0.0" />

    <EventID Qualifiers="49154">3</EventID>

    <Level>2</Level>

    <Task>5</Task>

    <Keywords>0x80000000000000</Keywords>

    <TimeCreated SystemTime="2009-12-31T16:22:06.000000000Z" />

    <EventRecordID>99264</EventRecordID>

    <Channel>Application</Channel>

    <Computer>computername</Computer>

    <Security UserID="S-1-5-21-2646386180-1731120971-3670236442-1114" />

  </System>

  <EventData>

    <Data>System.ServiceModel.ServiceHostingEnvironment+HostingManager/2389992</Data>

    <Data>System.ServiceModel.ServiceActivationException: The service '/_vti_bin/client.svc'
cannot be activated due to an exception during compilation.
The exception message is: This collection already contains an address with scheme http.
There can be at most one address per scheme in this collection.

Parameter name: item. ---> System.ArgumentException:
This collection already contains an address with scheme http.
There can be at most one address per scheme in this collection.

Parameter name: item

   at System.ServiceModel.UriSchemeKeyedCollection.InsertItem(Int32 index, Uri item)

   at System.Collections.Generic.SynchronizedCollection`1.Add(T item)

   at System.ServiceModel.UriSchemeKeyedCollection..ctor(Uri[] addresses)

   at System.ServiceModel.ServiceHost..ctor(Type serviceType, Uri[] baseAddresses)

   at System.ServiceModel.Activation.ServiceHostFactory.CreateServiceHost(Type serviceType,
Uri[] baseAddresses)

   at System.ServiceModel.Activation.ServiceHostFactory.CreateServiceHost(String constructorString,
Uri[] baseAddresses)

   at System.ServiceModel.ServiceHostingEnvironment.HostingManager.CreateService(String normalizedVirtualPath)

   at System.ServiceModel.ServiceHostingEnvironment.HostingManager.ActivateService(String normalizedVirtualPath)

   at System.ServiceModel.ServiceHostingEnvironment.HostingManager.EnsureServiceAvailable(
String normalizedVirtualPath)

   --- End of inner exception stack trace ---

   at System.ServiceModel.ServiceHostingEnvironment.HostingManager.EnsureServiceAvailable(
String normalizedVirtualPath)

   at System.ServiceModel.ServiceHostingEnvironment.EnsureServiceAvailableFast(
String relativeVirtualPath)</Data>

    <Data>w3wp</Data>

    <Data>3924</Data>

  </EventData>

</Event>

 

The problem, although not obvious from the error message is related to the WCF framework not being able to work with multiple names on the same schema (where schema means HTTP).

Solution

The solution to this is to Extend the web application to another IIS web site.  It will make use of the same application pool but is likely to add a small overhear to the server.  The biggest challenge is to ensure that you get your deployments automated so that SharePoint can keep the web.config’s inline.   And with the introduction of the Sandboxed solution the other issues with artifacts deployed to the 80 (inetpub) folder will likely go away over time.

Many people have used the the Content Editor WebPart to provide an easy way to inject some JavaScript into a web page to do things like hide the left navigation or show snowflakes. 

In SharePoint 2010 the good old Content Editor WebPart is still around and even has some nice new features like format as XHTML,  however I discovered one feature that I suspect is more of a bug – or perhaps an undocumented feature

Scenario

You have used the content editor web part to put some placeholders on the page,  a couple of DIV tags and the like.   You have another web part (it could be the same one) that does some funky Client API calls to get some information to render in those place holders <  all looking good for a simple no code solution so far.

Now you decide that you want to copy you handy work to another site,  this is just a quick ‘Information Worker’ hack so no need for any Visual Studio or WSPs here.

Edit the page and select the content editor web part,  the one that has has some content dynamically added.   Click on the HTML > Edit HTML Source option and da-dah the dynamic content is included in the source.

Upshot

This could be a great thing if you wanted to get at all of the code,  but I suspect this is not what you expected.  The problem is that if you now make a minor change – say you wanted to add something and click save,  you will hard code whatever values had been added.

I would prefer the Content Editor Web Part source Editor to ONLY show me what I has actually added to it in the web part properties.   I suspect the reason for the bug is likely someone trying to reduce the calls to the server.

Serious Problem

The problem is actually quite serious as you only need to visit the page,  click Edit Page and then Stop Editing for it to update the content.   This really does mean that you will have to stop using the CEWP in this way at least for the Beta.

Disclaimer:  I don’t work for Mindsharp or Combined Knowledge, but I do guest teach the developer course.

Had a great week attending Combined Knowledge’s first public beta of the SharePoint 2010 developer course,  in fact it was the first running of this course anywhere in the world!

The course was held in what I think is a great venue,  the board room in Ullesthorpe – I really like teaching here as this is a permanent setup so you know you have everything you need including lots of white boards << these are essential.  This week was slightly different on the food front as they were into the December Xmas menu,  no longer did we get the famous ‘Rice and Chips’ that Todd Bleeker is so keen on – instead we got a full choice from the bar menu,  I recommend the chicken and warm bacon salad starter.

Back to the course; you might be asking  ‘Why this course and not the Ignite Training?’ 

I have to say I am very impressed with Microsoft in the amount of information and training being provided this early in the delivery cycle,  if you compare this to what we had for 2007 and even worse what we didn’t have in 2003 you will know that they have really put the effort in.

So why did I not do the Ignite Training?  – Simple answer could be I missed the boat; and this is not untrue (double negative to make it sound better :S), but the real answer is that I am planning to guest teach this course for Combined Knowledge and it is a great way for me to feedback on what does/doesn’t work from an attendee’s perspective and help shape the course that people will get over the coming years  (the 2003 course is still being taught, so anything fixed now has a long time to live).

The course was being taught by Gary Yeoman,  long time trainer with Combined Knowledge and real world consultant when not training.  Attending the course were myself,  James Fisk, Adrian, David, Ian, Stephan (from Switzerland), Rehan and Ruth – plus we had the honour of having Todd Bleeker (the course author) taking a very active role throughout the week.

Big Topic or Lots of Topics

SharePoint in 2007 was a big product,  sure Windows SharePoint Services (WSS V3) wasn’t quite as big as MOSS but from a developers perspective there was a lot to cover.  Things in 2010 just get bigger.   To quote Todd  (and it may not have been word for word)

Custom Field types were probably the hardest thing you had to do in SharePoint 2007.  In 2010 there are lots of things that are more complicated.  however the great tools in Visual Studio 2010 help here a lot!

What I’m trying to say is we had a lot to cover and Gary was adopting a JIT based approach to some of the slides

If you skip the sleeping!

There are 24hours in a day (8 for work, 8 for sleep and 8 for yourself).  Based on this schedule there really is no way that there would be a beta SharePoint 2010 developer course with the depth of information and labs that people expect of a Combined Knowledge course for people to attend.   So how was this solved?

Well if your Todd Bleeker you make the call that you can adopt the following pattern 6 x 1/2 hour sleep and 21 hours work (per day, everyday)!   Yes Todd has been doing the Uberman sleep pattern (a form of polyphasic sleep) for the past 17 weeks – and he is amazingly still sane and took a very active role during the week.  Although on one occasion we returned after a break to see a pair of feet sticking out from under a desk which were attached to Todd fitting in one of his 1/2 hour sleeps! 

It was suggested at the start of the course that every student should also adopt this if they were to be able to get through the content we wanted to cover,  thankfully other attendees like to have some ‘down’ time to recuperate. 

‘Down’ time == ‘SharePint’ 

The Course

Being a beta course, and the first real proper teach ever, and based on the Beta 2 release that only dropped a few weeks ago it was expected that the labs would be a little rough and ready.   This couldn’t have been more wrong! The new format adopted with microlabs,improved consistency and the result of many hour of hard work has paid off in droves.  

We were all experienced SharePoint 2007 developers so we could skip any of the usual ‘What’s an SPWeb’ type discussion and head straight into some deep dives.

The course covered a lot of detail including

• - Powershell and how to extend it
• - A tour of VS2010 and the new SharePoint tools
• - Extending VS2010
• - Event Models – with focus on what’s new and improved
• - Deployment scenarios
• - Sandboxed and Farm Solutions
• - Workflow
• - SharePoint Designer <<   YES SharePoint Designer is really a useful dev tool in 2010
• - Developer Dashboard
• - and many many more

What stood out?

In truth SharePoint 2010 was the real star, including the great new tools in VS2010 (although the very poor story around Unit Testing is tainting my views here).

What I did get at the end of the course was a feeling that I just want to get out and start teaching it,  to start sharing the great new things that caused me so many long days in the past.  

I am looking forward to when I can run my first class and spend time working through your ideas, problems and solutions.

I’m excited about developing for SharePoint 2010. 

If I can offer one tip for now,   ‘always start with a Sandboxed solution first’