Agile Sharepoint development by 21apps and Andrew Woodward MVP

Normally I refrain for doing simple reposts of other people blogs,  however having read the post SharePoint’s Sasquatch Memory Leak by Todd Carter I just had to help get the information out to as many people as possible.

We are all being good developers and running SPDisposeCheck regularly on our code to make sure we don’t leak memory,  however for some we still experience spiralling memory leaks that seem untraceable.  until Todd’s post that is.

Rather than hang around here you MUST read it now,  and if your experiencing issues and have SPDisposeCheck’d your application to within an inch of it’s life with no success on controlling memory you will want to implement the workaround as soon as possible.

Big Thanks to Todd find Big Foot

Firstly what is a SharePoint SPI?

No it’s not another new version, it’s a generic term that is used to describe a SharePoint Project Item within the context of a Visual Studio project.

Visual Studio 2010 has introduced a lot of great SharePoint specific features making it significantly easier to get started with SharePoint development. This post however is not about the tools, it is intended to point out one little bit of the hidden functionality that unfortunately didn’t get completed in time.

The Visual Studio tools hide away so of the configuration options in .spdata files. This is normally great as you can change these via the UI tools and everything works. However I discovered a small issue during a recent refactoring of a project.

When you create a new SharePoint SPI (for example a web part) that needs to add a reference to the assembly it updates the .spdata file with information (as below)

<?xml version="1.0" encoding="utf-8"?>
<ProjectItem Type="Microsoft.VisualStudio.SharePoint.WebPart"
   DefaultFile="MyWebpart.cs"
   SupportedTrustLevels="All"
   SupportedDeploymentScopes="Site"
   xmlns="http://schemas.microsoft.com/VisualStudio/2010/SharePointTools/SharePointProjectItemModel">
  <Files>
    <ProjectItemFile Source="Elements.xml" Target="MyWebpart\" Type="ElementManifest" />
    <ProjectItemFile Source="MyWebpart.webpart" Target="MyWebpart\" Type="ElementFile" />
  </Files>
  <SafeControls>
    <SafeControl Name="MyWebpart"
       Assembly="$SharePoint.Project.AssemblyFullName$"
       Namespace="_21apps.Sample.SPSolution" TypeName="*" IsSafe="true" />
  </SafeControls>
</ProjectItem>

 

In here you will see references to the ProjectItemFiles (.weppart and element manifest) and also an entry for the SafeControls.

If you look closely at the SafeControl entry you will see the Assembly is using a Token

Assembly=”$SharePoint.Project.AssemblyFullName$”

This is great as it allows the SafeControl to pickup the actual assembly name at compile time. The problem comes with the next part of the entry

Namespace=”_21apps.Sample.SPSolution”

As you can see this is not token based, which means that if you later decide to rename you namespace, and this is case sensitive, the SharePoint safe control entry will no longer be valid and you will get an error like the one below when you deploy the solution.

A Web Part or Web Form Control on this Web Part Page cannot be displayed or imported because it is not registered on this site as safe.

For more info on what this actually means have a look at Maurice Prather’s post from back in 2005, the information is still applicable today.

Unfortunately this namespace is unlikely to be made into a Token any time soon. Perhaps one for the community to pickup in the meantime. For now it is case of being aware that you need to correct this manually if you do change your namespace.

On Thursday, 28th Jan 2010, I made my way down to the London SUGUK event.  I’ve been supporting the regional events and been unable to get down to London for a number of months.   The number of attendees was amazing – I didn’t count be I guess we have over 150 people and filled the room at Microsoft.

I did know quite a few regulars to the event,  but the great news again is the number of new faces.  SharePoint is really growing fast and wide and the need for information and community is growing with it.

Big Kudos to Matt for organising the event,  Matt has taken on the responsibilities for the South East and London,  to Ben Robb for doing a SharePoint Designer 2010 session  (although as he points out the whole Application Lifecycle Management is still not baked) and for Steve Smith for – umm not sure what Steve did

It was great to see Mike Fitzmaurice there  (this man made SharePoint happen),  and I suspect his presence on the panel was part of the reason the questions and discussion remained in the ITPro and User space.   Myself, Chris O’Brien, Matt Smith and Ben Robb tried hard to put a dev take on any questions where we could.

What I think would go down really well is to take a lead from the Nottingham group and organise focused events,  so we could have a Dev night, Design Night, End User night etc.

I have to apologies (thanks Alex Pearce for tweeting me this when we were in the Pub) – I did not advertise the #SharePint that took place afterwards, It was well attended and with more room than the pub we used to go to but not an ‘official’ #SharePint.

Hope to see you all at an event near you soon.

Being mainly focused on development aspects of SharePoint 2010 I haven’t spent as much time as I perhaps should have looking at the new features that have been added to make Administrators lives easier.  Having just upgraded one on my development machines from SharePoint 2010 Foundation to SharePoint 2010 Server for Internet Sites,  which on a single box installation with AD and SQL local went without a hitch, I thought it worth looking at what the Health Analyzer was telling me.

As you can see I have a few issues:

Security

The security Issue I can live with and I suspect everyone running the Beta has the same as using full admin accounts is the only way to get SharePoint working fully at this stage.

Performance

This is a dev box, so I am ok with SQL running on the same machine,  interesting that this is the only performance issue reported as the machine is a Hyper-V VM with 4 processors allocated and only 4GB RAM – but it seems to be running fine.

Configuration

3 items here – the second is about Email not configured – which is correct.  And again I have used built in accounts for the service identities as this was a single box install – I may fix these up at some point.

The 3rd item Missing server side dependencies caught my eye,  and I have to say the dialog that was displayed brought a smile to my face.

The server was reporting that I had referenced a web part TestJQuery.WebPart1.WebPart1 in two places and that the files to support these web parts were not on the system meaning the user will get an error.

This is awesome!

I know why these are here,  it was a spike test that I had done previously and had obviously forgot to clear it down.  What it does show is that the administrators (and developers on your own environments) can now proactively check for missing DLLs.

This was not one of the features I asked for in SharePoint 2010 – but is one that I am very glad exists. 

Availability

How many times have you been called in to find the logs have maxed out the drive space and SharePoint has crashed?   Here the health analyzer is telling me I may have an issue with space.    I’m ok with this,  I have a 40GB drive with 11GB free – I like to keep the drives around 40GB as I move then to my laptop and boot to VHD and this needs the full drive size as free space in order to boot – so I can let this one go as well.

The only thing left to say this is:

Thank you SharePoint Product Team.

Having completed the migration to Microsoft Business Productivity Online Suite (BPOS) on Christmas Eve, I thought it would be useful to post a few comments on how it has worked out in the first few weeks.  I’ll start with a few of the disappointments first and then move on to the things that I have liked.

Disappointments

My biggest disappointment has to be with Office Communicator 2007 R2.

Unable to do Video or Voice calls

With Office Communicator you cannot do Voice or Video calls unless all of the machines in the call are on the same LAN!   Really this is a major fail for something that is supposed to be cloud based – I mean you can do this in Messenger with no problem.

Also the Communicator Client continues to report an error stating that Communicator cannot determine the Exchange Web Services – so it won’t be able to automatically report if you are in a meeting or out of the office.

Effectively this makes Office Communicator almost useless as it offers less functionality than that provided by Messenger or Skype.

No Links

I also find the policy of blocking any URLs to be a bit over the top,  they should provide a user defined option to allow or not rather than completely block them.   Having to revert to Email to send a reference link mid conversation is really silly.

A better way solution?

I have looked for a beta of Communicator 2010 to see if this will solve the problem,  but it appears that there is little or no information out there on this. 

Having migrated from an Exchange only Environment the failings of Communicator are a disappointment,  but there are a number of other benefits to BPOS that do make it worth the move.

Things I Liked

Single Sign-On

The single sign on client, that also configures you applications, has to be one of the best features.  The ability to be able to have automatic sign on to your applications regardless of where you are is brilliant.  And the fact the sign-on client now also works on the Mac is a real bonus to those that have a mixed environment.

SharePoint

As you may be aware I am a big fan of SharePoint,  I have always looked at affordable ways to get a hosted solution but have found the costs to be very high with minimal storage space.   With the BPOS account you get to create a number of SharePoint site collections (all WSS based) and can allocate storage space as needed.   During the trial you get 5GB (20 users x 0.25GB) although not a significant amount should allow you to use it as a collaboration platform.

The biggest limitation with SharePoint is the core platform and the ability to customise this,  in 2010 things will be much better with the introduction of the Sandboxed solutions – but there are some great examples of doing ‘code-less’ deployments today, look at what the guys at Zeven Seas have done with ProduShare.

Live Meeting

Having failed to get video and voice going with Communicator we tried using LiveMeeting as an alternative,  sure it is perhaps a little over the top for a quick conf call – but actually we found it to be very useful.   I think when communicator is working right, you should be able to promote you call to a live meeting,  for now we will likely do this manually.

Live Meeting is the same product it always has been, but now in the BPOS package you get a license for everyone to be able to create meetings and can invite up to 15 people with the standard offering – which is enough for most collaborative meetings.  I am planning to look at pricing for upgrades to be able to do web casts – the professional offering supports up to 1250 people.

Live meeting proved to be a fantastic solution for us to do our sprint planning for Project Aberdovey – the team had met for a project kickoff day before Christmas and had worked through the Sprint 0 requirements ready for the first Sprint to start on 4th Jan.   Using Live Meeting, with web cams, white board and shared applications we were able to complete our Sprint planning game – decided what was in/out of the first sprint and agree and final adjustment to the story sizing.  

Live Meeting I feel really is the main solution to our initial challenges around communication,  it allows for very rich interaction and provides the quality of service that we need. 

So far so good

Overall I have been impressed with the BPOS offerings,  I have noticed no performance issues or service outages and have found that the value really is in the overall package.

I would recommend BPOS to people moving their email onto the cloud or for those looking for a more feature rich offering.

I think this was a best practice in SharePoint 2007, but was one that you could get away with.  You have internal and external URLs that will be used to access the same web application.   Internally for historical or vanity reasons this is set as something like http://companyname, externally you provide access, often via a proxy like ISA Server, via https://extranet.companyname.com – and you do SSL termination at the ISA server as well.

You create the web application using the internal name,  get everything setup and then do the following to provide external access

• - Add an Alternative Access Mapping for https://extranet.companyname.com in the Internet Zone
• - Add another internal URL to the Internet Zone http://extranet.companyname.com
• - In IIS add a new binding to the web application so it picks up http://extranet.companyname.com
• - Publish the SharePoint site via ISA, with SSL access externally and no SSL internally

This scenario would work well,  you would cut down on the web.configs to be maintained – assuming you didn’t use features to do this for you, and you would have a slightly lower overhead on the server – which was often needed in the 32bit setups.

Problems in 2010  (and for some in 2007)

With the introduction of the client API, and a few other services, Microsoft is now making use of the WCF for its web services.   These offer lots and lots of advantages,  but have one limitation (at least in 3.5) that makes the above scenario fail. 

You cannot have multiple bindings on any IIS website

If you do set your environment up this way and you are using the client API you will hit problems.  Assuming you have coded you client requests with a fail delegate (like this code snippet)

  1: var lists;

  2: var context;

  3:

  4: function sample() {

  5:     context = new SP.ClientContext.get_current();

  6:

  7:     this.site = context.get_web();

  8:

  9:     context.load(this.site);

 10:

 11:     lists = site.get_lists();

 12:     context.load(lists);

 13:

 14:     context.executeQueryAsync(onPass, onFail);

 15: }

 16:

 17: function onFail(sender, args) {

 18:     alert("Fail:" + args.get_message() + '\n' + args.get_stackTrace());

 19: }

 20:

 21: function onPass(sender, args) {

 22:     onPass(lists.getByTitle('List'));

 23: }

when the async mehtod is called it will fail and the onFail method will show a 500 error.  One of the best things is the logging in 2010 has been improved so a quick look on the server and you should see something like the following in the event log:

Log Name:      Application

Source:        System.ServiceModel 3.0.0.0

Date:          31/12/2009 16:22:06

Event ID:      3

Task Category: WebHost

Level:         Error

Keywords:      Classic

User:          domain\sp-admin

Computer:      computername

Description:

WebHost failed to process a request.
 Sender Information: System.ServiceModel.ServiceHostingEnvironment+HostingManager/2389992
 Exception: System.ServiceModel.ServiceActivationException: The service '/_vti_bin/client.svc'
cannot be activated due to an exception during compilation.
The exception message is: This collection already contains an address with scheme http.
There can be at most one address per scheme in this collection.

Parameter name: item. ---> System.ArgumentException:
This collection already contains an address with scheme http.
There can be at most one address per scheme in this collection.

Parameter name: item

   at System.ServiceModel.UriSchemeKeyedCollection.InsertItem(Int32 index, Uri item)

   at System.Collections.Generic.SynchronizedCollection`1.Add(T item)

   at System.ServiceModel.UriSchemeKeyedCollection..ctor(Uri[] addresses)

   at System.ServiceModel.ServiceHost..ctor(Type serviceType, Uri[] baseAddresses)

   at System.ServiceModel.Activation.ServiceHostFactory.CreateServiceHost(Type
serviceType, Uri[] baseAddresses)

   at System.ServiceModel.Activation.ServiceHostFactory.CreateServiceHost(
String constructorString, Uri[] baseAddresses)

   at System.ServiceModel.ServiceHostingEnvironment.HostingManager.CreateService(
String normalizedVirtualPath)

   at System.ServiceModel.ServiceHostingEnvironment.HostingManager.ActivateService(
String normalizedVirtualPath)

   at System.ServiceModel.ServiceHostingEnvironment.HostingManager.EnsureServiceAvailable(
String normalizedVirtualPath)

   --- End of inner exception stack trace ---

   at System.ServiceModel.ServiceHostingEnvironment.HostingManager.EnsureServiceAvailable(
String normalizedVirtualPath)

   at System.ServiceModel.ServiceHostingEnvironment.EnsureServiceAvailableFast(String relativeVirtualPath)
 Process Name: w3wp
 Process ID: 3924

Event Xml:

<Event xmlns="http://schemas.microsoft.com/win/2004/08/events/event">

  <System>

    <Provider Name="System.ServiceModel 3.0.0.0" />

    <EventID Qualifiers="49154">3</EventID>

    <Level>2</Level>

    <Task>5</Task>

    <Keywords>0x80000000000000</Keywords>

    <TimeCreated SystemTime="2009-12-31T16:22:06.000000000Z" />

    <EventRecordID>99264</EventRecordID>

    <Channel>Application</Channel>

    <Computer>computername</Computer>

    <Security UserID="S-1-5-21-2646386180-1731120971-3670236442-1114" />

  </System>

  <EventData>

    <Data>System.ServiceModel.ServiceHostingEnvironment+HostingManager/2389992</Data>

    <Data>System.ServiceModel.ServiceActivationException: The service '/_vti_bin/client.svc'
cannot be activated due to an exception during compilation.
The exception message is: This collection already contains an address with scheme http.
There can be at most one address per scheme in this collection.

Parameter name: item. ---> System.ArgumentException:
This collection already contains an address with scheme http.
There can be at most one address per scheme in this collection.

Parameter name: item

   at System.ServiceModel.UriSchemeKeyedCollection.InsertItem(Int32 index, Uri item)

   at System.Collections.Generic.SynchronizedCollection`1.Add(T item)

   at System.ServiceModel.UriSchemeKeyedCollection..ctor(Uri[] addresses)

   at System.ServiceModel.ServiceHost..ctor(Type serviceType, Uri[] baseAddresses)

   at System.ServiceModel.Activation.ServiceHostFactory.CreateServiceHost(Type serviceType,
Uri[] baseAddresses)

   at System.ServiceModel.Activation.ServiceHostFactory.CreateServiceHost(String constructorString,
Uri[] baseAddresses)

   at System.ServiceModel.ServiceHostingEnvironment.HostingManager.CreateService(String normalizedVirtualPath)

   at System.ServiceModel.ServiceHostingEnvironment.HostingManager.ActivateService(String normalizedVirtualPath)

   at System.ServiceModel.ServiceHostingEnvironment.HostingManager.EnsureServiceAvailable(
String normalizedVirtualPath)

   --- End of inner exception stack trace ---

   at System.ServiceModel.ServiceHostingEnvironment.HostingManager.EnsureServiceAvailable(
String normalizedVirtualPath)

   at System.ServiceModel.ServiceHostingEnvironment.EnsureServiceAvailableFast(
String relativeVirtualPath)</Data>

    <Data>w3wp</Data>

    <Data>3924</Data>

  </EventData>

</Event>

The problem, although not obvious from the error message is related to the WCF framework not being able to work with multiple names on the same schema (where schema means HTTP).

Solution

The solution to this is to Extend the web application to another IIS web site.  It will make use of the same application pool but is likely to add a small overhear to the server.  The biggest challenge is to ensure that you get your deployments automated so that SharePoint can keep the web.config’s inline.   And with the introduction of the Sandboxed solution the other issues with artifacts deployed to the 80 (inetpub) folder will likely go away over time.

Many people have used the the Content Editor WebPart to provide an easy way to inject some JavaScript into a web page to do things like hide the left navigation or show snowflakes. 

In SharePoint 2010 the good old Content Editor WebPart is still around and even has some nice new features like format as XHTML,  however I discovered one feature that I suspect is more of a bug – or perhaps an undocumented feature

Scenario

You have used the content editor web part to put some placeholders on the page,  a couple of DIV tags and the like.   You have another web part (it could be the same one) that does some funky Client API calls to get some information to render in those place holders <  all looking good for a simple no code solution so far.

Now you decide that you want to copy you handy work to another site,  this is just a quick ‘Information Worker’ hack so no need for any Visual Studio or WSPs here.

Edit the page and select the content editor web part,  the one that has has some content dynamically added.   Click on the HTML > Edit HTML Source option and da-dah the dynamic content is included in the source.

Upshot

This could be a great thing if you wanted to get at all of the code,  but I suspect this is not what you expected.  The problem is that if you now make a minor change – say you wanted to add something and click save,  you will hard code whatever values had been added.

I would prefer the Content Editor Web Part source Editor to ONLY show me what I has actually added to it in the web part properties.   I suspect the reason for the bug is likely someone trying to reduce the calls to the server.

Serious Problem

The problem is actually quite serious as you only need to visit the page,  click Edit Page and then Stop Editing for it to update the content.   This really does mean that you will have to stop using the CEWP in this way at least for the Beta.

Microsoft really do appear the be much better at supporting alternative platforms,  the most obvious being the Mac.  As part of the Business Productivity Online Suite (BPOS) they have recently released a single sign on application that will work with Mac OS.

 More details and download from Microsoft.

The Microsoft Online Services Team Blog has some information showing how the application works and some limitations. The most notable limitations are

• - SSO only works with Safari 4 being default browser
• - Communicator is not currently available for the Mac

I managed to get the application installed and working on the Mac within a few minutes,  the biggest issue I had was being unable to sign in and continually being prompted to say I had the wrong user name and password.   After a couple of resets I found the solution.

Wrong username or password

If you have confirmed the username and password are correct but you continually get the login rejected,  ensure that you have installed the application correctly.   When you open the package you need to drag this into the applications folder for it to work. 

Once signed in setting up Entourage was a single click process, very nice

I suspect I am not the only on to have been seduced by the ’shiny shiny’ ** – Setting things up on the iPhone is actually pretty simple,  the hardest part was finding the server URL .

Server URL for EMEA: red002.mail.emea.microsoftonline.com

I suspect you can guess the one for the USA, but to be sure you can get this by going to Accounts in Outlook,  select your BPOS account and choose Change. 

You will see a screen similar to this, and you will see that the Microsoft Exchange server is pointing to a local server – this is how they get the single sign on to work.

Click on More Settings -> Choose the Connection Tab -> Click on the Exchange Proxy Settings.

This will show you the actual server URL that you will need to use on the iPhone.

On the iPhone

Choose Settings -> Mail, Contacts, Calendars > Add Account.

Select Microsoft Exchange  (if you already have one setup you will need to delete it first as you can only connect to one exchange account).

Enter your email address, the username is also the email address and your password.   No need to enter a domain so leave it blank.

Click next and accept the certificate error.

Enter the server name (see above), excluding the https:// prefix

Click next and the account should be verified and you are good to go.

One of the great things about BPOS is that ActiveSync is included by default on all accounts,  previously I had to pay an additional fee for every account.

** Frankie Boyle – Look at the Shiny Shiny  (Please note Frankie Boyle is likely to offend)

A short post about my experiences doing my company email migration.  A project that went from original requirement on 21st Dec to being fully up and running on the new platform by 24th Dec.   The journey takes in a hunt for a for dns services, some tweets, a few false starts and a bit of Amazon EC2 for good measure.  Before starting it’s worth explaining what BPOS is for those that are not familiar.

What is BPOS?

Microsoft’s Business Productivity Online Suite is Microsoft’s first big effort at providing online services aimed at small to medium businesses.  Looking at what you get for your money it really is a very compelling solution.

With the recent (Nov 09) price drop in the UK from £10.04 to just £6.71 per user per per month you get hosted solutions for Exchange, SharePoint (WSS), Live Meeting and Office Communications (OCS)  all based on a secure, high availability platform.   There really is no way that a small/medium sized company could possible provide this level of functionality in house – I suspect the licenses are probably more expensive on their own.

Why did 21apps move?

Having recently grown the company, and being good agile practitioners (some like call it adaptive), we did a retrospective on Monday (21st Dec) looking at how the past 6 weeks had gone,  were there things that we could do better and how could we improve our own processes.  I will do a separate post on this, but for this post the problem area we needed to improve was Lines of Communication.

Area for improvement: Lines of Communication

We had been flipping between numerous platforms Skype, msn messenger, team viewer, mobile, landline, mesh folders, email, struggling with hotel broadband and clients firewalls etc, etc.

We needed a clear approach that was consistent and provided us with the ability to know when people were available an how they could be contacted.  And once contacted how to make the communication better.

Actions

Review BPOS as this looked to offer solutions for most of the issues around consistency in a single solution.

Get new phones (iPhones ) on the Orange network with tethering (remove dependency on hotel broadband)
* Those that follow me on twitter will know the hassle I have had with O2’s useless service and network coverage.  We’re not covering the iPhone bit here.

Looking at BPOS

21apps had been hosting email with Intermedia on Exchange 2007 since they first started offering it about 3 years ago.  The service has been great and I have had no issues.  They are also providing a BPOS equivalent (although only US based at time of writing, expected in the UK in Jan) – which uses Exchange 2010 (Microsoft don’t have this yet) but no Office Live Meeting.  The price is also slightly more for there offering.

The question was,  do I give the Microsoft offering a go or stick with Intermedia? 

Confession:  I had played with BPOS before as part of the beta program,  I like the concepts behind it so this is really a good time to see if it works for real and to be able to feedback directly with real world issues.   So I am doing this for everyone not just me

Signing Up

No win no fee, as they say a lot these days.  Microsoft are offering a 30 day trial with up to 20 users so you really have nothing to loose  (exception for some hair, sleep and hours in your day – but we’ll get to that later).

The sign up process was painless,  I blogged in more detail during my beta trial and basically followed the same steps

• Sign in with Live Id
• Create account
• Setup the various services

Microsoft have done a good job of providing you with detailed instructions on how to do things.   If you following them fully you should have very few issues – although I did find the amount of information tended to hide some of the key messages, this was likely down to me skipping forward.

Migrating Mailboxes

I knew that this was going to be the hardest thing to do.  You know you have to do the hardest things first, right?. First stop the migration pages and help section,  this lead me quickly to the Migrate to Microsoft Online Sevices white paper – all 54 pages and 18,000 words of it.

The biggest confusion in the document is that they right differ between On-Premise Exchange mailboxes from Internet-hosted POP3 or IMAP4 mailboxes,  however when your mail services is an Internet Hosted Exchange Service it is not clear that you should treat this just the same as you do for the POP3 and IMAP4 mailboxes – which is exactly what you should do.

Now normally people would want to spend some time planning this out,  and I did to an extent.   The advantage of working with a small company is that you can do things that would be impossible in a larger one.   I got everyone to take a backup of their emails (local .pst) – when I say everyone,  Ellie is using an iMac at the moment so has Entourage , I just took a gamble on that one

Its all local!

I had hoped that with BPOS being in the cloud and my current email being in the cloud that I could just get them to talk to each other and let the transfer begin.   Sadly this is not the case,  all of the work takes place in the migration tool – which means all of the data has to be dragged down locally and uploaded to the new location.   This I was not looking forward too,  but thought hey leave it running while I get into the Christmas spirit.

You need a CSV

Having progressed pass the document and downloaded the migration tool I was ready give the migration a go.   I had already manually created my users in the BPOS admin site and activated the services for them.

At first I was looking for some simple form to allow me to add the users,  however as this is really aimed at bulk processing even for small numbers of users you need to create a simple CSV.   The question is which one?

The document gives you examples and defines the fields that can be used

• · SourceIdentity (required)
• · SourceServer (required)
• · SourceLoginID (required)
• · SourcePassword (optional)
• · TargetIdentity (optional)
• · SourceRootFolder (optional)

but there is nothing for you to go – hey for my service it needs this information.  So it’s a bit of trail and error.   Eventually, after a few goes, my CSV file looked something like this

SourceIdentity,SourceServer,SourceLoginID,SourcePassword
andrew@21apps.com,owa015.msoutlookonline.net,andrew@21apps.com,$$$$$

I originally tried to get the process working with an admin password that I use for my Intermedia account but it didn’t work.   If I had more accounts I may have looked into this more deeply.   As it was I just reset everyones email password and added them to the file.   The import options allow you to then use this for each account imported.

Tip: When you use the migrate wizard if it shows Unknown as the mailbox size you are likely to hit a problem.

I added the users via the CSV and started the migration.  Note my broadband connection (should I say wet piece of string) only gives me about 1.2MB down and 450KB down.  One of the downsides to living in the middle of nowhere.

Although you might be saying,  12 minutes and you have about 5 % done thats not too bad.  Well this is after a couple of failed attempts, each of which took an hour to fail.

The error produced started like the text below and then went on to list the 5 items that had failed.

Elapsed time: 00:49:24

SMTP:andrew@21apps.com
Failed

Error:
The WebDAV injector has stopped migrating mail items for user SMTP:Andrew@21apps.com
because the number of failures has exceeded the failure tolerance of 5.
The injector will ignore any remaining items for this user's mailbox during this
migration session.

Try and Try again

Having tried the migration 3 times,  only one of the mailboxes (Jim’s) migrated as he only had about 20MB used,  mine with 800MB was another story.

I made various attempts are archiving off emails and got it down to about 200MB,  but still it failed.   Most likely cause was some really big emails that people send you with video and uncompressed 12mega pixel photos of the snow!

Time really was against me,  it was now about midnight – everything seemed to take ages and I was getting tired.   After a few rants on twitter – up popped an MSN messenge from Dan Usher.  The conversation stated along the lines of beer, but Dan quickly raised the idea of using the cloud for better bandwidth,  even hinting that

‘Make for a great blog post too… “Utilizing the Cloud for migration from one mail host to another”’

Using the Cloud

Having only played with Amazon EC2 this was an ideal opportunity to try and use it for something real.   A few minutes later, and what was a very slick web authorisation using telephone and pin numbers, I had an account ready to go. 

Creating a new machine and getting the admin password is actually quite a nice process.  Pick the setup, select/create a keypair (this is used to retrieve the admin password), setup a security group if needed allowing RDP access and go.   The process of provisioning and configuring the machine takes about 5 minutes. 

For some reason my first VM, a windows 2008 box, would not let me login.  May have been due to changing the keypair in the wizard (you need to add the site as trusted as you need to download it).  I also found that the message you get when you request the admin password whilst it is still provisioning does not show up in IE8.   I had to use Chrome to see the message.

I terminated the first and then setup an new one based on Windows 2003 R2,  waited,  got the admin password and RDP’s onto the box – all very slick.

Logged in to the BPOS admin service,  downloaded the migration tool – in under a second!   Had to also get Powershell installed as this is needed for the application.  Windows was reporting something like a 5-6MBs download speed   in fact too quick to get a real number.

Migrate in the cloud from cloud to cloud

I imported the csv of users and started the migration process again. It reported the size of the mailboxes (a good sign) and clicked start.  Wow it was actually possible to see the progress bar move

As you can see from the screenshot progress after only 6 minutes was way further forward than the previous one ever got in an hour,  and no reported errors so far.

Migration Complete

After only 25minutes I had migrated both email boxes,  that’s over 400MB of emails from Intermedia to BPOS.   I had a couple of errors,  but not worth worrying about – if they were important I should have dealt with them by now !

The final steps

Migration was complete,  now I need to get get the DNS configured and switched over.   Based on recommendations by Spence Harbar I decided to use a DynDns account so I can once and for all split the DNS management from my mail/web hosting.  This does cost $29.95 per year but to quote Spence it’s really ‘Buttons’ – considering the flexibility and level of service you get.

With the DNS changes made,  it was 3:00am and time to go to bed and let the DNS mice do their stuff.

In Summary

Having had a slight lie in until 9:30 – I don’t do lie ins’ – I came to check on the services,  fired up Communicator and ping – James was already on playing with stuff and using the services – I hadn’t even emailed him to say it was done.

So in summary

• 21st Dec – Decided we need a better line of communication
• 22nd Dec – Looked at the options
  • Intermedia v Microsoft
  • better DNS management
• 23rd Dec – Decided to go the Microsoft BPOS route
  • Setup trial account
  • Attempted migration of mail boxes – Failed
  • Tweeted and messenger’d a bit
  • Setup Amazon Cloud Compute account
  • Create cloud machine
  • Migrated from Cloud to Cloud in the Cloud
  • Setup DNS management and change name servers for 21apps.com
• 24th Dec
  • Go Live

Having only done the process here once it was a learning curve,  I think next time it would be possible to get everything done and dusted in an afternoon.

How much?

Is this really such a good deal and worth migrating?

Before – monthly

 * Intermedia now offer BPOS style plans,  this costs is based on what I was actually paying.

Migration – one off.

Yes that’s correct only $0.71 cents to do the migration using the cloud!  (see below).  I have not added the days here as these activities are part of doing business.  You could substitute your overhead costs in here if you wanted to.

After Monthly

** I’ve included VAT/Tax in these figures as it’s the real cost to the company as we use flat rate VAT scheme.

What’s the ROI?

The costs to the business have risen by about £10 per month,  but with this we get a significant number of additional features and should easily see a ROI on this small investment.  I know that some of the current versions offer limited capability, like SharePoint being unable to add custom code,  but these are being worked on for the next releases of the product.   I’m expecting the service offering based on BPOS to be a good first step but with major improvements with the 2010 product wave.

I will blog more on my experiences of using the Microsoft BPOS solution over time and would be very happy to help out if anyone is also looking to make this move.

I will also be looking much more at how we can utilise the bandwidth in the cloud to make ourselves more productive – we already have plans to use the cloud as an extension of our internal development process and this experience will likely accelerate those activities.