Making your working day better for you, your colleagues, your organisation and customers!

Of course there’s still SharePoint stuff, but we’re evolving as an organisation, so we can now offer a greater breadth of services to you like “Serious Games”, “Innovation Consultancy” and SharePoint focussed personal development!

We’ve updated our “What we Do” page, go and check-it out for a greater insight to the awesome services we can now offer you!

Take a look below at an overview of what services we can now offer you, and get in touch with us to discuss how we can help change the way you work, bring innovation to the workplace and make your business life much easier!

Innovation Services:

• Serious Games Consultancy & Facilitation
  The seriously fun way to do serious work – seriously..!
  Let 21apps (The UK’s only Qualified Innovation Games® Instructors) work with your organisation to bring Serious Games such as Gamestorming, Innovation Games® and other innovative techniques into your business and get the leading edge on your competitors. Serious Games can help you be successful in delivering your projects, innovating, engaging stakeholders, prioritisation, identifying requirements, gaining customer feedback, product development, market research, workplace development and more..
  Find out more on the 21apps website
• Technology Innovation Consultancy
  Work with 21apps to deliver innovation into your organisation, team, projects and technology platforms including Microsoft SharePoint, Spigit, social media and more.
  We can help you with technology change management, business and stakeholder engagement, technology platform choices and project delivery.
  Give us a call to find out more.

SharePoint Rescue Missions:

• Emergency!
  Our “Emergency! “ offering is a full-on, targeted, sort-us-out service, designed to help you resolve your most pertinent, “I need help now!” SharePoint problems, whether its technology, business, governance, IA, infrastructure or user adoption issues you’re having.
  We will leave you with a pragmatic action plan and a huge sigh of relief!
  Find out more on the 21apps website
• SharePoint Governance Kick-Start
  We’ll review your current Governance and work with you to take action to start your SharePoint Governance on the right foot.
  Find out more on the 21apps website
• SharePoint Business Analysis Kick-Start
  We’ll work with you to bring your BA’s up to speed in the world of SharePoint, think facilitation not analysis and a whole lot more.
  Give us a call to find out more.

Be Awesome:

• SharePoint Team Awesome
  We’ll work on-site with you or remote-mentor your SharePoint team to become truly awesome in all aspects of delivering SharePoint Business Value.
  Give us a call to find out more.
• SharePoint Sensei
  We’ll work one-on-one on-site or remote-mentor your ‘Wannabe SharePoint Sensei’s” and train them like Ninjas in the craft of delivering SharePoint projects.
  Find out more on the 21apps website
• SharePoint Awesome++
  “SharePoint Awesome++” parachutes someone from 21apps into your organisation to work directly with you and mentor you in the most advanced concepts to ensure that your SharePoint capability rocks from both a business and technology perspective! Not for the feint hearted!
  Give us a call to find out more.

Training Services:

• Innovation Games® Training Services (www.InnovationGames.com)
  21apps are the UK’s only Qualified Instructors!
  Let us show you how “Serious Games = Serious Fun” and how these collaborative play techniques can help your organisation innovate, engage stakeholders, prioritise, identify requirements, gain customer feedback and more..
  Give us a call to find out more.
• SharePoint Innovation Games® Training Services
  We offer SharePoint specific Innovation Games® Training or Facilitation Services designed to help you be successful in delivering your SharePoint projects, realising business value and getting the leading edge on your competitors.
  Give us a call to find out more.
• 21apps SharePoint Delivery Workshop
  1 day workshop based on our proven method for delivering high quality, business value focussed SharePoint Solutions (www.21apps.com/ourmodel)
  Give us a call to find out more.
• SharePoint Governance and IA Master Class (www.spgovia.com)
  Run as either public (See www.spiauk.Eventbrite.com) or private courses (Contact us).
  Let us show you how to view the SharePoint world through a different lens. This Master Class pinpoints the critical success factors for SharePoint governance and Information Architecture and rectifies this blind spot with great tools and techniques

Just a Resource:

Subject to availability, we can deliver awesome SharePoint  resources and guru’s from a range of other disciplines from within 21apps and our trusted friends across the globe for your project whether you need someone for pre-sales, a client workshop, onsite consultancy or project delivery…
Find out more on the 21apps website

————————————-

We hope you like what you see here and hope that you’ll give us a call today to discuss how we can help you make a difference in your organisation!

By email: Ant Clay (Ant@21apps.com) / Andrew Woodward (Andrew@21apps.com)

Or phone: Ant Clay +44 7887 553551 / Andrew Woodward +44 7970 126829

Or on Twitter: @21apps / @Soulsailor / @AndrewWoody

My work has shifted away from development and towards business – and it is in the business context that I am seeing so much opportunity to improve.  It might be me but it feels like stepping back in time,  back to the days of battling with “because that’s what the policy says” mentality.  The focus on micro management, performance reviews and over bureaucratic HR.   People don’t ask the questions ‘why’ they just seem to accept this is how it’s done and it’s their job to enforce it, rather than thinking about what’s most valuable to the business.

I watched this video,  and it’s where I borrowed the title of the post as I think Craig Smith covers a lot of my thoughts on how an agile approach in business can really add value.

http://www.infoq.com/interviews/caig-smith-business-agile

And the related presentation.

Richard Harbridge was looking for a decent template to capture SharePoint requirements to which 21apps own Ant Clay responded:

Ant was referring the the 21Shift Model where we focus teams on outcomes.  This started what turned out to be a very lively and ultimately agreeable discussion

Richard responded with the idea that requirements were a journey, an opportunity for Shared Understanding and more than just outcomes.

At this point there was agreement in that we needed to get a Shared Understanding, but a clear view from Ant that Requirements documents were not what was needed; and the journey was absolutley part of the the process of defining outcomes and being able to track these back to the vision.  Requirements documents carry too much baggage and tend to focus on stakeholder ‘wants’

Dux joins the conversation, noting that ideally requirements should not be a document of ‘wants’ which, and I think this is shared by Dux as he used the ‘ideally’ word, is often not the case.

I joined in the conversation to reiterate where we (21apps) see the value in aligning the organisation vision to measureable outcomes mapped to a solution, which some may choose to document as Requirements.  This was in agreement with Richard in that there needs to be a journey in order to get Shared Understanding – but I disagree that Requirements collection facilitate that journey in and of themselves, and as Ant pointed out more often than not they result in a lack of alignment to a vision or worse a list of technology features with no coherent view of what the goal is.

The conversation, as is often the case with twitter chats or conversation over beer, went on to talk about the baggage that is associated with terms and how words like Requirements or Project Management can carry a lot of implied meaning.  Something for another blog post perhaps, for this post I want to reflect on Requirements and why we don’t need them!

But we ended with agreement that we are all on the “Same Page”, and that the journey, discovery and Shared Understanding are the essential elements.

Which is why I reiterate that Requirements – we don’t need them!

What no Requirements?

My background is in development, I worked with eXtreme Programming in early 2000 and have used agile techniques ever since.  The benefits of agile development over the traditional waterfall approach has been debated long and hard and its generally accepted that adopting agile approaches lead to delivery of solutions much more closely aligned to what the business needs.

So why do I think that we need to move on from the use of Requirements in our SharePoint Projects?

The Waterfall Model

http://en.wikipedia.org/wiki/Waterfall_method

Right there at the top of the Waterfall Model is the Requirements step.  This is where, in the waterfall approach, you write down everything it is that you want from the system and then this is fed into the next steps until it ultimately is delivered.

The problem with the waterfall approach is that it assumes you (and I mean the users, managers, business) actually know what it is you want.  What actually happens is people are given ‘one shot’ at getting their requirements written down before it gets taken on by the SharePoint team to build and configure and six months later presented back to the users as the solution they supposedly asked for.

Most users will never have seen the version of SharePoint being deployed, they won’t really have had time to think about what they want to use it for or how it could help them change the way they work – how do you expect the user to be able to write down their requirements?

Even if you are an experienced SharePoint consultant it is hard for you to be able to get the users to articulate exactly what it is they need – you and the users will find it hard to be on the same page.

A warning: Often you will attend meetings where the technology solution is discussed, agreements are reached and everyone is happy with what’s being decided.  A good SharePoint consultant will be able to ensure that the solution doesn’t have any horrors, but they will often get carried away by the desire to cover all of the product features and forget about the real goals – it’s not their fault Microsoft themselves promote solutions based on product features.

Lack of user involvement traditionally has been the No. 1 reason for project failure. Conversely, it has been the leading contributor to project success. Even when delivered on time and on budget, a project can fail it it doesn’t meet user needs or expectations.

Project Management: Criteria for Success – Feb 2001

Lesson from Agile

A lot of what I will cover here is based on a great post by James Shore, Agile Requirements Collaboration. James talks about this with regards to software development, which is where a lot of the innovation and mainstreaming of these techniques has developed, but the concepts are equally valid in SharePoint projects and form a basis of our 21Shift Model.

In agile projects, be they Scrum or otherwise, we refer to ‘requirements’ as stories.  In the 21Shift model we also use the term ‘outcomes’ – the stories define the outcomes that we use to measure the success of the project.

So where do stories come from in the first place? Well… somebody makes them up. Somebody has to have a vision of what the product is going to be. On an XP project, that person is usually called the “on-site customer.” You’ll also hear terms like “lead customer” and “product manager.” or “Product Owner” in Scrum.

This is the first and one of the biggest challenge we face on SharePoint projects, in fact a lot of agile development projects also face this problem.  Who is able to be “Product Owner”

The product owner needs to have a strong, clear vision of the solution, but that doesn’t mean he or she is solely responsible for it. The product owner will work with interaction designers, business analysts, real customers, marketers, users… Ultimately, though, somebody has to coalesce all these conflicting views and come up with a clear, unifying vision. That’s the role of the product owner.

The Product Owner needs a clear, unifying vision.

In order to define the outcomes, or requirements if you insist, the Product Owner needs to have a clear vision of what it is the SharePoint project is going to deliver. They have to work with a diverse group of people and ensure they have a Shared Understanding amongst all stakeholders so that they can define these outcomes.

See the SharePoint Governance and Information Architect Master class for ways to get that Shared Understanding and the Vision stage of the 21Shift Model

Iterations

Unlike the the waterfall approach we do not expect the Product Owner to define all of the outcomes upfront, the solution should be delivered in an iterative approach adding value to the solution in manageable chunks.  Why is this so important?

In waterfall we expect all requirements upfront,  but how can you expect a user to tell you want they want when they don’t know what they want?  By using an iterative approach your users get to see the solution as it’ develops, this results in a great understanding of the solution but also the problem they are trying to solve.

The high level vision could remain the same, but through each iteration the details of how to achieve that vision are discovered.

This brings us to the idea of “latest responsible moment.” “Latest responsible moment” is a core agile idea. The term comes from Lean Software Development. The idea is that you want to put off decisions as long as you can: to the latest responsible moment. But it’s the latest responsible moment, not the “last possible” moment. That wouldn’t be responsible.

The reason we do this is because putting off decisions is cheaper and results in better decisions. Money we spend tomorrow is cheaper than money we spend today, and between now and tomorrow, we’ll learn more things about our environment and the decisions we need to make. Plus, things could change so much that the decision isn’t relevant any more. If we’ve spent the time and effort to make the decision, that money is wasted. So we wait as long as we responsibly can.

Focus on more detail over time

The diagram above I have heard referred to as the “Cone of Uncertainty”,  the further away from the delivery of the solution the less certain the Product Owner is about the precise details needed to implement the solution.

As an example:

Vision:

The Vision could state that we need a consistent approach to project documentation across the organisation to reduce rework, improve quality and align our delivery to the professional image we aspire to.

Outcomes:

We understand what it is the business needs, avoid technology discussions, and focusing on measurable improvements.  This stage really does help with an Retrun on Investment (ROI) needs.
In this example a measurable outcome could include (avoiding platitudes):

• 80% of  project documents based on the correct template.
• 50% decrease in the time taken to locate previous project documents based on similar technology.

Solutions:

As we move into the solutions stage we could identify that we needed to have standard document templates, with defined metadata, that are managed centrally and used for all projects and that the refinements are reflected in search centre.

Delivery:

We need 5 document templates, 8 metadata columns defined, any integration with the actual document template, perhaps use of a content type hub for central management and a policy around the use of the platform. We also need to develop features to configure the project templates automatically.

What this allows us to do is break the solution down into manageable iterations, where the specific details around each of the areas of the vision are refined in greater detail as they get closer to the actual delivery.   The Product Owner is also closely integrated with the delivery team so they can actually see what the solution will look like, how it actually works in SharePoint.

Traditionally agile teams have worked from a Backlog, a list of stories that define the solution, and as they get closer to delivery more detail is added.  In the 21Shift model we have found that reviewing the vision and outcomes for every iteration ensures they still apply, but also to keeps them fresh in the teams mind.

By working at the “latest responsible moment” you can avoid the need to write long requirements documents and allow the Product Owner to refine the details from Vision to Outcomes, with details added as the solution is defined and delivered,  before moving onto the next iteration.  Thus avoid the need for Requirements in the traditional sense, and relying more on the use of stories and collaboration.

Conclusion

To summarize, you need a visionary product owner to work with your team. If you have one, you don’t need to define all the details of your requirements in advance: you can wait until the last responsible moment. If a release is more than three months away, all you may need is a vision for that release. Up until six weeks before implementing a feature, a basic description of the feature on a card may be enough. You’ll need stories for planning about six weeks before implementing, and once you start implementing a feature, you should specify it in detail.

By working in iterations, understanding the vision and defining measurable outcomes it is possible to deliver solutions that the meet the users needs and expectations and align the to vision for the organisation.  All without the need to write Requirements documents in their traditional form.

In SharePoint projects this iterative approach allows us to make much more use of the out of the box features and functionality, we can keep things simple and also have the ability to stop early if the solution is good enough.

Update

21 June 2010:  We are pleased to annouce the 21Scrum has been released.   Please visit www.21Scrum.com for more details and to get your copy.

Having been an advocate of SharePoint and Agile development for some time I have always found it disappointing that there has not been a decent solution for managing my Agile Projects in SharePoint.  I initially floated the idea of a scrum tool for SharePoint just under a year ago and have at last managed to dedicate some time in order to make this idea a reality – internally this is called Project Aberdovey.

Today we announce two things

• Project Aberdovey will be called “21Scrum – Scrum for SharePoint“

21Scrum

21Scrum is a Sandboxed solution that works with all editions of SharePoint 2010 including SharePoint Foundation.

The solution is made of up 3 key areas

• Sprint Planning
• Burndown Chart
• Story Board

Sprint Planning

All of your backlog items are maintained in a SharePoint list,  you are free to extend the list to add additional metadata and views to make management of the backlog easier.   Planning your sprints is done using the Sprint Planning board where you plan as you would with a white board moving stories between the backlog, current and next sprints.

Sprint Planning

Visibility is central to Scrum projects and the Burndown chart provided gives you a real time view of how you sprint is progressing against the ideal.

Story Board

The story board is what the team uses during the Sprint, ideally this is a big white board in the team room – with distributed teams this is not viable.  The 21Scrum story board provides this white board view and, like the Sprint Planning, allows the team to drag and drop stories as they progress through the stages from Not Started to Done.

Getting Started

A quick guide to getting started with 21Scrum

1) Download the 21Scrum Sandboxed solution – rename the file from .zip to .wsp

21Scrum has now been released – check out the fully featured trial.

2) Navigate to the Solution Gallery in the Site Collection you want to deploy 21Scrum and click upload solution (in the solutions Tab)

3) Upload the 21Scrum solution and activate it

4) Activate the Features

21Scrum is divided into two part

• * Site Collection Feature which has things like the WebPart definitions
• * Site Feature which enables the lists to support the team

This split allows you to have multiple independent Scrum teams working in the same site collection.

At the root site collection activate the Site Collection Feature

And activate the Site level feature in the site that you want to manage your Scrum team

5) The Burndown Chart is provided as a web part,  add this web part to a page.

As you haven’t started doing any work yet your Burndown chart will look like this.   To make testing of the beta easier we have provided another web part.  The “Populate Demo Data” web part will create sample data as if you were half way through a sprint. Simply add this web part to a page and click “Populate Demo Data” and have a play with the Sprint Planning, Story Board and Burndown.

Giving Feedback

Having used a lot of scrum tools and often found them trying to do too many things we have kept 21Scrum very focused,  the ability to be able to get up and running quickly and for the tool to add value and not become an overhead was crucial.   However we may have missed something that you can’t live without!

The beta release of 21Scrum is available to anyone to download and use,  it is not restricted in time-boxed so please feel free to use this on real life projects.

All we ask is that you provide feedback so we can make sure that when we release 21Scrum it is the best Scrum tool available.

If you have a feature request or find a bug please add comments on this page to send an email to andrew@21apps.com

Here I talked about what a Kata is and how it compares to Kumon Maths and I also took the opportunity to practice my reviewing techniques by looking at the best and reviewing Uncle Bobs prime factors TDD Kata.

I had promised to provide a web cast of the TDD SharePoint session that I did at the Best Practices conference, but I feel now that I need to really improve my keyboard mastery before taking on that one.  Instead I have taken the opportunity to record my current TDD Kata which is based on Roy Osheroves String Calculator.  I have limited the steps I have done in this recording as I think 30 minutes would get a little boring.

String Calculator

1. Create a simple String calculator with a method int Add(string numbers)
   1. The method can take 0, 1 or 2 numbers, and will return their sum (for an empty string it will return 0) for example “” or “1″ or “1,2″
   2. Start with the simplest test case of an empty string and move to 1 and two numbers
   3. Remember to solve things as simply as possible so that you force yourself to write tests you did not think about
   4. Remember to refactor after each passing test
2. Allow the Add method to handle an unknown amount of numbers
3. Allow the Add method to handle new lines between numbers (instead of commas).
   1. the following input is ok:  “1\n2,3″  (will equal 6)
   2. the following input is NOT ok:  “1,\n” 
   3. Make sure you only test for correct inputs. there is no need to test for invalid inputs for these katas
4. Allow the Add method to handle a different delimiter:
   1. to change a delimiter, the beginning of the string will contain a separate line that looks like this:   “//[delimiter]\n[numbers.]” for example “//;\n1;2″ should return three where the default delimiter is ‘;’ .
   2. the first line is optional. all existing scenarios should still be supported

The idea behind TDD is to work through one thing at a time,  so in the example we start with 0 numbers (or empty string) and progress through each requirement continuing to adopt the Red, Green, Refactor approach – hopefully this will be clear from the video.

The Kata is not about knowing how to solve this problem, it is about knowing how to do things quickly and also knowing how to approach problems using TDD. 

In this example,  as in my day job, I am using ReSharper to aid refactoring,  NUnit 2.5 for my tests as this supports TestCase and TestDriven.Net to run the tests.

I have setup some live templates for the Test method and common Assert.AreEquals statements and have added a Keyboard shortcut to Re-Run the tests in TestDriven.Net (I use Alt + R).

Hope you enjoy the video, I have left the sound off - if you perfer some nice background music I can edit and add some

Would love to hear your thoughts on this.

TDD Kata – Calculator Part 1 by AndrewWoody from Andrew Woodward on Vimeo.

You can download the WMV video to watch locally from the Vimeo site.

One interesting test for the project was that it was not going to be developed in one go,  the days spent on this project would need to fit around other work and I want to see how using agile techniques could help to keep the momentum and also retain the knowledge from the previous activities in a format that would be easy to pick up.

Sprint 1 – Day 1 was 15th May 2009

Sprint 1 – Day 2 was 4th June 2009 a full 20 days gap!

Daily Scrum

Always start the day with a scrum and ensure that you come prepared.  This meant that I needed to be able to answer the following questions

• What did you achieve yesterday?
• What are you going to achieve today?
• Any blockers preventing you from doing this?

Take a minute,  try and think back 20 days and remember exactly what you had achieved?   Difficult isn’t it!  This is where using a process really helped,  I started by looking at the my white board to see where I left the project.

Interestingly I had not ‘Done’ anything (refer to Day 1 for the definition of Done and how I structure my white board), my Burndown for Day 1 was looking very sad so I won’t even show it.  What I could however see was that I had completed the work on one story,  the second story was in progress and that I had not started the 3rd.  The observant will also notice I have a couple of new stories in the backlog.

If I had not been working on the project for the past 20 days how come there are new stories?

Adding to the Backlog

One of the tenets of being agile is that you work at a sustainable pace,  you don’t do more work than is possible and you allow your team to go home, see the family and have a beer or two.  What happens when you team is having that beer?  They think about work, and have ideas about ways to do things or new requirements and then write these down as new stories.  The same is true for the business, whilst the team is busy working on the current iteration the business is thinking about what they need, reacting to change and then feeding this into the backlog as new stories.  They may even take away some stories that are no longer needed.

What does this mean?

Well for Sprint 1 nothing,  we have agreed a short period of work where we can focus on the deliverables.   The new stories will be considered for Sprint 2 and for any removed stories we will have the warm feeling that we had not spent too long on in big up front design tasks and documentation.

The Rest of Day 2

After the scrum I reviewed the solution I had put together,  and compared this to the stories.  At the end of Day 1 I had a WSP  and some features that meant I could see what I had done and actually use it.  The ability to use the solution very early in the process is another of the big benefits to being agile.  It allows people to see it for real, to make comment and also to make changes if it is not how they envisaged it.

Needless to say I made some changes,  there were bits that I had added that on reflection were not required,  I removed the extra items and did some refactoring of the solution to make it more compact and understandable.

The rest of the day went pretty well,  and at the end of Sprint 1 I had not actually written any C# code or created any Unit Tests.  The solution at the end of the sprint I had completed the three stories I had planned.  I have to confess that I did do a few extra hours work at the end of this day (Sprint) in order to complete the stories. This can be a common scenario where the a stress levels and work will increase towards the end of the Sprint.  The same if true of Waterfall projects but at the end of these projects there is just so much more that is expected to be done the stress levels go off the scale.

The final tasks for the end of Sprint 1 was to do a “Show and Tell” to the business.  To make this more real,  and to be able to provide a summary once the project is released I record a Screencast demonstrating what had been done.   It may sound silly doing the screen cast or doing a show and tell to yourself, however what it does do is make you bring together everything and ensure that what you said was done, actually was done.

Sprint 1 Retrospective

In order to improve our processes, to produce better quality solutions and to make ourselves more productive we need to look at what happened in the Sprint, the good things as well as the challenging ones and also look for ways to improve.

Things that went well

• Adopted great agile techniques
• Was able to pickup after a break
• Good use of tools like WSPBuilder and hosted source control

Things to improve on

• Testing was manual – need more automation of integration and acceptance tests
• Testing was on single machine – need to test on multi server farm configuration
• Project break was too long – final delivery will be too late at current pace

If you want to learn how to be better at retrospectives I recommend getting a copy of Agile Retrospectives, Making Good Teams Great by Esther Derby and Diana Larsen.

Summary

Based on the experience so far I can say that adopting agile techniques in a team of one really is something that everyone should do.   I agree just cranking out code some developers could be a fair bit more productive up front than I have been. However if the project takes off and you need to bring in more help or you have a long break the project will eventually fail without these process in place.

And the biggest benefit to doing this is that you learn the benefits of being agile,  you understand why it works and you will be able to add this to your CV and Life experience and go on to become a much better developer.  You will be able to challenge the traditional practices in your organisation based on experience and not just hearsay.

Isn’t it quicker to just crack open Visual Studio and start coding?   After all your the customer so you know what you want right?   You not going to have to deal with any of the social complexities that conspire against you. You’re going have the hottest of communication with the dev team,  the dev and test roles will have an interment knowledge and the scrum master won’t really have to do anything as there is not going to be any blockers on this project.

Right?

So I started down this path and before I knew it I had some weird project created with bits of code and SharePoint artefacts but not really seeming to make any progress.   I was easily distracted by other things and didn’t really have a feel for how long it would take and when I could say I was done.

STOP!

What was I doing!  I spend my time trying to help people develop better, to take small steps and sometimes big steps to help produce better software..   why was I not practicing what I preached?

I stopped!

Deep breath, deleted all the rubbish I can previously created and got back to basics.

The Team

I have a team,  the fact that some (in this case all) of the roles were done by the same person was ok; that happens often in agile teams.

Design

Yes, you heard me right here.  Eric Shupps would be proud of me .   I’m not going to jump into Visual Studio and crank out code starting [TestFixture] I need to do some design work first, even though this application is going to be small I still need to put in the time on the upfront design.  I need to think through the solution at a high level.

Here I used the opportunity to try out the great Balsamiq application,  which I stumped up the $79 license fee for and download the desktop version.

I’ve deliberately blurred the image but you get the idea of what the mock ups look like, and these take only a matter of minutes to produce.  Definately something you should consider if your currently trying to do this in Visio or heaven forbid cranking out HTML mockups.

The few UI based mockups actually helped me to understand how the application would work for the end user, which in turn provided a clear vision for the way the solution would be developed.

User Stories

I didn’t need to do any remote collaboration on the User Stories so I was able to get back to basics and use one of the white boards in my office as the task board.   I wrote the user stories out on post-it notes and added them to the board along with rough estimates for each one.

Here you can see I’m using a very common step of status on my task board

Backlog - This is where I add all stories that came out of the design,  they are all user focused with nothing specific about the implementation.  No stories like ‘Use jQuery to make it nice’  these are user stories.  If I think of something while i’m coding or testing, or even having a beer I can just write it to a post-it and stick it on the backlog.

Sprint – I like Scrum so adopt a Scrum Esq. style to my process.  The sprint will be where I put the stories I plan to do next.  The amount is based on the estimate and also the sprint length.   In this project I’m doing stupily short sprints of 2 days each.

In Progress – When I pick off a story to work on it goes in here,  if you’ve ever looked at the Kanban approaches your looking to keep this ‘In Progress’ to a defined level.  I like to try and move things through to ‘Done’ before picking off new stories.

Verify – When I think it’s been coded properly and dev tested I move the story here.   Verify is where I would validate the code on another environment – i.e. a non-dev box,  one with multiple web front ends.  At this point I also look at introducing some form of automation for both the build/deployment and also the acceptance testing.

Done – This is when it is really really done,  there is nothing left to do, no documentation, no tests or tweaks,  the code is production ready.

Sprint 1 – Day 1

Having selected the user stories for the sprint it was time to setup my dev project and get some of the foundations in place.   If you follow me on twitter you will know that I previously spent time producing a decent SharePoint development Virtual Machine so it was very quick for me to roll out the few things I needed to get going.   A SharePoint team site in which I could spike and test my code,  a VS2008 solution and class library project in which to start cranking out some code.

At the end of Day 1 my board looked like this:

One of the stories I felt was ready for verification and another I was making good progress on.   At this point I had nothing ‘Done’ my burndown in the morning would look a bit sad,  but that’s OK that is often the case at the start of a project or sprint.  I also don’t at this point have a verification setup so this is going to slow me down a bit on Day 2.

Day 1 has been good,  I have the basic project structure defined.   I was able to build a WSP with the SharePoint features I needed to complete the first story and was able to deploy and test this on my dev machine.

People that have read my blog or seen me talk at conferences and user groups know that I am very keen on doing things right; I like to see Unit Tests and better still I want to see people at least trying Test Driven Development.   However at the end of Day 1 I was yet to write a single line of C# code, neither in test form or production code.   So did I really do development on day one?

Well I think I did, only the language used was XML and a little bit of CAML and for these items I do not know of any way, that makes sense, to unit test.  I often think that the XML we define as part of SharePoint projects are more inline with doing configuration,  we are just setting up SharePoint using a specific configuration and the testing for this should naturally falls into the integration space and will be picked up during verification.   That is not to say as a developer I do not validate it,  I need to ensure that it works as expected in my environment before I can move it over to the Verify column and it’ is the job of the tester to make sure this happens.  For this project the tester (me) was adamant that it all worked as they didn’t want to have to waste time on silly errors.

Interesting Test

One thing that I will be having to do with this project is fit it around real work, the sort that helps to keep food on the table sort of work.   This means that the sprints although 2 days in length the elapsed time may be significantly longer.  I know that this will impact on the velocity as I will lose some of the momentum during these breaks, however the benefit I have from adopting a more defined process is that it will be significantly easier for me to pick up the project.   Imaging if I had to go back to the mess of code I had originally created after a weeks break!

I’m really interested to see if my agile development with a team of one really does make a significant impact on the way I develop the code, on the time it takes to complete it and on the quality of the solution created at the end.

Find out what happens on Day 2.

UPDATE:  Project Aberdovey is now live and called 21SCRUM

Download a trial from www.21scrum.com

Being an advocate of SharePoint as an application platform I looked long and hard for a decent implementation of a Scrum and have always been found wanting.  I have seen simple examples where any power user could set things up but found the lack of any Burndown charts meant you lost probably the most important part of Scrum – the visibility.

I have found that VersionOne is one of the best products on the market, although it has a lot of options which tends to put off new teams.

For those that use Team Foundation Server your probably going to head towards the Scrum for Team System developed by Conchango.  I’ve not used this in anger so am unable to comment on how well it works however this solution is not really a SharePoint specific solution as it requires you to make the commitment to using Team Foundation Server.

Bil Simser did a good post on scrum tools,  although its nearly 3 years old still has some good links.   Again however the tools are not specifically SharePoint.

Why the SharePoint Obsession?

SharePoint is the collaboration tool of choice, companies have deployed it and it is being used with varying degrees of success.  People have become familiar with the how to add items to a list, how to upload documents into a document library and for the more advanced how to build engaging dashboards.

What is the biggest thing you get from adopting scrum?  ‘Visibility’

What do you need to encourage to make Scrum work?  ‘Collaboration’

SharePoint is the natural platform choice on which to build a Scrum tool.

Introducing Project Aberdovey

As with all good development teams I think it is right that your development project should have a name,  and in keeping with some big companies in the Seattle area 21apps uses place names.   Aberdovey, or as it’s spelt in Welsh Aberdyfi, has probably one of the best beaches in the UK and is located 50 miles west of the office.

Put simply, Project Aberdovey is a Scrum tool for SharePoint.

There are a few aims for the project:

1. Produce a fantastic Scrum tool built on SharePoint
2. Develop the solution using agile techniques including TDD
3. Dogfood the solution as soon as possible – what better way to develop the right solution
4. Open development – I want to give feedback on the challenges but also welcome your input

I will be looking for ways to give people visibility,  I am looking at codeplex as an option but welcome any suggestions.

Disclaimer:  Red Gate ANTS Profiler is a paid for application.  There are other products on the market that do code profiling including Visual Studio.  The reason for using ANTS in this post is due to the ease with which you can get up an running and the quality of the user experience.  I recommend you set aside a day to look at the profiling in Visual Studio and then a few hours to get that much further with ANTS.

Software Performance Analysis – Wikipedia

In software engineering, performance analysis, more commonly today known as profiling, is the investigation of a program’s behavior using information gathered as the program executes (i.e. it is a form of dynamic program analysis, as opposed to static code analysis). The usual goal of performance analysis is to determine which sections of a program to optimize – usually either to increase its speed or decrease its memory requirement (or sometimes both).

As the quote says this technique is specifically aimed at the optimization of code; the tools enable the developer to quickly target the areas of code that need optimization and lead to an improved return for the effort. (seems strange saying ROI when talking about dev techniques, but this is really what it is).

Getting started

Red Gate provide a 14 day trial, so you can try before you buy.  The latest release, version 4.3 as at the time of writing, has made some great improvements over the 4.1 version – most notably the removal of the need to stop all the IIS web sites listening on port 80 .

As you would expect this post will focus on profiling SharePoint code, I’m going to use the Magic 8 ball application developed for my TDD session at the Best Practices SharePoint Conference in San Diego.

More details on Test Driven SharePoint development can be in the Agile section of this site.

Environment

The environment that I will do the testing is a typical SharePoint developers Virtual Machine.

1. Windows 2008 Server (32bit)
2. Visual Studio 2008
3. Microsoft Office SharePoint Server 2007
4. SQL Server 2005

As this is running on Windows 2008 I will be testing against IIS7.   The requirements are identical for Windows 2003 and IIS 6.0

Installation

Download ANTS Profiler and install, not too many options to worry about here.

Luckily for my first run through the team have provided a technical paper on installing, using and getting ANTS Profiler to work with SharePoint, How to profile a SharePoint 2007 Site Collection.  There were a few things in here that I prefer not to do,  I will document what I do below and will pass this back – I’m not criticizing, there’s no way I would even attempt to write a Profiler so mucho respect from me to the Red Gate team

Configuration

We are looking for the ability to profile any site collection within a Web Application.  This involves getting the account used for the associated application pool, giving it some additional rights and then configuring the ANTS services to use this account.

Web Application

Depending on your development machine configuration you will have a number of web applications configured in SharePoint, the configuration below should be reasonably familiar to most of you.

The key here is to identify the web application (IIS Website) as we need to get the account name from the associated Application Pool.

Load IIS Manager, select the web application and chose Basic Settings from the Actions menu on the right.  This will show the associated application pool as below.

Select the Application Pools from the connections menu, choose the application pool identified in the previous step and click advanced settings.  This will give you a properties screen showing the account we need to work with.

In this case 21apps\svc-intranet-ap – if you have a local installation this may be set to Local System in which case the steps below may not be required.   I recommend that dev’s don’t have a default install as you really need to understand how SharePoint is setup to undertake any large scale developments.

Web.Config

In IIS 6 days you would need to edit the web.config file to set the application into debug mode.   In IIS 7 the common ASP.NET configuration options have been promoted to the IIS console.  With the web application still selected click on the .NET Compilation option and change the Debug behaviour to True and click Apply.

Like all good SharePoint developers you will be working with WSS_Minimal trust settings or a custom policy file.  To do profiling however you will need to set the Trust level to Full.  Again IIS 7 provides a UI or you can edit it manually.

It is possible to run the code without setting a Full trust level, but you need to remove the .pdb files and will lose the ability to profile the lines of code.  Which is sort of the point of profiling!  If you don’t set Full trust you will get an error like the one below when loading a web part

Web Part Error: An error has occurred.

Show Error Details
Hide Error Details 

[TargetInvocationException: Exception has been thrown by the target of an invocation.]
  at System.RuntimeTypeHandle.CreateInstance(RuntimeType type, Boolean publicOnly, Boolean noCheck, Boolean& canBeCached, RuntimeMethodHandle& ctor, Boolean& bNeedSecurityCheck)
  at System.RuntimeType.CreateInstanceSlow(Boolean publicOnly, Boolean fillCache)
  at System.RuntimeType.CreateInstanceImpl(Boolean publicOnly, Boolean skipVisibilityChecks, Boolean fillCache)
  at System.RuntimeType.CreateInstanceImpl(Boolean publicOnly)
  at System.Activator.CreateInstance(Type type, Boolean nonPublic)
  at System.Activator.CreateInstance(Type type)
  at Microsoft.SharePoint.WebPartPages.SPWebPartSerializer.get_DefaultControl()
  at Microsoft.SharePoint.WebPartPages.BinaryWebPartSerializer.SerializedWebPart..ctor(SPWebPartManager webPartManager, XmlNamespaceManager xmlnsManager, Byte[] sharedData, Byte[] userData, String[] links, Type type, SPWeb spWeb)
  at Microsoft.SharePoint.WebPartPages.BinaryWebPartSerializer.Deserialize(SPWebPartManager webPartManager, XmlNamespaceManager xmlnsManager, Byte[] userData, Byte[] sharedData, String[] links, Type type, SPWeb spWeb)
  at Microsoft.SharePoint.WebPartPages.SPWebPartManager.CreateWebPartsFromRowSetData(Boolean onlyInitializeClosedWebParts)

[VerificationException: Operation could destabilize the runtime.]

Services

ANTS runs as two services

1. ANTS Memory Profiler 4 Service
2. ANTS Performance Profiler 4 Service

Change the accounts on these two services to the same accounts used in the web application we identified above.

If this is the first time the account has been associated with a service it will be automatically granted the ‘Log On As A Service’ right.

Local Security Policy

From the Administrators menu choose Local Security Policy and grant the application pool account the following rights

1. Act as Part of the Operating System
2. Impersonate Client after authentication

Open a command prompt and run gpupdate /force.

Profiling your code

Your environment is now configured, we can start profiling.  As this is SharePoint I tend to just hit the bit i’m planning to profile first to make sure this is working before I start profiling.  Many hours can be lost trying to solve a configuration issue with a profiler that is not at fault.

Start Red Gate ANTS Performance Profiler.  It knows we’re running on a system a system that has UAC, it offers to make our life easy and always run as an Administrator.  I normally click yes here.

ANTS loads the Profile Settings window is automatically.

There are only a few things to set.

1. Choose the ASP.Net web application (hosted in IIS)
2. The URL to the SharePoint site
3. Use the original port

Further profiling counters can be set, but we just want to get our first run going for now.  Click on Start Profiling.  (In this example I have added a long running process to show how ANTS can identify this as a hotspot).

Profile Results

Profiling will start and a new browser will open at the location specified.  As this is SharePoint the profiler restarts the service so you will always have a small delay while SharePoint is JIT compiled. The profile timeline will show this impact on the processor.

Once the processor has levelled out its time to start profiling your code.   Navigate and interact with the bit of the application you are interested in.  If it is a  web part or navigation load you are interested in then you can refresh the page.

Once you are happy you have performed enough tasks,  I recommend keeping these focused, stop the profiling. ANTS will present you with the HOT code, the bit of code that is the worst performing. You can focus the analysis on any part of the profile graph to show poor performing code around a specific spike in the processor or memory.

In the example above I am focused on the code that runs (including my long running process) when I ask the magic 8 ball a question.

Viewing source code when you have enabled Line level profiling shows every line, the hit count and the % of time taken to perform that operation.

The really cool thing is you can click through the methods to follow the slow running process through the code!

Showing the method grid allows you to open up a call graph for any method to see how the processing is broken down in a dynamic tree view.  This really is just brilliant!  The image below does not do it justice you really do need to see this for yourself to appreciate it.

Conclusion

The long running process I added for the demonstration was picked up and allowed me to drill down into the lines of code that were the slowest.  It even showed how the internal calls within the base classes performed.

Red Gate ANTS Profiler is easy to setup and proves to be an invaluable tool in your software development tool kit.  I suspect profiling will not be something that you do every day, however with the speed and ease of use I would like to think that you will consider the inclusion of profiling a must have rather than a nice to have.

1. You are not allowed to write any production code unless it is to make a failing unit test pass.
2. You are not allowed to write any more of a unit test than is sufficient to fail; and compilation failures are failures.
3. You are not allowed to write any more production code than is sufficient to pass the one failing unit test.

I did have some slides to set the scene (see below), to give some background to TDD and raise the issues around the use of the word TEST.  We also looked at 10 reasons TDD sucks, with a slight SharePoint slant,  and quickly got into Visual Studio where the real learning was to happen.

The user story was to develop a Magic 8 Ball web part that could have custom answers defined in a SharePoint list.  The finished code is available on CodePlex and it is planned to extend this solution over time to expand on the ideas.  My main issue with just getting the finished code is the learning is in the process and it can be hard to see how the design was driven by the code.   I am planning to do a web cast on this to show this process.

CodePlex project now available  http://www.codeplex.com/sp8ball

Winners

The guys at Typemock we very generous in donating 3 copies of Isolator for SharePoint which is an essential component for anyone doing unit testing in a SharePoint environment.  The lucky winners were

Mark Freeman
Shahar Tamari
Kennedy Duncan

Each of you will receive a free license for Isolator for SharePoint,  for those that did not win Typemock are running a special discount.