Agile Sharepoint development by 21apps and Andrew Woodward MVP

For almost my entire career I have been involved in software development, and probably uniquely more often than not in an environment or role where I have needed to look at process improvement. This started with my first role where I had to ensure development adhered to ISO9001 standards, through to today where I work with teams to help them adopt agile techniques and continuous improvement.

Along the way almost everyone I talked to understands the basic idea that the earlier in the cycle you find a defect the cheaper and easier it is to fix, and from this most (if not all) agree that Unit Testing is one of the most cost effective ways to catch these defects. The problem is, it appears, developers are lazy; they understand they should do it they just never get around to it – ‘perhaps on the next project’.

As part of my own development I like to work with, be trained by or just hang with people that I see as having reached a higher level of knowledge and skill than me in a particular field. You could say a greater Mastery of the subject. One area that I am very passionate about, many may have seen some great interviews on the subject, is Test Driven Development (TDD). TDD is more than just doing Unit Testing; it is a technique that once you understand and are willing to invest time in helps you to become a better developer.

I have been doing TDD, although not on every project (sadly), for about 2 years and have shared the knowledge I have gained through white papers, blogs and talking at conferences. In trying to lead the way in my area of expertise, SharePoint, I felt that I was missing something. I had yet to reach a place where I felt that I had mastered the art, where I had moved into the phase of challenging myself to do more than just the practice of TDD.

As part of my work with Typemock, the only solution of working with SharePoint’s sealed API, I found that these guys really did get it, they had the battle scars and were practicing what they preached, and they were challenging themselves to do it different, better. Roy Osherove is the Chief Architect at Typemock and I have seen him speak at conferences in a way that was engaging and thought provoking. The opportunity therefore to be able to spend 5 days with Roy doing a TDD Master Class was one that I could not miss.

I’m not going to do the, on day 1 we did this, on day two we did this, as I think that Sara (fellow attendee) has covered this quite well in her post. Instead I will talk about how the course was much more than a lesson in Test Driven Development; we paired up to cover all of the practices looking at the basics of unit testing frameworks (Nunit, MSTest), understanding isolation frameworks (Moq, Rhino Mocks, Typemock Isolator) and how they all look to solve the same issues whilst using slightly different syntax. We looked at how the techniques around TDD have evolved over time, and the how the tools have really moved on significantly in recent years making the barrier to entry and the ability to create readable and maintainable code possible. We looked at techniques, like the daily Kata, that will help perfect the practices, and discussed the ideas of Shu -Ha -Ri where the student moves from the fundamental techniques, to finding new ways and challenging tradition and then onto surpass the teacher and ultimate mastery.

I liked the way Roy questioned the idea that the lazy coder was in fact the one looking to do it as easily as possible, the one who knew all of the keyboard short cuts, the one with all the Live Templates in ReSharper. The Lazy code is really the one who gets the job done well, with the least amount of effort – unlike the average developer, who I initially thought of as lazy, who just doesn’t see development as a craft but more a way to pay the bills.

My view that learning from someone, who is at a higher level of knowledge and understanding has been reaffirmed. The same principles should apply to you, when you look at training and work, ensure that the person teaching you is someone you admire, someone who you feel has the level of knowledge and experience that you aspire to.

Having attended Roy’s TDD Master Class I can say that my knowledge has been enhanced, the core values have been reaffirmed and I am now in a much better place with regards my own capability, but more importantly my own ability to help others.

So the question is should you attend Roy’s TDD Master Class?

If you want to really Master the Art of Test Driven Development then definitely.

Before last week, I attended Roy Osherove’s TDD Masterclass,  I do not think I would have had the “you know what’s” to be able to put up an alternative solution to one of Uncle Bobs (aka Robert Martin) examples of doing Test Driven Development,  but here I am publishing the the solution I arrived at and suggesting that my solution better servers the values of Readability, Maintainability and trustworthiness.

In Uncle Bobs defence his post is dated Apr 2006 and the art of Test Driven Development has moved on leaps and bounds in the past few years.

The Problem

Write a class named “PrimeFactors” that has one static method: generate.

The generate method takes an integer argument and returns a List<Integer>. That list contains the prime factors in numerical sequence.

To see the details and how it was previously solved.

My Solution

Although the original is coded in Java and mine in C#, there is enough similarities for you to be able to understand both – weird that really

The Tests

The Production Code

I understand that showing the final solution does not prove the value of TDD, I will be posting a web cast of this process to really show how TDD helps you code to evolve and also how I adhere to the Red, Green, Refactor steps to achieve readable, maintable and trustworthy code.

Where do I think mine is better?

The Tests

The tests all adopt the naming convention Method_Scenario_Behaviour

The Act and Assert are kept separate for readability.

I have not used any logic in my tests, unlike the list()method used in Uncle Bobs.

The Production Code

No magic numbers, I have extracted well named constants.

Refactored the logic in ther generate method into more readable methods. For example line 14.

Is much easier to understand than the line

I don’t like the way the while loops were refactored into for loops to reduce the code. If you look closely at the for loop above you will see that it does not follow the normal convention where the evaluation part is related to the incremental part.  This I feel makes it easy to misread and hard to read if you don’t miss it.

Overall I think that the my code, although longer than Uncle Bobs is much easier to read and therefore easier to maintain.

I would love to hear feedback on the code, perhaps I have a couple of refactoring’s that could improve it still further.

Last week I had the pleasure of attending Roy Osherove’s Test Driven Development Masterclass,  I will post in a few days more on this but can say this is one of the best training courses I have ever attended.

During the course I got to learn about the Kata technique, something I had heard about but had never investigated.

Kata is a Japanese word describing detailed choreographed patterns of movements practiced either solo or in pairs. most commonly known for the presence in the martial arts.

The theory is that if you practice a common technique regularly (every day) over time the actions will become natural and automatic, you will have muscle memory.  During the course Roy introduced the idea of coding a reasonably detailed solution, using all of the good TDD practices (Red, Green, Refactor), every day so that the techniques used become second nature, the act of coding something using TDD is natural.

Over the course of the week the class went from being unable to code half of the problem in 30 minutes, to being able to complete the solution.  The obvious productivity’s aside, I will discuss this in a future post, it was clear that every developer in the class was able to master a new technique.   In fact I have tweeted with some of the class members who were doing the same code Kata on Saturday morning.

So what does Jack do?

Although he did join me in my Sunday morning Kata I am not saying that Jack has been doing these coding Katas for years. No, Jack has been doing Kumon mathematics since he was 4 years old. 

Kumon is a math and reading enrichment program. Students do not work together as a class, but progress through the curriculum at their own pace, moving on to the next level when they have achieved mastery of the previous level. Mastery is defined as speed and accuracy.  They only progress when they have mastered the skills through practice and repetition.

Kumon sheets (numbers of questions) are completed daily, they build up the students skills in mental arithmetic and techniques by repeating them often.   Jack, like all new students, started at a level that is below their current ability but through which they can quickly progress.  Jack started at level 6A and moved quickly through the numbers bonds and now has an amazing ability to do mental maths.  Although challenging at times Jack has continued with his daily Kumon (KATA) and had reached the G-League before starting high school, an amazing achievement.

So I can now say that Jack and I will be doing our daily Kata together,  perhaps aiming to master a different art, but taking a daily step closer to achieving that mastery.

Kata is definitely something that will become a way of life for people at 21apps, the value is real and measurable, the personal satisfaction is infectious.

If you’ve read my blog you’ll know that I am a fan of Test Driven Development and an avid user of Typemock Isolator, specifically for the hard to test SharePoint code.  

If you’re not sure of the benefits of doing TDD or need to get some justification from you boss as to how quickly he could see a return on investment, have a read of this empirical evidence.

Roy Osherove is giving an hands-on TDD Masterclass in the UK, September 21-25. Roy is author of “The Art of Unit Testing” (http://www.artofunittesting.com/), a leading tdd & unit testing book; he maintains a blog at http://iserializable.com (which amoung other things has critiqued tests written by Microsoft for asp.net MVC – check out the testreviews category) and has recently been on the Scott Hanselman podcast (http://bit.ly/psgYO) where he educated Scott on best practices in Unit Testing techniques. For a further insight into Roy’s style, be sure to also check out Roy’s talk at the recent Norwegian Developer’s Conference (http://bit.ly/NuJVa). 

Full Details here: http://bbits.co.uk/tddmasterclass

bbits are holding a raffle for a free ticket for the event. To be eligible to win the ticket (worth £2395!) you MUST paste this text, including all links, into your blog and email Ian@bbits.co.uk with the url to the blog entry.  The draw will be made on September 1st and the winner informed by email and on bbits.co.uk/blog

How and I forgot to say.   I will be there as well so we can look to do a SharePint TDD edition.

Having worked for more years than I can remember as a consultant I have lived on my laptop,  everything I do is on the laptop including all of the development.   My current, and have to say the best one ever owned, is a Lenovo T61p.  The plus points it has a proper caddy so I run two drives internally (VMs on one, OS on the other) and all of the drivers just seem to work.   I’ve upgraded the drives a few times to squeeze in some more space (now with 500GB available) and for speed 7200K disks,  I plan to upgrade the main drive to use SSD so that I can keep the old friend going a little while longer.

Many of you are likely in the same place and are thinking,  heck that seems like a reasonable bit of kit, what’s the problem?

The problem is the laptop is OK for single machine development,  as in it’s usable but nothing lightening.   I could, as some have done, pave the way and go Windows 2008 server native to do my development but then I loose all the nice things like just closing the lid and it going to sleep.  But the biggest problem is that I want to do my development right!  I want to get my Continuous Integration running,  be able to test in a scaled out environment and also be able to play with all the new software that is being released.   Sadly my laptop is not really going to cut the mustard here.

I was encouraged by the rig that Andrew Connell put together and this really fitted the bill for what I wanted – I knew this was not going to be cheap,  but these are the tools of my trade.  I wouldn’t expect the local mechanic to be using 99p spanners from the market, they buy the tools that are right for the job. 

Decision  #1

At the time the new i7core processors where starting to hit the shelves,  would it be possible to get a ‘workstation’ setup with dual processors using this new chip architecture?   The answer was you could,  you would be bleeding edge and you would pay through the nose.  So the hard choice was made to stick with the tried and tested XEON proc’s.

Decision  #2

I wanted to do a custom build,  I spent ages looking at the components, deciding what case, proc speeds, heat sinks etc.   The problem I had was in the UK I could not source all of these components from a single supplier.   Why a single supplier you ask?  Well in the UK to make it easier for small businesses you have the option to us register for flat rate VAT scheme, this means calculating the VAT payment based on sales rather than reclaim for everything.  Generally it works well,  the problem is that for larger capital purchases the flat rate doesn’t work.  To support this if a single purchase is above a set amount (£2000) then you can claim the VAT back in addition,  the problem is that it has to be on a single invoice.   So if I purchased the components from the various suppliers I was looking at an added cost of 17.5% – a not insignificant amount.

The Rig

Being a fan of Lenovo since owning my T61p I decided to give their D10 workstation a go.   I had read some good reviews and it was designed with one of my biggest criteria in mind,  not sounding like an aeroplane in my office.

Problem #1

I put together my original spec and placed the order with Dabs.com for the D10, an additional processor, RAM upgrade and some additional drives.   I waited,  and waited eventually I received an email from Dabs saying this product was discontinued!  I knew this was not the case so I went to the go direct and ordered from the Lenovo site and purchased the main rig (with all the lovely 4 year next day onsite warranty) and then upgraded this where the cost for the components was cheaper sourced elsewhere. 

Note: About 1 week after buying mine they announced the D20,  If I were buying today this is what I would get.  The problem was I wouldn’t be able to get a D20 for 3-4 months in the UK,  so this was not an option.

The Rig Continued

I did manage to get a good deal on the kit,  making use of an autocad discount code so was not overly concerned about the D20 release.  Working the numbers carefully I got a the following kit from Lenovo to start my build

• D10 Workstation
• Single Xeon 2.5GHz processor (dual capable board)
• 18GB RAM  (2GB with the machine, + 4×4GB additional chips)
• 250GB 7200disk

The big seller for the D10 was the onboard SAS and SATA RAID with the SATA supporting RAID 10

The following additional items were purchased independently from Scan

• 4 x Western Digital Cavier Black drivers
• 1 x Xeon E5420P Quad Core Proc
• 2 x Thermalright HR-01X with 120mm fan and fixings

I also ordered a pair of LG 20” widescreen monitors from PCWorld Business,  was trying to justify 24” but my room is just not that big .   The monitors had good reviews,  support portrait mode and were surprisingly cheap.

The additional kit from Scan arrived first,  followed by the additional memory, I was counting down the days for the workstation.   Finally I had an excuse to convert the spare room into a spare room/office

It arrived

Lenovo Delivered

First thoughts,  this really is going to be a DOA,  I like to treat my machines with a little bit of respect, it looked like the couriers had used this for target practice!  A few photos later just in case I needed to send it back and I had the rig out.   One thing to remember with these Workstations, they are significantly bigger than a standard desktop or tower.

Lenovo Size

Shown here with my 15” T61p laptop you get a sense of the size.

First Impressions

The D10 is really well put together,  hit had survived the crash testing by the couriers and everything was all still connected and working.  This bodes well for the build quality of the machine.

When I first turned on the machine it sounded like a 747 was spinning up ready for take off..  thankfully after a few seconds the fans all slowed down to a give a near silent machine

Once I proved the machine was working and had it booted up into the shipped software I was ready to start customising.  I decided to get the base OS replaced before doing any hardware updates,  my Virtualisation of choice was Hyper V so I started with the latest Windows 2008 release,  I decided not to go R2 beta as I wasn’t planning to refresh the base OS anytime soon.  My long term plan was to have the OS on a RAID 0 SSD setup,  for now it would just run on the shipped drive.

Inside the rig is fairly spacious and it is really well put together.  No need for screwdrivers for adding disks and the ductings allows the air to flow where it’s needed.  There is a lot of fans in the box but it really does not make much noise at all, in fact the loudest part if the noise from the hard disks.

Inside the Thinkstation

I really like the memory configuration with the rear fan pulling all of the air over the RAM to keep it cool. I see that Andrew Connell was having issues on his rig and had to install memory cooling.

Memory Ducting

The machine runs at a reasonable temp I think,  and not had any issues even during the really hot spell we had in the UK and like almost every UK home I do not have aircon so when it’s hot outside it’s hot inside

Internal Temps

The OS install Part 1 & 2

First install of Windows 2008 went very smoothly,  the software went on really quickly and I had a base setup working with.  Next lets get some ram and hard disks into this baby and see how it flies.

Added the 4 x 1TB drives,  these are SATA drives and were configured very easily using the built in configuration to provide a single 2 TB RAID 10 configuration.   RAID 1 is used for resiliance, every thing is copied to both drives,  RAID 0 is for performance where the information is written across the drives giving you much better I/O.   Doing this in a RAID 10 gives you the best of both worlds.

More Info on Raid configurations

I moved the OS drive over to the SAS controller which also support SATA drives as I had now used up all of the ports on the SATA controller (one was still used for the DVD).

Rebooting the machine and nothing,  it was unable to recognize the boot drive on the SAS controller .   I tried various configurations, boot options and changes all to no avail.   Eventually I decided that i would do another install,  this time using the RAID 10 configuration for booting as well.   (Looking back this is probably the right thing to do as I now have resilience for the OS as well).

OS installation Part 2 when smoothly and it was amazing how much quicker the install was now it was writing to the stripped and mirrored drives – ‘I’m going to like this’

Disk Config

Here you can see the basic disk configuration,  226GB for the OS and 1.6TB for the virtual machines.  I also used the 250GB drive as a file share for ISOs and software.

Hyper-V and Graphics Drivers

Still running on one monitor I setup Hyper-V  and installed my first VM,  WOW = this thing flies,  why oh why have I waited so long to get something like this and move away from the about bearable performance from my laptop.

Time came to get my dual display hooked up and working,   i added the additional monitor – the NVidia Quadro FX1700 shipped with the box supports 2 direct connections and I think can support upto four monitors.   I added two in and looked for config options.   Sadly the currently installed SVGA driver that ships with Windows 2008 can’t support external monitors, or dual monitors or anything other than a very very simple single monitor with limited resolution options.   A quick dig into the drivers on lenovo and I was installing the 64bit Vista XP driver,  install went well everything seemed to be working fine.

Note: I’m not planning to do any video editing or normal day to day stuff on here,  this is really a dev rig.  My laptop is still my main day to day machine/

I fired up the VM and connected,  there was a noticeable delay – it was only a few seconds but compared to the lightening speed before this seemed odd.   On researching this more I found that this is a known issues (but not an issue Microsoft have as at today anything planned to resolve).   I switched back to SVGA and lightening fast,  NVidia and a delay in connecting,  the thing to note is the delay is only really when connecting via the Hyper V manager,  if you remote desktop to the VM then it works just fine.

I’m not thrilled about the delay, but the value of the duel monitor and better resolution is more valuable to me.   I had considered perhaps using VMWare workstation at one point but really wanted to get to know Hyper V so kept the VMWare for the laptop.

A few posts on the subject,  I am hoping that Microsoft do solve this

http://blogs.msdn.com/virtual_pc_guy/archive/2009/01/07/bad-performance-with-high-end-graphics-and-hyper-v.aspx

http://social.technet.microsoft.com/Forums/en-US/winserverhyperv/thread/4e1c53f5-0400-4ca9-8819-f942c10881c1/

Adding a second CPU

Now that my machine was running,  VM installed and ready for some serious work I though it the right time to add the second processor and put in those nice Thermalright coolers.

A quick review of the Lenovo site and I had instructions, in fact a fairly simple process for installing the new CPU and adding the heat sinks. As you can see these heat sinks are beasts compared to the chip that they are there to cool.

Heat sink to processor

After some careful work removing the original heat sink,  adding the new on,  adding the additional CPU and it’s heat sink I marvelled at just how good it was to build you own rig.

Problem #2

Next step,  lets get the cover on and give it a whirl.   I had removed some of the internal air ducting as this didn’t fit with the heat sinks but I was more than annoyed to find that I couldn’t put the case back on!!!   The case has a nice handle and a key lock that you can see from above,  the result is that these monster heat sinks will not fit!.   After a few choice swear words I decided to test it anyway, with the side off.  I could look for smaller heat sinks later.

Problem #3

Started the machine and bang!  The famous BSOD!

I knew this was something to do with the new CPU,  but had to start investigating it to see what it was.   Some more fiddling with the machine,  removed the original CPU and heat sink started the machine and it booted up and worked fine.  So there was nothing wrong with the CPU,  was it to do with the motherboard?

A thing to note, taking CPUs and Heat sinks in and out of the workstation is not something I recommend,  and ensure you have some spare heat paste as you will need it.

Time to turn to the internet,  the benefit of buying a branded workstation was that it is easy to narrow down searches.  I was lucky to find a whole support forum dedicated to the Lenovo Workstations and it was here I found the reason for my problem.

http://forums.lenovo.com/lnv/board/message?board.id=ThinkStation&thread.id=755&view=by_date_ascending&page=1

It appears that two Xeon E5420 processors are not always the same.  They rely on both processors having the same stepping code and have a slightly different revision code SLBBR and SLANV.   The problem is that there is no way through any of the suppliers I have seen to be able to order a specific processor revision.

I phoned Scan with my request and a desire to replace the chip,  they were not willing to do this as I had obviously opened the box not expecting to have this problem.   To be fair the stepping code is shown on the box.  I asked the guy at Scan so can I order this particular stepping model to which he said no, sorry we don’t stock them as different items so all you can do is order it and if its the wrong one send it back.

At this point I’m really hitting a low point in my affections with my dev rig,   realising I now have over £350 worth of upgrades that are worthless is not a way to make me happy.

Getting the 2nd CPU

I am not a big user of Ebay,  but found that this was the only way I could get a CPU with the exact stepping code I wanted.   I eventually got one for around £200, was able to install it and it worked first time.  +1 for Ebay

I still have a few left over components I need to shift,  including the Xeon Processor and some heat sinks.   Time to get back and make use of Ebay again.

Living with my Dev Rig

I have found the snapshots and vm revision tree in Hyper V to be really cool and very very quick.   It was a bit confusing at first understanding that you select a snapshot and choose apply to go to that.  I have missed the idea of a Team in VMWare workstation and also the cloning of images using export/import is less intuitive.

I have recently ventured into using System Centre Virtual Machine Manager 2008 R2 to see what value add this can give me.  I know it’s targeted at bigger setups but I think this will be a great investment as I start to automate more of my development and test environments. 

SCVMM does rely on AD so I am now in the strange position where my AD and SCVMM machines are running as Virtual Machines on my Dev Rig and the SCVMM is managing the Hyper-v as a perimeter  setup which includes itself.  I’m pretty sure this is not a Best Practice 

Going forward I can actually see my Dev Rig taking on much more of a server role,  in fact this is really where it is now with very little installed on the host OS.

What I do like is that I now have the power and storage to really start ramping up on all the new cool software that is coming down the line.

I did however notice that my dev rig has caught the attention of someone else in the family.

Someone else using my dev rig

At least he’s working on something important,  School Project: The History of Chelsea Football Club.

Scrum as an agile approach is great, it allows teams to get up to speed quickly with agile and also enable the business to understand what being agile means.  The problem is that people often just do Scrum.  Doing Scrum is not enough,  it is a great place to start and will help you with your projects but your team still need to get the engineering disciplines in place, they still need to work on doing the right thing the right way and with level of quality demanded by the business.

Jack Milunsky posted a nice article that talks about why Measuring Velocity is not enough.

Like burndown charts, velocity is just another metric which the team can use to reach what I believe is the ultimate goal – sustainable throughput. Velocity in my opinion, is not a metric for determining productivity.

What I really liked, and is something that I am going to be covering a lot more over the coming months is the conclusion

.in order to enhance productivity there are many things one can do:

* You have to get the requirements down pat. The best way to do this is to work with the customer to define the user stories and get their intent up-front.
* Ensure you are capturing the acceptance test criteria from the customer up-front. This helps solve the greatest risk of all – getting the wrong product.
* Define your definition of "done" up-front so that there’s no argument at the end.
* Ensure you build quality in right from the start.

The conclusion can be split into two areas

1. Understanding the Requirements
2. Getting your process right

Getting your process right is something the development teams control,  and there is a lot of information out there today about techniques like Test Driven Development, Continuous Integration and the like that all focus on the development process and quality.  

Understanding the the requirements on the other hand and getting the business to provide you with valid and testable user stories is an area of agile development that I feel has been neglected.  There are great books that have been written on how to write user stories and how to ensure they are testable.   There are other books aimed at the business analysts about how to gather requirements.  However these have never come together, agile development teams don’t look much be the user stories. 

This pushing back beyond the user stories and into the way requirements are gathered and the business is understood is something I have been working on for a while.  And with this push back beyond the user story it is clear the way to get business and customer buy in to the agile development approach is to really change the way businesses approach the complex problems they are trying to solve.

I have some really exciting things to announce over the coming weeks that will really take this discussion forward, working with some of the most visionary guys in the field.

Rest assured there will be a very big SharePoint focus.

I recently wrote about my experience of doing agile development in a team of one,  where I questioned the idea that ‘If your the only member of the team is it really worth going through the same agile development process that you would in a team of eight?’

One interesting test for the project was that it was not going to be developed in one go,  the days spent on this project would need to fit around other work and I want to see how using agile techniques could help to keep the momentum and also retain the knowledge from the previous activities in a format that would be easy to pick up.

Sprint 1 – Day 1 was 15th May 2009

Sprint 1 – Day 2 was 4th June 2009 a full 20 days gap!

Daily Scrum

Always start the day with a scrum and ensure that you come prepared.  This meant that I needed to be able to answer the following questions

• What did you achieve yesterday?
• What are you going to achieve today?
• Any blockers preventing you from doing this?

Take a minute,  try and think back 20 days and remember exactly what you had achieved?   Difficult isn’t it!  This is where using a process really helped,  I started by looking at the my white board to see where I left the project.

Interestingly I had not ‘Done’ anything (refer to Day 1 for the definition of Done and how I structure my white board), my Burndown for Day 1 was looking very sad so I won’t even show it.  What I could however see was that I had completed the work on one story,  the second story was in progress and that I had not started the 3rd.  The observant will also notice I have a couple of new stories in the backlog.

If I had not been working on the project for the past 20 days how come there are new stories?

Adding to the Backlog

One of the tenets of being agile is that you work at a sustainable pace,  you don’t do more work than is possible and you allow your team to go home, see the family and have a beer or two.  What happens when you team is having that beer?  They think about work, and have ideas about ways to do things or new requirements and then write these down as new stories.  The same is true for the business, whilst the team is busy working on the current iteration the business is thinking about what they need, reacting to change and then feeding this into the backlog as new stories.  They may even take away some stories that are no longer needed.

What does this mean?

Well for Sprint 1 nothing,  we have agreed a short period of work where we can focus on the deliverables.   The new stories will be considered for Sprint 2 and for any removed stories we will have the warm feeling that we had not spent too long on in big up front design tasks and documentation.

The Rest of Day 2

After the scrum I reviewed the solution I had put together,  and compared this to the stories.  At the end of Day 1 I had a WSP  and some features that meant I could see what I had done and actually use it.  The ability to use the solution very early in the process is another of the big benefits to being agile.  It allows people to see it for real, to make comment and also to make changes if it is not how they envisaged it.

Needless to say I made some changes,  there were bits that I had added that on reflection were not required,  I removed the extra items and did some refactoring of the solution to make it more compact and understandable.

The rest of the day went pretty well,  and at the end of Sprint 1 I had not actually written any C# code or created any Unit Tests.  The solution at the end of the sprint I had completed the three stories I had planned.  I have to confess that I did do a few extra hours work at the end of this day (Sprint) in order to complete the stories. This can be a common scenario where the a stress levels and work will increase towards the end of the Sprint.  The same if true of Waterfall projects but at the end of these projects there is just so much more that is expected to be done the stress levels go off the scale.

The final tasks for the end of Sprint 1 was to do a “Show and Tell” to the business.  To make this more real,  and to be able to provide a summary once the project is released I record a Screencast demonstrating what had been done.   It may sound silly doing the screen cast or doing a show and tell to yourself, however what it does do is make you bring together everything and ensure that what you said was done, actually was done.

Sprint 1 Retrospective

In order to improve our processes, to produce better quality solutions and to make ourselves more productive we need to look at what happened in the Sprint, the good things as well as the challenging ones and also look for ways to improve.

Things that went well

• Adopted great agile techniques
• Was able to pickup after a break
• Good use of tools like WSPBuilder and hosted source control

Things to improve on

• Testing was manual – need more automation of integration and acceptance tests
• Testing was on single machine – need to test on multi server farm configuration
• Project break was too long – final delivery will be too late at current pace

If you want to learn how to be better at retrospectives I recommend getting a copy of Agile Retrospectives, Making Good Teams Great by Esther Derby and Diana Larsen.

Summary

Based on the experience so far I can say that adopting agile techniques in a team of one really is something that everyone should do.   I agree just cranking out code some developers could be a fair bit more productive up front than I have been. However if the project takes off and you need to bring in more help or you have a long break the project will eventually fail without these process in place.

And the biggest benefit to doing this is that you learn the benefits of being agile,  you understand why it works and you will be able to add this to your CV and Life experience and go on to become a much better developer.  You will be able to challenge the traditional practices in your organisation based on experience and not just hearsay.

If your the only member of the team is it really worth going through the same agile development process that you would in a team of eight?

Isn’t it quicker to just crack open Visual Studio and start coding?   After all your the customer so you know what you want right?   You not going to have to deal with any of the social complexities that conspire against you. You’re going have the hottest of communication with the dev team,  the dev and test roles will have an interment knowledge and the scrum master won’t really have to do anything as there is not going to be any blockers on this project.

Right?

So I started down this path and before I knew it I had some weird project created with bits of code and SharePoint artefacts but not really seeming to make any progress.   I was easily distracted by other things and didn’t really have a feel for how long it would take and when I could say I was done.

STOP!

What was I doing!  I spend my time trying to help people develop better, to take small steps and sometimes big steps to help produce better software..   why was I not practicing what I preached?

I stopped!

Deep breath, deleted all the rubbish I can previously created and got back to basics.

The Team

I have a team,  the fact that some (in this case all) of the roles were done by the same person was ok; that happens often in agile teams.

Design

Yes, you heard me right here.  Eric Shupps would be proud of me .   I’m not going to jump into Visual Studio and crank out code starting [TestFixture] I need to do some design work first, even though this application is going to be small I still need to put in the time on the upfront design.  I need to think through the solution at a high level.

Here I used the opportunity to try out the great Balsamiq application,  which I stumped up the $79 license fee for and download the desktop version.

I’ve deliberately blurred the image but you get the idea of what the mock ups look like, and these take only a matter of minutes to produce.  Definately something you should consider if your currently trying to do this in Visio or heaven forbid cranking out HTML mockups.

The few UI based mockups actually helped me to understand how the application would work for the end user, which in turn provided a clear vision for the way the solution would be developed.

User Stories

I didn’t need to do any remote collaboration on the User Stories so I was able to get back to basics and use one of the white boards in my office as the task board.   I wrote the user stories out on post-it notes and added them to the board along with rough estimates for each one.

Here you can see I’m using a very common step of status on my task board

Backlog - This is where I add all stories that came out of the design,  they are all user focused with nothing specific about the implementation.  No stories like ‘Use jQuery to make it nice’  these are user stories.  If I think of something while i’m coding or testing, or even having a beer I can just write it to a post-it and stick it on the backlog.

Sprint – I like Scrum so adopt a Scrum Esq. style to my process.  The sprint will be where I put the stories I plan to do next.  The amount is based on the estimate and also the sprint length.   In this project I’m doing stupily short sprints of 2 days each.

In Progress – When I pick off a story to work on it goes in here,  if you’ve ever looked at the Kanban approaches your looking to keep this ‘In Progress’ to a defined level.  I like to try and move things through to ‘Done’ before picking off new stories.

Verify – When I think it’s been coded properly and dev tested I move the story here.   Verify is where I would validate the code on another environment – i.e. a non-dev box,  one with multiple web front ends.  At this point I also look at introducing some form of automation for both the build/deployment and also the acceptance testing.

Done – This is when it is really really done,  there is nothing left to do, no documentation, no tests or tweaks,  the code is production ready.

Sprint 1 – Day 1

Having selected the user stories for the sprint it was time to setup my dev project and get some of the foundations in place.   If you follow me on twitter you will know that I previously spent time producing a decent SharePoint development Virtual Machine so it was very quick for me to roll out the few things I needed to get going.   A SharePoint team site in which I could spike and test my code,  a VS2008 solution and class library project in which to start cranking out some code.

At the end of Day 1 my board looked like this:

One of the stories I felt was ready for verification and another I was making good progress on.   At this point I had nothing ‘Done’ my burndown in the morning would look a bit sad,  but that’s OK that is often the case at the start of a project or sprint.  I also don’t at this point have a verification setup so this is going to slow me down a bit on Day 2.

Day 1 has been good,  I have the basic project structure defined.   I was able to build a WSP with the SharePoint features I needed to complete the first story and was able to deploy and test this on my dev machine.

People that have read my blog or seen me talk at conferences and user groups know that I am very keen on doing things right; I like to see Unit Tests and better still I want to see people at least trying Test Driven Development.   However at the end of Day 1 I was yet to write a single line of C# code, neither in test form or production code.   So did I really do development on day one?

Well I think I did, only the language used was XML and a little bit of CAML and for these items I do not know of any way, that makes sense, to unit test.  I often think that the XML we define as part of SharePoint projects are more inline with doing configuration,  we are just setting up SharePoint using a specific configuration and the testing for this should naturally falls into the integration space and will be picked up during verification.   That is not to say as a developer I do not validate it,  I need to ensure that it works as expected in my environment before I can move it over to the Verify column and it’ is the job of the tester to make sure this happens.  For this project the tester (me) was adamant that it all worked as they didn’t want to have to waste time on silly errors.

Interesting Test

One thing that I will be having to do with this project is fit it around real work, the sort that helps to keep food on the table sort of work.   This means that the sprints although 2 days in length the elapsed time may be significantly longer.  I know that this will impact on the velocity as I will lose some of the momentum during these breaks, however the benefit I have from adopting a more defined process is that it will be significantly easier for me to pick up the project.   Imaging if I had to go back to the mess of code I had originally created after a weeks break!

I’m really interested to see if my agile development with a team of one really does make a significant impact on the way I develop the code, on the time it takes to complete it and on the quality of the solution created at the end.

Find out what happens on Day 2.

How many times have you added an additional parameter or bit of logic to your code because you thought it could be useful and it’s easier to add whilst your already changing the code?   If you are your introducing Future Creep!  and should stop doing it.

What does Future Creep mean?

If your from from an XP (Extreme Programming) background you would know this better as YAGNI,  You Aren’t Gonna Need It!   I actually prefer the term Future Creep I first saw on the post, beware of future creep by Jamis of 37 signal.   Everyone understands the terms Scope Creep and Feature Creep and its easy to spot these, although often ignored.  Future creep is the adding of additional code by the developer because they can see a possible future requirements.

DONT DO IT!

You may think, hey it’s only gonna take me a couple of minutes to add this additional parameter which I can see being really useful in the future.  You add the code and then move on,  from that point forward that additional code needs to be maintained, every time a developer interacts with your code they need to read it and understand it – often taking more time as that bit of the code is never actually used, which leads to uncertainty about the code.

It happened to me

The reason I wrote this blog post was an example of future creep on a recent project that had a measurable impact on time and effort. The future creep was was simple, the developer was writing a light weight logging library and decided as part of the Logging.Log() method that if the code had a HTTPContext they would also initialise the current web using SPContext.Current.   None of the code within the internal methods made use of this it was just setup and passed into the method.  This in itself does not sound to bad and for most scenarios the code did run with a valid SPContext h0wever this future creep caused a number of issues which forced developers trying work around it

1. The logging library could not be used in code that had elevated privileges as SPContext.Current throws an error if accessed under a different security context
2. The logging library could not be used where a HTTPContext existed but no SPContext, as was the case for some of the Exchange integration code

Having battled with solutions to the second scenario and getting comments about not wanting to make changes to the Logging Library because the dev was not comfortable with the way it had been implemented I challenged the original author of the code as to the requirement for SPContext.

The answer:  Future Creep

A few minutes later the extra code was refactored out, and the library could be used.

Lessons Learnt and how to prevent it?

Don’t do it,  resist the temptation, remember:

The best way to implement code quickly is to implement less of it.

The best way to have fewer bugs is to implement less code.

And adopting techniques like Test Driven Development force you down a path that doesn’t allow it.

A short and to the point post by Dror Helper at Typemock looks at industry findings on the cost benefits of doing TDD.

The Cost of Test Driven Development

..the research proved two points:

1. Using TDD reduce the amount of bugs in the code significantly
2. Using TDD takes more time then not using TDD

This is in line with what was expected, the question is does the extra time doing TDD reduce the quality or is it the unit testing.  Could doing Test Last unit testing give similar results?

There is also the unquantifiable part to the numbers in the quality of the developers in each team.  This has a significant impact on the code quality.  Using processes like TDD help to raise the bar for all of the developers and the cost benefit will improve over time.  It would be a very interesting exercise to see how these numbers work out in a typical SharePoint project.

On the same lines Phil Haack (great name) has a post on Bug Driven Development where he quotes Robert Glass and Steve McConnell on the costs not doing some form of automated testing which reinforces the value in getting the right processes in place.