Techdays 2012 – Slides from my presentation

I held a session at Techdays in Örebro, Sweden about TDD and Legacy Code. I’ve uploaded the slides to speakerdeck.com for those who want to look through them again.

Thank you for coming to my session and I hope you got something out of it! I had some great discussions afterwards about working with legacy code. So good luck to all of you with getting started with testing your legacy systems.

Here’s the link to the slides.

TDD when up to your neck in Legacy Code

So you are working with Legacy Code and want to deliver new features on time and without introducing new bugs; what are your options?

Edit and Pray

Up To Your NeckIf you are not writing unit tests then you are doing Edit and Pray. I spent the first 5 years of my career as a programmer perfecting Edit and Pray. I thought of it as working carefully. I didn’t think of it as me making a change and then just hoping that nothing bad had happened. I learned as much as I could about the system so as to be able to make a good guess about the possible consequences of any changes. Then I painstakingly double- and triple-checked everything before crossing my fingers and deploying. And most of the time it worked but new members on the team had to spend a few months learning the code base before they could make any significant changes.

Cover and Modify

A much better option is to write tests to cover the legacy code that you want to change and make a safety net for it. This is the first step to isolating it from the rest of the code and stopping any changes rippling through the system. Tests written to cover code can be quite different from unit testing and from testing at the GUI level. Their role is to be the canary in the coal mine; they detect if any changes to the code have modified the existing behaviour of the code under test.

Once these covering tests are in place, then the process of refactoring and breaking the code up into units suitable to be unit tested can begin. Later on when you are confident that the code under test is now covered with proper unit tests then the covering tests can be deleted.

Why Unit Test?

If you are lucky then you might have a few large integration tests for the critical parts of your system. For example, you might have a test, taking 5 –15 minutes to run, that initialises test data and then executes all the invoicing logic and checks the result afterwards. A test like this has value but the feedback is very coarse-grained. You won’t be running this after every change in the code and if it throws an exception or returns an incorrect result it might take a few minutes to find out why (or longer if you have to debug it).

Unit tests give you instant feedback at a very local level. They can be run after every change you make and it takes seconds to find any error. This is vital to allowing a programmer to continuously refactor with confidence. I can make a small change to improve the code and know almost instantly if I have broken anything. And in contrast to practising Edit and Pray, a new programmer can come in and change code and feel confident that they have not broken anything important. The learning curve will be much easier as well due to the unit tests acting as living documentation.

TDD and Legacy Code is too hard

A lot of programmers might agree that introducing unit tests into legacy code would be very helpful but think that the process is too hard or time consuming. I can’t count the number of times I have heard someone lamenting that they would love to learn TDD but can’t because they are working with legacy code. Everything is hard if you don’t know how. There are techniques to introduce tests that any programmer can learn and these techniques belong to the core of TDD and good code design. If you learn how to unit test legacy code, then you are on the path to mastering TDD and becoming a much better programmer.

The great thing about TDD is that it encourages taking small, baby steps and this is perfect for working with legacy code. You do not want to make big changes and risk introducing unnecessary bugs. As you begin to master the techniques of refactoring legacy code, you will nearly always be able to find a way to progress with a new baby step.

But first…

Stop writing legacy code. Do not write new features without unit testing. You are just making the code base worse and getting further away from introducing tests into your system. Get your testing infrastructure set up and a few tests running successfully and it will be much easier to think about introducing tests for the rest of the code.

Sprout Method

When  adding new functionality, write the code in a new method and with TDD and then call this method from the old code. So even if you can’t test the code where your method is being called, at least the new code has tests. This technique (from Michael Feathers again) is called Sprout Method.

I use this technique all the time when I am working in an ASP.NET Web Forms project. As it is very difficult to test the code-behind, I write all new functionality in a separate class that is then called by the code-behind class (in Page Load or a button click event handler etc.)

Wrap Method

Another technique to introduce new tested code is Michael Feathers’ Wrap Method pattern. It works with new code that is called directly before or after the old code. Adding a new logging method is the typical example.

Here we have an a method for sending an invoice that carries out lots of different, complicated tasks. I don’t really understand all this code yet but I need to log that the invoice was sent.

public void SendInvoice()
{
	//Code to get invoice data
	//Code to send invoice by email
}

There are two steps to this method:

  1. Extract all the code in SendInvoice into a private method which I call SendInvoiceToCustomer.
  2. Call SendInvoiceToCustomer and my new LogInvoiceSent method (created using TDD) from the SendInvoice method.
public void SendInvoice()
{
	SendInvoiceToCustomer();
	LogInvoiceSent();
}

Now existing code will call SendInvoice in the same way as before but now will also be logged. I haven’t changed the logic or flow of the original SendInvoice method and therefore haven’t risked introducing a bug. However, this technique will only work for code that can be called either before or after the original method.

There are other related patterns (Sprout class and Wrap class) and variations in Michael Feather’s book Working Effectively with Legacy Code.

Next Step

Now all new code is produced with TDD but we are still making untested changes in the old code when we call our new methods. After you have got to know the system a little better it is time to take more baby steps. Next baby step: breaking hidden dependencies.

The Legacy Code Lifecycle

If you have worked for more than couple of years as a programmer then you’ve seen a legacy system. If you are lucky then you have only dabbled with them and not been drawn into the epicentre of a full-blown legacy code project. If you’ve been unlucky then your very first job was maintaining an old legacy system that made you question your decision to choose this career.

Inception of a Legacy SystemFrankenstein

So what do I mean when I say Legacy Code? There are lots of definitions: old code, somebody else’s code, bad code etc. But in the extreme cases nearly everyone recognizes legacy code. Have you ever made a change to a system that should have taken one day but took five? Or made a seemingly simple change that rippled through the system introducing a bug in a totally different module?

I think that the concept of legacy code is tightly coupled to how we as programmers write code. To illustrate this, let’s start with the lifecycle of a project gone wrong.

First Phase – The Land of Milk and Honey

A company decides to build a new system. The developers rejoice as they get to work on a shiny, new greenfield project. This is developer paradise. If the original team is reasonably competent then this phase is a joy to work in. It is easy to add new features and the customers are happy as their requests are fulfilled within a couple of days. The developers get to use the latest, shiny technology and enjoy the feeling of fast flowing development.

However, it is very possible to sabotage the whole system from the very start. An example is the prototype trap; first the team hacks together a prototype which the customer loves, then the team decide to build on the prototype instead of throwing it away. And with that decision made, they are well on their way to building a spaghetti code jungle.

Second Phase – Getting Fatter

Life is not too bad. The features are not flying out the door at the same speed anymore but the customer is reasonably happy. The code might be starting to get a bit clunky, the controllers/code-behind classes are not as skinny anymore. Class size is growing as the classes accumulate more methods and more lines per method. The bug count increases as new features involve changing existing code and not just adding new code. It is high-time to set up bug tracking and a process for change requests.

Third Phase – A Bump in the Road

It is common for programmers to change jobs regularly and in most companies the key people will eventually leave for greener pastures. This means that the two or three developers that built the system and know every nook and cranny are gone. And so is their knowledge. New developers come in and do not manage to grasp the structure and logic of the system. Perhaps it is not even their main task and they are just helping out occasionally.

The era of quick fixes has begun. The new developers never went through the first phase and don’t remember when the code base was a thing of beauty. They see a chunky, inelegant code base and they want to get in and out as quickly as possible. Maybe the customer is putting the pressure on or they just want to get back to their other (greenfield) project.

Fourth Phase – The Big Project

Good news! The customer is delighted with the system. It is saving/earning them buckets of money. They have loads of ideas for new features and want them implemented pronto.

But this is where everything goes wrong. The current team of developers have allowed the code base to rot by applying quick fixes and not understanding the original vision for the system. There is not much structure left and new code has been dumped in inappropriate places. The team are trying to build on a foundation of sand and it is now the bug-fix death march begins.

They try to estimate how long a new feature is going to take but this is extremely difficult due to no-one knowing how big the ripple effect will be. So they triple their estimate to be on the safe side. The problem is this feature will never be done. They might get to 90% done but that last 10% is unattainable. In an entangled code base most large changes introduce bugs. Fixing those bugs introduces new bugs. It is near impossible to complete this project without bugs and meet a deadline.

This is where it turns into a death march. The team has a deadline to meet and just throws code at the bugs in a desperate attempt to get everything done on time. People work long hours, the code quality worsens all the time. In the end, the customer gets a bug-ridden Big Ball of Mud. The customer is not happy and crisis talks begin about the future of the project.

Fifth Phase – What do we do now?

At this stage, you have a burnt out team (or maybe no team at all) and a system that is a maintenance nightmare . What happens now? Throw away the code (and the money invested) and rewrite it? Limp along for years and spend your days fixing all the bugs and taking an eternity to add new features? This system might only be a few years old and already could be thrown on the scrap heap.

A lot of us have seen and experienced projects like this and agree that this system qualifies as legacy code. But I favour Michael Feathers’ much tougher definition:

Code without tests is bad code. It doesn’t matter how well written it is; it doesn’t matter how pretty or object-oriented or well-encapsulated it is. With tests, we can change the behavior of our code quickly and verifiably. Without them, we really don’t know if our code is getting better or worse.

So yes, my example qualifies as legacy code but so does most of code I’ve written during my career as a programmer. Code without tests inevitably rots and becomes harder to maintain. It might never get to the fifth phase but will require a lot of care and attention to stay at the second phase and not get out of control.

In my experience, writing unit tests is the key to changing the lifecycle of a project and to start improving code quality and reducing the bug count. But how can you learn unit testing and TDD if you work with a legacy system? If you are stuck with a legacy system and want to make programming fun again then get started by reading my next article on TDD and Legacy Code.