When Kent Beck published his book on XP his message was clear. Refactor mercilessly.

Refactoring is a form of rework and in lean that is considered waste. So was Kent mistaken about this?

Is there any proof that refactoring and Evolutionary Design (ED) are better than Big Upfront Design (BUD)? Agilist seem to have a strong believe that BUD leads to more waste than ED. The cost of refactoring is considered lower than the cost of building the wrong system.

However, is there any proof to this or is it just beliefs and gut feelings? I have certainly seen absolute poor design coming from ED, but so have I seen this with BUD. The difference? BUD seems to deliver structured crap. ED seems to deliver unstructured spaghetti crap.

“Now hold on”, I hear you saying. That must mean they did not refactor! Maybe… The success of refactoring seems to depend on several things. Skill, experience, motivation, courage, discipline. Discipline is only easy when it has become a habit. As long as we need to think about it, it is doomed to fail. If we are still dependent on willpower, then there is a fair chance we won’t refactor when we know it is needed.

Don’t get me wrong. I much more like the ED way of working, but I am just not sure it can be considered a superior to BUD.

ED is allows earlier feedback. This makes it easier to adjust when you find out you are working in the wrong direction. ED allows responding to change, but this also has it’s limits. If the entire goal of the software system changes, it may mean the whole software must be changed.

Refactor mercilessly. Is that even a good idea? I have seen teams stranding and take forever to deliver the next feature, because they needed to refactor more. When you encounter such a situation, stop them? Refactoring is only allowed on code that is impacted by adding the next feature.

7 thoughts on “Refactoring = waste

  1. Big design up front means betting on the set of features you need to deliver over the next [arbitrary period of time]. Surely, on average, this creates much more waste than an evolutionary design approach that avoids committing in advance to design decisions that intend to support future features that we will not implement. Yes, refactoring is rework, but increasingly less rework on average than big design up front, the farther in advance we forecast our features.

    Of course, this suggests that we should forecast less far in advance which features we will implement, and design only for the features we intend to implement over that forecast. Maybe this points to “little design up front” as a useful middle ground? Maybe some groups should try designing based on a 3-month forecast of features? I admit, I find it intriguing, although I have my doubts.

    The larger point remains, though: I see refactoring and testing as the same kind of waste. In theory, we should require neither, but I’ve never seen anyone so effective as a programmer that they can produce highly valuable, non-trivial work without them. Refactoring is waste; big design up front is even bigger waste, depending on the size of “big”.

    This relates to the Classicist/Mockist argument. I know people who have seen me test-drive with mocks who have called it “not TDD”, because I do “small design up front” whenever I introduce a new interface. I make a guess about which types I want based on a desire to avoid putting too many responsibilities in the same place. I feel comfortable doing this, because I don’t need to fall back on Triangulation to in order to implement an algorithm I’ve built a thousand times before. One could argue that, by doing this, I’m admitting that some refactoring is rework and choosing to jump ahead to “the solution”, which nudges in the direction of your thesis. So although it sounded crazy when I first read it, it doesn’t seem so crazy…. except that even when I design with mock objects, I refactor pretty relentlessly and I take very small steps compared to the average programmer. Even compared to the average TDD practitioner.

    I’m willing to bet what’s in my wallet that evolutionary design done well beats big design up front done well. I’m even more willing to bet that evolutionary design done just OK beats big design up front done just OK. I’m not so sure that evolutionary design done in a mediocre fashion is any better than big design up front done in a mediocre fashion, except that people practising evolutionary design who pay attention will at least learn a lot about design, whereas those practising big design up front will probably mostly fail, paint themselves into a corner, and give up. (Maybe.)

  2. One of the reasons why BUD does not work is that the user does not know beforehand what he really needs. This will be gradually discovered. This has nothing to do with the way something is built. BUD leads us to build all sorts of things that are unnecessary and not build things that are needed.

  3. I think the proof of ED over BUD is that BUD always devolves into ED, but ED never devolves into BUD.

    It is true that ED requires discipline, but then so does any successful software process. ED is not unique in that regard.

    In the end it seems obvious to me that a process of successive refinement and incremental improvement is always superior to the long term guesswork necessary for BUD.

    As for Refactoring being rework, that’s like calling the successive refinement of an article, term paper, thesis, short story, novel, oil painting, or clay sculpture, rework. It is not rework, because the product is not finished. You can’t rework something that isn’t done.

  4. The proof that ED is superior to BUD is that BUD always devolves into ED; but ED never devolves into BUD.

    Yes ED requires discipline, but then what successful software process does not. Surely we are not hoping to find a software process that is undisciplined.

    In the end it seems obvious to me that a process of successive refinement and incremental improvement must be superior to the long term guesswork required for BUD.

    As for Refactoring being rework, that’s like saying the successive refinement necessary to finish a term paper, thesis, article, novel, painting, or sculpture is rework. It is not rework because the product it’s not done until that refinement is complete. You can’t rework something that isn’t done.

  5. Thanks for the replies. I never thought this would draw so much attention and get so many responses on Twitter too. Awesome. I have been inspired by all who responded to this. Those who got my intend and those who tried to explain to me what refactoring is all about. This deserves a response from my part. See you soon!

  6. Refactoring leads to learning and ED allows you to apply that learning. The value of learning is not accounted for in this discussion.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s