01 May 2011

Driven Development

I have recently made the transition to working at a TDD (Test Driven Development) shop, I have no previous experience with it and I am still not sure if I am really feeling it, this may be based on my perception that in order to do it right it seems SO tedious that it is going to seriously harsh my creative coding buzz. Also based on my, albeit limited, knowledge of Combinatorics it seems like it should be called "Smoke Test Driven Development" as the number of possible combinations needed to fully test any software is truly astronomical. On the other hand I have seen it be very useful and the idea of a verification process for software is invaluable, additionally many respected Software Process and Design luminaries such as Martin Fowler and Bob Martin promote it.

I admit part of my problem is that there are probably a lot of things about TDD that I still don’t really get, actually one of my shortcoming as a developer is that I am really not that interested in software process, it’s the structure of software that gets me jazzed. Also I think I have not been lucky enough to see good process applied by good people and my lack of passion for this discipline has resulted in my taking the path of least resistance.

I have been doing this Software Development thing for quite a number of years, more than I want to say, and I have to confess I still don’t really have an explicit methodology, I have acted as a lead developer, although I hate to officially have that title, on many small and mid size projects over the years and in many cases feel that I have made the difference between success and failure of projects, I of course mean that the projects were considered successful, some of which were quite complicated, and in several case have been recognized for helping the success of the project, yet I feel that I am still essentially methodologyless.

Of my last three projects, two claimed themselves as Agile and one declared itself RUP CMMI level 3. The two Agile projects were not even close, they were mostly chaotic waterfall processes, the RUP CMMI project was true to its name, but the process implementation was ridiculously self absorbed in the process itself to the point of being essentially useless, as it was guided by the assumption that checking off all of the process step checkboxes meant the project would run fine on autopilot with people who had little or no software management experience and no thought need be given to the structure or architecture of the software. Admittedly being a seasoned developer this has left me a bit jaded and thinking: "Process, I don’t need no stinking process."

As for TDD I am still on the fence, when I first switched to it I did some light research on it and in the process I encountered BDD (Behavior Driven Development) this started to seem like a YADD (Yet Another Driven Development), so then I googled "Driven Development" to see what else was out there, this of course is still mostly TDD and BDD, but there are many new YADDs, the best one was a humorous yet enlightening blog entry by Scott Berkun called "Asshole Driven Development" which lays out the following (and many more in the comments):

  • Asshole Driven Development (ADD)
  • Cognitive Dissonance Development (CDD)
  • Cover Your Ass Engineering (CYAE)
  • Development By Denial (DBD)
  • Get Me Promoted Methodology (GMPM)

Ironically I think one of my Agile projects was a combination of Scott Berkun’s ADD and CDD, and what I would call RDD, Resume Driven Development, that’s where developers pick technologies solely to add them to their resumes, I was against this practice and reviled for it, thus I was on one side of the CDD conflict.

As I have wandered through the desert of pseudo-process for the last few years, I have been increasingly researching, observing and thinking about why am I able to succeed where others fail and what is the best way to do this, also there are many things I have observed where the management of the project was incredibly inefficient and dysfunctional, sometimes I would openly comment on and other times I would keep to myself.

I figured that I would jump on the YADD proliferation bandwagon and propose two of my own YADD’s, the first is my FDD, Framework Driven Development, the basic structural principles of which are outlined in my blog entry "What does your framework look like?"

In my recent experience I have noticed that software projects can become too focused on process and that process can become a magic bullet. It seems that in some cases process trumps structural concerns. The process component is easier for mangers to grasp and is more congruent with their formal methods, naturally. However, I think that the two need to be balanced, and I feel that something like my FDD concept is a step in that direction, as how to integrate it into process, that is something that still needs to be thought through.

The second YADD is what I would call DSDD, Developer Skill Driven Development. My previous ADD, CDD, and RDD "Agile" project was run by very young non-technical "managers", who had or were in the process of acquiring their PMI (Project Management Institute) PMP (Project Management Professional) suffixes, this one tends feel like a GMPM to me. Needless to say these "mangers" had only a rudimentary, at best, understanding of software development, I should also mention that this was government work so the tolerance for inefficiency was high, and there were a number of very good and good developers so the project mostly ran, as one developer pointed out, by itself. The most frustrating thing about the project was how the work was assigned, the better developers, myself included would often watch, sometimes in anger, as poor and junior developers were given the more challenging work, with which they always stumbled. Another developer would often say to me in jest, "We are all just 25 horsepower engines," I guess sometimes you have to laugh otherwise you’ll be crying.

I have since seen and also reflected back to previous projects in regards to how often managers seem to not understand or take into account the strengths of their team members, and it seems to be a reoccurring antipattern which I would call SDD, Slot Driven Development. (How many YADD’s are we up to now?) I think part of the problem is the management philosophy of divide and concur, where you split up the work into tasks, apply your "resources" and repeat.

DSDD is the idea that not all project tasks have the same level of difficulty and not all developers have the same level or set of skills and the two should be matched as effectively as possible. Another ramification of this approach is to probably a need rethink how project work is partitioned, think framework builder i.e. the 20%.

Of course I am just having some fun ranting here and am not really seriously proposing the creation of more YADDs. Some of my bad experiences may be due to the type of contracts I have taken recently, more on that later, but it does seem that in general software projects are poorly managed, please feel free to share your thoughts, experiences and YADDs in the comments.


  1. I enjoyed reading this blog. In particular I had recently experienced some unexpected consequences of Test-Driven Development that I thought I would share here. I have been practicing TDD for quite a while now but am always reexamining whether or not it is worth the trouble.

    Recently I had the luxury of having extra time to hone my skills in Java Script. I decided to try TDD in Java Script and knew that I needed to create some infrastructure for a looming RIA application that I was assigned. So I used the Yahoo YUI test library and began writing tests around all the infrastructure I envisioned myself needing for that future project. After a few weeks I put this work on the back burner and was consumed by developing the Java API around the database for that same application. The later took a lot longer than I had anticipated and the project had a hard deadline.

    We had a government customer that had remedial infrastructure and I had a really terrible development machine at the customer site. And the RIA was supposed to work in IE 7. But I had no development tools for IE 7 and no debugger.

    So I debugged the UI stuff using Firefox. By the time I had the app working I only had 1 day to make the port to IE 7. Then I remembered my unit tests. I thought that If I could get them to work in IE I would be along way toward completing the port.

    I ran the tests and they failed in ways that were different than they had previously failed in Firefox. The process of debugging the code revealed IE bugs that I needed to work around. The deploy/test/fix cycle was much faster using this technique. If I had not had the tests, I would have had to deploy the web app and test it functionally in IE. And the deploy process on my memory-constrained machine took 15 minutes. But I could point my browser to the tests on my file system without deploying the web app at all. Without this ability I would never have succeeded in the port.

  2. I like your acronyms. ADD, CAYE ...!

  3. "I am really not that interested in software process, it’s the structure of software that gets me jazzed." same here.

    About TDD: Test Driven Development is one of these ideologies ,which sound great in theory but fall short when put in practice. What I mean is, in most of the places I worked at and the teams I worked in had limited resources, non-surprisingly :D I was always working for companies < 200 and not willing EVER to work for corporations, unless I have a mental/retardness disease. So, to the point, usually the time required to create and maintain such test-cases was prohibitive and we only ended up creating functional tests to save ourselves from the dreaded regression bugs :)

    Anyway, TDD and many other YADDs, fall into the Context-less Driven Development. If you fail to analyze your context first and jump blindly on following rigorously any methodology, chances are you gonna have a miss by a long shot.

    Also consider this: isn't TDD rudimentary in contradiction with Agile? To employ TDD successfully you need a very clear set of requirements, otherwise it is close to impossible to come up with a set of self contained units with well defined input-output behavior ! What if you requirement change and you need to adapt to customer requirements? What happens is you need to disregard not only the component/unit/lib you worked on, but also all the time associated with defining the test cases and implementing the tests along with it :( Such an unfortunate WASTE :(

    Yet, there are cases where TDD makes perfect case and, up to my understanding,had proved itself. Consider component development (like .Net web components),e.g. Telerik, ComponentOne etc.
    If you indeed deliver a component with well defined behavior, perfect! TDD will save you day and secure your career too :P

    Keep the good job, another amazing post.. hope I wont loose my job of reading too much of your blog ;)