15 March 2014

Are Your Developers Treating Your Project as Their Own Personal Science Project?

Years ago I worked on a project that had a custom role based security system and for this system we needed a way to load menus according to the users’ assigned roles.  Two of the dominant developers on the project decided that this was a perfect use for AJAX mainly because they wanted to learn and use AJAX.  So they build a separate deployable menu application to serve up the menu by emitting XML from a Spring Controller URL, it was very comparable to a simple Restful service. It was called directly from JavaScript in the application’s web page.  Since there would be a lot of calls for any given user’s menu they built a cache for each user’s menu options, it was a custom cache where objects never expired.  This work was completed as I joined the project and as I got acquainted with this project my first thought was why do all this work?  Why not just load the menu when the user is authenticated and then cache it into the HTTP Session, I raised the issue but was ignored.  It was their pet project and it was sacrosanct. 

Later, while testing the apps that used the AJAX menu call the testers ran into a problem, the cache was interfering with testing because it was keeping the database changes from be propagated and testing the applications created a need to continuously restart the AJAX menu app.  So a new admin interface was added to the AJAX menu application to allow the testers to clear the cache.  This created a whole separate application that the client would have to support just to enable this unnecessary AJAX menu delivery mechanism.  So instead of just caching the data in the HTTP Session which is easier and any cache problem would be fixed by just logging out and logging back in, these two developers built a pet project which  consists of an extra application and codebase that has to be understood and maintained.  Ultimately the biggest problem this creates boils down to simple economics, it more expensive to maintain a solution that requires a whole separate application.

I think there is a tendency for all developers both good and bad to sometimes want to make projects more interesting and there can be a lot of temptations.  It is often tempting to create a custom implementation of something that could be found already written because it’s a fun project or the temptation to shoehorn a new hot technology into a project or even the temptation to add a solution to a problem that doesn’t need to be solved or just strait up over engineering.    I am guilty of some of these transgressions myself over the years. 

My anecdote about the menu app is just one of at least a dozen of egregious examples of these types of software project blunders that I have seen over the years.  My anecdote shows a manifestation of this problem and it also shows that these types of software project issues have a real tangible cost.  They create more code and often more complex code.  In some cases they create the need for specialized technical knowledge that wasn’t even necessary for the project.  While these approaches may only result in a small cost increases during the construction of the software the real consequence will be the long term maintenance costs of a software system that was needlessly overcomplicated.   In some cases these costs will be staffing costs but they can also be time costs in that changes and maintenance take considerably longer.  These problems may also result in costs to business as unwieldy systems cause the loss of customers to competitors and can impede the ability to gain new customers.

These types of decisions can be viewed as software project blunders.  In some cases the developers know better but choose to be selfish in that they put their own desires above the project, in these cases these decisions can potentially be viewed as unethical.  The two developers from the anecdote were extreme examples and were pushing into unethical territory since they treated every new project as a personal opportunity to use some new technique or library or some custom idea that they were interested in.  However, I think most of these blunders are simply that, mistakes in judgment or lack of good oversight.  It is always easier in hindsight to see these problems, but when one is in the heat of the moment on the project one can lose perspective.  The real challenge is trying to avoid these blunders. Really these are problems that fall into a larger spectrum of software project management problems and sadly I have mostly seen terrible software project management throughout my career.

I wish I could say I have a solution to this problem.  I think developers have some responsibility here especially senior developers, well at least the good ones.  Software development is really about decisions.  Developers make many decisions every day including naming, project structure, tools, libraries, frameworks, languages, etc.  Too often these decisions are made in isolation without any review and sometimes they can have serious long term consequences.  When making so many decisions in such a complex domain mistakes are inevitable.  Good software developers and good teams make an effort to deliberate and review their choices.