26 September 2016

Software Development is a Risky Business

Software projects are inherently risky.  Some statistics show that roughly seventy percent to fifty percent of software projects get classified as “challenged”.   Where challenged can mean cost overruns and/or delivery delays, or it can mean complete failure and the cancellation of the project.  Some statistics indicate as the size of the project increases so does the probability of problems and failure.  Many articles and blog posts have been written about the long history of software project failure.  At the time of writing this post the most recent and very public software project debacle was the healthcare.gov project.  It seems as IT becomes more pervasive and essential the problems with projects may be getting worse not better.

In spite of these numbers and all too often in my experience I have found that individuals and organizations seem to have a very optimistic outlook about their own projects especially at the beginning of the project.  Perhaps it’s the classic: “that won’t happen to me” attitude.  Even as projects start to run into problems I have seen people especially managers not take issues seriously or ignore them all together.  In some of the analysis of healthcare.gov, this was cited as one of the problems.

Risk management and risk assessment/analysis are developed areas in project management and engineering and are often discussed in software projects.  Many risks will end up falling into two categories.  One category is actual risks to the end product, such as unforeseen security vulnerabilities in software systems or other engineering issues that lead to flaws or critical failures of the system or components. The other is risks in the management and execution of constructing product or system. 

Risk management is one of those areas where the engineering aspects and the engineering management aspects can become tightly coupled.  An example can be derived from the XKCD comic “Tasks”: https://xkcd.com/1425/.  Now the point of this comic is to demonstrate the difference between an easy problem and hard one, and presumably the conversation takes place between a manager and developer.  In a risk setting, perhaps solving the hard problem is the goal of the project.  So a risk might be that the research team fails to solve that problem at the end of the five years, or that it takes ten. So in a sense the task is high risk but there are team related risks, maybe the budget is too low and the team is subpar due to low salaries so they can never even solve it.  Another possible scenario is that the conversation never happened and the project management and team think it is solvable in six months.  This second scenario is the risk I have seen though out most of my career, the project management and team are overly optimistic or really don’t have a good understanding of the complexity of the task at hand.  In some cases these types of issues seem to be due in part to pluralistic ignorance.  Often in cases where developers don’t want to contradict management.  There’s a great dramatization of this type of behavior in the show Silicon Valley when the Hooli developers are working on the Nucleus project, where the information is conveyed up the management chain but nobody wants to tell Gavin the truth.

Sadly it seems like the risks that are identified are almost peripheral to the real risks on projects.  It is usually things like the hardware delivery will be delayed which will delay setting up our dev/test/prod environments and delay the start of development.  While these are legitimate risks, there is always the risk of all of the unknowns that will cause delays and cost over runs, even things like developer turnover never seem to be taken into account, at least in my experience.  Also there seems to no cognizance of the long term cost risks like the project was poorly managed or rushed or was done on the cheap which caused the delivered codebase to be an unmaintainable big ball of mud that will incur a much higher long term costs.

In order for software engineering to grow up and become a real engineering discipline it will need formal risk assessment and risk management methodologies both on the project management side and on the work product side.  This is also an area where ideas and methodologies can potentially be taken from existing work in other engineering disciplines.  These methodologies will also most likely draw from probability and game theory.

References and Further Reading

31 May 2016

My Ant Symbiosis Stories

This post is a little off topic from my normal posts.  Although, since I consider my blog to be about programming, software engineering, math and computer science, perhaps taking a brief foray into the natural sciences is warranted as it is science and I am a science nerd, and of course it is my blog.  The book that I am referencing for this post is: The Ants by E.O. Wilson and Bert Hölldobler. It has many great examples of data visualization.  There is also a CS relevant area of study called ant hill algorithms.  Additionally, I believe that the study of social biology is basically just game theory which again is a topic that comes up in CS.

The natural world is fascinating and it teams with many interesting life and death stories that play out every minute of every day in the theatre of nature.  Some of the most fascinating plots include various types of symbiosis that occur between species. Symbiosis, also called mutualism, is a cooperative relationship between two species that sometimes includes an exchange of “survival favors”.  A fairly famous example of this type of species interdependence is the story of why Brazil nuts produce very poorly in plantations.  It turns out the Brazil nut tree Bertholletia excelsa is dependent on a pollinator insect, an orchid bee Euglossa, which is attracted by an orchid, Coryanthes Vasquezii  which does not grow on the tree but is present in the undisturbed forest habitat.  Finally the tough outer shell needs large rodents like the Agouti to gnaw it open, they much like squirrels consume some of the seeds and also bury some for later which allows some of the forgotten seeds to germinate.  There are many examples of symbiotic relationships in rainforest and other habitats.

Ants are notorious for having symbiotic relationships with plants and other insects and many such cases are well documented. One case is the Neotropical Cecropia tree which provides hollow stems for Azteca ants to inhabit and even food in the form of gylcogen-rich Müllerian bodies.  There was another story about a relatively recent discovery of a plant that grows on the harsh cliffs of Pyrenees that depends on three species of ants for pollination and seed dispersal.  These types of stories are not uncommon in the natural world, however, one often reads of them playing out in the above and often exotic and/or tropical locations, so I found myself rather surprised to discover this type of drama transpiring literally inches from my living room window in suburban Arlington Virginia. 

The story I am going to tell happened during the summer of 2011 and I had originally labored under the assumption that it was a somewhat unique undocumented story.  However, my subsequent research has yielded a number of pictures which depict many of the events that I will describe and show here often with better photographic quality than what I what I captured.  Not that my photos are bad but they lack the fidelity of others I have seen.  I make up for this in quantity which captures various events over an extended period of time.

One of my hobbies is identifying eastern forest native plants and experimenting with growing them in my yard.  This activity, probably to the chagrin of my mcmansioned neighbors, has turned my yard into a small native plant and wildlife habit.   A few years ago I collected some seeds of a plant known as Yellow Leaf Cup, Bearpaw, or Bearsfoot, which I believe goes by the Latin name Smallanthus Uvedalia formerly known as Polymnia Uvedalia.  It is in the Aster Family (Asteraceae).  I planted some in pots and scattered some seeds in my yard, the potted ones germinated but did not make it however some took root right in front of my living room window.  To me this plant really stands out, for a temperate zone plant it has very large leaves, 12 to 18 inches long, and as a result it almost completely obscures my living room window shrouding it in giant green veined leaves as you can see in the picture at the top of this post.  The location of these leaves put me in a unexpected and unique position to watch an incredible and probably epoch old symbiotic drama of nature play out between Ants(Hymenopterans) and Hemipterans

In June 2011 as the stand of Large Flowered Leaf-Cup reached about eight or ten feet in height and completely blocked my living-room window providing a nice screen of green privacy, an odd looking insect, pictured below, showed up.  As you can see it’s shaped like a thorn which has garnered its ilk the general common name of thorn bug it is also referred to as a tree hopper this class of insects that falls under the order Hemiptera, and I believe it is the species Entylia carinata. It does not have a common name.  I think it needs a common name and in that picture I see a rhinoceros looking insect, thus I am coining it the rhinoceros thorn bug or just the shorter version of rhino thorn bug.  If you look closely at the picture below you can see its eyes and legs which are hard to see in many of the photos, my guess is this is by design as thorns don’t have eyes and legs.

Shortly after its arrival the ants showed up to tend to it:

I contacted a biologist who specializes in ants and he identified them as a species of carpenter ants in the genus Camponotus. He told me the possible species for this area are: Pennsylvanicus, Nearcticus, and Subbarbatus.

Over the following months I watched the interesting drama play out with these two species. Owning a copy The Ants gave some perspective to the traits and behaviors of these species.  Over time I captured over one thousand pictures of what played out between these two species.  This collection is really a dataset of photos and it revealed some interesting observations, which while not novel, still make for an interesting story.

Rhino Thorn Bugs: Masters of Camouflage

First let’s start with how Rhino Thorn Bug disguises itself.  A number of the photos of it were taken in different lighting conditions.  In the above photos the light is more direct and the yellow and green “veins” of both the plant and insect mesh and it looks like a thorn on the stem.  In the following lighting the Rhino Thorn Bug appears to have purplish veins that blend with the background of plant veins.

 Another outcome of the Rhino Thorn Bug living on smallanthus uvedalia is the damage to the leaves which also makes for a nice abstract photo:

I’m making a leap here, but it seems that this damage extends the camouflage range for the rhino thorn bug such that it can hang out on the leaf and look like damaged parts of the leaf that more closely match its brownish color and pseudo-veins in back lit conditions:

I also noticed that it seemed to stay in one place and was able to feed by making multiple bites into the thick main leaf vein, I suspect that the major vein might be a bundle of veins and it was tapping into individual sub-veins each time.  This is visible in a subsequent photo.

Protection and Feeding

Ant Symbiosis with other insects takes a number of forms. A common symbiosis is known as Trophobiosis where ants feed on what is known as honeydew.  This substance is a mixture of sugars that the insect excretes from its posterior.  It is often the case that the provider of this mixture breaks down more complex polysaccharides into simpler sugars that the ants can digest these include glucose, fructose, etc.   The not so great image below shows this an ant feeding from the posterior of the rhino thorn bug:

This relationship is not unfamiliar to humans. It seems to mirror our use of livestock especially dairy cows, although we get the fluid we consume from a different organ.  However, the behavior of protecting, caring for and in some cases herding these insects is exactly what we do with our livestock.  The ants have been doing this millions of years prior to our existence as the ant Hemiptera symbiosis behavior is thought to have originated in the Eocene period.  There is evidence of this, shown below, from about 15 million years ago found in Dominican Amber which shows a queen from the extinct species Acropyga glaesaria on her mating flight with a “seed” mealybug in her mandibles for use when she establishes her new colony.

Another ant behavior that I believe I caught is ants transferring food to each other, which is known as Trophallaxis.  In the image below you can see the two ants at the top with their mandibles interlocked, in a previous picture one of them was feeding and now seems to be transferring food to the other.


Ants have different castes of workers. The image below shows two different castes I observed.  The one at the top is a Major worker.  Ants of this caste are in some cases referred to as soldiers as they in some species take on roles like acting as “doormen” at colony entrances to ensure that only ants of that colony are allowed.  It easy to see that the larger body size with the large head and large mandibles makes them more formidable, although I don’t know if the solder distinction truly applies in this case.  The other two are minor workers which have smaller bodies and heads. 

Rhino Thorn Bugs, the Next Generation

As the story unfolded it got even more interesting. I recall, although I don’t have photos of it that the ants seems to escort the thorn bugs to meet each other for mating.  The evidence of mating was soon obvious as can be seen in the photo below with the ants in attendance of the brood of recently hatched nymphs.  The ants tended to the nymphs which included chasing them around as they wandered off. Some of the nymphs seemed to stray and many of them died.  Also visible in this photo are the multiple bites left by what was presumably the female rhino thorn bug made over her extended time staying in one location.

Pretty soon the nymphs are providing the same trophobiosis as their parents as shown below:

As the nymphs grow up you can see that they start to take on their adult shape:

The Next Year, New Ants and Aphids

I was curious to see if the thorn bugs would return the following year. They did not.  However, I was surprised to a whole new ant drama play out with new actors.   I reached out to the ant biologist who helped me the previous year and he identified these as Tapinoma sessile.  I was not able to identify the aphids.  I was told they would be plant specific.

Their activity was concentrated around the flowers and spanned the time that they were in bloom.  The “infestation” of the flower heads can be seen in the following photo:

Another difference was the ratio of ants to honeydew producers, as you can see there are many aphids to each ant which in contrast to the rhino thorn bugs which often had multiple ants in attendance of each one.  The biologist that I contacted said that the ants looked fat and happy, and their abdomens or gasters in ant biology speak, seem to be bursting and almost glistening, presumably with honeydew.   I once saw David Attenborough eat a honeypot ant and it got me thinking I should have tried it with these guys. Well that was until I learned that their common names include odorous house ant and stink ant.

The photo below shows how the flower’s sepal seems to serve as a protected staging area for the aphids to develop to full maturity:

All of this happened in 2012. I have not seen the aphids since. I do see the rhino thorn bugs at least one or two each year over the last few years.  I have never seen the level of activity that I saw back in 2011, I am always hopeful that each new year will bring some new drama. Actually last year it was an opiliones explosion, but that’s a whole other story. 

The story about the Brazil nut symbiosis is interesting and was a little bit of a setup to show my cute seed disperser below.  Although the plant is obscured by another (Wingstem, Verbesina alternifolia) It is collecting Large Flower Leafcup seeds, the flower heads are visible slightly right and below it.

References and Further Reading

30 April 2015

What is Software Engineering?

The terms software engineering, software engineer, and sometimes just engineer as in "full stack engineer" appear a lot in the software development field.  Some people including myself find the use of the term "Software Engineering" to be somewhat dubious. I also feel that there are too many people calling themselves engineers who really shouldn’t be.  As a result of my skepticism I have been trying to devise a way to determine exactly what is Software Engineering?

The happy path is to go to the web and search for it or just find a book. Both will lead to many answers.  But that is really just getting someone else’s opinion.  A little later we will look at other people’s opinions as there are very established authors who readily answer this question.

To get some perspective, let’s step back and ask a more basic question. What is engineering?  For this I will rely on the ultimate oracle of our time. Some top responses from the Google are:

The Google dictionary definition:



the branch of science and technology concerned with the design, building, and use of engines, machines, and structures.

• the work done by, or the occupation of, an engineer.

• the action of working artfully to bring something about.

"if not for Keegan's shrewd engineering, the election would have been lost"

The Georgia Tech School of Engineering provides the following:

Engineering is the practical application of science and math to solve problems, and it is everywhere in the world around you. From the start to the end of each day, engineering technologies improve the ways that we communicate, work, travel, stay healthy, and entertain ourselves.

What is Engineering? | Types of Engineering:

Engineering is the application of science and math to solve problems.  Engineers figure out how things work and find practical uses for scientific discoveries.

Wikipedia: defines it as

Engineering (from Latin ingenium, meaning "cleverness" and ingeniare, meaning "to contrive, devise") is the application of scientific, economic, social, and practical knowledge in order to invent, design, build, maintain, research, and improve structures, machines, devices, systems, materials and processes.

So to summarize and synthesize the above for a working definition, we get:

Engineering is a branch of science and technology where mathematical, scientific, economic, and social knowledge are applied artfully, creatively, and efficiently to invent, design, build, maintain, research, and improve structures, machines, devices, systems, materials and processes.

Another aspect of engineering that should be considered is its diversity. Just searching around I came up with the following list of types of engineering, which should by no means be considered complete, they include:  Aeronautical, Agricultural, Audio, Biomedical, Biomechanical, Building, Ceramics, Chemical, Civil, Computer, Electrical, Electronics, Environmental, Food, Geological, Industrial, Marine, Mechanical, Military, Mineral, Military, Nuclear, Optical, Petroleum, Photonics, Systems, and Textile.

Often discussions of software engineering draw parallels to construction and civil engineering and it is easy to see why, most people often see the results of these disciplines and even live in them.  Not to trivialize these disciplines, but bricks and mortar, in addition to wood, glass, and rebar are things that support our lives and are physical thus somewhat easy to conceptualize.  But all engineering disciplines now touch and perhaps run our lives in ways that we rarely think about.  I previously posited that chemical engineering, prior to the software era, was probably the most pervasive engineering in our lives not to mention that the computer hardware we use wouldn’t exist without it.  It serves as an interesting area to look at in regards to software engineering comparisons too, as it is a relatively recent discipline and has some parallels like the control of processes and "flows" in this case of materials.  Perhaps the closest "relative" to software engineering might be industrial engineering.  Like software it deals with human factors.  It also deals with stochastic processes, something that is becoming increasingly more important in software systems.

Now that we have some general engineering perspective, let’s revisit the question "What is software engineering" by looking at what the established organizations and authors have to say.

Wikipedia gives the following definition:

Software engineering is the study and an application of engineering to the design, development, and maintenance of software.

Typical formal definitions of software engineering are:

  • the application of a systematic, disciplined, quantifiable approach to the development, operation, and maintenance of software
  • an engineering discipline that is concerned with all aspects of software production
  • the establishment and use of sound engineering principles in order to economically obtain software that is reliable and works efficiently on real machines.

K.K. Aggarwal provides us with a bit of history:

At the first conference on software engineering in 1968, Fritz Bauer defined software engineering as "The establishment and use of sound engineering principles in order to obtain economically developed software that is reliable and works efficiently on real machines".

Stephen Schach defined the same as "A discipline whose aim is the production of quality software, software that is delivered on time, within budget, and that satisfies its requirements".

The ACM weighs in with:

Software engineering (SE) is concerned with developing and maintaining software systems that behave reliably and efficiently, are affordable to develop and maintain, and satisfy all the requirements that customers have defined for them. It is important because of the impact of large, expensive software systems and the role of software in safety-critical applications. It integrates significant mathematics, computer science and practices whose origins are in engineering.

There are a number of books on software engineering which have an obligation to address the question.

The answer in Ian Sommerville’s 8th Edition of Software Engineering  is:

Software engineering is an engineering discipline that is concerned with all aspects of software production from the early stages of system specification to maintaining the system after it has gone into use. In this definition, there are two key phrases:

1. Engineering discipline Engineers make things work. They apply theories, methods and tools where these are appropriate,. but they use them selectively and always try to discover solutions to problems even when there are no applicable theories and methods. Engineers also recognise that they must work to organisational and financial constraints, so they look for solutions within these constraints.

2. All aspects of software production Software engineering is not just concerned with the technical processes of software development but also with activities such as software project management and with the development of tools, methods and theories to support software production.

What Every Engineer Should Know about Software Engineering by Philip A. Laplante offers the following:

Software engineering is "a systematic approach to the analysis, design, assessment, implementation, test, maintenance, and reengineering of software, that is, the application of engineering to software. In the software engineering approach, several models for the software life cycle are defined, and many methodologies for the definition and assessment of the different phases of a life-cycle model" [Laplante 2001]

In Software Engineering A Practitioner's Approach 8th Edition Roger S. Pressman and Bruce R. Maxim provide a terse explanation:

Software Engineering encompasses a process, a collection of methods [practice] and an array of tools that allow professionals to build high-quality computer software.

Roger Y. Lee describes it in Software Engineering:A Hands-On Approach as:

Software engineering is the process of designing, developing and delivering software systems that meet a client’s requirements in an efficient and cost effective manner. To achieve this end, software engineers focus not only on the writing of a program, but on every aspect of the process, from clear initial communication with the client, through effective project management, to economical post-delivery maintenance practices. The IEEE Standard glossary of software engineering terminology defines software engineering as the application of a systematic, disciplined, quantifiable approach to the development, operation and maintenance of software (Ghezzi et al. 1991). As made clear by the above definition, software engineering came about in order to reverse early development practices by establishing a standard development method that could unify and normalize development operations, regardless of the people involved or the goals sought.

The laborious effort of wading through all of this erudition gives me the following distillation:

Software Engineering is:

  • An engineering discipline
  • Application of a systematic, disciplined, quantifiable approach to the development, operation, and maintenance of software"
  • Software engineering (SE) is concerned with developing and maintaining software
  • Software engineering is the process of designing, developing and delivering software systems
  • A standard development method that can unify and normalize development operations, regardless of the people involved or the goals sought.

Software Engineering Includes:

  • All aspects of software production from the early stages of system specification to maintaining
  • The system after it has gone into use
  • Non technical activities such as software project management and with the development of tools, methods and theories to support software production.
  • The integration of significant mathematics, computer science and practices whose origins are in engineering.
  • The application of theories, methods and tools where these are appropriate
  • Many methodologies for the definition and assessment of the different phases of a life-cycle model
  • Several models for the software life cycle
  • A process, a collection of methods [practice] and an array of tools that allow professionals to build high-quality computer software.
  • The application of sound engineering principles to the analysis, design, implementation (development), assessment, test, maintenance and reengineering of software that is reliable and works efficiently on real machines

Software Engineering Produces:

  • Quality software
  • Software that is delivered on time
  • Cost effective production of software (within budget)
  • Software that satisfies its requirements.
  • Software systems that behave reliably and efficiently
  • Software that is affordable to develop and maintain
  • Solutions that conform to organizational and financial constraints

I tried to break the multitude of ideas into three categories, but they are admittedly a bit fuzzy and you could probably move some of these around.  The point of the categorization exercise is to attempt to separate a definition from the broader idea of what the discipline encompasses and the attributes of the structures and systems that it produces, again things can get fuzzy here.  In thinking about these and my own experience with software development I came up with a list of things that I feel should be defined to make software engineering a real discipline and they are:

  1. What is the definition of the engineering discipline we are describing?
  2. What are the produced artifacts, e.g. structures and processes that the discipline creates?
  3. What non-managerial processes and methodologies are used to create the produced artifacts?
  4. What types of documentation (descriptive artifacts) are used to describe the produced artifacts and the processes to create those produced artifacts?
  5. How does the underlying math and science support the engineering discipline and how do these foundational disciplines allow one to reason about the produced artifacts that the engineering discipline is trying to construct?  How do these disciplines define and drive the descriptive artifacts?
  6. How does one organize and manage the effort including various roles people will play to actually carry out the work to bring the system or structure to fruition?

So coming back to the original purpose of this post, my answer to the question of what is software engineering, number one above, is:

Software engineering is the branch of science and technology where mathematical, computer science, linguistic, social knowledge and system relevant domain specific knowledge are applied artfully, creatively, and efficiently to invent, design, build, maintain, research, and improve software systems.

That’s the easy part. The hard part is what are the answers to the other five questions? I know that there a number of industry standard answers for these that include things like UML, Agile, Scrum, etc., but are those really valid and are they really working? I don’t think so, so maybe it’s time to look for other answers.

23 March 2015

What is Software Correctness?

Software correctness which is really software quality is not one thing.  It is comprised of a number of different and sometimes conflicting attributes.  As always Gerald J. Sussman provides interesting insights and in this case it is in his "We really don’t know how to compute" talk.   He makes the point that "correctness" may not be the most important attribute of software, the example he give is if Google’s page craps out on you, you will probably try again and it will probably work.  Actually this example might be more about reliability but it got me thinking as I have always thought that "software correctness" meant the combination of both accuracy and reliability and that was this the "correctness" that I always thought was the most important feature of software.  But this correctness is really two software quality attributes and the Google search software exemplifies these two aspects of "software correctness", one is availability and the other is search accuracy or the fulfillment of the users expectations.  One could argue that availability is not a software attribute but it is affected by the quality of the software and it is an attribute of a software system even if some of that might be driven by hardware.   So these two attributes for Google search do not have to be perfect.  The search quality needs to be at least better than the competition and the availability has to be high enough so that people won’t be discouraged from using it and for these two attributes they are both very high.   These parameters of quality vary in other domains, a machine administering a dose of radiation to a cancer patient would need to have stringent requirements as to the correctness of the dose, and in fact there have been deaths due to exactly this type of problem.  Another is software guiding a spacecraft or a Mars rover which needs to be extremely reliable and correct and when it hasn’t been, well there are a few space bricks floating around the solar system due to software errors.  In the real world, especially now, software is everywhere and it now lives in a variety of contexts.  So which quality attributes are important will vary based on its domain and its context.

So what makes software good?  Google first conquered search with Map-Reduce and other processes running on custom commodity hardware that allowed for flexible fault tolerant distributed massive data crunching.  Amazon has commoditized its server infrastructure to sell spare capacity that can be easily used by anyone.  Apple created easy to use attractive interfaces for the iPhone and iPad that consumers loved with similar usability now being offered on Android.  NASA has successfully landed on the frigid world of Titan, touched the edge of the heliosphere, and has run rovers on Mars.  While there are obviously hardware components to these systems, it was the various types of software quality that played a pivotal role in these and the many other great software successes.

It is easy to look back at these successes.  With the exception of NASA, I doubt there was a deliberate effort on which quality aspects to focus on.  Most were probably chosen intuitively based on business needs.  Software quality is essential to any organization that creates software.  It is an established subdomain or perhaps a related separate sibling field of software development.  It has consortiums, professional organizations and many books written on the subject.  So I delved into some of these and you find mention of Agile and TDD.  You also start to see things like CMMI, ISO 9000, Six Sigma.  Some of the resources seem to be written in some alien language, perhaps called ISO900ese, that seems related to software development but leaves me feeling very disconnected from it.   I think that the problem might be that these attempts to formalize software quality as a discipline are suffering from the same problem that plagues the attempt to formalize software engineering as a discipline.  It conflates the development process with the final product and the software quality formalization might be further exacerbated by the influence of the domain of manufacturing and its jargon.  I think these established methodologies have value, for example makes sense, but it seems like in general with these standards there is a lot to weed through and the good parts seem to be hard to pick out.  So I thought it might be fruitful to look at the problem from another angle that might provide a better way to reason about the quality of a system not only after and during its development but also before development begins.

A list of "quality parameters" for software is a requirement for software engineering to be a real engineering discipline.  These parameters could then be viewed qualitatively and quantitatively and perhaps even be measured.  In engineering disciplines various parameters are tradeoffs, you can make something sturdier but you will probably raise the cost and or the weight, for example plastic bumpers on cars, metal would be more durable but would be heavier and more costly.  If one were to think of some parameters in software as tradeoffs you can see that improvements on some parameters may cause others to go down.  In my post "The Software Reuse Complexity Curve" I posit that reuse and complexity are interrelated and that the level of reuse can affect understandability and maintainability.  Another example is performance, increasing performance of certain modules or functions might lead to an increase in complexity which without proper documentation could lead to harder understandability which might cause a higher long term maintenance cost.

So to come back to the question and to rephrase it:  What are the attributes that constitute software quality?   I am not the only person to ask this question as I have seen other blog posts that address this. There are quite a few people who have asked this question and have compiled good lists.  One post by Peteris Krumins lists out some code quality attributes that Charles E. Leiserson, in his first lecture of MIT's Introduction to Algorithms, mentions  as being more important than performance.   Additionally Bryan Soliman’s post "Software Engineering Quality Aspects" provides a nice list with a number of references.  I thought I’d create my own list derived from those sources and my own experience.  I am categorizing mine in three groups, Requirements based (including nonfunctional requirements), structural, and items that fall into both categories.


  • Performance
  • Correctness (Requirements Conformance)
  • User Expectations
  • Functionality
  • Security
  • Reliability
  • Scalability
  • Availability
  • Financial Cost
  • Auditability


  • Modularity
  • Component Replaceability
  • Recoverability
  • Good Descriptive Consistent Naming
  • Maintainability/ Changeability
  • Programmer's time
  • Extensibility/Modifiability
  • Reusability
  • Testability
  • Industry standard best practices (idiomatic code)
  • Effective library use, leveraging open source (minimal reinvention)
  • Proper Abstractions and Clear code flow
  • Good and easy to follow architecture/ Conceptual Integrity
  • Has no security vulnerabilities
  • Short low complexity routines


  • Robustness/Recoverability
  • Usability
  • Adaptability
  • Efficiency
  • Interoperability/Integrate-ability
  • Standards
  • Durability
  • Complexity
  • Installability

This is by no means intended as a definitive list.  The idea here is to put forth the idea of enumerating and classifying these attributes as a way to reason about, plan and remediate the quality of software projects, basically create a methodical way to address software quality throughout the software development lifecycle.  In the case of planning software quality attributes can be potentially rated with respect to priority.  This could also allow conflicting attributes to be more effectively evaluated. 

The attribute of availability is one that is fairly easy to quantify and often gets addressed usually in some type of Service Level Aggreement (SLA).  Although availability cannot be predicted, the underlying system that "provides" it can be tested. Netflix has developed chaos monkey to in part to verify (test) this quality attribute of their production system.  Tests and QA testers are a way to verify the requirements correctness and the correctness of system level components and services contracts of software. Static analysis tools like PMD, Checkstyle, and Sonarqube can help with code complexity, code conventions and even naming. 

Complexity is an elusive idea in this context it also crosses many of the above ideas.  Code can be overly complex, as can UIs, Usability (think command line), Installability, etc.  Another cross concept is understandability which can affect code, UIs, etc. which is in part related to complexity but can manifest itself in the use of bad or unconventional nomenclature as well.  There may be other cross concerns and it seems that the spectrum of quality attributes might better reasoned about by separating out these concepts and developing general ways to reason about them.

I should note that I include financial cost as one of the attributes.  While it might not be a real quality attribute, higher quality may necessitate higher cost, but not necessarily.  If one were to develop two systems that had the same level of quality across all of the chosen attributes, but one system had twice the cost, the cheaper one would be a better value, hence more desirable. So cost is not a quality attribute per se, but it is always an important factor in software development. 

In summary my vision is a comprehensive methodology where the quality attributes of a software system are defined and prioritized and hopefully measured.  They are planned for and tracked throughout the life cycle and perhaps adjusted as new needs arise or are discovered.  The existing tools would be used in a deliberate manner and perhaps as thinking about the quality attributes becomes more formalized new tools will be developed to cover areas that are not currently covered.  It will no longer be about code coverage but quality coverage.

References and Further Reading

Anthony Ferrara, Beyond Clean Code,  http://blog.ircmaxell.com/2013/11/beyond-clean-code.html

David Chappell,  The Three Aspects of Software Quality: Functional, Structural, and Process, http://www.davidchappell.com/writing/white_papers/The_Three_Aspects_of_Software_Quality_v1.0-Chappell.pdf

Rikard Edgren, Henrik Emilsson and Martin Jansson , Software Quality Characteristics,


Bryan Soliman, Software Engineering QualityAspects, http://bryansoliman.wordpress.com/2011/04/14/40/

Ben Moseley and Peter Marks , Out of the Tar Pit, http://shaffner.us/cs/papers/tarpit.pdf

Brian Foote and Joseph Yoder, Big Ball of Mud, http://laputan.org/mud/

Wikipedia , Software quality, http://en.wikipedia.org/wiki/Software_quality

Franco Martinig, Software Quality Attributes, http://blog.martinig.ch/quotes/software-quality-attributes/

MSDN, Chapter 16: Quality Attributes, http://msdn.microsoft.com/en-us/library/ee658094.aspx

Advoss, Software Quality Attributes, http://www.advoss.com/software-quality-attributes.html

Eric Jacobson, CRUSSPIC STMPL Reborn, http://www.testthisblog.com/2011/08/crusspic-stmpl-reborn.html