<img height="1" width="1" style="display:none" src="https://www.facebook.com/tr?id=266211600481844&amp;ev=PageView&amp;noscript=1">

Unit Testing - Days of Future Past for Software Development

May 20, 2014 Software Eric Potter

Software bugs plague our industry. Recently, the Heartbleed bug compromised millions of passwords. A unit conversion bug was responsible for the loss of the Mars Climate Orbiter. This is not a new problem. Computer scientists identified this problem as the 'Software Crisis' as far back as the 1960s.

The major cause of the software crisis is that the machines have become several orders of magnitude more powerful! To put it quite bluntly: as long as there were no machines, programming was no problem at all; when we had a few weak computers, programming became a mild problem, and now we have gigantic computers, programming has become an equally gigantic problem.[1]

Edsger W. Dijkstra

That quote is from Edsger Dijkstra, as are all of the quotes in this post. He was a brilliant computer scientist in the early days of the profession. He coined the term 'structured programming'. He invented the graph search algorithm that bears his name. He used his considerable influence to push the GOTO statement out of favor. In 1972, the ACM honored him with the Turing Award. In his acceptance speech, Dijkstra proposed putting an end to bugs by the end of the 1970s.

The vision is that, well before the seventies have run to completion, we shall be able to design and implement the kind of systems that are now straining our programming ability, at the expense of only a few percent in man-years of what they cost us now, and that besides that, these systems will be virtually free of bugs.[1]

Dijkstra knew that developers cannot test bugs entirely out of system, they have to prevent bugs from entering in the first place.

Program testing can be used to show the presence of bugs, but never to show their absence![2]

But his proposed solution was flawed. He wanted reliability to come through rigorous mathematical proofs. Proofs proved too cumbersome to gain widespread adoption and are difficult to apply in many circumstances. But he was right about many things. He knew that whatever validated the software had to grow up with the software.

But one should not first make the program and then prove its correctness, because then the requirement of providing the proof would only increase the poor programmer’s burden. On the contrary: the programmer should let correctness proof and program grow hand in hand.[1]

What is fascinating about that statement is how much it sounds like he could have been talking about unit testing. Advocating that the developer write the proof at the same time as the program is the heart of test driven development. To be clear, Dijkstra was not talking about what we now know as unit testing, but it is an interesting thought experiment to imagine an alternate reality where that is what he did.

Days of Future Past

In the movie X-Men: Days of Future Past, the X-Men live in a world where they have almost destroyed each other because of their infighting. The general population doesn't trust them and wants to be rid of them. The solution is to send someone back to the 70s to convince a young Professor Xavier to use his influence to unite the X-Men and prevent the mutant wars.

Days of Future Past Professor X

In our industry, we live in a day when software developers have wasted a lot of time and energy with needless infighting. The general population doesn't trust our software because historically it has been unreliable. What if we could send someone back to convince Professor Dijkstra to use his influence to bring unit testing into widespread use in the 70s?

220px Edsger Wybe Dijkstra

Let's do a thought experiment in which Dijkstra is referring to unit tests whenever he used the world "proof". This will allow us to gain insight into the modern development process from his genius. This is a viable thing to do because unit tests provide many of the outcomes that he wanted to achieve with mathematical proofs.

The goal of a proof was to give the programmer confidence that the code behaved as desired without needing to do system testing. The programmer would develop the proof for a module or subroutine before writing the code, that way the problem was totally understood. Then the programmer would write the code, with the proof ready to validate the code as soon as it was finished. Unit testing can accomplish all of this, so let's begin our experiment.

For, how does one develop a program that admits a nice correctness proof? Well, by developing the program and its correctness proof hand in hand. In actual fact the correctness proof is often even developed slightly ahead of the actual program text: as soon as the next step in the correctness proof has been chosen, the next refinement of the program is made in such a way that the chosen step in the correctness proof is applicable to it. Instead of seeking for a proof to go with a given program, we now construct a program to go with a chosen correctness proof! [3]

Dijkstra could see the importance of developing the proofs "hand in hand" with the program. If tests are developed after the software is written, they become an additional burden to the developer. If they are written at the same time as the code, they provide value to the developer, making the development task easier. If the code needs to be changed later, the proof becomes an instant regression test.

Dijkstra also saw the importance of building software that can be tested.

it was no longer sufficient to design a correct program, in addition the program should be designed in such a way that its correctness could, indeed, be established. [3]

In order for software to be testable, it must have a healthy separation of concerns. This also improves the understandability, maintainability, extensibility, and scalability of the code. Making the code testable improves the code even if the tests aren't written.

Why do we need it

One of the reasons that it is so difficult to write reliable software is that modern software is so complex. For most programs, it would be impossible for any one person to know how the entire system worked. As far back at the 1960s, Dijkstra saw software getting to be unmanageably complex.

I am a programmer and my direct concern is how to avoid the complexity that makes larger programs intellectually unmanageable and, as a result, not a safe tool to use. [5]

If that is what he thought about software in the 60s and 70s, imagine what he would think about our software today. One of the key things a programmer must do is manage the mental complexity of the code.

...the core challenge for computing science is hence a conceptual one, viz., what (abstract) mechanisms we can conceive without getting lost in the complexities of our own making.[6]

The primary solution that Dijkstra proposed for the complexity problem was abstraction.

Argument four has to do with the way in which the amount of intellectual effort needed to design a program depends on the program length. It has been suggested that there is some kind of law of nature telling us that the amount of intellectual effort needed grows with the square of program length. But, thank goodness, no one has been able to prove this law. And this is because it need not be true. We all know that the only mental tool by means of which a very finite piece of reasoning can cover a myriad cases is called “abstraction”; as a result the effective exploitation of his powers of abstraction must be regarded as one of the most vital activities of a competent programmer.[1]

Abstraction allows us to suppress details in order to focus our thought on big pictures problems. Imagine how difficult it would be to drive a car if you had to think about all the things that happened when you put your foot on the accelerator. We should focus on getting home. So we simply think about pushing the pedal and appreciate that the car goes forward.

Long pieces of code can be difficult to understand, in turn making them difficult to develop and maintain. If chunks of the code can be extracted into other methods or other classes, the understandability is vastly improved. But understandability doesn't guarantee reliability. But when the code what was extracted is unit tested, we can be confident in it's functionality. This allows us to abstract the code mentally.

Provide Value

There is an ongoing debate in our industry about exactly how to use unit testing during software development. I believe in the pragmatic approach. Let's use unit testing in ways that provide value to our customers by producing better software. Unit tests are not our goal.

We must not forget that it is not our business to make programs, it is our business to design classes of computations that will display a desired behavior.[1]

Our goal is to provide value to our users and to do so on time and on budget. An important aspect of providing value to is to make sure the software is reliable. If there is one theme in all of Dijkstra's writing, it is that as an industry we must improve our standards of quality.

I want you to gain, for the rest of your lives, the insight that beautiful proofs are not "found" by trial and error but are the result of a consciously applied design discipline. I want to inspire you to raise your quality standards. I mean, if 10 years from now, when you are doing something quick and dirty, you suddenly visualize that I am looking over your shoulders and say to yourself "Dijkstra would not have liked this.", well, that would be enough immortality for me.[4]

Next time you are about to commit that huge method that seems to work, picture Dijkstra looking over your shoulder. Refactor it into clear and cohesive methods that abstract away implementation details. Write unit tests that prove that it works for all inputs. Don't do it because you are committed to a process. Do it because you are committed to providing value to your users.

Some might say that I have taken some of the quotes slightly out of context. I'll admit that I am trying to coerce some of Dijkstra's ideas into ideas about unit testing. That was part of the thought exercise. I strongly encourage you to read the original papers for yourself.

Posted in:

Topics: Software Code Programming

Comments