Why is design-by-contract not so popular compared to test-driven development?

Question

You may think this question is like this question asked on StackOverflow earlier. But I am trying to look at things differently.

In TDD, we write tests that include different conditions, criteria, verification code. If a class passes all these tests we are good to go. It is a way of making sure that the class actually does what it's supposed to do and nothing else.

If you follow Bertrand Meyers' book Object Oriented Software Construction word by word, the class itself has internal and external contracts, so that it only does what its supposed to do and nothing else. No external tests required because the to code to ensure contract is followed is the part of the class.

Quick example to make things clear

TDD

1. Create test to ensure that in all cases a value ranges from (0-100)

2. Create a class containing a method that passes the test.

DBC

1. Create a class, create a contract for that member var to range from (0-100), set contract for contract breach, define a method.

I personally like the DBC approach.

---

Is there a reason why pure DBC is not so popular? Is it the languages or tools or being Agile or is it just me who likes to have code responsible for itself?

If you think I am not thinking right, I would be more than willing to learn.

Langan

github.com/ljr1981/stack_overflow_answers/blob/main/testing/… – Protoactinium

I pasted a link in my last comment. It takes you to a TDD class written in Eiffel. The thing to notice is how the TDD test is literally two lines. One to call the method (feature) under test and another to test that the result is as expected. – Protoactinium

github.com/ljr1981/stack_overflow_answers/blob/main/src/… – Protoactinium

The code (linked in the last comment) now shows you the power of DbC. When the TEST_SET class test methods call the `parse' method on the class, then all of the appropriate DbC contract assertions will be executed. There is no need to place these in the TDD test code. They are not appropriate there. This little demo is a great demonstration of DbC and TDD working in concert with each other. – Protoactinium

Answer 1

The main problem with DBC is that in the vast majority of cases, either the contract cannot be formally specified (at least not conveniently), or it cannot be checked with current static analysis tool. Until we get past this point for mainstream languages (not Eiffel), DBC will not give the kind of assurance that people need.

In TDD, tests are written by a human being based on the current natural-text specifications of the method that are (hopefully) well-documented. Thus, a human interprets correctness by writing the test and gets some assurance based on that interpretation.

If you read Sun's guide for writing JavaDocs, it says that the documentation should essentially lay out a contract sufficient to write a test plan. Hence, design by contract is not necessarily mutually exclusive with TDD.

Johnsonian

Eiffel has a lot of interesting ideas, it's just not ready for mainstream use. A bunch of my friends are working on DBC formalisms for Java that are pretty cool. I've been working on improved documentation by formalizing "directives" – Johnsonian

Static analysis for contract specifics doesn't work for Eiffel either,it only has nominative typechecking, a really nice implementation, but not contract specific. – Tweeny

Documentation WILL go stale. Devs will not read it (possibly because it's likely stale). Tests you have to run separately, and even then they're very ignorable. DbC's pre/post conditions and invariants execute inline, every invocation, non-bypassable. – Eyot

If you use Python, you can use github.com/Parquery/icontract to define contracts and automatically generate tests with github.com/mristin/icontract-hypothesis or do static analysis with github.com/pschanely/CrossHair. So we are getting there for Python. – Natoshanatron

"The main problem with DBC is that in the vast majority of cases, either the contract cannot be formally specified (at least not conveniently), or it cannot be checked with current static analysis tool." I am answering as an Eiffel programmer: 1. Contracts are "formally specified" by definition. They are Boolean assertions written in code. 2. Because contracts are just Boolean assertion expressions written as formal specification in code, they can be statically checked. Both of these are true in EiffelStudio and have been since the beginning (i.e. the last 35 years or so). – Protoactinium

Answer 2

TDD and DbC are two different strategies. DbC permits fail-fast at runtime while TDD act "at compile time" (to be exact it add a new step right after the compilation to run the unit tests).

That's a big advantage of TDD over DbC : it allows to get earlier feedback. When you write code the TDD way, you get the code and its unit-tests at the same time, you can verify it "works" according to what you thought it should, which you encoded in the test. With DbC, you get code with embedded tests, but you still have to exercise it. IMO,this certainly is one reason that Dbc is not so popular.

Other advantages : TDD creates an automatic test suite that allow detecting (read preventing) regressions and make Refactoring safe, so that you can grow your design incrementally. DbC does not offer this possibility.

Now, using DbC to fail-fast can be very helpful, especially when your code interfaced other components or has to rely on external sources, in which case testing the contract can save you hours.

Gar

I have found DbC useful when creating prototypes where I do not have a clear idea of the data model and business rules, hence I am making big changes to the code as I search for a workable application design. In this situation, TDD would slow down the search for a good design. DbC does not slow down changing the design. The application I developing does complex things with databases, files and metadata within files I would have trouble using TDD due to not easily being able to isolate code that normally reads live data for unit testing. – Stearne

Not sure how DbC permits fail-fast at runtime moreso than TDD, since contracts should in theory provide a more comprehensive compile-time check by covering all possible cases whereas unit tests only cover a few hard-coded cases at compile-time. – Wolfish

As a 20 year Eiffel programmer, I can tell you that there is no such moniker as "fail-fast" (runtime or otherwise). TDD assertions are not applied at compile time either. DbC assertions are embedded as pre and post conditions that execute and pass/fail at run time within whatever context they are executed—that is—testing or production (if you want). Therefore, DbC contract assertions are applied in testing by executing TDD test code. In production code, one has a choice between leaving some, all or no contracts in the finalized executable. – Protoactinium

TDD is not an advantage over DbC. It is the other way around. The DbC contract assertions in a testing environment are always applied no matter how they are instantiated and called. Therefore, using DbC, one writes far less TDD code because the relational rules between object properties and methods are expressed as contract assertions that run any time that class method or property is accessed. Therefore, DbC and TDD are not competitors, but complementary to each other. Treating them as competitors is just ignorance talking. – Protoactinium

TDD does not create automatic test suites. There are tools within testing suites that do this. Eiffel has this as well. However, in 20 years of Eiffel coding I have never found the need to use such a tool because DbC contracts do this job much better because each contract assertion is about the Client-Supplier relationship specifically and they work extremely well. TDD is largely DbC applied externally in testing code instead of bringing it in where it properly lives as pre and post condition code in the production code where it belongs. – Protoactinium

Answer 3

First of all, I am an Eiffel software engineer, so I can speak to the matter from experience.

---

The premise of TDD vs DbC is incorrect

The two technologies are not at odds with each other, but complementary to each other. The complement has to do with the placement of assertions and purpose.

The purpose of TDD has both components and scope. The basic components of TDD are boolean assertions and object feature (e.g. method) execution. The steps are simple:

1. Create an object.
2. Execute some code in a feature.
3. Make assertions about the state of the data on the object.

Assertions that fail, fail the test. Passing all assertions is the goal.

Like TDD, the contracts of Design-by-Contract have purpose, scope, and components. While TDD is limited to unit-test-time, contracts can live through the entire SDLC (Software Development Life-cycle)! Within the scope of TDD, execution of object methods (features), will execute the contracts. In an Eiffel Studio Auto-test (TDD) setup, one creates an object, makes the call (just like TDD in other languages), but here is where likeness ends.

In Eiffel Studio with Auto-test and Eiffel code with contracts, the purpose changes somewhat. We want to test the Client-Supplier relationship. Our TDD code is pretending to be a Client of our Supplier method on its object. We create our objects and call the methods based on this purpose, and not just simplistic "TDD-ish method testing". Because the calls to our methods (features) have contracts, those contracts will execute as a part of our TDD-ish code in Auto-test. Because this is true, contract assertions (tests) that we place in our code do NOT have to appear in our TDD test code. Our job (as a programmer) is to simply ensure: A) The code + contracts are executed along all N-paths, and B) The code + contracts are executed using all reasonable data types and ranges.

There is perhaps more to write about the TDD-DbC complement relationship, but I won't be boorish with you on the matter. Suffice it to say that TDD and DbC are NOT at odds with other—not by a long shot!

The power of the contracts of DbC beyond where TDD can reach

Now, we can turn our attention to the power of the contracts of Design-by-Contract beyond where TDD can reach!

Contracts live in the code. They are not external to it, but internal. The most powerful bit (beyond their client-supplier contract relationship basis) about contracts is that the compiler is designed to know about them! They are NOT a bolt-on addition to Eiffel! Thus, they participate in every aspect of inheritance (both traditional vertical is-a inheritance and in lateral or horizontal Generics). Moreover, they reach to a place that TDD cannot reach—inside the method (feature).

While TDD can mimic pre-conditions and post-conditions with some ease, TDD cannot reach inside the code and perform loop-invariant contracts, nor can it do periodic spot-check "check" contracts along a block of code as it is executing. This is a powerful logical and qualitative paradigm, and a reality about how design-by-contract works.

Moreover, TDD cannot do class invariants but in the faintest of ways. I have tried my hardest to get my Auto-test code (which is really just Eiffel Studios version of applied-TDD) to do class invariant mimicry. It is not possible. To understand why you would have to know the in's-and-out's of how Eiffel class invariants work. So, for the moment, you will simply have to either take my word for it (or not) that TDD is incapable of this task, that DbC handles so easily, well, and elegantly!

The reach of DbC does not end with the above notions

We noted above that TDD lives at unit-test-time. Contracts, because they are applied in code under the supervision and control of the compiler, apply anywhere that the code can be executed:

1. Workbench: you, as a programmer, are using the code to see it work (e.g. before TDD-time or in conjunction with TDD-time).

2. Unit-test: your continuous integration testing, unit-testing, TDD, etc.

3. Alpha-test: your initial test users will trip over contracts as they run the executable

4. Beta-test: a wider audience of users will also trip over contracts.

5. Production: the final executable (or production system) can have continual testing applied through contracts (TDD cannot).

In each of the situations above, one will find that one has control over just which contracts run and from what sources! You can selectively and fine-grainly turn on and off various forms of contracts and control with extreme precision where and when they are applied by the compiler!

And if all of this was not enough, contracts (by design) can do something that no TDD assertion can ever do: tell you where in the call-stack and which client-supplier relationship is broken, and why (which also immediately suggests how to fix it). Why is this true?

TDD assertions are designed to tell you about the results of a code-run (execution) after the fact. TDD assertion can only see as far as the current state of the method under examination. What TDD assertions cannot do from their position on the outside of the code-base is to tell you precisely which call failed and why! You see—your initial TDD call to some method will trigger that method. Many times, that method will call another, and another, and another—sometimes, as the call-stack winds up and down and hither and yon, there is a breakage: Something writes data wrong, does not write it at all, or writes it when it ought not.

TDD is like the police showing up to the crime scene after the murder has already happened. All they have left is forensic clues that they hope will lead them to a suspect and a conviction. But what if we could be there as the crime was taking place? That is the difference between the placement of TDD assertions and contract assertions. Contracts are there to catch the crime in progress and they point directly at the offender as it is committing the offense!

Recap

Let's recap.

• TDD is not at odds with DbC.

• It is a complement and a cooperative set of technologies, but with different functions and purposes, as well as tools to work with them.

• Contract reach further and reveal more about your code when it breaks.

• TDD is one form of catalyst for contracts to be executed.

At the end of the day: I want both! After reading all of this (if you survived), I hope you do as well.

Protoactinium

After many years of thinking and rethinking. I now know its not TDD vs DBC but TDD + DBC – Langan

Answer 4

Design-by-contract and test-driven development are not mutually exclusive.

Bertrand Meyer's book Object Oriented Software Construction, 2nd Edition doesn't say that you never make mistakes. Indeed, if you look at the chapter "When the contract is broken", it discusses what happens when a function fails to accomplish what its contract states.

The simple fact that you use the DbC technique doesn't make your code correct. Design-by-contract establishes well-defined rules for your code and its users, in the form of contracts. It's helpful, but you can always mess things up anyway, only that you'll probably notice earlier.

Test-driven development will check, from the outside world, black box style, that the public interface of your class behaves correctly.

Neolamarckism

Answer 5

I think it is best to use both methods in conjunction rather than just one or the other.

It has always seemed to me that fully enforcing a contract within the class and its methods themselves can be impractical.

For example, if a function says it will hash a string by some method and return the hashed string as output, how does the function enforce that the string was hashed correctly? Hash it again and see if they match? Seems silly. Reverse the hash to see if you get the original? Not possible. Rather, you need a set of test cases to ensure that the function behaves correctly.

On the other hand, if your particular implementation requires that your input data be of a certain size, then establishing a contract and enforcing it in your code seems like the best approach.

Biometrics

You can have a naive, but slow implementation of your hashing function and ensure that your optimal implementation returns the same answer. You can then turn off this particular contract in the production. – Natoshanatron

Answer 6

In my mind TDD is more "inductive". You start with examples (test cases) and your code embodies the general solution to those examples.

DBC seems more "deductive", after gathering requirements you determine object behavior and contracts. You then code the specific implementation of those contracts.

Writing contracts is somewhat difficult, more so than tests that are concrete examples of behavior, this may be part of the reason TDD is more popular than DBC.

Tweeny

Answer 7

I see Design By Contract as a specification for success/failure in ALL cases, whereas Test Driven Development targets ONE specific case. If the TDD case succeeds, then it is assumed a function is doing it's job, but it doesn't take into account other cases that could cause it to fail.

Design By Contract on the other hand doesn't necessary guarantee the desired answer, only that the answer is "correct." For example, if a function returns is supposed to return a non-null string, the only thing you can assume in the ENSURE is that it will not be null.

But maybe it doesn't not return the string that was expected. There is no way for a contract to be able to determine that, only a Test can show that it was performing according to the specification.

So the two are complementary.

Greg

Idioblast

"If all you have is a hammer to work with as a tool, then everything begins to look like a nail". I've been trying to read up on TDD but the more I read the more it sounds like hysteria; it's refreshing to read something a little more level-headed. TDD is a tool, nothing more. – Hagridden

Answer 8

I see no reason why both cannot co-exist. It is wonderful to look at a method and know at a glance exactly what the contract is. It is also wonderful to know that I can run my unit tests and know that nothing was broken with my last change. The two techniques are not mutually exclusive. Why design by contract is not more popular is a mystery.

Waggery

Answer 9

I've used both in the past and found DBC-style less "intrusive". The driver for DBC may be regular application running. For Unit Tests you have to take care of setup because you expect (validate) some responses. For DBC you don't have to. Rules are written in data-independent manner, so no need to setup and mocking around.

More on my experiences with DBC/Python: http://blog.aplikacja.info/2012/04/classic-testing-vs-design-by-contract/

Wanting