Software Development Costs Pyramid [closed]
Asked Answered
H

5

11

A friend was telling me the other day that there is a pyramid for the costs of fixing a problem in the software development life cycle. Where could I find this?

He was referring to the cost of fixing a problem.

For example,

To fix a problem at the requirements stage costs 1.

To fix a problem at the development stage costs 10.

To fix a problem at the testing stage costs 100

To fix a problem at the production stage costs 1000.

(These numbers are just examples)

I would be interested in seeing more about this if anyone has references.

Headroom answered 9/11, 2010 at 2:47 Comment(3)
I'm voting to close this question as off-topic because its about deployment costSemifinalist
I'm voting to close this question as off-topic because it is about managing a project, not programming.Gilpin
I originally asked this question but it's closed now so I can't add an answer. I came across the following chart in "Best Kept Secrets of Peer Code Review" page 123 from a study at IBM. Phase - Cost to fix Requirements - $0 Design - $45 Development - $275 QA/Test - $33,000 Customer - $141,000Headroom
I
19

The Incredible Rate of Diminishing Returns of Fixing Software Bugs

Stefan Priebsh: OOP and Design Patterns: Codeworks DC in September 2009

(Stefan Priebsh: OOP and Design Patterns: Codeworks DC in September 2009)

Irrelevant answered 9/11, 2010 at 2:54 Comment(0)
S
13

This is a well-known result in empirical software engineering that has been replicated and verified over and over again in countless studies. Which is very rare in software engineering, unfortunately: most software engineering "results" are basically hearsay, anecdotes, guesses, opinions, wishful thinking or just plain lies. In fact, most software engineering probably doesn't deserve the "engineering" brand.

Unfortunately, despite being one of the most solid, most scientifically and statistically sound, most heavily researched, most widely verified, most often replicated results of software engineering, it is also wrong.

The problem is that all of those studies do not control their variables properly. If you want to measure the effect of a variable, you have to be very careful to change only that one variable and that the other variables don't change at all. Not "change a few variables", not "minimize changes to other variables". "Only one" and the others "not at all".

Or, in the brilliant Zed Shaw's words: "If you want to measure something, then don't measure other shit".

In this particular case, they did not just measure in which phase (requirements, analysis, architecture, design, implementation, testing, maintenance) the bug was found, they also measured how long it stayed in the system. And it turns out that the phase is pretty much irrelevant, all that matters is the time. It's important that bugs be found fast, not in which phase.

This has some interesting ramifications: if it is important to find bugs fast, then why wait so long with the phase that is most likely to find bugs: testing? Why not put the testing at the beginning?

The problem with the "traditional" interpretation is that it leads to inefficient decisions. Because you assume you need to find all bugs during the requirements phase, you drag out the requirements phase unnecessarily long: you can't run requirements (or architectures, or designs), so finding a bug in something that you cannot even execute is freaking hard! Basically, while fixing bugs in the requirements phase is cheap, finding them is expensive.

If, however, you realize that it's not about finding the bugs in the earliest possible phase, but rather about finding the bugs at the earliest possible time, then you can make adjustments to your process, so that you move the phase in which finding bugs is cheapest (testing) to the point in time where fixing them is cheapest (the very beginning).


Note: I am well aware of the irony of ending a rant about not properly applying statistics with a completely unsubstantiated claim. Unfortunately, I lost the link where I read this. Glenn Vanderburg also mentioned this in his "Real Software Engineering" talk at the Lone Star Ruby Conference 2010, but AFAICR, he didn't cite any sources, either.

If anybody knows any sources, please let me know or edit my answer, or even just steal my answer. (If you can find a source, you deserve all the rep!)

Salas answered 9/11, 2010 at 4:37 Comment(1)
I work at a company that basically spends no time in the requirements phase (and the closest approximation to our development process is waterfall...). Ignoring this leads to so many changes that the software we initially started writing has no similarity to the end product... and it costs a ton more. On top of that, since we don't spend time in requirements we end up making a lot of changes to production after the fact that are not easy to implement because of the way the software was structured around the requirements...Noriega
P
3

See pages 42 and 43 of this presentation (pdf).

Unfortunately the situation is as Jörg depicts, in fact somewhat worse: most of the references cited in this document strike me as bogus, in the sense that the paper cited either is not original research, or does not contain words supporting the claim being made, or - in the case of the 1998 paper about Hughes (p54) - contains measurements that in fact contradict what is implied by the curve in p42 of the presentation: different shape of the curve, and a modest x5 to x10 factor of cost-to-fix between the requirements phase and the functional test phase (and actually decreasing in system test and maintenance).

Precession answered 3/1, 2012 at 7:37 Comment(4)
The linked pdf is no longer available. Do you have a new link for it?Unfix
Try this seaspin.org/files/folders/205/download.aspx or this seaspin.org/files/folders/22/download.aspxPrecession
Also I recommend my book: leanpub.com/leprechauns with much more on the topicPrecession
I just read your book and it made me think of this question I asked 13 years ago. Then I came here to find you had replied. Thanks for your book!Headroom
C
0

Never heard of it being called a pyramid before, and that seems a bit upside-down to me! Still, the central thesis is widely considered to be correct. just thick about it, the costs of fixing a bug in alpha stage are often trivial. By beta stage it might take a bit more debugging and user reports. After shipping it could be very expensive. a whole new version has to be created, you have to worry about breaking in-production code and data, there may also be lost sales due to the bug?

Creativity answered 9/11, 2010 at 2:53 Comment(0)
P
-1

Try this article. It uses the "cost pyramid" argument (no naming it), among others.

Palaver answered 18/4, 2013 at 15:3 Comment(1)
Please include an actual answer, instead of posting sole link. Thanks.Woodie

© 2022 - 2024 — McMap. All rights reserved.