How bad is SLOC (source lines of code) as a metric? [closed]

Question

We are documenting our software development process. For technical people, this is pretty easy: iterative development with internal milestones every four weeks, external every 3 months.

However, the purpose of this exercise is to expose things for our project management in terms that they can understand. Specifically, these non-technical managers need metrics that they can understand.

I understand our options for metrics well and have proposed a whole set (requirements met and actual costs vs. budgeted costs are two of my favorites). However, we do have some old hands involved and they tend to hang onto metrics like SLOC.

I understand the temptation of SLOC: it seems easy for non-software people to understand and it seems like the closest analog of a physical thing (it's just like counting punched cards back in the old days!).

So here's the question: how can I explain the dangers of SLOC to a non-technical person?

Here's some concrete motivation: we work on a fairly mature deployed system that has years of history behind it. As we add features, SLOC tends to stay approximately level or even decrease (refactoring removes old / dead code, new features are really just adjustments of existing, etc). To a non-programmer manager, a non-increasing SLOC in a development project is perplexing at best....

Clarifying in response to a recent answer below: remember, I'm arguing that SLOC is a bad metric for the purposes of measuring project progress. I'm not arguing that it is a number that's not worth collecting. It requires extensive context to do anything useful with it and most program managers don't have that context.

Washerman

if you want to put in a nail somewhere, do you count the hammer strokes? – Otic

"Can you tell how good a book is by the number of pages?" – Nitty

SLOC very good metric!! ...for how bad stuff is. The more SLOC you have, the worse position you are in. – Benzyl

Just tell them that "1 line of code" is worst "2 lines\n of code". – Australopithecus

I know this is an old question but I think it is still highly relevant! The more abstraction we get with frameworks, the more pointless SLOC becomes for measuring productivity. – Luann

Answer 1

Someone said :

"Using SLOC to measure software progress is like using kg for measuring progress on aircraft manufacturing"

It is totally inappropriate as it encourages bad practices like :

• Copy-Paste-Syndrome

• discourage refactoring to make things easier

• Stuffing with meaningless comments

• ...

The only use is that it can help you to estimate how much paper to put in the printer when you do a printout of the complete source tree.

Hasty

I had heard that comment before but don't know the citation. Do you have a reference for it? – Washerman

@Bob According to an article I found online it was Bill Gates : dev102.com/2008/09/09/… – Hasty

@Bob Cross: I think it is usually attributed to Bill Gates. (Which is kind of ironic, considering what bloated monsters Windows and Office are. Windows is around 50 million SLOC, Office around 270 million. That's over 300 million SLOC, using all the tools 2010 has to offer, to achieve pretty much the same thing Alan Kay and his group did in 1976 with 60 thousand SLOC.) – Henson

And like Bill's comment about "Who'd need more than 640K?" he was totally off the mark. If you talk to people that design airplanes, they fight to remove every last ounce of weight. This is the entire excuse for the Boeing Dreamliner. People should do that for code, too. (Its good to be an icon, people will quote you no matter how dumb your remark). – Swoon

As a metric by itself it's bad. As a value to derive other metrics it can be useful, such as predicting defect density in your software. – Symbiosis

Bill Gates never said "640K ought to be enough for anybody". wired.com/politics/law/news/1997/01/1484 – Vitalis

Answer 2

The issue with SLOC is that it's an easy metric to game. Being productive does not equate to producing more code. So the way I've explained it to people baring what Skilldrick said is this:

1. The more lines of code there are the more complicated something gets.
2. The more complicated something gets, the harder it is to understand it.
3. Before I add a new feature or fix a bug I need to understand it.
4. Understanding takes time.
5. Time costs money.

Smaller code -> easier to understand -> cheaper to add new features

Bean counters can understand that.

Quadrate

okay, that 5 step process is actually quite compelling. Thanks. – Washerman

This is a great argument for SLOC as a metric. Once a project has matured, we should be trying to plateau the SLOC count, and not allow it to increase wildly. – Nitty

Answer 3

Show them the difference between:

for(int i = 0; i < 10; i++) {
    print i;
}

and

print 0;
print 1;
print 2;
...
print 9

And ask them whether 10 SLOC or 3 SLOC is better.

---

In response to the comments:

1. It doesn't take long to explain how a for loop works.

2. After you show them this, say "we now need to print numbers up to 100 - here's how you make that change." and show how much longer it takes to change the non-DRY code.

Zipah

I would agree with you that the first case is better, but the 'non-technical' person might agree that the second case is. And, they might be right, the second case will compile/run faster and require less memory =) – Phocaea

I agree that you and I both look at this example (or similar on Wikipedia) and say, yes, of course. However, non-programmers see "gibberish" and "more gibberish". It's sort of like using a word in its own definition.... – Washerman

@mkoistinen: The compiler can unroll the loop for you if it's actually a good idea. And bloated binaries with hand-unrolled loops like this often perform worse even though intuitively it seems like the computer should be doing less "work". – Romeo

@Phocaea Second example is better?? Are you sure, what if you do a million, ten million are you going to spend all day typing out code that takes someone else 10 seconds to type out?? Are you going to spend 10 million more $$ on someone for the 10 million version which does the same thing as the three line version, not to mention the 10 million line version cannot be scaled while the three line version can.... – Arianism

@YunfeiChen yea, you're probably right... 10 years ago, I was wrong. – Phocaea

@Phocaea I do not get the 10 years part, unless it is a joke about how it took you 10 years to write out those print statements :)..... – Arianism

@YunfeiChen you were responding to a comment I posted 10 years ago. – Phocaea

Answer 4

Uptodate

sure, it's easy to see Dilbert in everything if you try to. That said, we can't just say "all metrics are dumb." We have to suggest something better. That said, SLOC definitely IS dumb. – Washerman

You could edit the strip to remove panels 3, 4 and 5. Then the message would become less "metrics are dumb" and more "ill-considered metrics are dumb", and the analogy between SLOC and PHB's "number-of-words" metric would remain clear. – Uptodate

We'll have to subtract panels you delete. – Sclerometer

Is it bad that my gut instinct is to agree that everything is measurable if you try hard enough? SLOC is an absolutely terrible metric, but that doesn't mean that there aren't relevant ones. – Neurotic

@JoshLee We will have to substract bugs you delete... I think that should be more programming related... – Arianism

I don't think the comic suggests "all metrics are dumb", but rather that it's dumb to try to measure certain things in certain ways. Pretty sure Alice understands that the cartesian plane under a Euclidean metric is super useful. – Its

Answer 5

I disagree on SLOC being a bad metric. It may be moot to go into a years-old question with eleven answers, but I'll still add another.

Most arguments call it a bad metric because it is not suited to directly measure productivity. That is a strange argument; it assumes the metric to be used in an insane way. With this reasoning, one could call the Kelvin a bad unit because it is unsuited to measure distance.

Code length is a viable measure of ballast.

The amount of non-comment code lines correlates with:

• undetected errors
• maintenance costs
• training time for new contributors
• migration costs
• new feature costs

and many more similar kinds of costs, like the cost of optimization.

Of course SLOC count isn't a precise measure of any of these. Code can be anywhere between very nice and very ugly to manage. But it can be assumed that code length is rarely free, and thus, longer code is often harder to manage.

If I were managing a team of programmers, I would very much want to keep track of the ballast it creates or removes.

Interlocutor

If you read the last paragraph of my question (and several of the answers), you'll see agreement with your point. Measuring SLOC is a good idea (and can be handled automatically): e.g., when measuring test coverage. What I continue to argue (and so do most of the answers) is that "new SLOC written" is not a good metric for project progress. If you can implement more requirements near zero SLOC increase, your value per line of code increases. – Washerman

@BobCross I guess we all agree in general. It just might be better to focus on what SLOC does mean instead of what it does not mean. That is more constructive; the managers get their metric, which is indeed useful, and learn how to interpret it. The question whether it directly measures productivity barely needs extra mentioning. – Interlocutor

Thanks for your suggestion. If you reread the original question, I suggest two metrics that communicate progress and value to the project management better than SLOC. The original question was focused on the problem of lazy metrics: the equivalent of counting punch cards and calling that progress. – Washerman

@Interlocutor More constructive, writing redundant repeated code and introducing harder to debug code is more constructive??? – Arianism

Answer 6

Pretty bad (-:

A much better idea would to cover the test cases, rather than code.

The idea is this: a developer should commit a test case that fails, then commit the fix in next build, and the test case should pass ... just measure how many test cases the developer added.

As a bonus collect coverage stats (branch coverage is better than line coverage here).

Rochelrochell

I totally agree that test cases and coverage are better. That said, I actively refuse to allow measurement of individual people for project management people. We're a team and that sort of thing stays inside the team. – Washerman

Answer 7

Explain that SLOC is an excellent measurement of the lines of code in the application, nothing else.

The number of lines in a book, or the length of a film doesn't determine how good it is. You can improve a film and shorten it, you can improve an application and reduce the lines of code.

Burnout

I agree that you and I would understand that length != better. However, for non-technical people that are used to projects that need to produce very long cables (for example), length is a measure that they understand. The breakdown is that software == knowledge. It doesn't have a physical analog. – Washerman

Answer 8

I believe SLOC is a great metric. It tells you how large your system is. That is good for judging complexity and resources. And it helps you prepare the next developer for working on a codebase.

But SLOC count should be analyzed only AFTER other appropriate code quality metrics have been applied. So...

• Do NOT write 2 lines of code when 1 will do, unless the 2-line version makes the code 2 times easier to maintain.
• Do NOT fluff code with unnecessary comments just to fluff SLOC count.
• Do NOT pay people by SLOC count.

I have been managing software projects for 30 years. I use SLOC count all the time, to help understand mature systems. I have never found it useful to even glance at SLOC count until a project is near version 1.0 release.

Basically, during the development process, I worry about quality, performance, usability, and conformance to specifications. Get those right, and the project will probably be a success. When the dust settles, look at SLOC count. You might be surprised that you got SO much out of 5,000 lines of code. And you might be surprised that you got SO little! (But SLOC count does not affect quality, performance, usability, and conformance to specification.)

And always code like the person who will be working on your code next is a violent psychopath who knows where you live.

Cheers, Uncle Chip

Bricole

I disagree with your first point there, Do NOT write 2 lines of code when 1 will do, unless the 2-line version makes the code 2 times easier to maintain. – Arianism

By that logic Selection sort is the best algorithm for sorting because it is the shortest, QuickSort, HeapSort and MergeSort are unnecessary.... – Arianism

Quicksort is much more than 2 times faster than Selection sort for large n. n^2 versus nlogn. – Its

Answer 9

You don't judge how good(how many features,how it performs..) a plane is based on its weight(sloc).

When you want your plane to fly higher, longer and perform better, you don't add weight to it. You replace parts of it with lighter/better materials. You strip off parts you don't need as to not add unnecessary weight.

Scot

“Measuring programming progress by lines of code is like measuring aircraft building progress by weight.” — Bill Gates – Sclerometer

Answer 10

Why SLOC is bad as an individual metric of productivity

Think of code as a block of clay/stone. You need to carve, say 10 statues. It's not how many statues you carve that counts. It's how well you've carved it that counts. Similarly it's not how many lines you've written but how well they are functioning. In case of code LOC can backfire as a metric this way.

Productivity also changes when writing a complex piece of code. It takes a second to write a print statement but a lot of time to write a complex piece of logic. Not all fingers are equal.

How SLOC can be used to your benefit

I think SLOC for defect % is a good metric. Yes the difficulty level comes into play but this is a good parameter that the managers can throw around while doing business. Try to think from their perspective too. They don't hate you or your work, but they need to tell customers that you're the best and for that they need something tangible. Give them what you can :)

Caen

in a modern language, I feel that SLOC is too fine-grained. Example: Findbugs reports bugs per Java class and package. THAT is more useful - we can look at a list of classes and say that "this is the worst class / package" and we should organize a focused effort. Bugs per line of code is hard to take action on - what should we do? – Washerman

@Bob If you have better metrics then there's no point in sticking to SLOC. You can explain this to the managers. Why they are still struck with it is because the industry still holds on to it. Tell them that from QA point of view it's better to have findbug type of reports. Try and understand "their" focus, if it's internal monitoring they'll agree on findbugs, if it's external then they'll still need SLOC because that's what the client understands. – Caen

@Sidarth, in case it isn't clear, some of the people involved ARE the clients. They aren't savvy enough to understand a Findbugs (especially since they're looking for a "size" metric in this category) - they aren't technical in our area. – Washerman

Even I do not know what FindBugs is , And I am a C++ programmer in software Developing, Is it a new tool for finding bugs in the code or an IDE or something?? – Arianism

Answer 11

even modern code metrics tools criticize SLOC conting, i like the point made in the ProjectCodeMeter FAQ:

What's wrong with counting Lines Of Code (SLOC / LLOC)?

Indiscerptible

Answer 12

SLOC can be changed dramatically by putting extra empty lines ("for readability") or by putting or removal of comments. So relying on SLOC only can lead to confusion.

Railroad

Whitespace (extra lines) and/or comments are not counted as part of SLOC. – Doomsday

this is not something non-technical staff needs to know. – Haldeman

Answer 13

Why don't they understand that the SLOC hasn't changed, but the software does more than it did yesterday because you've added new features, or fix bugs?

Now explain it to them like this. Measuring how much work was done in your code by comparing the lines of code is the same as measuring how many features are in your cell phone comparing it by size. Cell phones have decreased in size over 20 years time while adding more features because of technological improvements and techniques. Good code follows this same principal as we can express the same logic in fewer and fewer lines of code, making it faster to run, easier to maintain, and simpler to understand as we improve our understanding of the problem and introduce new techniques for development.

I would get them to focus on the business value returned through feature development, maintenance, and bug fixes. If whoever is happy with the software says they can see improvement don't sweat the SLOC.

Go read this:

https://stackoverflow.com/questions/3800707/what-is-negative-code

Hydrofoil