How do I create and maintain a code reuse library?

Question

I am trying to setup a repository of reusable code. I was thinking about having each reusable code module have a certain “Maturity Level” rating. The rating would be defined as the level at which a reusable code lies within a certain set of requirements. The highest maturity level will be the highest degree of standard across a predefined set of requirements.

For example:
Level; Requirements; Description
Level 0; Code is legal to use; Is the code legal to use in commercial industry/across multiple contracts/etc?
Level 1; Base codeline and meets level 0 requirements; Prototyped code, 3rd party tools, etc
Level 2; Has Function Interface and comments and meets level 1 requirements; Sufficient documentation for each class and function; Able to determine functionality from comments
Level 3; Adheres to coding standards and meets level 2 requirements; Follows defined coding standards and passes code checking utility test
Level 4; Includes test cases and meets level 3 requirements; Has sufficient test cases to test all functionality of code
Level 5; Approved by reuse committee and meets level 4 requirements; Reviewed by reuse experts and peers and verified it meets all levels of maturity

I’m wondering if this maturity level should be a hierarchical structure, where in order to move to the next level you need to meet the requirements of all previous levels (as I have shown above)?

Or if it should be a subset of requirements to meet the next level?

For example, we have meet x out of y requirements, we can move to the next level (requirements would be the same as mentioned above).

Level 0, meets 0 out of 6 requirements
Level 1, meets 1 out of 6 requirements
…

The problem I see with the subset approach is some requirements should have a stronger weighting, and in this approach that will not be taken into account (unless I start getting specific like, meets a out of b and x out of y, etc). But then it could start to get complicated.

Has anyone done this before, and if so, how did you setup your library? Do you have a maturity level at all or some other structure? Any input would be greatly appreciated.

Bowing

I think when you have to worry about maintenance of a code reuse library, you're on the wrong track... :p – Timothytimour

@jalf - Then you obviously dont have a reuse library :) All reusable code you've ever written has never had bugs or needed features added? – Bowing

@jalf That's why he's putting effort and attention into the design and structure now, to save him work later... – Maeganmaelstrom

my experience is that code repositories never works. if code is simple - you can find it in internet or write yourself with a few efforts. if it is complex - it is unlikely to be reused in other projects – Combust

Answer 1

Setting up a code reuse repository is a difficult task. The main difficulty is not in how you set it up but in how you communicate the existence of the various libraries in the repository. Reuse libraries only good if they are used, and they are only used if they are known, and they are only used widely if the quality of the code is high and if it meets the needs of the users.

I like the idea of maturity levels, but as others have posted, there is probably quite a bit of setup/build work to do. I have thought of a similar approach to builds of an application - I called them confidence levels. In the application-build arena, a low confidence build is one that did not pass unit tests; a medium confidence might include passing unit tests, but not integration tests, and so on. This was a good mechanism for communicating to QA and users what to expect. A similar mechanism might be appropriate for libraries.

Documentation comments are a must, and must also have as much care put into them as you put into the code. The comments should communicate what, why, where, when, how, which, etc. Your build process should publish the documentation to a well-known location (again, communication is key).

Along the lines of communication, it doesn't hurt to present from time-to-time just what is there. Again! communication.

So, at a minimum your build of each library should:

• publish the library (maybe notify subscribers)
• publish the documentation
• run unit tests
• publish the maturity level

As to maturity levels, I would define them with a "level name" and a description of what the level means. Publish the criteria for what it means to move up or down a level. Actually, now that I think about it, perhaps you want a set of orthogonal criteria: a level for the code, a level for the documentation, use-policies (i.e. must have a license for XYZ), and other attributes. I do recommend you approach this in small increments though. At the end of the day, delivering functionality to end-users is what matters.

You have to also communicate a mindset of naturally pushing reusable bits into the repository. Developers have to have incentive to do this usually. Static code checking tools that look for duplication and peer reviews only go so far. Someone has to actually do the work of moving code to the repository.

Finally, I recommend that you use as much tool support as possible in the setup, build, maintenance, and communication of the repository. Otherwise, like any non-code artifact, you will face a certain amount of entropy which lowers the value as the non-code artifact becomes dated.

Tropicalize

+1 for "how you communicate the existence of the various libraries" and the Otherwise... An even worse outcome is the prevalence of "cut n paste" culture, it doesn't do quite what you want, the library release process is impenetrable, so we... It's scary how much of this goes on. – Epicedium

Right. Cut-and-paste is really only justified once, and then if generalization is not immediately apparent and you're crammed for time. That third instance of the code should be a red-flag to generalize. Agreed - cut-and-paste is too prevalent and can be detrimental: multiple points of maintenance. – Tropicalize

Answer 2

I think you will find it difficult to ensure that the entire development team follows these guidelines accurately enough. Especially when the guidelines may be interpreted one way or another. Moreover, it will be a major pain if somebody improves a piece of code by adding tests and suddenly it has to move to a different project. More likely than not, such code will stay in the project it was originally placed into, and over time the maturity levels will become meaningless.

One approach I saw working fine in a large company is this:

• All third party libraries are committed to a special directory and always include a version number.
• Our own common libraries are divided based on the references they have to other things. E.g. if the utility code references the Infragistics library then this bit of utility code goes into an InfragisticsUtils library.
• Our own common libraries that form clearly identifiable "units" go into separate libraries. For example, a library of code that deals with pricing securities is a separate project.
• All reusable code that doesn't satisfy any of the above goes into a catch-all Utilities project.
• Our own libraries are compiled and released to a shared location where projects can reference them. It is up to the projects' development team to decide whether they want to reference a compiled binary or just include the utility project into their solution.

Obviously the quality of the code you find in the catch-all Utilities library can vary significantly. To alleviate this we simply ensured that two people from different development teams reviewed all checkins to Utilities. This weeds out a lot of stuff that has no place there!

Alow

Answer 3

I think a great code repository would include a CM tool and a Wiki tool. The CM tool should be structured using the level idea (as you proposed), since it structures the code by quality. The wiki should act as an "advertisement" of what the software can do and how it can help you out. This wiki could also keep information like, which projects are using the code, rating of how usable it is, and examples of how to use it. If anyone is worried about the development team following the level guidelines, it should be pointed out how self policing works and give the example of how well it works with wikis.

Now, the structuring of the CM tool is important. It is designed to convey the quality of the code, so you know what you get into when you use it. For example, if you write a simple class with barely any comments and it doesn't really uphold to coding standards (maybe a prototype) then it would be entered into \sw_repository\level0\ExamplePrototype.

Maybe someone then takes that piece of code and added comments and brings it up to standards. Then that person would place it in \sw_repository\level1\ExamplePrototype.

Then that same person, a while later, creates unit tests for the ExamplePrototype. This would then go to level2 and so on.

Defining the levels should take some thought. They really should be somewhat sequential and even if for example, you had written unit tests for the prototype code but it didn't satisfy the comments and coding standard then it is placed in level0 anyway. However, it would be easy to go to level2 if those comments and standards were satisfied.

Salivate

Answer 4

For my library, I just put in code that I wrote that can be used across multiple applications. If code is specific to a particular app then it doesn't go into the library. As more apps use it, the bugs get worked out so I never expect it to be bug free right away. Bugs will be constantly found and fixed as your library matures and is stressed with different apps. It will never be bug free but over time will approach reliability. Also when I realize that API for some stuff is wrong, I don't worry about it and refactor the API as soon as possible.

Here is my library in c++
http://code.google.com/p/kgui/

Stodgy

So you don't have any standard as to what code goes into your library other than the basic requirement of being across applications? – Bowing

In response to SwDevMan81, my standard is to put code that can be reused in other apps into my library. Just because it is in the library doesn't necessarily make the apps bigger as the linker is usually smart enough to only include the bits of the library that are actually called/referenced in each app. – Stodgy

Answer 5

For years Microsoft has been a big advocate for what is known as software factories, a big part of which is devoted to solving the problems of reuse.

What are the problems of reuse? First of all, it's hard. It's very hard to come up with a library and API that will serve beyond the immediate needs of the project at hand. How do you predict the future? Also, the problem of a centralized repository that serves as both a knowledge base and a vibrant community of practice is very challenging. How do you make something that is both very flexible yet easy to use? Many have tried and failed. Both software factories and software product lines attempt to address these problems.

Good luck!

Waltraudwaltz

Answer 6

Kit mentioned the most important problem: the reuse. the rest of the idea is nice, but it's not more than a detail.

i mean, i myself have trouble maintaining my very own reuse library. sometimes i do an implementation that is very project-specific, or i do the n-th prototype for some idea, and none of those ever gets into my library.

if you really succeed in having a code reuse library, that is used and maintained by many developers, in a disciplined way, than that's victory. you need a version control system and a bug tracker, the latter being used by both project owners and users. you need communication and means of contribution. having a handful of devs using a project dramatically improves its quality. implementation gets better. documentation is created. API and feature design are on a much higher level. a committee is a nice thing, but it cannot decide, how good given code is, without actually using it. it can decide whether the code meets specific standards, but meeting standards is not sufficient for good libraries.

you need to do your best to make sure, you don't have tons of code snippets floating around, all of which can more or less do something. ok, any design decision has pros and cons, but i think, it makes more sense to start with ONE project for a given task, and branch it, if really necessary, but keep alive communication between project teams, and consider (partially) merging back.

don't worry too much about categorizing the quality of different projects. if a project is bad, then users/devs will push it to a better level. most people are clever enough to see, when a library is good, and when it isn't. you really need to put your energy in these symbiotic effects, rather than trying to burden participants with strict rules.

just my 2 cents ... ;)

Caelian

Yes, no strict rules. If there is fear of possibly messing something up in the library, people won't contribute, and it's value goes down. With enough good tools like refactoring, unit testing, source control, and coverage - fear should (I hope!) be reduced. – Tropicalize

Answer 7

Look at Will Tracz's "Confessions of a Used Program Salesman", and stuff by HP's reuse Rabbi, Martin Griss.

Salutary

Answer 8

I think you are thinking too much in a non problem.

How did you setup my library? Easy, if you use the same (or almost the same) method in two or more projects, move it to the library.

Faulk

I think it involves a little more than "move it to a library". What about 3rd party software, subcontracted software, etc. In a large company you might have a lot of projects where reusable modules could be used. How do we manage them? – Bowing

Answer 9

It's considered good approach to have one's own library, but a thousands lines one is a ruin !

Excursus