The best approach to modular programming in Delphi

Question

this is a continuation of the discussion I started here. I would like to find the best way to modularize Delphi source code as I'm not experienced on this field. I will be gratefull for all your suggestions.

Let me post what I have already written there.

The software developed by the company I work for consists of more than 100 modules (most of them being something like drivers for different devices). Most of them share the same code - in most cases classes. The problem is that those classes are not always put into separate, standalone PAS units. I mean that the shared code is often put into units containing code specific to a module. This means that when you fix a bug in a shared class, it is not enough to copy the PAS unit it is defined in into all software modules and recompile them. Unfortunately, you have to copy and paste the fixed pieces of code into each module, one by one, into a proper unit and class. This takes a lot of time and this is what I would like to eliminate in the nearest future by choosing a correct approach - please help me.

I thought that using BPLs distributed with EXEs would be a good solution, but it has some downsides, as some mentioned during the previous discussion. The worst problem is that if each EXE needs several BPLs, our technical support people will have to know which EXE needs which BPLs and then provide end users with proper files. As long as we don't have a software updater, this will be a great deal for both our technicians and end users. They will certainly get lost and angry :-/.

Also compatibility issues may occur - if one BPL is shared by many EXEs, a modification of that BPL can bee good for one EXE and bad for some other ones.

What should I do then to make bug fixes quicker in so many projects? I think of one of the following approaches. If you have better ideas, please let me know.

• Put shared code into separate and standalone PAS units, so when there is a bug fix in one of them, it is enough to copy it to all projects (overwrite the old files) and recompile all of them. This means that each unit is copied as many times as many projects it is used by.

This solution seems to be OK as far as a rarely modified code is concerned. But we also have pas units with general use functions and procedures, which often undergo modifications. It would be impossible to do the same procedure (of copying and recompiling so many projects) every time someone adds a new function to this file.

• Create BPLs for all the shared code, but link them into EXEs, so that EXEs are standalone.

For me it seems the best solution now, but there are some cons. If I make a bug fix in a BPL, each programmer will have to update the BPL on their computer. What if they forget to do that? However, I think it is a minor problem. If we take care of informing each other about changes, everything should be fine. What do you think?

• And the last idea, suggested by CodeInChaos (I don't know if I understood it properly). Sharing PAS files between projects. It probably means that we would have to store shared code in a separate folder and make all projects search for that code there, right? And whenever it is necessary to modify a project, it would have to be downloaded from SVN together with the shared files folder, I guess. Each change in the shared code would have to cause recompilation of each project that uses that code.

Please help me choose a good solution. I just don't want the company to lose much more time and money than necessary on bugfixes, just because of a stupid approach to software development. So far nobody has cared about it and you can imagine how many problems it causes.

Thank you very much.

Bootblack

Don't have two copies of the same function. Ever. SVN makes it hard to achieve what you are looking to do but I think you need SVN externals. I vote to move to Programmers where you should get a better response. – Bronchus

@David Microsoft does it, in the source code for their Office products. (Scott Berkun, "The Art of Project Management") Why? If a bug is introduced in one copy, the other products will not be affected. – Warman

@Warman I cannot believe that. I think you are re-telling the story slightly twisted. – Bronchus

@Warman and if a bug is fixed in one, they don't fix it in the others? – Robot

Answer 1

You say:

• Create BPLs for all the shared code, but link them into EXEs, so that EXEs are standalone.

You can't link BPLs into an executable. You are simply linking in the separate units that are also in the BPL. That way you don't actually use or even need the BPL at all.

BPLs are meant to be used as shared code, i.e. you put the code that is shared into one or several BPLs and use that from each of the .exes, .dlls or other .bpls. Bugfixes (if they don't change the public interface of the BPL) merely require the redistribution of that one fixed BPL.

As I said, decide on the public interface of a DLL and then don't change it. You can add routines, types and classes, but you should not modify the public interfaces of any existing classes, types, interfaces, constants, global variables, etc. that are already in use. That way, a fixed version of the BPL can easily be distributed.

But note that BPLs are highly compiler version dependent. If you use a new version of the compiler, you will have to recompile the BPL too. That is why it makes sense to give BPLs suffixes like 100, 110, etc., depending on the compiler version. An executable compiled with compiler version 15.0 will then be told to use the BPL with suffix 150, and an executable compiled with version 14.0 will use the BPL with suffix 140. That way, different versions of the BPLs can peacefully co-exist. The suffix can be set in the project options.

How do you manage different versions? Make a directory with a structure like I have for my ComponentInstaller BPL (this is the expert you can see in the Delphi/C++Builder/RAD Studio XE IDE under menu Components -> Install Component):

Projects
  ComponentInstaller
    Common
    D2007
    D2009
    D2010
    DXE

The Common directory contains the .pas files and resources (bitmaps, etc.) shared by each version, and each of the Dxxxx directories contains the .dpk, .dproj, etc. for that particular version of the BPL. Each of the packages uses the files in the Common directory. This can of course be done for several BPLs at once.

A versioning system might make this a lot easier, BTW. Just be sure to give each version of the BPL a different suffix.

If you actually want standalone executables, you don't use BPLs and simply link in the separate units. The option "compile with BPLs" governs this.

Melonymelos

Thank you for sharing your experience. Let me ask you one more question. Your example is based on several versions of the same BPL. Each BPL version uses the same directory with .pas files. But let me create another BPL with a completely different code that references some units from your common directory. And now, should it refer directly to those files (attached to the project) or should it just use one of those common BPLs (by referencing it in project's requirements list)? Which method do you consider better? – Bootblack

Units should only be in one and one only BPL at the same time. Other BPLs that need such a unit should require the BPL that contains it. It makes sense to think a little longer about which unit should go where. I would draw a graph which executable and which unit needs what, rearrange it until you are satisfied and then decide. – Melonymelos

OK, this is what I wanted to hear. Thank you. – Bootblack

Let me conclude, to finalize the discussion. Shared code is put into BPLs. Pas files are never copied - only BPL dependencies are accepted in this approach. Each programmer installs all shared BPLs on their machine and updates them (together with .pas files belonging to them) whenever somebody modifies any of those BPLs. It seems to be easy as long as SVN is used. Is that all correct? – Bootblack

If one of the BPLs is bugfixed, but the interface (the functions, procedures, classes, types, consts, etc. it exposes) is not modified, only that BPL needs to be replaced. BPLs or executables depending on it do not have to be replaced. You can add to the interface of a BPL, but not remove anything or modify it. – Melonymelos

Answer 2

From my point of view trying to manage artifacts like Delphi units, libraries and executable files, you search at wrong place. I suggest you to turn around and start with refactoring of code, based on Design patterns implementation.

E.g. all common functions can be placed into one Singleton class, instances of common classes can be constructed with Abstract Factory, classes can interact through native Delphi implementation of interfaces instead of direct usage and so on. Even you can choose to implement Facade for all common parts of projects.

Of course, concrete choice of patterns and details of implementation depends on project specific and only you can decide what applicable in your case. I suppose, that after looking to project in this vein you can find more natural ways of code organization and solution for your problems.

Some other things:

1. Of course, you must follow @CodeInChaos suggestion and share one copy of source file between all projects instead of copying it to each project manually. It may be useful if you adopt some standard for building environment, which will be mandatory for all developers (same folder structure, location of libraries, environment settings).
2. Try to analyze building and deployment process: for me it's looking abnormal when solution not built with latest version of code and not tested before deployment. (it's for your "If I make a bug fix in a BPL, each programmer ..." phrase).
3. Variant with standalone executable files looks better because significantly simplifies organization of testing environment and project deployment. Just choose adequate conventions for versioning.

Poulterer