Moving to use of Delphi packages - best practice please?

Question

I am trying to start making my own libraries avaialble as packages prior to compiling my Apps with these packages hence modularising my code. For years I've 'sort of' understood packages, breathing a sigh of relief when I load a component package and click on 'Install' and it does. I understand that the process of installing a component (or components) is via the creation of a BPL which is then registered with the IDE.

Where I begin to get lost is how to make files available so that I can compile with EITHER a package OR pre-compiled dcu's (like the third party vendors do) and without pointing my project at the source code all the time. I can create a package with the following settings:

where I've specified that all my output will go into 'c:\scratch\wow'. After a build I find TEST.BPL, TEST.DCP and lots of DUC's. Now, when I point another project at this folder to use the DCU's, I get a missing DFM error (one of the units is a form). Should I be manually copying needed DFM's into this output folder? The DPK knows about this form, so why do I not get the DFM copied for me? I presume that using TEST.BPL, that file contains everything, but I wish to work in the two modes. Of course I can get around this by including the source folder in my project search path to find the DFM but third party libraries seem to already have the DFM's in their output folder. Did they install them there using the installer? Thanks

instead

Laufer

Why do you prefer packages to static linking? It sounds like it will give you lots of problems with no benefits. – Condign

Answer 1

Yes, you should copy the .dfm to the directory with the compiled units (.dcus), if that is the only directory you want in your search path. The BPL will of course contain the .dfms, and you need a .dcp to be able to link a BPL with your app.

Third party tools must have put the .dfms together with the .dcus in the directory using their installer, indeed.

Ingeingeberg

Answer 2

As others say you could use post-build events to copy your DFM files into place. Other people use a one-time external batch file that copies the DFMs to the DCU folder.

Personally I see very little benefit to making packages for things which are not developed also as reusable components. I also see very little benefit in partitioning an existing application into packages, when you don't reasonably need to use the same subsection or package more than once, or at designtime.

Things I would put into packages:

1. Delphi visual and non-visual components.

2. Things which absolutely must be plugged in at runtime, or left out. For example, supposing I sell MetaWare Light and MetaWare Pro, and instead of using compiler IFDEFs to build a differnt binary, I preferred for some reason to simply not ship the ADVANCEDFEATURE.BPL with my systems.

Things to beware of with packages:

1. I have run into a lot of compiler bugs when combining packages with generics. I have also run into IDE crashes and lockups, in Delphi 2009, 2010, XE and XE2. (I believe XE3 is better)

2. You should learn a bit about BorlandMM.dll and shared memory management in the BPL world before you move to it. There are some subtleties.

3. Packages limits the ability of the linker to decide what to remove. In fact, it pretty much destroys it. Packages contain everything that is linked into them, and nothing publically accessible can be removed.

4. Once you've created a binary package and shipped it to even one customer, you have a pretty difficult to modify contract (this BPL contains a particular signature or application binary interface) you have to be careful in the future to never change them, or mix and match them. Beware of DLL hell, even among your own customers, and be prepared to use versioning on your packages. Just as delphi packages have a version suffix, I recommend you use version suffixes in your own packages right off the bat, and bump them whenever binary compatibility has changed.

5. Delphi handles build dependencies between packages about as well as could be hoped, which is less well than a single monolithic application. In applications that I have that make heavy use of packages, I find project groups that contain a bunch of packages that depend on each other are very difficult to manage and build quickly. In fact, I have experienced that both compiles and builds are slower and more frustrating than they would be in a singular 750Kline megaproject.

I really wonder if you're not that into the package area of Delphi (you breath a sigh of relief whenever a delphi component actually builds and installs without issue?) if you really want to move into the Packages World totally. By all means, you should experiment. But I wouldn't bet the farm on it yet. Learn some more first.

Postmistress

+1 for a very nice explanation, thanks. I was aware of those issues (DLL hell) which is why I've held off so long, but I'm trying to catch up on all of this stuff including finding ways of further automating the installation of components that we use that have no installer - all of which requires me to understand packages. I have a fond hope that with XE I can move to BPL's with more pros than cons (my Apps are approx 750k lines). – Laufer

Interesting that now with Delphi 10 Seattle, with a 750K line project, you should be able to work WITHOUT PACKAGES and have a much better time than you could in Delphi XE era. – Postmistress

You seem to be saying that packages are more trouble then they're worth (and based on my experience I would tend to agree), however what about the case when you have large sections of code shared between multiple applications (e.g. many device objects), and you also want to enforce boundaries to reduce dependencies and maintain architecture. With standard DLLs you would have masses of flat function calls to maintain and it would be nice to access the object interfaces directly? Is it still a bad idea? – Whereinto

It's not that they're not worth it. Note that there is a difference between USING DESIGNTIME PACKAGES in the ide (which is core element of the delphi experience). It's whether or not you chose to build your own target applications using BPLs as part of it, that is really the problematic. You are not considering a sane alternative (dlls versus bpls). THe question is "dll or NOT". Absolutely EVERY Problem above with BPLs applies to DLLs because (get this one) BPLS ARE DLLS. Go static. Avoid making your own dlls. – Postmistress

Answer 3

Instead of copying *.DFM manually you can use Post-Build Event (Project/Options/Build Event), ex:

copy “$(PROJECTDIR)\Unit1.DFM” “c:\Scratch\wow\Unit1.DFM”

Jessie

Good idea. I'm already using that same method to use madExcept to patch the binary. – Laufer

@Brian: if you install madExcept into the IDE, you don't need a post build event to patch the exe. madExcept will do it itself (if its option is turned on, see Project|madExcept options once installed). – Ilario

@Serg: how is the performance of build events? I tried once and each build then took almost an additional 30 seconds just to copy the exe to another location... Gave up on it, but maybe too early? – Ilario

@Marjan - That is strange. I have never experienced problems with postbuild events, though I used them to copy *.res and *.dfm files only (while building packages). – Jessie

@Serg: thanks that's good to know. It may then just be dear ol' Kaspersky (anti-virus) getting its grubby hands in the mix when copying an executable... – Ilario

@Marjan Venema: is pretty common with AVs - since they try to monitor everything, they can cause such strange slowness (because they're analysing an big executable being copied)... It was the case on a ServerFault.com question some time ago..... – Header

@Marjan: Sorry, I was not clear - I use Post Builds for madExcept under control of a batch file. When in the IDE I let madExcept do its own thing. – Laufer

Answer 4

I found a way to do this without moving .dfm files to the directory of .dcu files, so you can have a directory for .dcu files only one for .dcp files only and another for .bpl files only.

All you need to do is to create another directory on your good structure, as I do. The directory is called RES and in it should be placed all the resource files (.res files, not .dcr files) that are used by applications compiled using your packages (components). In the Delphi Library Path, you must include in addition to the DCU directory (you should already have) a directory named RES.

On your component (design time) do everything you want with the form (design it, put other components, etc). In the source code of the unit you replace {$R *.dfm} with {$R UnitName.dfm}. In doing so, save all and close the DPK. Now move the .dfm file (do not copy, move!) to the RES folder (the .dfm file is a resource file to the Delphi. The {$R} directive is proof!) and after that open the DPK again to understand what has changed.

First realize that you may not open the form (F12) from his unit, though no error was issued by Delphi about "DFM missing".

Now, do a Build on your package and then install it. Realized again? No errors displayed! This happened because you have indicated the location of .dfm file in the Delphi library search path (RES directory).

Done! You can use your component and dfm will be found when your component is included in an application.

Many of you can now say that this way I will not longer be able to visually edit a form in the component design time. Yes this is true, but if you think about it, why would I want to change so often a form into a component that, in practice, should only be used and slightly edited? Draw your own conclusions ;)

Alveta

Wild guess: Does hardlinking the DFM file work? – Otiliaotina

@Ulrich Gerhardt - What do you mean by "hardlinking"? Place the contents of the .dfm file into a real .res file? I believe it can work, but I do not know how to do it ;) – Alveta

See en.wikipedia.org/wiki/Hard_link. It's a file system feature that allows you to have links to one file in multiple directories and with different names. – Otiliaotina

@Ulrich Gerhardt - Hi! I believe this will not work because the Delphi needs of the real files, but as this technique seems to be more sophisticated than making a normal link, you may want to try. If you are willing, as most know the subject, could try and post the results here. – Alveta

Hmm, a quick test indicates that Delphi breaks the hard link when saving the DFM. :-( – Otiliaotina