Solving Delphi BPL Package problems where BPLs won't load but you've already recompiled (Windows VirtualStore filesystem issue)

Question

My general question is how do you troubleshoot "My BPL won't load due to a dependency that just won't go away, no matter how much I clean up and recompile". Update You may think you have a clean recompiled system, but thanks to the inverse-miracle that is Windows and its file system virtualization mis-features, you haven't.

When I try to load my designtime package (in this case named dclFsTee.bpl) into my Delphi IDE (it's the fast report 4 teechart wrapper component package), it complains:

The program can't start because tee7100.bpl is missing from your computer. Try reinstalling ...

That tee7100.bpl is not referenced on any DCP or DCU file on my system THAT I KNOW OF. But clearly, something is wrong, and I can't find the problem.

All Delphi users face a hundred "won't compile or won't load" problems with BPLs. The universal refrain when asked what to do is to clean up your computer.

However, I've now spent hours cleaning up my computer, and while everything compiles file, clearly there must be something out of date hiding somewhere, because the resulting BPL file that I'm trying to load still wants to load a version of a TeeChart BPL that I removed from this system days ago, along with every trace I could find.

The traces of TeeChart stuff in Delphi 2007 that I removed include everything in the $(BDS)\Lib and $(BDS)\Lib\debug folder, and all DCP and BPL folders on the system. Also every TeeChart-unit-named dcu file is gone.

Once you've gotten to the end of the road, what do you try next? (Format the hard drive, buy new computer.) Seriously. I think I'm a smart guy, but I have a 1 tb hard drive, a library path that runs to 80+ folders, and a source code repository that seems to be well organized, but clearly something is hiding where I can't find it.

I have TeeChart Standard 2012, with full source code, and as far as I know, my development machine no longer contains any old TeeChart BPLs or DCP files from the "tee chart tee7100.bpl" version that ships with delphi.

I have run the "recompile.exe" wizard that comes with teechart, which appears to just run MSBuild and build the packages, after writing a {$DEFINE x} declaration to the tee.inc files (there are two of them in the source distribution).

However, somehow, silently it seems like one of the implicit imports into one of the packages is drawing in some stale file which has not been rebuilt, and which therefore tries to load the tee7100.bpl. The new bpl name is tee911.bpl.

Rather than ask the pretty-specific-to-fastreport question, I'm only mentioning it as a specific instance of a general world of hurt that I have faced dozens of times while developing in Delphi.

I'm only giving the fast-report details so you can see that this is in fact a specific instance of a general problem that one faces sometimes inside Delphi IDE when dealing with a component source code or package, or set of packages, with dependencies. Cleaning up your computer so that your code even builds can be tricky.

So here is my Delphi package-to-package-dependency-resolution question:

1. What is the most effective way to find or trace implicit-load-of-some-no-longer-wanted BPL-problems so that my code (which builds and compiles just fine!) will actually load into the Delphi IDE. The BPL file that results from running Recompile seems to be linking properly to the right DCP files, and no old/stale DCP or DCU files are present. The new DCP file name is tee911.dcp, for instance.

2. Can you get somehow, any idea of what package is actually stale, and what is being read and linked and imported statically when the .bpl links? (I'm thinking maybe like a special MAP-like file for BPL files?)

Update After many hours of fighting with this, and using every trick I know, I realized I hadn't checked for some VirtualStore related issues caused by file virtualization in Windows 7. That means that Windows 7 lies to the programs that run on top of it. It gives you another version of the file, that isn't the one you want. This can be deadly in several ways; One; You recompile a BPL but that's not the one that loads. The BPL that was killing me was in the SysWow64 folder that was part of the VirtualStore. Note that the virtualstore basically makes phantom files appear that are only there if you're a certain "low privelege" program, which Delphi 2007 on Win7/64 bit, apparently is. To remove BPL files in your SysWow64 VIRTUALSTORE folder for your current user account:

   del %HOMEPATH%\AppData\Local\VirtualStore\Windows\SysWow64\*.bpl

... Some days I just hate Windows architecture. Anyways, I'm not going to put the above as the answer, because I'd like to know if anyone has a better way or any tip or suggestion that might help next time.

Antebi

Wa are talking about a load of the bpl in the IDE (ie a design time package) aren't we and not the load of the bpl at run time by an exe of your own making? – Greyhound

Yes, designtime loading of a BPL which I've just recompiled from sources, but which must be pulling in something stale, that I just can't find... – Antebi

Assuming you already searched every folder on your environment's system path, I'd search the registry, keys and data, for the 'unfound' file name. Oh, and I'd check the C:\users\public and c:\users\all folders. – Greyhound

Was dclFsTee.bpl built against tee7100.bpl? If so, you can't replace TeeChart with a newer version that uses a different BPL name, unless you can also build dclFsTee.bpl from its source code. – Intercross

Oh, in your question I only noticed the recompile utility that comes with TeeChart, not that you also did the same for FastReport. – Intercross

When I install FR on my working system I always compile/install packages manually, from sources, one by one, sometimes edit sources if something goes wrong. Yes, it takes time. – Alejoa

If you have FR sources (I have no on the laptop I am right now) why don't you just edit the culprit package and rebuild it? It should contain the reference to tee71 or what it is in requires clause. – Alejoa

@Serg; I thought I made it clear that I DID REBUILD everything, including the culprit package. However it appears that the rebuilt .BPL is not the one loading, it's one that the BDS.exe is grabbing from the VirtualStore, another lovely Win7 misfeature. – Antebi

Interesting. I guess FR installation program put these bpl's to a windows system directory - another reason not to use component installation programs. – Alejoa

I can't decide which I'm more upset with : How crazy the copy-files-all-over-your-system behaviour of recompile.exe or the craziness that is the Windows 7 VirtualStore. – Antebi

Virtual store is fine. Why are you still using programs that are virtualized? – Curtis

What happens is the BPL is located in the list of folders to be virtualized by windows. Try to exclude BPL extension to be virtualized. The LUA File Virtualization Filter Driver is in the registry HKEY_LOCAL_MACHINE\System\CurrentControlSet\Services\Luafv\Parameters\ExcludedExtensionsAdd. You can add the BPL extension here, without the dot, and try if it is fixed the issue. – Gifferd

The cause of the issue was the installers from Steema (TeeChart) and the recompile.exe utility from FastReport. Together, they created a perfect storm of badness, because they like to install stuff in the system directory and in the "Program Files" folder, both are places that I don't want my Delphi components to install source code or binaries. I really abhor that practice. So much so that I strip away all their evil crap and just install from my own sources. But getting the initial copy of the sources requires running their evil installers, which makes a VirtuaStore mess. – Antebi

Answer 1

Okay nobody else answered so I'll put this here to be helpful for future people:

-- Remember Windows VirtualStore when cleaning up broken systems which have old versions of DLLs on them including TeeChart, FastReport, Indy and so on which tend to be involved in messes because they can exist both as "out of box packages that ship with delphi" as well as frequently installed as upgraded versions if you purchased and installed them from the vendors directly, or third, you may have your own compiled copy in your company's mega-component-pack-directory.

-- When searching for duplicate or out of date BPLs, doing a file search in windows doesn't look in the virtualstores, you'll have to locate and zap the whole virtualstore area for your process or user, or program, manually.

The second level of this issue is this:

The dependency graph for FastReports is complex:

• It depends on Indy and you might have your own version of Indy, and Delphi itself has one, and other things on your hard drive might have their own copy of Indy.

• It supports various editions of TeeChart, including the binaries that come with Delphi, and perhaps the Standard version or other purchased version of TeeChart that you might have bought from Steema.

• It uses a precompiled header include file to do the compilation and not just ONE but TWO different copies of an identically named include (.inc) file.

• When you use their own compiler tool (recompile FastReport) it works pretty reliably but isn't the best for when you want to build everything in your project from a single build script, thus the source of my problem.

• The key is to learn everything there is to know about the dependencies of all the components in your giant pile of packages, and to organize your system cleanly so that you don't have old stuff (like Indy and TeeChart bpls, dcp, or dcu files) lying around. Cleaning that up is quite a complex job if you don't know what you're doing.

  • A utility to really remove all traces of the version of Indy and TeeChart that ship with your system, and the "Embarcadero edition" of FastReports is key to getting this situation resolved. A general tip is that "if a version of X ships with Delphi and you are going to install a new version, prepare to suffer until your system is really cleaned up".

  • A really amazing technique to avoid all this crap is to just not install Indy, FastReport or TeeChart (uncheck them or skip them) during your initial Delphi IDE install, then install them yourself, one by one, from sources. Just because a version comes pre-installed in Delphi doesn't make that a good thing. (Update: You can no longer unselect Indy during install, it's part of the base Delphi product since at least Delphi XE8. A clean-up utility to remove the built-in Indy from Delphi's own lib dirs is necessary for anyone who builds their own.)

  • Another really amazing technique is to run the Installers for commercial components on a virtual machine, then just collect up the pascal source code and transfer that onto your clean development machine, and build it yourself. That way you can avoid the terrible things that happen when you've got BPLs and stuff scattered around your system, and even installed into C:\Windows\System32 (on 32 bit systems) and C:\Windows\SysWow64 (the equivalent path on 64 bit systems).

Antebi

I installed many components and haven't any problem but when I installed DevArt SDAC it is successful till I didn't exit the Delphi. Anytime I exit and rerun the Delphi show me the "program can't start because dac105.bpl is missing from your computer. Try reinstallin ..." I can't find away :( – Filomenafiloplume

Your path is probably wrong, missing either the directory containing dac105.bpl, or the directory that contains some other dll that dac105.bpl loads when it is loaded. – Antebi

Thanks@Warren. I found it. I forgot to install some .bpl maybe. Anyway I did as second way to install components to IDE (mentioned in last page of SDAC installer) and that is work perfectly. short section of second way is: "Find in the 'make.bat' line containing set IdeDir="D:\Program Files\Borland\Delphi and make sure that correct path to IDE is set and Run 'Make.bat' ...." – Filomenafiloplume

my path and file there but i still can't load bpl, any sugestion? – Bickford

comments are not where you ask questions on stack overflow. – Antebi

Answer 2

put that BPL (tee7100.bpl) under $(BDSCOMMONDIR)\Bpl

for XE: $(BDSCOMMONDIR)=  "C:\Users\Public\Documents\RAD Studio\8.0"
for XE5: $(BDSCOMMONDIR)=  "C:\Users\Public\Documents\RAD Studio\12.0"

Azoic

This wouldn't have helped in my case, but may help someone else who googles. +1 for trying to be helpful! – Antebi

Answer 3

The other issue that can cause this, is not having the folder where you've stored your .bpl files in your system path.

This happens because Delphi attempts to call the WinAPI function LoadLibrary with a file name, instead of an absolute path. So if Windows can't find the file, Delphi can't load it.

See this forum post for more information.

This seems to be an issue in Windows 7, though not in Windows 10.

Elf

For the simpler case of "My BPL won't load" (but the rest of my question which is about a transitive graph of dependencies failing) this is of course the simplest answer. In the case where we know that the path is not the cause, then the transitive graph of dependencies failing either due to the path or due to a mismatch of internal unit interfaces versions is the second cause. – Antebi

@WarrenP Absolutely. I just added this because yours was the only question I found about bpl files failing to load, and this was the solution for my case. – Elf