Windows Installer "Error 1308. Source file not found" when uninstalling patch in sequence scenario

Question

I require a series of uninstallable patches created using Patch Design with Installshield 2012. The first two patches work fine when uninstalling. However, the third patch, if and only if uninstalled when patch 1 and/or patch 2 have already been applied, produce errors:

MSI (c) (48:C4) [19:02:54:135]: Font created.  Charset: Req=0, Ret=0, Font: Req=MS Shell Dlg, Ret=MS Shell Dlg
Error 1308.Source file not found: {pathToFile}.  Verify that the file exists and that you can access it.

There are 26 of these errors regarding different files. There isn't an obvious pattern to the files or the components, or there features

Note: If I only have patch 3 applied, uninstalling does NOT produce this error..

I created all three patches with the same options in Patch Design. The only noticeable difference I understand is that patch 3 contains many more changes (file updates) than the first two. Let me repeat: MANY more changes.

My questions are:

1. Why does this happen only in the case where the series of patches are installed, instead of just the third patch itself?

2. What do I have to do in order to prevent the patch uninstall to try to fetch files from a location which should only be for design-time when building the patch? Or perhaps this is fetching is by design, however the cache is just too overloaded or confused..?

UPDATE - MORE INFO (requested by Glytzhkof): The patch contains 96 file changes, which is roughly half the size of base MSI package. It is actually off of 'Dev' branch work. Several new files have been added. Some were originally removed (had to put them back when I found we really were doing a patch...). If I described the situation any more, it might offend you as a professional in the field.

I have been trying to sell the Major Upgrade, and it would take just a few tweaks to the installer to make it obsolete the need for patches. The uninstall of our product requires parameter in order for it to be non-interactive (we would need this parameter to work in Major Upgrade scenario, it is currently only part of the Uninstall sequence). That is the only real issue - but fixing it would pay in spades. It was decided not to fix that issue, however. I try to 'bump' that issue every iteration. No dice. We need patches for major releases I am told - so here I am trying to get the tail to wag the dog.

And yes, patches can be faster (let me play devil's advocate here). But really, the difference between 30 and 90 seconds, when these things are automatically deployed anyway? And yes, I have also considered finding ways to optimize the installer with adjusting the file costing to see if it makes it any faster, but even then I'm sure there will be another reason why a patch will be requested.

ANOTHER UPDATE: The files mentioned in the 1308 errors are not on the target system's %windir%Installer\$PatchCache$\Managed\{PackedProductCodeOfMyBaseMSI??}

folder. This could cause the 1308, because if I remove more files from this cache, I get same error corresponding to the missing file. The question could be, why aren't ALL the file's in this PatchCache folder?

Coccid

Patching should in my opinion only be used for hotfixes of single files and for fixing errors in the uninstall sequence of existing products. This is what works in the real world in my experience. Can I ask why you require so many patches? Are you doing bit-level patching or do you include whole files? How many files in each patch? Are you patching the same product or different products? – Amplification

@ Glytzhkof I agree, and I kindly warned that this is not really what patching was designed for... Our first two patches were one file. The third patch, and this should make you laugh, around 200 file changes! Basically, a patch off Dev! I am doing Whole Files, and really only patching one product. – Coccid

I am still looking for a solution or at least some guidance, even though I agree that this is outside of normal good practices. – Coccid

Perhaps you can try your patch remade with Wix by decompiling the MSI file coming out of Installshield with dark.exe and then cleaning up the XML. Do you know Wix at all? It has a learning curve. codeproject.com/Tips/105638/… – Amplification

I have used WiX DTF CAs consumed by Installshield's tooling, that is about it. I may try this. Is this suggestion coming from a hunch about my latest update regarding the $PatchCache$ folder contents? Or just in general? – Coccid

It is coming from experience with Installshield as buggy: https://mcmap.net/q/104007/-what-installation-product-to-use-installshield-wix-wise-advanced-installer-etc-closed , patching as difficult and Wix as reliable but involved. Try the dark.exe tool on the compiled MSIs from Installshield and try to generate a patch with Wix. I haven't tried this at all, can't help there. – Amplification

I already have long-term plans to migrate to WiX, after a discussion with Christopher Painter and then a few hours of testing out). To me, because of Windows Installers complexity - control is king, even at the expense of having limited UI tools, and being more 'code driven'. Unfortunately there would be a lot of re-work because WiX doesn't appear to have a 'release flags' feature that Installshield has. That is the one reason I cannot move to WiX 'right now'. I will add that your comments here confirm my confidence that WiX move is the best move. – Coccid

Answer 1

I am still looking for a solution or at least some guidance, even though I agree that this is outside of normal good practices. – jJack 1 hour ago

I don't have access to my deployment tools, but I will try to provide a perspective. Since I don't fully grasp all aspects of what you write it will be generic comments. I hope it has at least something to do with what you are asking. It turned into a blog as I wrote.

To me an MSI patch is effective for 2 basic scenarios:

1. You fix an error in the uninstall sequence of an installed product with a minor upgrade patch.
2. You deliver a minor upgrade patch for a couple of files as a "hotfix" for a released product that might be huge and time consuming to reinstall.

For these two purposes I have used MSI patching effectively professionally several times. In every case there was no other good fix. Patching IMHO is for "hotfixing" - it is what the whole technology is designed for, not for deployment of frequent, incremental updates. Delivering 96 file updates is NOT hotfixing.

A Patch is a Working Upgrade: Remember that patching is just a more compact delivery mechanism for an upgrade that already works. It can be a major, minor or even small update, and each one will work differently. Before you do anything else at all, make sure you test your actual full upgrade MSI before attempting to package it as a patch. This is the best advice I can give you. All effort spent on patching is wasted if the full upgrade isn't working correctly. And yes, this includes install, uninstall, and upgrading in all interactions before making the patch itself. This is perhaps the most common patching mistakes of all.

Several obstacles exist preventing the uninstall of a patch. There are dozens of technical issues that may result in uninstallable patches (recommended read). This is a huge problem at times, since a patch that has deployed a hotfix may be found defective and hence should be pulled back entirely. In my view this is one of the core uses of a little patch - deploy a quick fix that can then be rolled back.

Patching & Custom Action Conditions: to me, one of the worst aspects of patching is that custom actions in packages may not be conditioned properly to NOT run when a patch operation is being performed as opposed to a normal installation. A patch features patch-specific properties such as PATCH and MSIPATCHREMOVE. Use these conditions on custom actions to make them run or not run during a patch depending on what is necessary. Be careful with the conditions on custom actions, they are complicated to get right. Here is a "MSI Conditions Cheat Sheet" to help you. I have not tested these conditions - testing is the only guarantee.

Some further patching advice:

• I would forget major upgrade patches entirely. I have tried them, and will try them again, but they tend to be less than ideal. An absolute requirement for a major upgrade patch to work is that RemoveExistingProducts is placed after InstallFinalize in the InstallExecuteSequence. The reason for this is that otherwise the files are uninstalled before the patch package tries to patch the existing files. Quite catch 22.
• A minor upgrade does not uninstall the existing installation, but rater re-caches a new MSI file to use for maintenance and uninstall operations. This means the patch can fix the uninstall sequence before it is run - one of the good patch uses that I listed above. In fact if the minor upgrade works, a patch may not be necessary at all. Just use the minor upgrade unless your MSI file is enormously big and you want to deliver a small "hotfix".
• If you need to include files in your patch, I recommend enabling "include whole files". Otherwise bit-level patching is done, and this is unnecessary complexity (unless your binaries are enormous). I am also not sure how bit-level patching works with signed files and digital signatures.
• Include only what you need in a patch. Add no files or settings that are not required, and you can make a reliable patch. Avoid adding custom actions if at all possible.
• As already mentioned: be aware that a patch uses the same InstallExecuteSequence as a regular install, but you can condition custom actions differently with patch-specific properties such as PATCH and MSIPATCHREMOVE. Use these conditions on custom actions to make them run or not run during a patch depending on what is necessary.
• Component referencing must be 100% correct for ANY type of patch to work. No exceptions.
• Minor upgrade patches must be run with the proper msiexec.exe command line to work unless delivered via a setup.exe / update.exe.
• Third party merge modules frequently cause patching problems in my experience.
• "Version lying" as they call it - the black art of ensuring files are always updated by adding a different version in the MSI file for the file on disk seem to cause patching errors.
• A patch will show the same GUI as the main install. In my opinion this is erroneous design. Custom actions in the GUI could mess up the patching process (should you accept new user input for values for a patch?).
• I believe each patch should be cumulative - replacing all previous patches. I never got this working properly back when I tested several patches installed in series and sequence. I concluded, for many reasons, that this was a futile patching approach to begin with. I had problems exactly along the lines of what you describe with the patching families, target releases etc... A patch isn't too smart, it is just a complex bundle of a few files trying to find the product it belongs to.

The obvious thing to conclude is that I really don't recommend that you go with this patching approach, even when asked to. However, I read this thread which seems to indicate successful patching for someone who has converted to using WIX instead of Installshield. You should check out the CodeProject link too.

As to your deployment scenario - I don't fully grasp all aspects of it, but it sounds like the developers want patches to convert a live application into the current QA version via a patch? I would never go along with this, or the scenario must be different from what it sounds like. Totally wasted effort to create a patch when you already must deliver minor or major upgrades in the first place - they are more than enough to deliver software for QA. You could use the dev-branch to deliver a separate MSI and then create a few patches now and then to test that the product is patchable, but I would never use patches to deliver installers of your product internally. I don't know if that is what you are asked to do.

Work with minor and major upgrades - preferably the latter for non-patches, and deliver a patch when you really need it. If installation duration is an issue, you could just schedule a daily major upgrade happening after the nightly build is complete on all developer and QA PCs? (including killing any running processes required to make the install work). I don't know if I am totally off track with what your scenario really is.

Check Stefan Kruger's installsite.org for more upgrades and patching tips.

Check out this well known wix tutorial for upgrades and patches. And MSDN.

Godgiven

The patch is absolutely not requested by QA or Dev. It is for the stakeholders who will be using, supporting, selling the product. Trust me, if it were up to me, it would be a well designed Major Upgrade, and that would save thousands and thousands of dollars and would just mean when Sys Admin runs SMS job to update our product, they have to sip coffee for an extra 60 seconds..I understand, in this scenario, it is truly a disservice to provide a patch. I don't make these decisions unfortunately. I 'just work here' :) – Coccid

It would be wrong for me to 'accept' this as the answer right now. I plan on figuring out a solution to the error and posting. But your blog-sized post here is great cheat sheet for everyone, nice work. – Coccid

I agree - I thought this was just about the worst answer I have ever written. It lacks structure and crucially a good answer to what you are asking :-), but sometimes it is good to provide content that can't be found in any books I guess. Looking forward to your solution post. – Amplification

Answer 2

I will have to add this as an answer since it is too long for a comment. 200 file changes? I guess file costing will take longer than installing a whole minor upgrade. If I were you I would refuse to deliver these kinds of patches. They are bound to fail, with almost 100% certainty due to the technology's complexity. Remember that an MSI patch is just a delivery mechanism of an already working upgrade with basically just added risk and complexity. This really is the case.

MSI patches are complex and registered for inspection and uninstall - they don't just dump and patch files like in the pre-MSI world. If the user demands patching, I would go to a different solution than MSI altogether.

Can you describe the scenario a bit better? I find that automated build processes delivering major upgrades work extremely well to deliver daily builds to QA. If installation speed is an issue you can use techniques to make things faster such as enabling MSIFASTINSTALL. With this property you can tell the system to perform less heavy and time consuming operations such as creating a system restore point and do thorough file costing (comparison of installed and installing files and directory sizes).

Godgiven

"automated build processes delivering major upgrades work extremely well". I agree. – Coccid

Answer 3

I'd look at any of the support articles about this error, of which there are several. They may be product specific in some cases but may point to an error (such as admininstrative installations or bugs in Windows Installer). Maybe you already did this.

Otherwise, these issues tend to occur when the patch supersedence settings are incorrect in some way. If you somehow said that one of the patches superseded another then by definition it includes all the patch changes from the earlier patches. Since this involves fiddling around with guids in various places perhaps a change was missed if the same base PCP file was used, although I'm not sure how much IS exposes or hides all that. The end result would be that Windows would think the other patches are no longer needed and would remove them. This might help:

http://msdn.microsoft.com/en-us/library/aa368345(v=vs.85).aspx

Pinelli

So far, patch sequencing doesn't seem to be the problem. Installshield so far has advised to use a Setup.exe launcher with 'Cache MSI Locally' enabled. Here is an interesting article about the Windows Insatller package caching (distinct from the 'Cache Msi Locally' setting in launcher??): blogs.msdn.com/b/heaths/archive/2009/02/02/… – Coccid

Answer 4

Addressing that IS recommendation is too long for a comment, so:

Caching a copy of the actual install MSI locally has always been a good idea (search for the Tao of Windows Installer rules) in case of repair, adding features etc, but Heath's article is about 5.0 changes to the internally cached MSI file which is not the same thing. IS may be saying "keep a copy of the actual MSI file available", Heath is simply describing an internal change to the "secret" cached MSI file but that internally cached MSI is not considered a valid source. Be sure which "cached" they mean. I think they mean that you should keep a copy of the actual MSI available when you install it.

My guess is that IS sees the underlying issue not so much as the missing patch, but as Windows needing to restore the original files on a patch uninstall, so if the msp file isn't available then it can get it from the original virtual MSI file that consists of the original plus patches. If you use the cached MSI option of IS I assume it copies the MSI file somewhere and installs it from there or otherwise makes that location a valid source. In the event of needing to restore files on a patch uninstall, their analysis is that it can be done (automatically?) using the (base MSI+any applicable patches) to restore the previous files. I'm trying to read IS's mind from your brief comment, and maybe it helps although I'm sure I haven't got it completely accurate.

Pinelli