Well, after three days of hopeless debugging and several attempts to find the root of this problem, we finally figured it out. See below for the workaround.
Delivery Optimization Service incorrectly caches the size of any HTTP resource it retrieves (it can be appinstaller file or MSIX package), and includes the Range
HTTP header in the subsequent requests with potentially outdated byte range values.
For example, if your appinstaller is 725 bytes long, first time Windows DO Service makes a well-formed HTTP request and downloads the whole XML. Request example:
GET https://foobar.com/baz.appinstaller HTTP/1.1
Connection: Keep-Alive
Accept: */*
Range: bytes=0-724
User-Agent: Microsoft-Delivery-Optimization/10.0
MS-CV: ......
Content-Length: 0
Host: foobar.com
But if you have updated the appinstaller file at your web server, and its size is increased, for example, up to 4096 bytes, DoSvc still requests only first 725 bytes, and obviously gets a broken XML it couldn't parse. Even if ETag
is handled correctly and has been changed for the appinstaller HTTP resource!
If your appinstaller file size decreases, it's likely that you will get 416 Range Not Satisfiable.
Here is the real response from Amazon S3 (where we host our appinstaller and MSIX packages) we have dumped using Fiddler:
HTTP/1.1 206 Partial Content
x-amz-id-2: ...
x-amz-request-id: ...
Date: Tue, 06 Apr 2021 21:45:12 GMT
Last-Modified: Tue, 06 Apr 2021 21:40:24 GMT
ETag: "af2d8bb5c638eca059cdb4dc6c694123"
Accept-Ranges: bytes
Content-Range: bytes 0-724/4096
Content-Type: application/appinstaller
Content-Length: 725
Server: AmazonS3
<?xml version="1.0" encoding="utf-8"?>
<AppInstaller
Uri="https://tvd-packages.tradingview.com/beta/latest/win32/TradingView.appinstaller"
Version="1.0.0.1071"
xmlns="http://schemas.microsoft.com/appx/appinstaller/2017/2">
<MainPackage
Name="TradingView.Desktop"
Version="1.0.0.1071"
Publisher="CN="TradingView, Inc.", O="TradingView, Inc.", STREET=470 Olde Worthington Road Suite 200, L=Westerville, S=Ohio, PostalCode=43082, C=US"
ProcessorArchitecture="x64"
Uri="https://tvd-packages.tradingview.com/beta/1.0.0-beta.1.5/win32/x64/TradingView.msix" />
<UpdateSettings>
<OnLaunch HoursBetweenUpdateChecks="0" />
</UpdateSettings>
<!--
- WHAT TH
Do you note Content-Range: bytes 0-724/4096
and the comment cut off?? This is the result of ill-formed HTTP Range request by Delivery Optimization Service.
WORKAROUND: Have you guessed it already? Yeah, just don't change the size of the appinstaller! For example, we padded it with XML comment up to 4096 bytes exactly, and if we need to add more sensible content to XML, we just remove some chars from XML comment to make it 4096 bytes again. We have even made a simple size test in our gitlab deployment pipeline to ensure that size of appinstaller was not accidentally changed.
You can check our current production appinstaller we use since April, 2021: https://tvd-packages.tradingview.com/stable/latest/win32/TradingView.appinstaller
P.S. The problem turns up if requests are performed to the same URL. If the URL changes, DoSvc HTTP client logic handles it as a different resource. That's why the appinstaller files are more likely victims of this problem.