Third party app breaks our WCF application
Asked Answered
P

4

26

Our application uses WCF over named pipes to communicate between two processes (note: neither process is a Windows Service.) Our application has been running in the field without incident for a couple of years now.

We are now receiving reports that the presence of a third party application (specifically, Garmin Express) is breaking ours. I've installed Garmin Express in house and confirmed the behavior. Specifically the "Garmin Core Update Service", when running, causes our application to fail.

When the Garmin service is running, the "service" side of our application starts and has no problems creating the WCF endpoint. But when the client starts and attempts to connect to the service, it fails with EndpointNotFoundException, as if the service is not even running.

At this point, I can literally stop the Garmin service from the Services control panel, then re-run the client successfully without even restarting our own service. If I start the Garmin service again, further attempts to start the client fail. So this at least proves that our WCF service is up and running the whole time, and the Garmin software is somehow blocking our client's ability to connect to it.

We are using our own name for the endpoint address (e.g. something like net.pipe://localhost/MyPrivateApplication). I've tried changing this address to various other names, but that has no effect on the issue.

How can another application, simply by running, break our own application's ability to use WCF?

Update: by request, here is a code snippet from the service side. I've simplified it from our original code in an attempt to isolate the issue. So far, not a single change I've made has had any effect on the issue.

MyService service = new MyService();
ServiceHost host = new ServiceHost(service);
string hostAddress = new Uri("net.pipe://localhost/MyWCFConnection");
host.AddServiceEndpoint(typeof(IMyService), new NetNamedPipeBinding(), hostAddress);
host.Open();
Platonic answered 12/4, 2013 at 21:47 Comment(5)
How does your service specify its Service URL? Is it base + relative address or an absoute address? Maybe you could post extracts from code/config showing how your service endpoint is established.Orlov
I've updated my post with a code snippet. I'm fairly sure this is an absolute address, correct? Again, this service-side code executes successfully whether the Garmin service is running or not. Only the client side is affected.Platonic
Yes, that is an absolute address, so there is something more subtle going on than my first suggestion.Orlov
FYI I emailed Garmin about it and they said they have a fix in testing and expect it to be out in version 2.2.x .Hollar
Hello, I am neding to reproduce this issue locally. Do you have the version number of the affected garmin software?Primarily
O
17

One possible answer to your question "How can another application, simply by running, break our own application...":

  1. The other application also uses the WCF NetNamedPipeBinding.
  2. Both applications create service endpoints using base + relative URLs.
  3. The applications' choice of base address, and HostNameComparisonMode is such that there is a name collision between the applications on one of the URL variants used by the client-side WCF stack to locate the metadata for the service.

I have no idea whether the Garmin service actually does use WCF NetNamedPipeBinding, but this is one possibility you should investigate. The problem can be avoided by always using absolute URLs for NetNamedPipe endpoints.


OK, so following the updates to the question, we now know that the Garmin service is using WCF NetNamedPipeBinding, and we know that your application registers its service using an absolute address, so the above explanation is not the complete story.

Here is another hypothesis:

  1. Suppose the Garmin service runs in a process which has the security privilege SeCreateGlobalPrivilege (which a Windows service will have unless specially coded to disable the privilege).
  2. Suppose it also registers its WCF Named Pipe endpoint with a base address of net.pipe://localhost and relative endpoint addresses.
  3. Now its service metadata will be published using a shared memory mapping object with a name in the Global namespace.
  4. Your service application is not a Windows service. My hypothesis is that its process does not have the security privilege SeCreateGlobalPrivilege. If this is the case, its service metadata will be published using a shared memory mapping object in its Local session namespace only.
  5. Now your client process tries to initiate a connection when the Garmin service is running... the WCF client-side channel stack NetNamedPipeBinding element tries to locate the service metadata for your service based on your service URL net.pipe://localhost/MyWCFConnection. As explained in the link above, it will execute the search using various variants of the service URL in turn to derive a name for the shared memory object containing the metadata. It looks in the Global namespace first, for the full list of variants in turn, before looking into the Local namespace.
  6. In this case, the first attempt will be for the name derived from "net.pipe://+/MyWCFConnection", and presumably it fails to find an object with this name in the Global namespace.
  7. However, the second attempt will be based on the variant "net.pipe://+/", and this will match the name of the Garmin service's shared memory mapping published in the Global namespace. Because of the search order, it will never get to your service's metadata published in the Local session namespace.
  8. Your client attempts to connect to the Garmin service's pipe. Let's assume the Garmin service has some security implemented which results in your client being rebuffed with an Access Denied (e.g. it may set an ACL on its pipe). The result might well surface as an EndpointNotFoundException. [LATER EDIT: Actually, most likely what is happening is that your client is actually connecting to the Garmin service, initiating the framing protocol's preamble handshake, and receiving back a framing protocol fault (http://schemas.microsoft.com/ws/2006/05/framing/faults/EndpointNotFound) because the URL requested in the Via record won't match what the Garmin service is expecting. The binding is then dropping the connection and surfacing this fault to your client code as an EndpointNotFoundException.]

What can you do about it? I would suggest:

  • If the above hypothesis or something similar can be confirmed, and Garmin are using base+relative addressing with a base of just net.pipe://localhost, best would be to get them to own the problem: they could fix such a problem very easily by changing their base address to something more likely to be unique.
  • You could perhaps work around it by finding some way for your service application to run with the security privilege SeCreateGlobalPrivilege: this isn't easy without making it a Windows service or running As Administrator, but maybe not impossible. Then your metadata would also be published in the Global namespace and the client's search would find it before Garmin's.
  • [Later edit] Maybe there is a workaround involving setting the HostNameComparisonMode property of the binding to Exact, and using a synonym for localhost as the host part of the service URL (e.g. net.pipe://127.0.0.1/MyWCFConnection). This may steer the search around the Garmin variants so that your client has a chance to consider names in the Local session namespace. I don't know that it will work, but worth a try, I would have thought.
  • And a very long shot: Does your company have a product support relationship with Microsoft? Arguably this is a serious design flaw in WCF: if you make a fuss about it you might possibly get Microsoft to issue a QFE patch for it e.g. to provide a binding property to tell the client-side stack to only try the Local namespace.
Orlov answered 13/4, 2013 at 11:4 Comment(5)
I'm fairly sure the Garmin Service is also using WCF NetNamedPipeBinding, based on the information in this article. When the Garmin service is started, a new named pipe appears with the same GUID-style name as you get with other WCF services that use named pipes.Platonic
I've confirmed that the problem goes away if I run the service side process as Administrator. That certainly lends support to your second theory. This won't help us solve the problem in the field but at least we have a possible explanation now.Platonic
So, it sounds like we are stuck. We need to fix this issue in the field now, we can't wait for Garmin to maybe or maybe not fix it at the leisure of their product release cycle, that could be months or even years. The service-side process portion of our product runs under the end user's account, so AFAIK there is no way to run it with SeCreateGlobalPrivilege.Platonic
See edits above for a couple of further thoughts on what you might do.Orlov
Thanks for the workaround suggestion, I had actually already tried exactly that (setting comparison mode to Exact and trying host names other than local host.) Unfortunately, it doesn't seem to help. Amazing that there is apparently no way to steer the search pattern around the global table from the client side.Platonic
P
11

I have found a method to display which applications use net.pipe (though not necessarily which are using it incorrectly).

First download the Handle application from Sysinternals (Microsoft). As a side-note: Process Explorer also lets you search for handles.

Then open a command prompt as administrator, and run Handle.exe net.pipe (minus quotes). This will list all applications using net.pipe that are currently running. From there, you can kill or disable one at a time, until your culprit is discovered. I almost never have more than 4-5 processes using it. If you fail to run command prompt as administrator, it may give zero or only irrelevant results.

Below are all the applications I've found that interfere with net.pipe:

  • "HP Support Solutions Framework Service" - only some versions affected

  • "Garmin Core Update Service" - fixed in newer versions but out of box is broken

  • "WBE Service" - used by a couple dell laptops in conjunction with a wireless docking station

  • "Intel(R) Security Assist" Service - I saw on a couple of Win10 laptops early 2016.

  • "Baraccuda WSA Service" - Web Security Agent. Probably would upset a customer if you disabled this.

  • "DropboxOEM.exe" - A variant of Dropbox for inclusion in store-bought PC's. Only noticed on Win10 so far. This one is unique, because it is the first I've found that is not a windows service, to the best I can tell.

  • "MTC Service" - Installed on some Getac brand PC's. Unsure what it does.

  • "pcdrcui.exe" - Not a service, but runs as admin. Component of Dell's SupportAssist.

  • "Mitchell1/Shopkey SE Connection" or "ShopHubService" or "Mitchel1/Shopkey Data Backup Service" - Data synchronization service. Unsure what all it does.

  • Procore Drive (Procore DriveService.exe). Uses net.pipe://+/:

     Procore DriveService.exe pid: 4204   type: Section        43C: \BaseNamedObjects\net.pipe:EbmV0LnBpcGU6Ly8rLw==
    
  • Keynetix.Cloud.Launcher.Service.exe. Uses net.pipe://+/:

     Keynetix.Cloud.Launcher.Service.exe pid: 5524   type: Section        4B8: \BaseNamedObjects\net.pipe:EbmV0LnBpcGU6Ly8rLw==
    
  • RevitAccelerator.exe (part of Autodesk Revit). This only gets run elevated immediately after installing Revit. Also, this issue is fixed in Autodesk Revit 2020.

  • Wonderware InTouch IData Service (SE.Scada.Asb.InTouchDataService.exe) that comes with the Wonderware InTouch HMI system

  • WKSSTrayNotification.exe (not a service, but a tray application part of ADDISON software)

  • EisBär SCADA V3.0 (service) by Alexander Maier GmbH

Software maker DATEV has another list in German here (archived).

I support an application that requires net.pipe, so I'll update this list as I find more services that do this.


Stripping up to and including the leading net.pipe:E from the name will also help in making out the culprit, because the remainder of the name is Base64-encoded (also here). So taking \BaseNamedObjects\net.pipe:EbmV0LnBpcGU6Ly8rLw== from above as an example with PowerShell we can decode the name to:

$ [System.Text.Encoding]::UTF8.GetString([Convert]::FromBase64String("bmV0LnBpcGU6Ly8rLw=="))
net.pipe://+/
Primarily answered 18/2, 2016 at 17:42 Comment(2)
Thanks a lot for this very helpful reply, saved us a lot of headache! We had the exact same situation as the OP and were able to identify the culprit by following your instructions; in our case it was "Wonderware InTouch IData Service" (SE.Scada.Asb.InTouchDataService.exe) that comes with the Wonderware InTouch HMI system (I don't know exactly what it does). Killing this process suddenly made our WCF-based application functional again.Gayn
In our case it was WKSSTrayNotification.exe (not a service, but a tray application part of Addison Software). Here is a knowledge base article of german software editor Datev with another list of culprits: datev.de/dnlexom/client/app/index.html#/document/1080431Reconstruction
B
8

We at Microsoft have zeroed down the issue to the way Garmin Core Update Service creates named pipes. Named pipes can be created in different Scopes – Global and Local. Global scope is essentially machine wide and Local is specific to the user.

Garmin’s application is

  1. running as System service, so the scope for listening named pipe service is global.
  2. listening on the root address of “net.pipe://localhost/” (e.g. without any sub-paths/segments).
  3. Using a StrongWildcard host name comparison mode.
  4. Items 1 through 3 mean that Garmin’s application is essentially a catch-all for any net-pipe connection that doesn’t match something more specific.
  5. It also means that Garmin is completely blocking all listeners that are using a local scope

The ideal fix for this would be a change in Garmin application such that it registers its net.pipe listener with a more specific URL.

Brede answered 8/5, 2013 at 4:2 Comment(3)
I think Microsoft should plan to fix it or hot fix it. We just found that service "HP Support Solutions Framework Service" is also causing this issue.Metry
Why can't this be fixed using appcompat shims for existing software?Poorhouse
@Poorhouse with .NET Framework 4.6.2 there's an app config switch to fix this. See my answer stackoverflow.com/a/74371039.Settler
S
3

With an application compiled against .NET Framework 4.x and running against.NET Framework 4.6.2 or higher the problem can be fixed without recompiling. There's a new App.config switch that changes the lookup behaviour for WCF named pipes to use the best matching base address.

So to fix it the following is required:

  • Use a unique base address with a relative path, e.g. net.pipe://127.0.0.1/<MySpecificApplication> (replace <MySpecificApplication> with your unique relative path)
  • Add an App.config to the project of the WCF client (for Desktop applications) or Web.config (for IIS and the ASP.NET Core Module) with the content. If this config file already exists in the project, add the XML elements to the config that don't exist yet.
  • For existing Desktop installations without recompiling: Copy the file and rename it to <name-of-the-wcf-client-exe-file>.exe.config and copy it next to the exe <name-of-the-wcf-client-exe-file>.exe in your application's folder.
<configuration>
    <appSettings>
        <add key="wcf:useBestMatchNamedPipeUri" value="true" />
    </appSettings>
</configuration>

I confirmed this fixes the problem for a desktop application compiled against .NET Framework 4.7.2 and running against .NET Framework 4.8.


Detailed explanation

WCF named pipes use an indirection via a file mapping object / section object that is called after the base address of the connection to find the named pipe to connect to.

Default lookup
By default the lookup strategy for this object first tries objects in the global namespace (services and applications run as adminstrator create their sections here) and afterwards in the local namespace (applications not run as administrator create their sections here).

In the global namespaces it starts with the specified base address (e.g. net.pipe://127.0.0.1/<MySpecificApplication>) and then walks up the path hierarchy until it reaches the root (e.g. net.pipe://127.0.0.1/). Only if no match is found in the global namespace, the local namespace is searched and walked up the hierarchy.

Lookup with wcf:useBestMatchNamedPipeUri=true
The order stays the same (global namespace first, then local and from most specific base address up to the root in each namespace), but the lookup doesn't stop at the first match and instead looks for the best match (longest substring of the base address).

Sources:

NetNamedPipeBinding best match

WCF has a new app setting that can be set on client applications to ensure they always connect to the service listening on the URI that best matches the one that they request. With this app setting set to false (the default), it is possible for clients using NetNamedPipeBinding to attempt to connect to a service listening on a URI that is a substring of the requested URI.

For example, a client tries to connect to a service listening at net.pipe://localhost/Service1, but a different service on that machine running with administrator privilege is listening at net.pipe://localhost. With this app setting set to false, the client would attempt to connect to the wrong service. After setting the app setting to true, the client will always connect to the best matching service.

Note

Clients using NetNamedPipeBinding find services based on the service's base address (if it exists) rather than the full endpoint address. To ensure this setting always works the service should use a unique base address.

To enable this change, add the following app setting to your client application's App.config or Web.config file: XML

<configuration>
    <appSettings>
        <add key="wcf:useBestMatchNamedPipeUri" value="true" />
    </appSettings>
</configuration>

Source: https://learn.microsoft.com/en-us/dotnet/framework/whats-new/#windows-communication-foundation

Settler answered 9/11, 2022 at 7:3 Comment(0)

© 2022 - 2024 — McMap. All rights reserved.