How to handle WCF exceptions (consolidated list with code)

Question

I'm attempting to extend this answer on SO to make a WCF client retry on transient network failures and handle other situations that require a retry such as authentication expiration.

Question:

What are the WCF exceptions that need to be handled, and what is the correct way to handle them?

Here are a few sample techniques that I'm hoping to see instead of or in addition to proxy.abort():

• Delay X seconds prior to retry
• Close and recreate a New() WCF client. Dispose the old one.
• Don't retry and rethrow this error
• Retry N times, then throw

Since it's unlikely one person knows all the exceptions or ways to resolve them, do share what you know. I'll aggregate the answers and approaches in the code sample below.

    // USAGE SAMPLE
    //int newOrderId = 0; // need a value for definite assignment
    //Service<IOrderService>.Use(orderService=>
    //{
    //  newOrderId = orderService.PlaceOrder(request);
    //}




    /// <summary>
    /// A safe WCF Proxy suitable when sessionmode=false
    /// </summary>
    /// <param name="codeBlock"></param>
    public static void Use(UseServiceDelegateVoid<T> codeBlock)
    {
        IClientChannel proxy = (IClientChannel)_channelFactory.CreateChannel();
        bool success = false;
        try
        {
            codeBlock((T)proxy);
            proxy.Close();
            success = true;
        }
        catch (CommunicationObjectAbortedException e)
        {
                // Object should be discarded if this is reached.  
                // Debugging discovered the following exception here:
                // "Connection can not be established because it has been aborted" 
            throw e;
        }
        catch (CommunicationObjectFaultedException e)
        {
            throw e;
        }
        catch (MessageSecurityException e)
        {
            throw e;
        }
        catch (ChannelTerminatedException)
        {
            proxy.Abort(); // Possibly retry?
        }
        catch (ServerTooBusyException)
        {
            proxy.Abort(); // Possibly retry?
        }
        catch (EndpointNotFoundException)
        {
            proxy.Abort(); // Possibly retry?
        }
        catch (FaultException)
        {
            proxy.Abort();
        }
        catch (CommunicationException)
        {
            proxy.Abort();
        }
        catch (TimeoutException)
        {
         // Sample error found during debug: 

         // The message could not be transferred within the allotted timeout of 
         //  00:01:00. There was no space available in the reliable channel's 
         //  transfer window. The time allotted to this operation may have been a 
         //  portion of a longer timeout.

            proxy.Abort();
        }
        catch (ObjectDisposedException )
        {
            //todo:  handle this duplex callback exception.  Occurs when client disappears.  
            // Source: https://mcmap.net/q/88338/-detecting-client-death-in-wcf-duplex-contracts/1428238#1428238
        }
        finally
        {
            if (!success)
            {
                proxy.Abort();
            }
        }
    }

Kesselring

For Petes sake, please get rid of the e in the throw e in those catch blocks. It throws away the entire stack trace before it, and makes logical troubleshooting into a guessing game. – Armourer

Answer 1

EDIT: There seems to be some inefficiencies with closing and reopening the client multiple times. I'm exploring solutions here and will update & expand this code if one is found. (or if David Khaykin posts an answer I'll mark it as accepted)

After tinkering around with this for a few years, the code below is my preferred strategy (after seeing this blog posting from the wayback machine) for dealing with WCF retries and handling exceptions.

I investigated every exception, what I would want to do with that exception, and noticed a common trait; every exception that needed a "retry" inherited from a common base class. I also noticed that every permFail exception that put the client into an invalid state also came from a shared base class.

The following example traps every WCF exception a client could through, and is extensible for your own custom channel errors.

Sample WCF Client Usage

Once you generate your client side proxy, this is all you need to implement it.

Service<IOrderService>.Use(orderService=>
{
  orderService.PlaceOrder(request);
}

ServiceDelegate.cs

Add this file to your solution. No changes are needed to this file, unless you want to alter the number of retries or what exceptions you want to handle.

public delegate void UseServiceDelegate<T>(T proxy);

public static class Service<T>
{
    public static ChannelFactory<T> _channelFactory = new ChannelFactory<T>(""); 

    public static void Use(UseServiceDelegate<T> codeBlock)
    {
        IClientChannel proxy = null;
        bool success = false;


       Exception mostRecentEx = null;
       int millsecondsToSleep = 1000;

       for(int i=0; i<5; i++)  // Attempt a maximum of 5 times 
       {
           // Proxy cann't be reused
           proxy = (IClientChannel)_channelFactory.CreateChannel();

           try
           {
               codeBlock((T)proxy);
               proxy.Close();
               success = true; 
               break;
           }
           catch (FaultException customFaultEx)
           {
               mostRecentEx = customFaultEx;
               proxy.Abort();

               //  Custom resolution for this app-level exception
               Thread.Sleep(millsecondsToSleep * (i + 1)); 
           }

           // The following is typically thrown on the client when a channel is terminated due to the server closing the connection.
           catch (ChannelTerminatedException cte)
           {
              mostRecentEx = cte;
               proxy.Abort();
               //  delay (backoff) and retry 
               Thread.Sleep(millsecondsToSleep  * (i + 1)); 
           }

           // The following is thrown when a remote endpoint could not be found or reached.  The endpoint may not be found or 
           // reachable because the remote endpoint is down, the remote endpoint is unreachable, or because the remote network is unreachable.
           catch (EndpointNotFoundException enfe)
           {
              mostRecentEx = enfe;
               proxy.Abort();
               //  delay (backoff) and retry 
               Thread.Sleep(millsecondsToSleep * (i + 1)); 
           }

           // The following exception that is thrown when a server is too busy to accept a message.
           catch (ServerTooBusyException stbe)
           {
              mostRecentEx = stbe;
               proxy.Abort();

               //  delay (backoff) and retry 
               Thread.Sleep(millsecondsToSleep * (i + 1)); 
           }
           catch (TimeoutException timeoutEx)
           {
               mostRecentEx = timeoutEx;
               proxy.Abort();

               //  delay (backoff) and retry 
               Thread.Sleep(millsecondsToSleep * (i + 1)); 
           } 
           catch (CommunicationException comException)
           {
               mostRecentEx = comException;
               proxy.Abort();

               //  delay (backoff) and retry 
               Thread.Sleep(millsecondsToSleep * (i + 1)); 
           }


           catch(Exception e)
           {
                // rethrow any other exception not defined here
                // You may want to define a custom Exception class to pass information such as failure count, and failure type
                proxy.Abort();
                throw e;  
           }
       }
       if (success == false && mostRecentEx != null) 
       { 
           proxy.Abort();
           throw new Exception("WCF call failed after 5 retries.", mostRecentEx );
       }

    }
}

Kesselring

I have the same question you do. Is that the approach you have used? And for calls that are gong to take a while (2-20 seconds) what pattern should be used there. – Magical

In general I've been using .NET 4 cancellation token to issue timeouts to my code, but haven't yet hooked it into WCF. MSDN has a good sample on cancellation tokens. – Kesselring

Worked perfect af correcting the line " if (mostRecentEx != null)" to if (success == false && mostRecentEx != null).. – Institute

I couldn't use T proxy =, since T doesn't have Close or Abort method. I had to stick with the IClientChannel proxy = as the code does in the post referenced. – Hallie

@makerofthings7, I think most of the catch statements are not required: ChannelTerminatedException, ChannelTerminatedException, ChannelTerminatedException - all descend from CommunicationException; just catching that you catch all of them. The only exception (a pun :) ) to this structure seems to be a System.ServiceModel.FaultException which is NOT a problem with channel, but a problem with target code on the other end throwing exception back into WCF channel. – Sandra

@Sandra Would the handler for CommunicationException always backoff, or would there be an exception to that ;)? E.g. "Prompt the user to make sure the server name is correct if not found (or start VPN)". Otherwise that makes sense. – Kesselring

@themakerofthings7, the order matters: in my understanding we have to catch FaultException first (as this is logic-related), then CommunicationException as a cause to retry, then TimeoutException, then Excpetion to catch all remaining. Unless I missed something else. – Sandra

@makerofthings7, one more thing: you reuse proxy in a loop after calling Abort() on it; did you see how it practically works? Both SO's responses and MS documentation says that once Close-d/Abort-ed it is no longer usable. – Sandra

@Sandra - You're right, I fixed that on my code, but it's good to update this SO link. I vaguely remember a reverse order working best for my purposes, but I'll +1 your thoughts on exception order in case others want the same route. – Kesselring

@port433 I finally got around to looking at this again... yes the order matters like you said. ServerTooBusyException ChannelTerminatedException, and EndpointNotFoundException must be above CommunicationException. Why? All the previous derive from communication exception, therefore CE must be last if you care to catch them. – Kesselring

Why is the ChannelFactory<T> not disposed of? Is this not necessary anymore or is it just not part of the example code? – Panda

We've seen one more WCF exception, ProtocolException. It occurs when the service on the other side for example stops negotiating HTTPS correctly, but otherwise connects. – Entablement

Any particular reason of "throw e" in the last catch block? – Sliding

Using Polly for retries ? – Caneghem

Great example! It was very inspiring to solve my problem :-) What if you had many methods returning a variety of things. For instance: "PlaceOrder" (returning a void), "GetOrder" (returning an int) and "GetAllOrders()" (returning an int[]). How would you do that? – Literate

Answer 2

I started a project on Codeplex that has the following features

• Allows efficient reuse of the client proxy
• Cleans up all resources, including EventHandlers
• Operates on Duplex channels
• Operates on Per-call services
• Supports config constructor, or by factory

http://smartwcfclient.codeplex.com/

It is a work in progress, and is very heavily commented. I'll appreciate any feedback regarding improving it.

Sample usage when in instance mode:

 var reusableSW = new LC.Utils.WCF.ServiceWrapper<IProcessDataDuplex>(channelFactory);

 reusableSW.Reuse(client =>
                      {
                          client.CheckIn(count.ToString());
                      });


 reusableSW.Dispose();

Kesselring

Answer 3

we have a WCF client that deal with almost any type of failure at the server. The Catch list is very long but does not have to be. If you look closely, you will see that many exceptions are child definitions of the Exception Class (and a few other classes).

Thus you can simplify things a lot if you want to. That said, here are some typical errors that we catch:

Server timeout
Server too busy
Server unavailable.

Rosarosabel

This is old, but if I recall correctly, we created our own exception classes in that project, that was a subclass of the exception class – Loverly

Sorry. no. This was over 6 years ago – Loverly

Answer 4

Below links may help to handle WCF Exceptions:

http://www.codeproject.com/KB/WCF/WCFErrorHandling.aspx

http://msdn.microsoft.com/en-us/library/cc949036.aspx

Shimberg

+1 - Great links. Can you assist (perhaps by editing your answer) with a text description of how I can use IExceptionToFaultConverter or the other classes to actually implement corrective WCF behavior for timeouts, etc? – Kesselring