How to use JWT tokens with WCF and WIF?

General notes

We're using IdentityServer3 and have been very happy with it so far.
We've gotten to secure MVC and ASP.NET Web API applications very easily with the help of both MS and Thinktecture OWIN middlewares.

The client we're working for still has a lot of SOAP WCF services, and this is where we're getting stuck.

The Setup

I'm not gonna lie, I'm far from being experienced with WCF, I've only used it for very basic scenarios - understand basicHttpBinding, no transport nor message security.

This is what I want to achieve:

A client gets a JWT access token from IdentityServer
Somehow the token ends up in the SOAP message headers
WCF reads and validates the token
WCF inspects the claims and performs authorization based on some criterion

I can't get the third step working.

The server setup

I'm using a ws2007FederationHttpBinding with TransportWithMessageCredential security mode. The message contains a BearerKey and the token is of type urn:ietf:params:oauth:token-type:jwt
The service uses the WIF identity pipeline, in which I added the JwtSecurityTokenHandler from the System.IdentityModel.Tokens.Jwt NuGet package

The client setup

The JWT token issued by the STS is wrapped in a BinarySecurityToken XML element, itself wrapped in a GenericXmlSecurityElement
This token is used as a parameter of the CreateChannelWithIssuedToken of the ChannelFactory

What happens

The token is found in the SOAP header and passed on to the JwtSecurityTokenHandler.
But then an exception is thrown:

System.ServiceModel.Security.MessageSecurityException: Message security verification failed. ---> System.IndexOutOfRangeException: Index was outside the bounds of the array.
   at System.Xml.XmlBufferReader.GetChars(Int32 offset, Int32 length, Char[] chars)
   at System.Xml.XmlBufferReader.GetString(Int32 offset, Int32 length)
   at System.Xml.StringHandle.GetString()
   at System.Xml.XmlBaseReader.ReadEndElement()
   at System.ServiceModel.Security.ReceiveSecurityHeader.ExecuteFullPass(XmlDictionaryReader reader)
   at System.ServiceModel.Security.ReceiveSecurityHeader.Process(TimeSpan timeout, ChannelBinding channelBinding, ExtendedProtectionPolicy extendedProtectionPolicy)
   at System.ServiceModel.Security.TransportSecurityProtocol.VerifyIncomingMessageCore(Message&amp;amp; message, TimeSpan timeout)
   at System.ServiceModel.Security.TransportSecurityProtocol.VerifyIncomingMessage(Message&amp;amp; message, TimeSpan timeout)
   --- End of inner exception stack trace ---

After JustDecompiling, it looks like there's an error when further reading the XML elements in the SOAP header. What's strange is that the token is the last element. Here's what the whole message looks like:

<s:Envelope xmlns:s="http://www.w3.org/2003/05/soap-envelope" xmlns:a="http://www.w3.org/2005/08/addressing" xmlns:u="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-utility-1.0.xsd">
    <s:Header>
        <a:Action s:mustUnderstand="1">http://tempuri.org/IService/GetListOfStrings</a:Action>
        <a:MessageID>urn:uuid:5c22d4e2-f9b8-451a-b4ca-a844f41f7231</a:MessageID>
        <ActivityId CorrelationId="554fc496-7c47-4063-9539-d25606f186b0" xmlns="http://schemas.microsoft.com/2004/09/ServiceModel/Diagnostics">1213dcd7-55b7-4153-8a6d-92e0922f76dd</ActivityId>
        <a:ReplyTo>
            <a:Address>http://www.w3.org/2005/08/addressing/anonymous</a:Address>
        </a:ReplyTo>
        <VsDebuggerCausalityData xmlns="http://schemas.microsoft.com/vstudio/diagnostics/servicemodelsink">uIDPo90CpMlUwLBOmEPkZ5C8fRQAAAAAVWkkf2rJS0qImBv+Yx1recUXdbBLjThDkAMkwfW3/2AACQAA</VsDebuggerCausalityData>
        <a:To s:mustUnderstand="1">https://localhost.fiddler:44322/Service.svc</a:To>
        <o:Security s:mustUnderstand="1" xmlns:o="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-secext-1.0.xsd">
            <u:Timestamp u:Id="_0">
                <u:Created>2015-05-21T06:41:45.362Z</u:Created>
                <u:Expires>2015-05-21T06:46:45.362Z</u:Expires>
            </u:Timestamp>
            <wsse:BinarySecurityToken ValueType="urn:ietf:params:oauth:token-type:jwt" EncodingType="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-soap-message-security-1.0#Base64Binary" xmlns:wsse="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-secext-1.0.xsd"><!-- Removed --></wsse:BinarySecurityToken>
        </o:Security>
    </s:Header>
    <s:Body>
        <GetListOfStrings xmlns="http://tempuri.org/" />
    </s:Body>
</s:Envelope>

Doesn't look like there's something malformed or anything. From the stack trace, the exception must be thrown when reading the </o:Security> end element since the token was properly read and handled.

Repro

I forked the samples repo so you can have a look if you feel like it. Here are the relevant projets:

SelfHost (Minimal) in the sources folder. This is the STS
In the Clients solution, the WCF service is in the APIs folder
In the Clients solution, the WCF client is the Console Client Credentials With Wcf project

Best way to fire it up is to start the STS first, then Right click -> Debug -> Start new instance on the WCF service, then the same on the WCF client.

Thanks in advance!