C#/IRS ACA - Sending a web service request with both MTOM attachment and GZip encoding using WCF 4.5

Question

We are trying to send data to IRS through an exposed web service for the ACA data transmission and we could not make the WSE 3.0 approach to work because of the order of Timestamp and Signature elements in the Security Header. XSD validation at the IRS side is showing an error when the TimeStamp element is appearing before Signature element. When we used policy assertion for manually generating the security header with the right order of Signature and Timestamp elements, IRS web service is showing "Invalid WS Security Header" Error.

Can anyone having the same problem, please let us know what could be a possible fix. Using WCF 4.5 instead of WSE 3.0 is the solution, can you please provide a working sample which handles both MTOM attachment & GZip encoding using WCF 4.5

Any help would be greatly appreciated.

EDIT: Following is the Status Request Soap Envelope we are sending now with the whole XML created by manually. It still shows TPE - 1122 Invalid Security Header Error. Are there any mistakes with this request?

<?xml version="1.0" encoding="utf-8"?>
<soapenv:Envelope xmlns:oas1="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-secext-1.0.xsd" xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/" xmlns:urn="urn:us:gov:treasury:irs:msg:irstransmitterstatusrequest" xmlns:urn1="urn:us:gov:treasury:irs:ext:aca:air:7.0" xmlns:urn2="urn:us:gov:treasury:irs:common" xmlns:urn3="urn:us:gov:treasury:irs:msg:acasecurityheader">
    <soapenv:Header xmlns:wsa="http://www.w3.org/2005/08/addressing">
        <wsse:Security xmlns:wsse="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-secext-1.0.xsd" xmlns:wsu="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-utility-1.0.xsd">
            <ds:Signature Id="SIG-E68EBBF1696C5DD4AA143353323390579" xmlns:ds="http://www.w3.org/2000/09/xmldsig#">
                <ds:SignedInfo>
                    <ds:CanonicalizationMethod Algorithm="http://www.w3.org/2001/10/xml-exc-c14n#WithComments" />
                    <ds:SignatureMethod Algorithm="http://www.w3.org/2000/09/xmldsig#rsa-sha1" />
                    <ds:Reference URI="#TS-82E7E6716E615C14D6144736030985954">
                        <ds:Transforms>
                            <ds:Transform Algorithm="http://www.w3.org/2001/10/xml-exc-c14n#">
                                <InclusiveNamespaces PrefixList="wsse wsa oas1 soapenv urn urn1 urn2 urn3" xmlns="http://www.w3.org/2001/10/xml-exc-c14n#" />
                            </ds:Transform>
                        </ds:Transforms>
                        <ds:DigestMethod Algorithm="http://www.w3.org/2000/09/xmldsig#sha1" />
                        <ds:DigestValue>[Digest_Value_Replaced]</ds:DigestValue>
                    </ds:Reference>
                    <ds:Reference URI="#id-82E7E6716E615C14D6144736030986559">
                        <ds:Transforms>
                            <ds:Transform Algorithm="http://www.w3.org/2001/10/xml-exc-c14n#">
                                <InclusiveNamespaces PrefixList="oas1 soapenv urn1 urn2 urn3" xmlns="http://www.w3.org/2001/10/xml-exc-c14n#" />
                            </ds:Transform>
                        </ds:Transforms>
                        <ds:DigestMethod Algorithm="http://www.w3.org/2000/09/xmldsig#sha1" />
                        <ds:DigestValue>[Digest_Value_Replaced]</ds:DigestValue>
                    </ds:Reference>
                    <ds:Reference URI="#id-82E7E6716E615C14D6144736030986558">
                        <ds:Transforms>
                            <ds:Transform Algorithm="http://www.w3.org/2001/10/xml-exc-c14n#">
                                <InclusiveNamespaces PrefixList="wsa oas1 soapenv urn1 urn2 urn3" xmlns="http://www.w3.org/2001/10/xml-exc-c14n#" />
                            </ds:Transform>
                        </ds:Transforms>
                        <ds:DigestMethod Algorithm="http://www.w3.org/2000/09/xmldsig#sha1" />
                        <ds:DigestValue>[Digest_Value_Replaced]</ds:DigestValue>
                    </ds:Reference>
                </ds:SignedInfo>
                <ds:SignatureValue>[Signature_Value_Replaced]</ds:SignatureValue>
                <ds:KeyInfo Id="KI-82E7E6716E615C14D6144736030986456">
                    <wsse:SecurityTokenReference wsu:Id="STR-82E7E6716E615C14D6144736030986457" xmlns:wsu="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-utility-1.0.xsd" xmlns:wsse="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-secext-1.0.xsd">
                        <wsse:KeyIdentifier EncodingType="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-soap-message-security-1.0#Base64Binary" ValueType="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-x509-token-profile-1.0#X509v3">[Cert_Value_Replaced]</wsse:KeyIdentifier>
                    </wsse:SecurityTokenReference>
                </ds:KeyInfo>
            </ds:Signature>
            <wsu:Timestamp wsu:Id="TS-82E7E6716E615C14D6144736030985954">
                <wsu:Created>2016-03-23T09:53:23:55Z</wsu:Created>
                <wsu:Expires>2016-03-23T10:03:23:55Z</wsu:Expires>
            </wsu:Timestamp>
        </wsse:Security>
        <urn:ACABusinessHeader wsu:Id="id-82E7E6716E615C14D6144736030986558" xmlns:wsu="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-utility-1.0.xsd">
            <urn1:UniqueTransmissionId>uuid:SYS12:tcc_cd::T</urn1:UniqueTransmissionId>
            <urn2:Timestamp>2016-03-23T09:53:23:55Z</urn2:Timestamp>
        </urn:ACABusinessHeader>
        <urn3:ACASecurityHeader/>
        <wsa:Action>RequestSubmissionStatusDetail</wsa:Action>
    </soapenv:Header>
    <soapenv:Body>
        <urn:ACABulkRequestTransmitterStatusDetailRequest version="1.0" wsu:Id="id-82E7E6716E615C14D6144736030986559" xmlns:wsu="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-utility-1.0.xsd">
            <urn1:ACABulkReqTrnsmtStsReqGrpDtl>
                <urn2:ReceiptId>[ReceiptId_Value_Replaced]</urn2:ReceiptId>
            </urn1:ACABulkReqTrnsmtStsReqGrpDtl>
        </urn:ACABulkRequestTransmitterStatusDetailRequest>
    </soapenv:Body>
</soapenv:Envelope>

EDIT 2:: Here is the method I am using to sign the envelope and create the Signature Element. STILL GETTING SECURITY HEADER ERROR... :(

 public static string getSignedXML(XmlDocument xmlDoc, RSACryptoServiceProvider key, string signatureNamespacePrefix,
                                        string sTimeStampId, string sManifestId, string sBusHeaderId)
        {
            xmlDoc.PreserveWhitespace = false;  //Ignore the whitespace in XML   

            SignedXml signedXml = new CustomIdSignedXml(xmlDoc); //If Id attribute needs to have a prefix. This is not needed as per latest Reference Guide Info.
            //SignedXml signedXml = new SignedXml(xmlDoc);

            // Add the key to the SignedXml document.
            signedXml.SigningKey = key;
            signedXml.SignedInfo.SignatureMethod = SignedXml.XmlDsigRSASHA1Url;
            signedXml.SignedInfo.CanonicalizationMethod = SignedXml.XmlDsigExcC14NWithCommentsTransformUrl;
            signedXml.SignedInfo.SignatureMethod = SignedXml.XmlDsigRSASHA1Url;
            signedXml.Signature.Id = "SIG-E68EBBF1696C5DD4AA143353323390579";

            //------------------------------------------------------------------------
            //START OF: Adding Manifest, BusinessHeader and TimeStamp References
            //------------------------------------------------------------------------
            //Adding Timestamp Reference
            XmlDsigExcC14NTransform timeStampTransform = new XmlDsigExcC14NTransform();
            timeStampTransform.InclusiveNamespacesPrefixList = "wsse wsa oas1 soapenv urn urn1 urn2 urn3";
            Reference reference = new Reference("#" + sTimeStampId);
            reference.AddTransform(timeStampTransform);
            // Add the reference to the SignedXml object.
            signedXml.AddReference(reference);

            //Adding Business Header Reference
            XmlDsigExcC14NTransform busHeaderTransform = new XmlDsigExcC14NTransform();
            busHeaderTransform.InclusiveNamespacesPrefixList = "wsa oas1 soapenv urn1 urn2 urn3";
            reference = new Reference("#" + sBusHeaderId);
            reference.AddTransform(busHeaderTransform);
            // Add the reference to the SignedXml object.
            signedXml.AddReference(reference);

            //Adding Manifest Request Dtl Reference
            XmlDsigExcC14NTransform manifestTransform = new XmlDsigExcC14NTransform();
            manifestTransform.InclusiveNamespacesPrefixList = "oas1 soapenv urn1 urn2 urn3";
            reference = new Reference("#" + sManifestId);
            reference.AddTransform(manifestTransform);
            // Add the reference to the SignedXml object.
            signedXml.AddReference(reference);
            //------------------------------------------------------------------------
            //END OF: Adding Manifest, BusinessHeader and TimeStamp References
            //------------------------------------------------------------------------

            signedXml.ComputeSignature();

            XmlElement xmlSignature = signedXml.GetXml(); //Get the singed XML block

            if (!string.IsNullOrEmpty(signatureNamespacePrefix))
            {
                //Here we set the namespace prefix on the signature element and all child elements to "ds", invalidating the signature.
                AssignNameSpacePrefixToElementTree(xmlSignature, "ds");

                //So let's recompute the SignatureValue based on our new SignatureInfo...

                //For XPath
                XmlNamespaceManager namespaceManager = new XmlNamespaceManager(xmlDoc.NameTable);
                namespaceManager.AddNamespace("ds", "http://www.w3.org/2000/09/xmldsig#"); //this prefix is arbitrary and used only for XPath
                XmlElement xmlSignedInfo = xmlSignature.SelectSingleNode("ds:SignedInfo", namespaceManager) as XmlElement;

                //Canonicalize the SignedInfo element
                XmlDsigC14NTransform transform = new XmlDsigC14NTransform();
                XmlDocument signedInfoDoc = new XmlDocument();
                signedInfoDoc.LoadXml(xmlSignedInfo.OuterXml);
                transform.LoadInput(signedInfoDoc);

                //Compute the new SignatureValue
                string signatureValue = Convert.ToBase64String(key.SignData(transform.GetOutput() as MemoryStream, new SHA1CryptoServiceProvider()));
                //Set it in the xml
                XmlElement xmlSignatureValue = xmlSignature.SelectSingleNode("ds:SignatureValue", namespaceManager) as XmlElement;
                xmlSignatureValue.InnerText = signatureValue;
            }

            //xmlDoc.DocumentElement.AppendChild(xmlDoc.ImportNode(xmlSignature, true));

            //----------------------------------------------------------------------------------
            //START OF: Add Key Info Element also to the XML after SignatureValue Node.
            //----------------------------------------------------------------------------------
            X509Certificate2 mycert = new X509Certificate2(<Cert_Path>, <Cert_Password>);
            //bulkReqTransmitService.ClientCertificates.Add(mycert);
            var exported = mycert.Export(X509ContentType.Cert, <Cert_Password>);
            var base64 = Convert.ToBase64String(exported);

            StringBuilder sbKeyInfo = new StringBuilder();
            string dsStartTagPrefix = "";
            string dsEndTagPrefix = "/";
            if (!string.IsNullOrEmpty(signatureNamespacePrefix))
            {
                dsStartTagPrefix = "ds:";
                dsEndTagPrefix = "/ds:";
            }
            sbKeyInfo.Append("<root xmlns:wsse=\"http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-secext-1.0.xsd\" xmlns:wsu=\"http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-utility-1.0.xsd\" xmlns:ds=\"http://www.w3.org/2000/09/xmldsig#\">");
            sbKeyInfo.Append("<" + dsStartTagPrefix + "KeyInfo Id=\"KI-82E7E6716E615C14D6144736030986456\">");
            sbKeyInfo.Append("<wsse:SecurityTokenReference wsu:Id=\"STR-82E7E6716E615C14D6144736030986457\">");
            sbKeyInfo.Append("<wsse:KeyIdentifier EncodingType=\"http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-soap-message-security-1.0#Base64Binary\" ValueType=\"http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-x509-token-profile-1.0#X509v3\">" + base64.ToString());
            sbKeyInfo.Append("</wsse:KeyIdentifier>");
            sbKeyInfo.Append("</wsse:SecurityTokenReference>");
            sbKeyInfo.Append("<" + dsEndTagPrefix + "KeyInfo>");
            sbKeyInfo.Append("</root>");

            XmlDocument tempDoc = new XmlDocument();
            tempDoc.LoadXml(sbKeyInfo.ToString());
            XmlNode oNode = tempDoc.DocumentElement;
            //necessary for crossing XmlDocument contexts
            XmlNode importNode = xmlSignature.OwnerDocument.ImportNode(oNode.FirstChild, true);
            xmlSignature.AppendChild(importNode);
            //----------------------------------------------------------------------------------
            //END OF: Add Key Info Element also to the XML after SignatureValue Node.
            //----------------------------------------------------------------------------------

            return xmlSignature.OuterXml;
        }


public class CustomIdSignedXml : SignedXml
    {
        public CustomIdSignedXml(XmlDocument xml) : base(xml)
        {
        }

        public CustomIdSignedXml(XmlElement xmlElement)
            : base(xmlElement)
        {
        }

        public override XmlElement GetIdElement(XmlDocument doc, string id)
        {
            // check to see if it's a standard ID reference
            XmlElement idElem = base.GetIdElement(doc, id);

            if (idElem == null)
            {
                XmlNamespaceManager nsManager = new XmlNamespaceManager(doc.NameTable);
                nsManager.AddNamespace("wsu", "http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-utility-1.0.xsd");

                //idElem = doc.SelectSingleNode("//*[@p5:Id=\"" + id + "\"]", nsManager) as XmlElement;

                string xml = doc.SelectSingleNode("//*[@wsu:Id=\"" + id + "\"]", nsManager).OuterXml;
                XmlDocument tempDoc = new XmlDocument();
                tempDoc.LoadXml(xml);
                XmlElement xEle = tempDoc.DocumentElement;

                idElem = xEle;
            }

            return idElem;
        }
    }

Spotless

I got the gzip working from a combination of this: msdn.microsoft.com/en-us/library/ms751458.aspx (Custom Message Encoder: Compression Encoder). But it required tweaks. Had to change the content type to show up as "text/xml" instead of "application/x-gzip" as they have set. Then had to set the "Content-Encoding" header manually using this: blogs.msdn.com/b/wsdevsol/archive/2014/02/07/… to the value "gzip" – Gorki

I am now trying to set up the signature part. – Gorki

Won't be able to verify MTOM is working properly until I'm past that, but right now I've got it set up as described here: #3639052 – Gorki

Answer 1

EDIT: Verified none of this will work due to WCF's messaging architecture. IRS structure requires modification of HTTP headers for MTOM to work properly which cannot be modified in WCF after the point during which XML serialization occurs. You will have to construct your XML manually and send it via HttpClient and do GZip and MTOM encoding yourself. Sorry for the bad news :(

WCF gzip: Take the sample here and integrate the classes and client config with your app. https://msdn.microsoft.com/en-us/library/ms751458.aspx

You'll have to tweak the content type to be "text/xml" instead of "application/x-gzip".

This still will be rejected by the IRS because this doesn't set the "Content-Encoding" header. You can set the content encoding this way:

using (new OperationContextScope(transmitterClient.InnerChannel))
{
// Add a HTTP Header to an outgoing request
HttpRequestMessageProperty requestMessage = new HttpRequestMessageProperty();
requestMessage.Headers["Content-Encoding"] = "gzip";
OperationContext.Current.OutgoingMessageProperties[HttpRequestMessageProperty.Name] = requestMessage;
return port.BulkRequestTransmitter(request);
}

Thanks to this blog post for that: http://blogs.msdn.com/b/wsdevsol/archive/2014/02/07/adding-custom-messageheader-and-http-header-to-a-wcf-method-call.aspx

Additionally, IRS's response is NOT gzipped, so you need to modify the class to not try to decompress the response.

Additionally, additionally, the innerMessageEncoding messageVersion default is NOT compatible with the IRS. You have to modify the ApplyConfiguration method of the GZipMessageEncodingElement to instantiate the TextMessageEncodingBindingElement to the MessageVersion.Soap11WSAddressing10 enum with Encoding.UTF8

MTOM My bindings look like this:

<bindings>
  <customBinding>
    <binding name="BulkRequestTransmitterBinding">
      <gzipMessageEncoding innerMessageEncoding="textMessageEncoding" />
      <security enableUnsecuredResponse="true" authenticationMode="MutualCertificate" messageSecurityVersion="WSSecurity10WSTrust13WSSecureConversation13WSSecurityPolicy12BasicSecurityProfile10"/>
      <httpsTransport />
    </binding>
  </customBinding>
</bindings>  

And then a custom behavior aiming at my cert.

Gorki

Signing is still broken in my solution though. So take this for what it is (or isn't) worth. – Gorki

Thanks for the reply Bon. We already tried GZip custom encoding classes provided by Microsoft sample. But worried to use that in production since many blogs had mentioned that the custom Gzip encoder classes has the memory leakage issue. Is there any other simpler solution with WCF to incorporate GZip encoding and sending the pay load file as MTOM? – Spotless

Not as far as I'm aware. I'm sure the memory leak issues could be debugged once they were observable and repeatable, though. I'm planning on using this solution in a console app fired on a scheduler to pick the next in a queue to send. So I'd expect that would release any of the unreleased memory inbetween executions. – Gorki

Thanks for the reply Bon. So were you able to create a working .NET solution for IRS A2A transmission? – Spotless

No, and I am pretty certain it is impossible when using WCF or Web Services client due to their strict namespace declaration and the fact that neither of those tools allow modification of the SOAP envelope, nor the declaration of multiple prefixed namespaces via object and property attributes. I am currently finishing my next pass using LINQ to Xml to construct the envelope and most headers, and will submit the SOAP request using HTTP POST manually within the next day or two. – Gorki

Oh OK. Still it is an automated A2A solution right. We are also using LINQ to XML for creating our Form Data file. Were you able to get all the security elements in the security header with the manual XML Node creation approach and if it (HTTP POST) works could you post the working sample here which would be of great help. Currently we are doing transmissions using the UI channel as an alternate approach which is the manual process of uploading the Manifest file and Form Data file. – Spotless

Nothing is working yet, but it will be automated once it is done. But I can't post the code unfortunately due to contractual obligations. I'll share what I can. – Gorki

Sure not a problem. Thank you. – Spotless

I got it working, SubCoder. For signing, only 1 part needs to be touched. The SignedXml GetXml output must be modified somewhat to use a BinarySecurityToken for the SignedInfo element. If you use LINQ to Xml to modify only that element and don't modify the 3 signed elements themselves, the digests remain in tact. – Gorki

Bon, Thanks for letting us know. If you could give some pointers of the whole approach, we will try it on our end. We are really stuck with the A2A transmissions at this point. – Spotless

Bon, Also how are you adding the MTOM payload file to the request if you are manually creating the XML file by-hand? – Spotless

Bon, Also how are you doing GZip Compression for the manually created request? – Spotless

Both MTOM and GZip happen after XML construction, before I assign the content to the post body of my pending HTTP Request. First MTOM, which is just done by string concatenation then gzip using the standard GZipStream class in .Net – Gorki

Cool Thanks. Could you share code snippets for the Security Header, GZip and MTOM. – Spotless

Sorry, not able to do that due to contractual requirements. There are no gotchas in this remaining part though, so standard documentation available on those features should get you this last bit to completion. – Gorki

Bon, Not a problem. Where can I find an example of gzip using GZipStream class in .net? Also how will I get the digest values manually for the three sections. – Spotless

dotnetperls.com/gzipstream was second result in Google. Digest values are calculated by using the SignedXml class and providing it with the Id attribute values for the 3 elements requiring signature. – Gorki

Thanks a ton Bon. One more question. How did you calculate the "SignatureValue"? And how could we make sure that the Signature Value Calculated is right? – Spotless

The SignedInfo class does it automatically when you call GetXml from the references you identified. There's a function in the SignedInfo class that runs the hash of the signed elements against the certificate it contains to verify they match. msdn.microsoft.com/en-us/library/… – Gorki

@Spotless have a look at the following SO posts. This SO explains how I am signing my signatures. This SO goes over adding the "ds" prefix to the Signature elements (something that may or may not be necessary). This SO, I am stuck on and receiving a error from. – Astilbe

I just answered your question there, I believe. Talk to you in that thread. – Gorki

Thanks a lot Bon and Russ. That was very good information and we are in the process of trying it out now. – Spotless

@Bon: After hand-writing the whole XML, we are past the GZip and forming the Security Header portion of it. Trying the Status Request to AATS system which is the simpler one. My Soap Envelope XML matches exactly with the one in IRS reference guide. But still getting TPE1122 - Security Header Error. I am posting it here. Could you please let me know if there is any glaring mistake I am doing? Please find an "EDIT" to my original post above with the soap envelope XML – Spotless

You are sure you aren't modifying the SignedInfo element or any of its descendants in the XML coming out of the GetXml function from the SignedXml class? It has been verified that the IRS will accept the SignedInfo (and entire Signature element) without the prefix and just the namespace inheritance override. My successful transmissions have namespace structure matching their docs exactly EXCEPT in the Signature element, and it works fine. – Gorki

@Bon: I am modifying and adding "ds" prefixes and recalculating the Signature value. But just tried the transmission without "ds" prefix also and same Security Header Error. I have posted the code doing the signing also in my post "EDIT 2" above. Please have a look. How does your Signature Element look? Could you post a snippet here? – Spotless

When you tried without modifying the ds prefix, did you use the SignatureValue generated from the SignedXml class, or did you calculate it yourself? In my solution I am not recalculating it. I am just grabbing the generated value from that Xml and replacing it with a structure matching how you've done to match their documentation. – Gorki

And just another thing to make sure, you're using a certificate uploaded to the AE AIR TCC integration page and your submission references the proper corresponding ASID? – Gorki

@Bon: Yes. I am also using the SignatureValue created by signedXML.ComputeSignature() method. Please refer my code above with "EDIT 2" in my original question. We just have one certificate which we uploaded and using the same in our application also. What is ASID? – Spotless

@Spotless 2 things that I notice that differ between yours and mine; which does not work either. 1.) You have the <urn3:ACASecurityHeader> element, which is not needed. This is needed only for UI submission. 2.) The order of your <References> element is different from mine: Timestamp, Header, Detail vs. (yours) Timestamp, Detail, Header. I'm not sure if this matters or not. Beyond that, I don't have the osa1 prefix defined in the Envelope, although it is defined in my InclusiveNamespaces.PrefixList? – Astilbe

@Spotless In your Signing: 1.) You have signedXml.SignedInfo.SignatureMethod defined twice. I don't have this defined at all, but the SignatureMethod is still generated correctly. 2.) I set idElem in the method you have commented out; and I don't have the other code you added. 3.) The way you're creating the replacement KeyInfo element, we are doing differently as well. – Astilbe

@Russ: 1) <urn3:ACASecurityHeader> element is available in the reference guide. Please refer Page #46 of latest reference guid. 2) I even tried the order you mentioned Timestamp, Header and Detail which does not make a difference. 3) oas1 is defined in InclusiveNameSpace prefix list only in my envelope too 4) signedXml.SignedInfo.SignatureMethod declaring twice is removed and it does not make a difference. 5) I am not sure which line you were referring to for idElem. – Spotless

@Russ: How can I make sure that my SignatureValue is correct and KeyInfo KeyIdentifier value is correct? – Spotless

@Spotless I think we have to trust the code is generating the SignatureValue appropriately. I was referring to the idElem in the GetIdElement method of CustomIdSignedXml class. See Table 5-4 of page 45 For a clearer definition of what the ACASecurityHeader. I think you and I are actually having the same problems. I'm just trying to convey what I know as to be correct. – Astilbe

@Russ: Removed ACASecurityHeader and same security header error. Also idElem doe not make any difference either. Please let me know if you find anything positive. And I am out of clue now as to what more to try... :( – Spotless

@Russ: For your information, couple months back when we were trying WSE3.0 to make this work, we got the status request working. But problem we had was, the response which was coming back as MTOM could not be handled by WSE3.0 classes. That is why we started researching other options. That time the issue was the correct certificate file was not uploaded to the IRS TCC Setup website. So check your certificate file which is uploaded to IRS website. – Spotless

I have a strong feeling that it is because of the SignatureValue of the Security Header. Not sure how to validate that though. :( – Spotless

@Spotless @Bon: Can the CheckSignature method of the SignedXml class return false positives? I'm using that to validate my signature, but maybe you're running into a validation issue on the IRS side? – Astilbe

I've never heard of such a false positive. It is a straight forward comparison of cryptographic hashes, essentially just a byte-by-byte comparison. – Gorki

@Gorki What part of the Certificate is required for us to send as the value of the KeyIdentifier element? I'm programmatically retrieving the certificate through the X509Store and X509CertificateCollection objects. Basically, is there a specific X509Certificate2 property I should be using? – Astilbe

Yeah but you need to make sure you're loading in the Microsoft.Web.Services3 namespace because that is where the BinarySecurityToken type comes from. In this next snippet, _cert is an X509Certificate2 object. BinarySecurityToken bst = new X509SecurityToken(_cert); var keyIdentifierValue = Convert.ToBase64String(bst.RawData); – Gorki

I was using the following approach for calculating the base64 value from the certificate which is from System.Security.Cryptography.X509Certificates.X509Certificate2 and this gives exact same value as the BinarySecurityToken from Microsoft.Web.Services3. Following is my code snippet. Still getting the security header error. X509Certificate2 mycert = new X509Certificate2(Cert_Path, Cert_Password); var exported = mycert.Export(X509ContentType.Cert, Cert_Pass); var base64 = Convert.ToBase64String(exported); – Spotless

@Astilbe Any more progress on this that made this work? – Spotless

@Spotless Unfortunately, no. I am using the same snippet of code Bon posted to retrieve and use my Certificate. I am still receiving a message was not formatted properly and/or cannot be interpreted error against the Transmitter Service; and a WS Security Header invalid error against the Status Service. I am truly stumped here. – Astilbe

Here's another straw grasp attempt. Do I need to add the X509Certificate2 object to the HttpWebRequest object (through request.ClientCertificates.Add method)? At the moment, I'm only using the certificate to create the signing in the SOAP Header. – Astilbe

No. There is no http level authentication. – Gorki

@Gorki Is the HttpWebRequest object the correct object to be using to submit the requests? – Astilbe

@Gorki I have a question about how you used a custom behavior to pull in your cert. I have that part of it working, but did you also have to specify the <serviceCertificate>? If so, how did you know what values to use? Or is it possible to tell the application to not check for the serverCertificate? And thanks to all you guys for the comments -- it's been very helpful! – Nowt

I abandoned WCF for this because I don't think it is possible to get it work with the required combination of MTOM, custom headers and outbound gzip. Sorry I don't think I can answer your question. – Gorki

@Gorki Oh, I misunderstood your original answer then. So that "edit" paragraph that you have means that the rest of the answer is no longer valid? I think I was reading it as though the entire thing was an edited response. I've started going down the manual route per your advice. – Nowt

@Astilbe (and SubCoder) did either of you guys ever get yours to work? I've been reworking my project for the last several days and feel like I've correctly implemented everything manually, but I'm still getting TPE1105. Losing my mind over this thing :( – Nowt

@Nate: No. Not working yet. I re-wrote the status request part alone manually to see if that works and I am still getting "WS Security Header Invalid (TPE1122)". I wanted to make that work and then try the bigger request with MTOM. If any of you get that working please let me know too. – Spotless

@Nowt (and SubCoder): I have not gotten either request to work yet. I have some certificate things to look into on Monday (when the person responsible for them gets back); but so far I have one solution which uses WCF for the Submission Request and manual SOAP XML (through HttpWebRequest) for the Status Request. The Submission is returning a WS Security error, and the Status is coming back and not being well-formed. I have another solution which does the Submission and Status both as Manual SOAP XML (through HttpWebRequest) and are returning the opposite errors on those. No progress :( – Astilbe

@Nowt Trust me when I say I feel your pain in RE: to losing your mind over this. – Astilbe

We are currently using UI Channel for transmissions in production. No luck with the A2A yet. – Spotless

Guys, Any updates or progress on the A2A transmissions? – Spotless

@Russ: Did you find a solution for making A2A approach work with a .NET application? – Spotless

@Spotless yes, we are successfully transmitting to the IRS through the A2A via a .NET application. Please see #36021540. I believe that was the last post which took me down the correct path. – Astilbe

@Russ: Glad to know that it worked for you. I will have a look at the thread you mentioned. Thank you. – Spotless

Answer 2

I do not yet have the reputation to actually post a comment on your posts >:(

Here are a couple of follow-up questions for you regarding the process you are using.

1. Are you using the WSDL Service Reference/Proxy classes to make your calls to the IRS through the ISS-A2A Channel?
2. After implementing the three class files from the samples, I am receiving the following error. Do you see anything out of the ordinary that I am doing incorrectly?

Additional information: The type '[Project].GZipEncoder.GzipMessageEncodingElement, GZipEncoder, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null' registered for extension 'gzipMessageEncoding' could not be loaded.

The <gzipMessageEncoding> element in my <customBinding> is giving me a warning which I think is related to the above error.

app.config

<system.serviceModel>
  <extensions>
    <bindingElementExtensions>
      <add name="gzipMessageEncoding" type="[ProjectName].GZipEncoder.GzipMessageEncodingElement, GZipEncoder, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null" />
    </bindingElementExtensions>
  </extensions>
  <client>
    <endpoint />
    <metadata>
      <policyImporters>
        <extension type="[ProjectName].GZipEncoder.GZipMessageEncodingBindingElementImporter, GZipEncoder, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null" />
      </policyImporters>
    </metadata>
  </client>
  <bindings>
    <customBinding>
      <binding name="BulkRequestTransmitterBinding">
        <gzipMessageEncoding innerMessageEncoding="textMessageEncoding" />
        <httpsTransport />
      </binding>
    </customBinding>
  </bindings>
</system.serviceModel>

Astilbe

Hi Russ, We using WSDL Service Reference/Proxy classes to make our calls to the IRS through the ISS-A2A Channel. We tried GZip custom encoder classes, but backed off from that approach since some of our online research show that it could cause memory leaks when large requests are processed. – Spotless

@Subhash, Thank you for the response. If not using the GZip custom encoder classes, what approach have you ended up using in order to compress your request to the IRS? – Astilbe

Additionally, how large were the requests you were testing which were causing the memory leaks? – Astilbe

Russ, We tried WSE 3.0 which worked for the most part for us except for the one issue with TImestamp element order is not correct in the security header. We are still stuck with the Security Header Error issue. We did not try GZip custom encoder for large files. We used only AATS test scenarios. – Spotless

@Astilbe did this specific issue ever have a solution in your config file? – Glantz

@Glantz Unfortunately, I don't recall the issue I was having above. My code is currently working without any errors. In reviewing at the app.config file, I do have a warning on the gzipMessageEncoding element saying it has an invalid child element; however, it is a warning and not an error, and it doesn't seem to be causing any issues. – Astilbe