IRS-A2A BulkRequestTransmitter message not formmatted properly and/or cannot be interpreted

Asked 15/3, 2016 at 20:3 Answered 17/6, 2016 at 20:26

I am receiving the following error when attempting to submit through the BulkRequestTransmitter Web Service. The Composition Guide is less than helpful as far as this message goes, and when I compare my SOAP XML with the SOAP from the Composition Guide, they seem to be apples-to-apples. I'm hoping that another set of eyes may be able to see where the problem is.

The message was not formatted properly and/or cannot be interpreted. Please review the XML standards outlined in Section 3 of the AIR Submission Composition and Reference Guide located at https://www.irs.gov/for-Tax-Pros/Software-Developers/Information-Returns/Affordable-Care-Act-Information-Return-AIR-Program, correct any issues, and try again.

What I've Tried:

Attempted to submit with (and without) whitespace in the SOAP Envelope.
Attempted to submit with the Form Data XML in XML format.
~~Attempted to submit with the Form Data in base64string format (as this submission was).~~
~~Added the ds prefix to the Signature elements. Used this SO post in order to add the prefix to the Signature elements.~~
Added the Form Data in "Pretty Print" format and as according to the updated Composition Guide (v4.2).
Copied the formatting of the MIME for the BulkTransmitterService request outlined in section 10.3 of the Composition Guide.
Created two solutions: 1.) Manually creating the XML necessary for the SOAP requests and sending via HttpWebRequest object; 2.) Sending a submission request via the WSDL imported to the project as a Service Reference, using custom encoders for GZip and Mtom Encoding and manually creating the XML necessary for the SOAP Status Request (sent via HttpWebRequest).

Update #1
Updated the request based on some new additions.

~~Added the ds prefix to the Signature elements.~~
Added the Form Data in "Pretty Print" format and as according to the updated Composition Guide (v4.2: Section 5.4.2).

Update #2
I began to manually create the SOAP .xml file within a new instance of Visual Studio importing the schema references as necessary. I'm doing this outside of any sort of application creation.

In doing so, I was able to find some additional bugs in the SOAP I was creating through my application (thank you for intellisense!). The bugs that I found were within the Manifest XML, as they didn't conform to the IRS schema.

I will be looking into these in the next 24 hours and update accordingly.

The urn:MailingAddressGrp should have a child of either urn:USAddressGrp or urn:ForeignAddressGrp. That child should then contain the proper address elements. My code is currently missing the direct child of the urn:MailingAddressGrp.
~~The value for urn1:DocumentSystemFileNm of Form1094C_Request_[TCC]_yyyyMMddThhmmssfffZ.xml is incorrect. I'm not entirely sure what it should be just yet.~~
~~The urn1:BulkExchangeFile element, is having an issue related to the xop:Include element I have within. The schema wants a base64Binary type.~~

Update #2.5

Updated my XML generation process to include the USAddressGrp element.
Discovered that I had one extra character in the milliseconds (four instead of three). Once I corrected this, along with removing the string "Form" from the beginning of the file name, the value for the urn1:DocumentSystemFileNm was able to validate against the schema successfully.

Update #3

Updated the Full Request based on the updates I have made. At this point, I am unable to deduce what is wrong with my request. If anyone sees anything glaring, please help!

Update #4

Updated the Full Request based on additional updates made. Removed the ds prefix from the Signature based on another SO user's feedback. This user has gotten these requests to work without having to append the ds prefix to the Signature after the fact and re-compute the signature.

The SO user also confirmed that his requests are working with an <inc:Include> element being setup as a child element of the <BulkExchangeFile> element.
Confirmed the MIME headers are correct as per the sample in section 10.3 of the Composition Guide.

Update #5

I currently have two solutions: one which is sending manually creating the XML necessary for the SOAP requests and sending via HttpWebRequest; and one which is using the WSDL Service Reference for the Submission Request, using the custom encoders outlined below, and manually creating the XML necessary for the SOAP Request of the Status.

As of this update, Solution 1 gives me the error above when making a Submission Request, and gives me the error below when making the Status Request. However, when using Solution 2, both requests (Submission and Status) give me the error below.

I am looking into possible certificate issues to see if they make any progress with either of these solutions.

Update #6

There were a number of issues I ran into which caused me to be delayed. I'll spare you the nitty-gritty details, however, the short of it is that we did not have the Security Certificate registered with the IRS system, nor did we have the Certificate installed properly so that I could access the information through the X509Store. Finally these things got done, and I was able to test submitting data to the IRS from the server (vs. my localmachine which did not have the proper certificate). Unfortunately, I am still receiving the WS-Security error detailed below. I have updated the Full Request with what I am currently sending.

An Error Occurred with message: The WS Security Header in the message is invalid. Please review the transmission instructions outlined in Section 5 of the AIR Submission Composition and Reference Guide located at https://www.irs.gov/for-Tax-Pros/Software-Developers/Information-Returns/Affordable-Care-Act-Information-Return-AIR-Program, correct any issues, and try again.

All line breaks in the MIME headers are as-is, and I believe the line breaks are what is expected. The FormData attachment is being sent as Pretty Print while the SOAP Envelope is not; The SOAP Envelope in this post is formatted for readability.

UPDATE #7:

Thanks to users: jstill and fatherOfWine with what they have posted below, and to Bon for earlier assistance on this project. I have broken through one wall in getting the Submission to work. It is now working. The Status request is also working. However, I need to figure out how to process it in order to pull the status and the attachment (error data file) out of it.

Full Request:

Content-Encoding: gzip
Accept-Encoding: gzip, deflate
Content-Type: multipart/related; type="application/xop+xml"; start="<rootpart>"; start-info="text/xml"; boundary="MIME_boundary"
SOAPAction: BulkRequestTransmitter
MIME-Version: 1.0
Host: la.www4.irs.gov

--MIME_Boundary
Content-Type: application/xop+xml; charset=UTF-8; type="text/xml"
Content-Transfer-Encoding: 8bit
Content-Id: <root_part>

<s:Envelope xmlns:s="http://schemas.xmlsoap.org/soap/envelope/">
    <s:Header>
        <Security xmlns:h="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-secext-1.0.xsd" xmlns="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-secext-1.0.xsd" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xsd="http://www.w3.org/2001/XMLSchema">
            <Signature Id="SIG-E77c57b78ebc54e989bfc9e43604a04a4" xmlns="http://www.w3.org/2000/09/xmldsig#">
                <SignedInfo>
                    <CanonicalizationMethod Algorithm="http://www.w3.org/2001/10/xml-exc-c14n#WithComments" />
                    <SignatureMethod Algorithm="http://www.w3.org/2000/09/xmldsig#rsa-sha1" />
                    <Reference URI="#TS-Eb4799bee41bb4df0a72f52832d283ef7">
                        <DigestMethod Algorithm="http://www.w3.org/2000/09/xmldsig#sha1" />
                        <DigestValue>[TimestampDigestValue]</DigestValue>
                    </Reference>
                    <Reference URI="#id-E5f1ed32aab8f4578adeee5debd851a62">
                        <DigestMethod Algorithm="http://www.w3.org/2000/09/xmldsig#sha1" />
                        <DigestValue>[ACABusinessHeaderDigestValue]</DigestValue>
                    </Reference>
                    <Reference URI="#id-E4a71164001994d7f865fc7ddb8055350">
                        <DigestMethod Algorithm="http://www.w3.org/2000/09/xmldsig#sha1" />
                        <DigestValue>[ManifestDigestValue]</DigestValue>
                    </Reference>
                </SignedInfo>
                <SignatureValue>[SignatureValue]</SignatureValue>
                <KeyInfo Id="KI-E2309cb142e1a4076a2e71373e6e6b75f">
                    <SecurityTokenReference d6p1:Id="STR-E2751169ee468470290fe5e8bfb34589e" xmlns:d6p1="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-utility-1.0.xsd" xmlns="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-secext-1.0.xsd">
                        <KeyIdentifier EncodingType="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-soap-message-security-1.0#Base64Binary" ValueType="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-x509-token-profile-1.0#X509v3">[KeyIdentifier]</KeyIdentifier>
                    </SecurityTokenReference>
                </KeyInfo>
            </Signature>
            <a:Timestamp a:Id="TS-Eb4799bee41bb4df0a72f52832d283ef7" xmlns="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-utility-1.0.xsd" xmlns:a="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-utility-1.0.xsd">
                <a:Created>2016-05-18T09:51:05.856Z</a:Created>
                <a:Expires>2016-05-18T10:01:05.856Z</a:Expires>
            </a:Timestamp>
        </Security>
        <ACATransmitterManifestReqDtl a:Id="id-E4a71164001994d7f865fc7ddb8055350" xmlns:h="urn:us:gov:treasury:irs:ext:aca:air:7.0" xmlns:a="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-utility-1.0.xsd" xmlns="urn:us:gov:treasury:irs:ext:aca:air:7.0">
            <PaymentYr>2015</PaymentYr>
            <PriorYearDataInd>0</PriorYearDataInd>
            <EIN xmlns="urn:us:gov:treasury:irs:common">000000301</EIN>
            <TransmissionTypeCd>O</TransmissionTypeCd>
            <TestFileCd>T</TestFileCd>
            <OriginalReceiptId />
            <TransmitterNameGrp>
                <BusinessNameLine1Txt />
            </TransmitterNameGrp>
            <CompanyInformationGrp>
                <CompanyNm>Selitestthree</CompanyNm>
                <MailingAddressGrp>
                    <USAddressGrp>
                        <AddressLine1Txt>6689 Willow Court</AddressLine1Txt>
                        <CityNm xmlns="urn:us:gov:treasury:irs:common">Beverly Hills</CityNm>
                        <USStateCd>CA</USStateCd>
                        <USZIPCd xmlns="urn:us:gov:treasury:irs:common">90211</USZIPCd>
                    </USAddressGrp>
                </MailingAddressGrp>
                <ContactNameGrp>
                    <PersonFirstNm>Rose</PersonFirstNm>
                    <PersonLastNm>Lincoln</PersonLastNm>
                </ContactNameGrp>
                <ContactPhoneNum>5559876543</ContactPhoneNum>
            </CompanyInformationGrp>
            <VendorInformationGrp>
                <VendorCd>I</VendorCd>
                <ContactNameGrp>
                    <PersonFirstNm>ContactFirstName</PersonFirstNm>
                    <PersonLastNm>ContactLastName</PersonLastNm>
                </ContactNameGrp>
                <ContactPhoneNum>ContactPhoneNumber</ContactPhoneNum>
            </VendorInformationGrp>
            <TotalPayeeRecordCnt>3</TotalPayeeRecordCnt>
            <TotalPayerRecordCnt>1</TotalPayerRecordCnt>
            <SoftwareId>PPACA</SoftwareId>
            <FormTypeCd>1094/1095C</FormTypeCd>
            <BinaryFormatCd xmlns="urn:us:gov:treasury:irs:common">application/xml</BinaryFormatCd>
            <ChecksumAugmentationNum xmlns="urn:us:gov:treasury:irs:common">6b2512ce28f603f76261923d297738e5</ChecksumAugmentationNum>
            <AttachmentByteSizeNum xmlns="urn:us:gov:treasury:irs:common">14076</AttachmentByteSizeNum>
            <DocumentSystemFileNm>1094C_Request_[TCC]_20160518T215105716Z.xml</DocumentSystemFileNm>
        </ACATransmitterManifestReqDtl>
        <ACABusinessHeader a:Id="id-E5f1ed32aab8f4578adeee5debd851a62" xmlns:h="urn:us:gov:treasury:irs:msg:acabusinessheader" xmlns:a="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-utility-1.0.xsd" xmlns="urn:us:gov:treasury:irs:msg:acabusinessheader">
            <UniqueTransmissionId xmlns="urn:us:gov:treasury:irs:ext:aca:air:7.0">51958882-c653-4eab-8dfb-287ecc555aaa:SYS12:[TCC]::T</UniqueTransmissionId>
            <Timestamp xmlns="urn:us:gov:treasury:irs:common">2016-05-18T14:51:05.8568594-07:00</Timestamp>
        </ACABusinessHeader>
    </s:Header>
    <s:Body xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xsd="http://www.w3.org/2001/XMLSchema">
        <ACABulkRequestTransmitter xmlns="urn:us:gov:treasury:irs:msg:irsacabulkrequesttransmitter" version="1.0">
            <BulkExchangeFile xmlns="urn:us:gov:treasury:irs:common">
                <inc:Include href="cid:1094C_Request_BB0S4_20160518T215105716Z.xml" xmlns:inc="http://www.w3.org/2004/08/xop/include" />
            </BulkExchangeFile>
        </ACABulkRequestTransmitter>
    </s:Body>
</s:Envelope>

--MIME_Boundary
Content-Type: text/xml; charset=us-ascii
Content-Transfer-Encoding: 7bit
Content-Id: <1094C_Request_[TCC]_20160518T215105716Z.xml>
Content-Disposition: attachment; name="1094C_Request_[TCC]_20160518T215105716Z.xml"

[PrettyPrintFormDataXml]
--MIME_boundary--

Floatation answered 15/3, 2016 at 20:3 Comment(13)

Content-Id: <rootpart>, What is <rootpart> here? what needs to be replaced in place of <rootpart>? – Levona 25/4, 2016 at 10:49

@Levona I don't think you need to replace <rootpart> with anything. I believe Content-Id is arbitrary, meaning it can be any string you want it to be as long as it matches with a reference. In the example above, the first Content-Id (the first MIME part) needs to match the start attribute in the Content-Type header. Lower in the example, The Content-Id of the second MIME part, should match with the <inc:Include href of the <urn1:BulkExchangeFile> element in the SOAP Body. – Floatation 25/4, 2016 at 15:17

We tried the same as you mentioned and still I stuck with the error "[TPE1105] The message was not formatted properly and/or cannot be interpreted." – Levona 26/4, 2016 at 4:32

I'm working on bundling some stuff up to share, but in the meantime I can tell you for sure that I had to remove all line breaks from the entire envelope. The form xml seems to allow it, but I specifically had issues with line breaks in the envelope where the fault responses said I had extra child elements that weren't expected because it was apparently treating those breaks as child text nodes (which I suppose is technically true, but every other framework I've worked with has been able to handle that) – Lorrainelorrayne 19/5, 2016 at 16:15

Thanks @jstill. The envelope I am sending is un-formatted (no line breaks); I have formatted it for this post with the appropriate line breaks for the sake of readability. I look forward to seeing what you are preparing to share. – Floatation 19/5, 2016 at 16:18

Hi @Floatation I'm too developing IRS related stuff.. Pls ping me so that we can code as a community and share the common problem faced, and its solutions – Warfourd 20/2, 2018 at 13:2

Hi @Floatation , I know it's a long shot but would you be willing to share the solution that finally worked for you? It would be rad if you felt comfortable with publishing your code. Thanks! – Effect 3/8, 2018 at 17:15

@MuthuGanapathyNathan Have you been successful in transmitting to the A2A service? – Effect 3/8, 2018 at 17:20

@Effect Unfortunately, I'm not sure I am legally able to publicly publish it. I am willing to answer questions if able. I the [irs] tag here and there to see of any new posts. – Floatation 3/8, 2018 at 18:25

Hi @Effect as Russ told, we are not able to publish the code, since its a private repo.. However, we could clear your doubts. Yes we finally successfully filed for a cx. – Warfourd 4/8, 2018 at 11:50

@Floatation Did you final solution end up using the consumed WSDL or your manual creation of the SOAP envelope? We are attempting to transition from UI to A2A (again lol) and this year's WSDL is failing when trying to build out a reference file using wsdl.exe. Currently stuck on "failed to import binding: BulkRequestTransmitterBinding". Wasn't sure if this was question worthy so I figured I'd throw it here first lol – Parting 24/10, 2018 at 16:49

The working solution I settled on (and have been using since 2016), has used a hybrid/manual approach to the creation of the SOAP Envelope. I created XML file templates for the Submission, Status and Form Data. I also use the Submission and Service objects found in the WSDL to capture data. Then I write the data found in the objects to the XML template. I hope this very high level explanation helps you out. – Floatation 25/10, 2018 at 0:13

Thanks for the reply Russ! I was definitely doing the wrong thing on trying to generate the reference file lol. I'm sure I'll be throwing actual questions up on the irs tag once AATS opens up again! – Parting 25/10, 2018 at 16:59

Don't know if it will resolve your issue, but nevertheless i give it a shot. Sometimes help comes from very unexpected sources :)

First of all timestamp fields are in a wrong format: one in businessheader should NOT contain milliseconds at all. I know it for a fact.
In security header timestamp only 3 digits are allowed to represent milliseconds.
Remove empty elements like "OriginalReceiptId" from ACATransmitterManifestReqDtl element: they don't like those.
I hope you are providing them with proper software id, because you have it empty in the payload, but I am sure they would love to have it, imho.:)
And I think the message you've got in the response also have something to do with Signature element. I think they want Signature element to have some prefix("ds" preferably, I guess). But here I am not sure on 100%.

You see, I am battling same battle as you. And my message security timestamp has prefix "u" and they do not complain about it. Though they didn't like binarysecuritytoken ever.:) I am struggling to generate signature to the IRS liking. WCF is very secretive and does not allow easy prefix changing on soap envelope or allow to choose CanonicalizationMethod algorithm for a signature.

UPDATE: Been able to successfully send request to the service. Tell you at once: prefixes are unimportant. What was important: CorrectedInd tag must be present in Form1095BUpstreamDetail and attributes recordType="String" lineNum="0" also must be present.

UPDATE2: Another thing that I've changed I've placed ACABusinessHeader before ManifestDtl. Here are my settings: I am using WCF as carrier and SignedXml to generate signature. Also I am using custom gZip encoder(for obvious reasons0 and custom MtomEncoder to read response from service(yes, yes it's MTOMed:)) can you believe those pokemons?!?!?) and that's not all: they send response as multipart document with only 1 part!:)) I had to adjust my encoder to handle that. And voilà, service started to behave. Hope it might help.

UPDATE3 First of all make sure data in attachment file correspond to the test scenario you are using as guinea pig. I, probably, sound like a broken record, but that's REALLY important. Now I'll cut the stuff and present what I have. It's a bit crude, but it does the trick.:)

1.Here is config file portion:
1.1.Make sure system.serviceModel element contains following portion:

<extensions>
  <bindingElementExtensions>
    <add name="gzipMessageEncoding" type="<namespaceWhereEncoderLives>.GZipMessageEncodingElement, GZipEncoder, Version=4.0.0.0, Culture=neutral, PublicKeyToken=null" />
  </bindingElementExtensions>
</extensions>

1.2. Make sure binding element contains this:

  <customBinding>
    <binding name="BulkRequestTransmitterBinding">
      <gzipMessageEncoding innerMessageEncoding="textMessageEncoding" />
      <httpsTransport />
    </binding>
  </customBinding>

1.3. Change binding of BulkRequestTransmitterPort endpoit under client element to "customBinding"(and change binding name to the name of the custom binding as well) and make sure it contains following portion:

    <identity>
      <dns value="domain from cert" />
    </identity>

Also client element should contain following portion:

  <metadata>
    <policyImporters>
      <extension type="NamespaceToToTheLocationOf.GZipMessageEncodingBindingElementImporter, GZipMessageEncoder, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null" />
    </policyImporters>
  </metadata>

GZip encoder you could get from following link: https://msdn.microsoft.com/en-us/library/cc138373(v=vs.90).aspx Just download WCF example and dully move whole GZipMessageEncoder project under your project.
Get MTOMEncoder(which I renamed from SwaEncoder for clarity reasons) from this link: Soap-with-Attachments Move following classes into GZipMessageEncoder project:
MimeContent, MimeParser, MimePart, MTOMEncoder

Modify GZipMessageEncoder class like this:
4.1. Add following code at the beginning of the class:

   //------------------- MTOM related stuff. Begin. ---------------------
    const string ATTCHMNT_PROP = "attachment_file_content";
    const string ATTCHMNT_CONTENT_ID = "Here goes content id";

    private string _ContentType;
    private string _MediaType;

    protected MimeContent _MyContent;
    protected MimePart _SoapMimeContent;
    protected MimePart _AttachmentMimeContent;
    protected GZipMessageEncoderFactory _Factory;
    protected MimeParser _MimeParser;
    private void SetupMTOM(GZipMessageEncoderFactory factory)
    {
        //
        _ContentType = "multipart/related";
        _MediaType = _ContentType;

        //
        // Create owned objects
        //
        _Factory = factory;
        _MimeParser = new MimeParser();

        //
        // Create object for the mime content message
        // 
        _SoapMimeContent = new MimePart()
        {
            ContentTypeStart = "application/xop+xml",
            ContentType = "text/xml",
            ContentId = "Here goes envelope MIME id from HTTP Content-Type header",   // TODO: make content id dynamic or configurable?
            CharSet = "UTF-8",                                  // TODO: make charset configurable?
            TransferEncoding = "8bit"                         // TODO: make transfer-encoding configurable?
        };
        _AttachmentMimeContent = new MimePart()
        {
            ContentType = "application/xml",                    // TODO: AttachmentMimeContent.ContentType configurable?
            ContentId = ATTCHMNT_CONTENT_ID,                    // TODO: AttachmentMimeContent.ContentId configurable/dynamic?
            TransferEncoding = "7bit"                         // TODO: AttachmentMimeContent.TransferEncoding dynamic/configurable?
        };
        _MyContent = new MimeContent()
        {
            Boundary = "here goes boundary id"  // TODO: MimeContent.Boundary configurable/dynamic?
       };
        _MyContent.Parts.Add(_SoapMimeContent);
        _MyContent.Parts.Add(_AttachmentMimeContent);
        _MyContent.SetAsStartPart(_SoapMimeContent);
    }
    //------------------- MTOM related stuff. End. ----------------------

4.2. Modify Method WriteMessage(Message message, int maxMessageSize, BufferManager bufferManager, int messageOffset) like this:

public override ArraySegment<byte> WriteMessage(Message message, int maxMessageSize, BufferManager bufferManager, int messageOffset)
        {
            ArraySegment<byte> buffer = innerEncoder.WriteMessage(message, maxMessageSize, bufferManager, 0);
            var requestSOAPEnvelopeXml = System.Text.Encoding.UTF8.GetString(buffer.Array);

            //Here you create Security node and sign the request. For ex:
            requestSOAPEnvelopeXml = SigngEnvelope(requestSOAPEnvelopeXml);
            //Here you are getting 1094\1095 forms xml payload.
            string fileContent = GetAttachmentFileContent();

            //Here comes the MTOMing...
            _SoapMimeContent.Content = System.Text.Encoding.UTF8.GetBytes(requestSOAPEnvelopeXml);
            _AttachmentMimeContent.Content = System.Text.Encoding.UTF8.GetBytes(fileContent);

            _MyContent.Parts.Where(m=> m.ContentId!=null && m.ContentId.Equals(ATTCHMNT_CONTENT_ID)).Single().ContentDisposition = GetFileName(envelope);
            // Now create the message content for the stream
            byte[] MimeContentBytes = _MimeParser.SerializeMimeContent(_MyContent);
            int MimeContentLength = MimeContentBytes.Length;

            // Write the mime content into the section of the buffer passed into the method
            byte[] TargetBuffer = bufferManager.TakeBuffer(MimeContentLength + messageOffset);
            Array.Copy(MimeContentBytes, 0, TargetBuffer, messageOffset, MimeContentLength);

            // Return the segment of the buffer to the framework
            return CompressBuffer(new ArraySegment<byte>(TargetBuffer, messageOffset, MimeContentLength), bufferManager, messageOffset);                
        }

4.3. Override couple more methods like this:

public override Message ReadMessage(ArraySegment<byte> buffer, BufferManager bufferManager, string contentType)
        {
            ArraySegment<byte> decompressedBuffer = DecompressBuffer(buffer, bufferManager);

            MtomEncoder mtomEncoder = new MtomEncoder(innerEncoder, _Factory);
            Message returnMessage = mtomEncoder.ReadMessage(buffer, bufferManager, contentType);
            returnMessage.Properties.Encoder = mtomEncoder;

            return returnMessage;
        }

        public override bool IsContentTypeSupported(string contentType)
        {
            return true;
        }

4.4. Make sure GZipMessage constructor looks like this:

        internal GZipMessageEncoder(MessageEncoder messageEncoder, GZipMessageEncoderFactory factory)
            : base()
        {
            if (messageEncoder == null)
                throw new ArgumentNullException("messageEncoder", "A valid message encoder must be passed to the GZipEncoder");
            innerEncoder = messageEncoder;

            SetupMTOM(factory);
        }

5. Make sure GZipMessageEncodingBindingElement class has following method:

    public override void ApplyConfiguration(BindingElement bindingElement)
    {
        GZipMessageEncodingBindingElement binding = (GZipMessageEncodingBindingElement)bindingElement;
        PropertyInformationCollection propertyInfo = this.ElementInformation.Properties;
        if (propertyInfo["innerMessageEncoding"].ValueOrigin != PropertyValueOrigin.Default)
        {
            switch (this.InnerMessageEncoding)
            {
                case "textMessageEncoding":
                    binding.InnerMessageEncodingBindingElement = 
                        new TextMessageEncodingBindingElement(MessageVersion.Soap11, Encoding.UTF8);
                    break;
                case "binaryMessageEncoding":
                    binding.InnerMessageEncodingBindingElement = new BinaryMessageEncodingBindingElement();
                    break;
            }
        }
    }

Modify MTOMEncoder class. Make sure that following method looks like this:

public override Message ReadMessage(System.IO.Stream stream, int maxSizeOfHeaders, string contentType)
{
    VerifyOperationContext();

    if (contentType.ToLower().StartsWith("multipart/related"))
    {
        byte[] ContentBytes = new byte[stream.Length];
        stream.Read(ContentBytes, 0, ContentBytes.Length);
        MimeContent Content = _MimeParser.DeserializeMimeContent(contentType, ContentBytes);

        if (Content.Parts.Count >= 1)
        {
            MemoryStream ms = new MemoryStream(Content.Parts[0].Content);
            //At least for now IRS is sending SOAP envelope as 1st part(and only part(sic!) of MULTIpart response) as xml. 
            Message Msg = ReadMessage(ms, int.MaxValue, "text/xml");//Content.Parts[0].ContentType);

            if( Content.Parts.Count>1 )
                Msg.Properties.Add(ATTCHMNT_PROP, Content.Parts[1].Content);

            return Msg;
        }
        else
        {
            throw new ApplicationException("Invalid mime message sent! Soap with attachments makes sense, only, with at least 2 mime message content parts!");
        }
    }
    else if (contentType.ToLower().StartsWith("text/xml"))
    {
        XmlReader Reader = XmlReader.Create(stream);
        return Message.CreateMessage(Reader, maxSizeOfHeaders, MessageVersion);
    }
    else
    {
        throw new ApplicationException(
            string.Format(
                "Invalid content type for reading message: {0}! Supported content types are multipart/related and text/xml!",
                contentType));
    }
}

GZipMessageEncoderFactory class constructor should look like this:

   public GZipMessageEncoderFactory(MessageEncoderFactory messageEncoderFactory)
{
    if (messageEncoderFactory == null)
        throw new ArgumentNullException("messageEncoderFactory", "A valid message encoder factory must be passed to the GZipEncoder");
    encoder = new GZipMessageEncoder(messageEncoderFactory.Encoder, this);
}

This is how I call the service:

   var requestClient = new BulkRequestTransmitterPortTypeClient("BulkRequestTransmitterPort");

        requestClient.Endpoint.Contract.ProtectionLevel = System.Net.Security.ProtectionLevel.None;
 #if DEBUG
        var vs = requestClient.Endpoint.Behaviors.Where((i) => i.GetType().Namespace.Contains("VisualStudio"));
        if( vs!=null )
         requestClient.Endpoint.Behaviors.Remove((System.ServiceModel.Description.IEndpointBehavior)vs.Single());
#endif                
   using (var scope = new OperationContextScope(requestClient.InnerChannel))
        {
             //Adding proper HTTP Header to an outgoing requqest.
            HttpRequestMessageProperty requestMessage = new HttpRequestMessageProperty();

            requestMessage.Headers["Content-Encoding"] = "gzip";
            requestMessage.Headers["Content-Type"] = @"multipart/related; type=""application/xop+xml"";start=""<Here goes envelope boundary id>"";start-info=""text/xml"";boundary=""here goes boundary id""";
            OperationContext.Current.OutgoingMessageProperties[HttpRequestMessageProperty.Name] = requestMessage;

            response = requestClient.BulkRequestTransmitter(request.ACASecurityHeader,
                                                                request.Security, ref request.ACABusinessHeader,
                                                                request.ACATransmitterManifestReqDtl, 
                                                                request.ACABulkRequestTransmitter);
        }

Modify Mime Part:

9.1. Add new method:

    public void GetHeader(StringBuilder Builder)
    {
        if (string.IsNullOrEmpty(ContentId) && string.IsNullOrEmpty(ContentType) && string.IsNullOrEmpty(TransferEncoding))
            return;

        if (!string.IsNullOrEmpty(ContentTypeStart))
        {
            Builder.Append(string.Format("Content-Type: {0}", ContentTypeStart));
            Builder.Append(string.Format("; type=\"{0}\"", ContentType));
        }
        else
            Builder.Append(string.Format("Content-Type: {0}", ContentType));

        if (!string.IsNullOrEmpty(CharSet)) Builder.Append(string.Format("; charset={0}", CharSet));
        Builder.Append(new char[] { '\r', '\n' });
        Builder.Append(string.Format("Content-Transfer-Encoding: {0}", TransferEncoding));
        Builder.Append(new char[] { '\r', '\n' });
        Builder.Append(string.Format("Content-Id: {0}", ContentId));
        Builder.Append(new char[] { '\r', '\n' });
        if (!string.IsNullOrEmpty(ContentDisposition))
            Builder.Append(string.Format("Content-Disposition: attachment; filename=\"{0}\"", ContentDisposition));
    }

9.2. Add property:

   public string ContentDisposition { get; set; }

Modify MimeParser SerializeMimeContent() method: replace this block of code:

       Builder.Append(string.Format("Content-Type: {0}", item.ContentType));
        if (!string.IsNullOrEmpty(item.CharSet)) Builder.Append(string.Format("; charset={0}", item.CharSet));
        Builder.Append(new char[] { '\r', '\n' });
        Builder.Append(string.Format("Content-Transfer-Encoding: {0}", item.TransferEncoding));
        Builder.Append(new char[] { '\r', '\n' });
        Builder.Append(string.Format("Content-Id: {0}", item.ContentId));

with this:

item.GetHeader(Builder);

And that's should be it! Kick off your shoes and dig the blues!:)))

Righthanded answered 16/3, 2016 at 15:10 Comment(51)

Thank you very much for your detailed response. In one of their documents I thought I had seen the Business Header Timestamp having milliseconds. Apparently, I imagined that cause I definitely can't find the reference when reviewing the documents; so I have fixed that. My Security Header timestamps, look proper according to the sample in the Composition Guide. Thanks for catching that the OriginalReceiptId was still present and not populated. I have removed it. I also have made sure that the SoftwareID is being populated (not sure why this wasn't there either). – Floatation 16/3, 2016 at 15:35

Here is how I ended up signing my request. It got me around the "WS-Security is Invalid" error, at least for the Transmittal. I am still receiving it when making a request for the Status. However, there could be an error on my part with that, I haven't had a chance to go back and review it as I've been concentrating on getting the Transmittal to work. – Floatation 16/3, 2016 at 15:37

Unfortunately, I don't think I can add the "ds" prefix to the signature, after the signature has been computed. When doing so, the signature is unable to be verified. – Floatation 16/3, 2016 at 21:23

u r welcome. Programmes have to help each other, rnt we?:)Yes, composition guide is not right is not right on account of businessheader timestamp.:) How I know it is true? I've got error message in the response complaining about it.:) After I removed miliseconds service started to complain about binarysecuritytoken.:) Thank for the hint about creating a signature. I've created signature myself before, but in my version I haven't been able create SecurityTokenReference element with KeyIdentifier theyare looking for. Do you know by any chance how to generate that? – Righthanded 17/3, 2016 at 0:3

After signing, I create a new <KeyInfo> element structure (according to the composition guide) and populate it with the required values. Then, I am replacing the <KeyInfo> element which was generated by the signing, with this new element just created. This does not have an effect on the signature, because they KeyInfo is outside of what is signed. – Floatation 17/3, 2016 at 15:26

After a bit of working I found this link which I was able to use to add the "ds" prefix to my Signature elements and re-sign my request. I have verified that the "ds" prefix was pre-pended to each element (as shown in the Composition Guide) and that the Signature could to be verified successfully. I did have to make some changes to the Signing method. Unfortunately, I am still stuck on the error initially reported above. – Floatation 17/3, 2016 at 22:32

Alright, I have the ACABusinessHeader element between the Security element and the Manifest element. Can I assume that you are creating the SOAP XML by hand and populating the values as necessary. How are you sending the request? Would you mind providing examples of how you utilized the custom GZipMessageEncoder and the custom MtomMessageEncoder (I have not seen this one yet). – Floatation 30/3, 2016 at 21:23

@Floatation I've Updated my answer. Hope you would be able make sense out of it.:) – Righthanded 31/3, 2016 at 19:0

This is great, thanks! Previous incarnations of my code, seemed to put me in the same block as the ballpark, as I had previously integrated the GZipMessageEncoding project from MSDN, and looked into the SoapWithAttachments project previously. Question: Do you have a link to the MtomEncoder classes? I presume it is similar in fashion to the GZIP stuff. – Floatation 31/3, 2016 at 20:47

When referencing the MtomEncoder do you actually mean the SwaEncoder from the example at the link? If so, then I presume you had renamed SwaEncoder (and references) from Swa to Mtom. Where there any additional changes to the SoapWithAttachments that you made, other than what you had mentioned above? Thanks so much. – Floatation 31/3, 2016 at 21:53

@Floatation Yep. I've renamed SwaEncoder into MtomEncoder. Forgot to mention that. My bad. On account of additional changes: I think I've mentioned everything I've done.... At least as I remember.:) Do you have any issues? – Righthanded 1/4, 2016 at 13:4

I think I have most everything setup as far as the class files go. However, I'm a little lost when it comes to integrating the two encoders: MtomEncoder and GZipEncoder. GZipEncoder is auto-magically integrated through the app.config. It looks like MtomEncoder is setup simply by getting called from GZipEncoder. You only mention setting up the MtomEncoder class, can I ignore the other classes: EncoderFactory, BindingElement, EncodingElement, etc? SetupMTOM method: This looks to setup the MIME information appropriately, but where does it get called? – Floatation 1/4, 2016 at 15:51

@Floatation No, no, no. You cannot ignore them. GzipMessageEncoder uses MtomEncoder. GZipMessageEncoder is main character here. All those classes are required by WCF framework. SetupMtom() method is called from GZipMessageEncode constructor. I've updated my answer with missing code. – Righthanded 1/4, 2016 at 17:6

if we are using the GZipMessageEncoder constructor and it (now) contains two parameters (the original code base only contained the MessageEncoder object), what are we passing in as the GZipMessageEncoderFactory parameter when creating a new instance of the GZipMessageEncoder from the GZipMessageEncoderFactory class? – Floatation 1/4, 2016 at 18:4

@Floatation Jee, it was more elaborate, then I thought!:) I'll update my answer with missing code. – Righthanded 1/4, 2016 at 18:27

Additionally, I'm receiving an error which states Unable to cast object of type 'System.ServiceModel.Channels.TextMessageEncoderFactory' to type 'GZipEncoder.GZipMessageEncoderFactory'. It seems to be occuring when I create a new OperationContextScope(client.InnerChannel) when attempting to submit the request. I use the OperationContext to add the Content-Encoding and Content-Transfer-Encoding properties and values to a HttpRequestMessageProperty object. – Floatation 1/4, 2016 at 18:33

It looks like the BuildChannelFactory<TChannel>(BindingContext context) method in GZipMessageEncodingBindingElement fires, this goes into the GZipMessageEncoderFactory(MessageEncoderFactory messageEncoderFactory) method, and errors on encoder = new GZipMessageEncoder(messageEncoderFactory.Encoder, this); – Floatation 1/4, 2016 at 18:43

@Floatation Verify what type _Factory in GZipMessageEncoder has. It should be GZipMessageEncoderFactory. – Righthanded 1/4, 2016 at 18:58

Looks like the error I was experiencing may have been because the two projects weren't building properly. (I would like these in a single project) One of those "gotchas". :( I was successfully able to debug into the WriteMessage method of the GZipMessageEncoder class. – Floatation 1/4, 2016 at 18:59

Great post! Do you happen to have this as a project you could share via GitHub or the like? – Thaw 2/4, 2016 at 15:46

@Thaw No, I do not. – Righthanded 4/4, 2016 at 12:23

@Floatation Listen I would like to ask you for a favor. I do not have enough credits to comment, but I would like to Ask Bon(he claimed in comments, that he got it working) what was his solution for the status service? Was it certificate or signature generation? Here is the link:#35183451 If you would be able to ask him I would appreciate it. – Righthanded 4/4, 2016 at 14:18

@Righthanded There is a lot of information in the answer comments. You should be able to post something there (if you can't post an answer as your question). It looks like Bon edited his answer as a notification that he could not get WCF to work properly. – Floatation 4/4, 2016 at 15:23

@Floatation He also claimed, that he got both services working and he has same issue: submit service working and status service getting "WS Security is invalid" error. Here is his comment:"Ok, I DID get this working finally for the RequestSubmissionStatusDetail as well. So I know my solution works now for both endpoints. "(c) But, unfortunately he didn't go into any details what he has done to make it work. – Righthanded 4/4, 2016 at 15:58

@Floatation Have you been able to put submit service to work, btw? – Righthanded 4/4, 2016 at 16:4

@Righthanded Unfortunately, I haven't been able to get any of it to work yet. :( I believe he has come up with a solution but, numerous times he has said he is unable to share any code due to contractual obligations with whatever company he is employed by. Which is unfortunate for all of us struggling. – Floatation 4/4, 2016 at 16:35

@Floatation I thought if he would be able to tell what was it, that made his solution to work, that's would be interesting to know.. We could write own code!;) What seems to be a hold up? – Righthanded 4/4, 2016 at 17:0

@Righthanded I ran out of time Friday after fixing the errors I was running into with your suggestions. At the moment, I have a class library setup with the GZip and Mtom encoders and a reference within the application project to the class library. I wasn't having any luck getting the app.config entries working when I had the class files directly in the application project. – Floatation 4/4, 2016 at 17:32

@Floatation app.config contains reference to GZipMessageEncoder. So you need to instruct WCF where that class lives. Just check item 1.1 and 1.3 in my last update and verify what namespace is there for the GZipMessageEncoder class. Make sure, that your config contains proper namespace for the custom encoder class. – Righthanded 4/4, 2016 at 17:45

Let us continue this discussion in chat. – Floatation 4/4, 2016 at 17:56

@Righthanded ContentId = "Here goes envelope id", Are you meaning that we need to setup an Id for SOAP envelop element and map that Id here? Could you please clarify what is envelope id? – Levona 25/4, 2016 at 11:1

@Levona Yes u r right. This is MIME id for the envelope part of the request, which is specified in Content-Type header. – Righthanded 25/4, 2016 at 13:18

@Levona Using my example, what you are referencing would be the string representation of <rootpart>. – Floatation 25/4, 2016 at 16:49

@Floatation I get passed the error TPE1105 (The message was not formatted properly and/or cannot be interpreted.) and now I"m getting the error TPE1114 (The request message must be MTOM encoded.) – Levona 26/4, 2016 at 8:56

@Righthanded Any luck getting the Status Check to work? I have no updates on my processing, personally, but may have something later today. – Floatation 11/5, 2016 at 15:50

@Floatation No, unfortunately there are no good news on my part either. IRS stopped communicating with us completely. So we are trying to find a way to turn back their interest. Still have no idea why status service doesn't work and submit does. – Righthanded 12/5, 2016 at 13:23

@Righthanded Good news, bad news. The good news is that I finally got all my ducks in a row and am able to begin actual testing of the submission. Bad news is that in the code for part 8 8 above (removing the Visual Studio generated endpoint), I am receiving an InvalidOperatonException on the Single() piece of code. I am receiving this error when executing the application on a server (but not on my local machine). Do we know that this is necessary for a successful submission? – Floatation 18/5, 2016 at 15:34

@Russ: with a combination of things you posted here and on many other questions/answers across SO (sorry to stalk you, but you had some good info) and what fatherOfWine provided, I've managed to get messages through and get a response back (captured via Fiddler). However, the code fatherOfWine provided does not seem to process the response properly as is (because the response is a single MIME part). Have you found a workaround for that? Anything prior to that step I might be able to help you with? – Lorrainelorrayne 18/5, 2016 at 16:11

@Floatation No, That code is not necessary in prod. Basically VS add debug info to the payload, which has not been jiving really well with IRS. That peace of code is for development environment only. – Righthanded 18/5, 2016 at 17:45

@Righthanded I have added a piece of code to the WriteMessage method which will check for that element, and remove it if it exists. Since that element is out of the Signature, signing it should not be a problem. I would rather have a check-and-remove (in the event that it does exist) even though it shouldn't exist. – Floatation 18/5, 2016 at 18:12

@Lorrainelorrayne Glad that we (fatherOfWine and myself) could provide useful information. This has been a pain of a project. Good to hear that you've gotten sending messages and capturing the returned response working appropriately (through Fiddler). Unfortunately, due to some delays on my end, I am just not getting around to testing out submitting data, and am working on getting through errors as they crop up. Currently, my SigningKey is not being loaded. I suspect this has to do with the way we are retrieving the Certificate. – Floatation 18/5, 2016 at 18:14

@Floatation This will work too. I think it would not validate against IRS schema of the request. Yes, signature would be unaffected, but validation process might. I think removing this tag in WriteMessage is more robust solution overall. – Righthanded 18/5, 2016 at 18:23

So I finally got my certificate issues sorted out, am now using the certificate installed correctly on the server; it has been registered with the IRS; and I am still getting the WS-Security error when attempting to send submissions for the test scenarios. :(:(:( – Floatation 18/5, 2016 at 21:58

I have updated my original post with the current request. – Floatation 18/5, 2016 at 22:15

@Righthanded So we attempted to re-register the Security Certificate. That didn't work, still receiving a WS-Security error. By chance, would you be able to provide a broad, yet step-by-step documentation regarding how to obtain a certificate, how to install the certificate, and how to register the certificate. I feel like we're missing something in that department, and it's unfortunately the blind leading the blind here on that subject. – Floatation 19/5, 2016 at 20:55

@Floatation I don't think I could. I know as much as you do in that department. – Righthanded 1/6, 2016 at 12:46

@Floatation I don't think I could. I know as much as you do in that department. Go to irs.gov/for-tax-pros and check that on e-Services client id created for you TCC and cert with public key is uploaded, don't forget to send same certificate in keyinfo tag. I am under immense pressure right now, so I cannot respond in timely manner. I've made status service working as well and now we are frantically pushing to complete communication tests. – Righthanded 1/6, 2016 at 12:53

@Righthanded Thanks for all your help with this project, I can't say that enough. :) We were able to resolve the Security Certificate issues we were having, and successfully send Submissions, receive the response back, send a request for status, and receive that response back as well. Now we're onto processing those responses, while testing the AATS scenarios (also frantically, like you). There is light at the end of the tunnel, and it is good. – Floatation 1/6, 2016 at 20:23

@Floatation Glad it all worked out for u!:) Congrats! U r welcome, buddy.:) Programmers should help each other. We are all brothers in arms, aren't we?;)) – Righthanded 2/6, 2016 at 17:44

@Righthanded This is a super old thread but I figured I'd ask here, where exactly should the <identity> block go in the config file? Does it go directly inside the endpoint tag or client? EDIT: I'm guessing inside endpoint since outside it throws a warning, it's just not changing my current error lol – Parting 2/1, 2019 at 15:27

@Parting It goes inside <Client> tag under BulkRequestTransmitterPort endpoint tag. Just read carefully item 1.3. But I would suggest you try without specifying it at all. It might work for u just fine. – Righthanded 3/1, 2019 at 1:42

First off, a quick disclaimer. This answer was made possible by the great stuff provided by fatherOfWine, Russ, and Bon across this and other SO questions. All I really did was combine a bunch of stuff from them them and hack through the issues I still had until it worked. More importantly, the code provided here is BAD and should probably not be used as-is. I plan on cleaning this up quite a bit now that I know what works, and I'd recommend anyone making use of it to do the same. A big thing that will likely jump out to anyone looking at this is the plethora of static variables I used as a quick hack to get to things all through the pipeline. Seriously, don't use this as-is in production, it is the product of many hours of just throwing things at the wall until something stuck, but it should provide a good starting point to get something better going.

There's too much code to really include it all here, so I'll just go through some highlights and general discoveries then include a link to the VS solution.

Make sure you have your TCC and various other IDs setup already, that you've purchased the right kind of certificate (Page 41 of this doc) and that you've registered the cert properly (see this doc)
I found that removing all the CR-LFs in the soap envelope was necessary to get the message to be accepted. With them in there I would gets faults on some elements of "unexpected child elements" or something like that.
The documentation contradicts itself in several place (see pages 74 and 84 and look at what they say the BulkExchangeFile element should contain for an example) and the wsdl/xsds are just straight up wrong as well. Maybe I just had old ones somehow, but I had to make changes and try things until I found what the service on their side would actually accept.
It is very important that you add the keyinfo to the signedxml section properly, that all of your references there are built right and include the proper InclusiveNamespaces lists, and that once you call ComputeSignature the only change you make to your envelope is to add the signature element to it.
Speaking of the signature element, if it appears after the timestamp element inside the security element the IRS system will return a fault. It has to be first.
Because the namespace prefixes were so important when it came to generating the signature references, I went the route of building the envelope xml by hand so I could be certain that everything matched up exactly with what they wanted. Even then, there were several elements whose prefixes I had to change as I tested because what the XSD or some page in the docs said it should be was not what their service actually wanted. Luckily the faults returned by the service actually provided some help by indicating which namespace it was expecting a value from.

Outside of getting all the gzip and MTOM stuff setup (again, thanks a million to fatherOfWine for that help) the bulk of what finally worked for me is done in single general-use class (which I cleverly called "General"). Again, this is bad code and was the product of just needing to get something (anything!) to work properly. I'll go ahead and include it here in the answer though in case it provides a quick "ah ha!" to anyone else working this problem.

using System;
using System.IO;
using System.Net;
using System.Security.Cryptography.X509Certificates;
using System.Security.Cryptography.Xml;
using System.ServiceModel;
using System.ServiceModel.Channels;
using System.Text;
using System.Xml;
using IrsAcaClient.ACABulkRequestTransmitterService;

namespace IrsAcaClient
{
public class General
{
    /*****************************************************
     * 
     * What I'm doing here (with static vars) is VERY BAD but this whole thing is just a dirty hack for now.
     * Hopefully I can clean this up later.
     * - JRS 2016-05-10
     * 
     *****************************************************/
    public const string SecurityTimestampStringFormat = "yyyy'-'MM'-'dd'T'HH':'mm':'ss'.'fffZ";

    public const string EnvelopeContentID = "<rootpart>";

    public static string AttachmentFilePath;

    public static string AttachmentFileName { get { return Path.GetFileName(General.AttachmentFilePath); } }
    public static string AttachmentContentID {get { return string.Format("<{0}>", General.AttachmentFileName); }}

    public const string MIMEBoundary = "MIME_boundary";

    public static string TCCode;

    public static Guid TransmissionGuid;

    public static string UniqueTransmissionId
    {
        get { return string.Format("{0}:SYS12:{1}::T", TransmissionGuid, TCCode); }
    }

    public static string SecurityTimeStampWsuId;
    public static string ManifestWsuId;
    public static string BusinessHeaderWsuId;
    public static string SignatureWsuId;

    public static string CertificatePath;
    public static string CertificatePassword;

    public static DateTime SecurityTimestampUTC;

    private static string _replacementSoapEnvelope;

    public static string ReplacementSoapEnvelope{get { return _replacementSoapEnvelope; }}

    private static void GenerateReference(string elementID, string inclusivePrefixList, SignedXmlWithId xSigned)
    {
        var reference = new Reference()
        {
            Uri = "#" + elementID
        };

        XmlDsigExcC14NTransform env = new XmlDsigExcC14NTransform();
        env.InclusiveNamespacesPrefixList = inclusivePrefixList;
        reference.AddTransform(env);

        xSigned.AddReference(reference);
    }

    public static string GetAttachmentFileContent()
    {
        //probably not ideal
        return File.ReadAllText(AttachmentFilePath);
    }

    public static string GetFileName()
    {
        //TODO: this may need to be tweaked slightly from the real filename
        return General.AttachmentFileName;
    }

    public static string GenerateWsuId(string prefix)
    {
        return string.Format("{0}-{1}", prefix, Guid.NewGuid().ToString().Replace("-", "").ToUpper());
    }

    internal static void GenerateReplacementSoapEnvelope(ACABulkRequestTransmitterService.SecurityHeaderType securityHeader, ACABulkRequestTransmitterService.ACABulkBusinessHeaderRequestType businessHeader, ACABulkRequestTransmitterService.ACATrnsmtManifestReqDtlType manifest, ACABulkRequestTransmitterService.ACABulkRequestTransmitterType bulkTrans)
    {
        //load the base envelope xml
        var doc = new XmlDocument();
        doc.PreserveWhitespace = false;
        doc.Load("BaseSoapEnvelope.xml");

        /* Need a bunch of namespaces defined
         * xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/"
         * xmlns:urn="urn:us:gov:treasury:irs:ext:aca:air:7.0"
         * xmlns:urn1="urn:us:gov:treasury:irs:common"
         * xmlns:urn2="urn:us:gov:treasury:irs:msg:acabusinessheader"
         * xmlns:urn3="urn:us:gov:treasury:irs:msg:irsacabulkrequesttransmitter"
         * xmlns:wsa="http://www.w3.org/2005/08/addressing"
         * xmlns:wsse="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-secext-1.0.xsd"
         * xmlns:wsu="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-utility-1.0.xsd"
         * xmlns:ds="http://www.w3.org/2000/09/xmldsig#"
         * xmlns:xop="http://www.w3.org/2004/08/xop/include"
         */
        XmlNamespaceManager nsMgr = new XmlNamespaceManager(doc.NameTable);
        nsMgr.AddNamespace("soapenv", "http://schemas.xmlsoap.org/soap/envelope/");
        nsMgr.AddNamespace("urn", "urn:us:gov:treasury:irs:ext:aca:air:7.0");
        nsMgr.AddNamespace("urn1", "urn:us:gov:treasury:irs:common");
        nsMgr.AddNamespace("urn2", "urn:us:gov:treasury:irs:msg:acabusinessheader");
        nsMgr.AddNamespace("urn3", "urn:us:gov:treasury:irs:msg:irsacabulkrequesttransmitter");
        nsMgr.AddNamespace("wsa", "http://www.w3.org/2005/08/addressing");
        nsMgr.AddNamespace("wsse", "http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-secext-1.0.xsd");
        nsMgr.AddNamespace("wsu", "http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-utility-1.0.xsd");
        nsMgr.AddNamespace("ds", "http://www.w3.org/2000/09/xmldsig#");
        nsMgr.AddNamespace("xop","http://www.w3.org/2004/08/xop/include");


        //start replacing values in it
        //for securityHeader, should have the following
        /*
         * securityHeader.Signature.Id
         * securityHeader.Timestamp.Id
         * securityHeader.Timestamp.Created.Value
         * securityHeader.Timestamp.Expires.Value
         */
        //doc.SelectSingleNode("//wsse:Security/ds:Signature", nsMgr).Attributes["Id"].Value = securityHeader.Signature.Id;
        doc.SelectSingleNode("//wsse:Security/wsu:Timestamp", nsMgr).Attributes["wsu:Id"].Value = securityHeader.Timestamp.Id;
        doc.SelectSingleNode("//wsse:Security/wsu:Timestamp/wsu:Created", nsMgr).InnerText = securityHeader.Timestamp.Created.Value;
        doc.SelectSingleNode("//wsse:Security/wsu:Timestamp/wsu:Expires", nsMgr).InnerText = securityHeader.Timestamp.Expires.Value;


        //for businessHeader, should have the following
        /*
         * businessHeader.UniqueTransmissionId
         * businessHeader.Timestamp
         * businessHeader.Id 
         */
        doc.SelectSingleNode("//urn2:ACABusinessHeader", nsMgr).Attributes["wsu:Id"].Value = businessHeader.Id;
        doc.SelectSingleNode("//urn2:ACABusinessHeader/urn:UniqueTransmissionId", nsMgr).InnerText = businessHeader.UniqueTransmissionId;
        doc.SelectSingleNode("//urn2:ACABusinessHeader/urn1:Timestamp", nsMgr).InnerText = businessHeader.Timestamp.ToString("yyyy'-'MM'-'dd'T'HH':'mm':'ssZ");


        //for manifest, should have the following, some of which will need some conversions
        /*
         * manifest.Id
         * manifest.BinaryFormatCd - convert from enum
         * manifest.PaymentYr
         * manifest.PriorYearDataInd - convert from enum
         * manifest.EIN
         * manifest.TransmissionTypeCd - convert from enum
         * manifest.TestFileCd
         * manifest.TransmitterNameGrp.BusinessNameLine1Txt
         * manifest.CompanyInformationGrp.CompanyNm
         * manifest.CompanyInformationGrp.MailingAddressGrp.Item.AddressLine1Txt
         * manifest.CompanyInformationGrp.MailingAddressGrp.Item.CityNm
         * manifest.CompanyInformationGrp.MailingAddressGrp.Item.USStateCd - convert from enum
         * manifest.CompanyInformationGrp.MailingAddressGrp.Item.USZIPCd
         * manifest.CompanyInformationGrp.ContactNameGrp.PersonFirstNm
         * manifest.CompanyInformationGrp.ContactNameGrp.PersonLastNm
         * manifest.CompanyInformationGrp.ContactPhoneNum
         * manifest.VendorInformationGrp.VendorCd
         * manifest.VendorInformationGrp.ContactNameGrp.PersonFirstNm
         * manifest.VendorInformationGrp.ContactNameGrp.PersonLastNm
         * manifest.VendorInformationGrp.ContactPhoneNum
         * manifest.TotalPayeeRecordCnt
         * manifest.TotalPayerRecordCnt
         * manifest.SoftwareId
         * manifest.FormTypeCd - convert from enum
         * manifest.ChecksumAugmentationNum
         * manifest.AttachmentByteSizeNum
         * manifest.DocumentSystemFileNm
         */
        doc.SelectSingleNode("//urn:ACATransmitterManifestReqDtl", nsMgr).Attributes["wsu:Id"].Value = manifest.Id;
        doc.SelectSingleNode("//urn:ACATransmitterManifestReqDtl/urn:PaymentYr", nsMgr).InnerText = manifest.PaymentYr;
        doc.SelectSingleNode("//urn:ACATransmitterManifestReqDtl/urn:PriorYearDataInd", nsMgr).InnerText = manifest.PriorYearDataInd.GetXmlEnumAttributeValueFromEnum();
        doc.SelectSingleNode("//urn:ACATransmitterManifestReqDtl/urn1:EIN", nsMgr).InnerText = manifest.EIN;
        doc.SelectSingleNode("//urn:ACATransmitterManifestReqDtl/urn:TransmissionTypeCd", nsMgr).InnerText = manifest.TransmissionTypeCd.ToString();
        doc.SelectSingleNode("//urn:ACATransmitterManifestReqDtl/urn:TestFileCd", nsMgr).InnerText = manifest.TestFileCd;
        doc.SelectSingleNode("//urn:ACATransmitterManifestReqDtl/urn:TransmitterNameGrp/urn:BusinessNameLine1Txt", nsMgr).InnerText = manifest.TransmitterNameGrp.BusinessNameLine1Txt;
        doc.SelectSingleNode("//urn:ACATransmitterManifestReqDtl/urn:CompanyInformationGrp/urn:CompanyNm", nsMgr).InnerText = manifest.CompanyInformationGrp.CompanyNm;
        doc.SelectSingleNode("//urn:ACATransmitterManifestReqDtl/urn:CompanyInformationGrp/urn:MailingAddressGrp/urn:USAddressGrp/urn:AddressLine1Txt", nsMgr).InnerText = ((USAddressGrpType)manifest.CompanyInformationGrp.MailingAddressGrp.Item).AddressLine1Txt;
        doc.SelectSingleNode("//urn:ACATransmitterManifestReqDtl/urn:CompanyInformationGrp/urn:MailingAddressGrp/urn:USAddressGrp/urn1:CityNm", nsMgr).InnerText = ((USAddressGrpType)manifest.CompanyInformationGrp.MailingAddressGrp.Item).CityNm;
        doc.SelectSingleNode("//urn:ACATransmitterManifestReqDtl/urn:CompanyInformationGrp/urn:MailingAddressGrp/urn:USAddressGrp/urn:USStateCd", nsMgr).InnerText = ((USAddressGrpType)manifest.CompanyInformationGrp.MailingAddressGrp.Item).USStateCd.ToString();
        doc.SelectSingleNode("//urn:ACATransmitterManifestReqDtl/urn:CompanyInformationGrp/urn:MailingAddressGrp/urn:USAddressGrp/urn1:USZIPCd", nsMgr).InnerText = ((USAddressGrpType)manifest.CompanyInformationGrp.MailingAddressGrp.Item).USZIPCd;
        doc.SelectSingleNode("//urn:ACATransmitterManifestReqDtl/urn:CompanyInformationGrp/urn:ContactNameGrp/urn:PersonFirstNm", nsMgr).InnerText = manifest.CompanyInformationGrp.ContactNameGrp.PersonFirstNm;
        doc.SelectSingleNode("//urn:ACATransmitterManifestReqDtl/urn:CompanyInformationGrp/urn:ContactNameGrp/urn:PersonLastNm", nsMgr).InnerText = manifest.CompanyInformationGrp.ContactNameGrp.PersonLastNm;
        doc.SelectSingleNode("//urn:ACATransmitterManifestReqDtl/urn:CompanyInformationGrp/urn:ContactPhoneNum", nsMgr).InnerText = manifest.CompanyInformationGrp.ContactPhoneNum;
        doc.SelectSingleNode("//urn:ACATransmitterManifestReqDtl/urn:VendorInformationGrp/urn:VendorCd", nsMgr).InnerText = manifest.VendorInformationGrp.VendorCd;
        doc.SelectSingleNode("//urn:ACATransmitterManifestReqDtl/urn:VendorInformationGrp/urn:ContactNameGrp/urn:PersonFirstNm", nsMgr).InnerText = manifest.VendorInformationGrp.ContactNameGrp.PersonFirstNm;
        doc.SelectSingleNode("//urn:ACATransmitterManifestReqDtl/urn:VendorInformationGrp/urn:ContactNameGrp/urn:PersonLastNm", nsMgr).InnerText = manifest.VendorInformationGrp.ContactNameGrp.PersonLastNm;
        doc.SelectSingleNode("//urn:ACATransmitterManifestReqDtl/urn:VendorInformationGrp/urn:ContactPhoneNum", nsMgr).InnerText = manifest.VendorInformationGrp.ContactPhoneNum;
        doc.SelectSingleNode("//urn:ACATransmitterManifestReqDtl/urn:TotalPayeeRecordCnt", nsMgr).InnerText = manifest.TotalPayeeRecordCnt;
        doc.SelectSingleNode("//urn:ACATransmitterManifestReqDtl/urn:TotalPayerRecordCnt", nsMgr).InnerText = manifest.TotalPayerRecordCnt;
        doc.SelectSingleNode("//urn:ACATransmitterManifestReqDtl/urn:SoftwareId", nsMgr).InnerText = manifest.SoftwareId;
        doc.SelectSingleNode("//urn:ACATransmitterManifestReqDtl/urn:FormTypeCd", nsMgr).InnerText = manifest.FormTypeCd.GetXmlEnumAttributeValueFromEnum();
        doc.SelectSingleNode("//urn:ACATransmitterManifestReqDtl/urn1:BinaryFormatCd", nsMgr).InnerText = manifest.BinaryFormatCd.GetXmlEnumAttributeValueFromEnum();
        doc.SelectSingleNode("//urn:ACATransmitterManifestReqDtl/urn1:ChecksumAugmentationNum", nsMgr).InnerText = manifest.ChecksumAugmentationNum;
        doc.SelectSingleNode("//urn:ACATransmitterManifestReqDtl/urn1:AttachmentByteSizeNum", nsMgr).InnerText = manifest.AttachmentByteSizeNum;
        doc.SelectSingleNode("//urn:ACATransmitterManifestReqDtl/urn:DocumentSystemFileNm", nsMgr).InnerText = manifest.DocumentSystemFileNm;


        //for bulkTrans, should have the following
        /*
         * bulkTrans.BulkExchangeFile.Include.href
         */
        doc.SelectSingleNode("//urn3:ACABulkRequestTransmitter/urn1:BulkExchangeFile/xop:Include", nsMgr).Attributes["href"].Value = bulkTrans.BulkExchangeFile.Include.href;


        //now do some more security setup
        var cert = new X509Certificate2(CertificatePath, CertificatePassword, X509KeyStorageFlags.MachineKeySet);

        var exported = cert.Export(X509ContentType.Cert, CertificatePassword);
        var base64 = Convert.ToBase64String(exported);

        //now compute all the signing stuff
        var xSigned = new SignedXmlWithId(doc);
        xSigned.Signature.Id = securityHeader.Signature.Id;

        // Add the key to the SignedXml document.
        xSigned.SigningKey = cert.PrivateKey;
        xSigned.Signature.Id = SignatureWsuId;
        xSigned.SignedInfo.CanonicalizationMethod = SignedXml.XmlDsigExcC14NWithCommentsTransformUrl;

        var keyInfo = new KeyInfo
        {
            Id = GenerateWsuId("KI")
        };

        //need to get the keyinfo into the signed xml stuff before we compute sigs, and because it is using some stuff that
        //doesn't appear to be supported out of the box we'll work around it by adding a node directly
        var sbKeyInfo = new StringBuilder();
        sbKeyInfo.Append("<root xmlns:wsse=\"http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-secext-1.0.xsd\" xmlns:wsu=\"http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-utility-1.0.xsd\" xmlns:ds=\"http://www.w3.org/2000/09/xmldsig#\">");
        sbKeyInfo.Append("<wsse:SecurityTokenReference wsu:Id=\"" + GenerateWsuId("STR") + "\">");
        sbKeyInfo.Append("<wsse:KeyIdentifier EncodingType=\"http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-soap-message-security-1.0#Base64Binary\" ValueType=\"http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-x509-token-profile-1.0#X509v3\">" + base64.ToString());
        sbKeyInfo.Append("</wsse:KeyIdentifier>");
        sbKeyInfo.Append("</wsse:SecurityTokenReference>");
        sbKeyInfo.Append("</root>");
        XmlDocument tempDoc = new XmlDocument();
        tempDoc.LoadXml(sbKeyInfo.ToString());

        keyInfo.AddClause(new KeyInfoNode((XmlElement)tempDoc.FirstChild.FirstChild));

        xSigned.KeyInfo = keyInfo;

        GenerateReference(SecurityTimeStampWsuId, "wsse wsa soapenv urn urn1 urn2 urn3", xSigned);
        GenerateReference(BusinessHeaderWsuId, "wsa soapenv urn urn1 urn3", xSigned);
        GenerateReference(ManifestWsuId, "wsa soapenv urn1 urn2 urn3", xSigned);

        // Compute the Signature.
        xSigned.ComputeSignature();

        //signing stuff must come before the timestamp or the IRS service complains
        doc.SelectSingleNode("//wsse:Security", nsMgr).InsertBefore(xSigned.GetXml(), doc.SelectSingleNode("//wsse:Security", nsMgr).FirstChild);

        //
        _replacementSoapEnvelope = doc.OuterXml;
    }

    public static ACABulkRequestTransmitterResponseType Run(ACABulkRequestTransmitterService.SecurityHeaderType securityHeader, ACABulkRequestTransmitterService.ACABulkBusinessHeaderRequestType businessHeader, ACABulkRequestTransmitterService.ACATrnsmtManifestReqDtlType manifest, ACABulkRequestTransmitterService.ACABulkRequestTransmitterType bulkTrans)
    {
        //had some issues early on with the cert on the IRS server, this should probably be removed and retested without it
        ServicePointManager.ServerCertificateValidationCallback += (sender, certificate, chain, errors) => true;
        ServicePointManager.SecurityProtocol = SecurityProtocolType.Tls12 | SecurityProtocolType.Ssl3 |
                                               SecurityProtocolType.Tls11 | SecurityProtocolType.Tls;

        var acaSecurityHeader = new ACABulkRequestTransmitterService.TransmitterACASecurityHeaderType(); //leave this empty for transmitting via ISS-A2A

        var requestClient = new ACABulkRequestTransmitterService.BulkRequestTransmitterPortTypeClient("BulkRequestTransmitterPort");

        requestClient.Endpoint.Contract.ProtectionLevel = System.Net.Security.ProtectionLevel.None;
        //var vs = requestClient.Endpoint.Behaviors.Where((i) => i.GetType().Namespace.Contains("VisualStudio"));
        //if (vs != null)
        //    requestClient.Endpoint.Behaviors.Remove((System.ServiceModel.Description.IEndpointBehavior)vs.Single());

        //generate the real envelope we want
        GenerateReplacementSoapEnvelope(securityHeader, businessHeader, manifest, bulkTrans);

        using (var scope = new OperationContextScope(requestClient.InnerChannel))
        {

            //Adding proper HTTP Header to an outgoing requqest.
            HttpRequestMessageProperty requestMessage = new HttpRequestMessageProperty();

            requestMessage.Headers["Content-Encoding"] = "gzip";
            requestMessage.Headers["Content-Type"] = string.Format(@"multipart/related; type=""application/xop+xml"";start=""{0}"";start-info=""text/xml"";boundary=""{1}""", General.EnvelopeContentID, General.MIMEBoundary);
            OperationContext.Current.OutgoingMessageProperties[HttpRequestMessageProperty.Name] = requestMessage;

            var response = requestClient.BulkRequestTransmitter(acaSecurityHeader,
                                                                securityHeader,
                                                                ref businessHeader,
                                                                manifest,
                                                                bulkTrans);

            //we got a response!  now do something with it
            return response;

        }
    }
}

Here is the complete solution, just needs all of your own data supplied (including the complete attachment file with all the payee and payer records, which is outside the scope of this but should be pretty easy to generate). Also note that this is submission of forms only, not status checks. When I get that working I'll try to remember to return and update this answer (but if someone else already has it and wants to share, that'd be pretty rad as well).

Edit for Status Service

I've combined a cleaned up version of the classes generated from the wsdl and my own junk code to get messages through and process the responses. Note that this isn't 100% tested yet, needs sanity checks, etc. but like the previous stuff should at least help anyone else struggling with this mess. Usage here is pretty straightforward:

var statusResponse = StatusService.CheckStatus(receipt, tCCode, certificatePath, certificatePassword, "https://la.www4.irs.gov/airp/aca/a2a/1095BC_Status_Request_AATS2016");

And here is the full class (with bonus generated classes namespace):

See my second answer for the status service code

Lorrainelorrayne answered 20/5, 2016 at 13:25 Comment(8)

This is fantastic! I am going to begin reviewing your code and comparing it to what I have to see what exactly I'm missing. As far as the Status Service goes, you can add another (separate) answer for that piece. – Floatation 24/5, 2016 at 15:37

Did you have to add code to the generated Reference.cs file? I see comments related to an updated of the BulkExchangeFile to change the definition from byte[]. It looks like you also added code for BulkExchangeFileType and IncludeType? What is your plan for when IRS ultimately updates the WSDL to a newer version? – Floatation 24/5, 2016 at 16:7

Yeah, I did have to manually modify the code generated by the WSDL, because their service returned faults that directly contradicted the WSDL itself. Even under a normal WCF setup, any sort of meaningful WSDL update requires code changes, so in this case I'll likely just regen classes from the new WSDL with xsd.exe, replace the current ones, and work through the issues. Sorry about no status code yet, got busy and forgot. Will get it posted shortly. – Lorrainelorrayne 24/5, 2016 at 19:53

No problem, it's great seeing another solution and seeing how close two different programmers are to one another while still being widely different in their solutions. One thing I have noticed in your solution, is you do not have the OriginalReceiptId element defined in your manifest. This isn't a huge deal, however, the element is relavant if needing to submit a Replacement Transmission. – Floatation 24/5, 2016 at 20:10

Ah, yeah, hadn't gotten into testing retransmissions, but adding that in at the point of generating the replacement soap envelope (or including it in the base and removing it when not needed) should be cake. Status code is posted now, hope it helps as well. – Lorrainelorrayne 24/5, 2016 at 20:24

@Lorrainelorrayne Thank you for the awesome start!! It's been a help. The DropBox link for IrsAcaClient.zip is no longer working. I know it's a long shot but do you still have that laying around and can share it again? It would save me days of effort! Cheers! – Effect 3/8, 2018 at 0:48

@Effect Amazingly, I did still have that zip. I updated the link in the post, but here it is as well: dropbox.com/s/o8yu63ep2nn1p3t/IrsAcaClient.zip (no guarantees that any of it still works, though, I haven't touched it in ages) – Lorrainelorrayne 7/8, 2018 at 18:54

@Lorrainelorrayne Thank you SOO much!! Just knowing that it worked at some point in the past is already working better than what I have going! Cheers! – Effect 8/8, 2018 at 1:8

Seconds answer to include status service, rather than just another link that could disappear.

Here is the main class:

using System;
using System.IO;
using System.IO.Compression;
using System.Net;
using System.Security.Cryptography.X509Certificates;
using System.Security.Cryptography.Xml;
using System.Text;
using System.Text.RegularExpressions;
using System.Xml;
using System.Xml.Serialization;

namespace IrsAcaClient
{
    public class StatusService
    {
        private const string SecurityTimestampStringFormat = "yyyy'-'MM'-'dd'T'HH':'mm':'ss'.'fffZ";


        public static ACABulkRequestStatusService.ACABulkRequestTransmitterStatusDetailResponseType CheckStatus(string receiptID, string tCCode, string certificatePath, string certificatePassword, string statusServiceUrl)
        {
            //go ahead and generate some of the ids and timestamps we'll need
            var securityTimeStampWsuId = GenerateWsuId("TS");
            var businessHeaderWsuId = GenerateWsuId("id");
            var detailRequestWsuId = GenerateWsuId("id");
            var signatureWsuId = GenerateWsuId("SIG");

            var securityTimestampUTC = DateTime.UtcNow;
            var securityTimestampCreated = securityTimestampUTC.ToString(SecurityTimestampStringFormat);
            var securityTimestampExpires = securityTimestampUTC.AddMinutes(10).ToString(SecurityTimestampStringFormat);

            //build the envelope
            //load the base envelope xml
            var doc = new XmlDocument();
            doc.PreserveWhitespace = false;
            doc.Load("BaseStatusRequestEnvelope.xml");

            /* Need a bunch of namespaces defined
             * xmlns:oas1="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-secext-1.0.xsd"
             * xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/"
             * xmlns:urn="urn:us:gov:treasury:irs:msg:irstransmitterstatusrequest"
             * xmlns:urn1="urn:us:gov:treasury:irs:ext:aca:air:7.0" 
             * xmlns:urn2="urn:us:gov:treasury:irs:common"
             * xmlns:urn3="urn:us:gov:treasury:irs:msg:acasecurityheader"
             * xmlns:wsa="http://www.w3.org/2005/08/addressing"
             * xmlns:wsse="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-secext-1.0.xsd" 
             * xmlns:wsu="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-utility-1.0.xsd"
             * xmlns:ds="http://www.w3.org/2000/09/xmldsig#");
             */
            XmlNamespaceManager nsMgr = new XmlNamespaceManager(doc.NameTable);
            nsMgr.AddNamespace("soapenv", "http://schemas.xmlsoap.org/soap/envelope/");
            nsMgr.AddNamespace("urn", "urn:us:gov:treasury:irs:msg:irstransmitterstatusrequest");
            nsMgr.AddNamespace("urn1", "urn:us:gov:treasury:irs:ext:aca:air:7.0");
            nsMgr.AddNamespace("urn2", "urn:us:gov:treasury:irs:common");
            nsMgr.AddNamespace("urn3", "urn:us:gov:treasury:irs:msg:acasecurityheader");
            nsMgr.AddNamespace("wsa", "http://www.w3.org/2005/08/addressing");
            nsMgr.AddNamespace("wsse", "http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-secext-1.0.xsd");
            nsMgr.AddNamespace("wsu", "http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-utility-1.0.xsd");
            nsMgr.AddNamespace("oas1", "http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-secext-1.0.xsd");
            nsMgr.AddNamespace("ds", "http://www.w3.org/2000/09/xmldsig#");

            //start replacing values in it
            //for securityHeader, should have the following
            /*
             * securityHeader.Timestamp.Id
             * securityHeader.Timestamp.Created.Value
             * securityHeader.Timestamp.Expires.Value
             */
            doc.SelectSingleNode("//wsse:Security/wsu:Timestamp", nsMgr).Attributes["wsu:Id"].Value = securityTimeStampWsuId;
            doc.SelectSingleNode("//wsse:Security/wsu:Timestamp/wsu:Created", nsMgr).InnerText = securityTimestampCreated;
            doc.SelectSingleNode("//wsse:Security/wsu:Timestamp/wsu:Expires", nsMgr).InnerText = securityTimestampExpires;

            //for businessHeader, should have the following
            /*
             * businessHeader.UniqueTransmissionId
             * businessHeader.Timestamp
             * businessHeader.Id 
             */
            doc.SelectSingleNode("//urn:ACABusinessHeader", nsMgr).Attributes["wsu:Id"].Value = businessHeaderWsuId;
            doc.SelectSingleNode("//urn:ACABusinessHeader/urn1:UniqueTransmissionId", nsMgr).InnerText = GetUniqueTransmissionId(Guid.NewGuid(), tCCode);
            doc.SelectSingleNode("//urn:ACABusinessHeader/urn2:Timestamp", nsMgr).InnerText = securityTimestampUTC.ToString("yyyy'-'MM'-'dd'T'HH':'mm':'ssZ");

            //for ACABulkRequestTransmitterStatusDetailRequest, should have the following
            /*
             * ACABulkRequestTransmitterStatusDetailRequest.Id
             * ACABulkRequestTransmitterStatusDetailRequest.ACABulkReqTrnsmtStsReqGrpDtl.ReceiptId
             */
            doc.SelectSingleNode("//urn:ACABulkRequestTransmitterStatusDetailRequest", nsMgr).Attributes["wsu:Id"].Value = detailRequestWsuId;
            doc.SelectSingleNode("//urn:ACABulkRequestTransmitterStatusDetailRequest/urn1:ACABulkReqTrnsmtStsReqGrpDtl/urn2:ReceiptId", nsMgr).InnerText = receiptID;

            //now do some more security setup
            var cert = new X509Certificate2(certificatePath, certificatePassword, X509KeyStorageFlags.MachineKeySet);

            var exported = cert.Export(X509ContentType.Cert, certificatePassword);
            var base64 = Convert.ToBase64String(exported);

            //now compute all the signing stuff
            var xSigned = new SignedXmlWithId(doc);

            // Add the key to the SignedXml document.
            xSigned.SigningKey = cert.PrivateKey;
            xSigned.Signature.Id = signatureWsuId;
            xSigned.SignedInfo.CanonicalizationMethod = SignedXml.XmlDsigExcC14NWithCommentsTransformUrl;

            var keyInfo = new KeyInfo
            {
                Id = GenerateWsuId("KI")
            };

            //need to get the keyinfo into the signed xml stuff before we compute sigs, and because it is using some stuff that
            //doesn't appear to be supported out of the box we'll work around it by adding a node directly
            var sbKeyInfo = new StringBuilder();
            sbKeyInfo.Append("<root xmlns:wsse=\"http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-secext-1.0.xsd\" xmlns:wsu=\"http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-utility-1.0.xsd\" xmlns:ds=\"http://www.w3.org/2000/09/xmldsig#\">");
            sbKeyInfo.Append("<wsse:SecurityTokenReference wsu:Id=\"" + GenerateWsuId("STR") + "\">");
            sbKeyInfo.Append("<wsse:KeyIdentifier EncodingType=\"http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-soap-message-security-1.0#Base64Binary\" ValueType=\"http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-x509-token-profile-1.0#X509v3\">" + base64.ToString());
            sbKeyInfo.Append("</wsse:KeyIdentifier>");
            sbKeyInfo.Append("</wsse:SecurityTokenReference>");
            sbKeyInfo.Append("</root>");
            XmlDocument tempDoc = new XmlDocument();
            tempDoc.LoadXml(sbKeyInfo.ToString());

            keyInfo.AddClause(new KeyInfoNode((XmlElement)tempDoc.FirstChild.FirstChild));

            xSigned.KeyInfo = keyInfo;

            GenerateReference(securityTimeStampWsuId, "wsse wsa oas1 soapenv urn urn1 urn2 urn3", xSigned);
            GenerateReference(businessHeaderWsuId, "wsa oas1 soapenv urn1 urn2 urn3", xSigned);
            GenerateReference(detailRequestWsuId, "oas1 soapenv urn1 urn2 urn3", xSigned);

            // Compute the Signature.
            xSigned.ComputeSignature();

            //signing stuff must come before the timestamp or the IRS service complains
            doc.SelectSingleNode("//wsse:Security", nsMgr).InsertBefore(xSigned.GetXml(), doc.SelectSingleNode("//wsse:Security", nsMgr).FirstChild);

            //get the completed envelope
            var envelope = doc.OuterXml;

            //start the webrequest
            //get the request object
            var request = CreateWebRequest(statusServiceUrl);

            //get the request stream and then get a writer on it
            using (var stream = request.GetRequestStream())
            using (var gz = new GZipStream(stream, CompressionMode.Compress))
            using (var writer = new StreamWriter(gz))
            {
                //start by writing the soap envelope to the stream
                writer.WriteLine(envelope);
                writer.Close();
                stream.Close();
            }

            //get the response
            WebResponse response;

            //let an exception get thrown up the stack
            response = request.GetResponse();

            //get the response stream, get a reader on it, and read the response as text
            using (var responseStream = response.GetResponseStream())
            using (var reader = new StreamReader(responseStream, Encoding.UTF8))
            {
                var responseText = reader.ReadToEnd();

                //rip the one element (and children) we need out
                var match = Regex.Match(responseText, @"<(?'prefix'[\w\d]*):ACABulkRequestTransmitterStatusDetailResponse.*<\/\k<prefix>:ACABulkRequestTransmitterStatusDetailResponse>");

                return Deserialize<ACABulkRequestStatusService.ACABulkRequestTransmitterStatusDetailResponseType>(match.ToString());
            }
        }

        private static string GetUniqueTransmissionId(Guid transmissionGuid, string tCCode)
        {
            return string.Format("{0}:SYS12:{1}::T", transmissionGuid, tCCode);
        }

        private static string GenerateWsuId(string prefix)
        {
            return string.Format("{0}-{1}", prefix, Guid.NewGuid().ToString().Replace("-", "").ToUpper());
        }

        private static void GenerateReference(string elementID, string inclusivePrefixList, SignedXmlWithId xSigned)
        {
            var reference = new Reference()
            {
                Uri = "#" + elementID
            };

            XmlDsigExcC14NTransform env = new XmlDsigExcC14NTransform();
            env.InclusiveNamespacesPrefixList = inclusivePrefixList;
            reference.AddTransform(env);

            xSigned.AddReference(reference);
        }

        /// <summary>
        /// creates a webrequest object and prefills some required headers and such
        /// </summary>
        /// <param name="url"></param>
        /// <returns></returns>
        private static HttpWebRequest CreateWebRequest(string url)
        {
            //setup a web request with all the headers and such that the service requires
            var webRequest = (HttpWebRequest)WebRequest.Create(url);

            webRequest.Method = "POST";
            webRequest.ProtocolVersion = HttpVersion.Version11;
            webRequest.Headers.Add(HttpRequestHeader.ContentEncoding, "gzip");
            webRequest.Headers.Add(HttpRequestHeader.AcceptEncoding, "gzip, deflate");
            webRequest.ContentType = "text/xml;charset=UTF-8";
            webRequest.AutomaticDecompression = DecompressionMethods.GZip | DecompressionMethods.Deflate;
            webRequest.Headers.Add("SOAPAction", "RequestSubmissionStatusDetail");
            webRequest.KeepAlive = true;

            return webRequest;
        }

        /// <summary>
        /// deserializes the xml string into an object
        /// </summary>
        /// <param name="xmlString"></param>
        /// <returns></returns>
        public static T Deserialize<T>(string xmlString) where T : class
        {
            //if the string is empty, just return null
            if (xmlString.Length <= 0)
            {
                return null;
            }

            //create a serializer
            var serializer = new System.Xml.Serialization.XmlSerializer(typeof(T));
            T output;
            //create the reader that the serializer will read from, passing it the string
            using (var reader = new System.IO.StringReader(xmlString))
            {
                //rebuild the list object
                output = (T)serializer.Deserialize(reader);
            }
            //return the list
            return output;
        }
    }
}

Here is the relevant base xml:

<?xml version="1.0" encoding="utf-8" ?>
<soapenv:Envelope
xmlns:oas1="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-secext-1.0.xsd"
xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/"
xmlns:urn="urn:us:gov:treasury:irs:msg:irstransmitterstatusrequest"
xmlns:urn1="urn:us:gov:treasury:irs:ext:aca:air:7.0" xmlns:urn2="urn:us:gov:treasury:irs:common"
xmlns:urn3="urn:us:gov:treasury:irs:msg:acasecurityheader">
  <soapenv:Header xmlns:wsa="http://www.w3.org/2005/08/addressing">
    <wsse:Security xmlns:wsse="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-secext-1.0.xsd" xmlns:wsu="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-utility-1.0.xsd">
      <wsu:Timestamp wsu:Id="XXXXXXXXXXXXXXXXXX">
        <wsu:Created>XXXXXXXXXXXXXXXXXX</wsu:Created>
        <wsu:Expires>XXXXXXXXXXXXXXXXXX</wsu:Expires>
      </wsu:Timestamp>
    </wsse:Security>
    <urn:ACABusinessHeader wsu:Id="XXXXXXXXXXXXXXXXXX" xmlns:wsu="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-utility-1.0.xsd">
      <urn1:UniqueTransmissionId>
        XXXXXXXXXXXXXXXXXX
      </urn1:UniqueTransmissionId>
      <urn2:Timestamp>XXXXXXXXXXXXXXXXXX</urn2:Timestamp>
    </urn:ACABusinessHeader>
    <urn3:ACASecurityHeader />
    <wsa:Action>RequestSubmissionStatusDetail</wsa:Action>
  </soapenv:Header>
  <soapenv:Body>
    <urn:ACABulkRequestTransmitterStatusDetailRequest version="1.0" wsu:Id="XXXXXXXXXXXXXXXXXX" xmlns:wsu="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-utility-1.0.xsd">
      <urn1:ACABulkReqTrnsmtStsReqGrpDtl>
        <urn2:ReceiptId>XXXXXXXXXXXXXXXXXX</urn2:ReceiptId>
      </urn1:ACABulkReqTrnsmtStsReqGrpDtl>
    </urn:ACABulkRequestTransmitterStatusDetailRequest>
  </soapenv:Body>
</soapenv:Envelope>

For this one, the main change I needed to make to the WSDL-generated classes was the following:

    [System.SerializableAttribute()]
    [XmlRoot("ACABulkRequestTransmitterStatusDetailResponse", Namespace = "urn:us:gov:treasury:irs:msg:irstransmitterstatusrequest")]
    public class ACABulkRequestTransmitterStatusDetailResponseType
    {
        private ACABulkRequestTransmitterResponseType aCABulkRequestTransmitterResponseField;
        private ACABulkReqTrnsmtStsRespGrpDtlType aCABulkReqTrnsmtStsRespGrpDtlField;
        private string versionField;
        public ACABulkRequestTransmitterStatusDetailResponseType()
        {
            this.versionField = "1.0";
        }
        [System.Xml.Serialization.XmlElementAttribute(Namespace = "urn:us:gov:treasury:irs:ext:aca:air:7.0", Order = 0)]
        public ACABulkRequestTransmitterResponseType ACABulkRequestTransmitterResponse
        {
            get
            {
                return this.aCABulkRequestTransmitterResponseField;
            }
            set
            {
                this.aCABulkRequestTransmitterResponseField = value;
            }
        }
        [System.Xml.Serialization.XmlElementAttribute(Namespace = "urn:us:gov:treasury:irs:ext:aca:air:7.0", Order = 1)]
        public ACABulkReqTrnsmtStsRespGrpDtlType ACABulkReqTrnsmtStsRespGrpDtl
        {
            get
            {
                return this.aCABulkReqTrnsmtStsRespGrpDtlField;
            }
            set
            {
                this.aCABulkReqTrnsmtStsRespGrpDtlField = value;
            }
        }
        [System.Xml.Serialization.XmlAttributeAttribute()]
        public string version
        {
            get
            {
                return this.versionField;
            }
            set
            {
                this.versionField = value;
            }
        }
    }

Lorrainelorrayne answered 24/5, 2016 at 20:21 Comment(7)

Did you replace the public partial class ACABulkRequestTransmitterStatusDetailResponseType with the above declared public class? Biggest change between the two is removing the PropertyChangedEventHandler and RaisePropertyChanged method? – Floatation 24/5, 2016 at 21:44

I removed all those handlers just to save space for pasting here. The bigger change was the XmlRoot attribute, to simplify deserialization. The xml serializer was unhappy with the snippet I was feeding it (via the regex match) because of the name difference between the type and the xml element, and the fact that it didn't expect that to be a root element. That attribute was the quick fix. – Lorrainelorrayne 25/5, 2016 at 13:7

Okay, I added the event handler back in. Did you keep the original attributes or slim it down to only the XmlRoot and System.SerializableAttribute()? – Floatation 25/5, 2016 at 15:35

XmlRoot and Serializable are all mine has – Lorrainelorrayne 25/5, 2016 at 17:4

Good news! My Status piece is working so far, in that it is returning the status as expected. However, how are you processing a status which has errors or that has been rejected? – Floatation 26/5, 2016 at 21:49

Is it not deserializing those types of responses properly? I had only tested a couple of scenarios when I posted this stuff, so I may not have run into any that had errors or was rejected. If it is deserializing and you're meaning more of "what do you do once you know that is the status"...I'm not sure, actually. My task was just to get the communication working, someone else is handling what to do with those responses, getting updates for retransmission, etc. – Lorrainelorrayne 27/5, 2016 at 13:30

Your Deserialize method throws an error of There is an error in XML document (1, 2). at the piece of code when rebuilding the list object. – Floatation 27/5, 2016 at 15:17

Adding the ds prefix is actually what is breaking this. The SignatureValue element contains the x509 hash of the serialized SignedInfo element (this is where having consistent whitespace matters).

By modifying the SignedInfo element, the authentication will fail when the IRS compares the SignatureValue hash you send against their computation of the SignedInfo hash using the certificate you uploaded and associated with the provided TCC.

Just remove your modification of the SignedInfo element and all should be good. It works for me.

Matusow answered 19/3, 2016 at 19:44 Comment(6)

Is your entire document formatted in "Pretty Print" (having whitespace); or are you only formatting the form data attachment? I don't doubt that you got yours to work without the ds prefix being there, and without it being necessary. I did find a way to add the prefix in an attempt to eliminate the reason for the error being the missing prefix. After I implemented this, the signature was still able to verify successfully, so it seemed to me that doing this would not negatively impact anything. – Floatation 21/3, 2016 at 15:56

My attachment is pretty with indentation and line breaks. My SOAP envelope is not and has no indentation or line breaks. I am able to fully send messages, receive a receipt, and use it query the status of the transmission. And I have received full verification from their system that the submission was fully processed successfully to completion. – Matusow 21/3, 2016 at 20:52

And is your reference for the Form Data attachment formatted as:

<urn3:ACABulkRequestTransmitter version="1.0"><urn1:BulkExchangeFile>cid:blah</urn1:BulkExchangeFile></urn3:ACABulkRequestTransmitter>

<urn1:BulkExchangeFile><xop:Include herf"cid:blah" xmlns:xop:"http://www.w3.org/2004/08/xop/include" /></urn1:BulkExchangeFile>

or something completely different? – Floatation 21/3, 2016 at 21:0

It is using xop include :) – Matusow 22/3, 2016 at 1:22

And yes, their documentation is absolutely zero help in this regard other than pointing to the w3c spec. – Matusow 22/3, 2016 at 1:22

Well, even worse is that I wrote the SOAP requests in an XML file in Visual Studio, so I could have intellisense with them. The BulkExchangeFile is the only element which wouldn't validate against their schema. I remove the Include tag, and replace it with some text, and it validates just fine because their xsd file is looking for base64Binary. I'm getting more and more frustrated with this project. I'm glad you were able to get it work, gives me hope that among all this mess there is actually a working solution. – Floatation 22/3, 2016 at 4:48

This was an edit made to the above post, which added more information to jstill's post. Unfortunately, peer reviewers rejected it.

In addition to the changes jstill made to the Status' Reference.cs file, I also had to include the BulkExchangeFileType and IncludeFileType updates he made to the Submission's Reference.cs file in order to get the Deserializer method to function partially.

The Deserializer method will return the TransmissionStatusCd and the ReceiptId elements as expected, however, it will not populate the ErrorDataFile element properly.

Since, at this time, I am unable to get the ErrorDataFile object populated properly, I am not utilizing the ACABulkRequestTransmitterStatusDetailResponseType object to capture the response returned from the Status Web Service. Instead, I have chosen to read the ResponseStream into a string object and parse the (up to) two MIME parts of the response, and process those as necessary.

Additional changes to the Status' Reference.cs

[System.CodeDom.Compiler.GeneratedCodeAttribute("System.Xml", "4.0.30319.34283")]
[System.SerializableAttribute()]
[System.Diagnostics.DebuggerStepThroughAttribute()]
[System.ComponentModel.DesignerCategoryAttribute("code")]
[System.Xml.Serialization.XmlTypeAttribute(Namespace = "urn:us:gov:treasury:irs:common")]
public partial class BulkExchangeFileType : object, System.ComponentModel.INotifyPropertyChanged
{
    private IncludeType includeField;

    /// <remarks/>
    [System.Xml.Serialization.XmlElement(Order = 0, Namespace = "http://www.w3.org/2004/08/xop/include")]
    public IncludeType Include
    {
        get { return this.includeField; }
        set
        {
            this.includeField = value;
            this.RaisePropertyChanged("Include");
        }
    }

    public event System.ComponentModel.PropertyChangedEventHandler PropertyChanged;
    protected void RaisePropertyChanged(string propertyName)
    {
        System.ComponentModel.PropertyChangedEventHandler propertyChanged = this.PropertyChanged;
        if (propertyChanged != null)
        {
            propertyChanged(this, new System.ComponentModel.PropertyChangedEventArgs(propertyName));
        }
    }
}

[System.CodeDom.Compiler.GeneratedCodeAttribute("System.Xml", "4.0.30319.34283")]
[System.SerializableAttribute()]
[System.Diagnostics.DebuggerStepThroughAttribute()]
[System.ComponentModel.DesignerCategoryAttribute("code")]
[System.Xml.Serialization.XmlTypeAttribute(Namespace = "http://www.w3.org/2004/08/xop/include")]
public partial class IncludeType : object, System.ComponentModel.INotifyPropertyChanged
{
    private System.Xml.XmlNode[] anyField;
    private string hrefField;

    /// <remarks/>
    [System.Xml.Serialization.XmlTextAttribute()]
    [System.Xml.Serialization.XmlAnyElementAttribute(Order = 0)]
    public System.Xml.XmlNode[] Any
    {
        get { return this.anyField; }
        set
        {
            this.anyField = value;
            this.RaisePropertyChanged("Any");
        }
    }

    /// <remarks/>
    [System.Xml.Serialization.XmlAttributeAttribute(DataType = "string")]
    public string href
    {
        get { return this.hrefField; }
        set
        {
            this.hrefField = value;
            this.RaisePropertyChanged("href");
        }
    } 

    public event System.ComponentModel.PropertyChangedEventHandler PropertyChanged;
    protected void RaisePropertyChanged(string propertyName)
    {
        System.ComponentModel.PropertyChangedEventHandler propertyChanged = this.PropertyChanged;
        if (propertyChanged != null)
        {
            propertyChanged(this, new System.ComponentModel.PropertyChangedEventArgs(propertyName));
        }
    }
}

Floatation answered 31/5, 2016 at 20:27 Comment(0)

For those running into the following error:

AIRMF3002 Rejected transmission - Unable to process your request because validation failure occurs in the Attachment Byte Size Number

It appears there is an issue on the IRS side with the size they are expecting (as per the Documentation) and the size they actually accept. Originally, I had the following code:

// Size in Bytes of File: This code returns the "Size" located on the File's Property Page.
// Result: TRANSMISSION REJECTED ON INCORRECT FILE SIZE!
manifestHeader.AttachmentByteSizeNum = new FileInfo(FormDataFilePath).Length.ToString();

I replaced the above code with the following, and the error I was receiving was resolved.

// Read the contents of the file, and retrieve the length of the content of the file itself..
// Result: TRANSMISSION WAS ACCEPTED USING THIS FILE SIZE.
manifestHeader.AttachmentByteSizeNum = File.ReadAllText(FormDataFilePath).Length.ToString();

It appears as though the Web Service is actually expecting the size of the file content and not the size of the actual file. The difference in size pertaining to the test scenarios was approximately 3 bytes. I assume that is because retreiving the size of the file adds some additional file-related information not belonging to the actual content.

I have notified the IRS about this issue regarding their documentation.

Floatation answered 17/6, 2016 at 20:26 Comment(0)

Hot tags

Godot Unity Godot Help Programming Godot 4.X GUI GDScript 3D 2D Physics CSharp Godot 3.X VR XR Projects C++

Recommended topics

Hot tags