Adding an attachment to SOAP request

Question

I am at a loose end as to how to add an attachment in my SOAP request. We have to consume a thrid party web service, built in java, which is the most convoluted thing I have ever come across. Any other web services we have used, that required attachments, have a method or property to add the attachment. Simple. However, this one provides no such method.

We have got a version of the SOAP message together that is exactly as we want the XML, however it is the MIME part of the file that we cannot add.

Example:

<soap:Envelope xmlns:soap="http://schemas.xmlsoap.org/soap/envelope/" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:wsa="http://schemas.xmlsoap.org/ws/2004/08/addressing">
<soap:Header>
<payloadManifest xmlns="http://<examplePayload>">
<manifest contentID="Content0" namespaceURI="http://<exampleManifest>" element="ProcessRepairOrder" version="2.01" />
</payloadManifest>
<wsse:Security xmlns:wsse="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-secext-1.0.xsd">
<wsu:Timestamp xmlns:wsu="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-utility-1.0.xsd">
<wsu:Created>2011-12-19T15:25:13Z</wsu:Created>
<wsu:Expires>2011-12-19T15:30:00Z</wsu:Expires>
</wsu:Timestamp>
<wsse:UsernameToken><wsse:Username>username</wsse:Username><wsse:Password>password</wsse:Password></wsse:UsernameToken></wsse:Security></soap:Header><soap:Body><ProcessMessage xmlns="<examplePayload"><payload><content id="Content0">

<s:ProcessRepairOrder xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://example.xsd" xmlns:s="http://<exampleManifest>" xmlns:gwm="http://example">
    <s:ApplicationArea>
        <s:Sender>
            <s:Component>Test</s:Component>
            <s:Task>ProcessAttachment</s:Task>
            <s:CreatorNameCode>Test</s:CreatorNameCode>
            <s:SenderNameCode>XX</s:SenderNameCode>
            <s:DealerNumber>111111</s:DealerNumber>
            <s:DealerCountry>GB</s:DealerCountry>
        </s:Sender>
        <s:CreationDateTime>2010-03-26T13:37:05Z</s:CreationDateTime>
        <s:Destination>
            <s:DestinationNameCode>GM</s:DestinationNameCode>
            <s:DestinationURI/>
            <s:DestinationSoftwareCode>GWM</s:DestinationSoftwareCode>
        </s:Destination>
    </s:ApplicationArea>
    <s:DataArea xsi:type="gwm:DataAreaExtended">
        <s:Process/>
        <s:RepairOrder>
            <s:Header xsi:type="gwm:RepairOrderHeaderExtended">
                <s:DocumentId/>
            </s:Header>
            <s:Job xsi:type="gwm:JobExtended">
                <s:JobNumber/>
                <s:OperationId>Test</s:OperationId>
                <s:OperationName/>
                <s:CodesAndComments/>
                <s:Diagnostics/>
                <s:WarrantyClaim xsi:type="gwm:WarrantyClaimExtended">
                    <s:OEMClaimNumber>00112233445566778899</s:OEMClaimNumber>
                    <gwm:Attachment>
                        <gwm:File><xop:Include xmlns:xop="http://www.w3.org/2004/08/xop/include" href="cid:test.gif"/></gwm:File>
                        <gwm:Filename>test.gif</gwm:Filename>
                    </gwm:Attachment>
                </s:WarrantyClaim>
                <s:LaborActualHours>0.0</s:LaborActualHours>
                <s:Technician/>
            </s:Job>
        </s:RepairOrder>
    </s:DataArea>
</s:ProcessRepairOrder>
</content></payload></ProcessMessage></soap:Body></soap:Envelope>

This is the XML part that we can generate and send off, however it is incorrect as we need a MIME part in there like:

Before XML:

--MIMEBoundary
Content-Type: application/xop+xml; charset=utf-8; type="text/xml"
Content-Transfer-Encoding: binary
Content-ID: <rootpart>

After XML

--MIMEBoundary
Content-Type: image/gif; name=test.gif
Content-Transfer-Encoding: binary
Content-ID: <test.gif>
GIF89a@�

--MIMEBoundary--

I have scoured the internet for answers but have come up blank. There doesn't seem to be much documentation around using WSE for this. I must stress that WSE is a requirement on the server side, and there is no way I can change the technology to address this issue.

Is there a way that these MIME sections can be added?

EDIT: I must add that I can get a working XML document sent through SoapUI with attachments, but cannot seem to find a way within our code.

I have added a bounty to try and get a solution to this problem. If anyone has any other ideas please let me know.

EDIT again: I know it has been a week since I was able to check the responses here, but while some give a good idea where to look I am still drawing a blank. The terrible documentation surrounding XopDocument and its methods is a big sticking point, if anyone has any examples of using SaveToXopPackage could they please provide because this is beginning to grate!

Onanism

WSE is obsolete and should not be used no matter what. – Decompose

Well John, how about coming up with a solution rather than going around just pretending you know everything. In this scenario, for me, WSE IS required. Without it, our requests will be rejected. I can't tell the third party vendor how they should be coding, I can try but they have always been the least helpful company in the world to work around. So, while WSE should not be used if you are creating your own software, it is a requirement in this instance. – Onanism

Clearly it's a business decision as to whether your company should continue to do business with a third party that requires you to use obsolete software, versus some competitor which does not. – Decompose

Clearly, but when its a matter of a much bigger company than ours, that we have a huge reliance on a lot of income, decides upon this we can't do anything about it. An example of how this working relationship might be viewed, say we are a small chain of sports shops and Adidas want us to do this, we can't just tell them that we are not using WSE and drop the brand from our stores. We are not a sports store and they are not Adidas, but it is a very similar situation. It's not as easy as just not working with WSE... – Onanism

I have little to say about that except that I hope someone has told the Emperor his clothes are obsolete. Also, if you pretend that using WSE is not an option, then you'll no doubt learn how to customize WCF to do what you need, only by using supported software. – Decompose

Answer 1

I was facing the same problem and the final solution I found was through HttpWebRequest. A sample code:

    public string ProcessAttachment(string fileInput)
    {
        HttpWebRequest req = (HttpWebRequest)WebRequest.Create(Settings.Default.GWM_WS_WebReference_GWM);
        req.Headers.Add("SOAPAction", "\"http://www.starstandards.org/webservices/2005/10/transport/operations/ProcessMessage/v1_01/ProcessAttachment\"");
        req.Headers.Add("Accept-Encoding", "gzip,deflate");
        req.ContentType = "multipart/related; type=\"application/xop+xml\"; start=\"<[email protected]>\"; start-info=\"text/xml\"; boundary=\"----=_Part_14_1350106.1324254402199\"";
        req.Method = "POST";
        req.UserAgent = "Jakarta Commons-HttpClient/3.1";
        req.Headers.Add("MIME-Version", "1.0");
        System.Net.ServicePointManager.Expect100Continue = false;
        Stream memStream = new System.IO.MemoryStream();
        FileStream fileStream = new FileStream(fileInput, FileMode.Open, FileAccess.Read);
        byte[] buffer = new byte[1024];
        int bytesRead = 0;
        while ((bytesRead = fileStream.Read(buffer, 0, buffer.Length)) != 0)
        {
            memStream.Write(buffer, 0, bytesRead);
        }
        fileStream.Close();
        Stream stm = req.GetRequestStream();
        memStream.Position = 0;
        byte[] tempBuffer = new byte[memStream.Length];
        memStream.Read(tempBuffer, 0, tempBuffer.Length);
        memStream.Close();
        stm.Write(tempBuffer, 0, tempBuffer.Length);
        stm.Close();
        HttpWebResponse resp = null;
        resp = (HttpWebResponse)req.GetResponse();
        stm = resp.GetResponseStream();
        StreamReader r = new StreamReader(stm);
        return r.ReadToEnd();            
    }

The parameter fileInput is the absolute path of the file that contains the SOAP Request containing also the raw binary data of the file to be attached at the end separated with MIME Boundary

Seemly

I don't really understand how this works. If you are saying include the raw data in the file inline, then that is not possible, in my scenario. It has to be contained in an XOP section like defined in the example I gave. If I am missing this in your code please advise. – Onanism

The file should have the 2 parts that you are describing in the first post: The XML part and the Ras data part separated with Mime Boundary. The Raw data part meaning your "GIF89a@�" part should be replaced as simple string using the below code: byte[] bt=File.ReadAllBytes(imageFile);string raw=System.Text.Encoding.Unicode.GetString(bt); Better contact me directly in order to explain,since I think that this goes beyond stackoverflow scope. – Seemly

HttpWebRequest req = (HttpWebRequest)WebRequest.Create(Settings.Default.GWM_WS_WebReference_GWM); I am unavailable to get Settings.Default.GWM_WS_WebReference_GWM how can I use this or is there any alternate for this – Famulus

Answer 2

I think that you may have a couple of options:

1) Use MTOM. This appears to automatically wrap the outgoing message in MIME blocks.

2) Microsoft actually provides support for generating and reading XOP with mime through the XopDocument class, which is what SoapEnvelope inherits from.

The save method is SaveToXopPackage and the read method is LoadFromXopPackage.

However, I think that this approach may require you to perform the sending of the message yourself through an HttpWebRequest. This blog has an example of how to implement this. The downside is that this requires a lot of extra code and configuration in order to work correctly.

The ideal solution would be to intercept the code that performs the envelope transmission, but I have been unable to locate the correct location for this in the pipeline.

Inconceivable

These are just a few of the number of references I have used when trying to get this to work. I can get it to send a MIME enclosed block, but I don't know how to add the binary data of the attachment to the MIME part. – Onanism

Also, there might as well be no documentation for SaveToXopPackage, Google returns 28 results for it, the majority of which are this exact page! – Onanism

Answer 3

I'm 90% confident I'm working on the exact same project as you guys. That soap request is a little too familiar :-)

We've gotten most of the way there by switching over to WCF and basically hand-coding the request object (creating classes that match the soap format and then using xmlelement attributes to decorate it so that it looks like their soap request. The file itself is declared as a Byte() on the Attachment class and also decorated with the xmlelement).

Here's what the WCF contract and part of the data model look like. The actual data model has a bunch of extra classes (Application Area, Data Area, Job, etc) but this gives you enough of a sense of how it's structured. The important piece is the File as Byte(). Here it is in Vb.net...

Public Class WarrantyClaim
    <XmlElement(Order:=0)> Public OEMClaimNumber As String = ""
    <XmlElement(Order:=1, namespace:="http://www.gm.com/2006/GWM")> Public Attachment As New Attachment
End Class

Public Class Attachment
    <XmlElement(Order:=0)> Public File As Byte()
    <XmlElement(Order:=1)> Public Filename As String
End Class

<ServiceContract(XmlSerializerFormat()> _
Public Interface IService
    <OperationContract(action:="http://www.starstandards.org/webservices/2005/10/transport/operations/ProcessMessage/v1_01/ProcessAttachment")> _
    Sub ProcessMessage(ByVal payload As WarrantyClaim)
End Interface

Next you've got your WCF client, this is pretty much the same as all WCF clients.

Public Class GmgwClient
    Inherits System.ServiceModel.ClientBase(Of IService)
    Implements IService

    Public Sub New()
        MyBase.New()
    End Sub
    Public Sub New(ByVal configName As String)
        MyBase.New(configName)
    End Sub
    Public Sub New(ByVal binding As System.ServiceModel.Channels.Binding, ByVal remoteAddress As System.ServiceModel.EndpointAddress)
        MyBase.New(binding, remoteAddress)
    End Sub

    Public Sub ProcessMessage(ByVal payload As Payload) Implements IService.ProcessMessage
        MyBase.Channel.ProcessMessage(payload)
    End Sub
End Class

Finally you've got the app.config. Here's the magic because we're telling WCF to use Mtom to send the message. This will take the Byte() and strip it out into a seperate MIME section replacing it with an XOP:Include. Note that for now I'm just sending it through localhost so I can see the request using tcpTrace. You can google that app but it'll basically capture the request so we can see how it looks. I setup tcpTrace to listen on port 84.

<system.serviceModel>
  <bindings>
    <wsHttpBinding>
      <binding name="WsHttpMtomBinding" messageEncoding="Mtom">
        <security mode="None">
          <transport clientCredentialType="Basic" proxyCredentialType="None" realm="" />
        </security>
        <reliableSession enabled="false" />
      </binding>
    </wsHttpBinding>
  </bindings>
  <client>
    <endpoint address="http://localhost:84/ProcessMessage" binding="wsHttpBinding" bindingConfiguration="WsHttpMtomBinding" contract="MyAppNameSpace.IService" name="preprod"/>
  </client>
</system.serviceModel>

Finally, here's the actual call to the WCF client to make the request.

Dim x As New WarrantyClaim
x.OEmClaimNumber = "12345"
x.Attachment = New Attachment
x.Attachment.Filename = "sample.gif"
x.Attachment.File = IO.File.ReadAllBytes("C:\sample.gif")

Dim y As New GmgwClient("preprod")
y.ProcessMessage(x)

And here's the trace we got through tcpTrace. It's got the basic structure right and it's managed to pull the binary data out of the xml and place it in a seperate MIME section.

POST /ProcessMessage HTTP/1.1
MIME-Version: 1.0
Content-Type: multipart/related; type="application/xop+xml";start="<http://tempuri.org/0>";boundary="uuid:501aa27d-9dd1-4f8a-b56d-3fbf327e7be6+id=1";start-info="application/soap+xml"
VsDebuggerCausalityData: uIDPoysDMCv023ZIjK0Cpp504ooAAAAA//jfaCaohkab2Zx/EU7gpLZDcUldWtlGr1j4ZnrfKl4ACQAA
Host: localhost:84
Content-Length: 55125
Expect: 100-continue
Accept-Encoding: gzip, deflate
Connection: Keep-Alive


--uuid:501aa27d-9dd1-4f8a-b56d-3fbf327e7be6+id=1
Content-ID: <http://tempuri.org/0>
Content-Transfer-Encoding: 8bit
Content-Type: application/xop+xml;charset=utf-8;type="application/soap+xml"

<s:Envelope xmlns:s="http://www.w3.org/2003/05/soap-envelope" xmlns:a="http://www.w3.org/2005/08/addressing">
  <s:Header>
    <a:Action s:mustUnderstand="1">http://www.starstandards.org/webservices/2005/10/transport/operations/ProcessMessage/v1_01/ProcessAttachment</a:Action>
    <a:MessageID>urn:uuid:a85374e6-c8ca-4328-ad32-6e8b88a5ca59</a:MessageID>
    <a:ReplyTo>
      <a:Address>http://www.w3.org/2005/08/addressing/anonymous</a:Address>
    </a:ReplyTo>
    <a:To s:mustUnderstand="1">http://localhost:84/ProcessMessage</a:To>
  </s:Header>
  <s:Body xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xsd="http://www.w3.org/2001/XMLSchema">
    <ProcessMessage xmlns="http://www.starstandards.org/webservices/2005/10/transport">
      <payload xsi:type="gwm:WarrantyClaimExtended">
        <OEMClaimNumber>12345</OEMClaimNumber>
        <Attachment xmlns="http://www.gm.com/2006/GWM">
          <File>
            <xop:Include href="cid:http%3A%2F%2Ftempuri.org%2F1%2F634618782531246992" xmlns:xop="http://www.w3.org/2004/08/xop/include"/>
          </File>
          <Filename>sample.gif</Filename>
        </Attachment>
      </payload>
    </ProcessMessage>
  </s:Body>
</s:Envelope>
--uuid:501aa27d-9dd1-4f8a-b56d-3fbf327e7be6+id=1
Content-ID: <http://tempuri.org/1/634618782531246992>
Content-Transfer-Encoding: binary
Content-Type: application/octet-stream

GIF89a<BinaryStuff>

Like I mentioned earlier - we've still got some issues. There are some tags missing from the Soap Header... but I think we'll be able to figure those out. The real problem is that the Content-ID is NOT in a format our partner can accept - they expect something like <[email protected]> and .net is formatting them as http://tempuri.org/1/634618782531246992. This is causing their Web Service handler to crash because it doesn't know how to read escaped content-id's inside the soap message.

Tenishatenn

Just noticed your edit for this now. Yeah it does look like the exact same project! Unfortunately your solution isn't going to work in our case as we are restricted to using WSE and not WCF...Reliance on VS 2005. It is a pain. But this is the result I am looking for but need to figure out the solution in WSE. :( However, with your problem, can you not set the Content-ID to anything you want? This works within SoapUI, when you specify the content-ID yourself and not relying on the default representation? My email address, not work one, is in my profile if you want to converse further. – Onanism

Answer 4

As you say you got it working through SoapUI, I would think you can just ask SoapUI for the generated XML it sent so you know how it should look, then modify your code to mimic that.

UPDATE: after your comment and reading the other answers in more detail: the solution looks to me just sending bytes directly, using HttpWebRequest like in ktsiolis's answer. In detail:

• Create your SOAP XML (the example you gave), encode this to bytes in UTF8 (1)
• Create a string with the initial mimeboundary (the part in your "Before XML"), encode to bytes in UTF8 (2)
• Create the bytes for the second mimeboundary (the part in "after XML"). So create the string containing "--MIMEBOUNDARY" etc., encode to UTF8 bytes, and append all bytes of your test.gif file (3)
• Append all bytes in the order (2), (1) and (3) and send that across the wire.

Shouldn't this do the trick?

Fight

Seriously? I know exactly how it should look, I just don't know how I can get the binary data attached to the MIME part. – Onanism

Answer 5

Ok so I got it to accept the data from the file in the <gwm:File> element. This is without using XOP, so the request now looks like:

<soap:Envelope xmlns:soap="http://schemas.xmlsoap.org/soap/envelope/" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:wsa="http://schemas.xmlsoap.org/ws/2004/08/addressing">  <soap:Header>  <payloadManifest xmlns="http://<examplePayload>">  <manifest contentID="Content0" namespaceURI="http://<exampleManifest>" element="ProcessRepairOrder" version="2.01" />  </payloadManifest>  <wsse:Security xmlns:wsse="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-secext-1.0.xsd"> <wsu:Timestamp xmlns:wsu="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-utility-1.0.xsd"> <wsu:Created>2011-12-19T15:25:13Z</wsu:Created>  <wsu:Expires>2011-12-19T15:30:00Z</wsu:Expires>  </wsu:Timestamp>  <wsse:UsernameToken><wsse:Username>username</wsse:Username><wsse:Password>password</wsse:Password></wsse:UsernameToken></wsse:Security></soap:Header><soap:Body><ProcessMessage xmlns="<examplePayload"><payload><content id="Content0">    <s:ProcessRepairOrder xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://example.xsd" xmlns:s="http://<exampleManifest>" xmlns:gwm="http://example"> 
    <s:ApplicationArea> 
        <s:Sender> 
            <s:Component>Test</s:Component> 
            <s:Task>ProcessAttachment</s:Task> 
            <s:CreatorNameCode>Test</s:CreatorNameCode> 
            <s:SenderNameCode>XX</s:SenderNameCode> 
            <s:DealerNumber>111111</s:DealerNumber> 
            <s:DealerCountry>GB</s:DealerCountry> 
        </s:Sender> 
        <s:CreationDateTime>2010-03-26T13:37:05Z</s:CreationDateTime> 
        <s:Destination> 
            <s:DestinationNameCode>GM</s:DestinationNameCode> 
            <s:DestinationURI/> 
            <s:DestinationSoftwareCode>GWM</s:DestinationSoftwareCode> 
        </s:Destination> 
    </s:ApplicationArea> 
    <s:DataArea xsi:type="gwm:DataAreaExtended"> 
        <s:Process/> 
        <s:RepairOrder> 
            <s:Header xsi:type="gwm:RepairOrderHeaderExtended"> 
                <s:DocumentId/> 
            </s:Header> 
            <s:Job xsi:type="gwm:JobExtended"> 
                <s:JobNumber/> 
                <s:OperationId>Test</s:OperationId> 
                <s:OperationName/> 
                <s:CodesAndComments/> 
                <s:Diagnostics/> 
                <s:WarrantyClaim xsi:type="gwm:WarrantyClaimExtended"> 
                    <s:OEMClaimNumber>00112233445566778899</s:OEMClaimNumber> 
                    <gwm:Attachment> 
                        <gwm:File>GIF89a@�</gwm:File> 
                        <gwm:Filename>test.gif</gwm:Filename> 
                    </gwm:Attachment> 
                </s:WarrantyClaim> 
                <s:LaborActualHours>0.0</s:LaborActualHours> 
                <s:Technician/> 
            </s:Job> 
        </s:RepairOrder> 
    </s:DataArea>  </s:ProcessRepairOrder>  </content></payload></ProcessMessage></soap:Body></soap:Envelope>

When passed into SoapUI this works perfectly, however in code it does give a response, but it throws an error saying Response is not well-formed XML. with an inner exception of WSE1608: No XOP parts were located in the stream for the specified content-id: <rootpart*[email protected]>

I will be opening a new question regarding this as it is technically a different issue.

The other question can be found at Soap response, not well formed XML, no XOP parts located, using WSE

Onanism

Answer 6

I am involved in exactly the same project and I have the same issues as discussed in this thread! I am using vb 2005 and WSE 3.0 enhancements and I got it working even it is a pain for now. When writing the content of the file directly in the File Property, the attachment will be accepted by the partner. In my case, this works for almost all transactions except PRA's. Here, the response is positive and an AttachmentID will be delivered but the attachment does not appear in the transaction.

Here is an example of the Attachment section:

                <gwm:Attachment>
                  <gwm:File>/9j/4AAQSkZJRgABAQEAYABgAAD/2wBDAAEBAQEBAQEBAQEBAQEBAQEBAQEBAQEBAQEBAQEBAQEBAQEBAQEBAQEBAQEBAQ...</gwm:File>
                  <gwm:Filename>intro2.jpg</gwm:Filename>
                </gwm:Attachment>

If I set RequireMtom for the Service to True, I'll get the following error:

Das Präfix '' kann nicht von '' in 'http://www.starstandards.org/webservices/2005/10/transport' innerhalb desselben Startelementtags neu definiert werden.

One the one hand, it works, on the other hand, I am not sure if it will be sent with XOP elements.

Disjuncture

I had a discussion with the web service developers about putting the data directly in the <File> element and they said this does not conform to their specification and they require a <xop:Include> element. See #8805595 for a further description of the problem we are having. If you want to discuss further outside of here please see my profile for my email address. – Onanism

Sure...but I can not find your email address. – Disjuncture

If you can't see it under bio, it is now in the About Me section in my profile. – Onanism

Thank you ... I've sent you an email. – Disjuncture