SOAP message to webservice - HTTP response code: 403 for URL

Question

I try to send a SOAP message in an XML file to a webservice and than grab the binary output and decode it. Endpoint uses HTTPS protocol, so I used TrustManager in my code to avoid PKIX problems. You can see my code here:

import javax.net.ssl.*;
import java.io.*;
import java.net.HttpURLConnection;
import java.net.URL;
import java.net.URLConnection;
import java.security.cert.X509Certificate;

public class Main{
    public static void sendSoapRequest() throws Exception {
        String SOAPUrl = "URL HERE";
        String xmlFile2Send = ".\\src\\request.xml";
        String responseFileName = ".\\src\\response.xml";
        String inputLine;

        TrustManager[] trustAllCerts = new TrustManager[] { new X509TrustManager() {
            public java.security.cert.X509Certificate[] getAcceptedIssuers() { return null; }
            public void checkClientTrusted(X509Certificate[] certs, String authType) { }
            public void checkServerTrusted(X509Certificate[] certs, String authType) { }

        } };

        SSLContext sc = SSLContext.getInstance("SSL");
        sc.init(null, trustAllCerts, new java.security.SecureRandom());
        HttpsURLConnection.setDefaultSSLSocketFactory(sc.getSocketFactory());

        // Create all-trusting host name verifier
        HostnameVerifier allHostsValid = new HostnameVerifier() {
            public boolean verify(String hostname, SSLSession session) { return true; }
        };
        // Install the all-trusting host verifier
        HttpsURLConnection.setDefaultHostnameVerifier(allHostsValid);

        // Create the connection with http
        URL url = new URL(SOAPUrl);
        URLConnection connection = url.openConnection();
        HttpURLConnection httpConn = (HttpURLConnection) connection;
        FileInputStream fin = new FileInputStream(xmlFile2Send);
        ByteArrayOutputStream bout = new ByteArrayOutputStream();

        copy(fin, bout);
        fin.close();

        byte[] b = bout.toByteArray();
        StringBuffer buf=new StringBuffer();
        String s=new String(b);

        b=s.getBytes();

        // Set the appropriate HTTP parameters.
        httpConn.setRequestProperty("Content-Length", String.valueOf(b.length));
        httpConn.setRequestProperty("Content-Type", "text/xml; charset=utf-8");
        httpConn.setRequestProperty("SOAPAction", "");
        httpConn.setRequestMethod("POST");
        httpConn.setDoOutput(true);

        OutputStream out = httpConn.getOutputStream();
        out.write(b);
        out.close();

        // Read the response.
        httpConn.connect();
        System.out.println("http connection status :"+ httpConn.getResponseMessage());
        InputStreamReader isr = new InputStreamReader(httpConn.getInputStream());
        BufferedReader in = new BufferedReader(isr);

        while ((inputLine = in.readLine()) != null)
            System.out.println(inputLine);
        FileOutputStream fos=new FileOutputStream(responseFileName);
        copy(httpConn.getInputStream(),fos);
        in.close();
    }

    public static void copy(InputStream in, OutputStream out) throws IOException {

        synchronized (in) {
            synchronized (out) {
                byte[] buffer = new byte[256];
                while (true) {
                    int bytesRead = in.read(buffer);
                    if (bytesRead == -1)
                        break;
                    out.write(buffer, 0, bytesRead);
                }
            }
        }
    }

    public static void main(String args[]) throws Exception {
        sendSoapRequest();
    }
}

I get following error code, when I execute this.

Exception in thread "main" java.io.IOException: Server returned HTTP response code: 403 for URL

Strut

You should provide credentials to gain access. Authentication header with login/password in base64 for basic authentication, simplest variant, but it depends of server side. You should provide what server required. – Arlyn

@Arlyn : No need for authentication on this server. – Strut

403-forbidden means that the request has reached the server and is valid but the server has denied the access to the requested resource. Summarizing, the SSL connection is ok, so you are invoking to the wrong endpoint or there are missing credentials in the SOAP header. – Amphetamine

@Amphetamine : I see, but when I use content of XML files in a SOAP request with SoapUI, I get response from the same endpoint. – Strut

If the content is the same, then inspect and compare the headers that is actually sending SOAPUI and your connection. It is possible that the server detects some incorrectness and treats it as 403 – Amphetamine

How to inspect the header with Java code? – Strut

@Amphetamine : Is there any other method to send a XML file to a given webservice and catch the repsonse? – Strut

You can use a local proxy like fiddler to inspect the requests that are being sent from your code. You can also use the chrome plugin Postman to invoke a URL providing content, headers, etc. – Amphetamine

@Amphetamine : Postman is only for REST as I know. – Strut

A SOAP request is sent over HTTP using a POST-like request. You can use postman perfectly for this – Amphetamine

@Amphetamine : I tried it with Postman with same content as in request.xml file and I got the response. I sent request as application/xml. It is binary, so couldn't read it. – Strut

If with postman and SOAP UI works well, then there is a problem with your Java code. Have you tried using application/xml instead of text/xml in the java code? Did you inspect the headers/payload with a proxy? If you did not install a proxy, try to post the request to an endpoint created here: requestb.in – Amphetamine

@Amphetamine : I tried Requestb.in, but body seems empty, it says: None. I tried with application/xml too. – Strut

It could be that you were sending an empty body. Try to remove String s=new String(b); b=s.getBytes(); and add out.flush(); before out.close() – Amphetamine

I removed and added mentioned lines, but same output: "http connection status :Forbidden" – Strut

Are you sure the URL used by SoapUI (from the wsdl) and the one you have in the code is the same (and has not funny encodings?) – Hallette

Answer 1

Your implementation is alright, the problem is related to your Content-Type header, in fact.

The value text/xml; charset=utf-8 is the default Content-Type of SOAP 1.1, which is probably not the version of yours. SOAP 1.2 expects a header of type application/soap+xml; charset=utf-8, so changing your line of code to this one below is gonna make it working:

httpConn.setRequestProperty("Content-Type", "application/soap+xml; charset=utf-8");

---

In SoapUI, it's possible to check the headers calling the request and going to the Headers tab on the bottom of the window:

Then, you can compare the differences between your application configs and the SoapUI ones.

Chiefly

I can't find any headers in SoapUI and I get the exact same error with this Content-Type too. – Strut

The headers are on the response side, not in the request. Make a request and check there. – Chiefly

It is application/timestamp-reply on the response side. – Strut

Is there possible for you to share the endpoint? Or at least the service project at GitHub? It's very hard to guess the specificities of your implementation. – Chiefly

Sorry, but it is reacheable only from local network. – Strut

@bosco : It solved my similar problem. Thanks a ton!. – Wiebmer

Answer 2

403 error might be related to your soap request headers being sent to the server. All Hosts valid will allow your Java App to trust the SSL Cert for the URL. Check if your server is expecting soap header with username/password. If you have access to this server, you can check through the web server logs on where your request is failing. Error code points to to missing Soap Header particularly Soap Headers with username and password

Batfish

No username/password is needed, it is not set in the current SoapUI request. – Strut

Answer 3

Wonder if your SOAP request contains any kind of authentication information in headers like SAML. One option is, in your above code where you read the file and send the data to server, instead of sending it to server you dump it to another file. Dump that byteoutputstream. Then copy text from that file and put it in SOAP UI and try running that. Does that work?

Placebo

Why is it better to make another file with the same content? – Strut

That's specifically in case your SOAP request contains SAML or some other binary data. That's to see if it is getting corrupted during file IO. Your code contains this small piece: byte[] b = bout.toByteArray(); StringBuffer buf=new StringBuffer(); String s=new String(b); b=s.getBytes(); where you are converting the data to String and back to bytes. Is there some specific need to do that? – Placebo

As @Amphetamine suggested, I deleted these lines earlier from code. – Strut

Oh, ok. And were you able to capture the headers being sent from SOAP UI vs the ones being sent from your code. Are those same too. – Placebo

Yes, I sent request to Requestbin, but body was empty. Is there something wrong with my Java code? – Strut

Can't say about body as your request is over SSL so the data is not really visible over wire, except as something unreadable. That might be why Requestbin is not displaying it. But just to be sure, I had requested you to try printing the payload. Dump it in a file or print it in console as you may wish. – Placebo

As an afterthought, try closing the output stream at the end of the processing. I think I faced similar issue of data not being sent over the wire, years ago. I checked and it works with Java 7 that I have, yet just try if you may. – Placebo

Answer 4

In a similar situation we have been some time before, and as long as trying TrustManager didn't work as expected, we managed to overcome this problem by installing the certificate from server to JVM's keystore (JVM used to run the application). More information about how to do it you can find in several posts, like How to import a .cer certificate into a java keystore?

I am aware that it is a try to force JVM to accept SSL certificates, and this functionality would be better to live in application context, but as long as we were building a web application which ran in specific application servers, the implemented solution was an accepted one.

Gasperoni