How do I write to an OutputStream using DefaultHttpClient?

Question

How do I get an OutputStream using org.apache.http.impl.client.DefaultHttpClient?

I'm looking to write a long string to an output stream.

Using HttpURLConnection you would implement it like so:

HttpURLConnection connection = (HttpURLConnection)url.openConnection();
OutputStream out = connection.getOutputStream();
Writer wout = new OutputStreamWriter(out);
writeXml(wout);

Is there a method using DefaultHttpClient similar to what I have above? How would I write to an OutputStream using DefaultHttpClient instead of HttpURLConnection?

e.g

DefaultHttpClient client = new DefaultHttpClient();

OutputStream outstream = (get OutputStream somehow)
Writer wout = new OutputStreamWriter(out);

Execratory

@KeithRandall , I've edited it. Hopefully it is clear now. – Execratory

Please explain what you want to write. In both your examples, the getOutputStream() returns a stream used to submit request data for http POST requests. – Titoism

@EugeneKuleshov , How do I get an output Stream using org.apache.http.impl.client.DefaultHttpClient ? – Execratory

Answer 1

You can't get an OutputStream from BasicHttpClient directly. You have to create an HttpUriRequest object and give it an HttpEntity that encapsulates the content you want to sent. For instance, if your output is small enough to fit in memory, you might do the following:

// Produce the output
ByteArrayOutputStream out = new ByteArrayOutputStream();
Writer writer = new OutputStreamWriter(out, "UTF-8");
writeXml(writer);

// Create the request
HttpPost request = new HttpPost(uri);
request.setEntity(new ByteArrayEntity(out.toByteArray()));

// Send the request
DefaultHttpClient client = new DefaultHttpClient();
HttpResponse response = client.execute(request);

If the data is large enough that you need to stream it, it becomes more difficult because there's no HttpEntity implementation that accepts an OutputStream. You'd need to write to a temp file and use FileEntity or possibly set up a pipe and use InputStreamEntity

EDIT See oleg's answer for sample code that demonstrates how to stream the content - you don't need a temp file or pipe after all.

Mucky

Question: What is this writeXml(writer) doing? Also, how are you passing a byte array to request.setEntity()? – Outshine

@MattGrogan it's implementing the application logic to actually produce the content that gets POSTed to the remote server (copied from the example code in the question). – Mucky

So if I have the data that I need to write, should I just call writer.write(data) in place of writeXml()? – Outshine

Yes, writeXml() is computing something to pass to writer.write(). – Mucky

Sorry for the extra questions, but I really need to get this working. How are you passing out.toByteArray() to request.setEntity(), I don't see any overload that takes a byte array. – Outshine

Sorry, that should have been wrapped in a call to new ByteArrayEntity(). See edited sample code. – Mucky

Answer 2

I know that another answer has already been accepted, just for the record this is how one can write content out with HttpClient without intermediate buffering in memory.

    AbstractHttpEntity entity = new AbstractHttpEntity() {

        public boolean isRepeatable() {
            return false;
        }

        public long getContentLength() {
            return -1;
        }

        public boolean isStreaming() {
            return false;
        }

        public InputStream getContent() throws IOException {
            // Should be implemented as well but is irrelevant for this case
            throw new UnsupportedOperationException();
        }

        public void writeTo(final OutputStream outstream) throws IOException {
            Writer writer = new OutputStreamWriter(outstream, "UTF-8");
            writeXml(writer);
            writer.flush();
        }

    };
    HttpPost request = new HttpPost(uri);
    request.setEntity(entity);

Thury

Is there any other example of this process? Where does writeTo get called? – Danaedanaher

@Haraldo when the request is executed, the HttpClient framework calls request.getEntity().writeTo(). This means that you can't write a single-threaded client that starts a POST then gives its OutputStream back to the caller to invoke multiple writes. – Telium

@oleg, my android undefined writeXml() function? did you wrote it? would you please show me your writeXml() code please? – Augustaugusta

Just to clarify this a bit, regarding isStreaming() - is it correct to return false here? I would have thought that this sort of logic would be considered streaming. – Plumbaginaceous

This answer is better than the-correct-answer for memory. – Familiarity

"... means that you can't write a single-threaded client that starts a POST then gives its OutputStream back to the caller to invoke multiple writes." - What if you had such a use case? Seems you'd have no choice but to flush first to disk or use PipedStreams (multi-threaded). – Deuteron

Answer 3

This worked well on android. It should also work for large files, as no buffering is needed.

PipedOutputStream out = new PipedOutputStream();
PipedInputStream in = new PipedInputStream();
out.connect(in);
new Thread() {
    @Override
    public void run() {
        //create your http request
        InputStreamEntity entity = new InputStreamEntity(in, -1);
        request.setEntity(entity);
        client.execute(request,...);
        //When this line is reached your data is actually written
    }
}.start();
//do whatever you like with your outputstream.
out.write("Hallo".getBytes());
out.flush();
//close your streams

Plumbago

Answer 4

I wrote an inversion of Apache's HTTP Client API [PipedApacheClientOutputStream] which provides an OutputStream interface for HTTP POST using Apache Commons HTTP Client 4.3.4.

Calling-code looks like this:

// Calling-code manages thread-pool
ExecutorService es = Executors.newCachedThreadPool(
  new ThreadFactoryBuilder()
  .setNameFormat("apache-client-executor-thread-%d")
  .build());


// Build configuration
PipedApacheClientOutputStreamConfig config = new      
  PipedApacheClientOutputStreamConfig();
config.setUrl("http://localhost:3000");
config.setPipeBufferSizeBytes(1024);
config.setThreadPool(es);
config.setHttpClient(HttpClientBuilder.create().build());

// Instantiate OutputStream
PipedApacheClientOutputStream os = new     
PipedApacheClientOutputStream(config);

// Write to OutputStream
os.write(...);

try {
  os.close();
} catch (IOException e) {
  logger.error(e.getLocalizedMessage(), e);
}

// Do stuff with HTTP response
...

// Close the HTTP response
os.getResponse().close();

// Finally, shut down thread pool
// This must occur after retrieving response (after is) if interested   
// in POST result
es.shutdown();

Note - In practice the same client, executor service, and config will likely be reused throughout the life of the application, so the outer prep and close code in the above example will likely live in bootstrap/init and finalization code rather than directly inline with the OutputStream instantiation.

Deuteron