How do I send Http trailers/footers in a chunked response from within a java servlet?

Question

Basically my response headers contain

Transfer-encoding=chunked,

Trailer=[some trailer I want to send say e.g "SomeTrailer"]

Once I'm done writing the data to the Servlet outputstream, I'm writing the trailer "SomeTrailer:[value]", but this is not being parsed by the httpclient correctly. The httpclient considers the whole of inputstream (including the trailer) as a single chunk. I've also tried writing the trailer in a response header after the data has been written to the outputstream but without success.

Please help

I haven't found any good sources on this.

Gilligan

Is this really required by the client? The TE and Trailer is very rarely used, I've actually never seen it in real world code. The servlet API has builtin support for chunked responses (even more, it sends that by default when you don't set the response's content length). But it doesn't have builtin support for chunked response trailers. As an alternative, you could set the desired trailer's value as a custom response header instead (only if the value is US-ASCII compatible and does not exceed a certain max length). – Dung

@Dung My use-case is that the server is sending an infinite stream of data and doesn't know its content-length. The server code is also calculating the checksum on the fly so that it can send it as a trailer, which the client will use to verify the data. – Gilligan

Answer 1

I ended up writing a simple single threaded webserver for this. Turned out it was quite easy. The server is pretty simple. The code's a bit rough though, but the main idea is there.

What it does it sends the filecontents as the first chunk and the checksum of the file as a footer.

import java.io.BufferedReader;
import java.io.DataOutputStream;
import java.io.IOException;
import java.io.InputStream;
import java.io.InputStreamReader;
import java.net.ServerSocket;
import java.net.Socket;

import org.apache.commons.codec.digest.DigestUtils;
import org.apache.commons.io.IOUtils;
import org.apache.log4j.Logger;

public class ChunkedResponseServer implements Runnable {
  private static final Logger LOGGER = Logger.getLogger(ChunkedResponseServer.class);

  // Space ' '
  static final byte SP = 32;
  // Tab ' '
  static final byte HT = 9;
  // Carriage return
  static final byte CR = 13;
  // Line feed character
  static final byte LF = 10;

  final int port;

  private volatile boolean cancelled = false;

  public ChunkedResponseServer(int port) {
    LOGGER.info("Chunked response server running on port " + port);
    this.port = port;
  }

  @Override
  public void run() {
    ServerSocket serverSocket = null;
    try {
      serverSocket = new ServerSocket(port);
      while (!cancelled) {
        final Socket connectionSocket = serverSocket.accept();
        handle(connectionSocket);
      }
    } catch (final IOException e) {
      throw new RuntimeException(e);
    }
  }

  public void cancel() {
    LOGGER.info("Shutting down Chunked response Server");
    cancelled = true;
  }

  private void handle(Socket socket) throws IOException {
    BufferedReader input = null;
    DataOutputStream output = null;
    try {
      input = new BufferedReader(new InputStreamReader(socket.getInputStream()));
      output = new DataOutputStream(socket.getOutputStream());

      addHeaders(output);
      addCRLR(output);

      final String filename = readFilename(input);
      final byte[] content = readContent(filename);
      addContentAsChunk(output, content);

      final String checksum = DigestUtils.md5Hex(content);
      addLastChunkAndChecksumFooter(output, checksum);
      addCRLR(output);

    } finally {
      IOUtils.closeQuietly(input);
      IOUtils.closeQuietly(output);
    }
  }

  private void addLastChunkAndChecksumFooter(DataOutputStream output, String checksum) throws IOException {
    output.writeBytes("0");
    addCRLR(output);
    output.writeBytes("checksum: " + checksum);
    addCRLR(output);
  }

  private void addContentAsChunk(DataOutputStream output, byte[] content) throws IOException {
    output.writeBytes(Integer.toHexString(content.length));
    addCRLR(output);
    output.write(content);
    addCRLR(output);
  }

  private void addCRLR(DataOutputStream output) throws IOException {
    output.writeByte(CR);
    output.writeByte(LF);
  }

  private void addHeaders(DataOutputStream output) throws IOException {
    output.writeBytes("HTTP/1.1 200 OK");
    addCRLR(output);
    output.writeBytes("Content-type: text/plain");
    addCRLR(output);
    output.writeBytes("Transfer-encoding: chunked");
    addCRLR(output);
    output.writeBytes("Trailer: checksum");
    addCRLR(output);
  }

  private String readFilename(BufferedReader input) throws IOException {
    final String initialLine = input.readLine();
    final String filePath = initialLine.split(" ")[1];
    final String[] components = filePath.split("/");
    return components[components.length - 1];
  }

  private byte[] readContent(String filename) throws IOException {
    final InputStream in = Thread.currentThread().getContextClassLoader().getResourceAsStream(filename);
    return IOUtils.toByteArray(in);
  }
}

Gilligan