How to stream large HTTP response in spring

Question

We have a HTTP request which when handled in server will create a response approximately 3GB in size, this data is an aggregation of 6 queries to database, how can we send this data as individual responses of 6 queries than as an aggregation.

Idola

You can only return a single response from a single request. – Orangery

Answer 1

StreamingResponseBody is used for asynchronous request processing where the application can write directly to the response OutputStream.

Checkout this article

http://www.logicbig.com/how-to/code-snippets/jcode-spring-mvc-streamingresponsebody/

http://shazsterblog.blogspot.in/2016/02/asynchronous-streaming-request.html

Eparchy

attaching a relevant post about trouble handling exception with StreamingResponseBody https://mcmap.net/q/412269/-exception-handling-in-streamingresponsebody-not-working/5777189 – Peroneus

Answer 2

I did this :

 @GetMapping("/{fileName:[0-9A-z]+}")
    @ResponseBody
    public ResponseEntity<InputStreamResource> get_File(@PathVariable String fileName) throws IOException {
        Files dbFile = fileRepository.findByUUID(fileName);

        if (dbFile == null)
            return new ResponseEntity<>(HttpStatus.NOT_FOUND);

        String filename = dbFile.getFileName();
        Resource file = storageService.loadAsResource(dbFile.getFileName());


        long len = 0;
        try {
            len = file.contentLength();
        } catch (IOException e) {
            e.printStackTrace();
        }


        MediaType mediaType = MediaType.valueOf(FileTypeMap.getDefaultFileTypeMap().getContentType(file.getFile()));

        if (filename.toLowerCase().endsWith("mp4") || filename.toLowerCase().endsWith("mp3") ||
                filename.toLowerCase().endsWith("3gp") || filename.toLowerCase().endsWith("mpeg") ||
                filename.toLowerCase().endsWith("mpeg4"))
            mediaType = MediaType.parseMediaType("application/octet-stream");


        InputStreamResource resource = new InputStreamResource(new FileInputStream(file.getFile()));

        return ResponseEntity.ok()
                .contentType(mediaType)
                .contentLength(len)
                .header(HttpHeaders.CONTENT_DISPOSITION, "attachment; filename=\"" + file.getFilename() + "\"")
                .body(resource);
    }

Moira

What about closing the file resource? – Sunn

The spring framework closes the FileInputStream: github.com/spring-projects/spring-framework/blob/v4.3.9.RELEASE/… – Butler

Answer 3

Call an API that is returning streams and then further you need to send the streamed response back to your client.

@PostMapping(value = "/path",produces =MediaType.APPLICATION_OCTET_STREAM_VALUE)
public ResponseEntity<StreamingResponseBody> downloadExtract(@RequestBody(required = false) Object requestObj,HttpServletResponse response) throws IOException {
    // Create the request body

    String requestBody = new ObjectMapper().writeValueAsString(requestObj);
    response.setContentType(MediaType.APPLICATION_OCTET_STREAM_VALUE);
    StreamingResponseBody responseBody = outputStream -> {
        // Make the HTTP request with the request body
        restTemplate.execute(
                url,
                HttpMethod.POST,
                requestCallback -> {
                    // Set the request body
                    OutputStream requestBodyStream = requestCallback.getBody();
                    requestBodyStream.write(requestBody.getBytes());
                },
                responseExtractor -> {
                    // Stream the response chunks directly to the client
                    InputStream responseStream = responseExtractor.getBody();
                    byte[] buffer = new byte[1024];
                    int bytesRead;
                    while ((bytesRead = responseStream.read(buffer)) != -1) {
                        outputStream.write(buffer, 0, bytesRead);
                        outputStream.flush();
                    }
                    return null;
                }
        );
    };
    return ResponseEntity.ok().body(responseBody);
}

Braley