nghttp2 multipart POST message

Question

I am currently trying to use nghttp2 to build a multipart message. The message should be strcuture as the following.

I should use the nghttp2_submit_request(here) function, with nva as my HTTP/2 header, and nghttp2_data_provider *data_prd for my data. However, I still dont understand how exactly can I create two message (with two message headers).

---

Update:

Is it possible for me to describe what I was think in my source code? Please see below. Here, I am using nghttp2_data_provider to open up an audio file, and write to the buffer.

ssize_t data_prd_read_callback(
    nghttp2_session *session, int32_t stream_id, uint8_t *buf, size_t length,
    uint32_t *data_flags, nghttp2_data_source *source, void *user_data) 
{

  printf("[INFO] C ----------------------------> S (DATA post body), length:%zu\n", length);

  int fd = source->fd;
  ssize_t r;
// writting my opened audio file into buffer
  while ((r = read(fd, buf, length)) == -1 && errno == EINTR); 
  printf("stream_id:%d, nread:%zu\r\n", stream_id, r);
  return nread;
}



void submit_postAudio(http2_session_data *session_data) {
  int32_t stream_id;
  http2_stream_data *stream_data = session_data->stream_data[STREAM_ID_REQUESTAUDIO];
  const char *uri = stream_data->uri;
  const struct http_parser_url *u = stream_data->u;
  char byLength = 0;

  const nghttp2_nv hdrs[] = {
  MAKE_NV(":method", "POST"),   MAKE_NV_CS(":path", stream_data->path),
  MAKE_NV(":scheme", "https"), MAKE_NV("accept", "*/*"),    
  MAKE_NV_CS("authorization", stream_data->access_token),
  MAKE_NV("content-type", "multipart/form-data; boundary=abcdefg123")
  };

  fprintf(stderr, "Request headers:\n");
  print_headers(stderr, hdrs, ARRLEN(hdrs));

  int fileDescriptor = open ("/my_audio.wmv", O_APPEND);  // open my audio file 
  nghttp2_data_provider data_prd;
  data_prd.source.fd = fileDescriptor   // set the file descriptor 
  data_prd.source.ptr = NULL;
  data_prd.read_callback = data_prd_read_callback;

  stream_id = nghttp2_submit_request(session_data->session, NULL, hdrs,
                                     ARRLEN(hdrs), &data_prd, stream_data);
  if (stream_id < 0) {
    errx(1, "Could not submit HTTP request: %s", nghttp2_strerror(stream_id));
  }

  stream_data->stream_id = stream_id;
}

What confused me is 1) how do I add a header (a message header to be more specific) to the audio. 2) how do I append it after my Json file.

Spireme

Answer 1

A multi-part mime header is usually defined like this:

-----boundary_id
name: value  # header stuff here, ie. "name: value" per line. No spaces in name, all ASCII
CRLF         # End of header is defined with an extra linefeed, AKA a "null line", see ref 1.

content here, ends at next boundary id

-----boundary_id

Ref 1

If the content is binary you would usually encode it as base-64 with a fixed line length (you can send binary data too but have to take precautions as to extra white-spaces such as a final line-feed). A header field-name stating this can be included:

Content-Transfer-Encoding: Base64

(for binary data use value "binary" instead).

The boundary id must be unique enough to not risk being part of any other content. It does not use double-dashes when defined in the main header, but you need to prepend it with such double-dashes everywhere it acts as a separator.

So in this case the entire message would look something like this:

name1: value1
name2: value2
name3: value3
x-comment: "as required by the specs (ignore this line)"
content-type: multipart/form-data; boundary: my_unique_boundary_id

--my_unique_boundary_id
Content-Disposition: form-data; name="metadata"
Content-Type: application/json; charset=UTF-08

{JSON data here}

--my_unique_boundary_id
Content-Disposition: form-data; name="audio"
Content-Type: application/octet-stream
Content-Transfer-Encoding: Base64
x-comment: "binary audio encoded as base-64 follows next"

UklGRuKlAABXQVZFZm10ICgAAAABAAEARKwAAIhYAQACABAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAc2NvdLQBAAAAAAAAVGhlIFJoeXRobSBNYWNoaW5lICAgICAgICAgICAgICAgICAgICAgICAg
IFRSTTEAICA6MDAAAAAAAAAvADAwMDAwMDAwMDAwMAAAALkBTQABAAAAJQAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAABKYXNvbiBE
b25uZWxseSAoYWthIGRqIHB1enpsZSkgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgIDAwQwAAAAAAADEyMTMwMAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
... etc.

--my_unique_boundary_id

Trucking

Empty line needed to separate the header from the body in the last MIME part. – Casie

@Casie ups, I accidentally deleted when removing a CRLF representation – Trucking

Thanks for your answer. From the ssize_t data_prd_read_callback( nghttp2_session *session, int32_t stream_id, uint8_t *buf, size_t length, uint32_t *data_flags, nghttp2_data_source *source, void *user_data) , I only have a buffer (uint8_t *)to write my payload to. Are you suggesting to simply write char in this format (with the correct spacing and new line ), and then follow by the audio file? – Spireme

@Spireme I'm not familiar with API in question, but in general, yes, the multi-part is basically a long text string. It can be copied to a byte buffer directly (just be aware of byte order if unicode is involved; shouldn't be a problem here though). Usually this step isn't necessary as the intent with multi-part is to make it transferable on its own as a string, but if a byte-buffer is required by the api that should work too, the audio data would already be included in the string as base-64 (which you need to convert the audio data to first. Or the raw binary if this method is chosen instead). – Trucking

@k3n, one more question, at the end of the audio file, do you know if I simply include a new line or should it be NULL terminated? – Spireme

@Spireme No need to null-terminate it as the content length is determined by the separators. When extracted just check for new line at the end of the buffer. If any is found (depending on how you scan for separators) they should be stripped off. With mp3 you might get away with not doing so for say mp3, but for structured formats that has length validated (f.ex. wav) this may not work. – Trucking

I am using wave – Spireme