Ricoh Theta S - live preview in javascript
Asked Answered
P

1

25

I am trying to create an app that talks to the Ricoh Theta S camera using HTTP requests ... they are compliant with Open Spherical Camera API Version 1.0 ... here is their api reference

So I am trying to implement the live preview they say the camera supports ... I am doing this in javascript so I tried every possible combination to get the binary data they say the api call returns ... tried vanilla js, jquery, angular ... nothing ... here is a print screen of the request ... the debugger shows it in red although the status code is 200

enter image description here

the response is empty ... so my question is ... is there a way I can implement a Motion JPEG (10 fps) in javascript, over HTTP, using a POST request that sends an object?

thanks, Rares

Puryear answered 4/11, 2015 at 9:44 Comment(10)
Did you managed to do it? I got it working in Java, but I also need it in JS.Centrum
nope ... i gave it up at that time ... i will need to resume so if u manage to do it ... i would appreciate posting here ... i will also do the same ... but the way i will do it in react and react-native but i think it can be adapted to whatever u needPuryear
OK thanks. I'm not sure it's possible in JS. I'll also gave up for now, because I have it working in Java.Centrum
The sessionId you are passing is the one you got before by executing a camera.startSession? Otherwise that could be the reason. Also the name probably should be camera.getLivePreview without the underscore.Didi
version one is with underscore ... and ye session id is from camera.startSession ... the actual problem is parsing a continous motion jpegs in javascript ... because the content-type is multipart/x-mixed-replace with boundary ... never did that ... i really do not know where to grab it from :)Puryear
ah, in this case which browser are you using? AFAIK support for multipart XHR is not present in many browsers and even has been removed from some that had it (e.g. mozilla: see bugzilla.mozilla.org/show_bug.cgi?id=843508). It has been implemented in some Gecko-based browsers, so you could check some of those.Didi
i am doing this on cordova ... and soon i will need to do it in react-native ...Puryear
then probably you are using webkit. I'm not sure how well multipart/x-mixed-replace is supported there, or if it is supported at all. Maybe you could avoid the problem writing a plugin fetching the contents using libcurl.Didi
Do you need to use POST? If you can pass the parameters using a GET, then simply use a <img> tag or an Image object and put the URL in the src parameter. That's how I used to access to MJPEG streams a few years ago. With that, the browser takes care of splitting the multipart headers and updates the image when a full frame is available. This used to work with very old browsers, so I guess support is still there. If later you need to convert it from spherical to view it, you can access the image from WebGL, I guess.Derwin
it is a must ... because this app i am building implements the Google Open Spherical Camera API ... developers.google.com/streetview/open-spherical-camera ... the url u call is /osc/commands/execute ... in order for the localhost on the camera to know what to reply u need to send a post with the info ... in this case it is ... { name: "camera.getLivePreview" } ... so u are telling the camera that u want to execute the getLivePreview command .. but u cannot put this info in the url as a GET param .. needs to be in the body of the POSTPuryear
M
1

There are a couple of things you could try here. A lot of server side technologies (including .NET and Java) are much stricter about what they consider to be valid JSON than Javascript is. Many have been adapted from earlier tech like SOAP that relied on validated XML and consider JSON to be a similar strict set of rules.

Also, the API you're connecting to was probably written by specialist embedded camera firmware engineers who've never written code for the web before. They're used to C++ and Java, which are a lot less forgiving than JS.

Firstly their API specifies that they expect the HTTP headers to be:

Content-Type: application/json;charset=utf-8
Accept: application/json

However, in the screenshot you're sending:

Content-Type: text/plain;charset=utf-8

This tells the server that the content you're sending is text, not JSON. Even though they only expect JSON that will cause a lot of out-of-the-box server side JSON implementations to fail.

The next thing to try is that a lot of JSON parsers that aren't actually Javascript add some quite specific rules to what they consider to be valid JSON.

You're sending:

{name:camera._getLivePreview, parameters:{sessionId:SID_0001}}

This is valid JS, but not actually valid JSON by the strict XML-like rules because they expect everything to be quoted (the only value types you don't quote are booleans and numbers).

So try:

{
    "name": "camera._getLivePreview", 
    "parameters": {
        "sessionId": "SID_0001"
    }
}

If you look at their getting started guide they format every request in this way - quote the properties and quote the values.

One way to ensure that you get this stricter JSON is to build your request JS object and then use JSON.stringify to set the body of the request, like this:

const content = {name:'camera._getLivePreview', parameters:{sessionId:'SID_0001'}};

const response = await fetch('.../osc/commands/execute', {
    method: 'POST', 
    body: JSON.stringify(content),
    headers:{ 'Content-Type': 'application/json' }
});

Finally what you're getting back is a video stream - support for this in fetch is fairly bleeding edge and basically missing from XMLHttpRequest. The server is going to keep sending you content and you keep piping it to something that can render it, and if you stop you'll see that target_closed error.

You need to keep iterating through the stream:

// Get the fetch response as a stream 
const reader = await response.body.getReader();

// Async loop the stream
let chunk = await reader.read();
while (chunk && !chunk.done) {
    for (let index = 0; index < chunk.value.length; index++) {
         // parse the response in chunks       
    }
    chunk = await reader.read();
}

There are plenty of JS MJPEG implementations already out there, here's a fairly simple one

Mathur answered 11/1, 2019 at 8:6 Comment(0)

© 2022 - 2024 — McMap. All rights reserved.