Worker using synchronous XMLHttpRequest to get data from GUI

Question

I'd like a Web Worker which is deep in a call stack to be able to make a synchronous request to get information from the GUI.

The GUI itself is not blocked--it's able to process messages. But the JavaScript on the worker's stack is not written in async / await style. It is just a lot of synchronous code. So if the GUI tried to send a response back to the worker with a postMessage, that would just be stuck in the onmessage() queue.

I've found at least one hack that works in today's browsers. The worker can postMessage to the GUI for the information it wants--along with some sort of ID (e.g. a UUID). Then it can make a synchronous XMLHttpRequest--which is not deprecated on workers--to some server out on the web with that ID.

While the worker is waiting on that http request, the GUI processes the information request. When it's done, it does an XMLHttpRequest to POST to that same server with the ID and the data. The server then uses that information to fulfill the blocking request it is holding open for the worker. This fulfills the synchronous request.

It may seem hare-brained to outsource synchronization between the GUI and the worker to a server. But I'll do it if I have to, because it does not fit the use case to force the worker code to be written in asynchronous style. Also, I'm assuming that someday the browser will be able to do this kind of synchronization natively. But it looks like the one mechanism which could have been used--SharedArrayBuffer, has been disabled for the time being.

UPDATE circa late 2018: SharedArrayBuffer was re-enabled in Chrome for desktop v67. It's not back on for Android Chrome or other browsers yet, and might be a while.

(More bizarre options like compiling a JavaScript interpreter into the worker so the JS stack could be suspended and restarted at will are not on the table--not just due to size and performance, but the inability to debug the worker using the browser's developer tools.)

So...

• Is there any way for a synchronous XMLHttpRequest to be fooled into making a request of something coming from within the browser itself (maybe via a custom link scheme?) If the GUI thread could directly answer an XMLHttpRequest that would cut out the middleman.

• Could the same functionality be provided via some kind of plugin? I'm thinking maybe synchronization could be done as an abstraction. If someone doesn't have the plugin, it falls back to using the network as a synchronization surrogate. (And presumably if they ever re-enable SharedArrayBuffer, it could just use that.)

I'm wondering also if there is some kind stock JS-ready service which already implements the protocol for the echo server...if anyone knows of one. Seems quite easy to write.

Dominy

"Is there any way for a synchronous XMLHttpRequest to be fooled into making a request of something coming from within the browser itself". Do you mean mock syncrhonous XMLHttpRequest? e.g. jsfiddle.net/07nvgrt8/17 ? – Alcott

@MarinosAn I don't see a mock like that working for what the OP is trying to do. The problem is that the same mock object would have to exist both in the script that starts the worker and in the worker. Ok, so you just send the mock to the worker, right? Well, no, because objects that are passed to workers are cloned. So any request made on the mock inside the worker won't be seen by the mock outside the worker. – Story

@Story Someone suggested to me to look at "service workers" which are distinct from plain workers, and are supported in most browsers. I hadn't heard of them, but I don't know that they'd bring anything new to the table or not--it seems to me at best, they might permit one doing a spinlock (bad) on the worker and using the service worker to relay. Haven't quite gotten my head around it, hoping lazyweb will help me. :-) – Dominy

@HostileFork The service workers examples I've seen mention providing "custom responses to requests" but all examples use the fetch event to provide the custom response. AFAIK, it is produced only when you actually use the fetch API specifically. An xhr won't generate a fetch event. And you cannot just use fetch in your specific situation instead of xhr because fetch does not operate synchronously. The specs mention a "synchronous flag", but it is not part of the API. – Story

@Story Your comments helped save me from wasting time looking into service workers...so if you'd like to digest them into a "you're out of luck" post before the bounty deadline (maybe with any insights you have on making the XMLHttpRequest service I describe, if you happen to have any), I'll award you the points if that is the unfortunate answer...! – Dominy

@HostileFork Done. I'm afraid I don't have any special insight to add into my answer though. – Story

"it does not fit the use case to force the worker code to be written in asynchronous style" - perhaps you could redo the question and ask how to change the worker code to be async. – Award

@Sando It's not a matter of any specific piece of code to be rewritten in asynchronous style. It is a matter of trying to provide an API service to an arbitrary client where the explicit goal is to empower them to code in a synchronous style. Just because JavaScript is fairly limp when it comes to allowing variations in programming style doesn't mean every programmer in the world needs to bend to JavaScript's design (or lack thereof). Anyway, apparently I didn't phrase myself clearly enough: the question is as it stands, and it is the question. – Dominy

Answer 1

I don't see a way to do what you're trying to do. Approaches that appear initially promising eventually run into hard problems.

Service Workers and fetch

In a comment, you suggested service workers as a possible solution. The service workers examples I've seen mention providing "custom responses to requests". However, all examples use the fetch event to provide the custom response. AFAIK, it is produced only when you actually use the fetch API specifically. An xhr won't generate a fetch event. (Yes, I've tried it and it did not work.) And you cannot just use fetch in your specific situation instead of xhr because fetch does not operate synchronously. The specs for fetch mention a "synchronous flag", but it is not part of the API.

Note that the fetch API and the associated event are not specific to service workers so you could use fetch in a plain worker, or elsewhere, if it solved your problem. You often see fetch mentioned with service workers because service workers can be used for scenarios where regular workers cannot be used and some of those scenarios entail providing custom responses to fetch requests.

Fake XMLHttpRequest

Marinos An suggested in a comment using a fake XMLHttpRequest object. In most cases, that would work. Testing frameworks like Sinon provide fake XMLHttpRequest that allow testing code to have complete control over the responses that the code under test gets. However, it does not work for your use-case scenario. If your fake xhr implementation is implemented as one JavaScript object, and you try sending it to the worker, the worker will get a complete clone of it. Actions on the fake xhr performed inside the worker won't be seen outside the worker. Actions on the fake xhr performed outside the worker won't be seen inside the worker.

It is theoretically possible to work around the cloning issue by having the fake xhr consist of two objects: a front end through which requests are performed, and a backend through which fake responses are established. You could send the front end to the worker, but the front end and the back end would have to communicate with each other and this brings you right back to the communication problem you were trying to solve. If you could make the two parts of the fake xhr talk to each other in a way that allows you to fake synchronous xhr requests, then by the same token you would be able to solve the communication problem without the fake xhr.

Story

The reason I brought up service workers is because there was some stuff about them working with XMLHttpRequest at one time. But this is no longer available, apparently.. I guess my remaining question is what might be done with a Chrome/Firefox plugin, Flash control (!), or otherwise to bridge the gap until SharedArrayBuffer or some other solution. At which point this question may need to accept a new answer, but looks like yours may be it for now! – Dominy

Alas, I cannot say anything much about plugins or Flash. I'd be willing to go through a lot of other hurdles, including rewriting the worker to work asynchronously, before I'd consider plugins or Flash. – Story

I'm working on a similar problem to OP. My experiments indicate that service workers do respond to synchronous XHRs, but only if they're sent from a web worker, not the main GUI thread. When you say "An xhr won't generate a fetch event. (Yes, I've tried it and it did not work.)" I suspect you either tried it with a request sent from the main thread OR browsers have changed things recently. – Benedictbenedicta

That was 3 years and a half ago. I don't quite remember how I tested it, and yes it is possible that browsers have changed how they operate regarding this specific problem. – Story

Answer 2

Hm...perhaps you could create your workers on the fly, like so

function startNewWorker (code) {
  var blob = new Blob([code], {type: "application/javascript"});
  var worker = new Worker(URL.createObjectURL(blob));
}

And then, for each new http request you need, you start its own worker:

const w1 = startNewWorker(yourCodeThatDoesSomething);
w1.onmessage = function () { /* ... */};

const w2 = startNewWorker(yourCodeThatDoesSomething);
w2.onmessage = function () { /* ... */};

Both will be asynchronous and will not block the interface for you user, and they both will be able to do their own work, and each of them will have its own listeners.

Notice that code is a string, so, if you have a function, you can use it .toString() concatenated with (), like this:

function myWorkerContent () {
  // do something ....
}

const code = "(" + myWorkerContent.toString() + ")()";
// or, if you want to use templateLiterals
// const code = `(${myWorkerContent.toString()})()`;

This way, when running your worker, it will create and execute the function instantly inside each worker of yours.

Sheik

As mentioned in the question, I'm not having a problem with the GUI being blocked. The problem is that the worker wants information from the GUI without having to unwind its stack and "yield" to its onmessage loop in order to get that information. Spawning another worker won't really help the problem, unless there were some privileged mode of worker-to-worker communication it could use while blocked (I'm fishing here for perhaps something that could answer a synchronous XMLHttpRequest, but looks like no dice) – Dominy