Speech gets cut off in firefox when page is auto-refreshed but not in google chrome

Question

I have this problem where in firefox the speech gets cut off if the page is auto-refreshed, but in google chrome it finishes saying the speech even if the page is auto-refreshed. How do I fix it so that the speech doesn't get cut off in firefox even when the page is auto-refreshed?

msg = new SpeechSynthesisUtterance("please finish saying this entire sentence.");
window.speechSynthesis.speak(msg);

(function ($) {
  'use strict';
  if (window == window.top) {
    var body = $('body').empty();
    var myframe = $('<iframe>')
      .attr({ src: location.href })
      .css({ height: '95vh', width: '100%' })
      .appendTo(body)
      .on('load', function () {
        var interval;
        interval = 750; 
        setTimeout(function () {
          myframe.attr({ src: location.href });
        }, interval);
      });
  }
})(jQuery);

Ardelia

How is the page being auto-refreshed? (e.g. via the html meta tag?) Or do you mean by pressing F5/clicking refresh? – Lakeishalakeland

@Frosty.. (1) Is it possible to make a basic demo page showing the whole problem in action? Maybe we can fix such code. (2) Posted code is not enough to re-create your problem. What exactly is your auto-refresh technique? (is via browser Extension? innerHTML code? Javascript code?).. If we know maybe a workaround can be thought of. – Tonsure

@Tonsure I posted the refresh script. – Ardelia

Hi, the best I could manage in Firefox is this: Web Speech demo (and its related source code). If it's doing what you want, then talk to me about making it work for your site/content also. – Tonsure

The reason I asked "Is it possible to make a basic demo page showing the whole problem in action?" was to find out (see) why you need a refresh every 750 ms. For example, if updating because of some sporting event scores. In such a case, it's better to update the div that hold "score" text itself (not refresh the whole page). – Tonsure

Answer 1

I have this problem where in firefox the speech gets cut off if the page is auto-refreshed, but in google chrome it finishes saying the speech even if the page is auto-refreshed.

The described behaviour for Firefox is a sane expected implementation.

Browsing the source code of Firefox and Chromium the implementation of speechSynthesis.speak() is based on a socket connection with the local speech server. That server at *nix is usually speech-dispatcher or speechd (speech-dispatcher). See How to programmatically send a unix socket command to a system server autospawned by browser or convert JavaScript to C++ souce code for Chromium? for description of trying to implement SSML parsing at Chromium.

Eventually decided to write own code to achieve that requirement using JavaScript according to the W3C specification SpeechSynthesisSSMLParser after asking more than one question at SE sites, filing issues and bugs and posting on W3C mailings lists without any evidence that SSML parsing would ever be included as part of the Web Speech API.

Once that connection is initiated a queue is created for calls to .speak(). Even when the connection is closed Task Manager might still show the active process registered by the service.

The process at Chromium/Chrome is not without bugs, the closest that have filed to what is being described at the question is

• Issue 797624: "speak speak slash" is audio output of .speak() following two calls to .speak(), .pause() and .resume()

• Why hasn't Issue 88072 and Issue 795371 been answered? Are Internals>SpeechSynthesis and Blink>Speech dead? (for possible reason why "but in google chrome it finishes saying the speech even if the page is auto-refreshed." is still possible at Chrome)

.volume property issues

• Issue 797512: Setting SpeechSynthesisUtterance.volume does not change volume of audio output of speechSynthesis.speak() (Chromium/Chrome)

• Bug 1426978 Setting SpeechSynthesisUtterance.volume does not change volume of audio output of speechSynthesis.speak() (Firefox)

The most egregious issue being Chromium/Chrome webkitSpeechReconition implementation which records the users' audio and posts that audio data to a remote service, where a transcript is returned to the browser - without explicitly notifying the user that is taking place, marked WONT FIX

• Issue 816095: Does webkitSpeechRecognition send recorded audio to a remote web service by default?

Relevant W3C Speech API issues at GitHub

• The UA should be able to disallow speak() from autoplaying #27

• Precisely define when speak() should fail due to autoplay rules #35 (ironically, relevant to the reported behaviour at Chromium/Chrome and output described at this question, see Web Audio, Autoplay Policy and Games and Autoplay Policy Changes)

  • Intent to Deprecate: speechSynthesis.speak without user activation

    Summary

    The SpeechSynthesis API is actively being abused on the web. We don’t have hard data on abuse, but since other autoplay avenues are starting to be closed, abuse is anecdotally moving to the Web Speech API, which doesn't follow autoplay rules.

    After deprecation, the plan is to cause speechSynthesis.speak to immediately fire an error if specific autoplay rules are not satisfied. This will align it with other audio APIs in Chrome.

• Timing of SpeechSynthesis state changes not defined #39

• Timing of SpeechSynthesisUtterance events firing not defined #40

• Clarify what happens if two windows try to speak #47

---

In summary, would not describe the behaviour at Firefox as a "problem", but the behaviour at Chrome as being a potential "problem".

Diving in to W3C Web Speech API implementation at browsers is not a trivial task. For several reasons. Including the apparent focus, or available option of, commercial TTS/SST services and proprietary, closed-source implementations of speech synthesis and speech recognition in "smart phones"; in lieu of fixing the various issues with the actual deployment of the W3C Web Speech API at modern browsers.

The maintainers of speechd (speech-dispatcher) are very helpful with regards to the server side (local speech-dispatcher socket).

Cannot speak for Firefox maintainers. Would estimate it is unlikely that if a bug is filed relevant to the feature request of continuing execution of audio output by .speak() from reloaded window is consistent with recent autoplay policies implemented by browsers. Though you can still file a Firefox bug to ask if audio output (from any API or interface) is expected to continue during reload of the current window; and if there are any preferences or policies which can be set to override the described behaviour, as suggested at the answer by @zip. And get the answer from the implementers themselves.

There are individuals and groups that compose FOSS code which are active in the domain and willing to help SST/TTS development, many of which are active at GitHub, which is another option to ask questions about how to implement what you are trying to achieve specifically at Firefox browser.

Outside of asking implementers for the feature request, you can read the source code and try create one or more workarounds. Alternatives include using meSpeak.js, though that still does not necessarily address if Firefox is intentionally blocking audio output during reload of the window.

Dyspnea

So thanks for explaining why it doesn't work, but do you have a solution to make it work? – Ardelia

Answer 2

Not sure why there's a difference in behavior... guest271314 might be on to something in his answer. However, you may be able to prevent FF from stopping the tts by intercepting the reload event with a onbeforeunload handler and waiting for the utterance to finish:

msg = new SpeechSynthesisUtterance("say something");
window.speechSynthesis.speak(msg);
window.onbeforeunload = function(e) {
  if(window.speechSynthesis.speaking){
    event.preventDefault();
    msg.addEventListener('end', function(event) {
      //logic to continue unload here
    });
  }
};

Harkness

Answer 3

EDITED: See more elegant solution with promises below initial answer!

---

Below snippet is a workaround to the browser inconsistencies found in Firefox, checking synth.speaking in the interval and only triggering a reload if it's false prevents the synth from cutting of prematurely:

(It does not NOT work properly in the SO snippet, I assume it doesn't like iFrames in iFrames or whatever, just copy paste the code in a file and open it with Firefox!)

<p>I'm in the body, but will be in an iFrame</p>

<script src="https://cdnjs.cloudflare.com/ajax/libs/jquery/3.3.1/jquery.min.js"></script>
<script>
	var synth = window.speechSynthesis;
	msg = new SpeechSynthesisUtterance("please finish saying this entire sentence.");
	synth.speak(msg);
	
	(function ($) {
		'use strict';
		if (window == window.top) {
			var body = $('body').empty();
			var myframe = $('<iframe>')
				.attr({ src: location.href })
				.css({ height: '95vh', width: '100%' })
				.appendTo(body)
				.on('load', function () {
					var interval;
					interval = setInterval(function () {
						if (!synth.speaking) {
							myframe.attr({ src: location.href });
							clearInterval(interval);
						}
					}, 750);
				});
		}
	})(jQuery);
</script>

A more elaborate solution could be to not have any setTimeout() or setInterval() at all, but use promises instead. Like this the page will simply reload whenever the message is done synthesizing, no matter how short or long it is. This will also prevent the "double"/overlapping-speech on the initial pageload. Not sure if this helps in your scenario, but here you go:

<button id="toggleSpeech">Stop Speaking!</button>

<script src="https://cdnjs.cloudflare.com/ajax/libs/jquery/3.3.1/jquery.min.js"></script>
<script>
	if (window == window.top) {
		window.speech = {
			say: function(msg) {
				return new Promise(function(resolve, reject) {
					if (!SpeechSynthesisUtterance) {
						reject('Web Speech API is not supported');
					}
					
					var utterance = new SpeechSynthesisUtterance(msg);
					
					utterance.addEventListener('end', function() {
						resolve();
					});
					
					utterance.addEventListener('error', function(event) {
						reject('An error has occurred while speaking: ' + event.error);
					});
					
					window.speechSynthesis.speak(utterance);
				});
			},
			speak: true,
		};
	}
	
	
	(function($) {
		'use strict';

		if (window == window.top) {
			var body = $('body').empty();
			var myframe = $('<iframe>')
				.attr({ src: location.href })
				.css({ height: '95vh', width: '100%' })
				.appendTo(body)
				.on('load', function () {
					var $iframe = $(this).contents();

					$iframe.find('#toggleSpeech').on('click', function(e) {
						console.log('speaking will stop when the last sentence is done...');
						window.speech.speak = !window.speech.speak;
					});
					
					window.speech.say('please finish saying this entire sentence.')
						.then(function() {
							if ( window.speech.speak ) {
								console.log('speaking done, reloading iframe!');
								myframe.attr({ src: location.href });
							}
						});
				});
		}
	})(jQuery);
</script>

NOTE: Chrome (since v70) does NOT allow the immediate calling of window.speechSynthesis.speak(new SpeechSynthesisUtterance(msg)) anymore, you will get an error speechSynthesis.speak() without user activation is no longer allowed..., more details here. So technically the user would have to activate the script in Chrome to make it work!

Gifted

Answer 4

Firefox:

First of all type and search for the “about: config” inside the browser by filling it in the address bar. This will take to another page where there will be a pop up asking to Take Any Risk, you need to accept that. Look for the preference named “accessibility.blockautorefresh” from the list and then right-click over that. There will be some options appearing as the list on the screen, select the Toggle option and then set it to True rather than False. This change will block the Auto Refresh on the Firefox browser. Remember that this option is revertable!

Flin

This has nothing to do with frosty's question, which is a question of why different browsers behave differently in a given scenario; you're giving browser usage advise on a different problem. Please try to be sure that you're actually answering the original question; thank you – Kaka

hello once again marcus, i gave solution to resolve in firefox as requested? – Flin

@Flin You say "This will block the Auto-Refresh on Firefox" bu Asker doesn't want to block it. He wants current audio to continue playing (no cuts or stops) even if doing some auto-refresh. – Tonsure