Is it possible to select the word that is being read while using the SpeechSynthesisUtterance API?

Question

Is it possible to select the word that is being read while using the SpeechSynthesisUtterance API?

Is there an event I can use to get the current spoken word and cursor position?

Here is what I have so far:

var msg = new SpeechSynthesisUtterance();
var voices = window.speechSynthesis.getVoices();
msg.voice = voices[10]; // Note: some voices don't support altering params
msg.voiceURI = 'native';
msg.volume = 1; // 0 to 1
msg.rate = 1; // 0.1 to 10
msg.pitch = 2; //0 to 2
msg.text = 'Hello World';
msg.lang = 'en-US';

msg.onend = function(e) {
  console.log('Finished in ' + event.elapsedTime + ' seconds.');
};

speechSynthesis.speak(msg);

Example from here.

Lashay

Answer 1

There was a related question that wrote out the words out to a span and I've extended that answer here to select the words as they are spoken.

var utterance = new SpeechSynthesisUtterance();
utterance.lang = 'en-UK';
utterance.rate = 1;

document.getElementById('playButton').onclick = function(){
    var text = document.getElementById('textarea').value;
    // create the utterance on play in case user called stop
    // reference https://mcmap.net/q/376303/-speechsynthesis-stops-working-after-first-utterance-in-firefox-but-works-in-chrome
    utterance = new SpeechSynthesisUtterance();
    utterance.onboundary = onboundaryHandler;
    utterance.text = text;
    speechSynthesis.speak(utterance);
};

document.getElementById('pauseButton').onclick = function(){
    if (speechSynthesis) {
      speechSynthesis.pause();
    }
};

document.getElementById('resumeButton').onclick = function(){
    if (speechSynthesis) {
      speechSynthesis.resume();
    }
};

document.getElementById('stopButton').onclick = function(){
    if (speechSynthesis) {
      speechSynthesis.cancel();
    }
};

function onboundaryHandler(event){
    var textarea = document.getElementById('textarea');
    var value = textarea.value;
    var index = event.charIndex;
    var word = getWordAt(value, index);
    var anchorPosition = getWordStart(value, index);
    var activePosition = anchorPosition + word.length;
    
    textarea.focus();
    
    if (textarea.setSelectionRange) {
       textarea.setSelectionRange(anchorPosition, activePosition);
    }
    else {
       var range = textarea.createTextRange();
       range.collapse(true);
       range.moveEnd('character', activePosition);
       range.moveStart('character', anchorPosition);
       range.select();
    }
};

// Get the word of a string given the string and index
function getWordAt(str, pos) {
    // Perform type conversions.
    str = String(str);
    pos = Number(pos) >>> 0;

    // Search for the word's beginning and end.
    var left = str.slice(0, pos + 1).search(/\S+$/),
        right = str.slice(pos).search(/\s/);

    // The last word in the string is a special case.
    if (right < 0) {
        return str.slice(left);
    }
    
    // Return the word, using the located bounds to extract it from the string.
    return str.slice(left, right + pos);
}

// Get the position of the beginning of the word
function getWordStart(str, pos) {
    str = String(str);
    pos = Number(pos) >>> 0;

    // Search for the word's beginning
    var start = str.slice(0, pos + 1).search(/\S+$/);
    return start;
}

<textarea id="textarea" style="width:100%;height:150px;">
Science Literacy is a way of approaching the world. It's a way of equipping yourself to interpret what happens in front of you. It's methods and tools that enable it to act as a kind of a utility belt necessary for what you encounter in the moment. It's methods of mathematical analysis, interpretation, some basic laws of physics so when someone says I have these two crystals and if you rub them together you get healthy. Rather than just discount it, because that's as lazy as accepting it, what you should do is inquire. 

So do you know how to inquire? Every scientist would know how to start that conversation. Where did you get these? What does it cure? How does it work? How much does it cost? Can you demonstrate? Science literacy is vaccine against charlatans of the world that would exploit your ignorance of the forces of nature. Become scientifically literate.
</textarea><br>
<input type="button" id="playButton" value="Play"/>
<input type="button" id="pauseButton" value="Pause"/>
<input type="button" id="resumeButton" value="Resume"/>
<input type="button" id="stopButton" value="Stop"/>

MDN SpeechSynthesis
MDN SpeechSynthesisEvent
MDN Boundary

Lashay

Is there a way to pass a value through to onboundaryHandler? My implementation does not use textarea, but rather text from classes - I am trying to figure out how to pass class instance information (event just an index) to onboundaryHandler so I can highlight relevant text. Thanks! EDIT: It seems you can with something like utterance.MY_GREAT_ID = index. – Juni

What is >>>, as used in pos = Number(pos) >>> 0? – Juni

right = str.slice(pos).search(/\s|$/); would avoid that right < 0 clause. ;-) – Indicative

getWordAt in this example only works for certain languages. It wont work for east Asian languages (Chinese, Japanese, Korean, Thai for example). Maybe you should have a look at charLength attribute on event. – Survival

How can I implement the highlighting functionality if I am using a div instead of textarea? – Kazbek

@LyubomirIvanovValchev You could set background color and text color via styles or rich text (editable text). I don't think it will be a small amount of work but it should be possible. – Lashay

Is it possible to use this code on general text in a div? – Hairdo

@LukePrior I don’t think so. IIRC that was the first thing I tried. And IIRC again they browser can’t select text that is not editable. However, your might be able to set content editable (see MDN) to true on the div and see if that changes things. – Lashay

@LyubomirIvanovValchev quick and dirty job would be using css to make textarea look like a div...you can disable some properties, such as delete, highlight, etc. – Reorder

Is it possible to add scroll to highlighted word? – Np

Answer 2

I've added 4 lines (from this answer) to add scroll functionality to 1.21 gigawatts answer above.

    var utterance = new SpeechSynthesisUtterance();
    utterance.lang = 'en-UK';
    utterance.rate = 1;

document.getElementById('playButton').onclick = function(){
    var text = document.getElementById('textarea').value;
    // create the utterance on play in case user called stop
    // reference https://mcmap.net/q/376303/-speechsynthesis-stops-working-after-first-utterance-in-firefox-but-works-in-chrome
    utterance = new SpeechSynthesisUtterance();
    utterance.onboundary = onboundaryHandler;
    utterance.text = text;
    speechSynthesis.speak(utterance);
};

document.getElementById('pauseButton').onclick = function(){
    if (speechSynthesis) {
      speechSynthesis.pause();
    }
};

document.getElementById('resumeButton').onclick = function(){
    if (speechSynthesis) {
      speechSynthesis.resume();
    }
};

document.getElementById('stopButton').onclick = function(){
    if (speechSynthesis) {
      speechSynthesis.cancel();
    }
};

function onboundaryHandler(event){
    var textarea = document.getElementById('textarea');
    var value = textarea.value;
    var index = event.charIndex;
    var word = getWordAt(value, index);
    var anchorPosition = getWordStart(value, index);
    var activePosition = anchorPosition + word.length;
    
    textarea.focus();
    // Added lines to scroll
    const fullText = textarea.value;
    textarea.value = fullText.substring(0, activePosition);
    textarea.scrollTop = textarea.scrollHeight;
    textarea.value = fullText;   
    // end added lines to scroll
    if (textarea.setSelectionRange) {
       textarea.setSelectionRange(anchorPosition, activePosition);
    }
    else {
       var range = textarea.createTextRange();
       range.collapse(true);
       range.moveEnd('character', activePosition);
       range.moveStart('character', anchorPosition);
       range.select();
    }
};

// Get the word of a string given the string and index
function getWordAt(str, pos) {
    // Perform type conversions.
    str = String(str);
    pos = Number(pos) >>> 0;

    // Search for the word's beginning and end.
    var left = str.slice(0, pos + 1).search(/\S+$/),
        right = str.slice(pos).search(/\s/);

    // The last word in the string is a special case.
    if (right < 0) {
        return str.slice(left);
    }
    
    // Return the word, using the located bounds to extract it from the string.
    return str.slice(left, right + pos);
}

// Get the position of the beginning of the word
function getWordStart(str, pos) {
    str = String(str);
    pos = Number(pos) >>> 0;

    // Search for the word's beginning
    var start = str.slice(0, pos + 1).search(/\S+$/);
    return start;
}

<textarea id="textarea" style="width:100%;height:150px;">
Science Literacy is a way of approaching the world. It's a way of equipping 
yourself to interpret what happens in front of you. It's methods and tools that enable it to act as a kind of a utility belt necessary for what you encounter in the moment. It's methods of mathematical analysis, interpretation, some basic laws of physics so when someone says I have these two crystals and if you rub them together you get healthy. Rather than just discount it, because that's as lazy as accepting it, what you should do is inquire.
So do you know how to inquire? Every scientist would know how to start that conversation. Where did you get these? What does it cure? How does it work? How much does it cost? Can you demonstrate? Science literacy is vaccine against charlatans of the world that would exploit your ignorance of the forces of nature. Become scientifically literate.
</textarea><br>
<input type="button" id="playButton" value="Play"/>
<input type="button" id="pauseButton" value="Pause"/>
<input type="button" id="resumeButton" value="Resume"/>
<input type="button" id="stopButton" value="Stop"/>

Np

Nice! Good job! – Lashay

Answer 3

//NOTE: A USER MUST INTERACT WITH THE BROWSER before sound will play.
const msg = new SpeechSynthesisUtterance();
const voices = window.speechSynthesis.getVoices();
msg.voice = voices[10]; // Note: some voices don't support altering params
msg.voiceURI = 'native';
msg.volume = 1; // 0 to 1
msg.rate = 1; // 0.1 to 10
msg.pitch = 2; //0 to 2
txt = "I'm fine, borderline, so bad it hurts Think fast with your money cause it can't get much worse I get told that I'm far too old for number one perks".split(" ")
msg.text = txt;
msg.lang = 'en-US';

msg.onend = function(e) {
  console.log('Finished in ' + event.elapsedTime + ' seconds.');
};

let gap = 240
let i = 0
speakTrack = setInterval(() => {
  console.log(txt[i++])
  //i++ < dont forget if you remove console log
  if (i >= txt.length) {
    i = 0
    clearInterval(speakTrack)
  }
}, gap)

speechSynthesis.speak(msg);

https://jsfiddle.net/Vinnywoo/bvt314sa

Hydrometallurgy

I like this effect but the timing seems to drift over time. – Lashay

Not bad but a little fast – Alfredoalfresco

@SithLee how to get function pause and resume in speakTrack – Whirlybird

Demo not working – Reorder

Swapped demo to JSFiddle. – Hydrometallurgy