How to get audio amplitude with speech recognizer?
Asked Answered
M

1

2

I'm trying to make a custom dialog while speech recognizing instead of using the official one. I got that part, but when then I decided to show the amplitude of the sound while recognizing, in order to make it more fancy, like Google Now search bar does (its a circle around the microphone that grows if voice its louder):

googlenow

Then I started to code how to obtain the amplitude of the sound, and finally I got it with AudioRecord Class.

The problem comes when I try to mix both (SpeechRecognizer and AudioRecord), because seems like they are not able to share microphone, or something like that...

In logcat I have this error:

03-03 21:16:07.461: E/ListenerAdapter(23359): onError
03-03 21:16:07.461: E/ListenerAdapter(23359): com.google.android.speech.embedded.Greco3RecognitionEngine$EmbeddedRecognizerUnavailableException: Embedded recognizer unavailable
03-03 21:16:07.461: E/ListenerAdapter(23359):   at com.google.android.speech.embedded.Greco3RecognitionEngine.startRecognition(Greco3RecognitionEngine.java:108)
03-03 21:16:07.461: E/ListenerAdapter(23359):   at java.lang.reflect.Method.invokeNative(Native Method)
03-03 21:16:07.461: E/ListenerAdapter(23359):   at java.lang.reflect.Method.invoke(Method.java:511)
03-03 21:16:07.461: E/ListenerAdapter(23359):   at com.google.android.searchcommon.utils.ThreadChanger$1$1.run(ThreadChanger.java:77)
03-03 21:16:07.461: E/ListenerAdapter(23359):   at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:390)
03-03 21:16:07.461: E/ListenerAdapter(23359):   at java.util.concurrent.FutureTask.run(FutureTask.java:234)
03-03 21:16:07.461: E/ListenerAdapter(23359):   at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:153)
03-03 21:16:07.461: E/ListenerAdapter(23359):   at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:267)
03-03 21:16:07.461: E/ListenerAdapter(23359):   at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1080)
03-03 21:16:07.461: E/ListenerAdapter(23359):   at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:573)
03-03 21:16:07.461: E/ListenerAdapter(23359):   at com.google.android.searchcommon.utils.ConcurrentUtils$2$1.run(ConcurrentUtils.java:112)

and some other times i have this:

03-03 21:47:13.344: E/ListenerAdapter(23359): onError
03-03 21:47:13.344: E/ListenerAdapter(23359): com.google.android.speech.exception.AudioRecognizeException: Audio error
03-03 21:47:13.344: E/ListenerAdapter(23359):   at com.google.android.speech.embedded.Greco3Recognizer.read(Greco3Recognizer.java:107)
03-03 21:47:13.344: E/ListenerAdapter(23359):   at dalvik.system.NativeStart.run(Native Method)
03-03 21:47:13.344: E/ListenerAdapter(23359): Caused by: java.io.IOException: couldn't start recording, state is:1
03-03 21:47:13.344: E/ListenerAdapter(23359):   at com.google.android.speech.audio.MicrophoneInputStream.ensureStartedLocked(MicrophoneInputStream.java:119)
03-03 21:47:13.344: E/ListenerAdapter(23359):   at com.google.android.speech.audio.MicrophoneInputStream.read(MicrophoneInputStream.java:159)
03-03 21:47:13.344: E/ListenerAdapter(23359):   at com.google.common.io.ByteStreams.read(ByteStreams.java:806)
03-03 21:47:13.344: E/ListenerAdapter(23359):   at com.google.android.speech.audio.Tee.readFromDelegate(Tee.java:374)
03-03 21:47:13.344: E/ListenerAdapter(23359):   at com.google.android.speech.audio.Tee.readLeader(Tee.java:267)
03-03 21:47:13.344: E/ListenerAdapter(23359):   at com.google.android.speech.audio.Tee$TeeLeaderInputStream.read(Tee.java:464)
03-03 21:47:13.344: E/ListenerAdapter(23359):   at java.io.InputStream.read(InputStream.java:163)
03-03 21:47:13.344: E/ListenerAdapter(23359):   at com.google.android.speech.audio.AudioSource$CaptureThread.run(AudioSource.java:193)

And this is how i launch both:

//previously in constructor
speechrec = SpeechRecognizer.createSpeechRecognizer(getActivity());
speechrec.setRecognitionListener(this);
//

public void launchListening()
{       
    if (speechrec.isRecognitionAvailable(getActivity()))
    {
        Intent intent = new Intent(RecognizerIntent.ACTION_RECOGNIZE_SPEECH);
        intent.putExtra(RecognizerIntent.EXTRA_LANGUAGE_MODEL,RecognizerIntent.LANGUAGE_MODEL_FREE_FORM);
        speechrec.startListening(intent);       
    }

    bufferSize = AudioRecord.getMinBufferSize(sampleRate, AudioFormat.CHANNEL_CONFIGURATION_MONO, AudioFormat.ENCODING_PCM_16BIT);// * bufferSizeFactor;
    audio = new AudioRecord(MediaRecorder.AudioSource.MIC, sampleRate, AudioFormat.CHANNEL_CONFIGURATION_MONO, AudioFormat.ENCODING_PCM_16BIT, bufferSize);
    audio.startRecording();

    captureThread = new Thread(new Runnable()
    {
        public void run()
        {
            //calculate amplitude here
        }
    });
    captureThread.start();
}

Any ideas on how creating a custom dialog for speech recognition, where I can show amplitude based on the noise, like Google does?

Magnetics answered 3/3, 2013 at 20:58 Comment(0)
B
3

The way to do it is to register a listener with the SpeechRecognizer and visualize the output of onRmsChanged. Note however that:

There is no guarantee that this method will be called.

So the speech recognizer that you are using needs to support this method. Note that the return value of SpeechRecognizer.createSpeechRecognizer(getActivity()) depends on the user's device's configuration.

(You cannot start an AudioRecord while the SpeechRecognizer is recording and vice versa.)

Bonspiel answered 4/3, 2013 at 9:6 Comment(2)
I know, the problem is that its not guaranted to be called... (in fact i tried and it wasnt called). I was thinking about capturing audio with AudioRecord and then sending it to SpeechRecognizer somehow, but seems imposible.Magnetics
Yes, using AudioRecord+SpeechRecognizer at the same time does not work in the current versions of Android. Regarding the RMS callback and Google's speech recognizer, it stopped working with Jelly Bean. This has been my impression while testing Babble (github.com/Kaljurand/babble). I don't know if this is a bug in Google's recognizer or their political decision to offer less via the API.Bonspiel

© 2022 - 2024 — McMap. All rights reserved.