Cant start service? (Speech recog)

Question

I want to listen for the word hello using pocketsphinx in a service continuously

I get the error. Here is the full stack trace. Here is a small portion of it.

Unable to create service curlybrace.ruchir.myApp.MyService: java.lang.RuntimeException: new_Decoder returned -1

It is caused by this:

            setupRecognizer(assetDir); //SETUP

and this:

                .getRecognizer();

In my onCreate:

 Log.v(TAG, "Voice recognition activated!");

        //Register voice recog listener :)

        Assets assets = null;
        try {
            assets = new Assets(MyService.this);
            File assetDir = assets.syncAssets();
            setupRecognizer(assetDir); //SETUP

            Log.v(TAG, "Set up listener");
        } catch (IOException e) {
            e.printStackTrace();
        }

Here is my setupRecognizer method:

  private void setupRecognizer(File assetDir) throws IOException {

        recognizer = defaultSetup()
                .setAcousticModel(new File(assetDir, "hmm/en-us-semi"))
                .setDictionary(new File(assetDir, "lm/cmu07a.dic"))
                .setKeywordThreshold(1e-5f)
                .getRecognizer();

        recognizer.addListener(this);
       // recognizer.addKeywordSearch("Hello", assetDir); //I don't know what this does...
    recognizer.startListening("Hello"); //Start listeneing


    }

Here is one of the implemented methods:

@Override
    public void onPartialResult(Hypothesis hypothesis) {

        String text = hypothesis.getHypstr();
        if (text.equals("Hello")) {
            //  do something

            Log.v(TAG, "SPEECH RECOGNIZED HELLO!");
        }

    }

I would appreciate any feedback. Positive, negative, even a comment. At this point I am desperate, after trying for 2 days!

Guenon

Just as an idea: did you register the microphone permission in your manifest? – Prolix

@Prolix No, I did not – Guenon

your app needs the permission to listen to the microphone, even if you use a library for it. Did you tried adding it? – Prolix

@Prolix Should I add these permissions? <uses-permission android:name="android.permission.WRITE_EXTERNAL_STORAGE" /> <uses-permission android:name="android.permission.RECORD_AUDIO" /> – Guenon

@Prolix Then why was I getting a service can't start, shouldn't I have gotten a security exception? – Guenon

@Prolix I added the permissions, but it still doesn't work. Giving me the same error... – Guenon

I think there is something wrong with the Sphinx file in your assets folder – Zeigler

Take a look at the logcat output, the error should be described there. – Alti

@HoanNguyen Hmm...that's odd. Heres what I did: I downloaded the CMUsphinx demo, and I copied the entire asset folder from the demo to my existing project. That is how I got my assets folder in my src/main. Was there something else that I was supposed to do? Did I do something wrong? Thanks so much for your help! – Guenon

@AlexanderSolovets I have, and I have even posted the link in my question, but I don't understand my error. What could the problem be? Thanks so much for your help, I have been stuck for almost 4 days, and I would really appreciate all feedback! P.S. Here is the log cat output, in case it was hard to find in the question: gist.github.com/anonymous/d68e9ac7e5d98315a5a4 Thanks so much :) – Guenon

@RuchirBaronia This doesn't look like a logcat output. – Alti

Did you add anything to the grammar file? – Zeigler

@HoanNguyen No, I didn't. What was I supposed to add to the grammar file? Please let me know. :) – Guenon

Are you just doing keyword spotting? – Zeigler

@HoanNguyen Yes, all I want to do is listen for the word "hello". Should I do something different? Please let me know! :) – Guenon

You will get probably get a lot of false positive with just one word. I will post my code for the keyword spotting I did with Sphinx. – Zeigler

@HoanNguyen Thanks, ill be looking forward to your answer! Actually, if you think it would not be accurate with one word, then I'll be happy to use 2 or 3 words to improve accuracy. I am planning on having the user provide their phrase, so I thought that I could just keyword spot for that phrase. Please let me know how I should do the keyword spotting. I am looking forward to your answer! :) – Guenon

.raw files are being created in /android/data/sync on my device – Diathermic

can u help me ? #39506771 – Hinton

@Hinton Your question seems to be deleted? What was the problem? – Guenon

Answer 1

You have this:

private void setupRecognizer(File assetDir) throws IOException {
        recognizer = defaultSetup() 
                .setAcousticModel(new File(assetDir, "hmm/en-us-semi"))
                .setDictionary(new File(assetDir, "lm/cmu07a.dic"))
                .setKeywordThreshold(1e-5f) 
                .getRecognizer(); 
        recognizer.addListener(this);
       // recognizer.addKeywordSearch("Hello", assetDir); //I don't know what this does... 
    recognizer.startListening("Hello"); //Start listeneing 
    } 

Try changing it to this:

private void setupRecognizer(File assetDir) throws IOException {
        recognizer = defaultSetup() 
                .setAcousticModel(new File(assetDir, "hmm/en-us-semi"))
                .setDictionary(new File(assetDir, "lm/cmu07a.dic"))
                .setKeywordThreshold(1e-5f) 
                .getRecognizer(); 
        recognizer.addListener(this);

    //Add this:
    File digitsGrammar = new File(modelsDir, "grammar/digits.gram");
    recognizer.addKeywordSearch(DIGITS_SEARCH, digitsGrammar);
    } 

To begin speech recon, call this from button. When it works, call it from a service, to keep things simpler:

    recognizer.startListening("Hello"); //Start listeneing 

Now, create a new file called digits.gram, and put it inside a folder called here: /youProjectRootFolder/grammar/digits.gram This file is really a .txt file, but change the extension to .gram when you are done putting this text inside:

hello /1e-1/
hi /1e-1/
bye /1e-1/
goodbye /1e-1/
...etc. /1e-1/

Here you will find a similar situation: Recognizing multiple keywords using PocketSphinx Good Luck!

Leadwort

Thanks so much for your answer! I cant seem to find the grammar folder in my root directory...Do I need to create it? Here is a screenshot of my directories in project view. Was I supposed to make the grammar file? Also, I was a bit confused on the setAcousticModel and the setDictionary methods, and why they take a file parameter. Why do we even need the assetDir file? I just got that from the demo. Please let me know :) Thanks so much Josh! – Guenon

Yes, you need to create your own "grammar" folder if your project doesn't have one, and then make the grammar file yourself: just copy paste the text I mention above and then change the extension from .txt to .gram,. I am not sure how does assetDir work deep below, but I do know that it allows you to load the dictionary and the acoustic model from their files. @RuchirBaronia – Leadwort

Hmm...I'm still having the same problem on this line .getRecognizer();. The same new decoder returned -1 error is happening...and I don't know why! Maybe I didn't add the digits.gram file properly, is it like this? snag.gy/VCCBH.jpg – Guenon

Also, I noticed that there is also another digits.gram file in the assets folder...are we supposed to reference that one? – Guenon

Please let me know on what the issue can be. If you need any more information, feel free to ask. I have been stuck on this for a very long time, and I would really appreciate your help. Thanks so much Josh! – Guenon

Wow, back then I did it in Eclipse so there are big differences, try 1) putting the grammar folder with the .gram file inside assets. 2) Have you checked this guide?:cmusphinx.sourceforge.net/wiki/tutorialandroid – Leadwort

Yeah, I've seen that guide, but there isn't much information on it. Anyway, there already is a .gram file in my assets file...should I reference that? How? Please let me know, thanks – Guenon

Also, I think I may know why this problem is happening. In the guide, they mention to edit the build.gradle file to run assets.xml, but I don't know how to do that. Here is a screenshot of the part of the guide that I didn't do. How do I do that? I tried adding that code they gave into the build.gradle file, but that came up with lots of errors. snag.gy/MuSFb.jpg – Guenon

Was I supposed to add this code in my gradle file? snag.gy/MuSFb.jpg – Guenon

Also, what does the string in the startListening method parameter do? – Guenon

control is not going to the onResult().it jumps to onPArtialResult() multiple times without saying anything – Diathermic

@Mansuu....onPartialResult triggers many times while the speech recognition takes place. If you stop the recognizer, then you will trigger onResult. However onPartialResult should return at least something, so you have another error elsewhere. – Leadwort

@Leadwort the main problem i face is without saying anything onResult() is called and gives me text from my digits.gram file – Diathermic

@Diathermic This type of speech recognition will only match to words from your digits grammar file. In this case, I think you are getting false positives because your threshold values are too permissive. Play with the values up and down until you don't get false positives when the mic is silent. – Leadwort

Answer 2

For command, the code below is what I did and it works well. If you do only keyword spotting then look at the keyword spotting example bundle in the Sphinx download and modify the code below.

Make sure that the assets --> sync folder contains only the following files

folder en-us-ptm
assets.lst 
cmudict-en-us.dict
cmudict-en-us.dict.md5
command.gram
your_preferred_name.dict

If you allow user to set the command then you do not need the command and your_preferred_name.dict. You can add it in code later and save it in the appropriate directory below. For keyword spotting replace the command.gram with whatever the name in the Sphinx example.

In the assets --> sync folder modify the files listed to have the content below. You can edit these file with notepad++

assets.lst

cmudict-en-us.dict
en-us-ptm/README
en-us-ptm/feat.params
en-us-ptm/mdef
en-us-ptm/means
en-us-ptm/noisedict
en-us-ptm/sendump
en-us-ptm/transition_matrices
en-us-ptm/variances 

command.gram

hello /1/

If the app has trouble understanding adjust the threshold parameter i.e. /1e-8/ The smaller the threshold the easier for the recognizer to pick up the word but also it is easier to get false positive. For keyword spotting replace the Sphinx keyword example with your keyword.

your_prefered_name.dict
Copy the whole line in the cmudict-en-us.dict having the word in command.gram in this example it is the word hello. I have a separate dict so that the file is much smaller so that the dict search is improved a bit. So your your_prefered_name.dict should looks like

hello HH AH L OW
hello(2) HH EH L OW

For keyword spotting I think you can string the words together (not sure you have to try to see if it will work) so for example hello world will be

hello world HH AH L OW .... (the dot is for world)

At the start of your app create a directory say "sphinx"

String createSphinxDir()
{
    SharedPreferences prefs = PreferenceManager.getDefaultSharedPreferences(this);
    String sphinxDir = prefs.getString("sphinx", null);
    if (sphinxDir == null)
    {
        Assets assets;
        try
        {
            assets = new Assets(this);
            File sphinxDirFile = assets.syncAssets();
            if (sphinxDirFile != null)
            {
                sphinxDir = sphinxDirFile.getAbsolutePath();
                Editor editor = prefs.edit();
                editor.putString("sphinx", sphinxDir);
                editor.commit();
                // Also save the command.gram and your_preferred_name.dict
                // to the sphinx dir here. Or save the them later to this
                // dir if you allow user to set the command or keyword
            }
        }
        catch (IOException e)
        {

        }
    }
    return sphinxDir;
}

Then wherever you initiate the speech recognizer

String sphinxDir = createSphinxDir();
        if (sphinxDir != null)
        {
            try
            {
                mSpeechRecognizer = defaultSetup()
                        .setAcousticModel(new File(sphinxDir, "en-us-ptm"))
                        .setDictionary(new File(sphinxDir, "your_preferred_name.dict"))
                        .setBoolean("-allphone_ci", true)
                        .getRecognizer();
                mSpeechRecognizer.addListener(your listener);

// check if file exists here I have a util called FileIOUtils, you should create a method to check.                 
if ((new File(sphinxDir + File.separator + "command.gram")).isFile())
                {
                    mSpeechRecognizer.addKeywordSearch("wakeup", 
                            new File(sphinxDir + File.separator + "command.gram"));
                }

                // Or wherever appropriate
                 startListening("wakeup");
            }
            catch (IOException e)
            {

            }
        }

For keyword spotting just change the above to the one in the Sphinx example.

Zeigler

Hey Hoan, I spent many hours going through that and trying to understand it, but I still am getting some errors. I have created this gist, and have commented all the places where there are errors. Please let me know on how I should resolve these, or what I am doing wrong. Thanks so much. gist.github.com/anonymous/e67e876dc1a33df25b2c – Guenon

gist.github.com/anonymous/e67e876dc1a33df25b2c – Guenon

Hey, I tried this, and the error went away! The problem is, it is still not working properly though... – Guenon

Basically, none of the call back methods are getting called (onResult, onPartialResult, onBeginingOfSpeech, etc.) What can I do to fix this? Here is the code: gist.github.com/anonymous/2fa9085937ad8106856a – Guenon

I know that none of the call back methods are getting called, because I am not getting the logs that I placed. – Guenon

Should be mSpeechRecognizer.startListening() instead of mSpeechRecognizer.startListening("hello"); – Zeigler

That gets me an error though. For me, there is no way to use the startListening method without any parameters. Here is a screenshot of the options: snag.gy/hvLH9.jpg – Guenon

Here is another screenshot of how there is no startListening method: snag.gy/VDE4k.jpg – Guenon

What can I do to fix this? Thanks so much for your help, I really appreciate it. :) – Guenon

What I did is so long ago, I forgot that I create a startListening method without a param. In your case it should be startListening("wakeup"); – Zeigler

Okay, so after 2 days of debugging, I figured out the problem. I placed logs everywhere, and it turns out, sphinxDir is null! That's why the listener doesn't even get setup, because sphinxDir is null. How can I fix this? – Guenon

private void setupRecog() { String sphinxDir = createSphinxDir(); Log.v(TAG, "ABOUT TO CREATE SETUP"); if (sphinxDir != null) { – Guenon

The part after the if(sphinxDir != null) just doesn't execute. That means it is null...Why is that? That must mean a problem with createSphinxDir() method... How can I fix this? Thanks so much Hoan! – Guenon

Why don't you put log in the createSphinxDir to see why the sphinx directory is not created. You might have to unplug your device from the computer and run the app. – Zeigler

Okay, I did what you said, and it seems that the entire try block in the createSphinxDir() isn't executing. Here is the stack trace: – Guenon

Here is the try block, with the part that doesn't execute highlighted: – Guenon

The variances.md5 should be on the en-us-ptm folder. What demo project did you download? The official Sphinx Android demo should contain this md5 file. – Zeigler

Actually, I do have the variances.md5 file in my en-us-ptm folder, I was wrong in the previous comment. But, why isn't it still working, even though I do have the md5? – Guenon

Here's a screenshot of the md5 file in my en-us-ptm folder: – Guenon

I've found a similar problem here, but here the person is using SD card...I am not...https://mcmap.net/q/341009/-getexternalfilesdir-fail – Guenon

What exception do you get? If it is not clear from the printStackTrace try log.d(TAG, "IOException: " + e.getMessage()); after the e.printStackTrace method. – Zeigler

Okay, after making that log, e.getMessage() is sync/en-us-ptm/variances .md5 – Guenon

it seems that the file is not found...which is odd, because the file is there (you can see in the screenshot too)...What should I do? Thanks! :) – Guenon

Let us continue this discussion in chat. – Zeigler

It looks like we were on chat at different times...anyway, the stacktrace does put a space, as you said in the chat. – Guenon

Is that normal? Is it looking for a file that doesn't exist? Thanks :) – Guenon

Okay, so I added the space in the variances.md5 file, but now I am getting a file not found on just the variances file...even though it is there... – Guenon

Hey, I added some more detail to the problem I am having: #35664797 – Guenon

How do I rename it with a space at the end...? – Guenon

It seems that you are right that it is looking for the variances file rather than the variances file, and so I want to add a space at the end of the variances file. – Guenon

I don't think it is possible to add a space to the end of a file – Guenon

See if the variances file name in the assets.lst have a space at the end. If it is just remove it and then remove the space at variances.md5 – Zeigler

Let us continue this discussion in chat. – Guenon

Hey Hoan, so I tried what you suggested in chat, and I am creating the commands.gram file programmatically now. Here is my new code that I made: gist.github.com/anonymous/67c45a580c4d1e54662a – Guenon

Unfortunately, I am still getting that error, but I noticed that before the error, the recognition does successfully start, as I get a log message like the following: 02-27 17:01:52.793 11696-11696/curlybrace.ruchir.emergency I/SpeechRecognizer: Start recognition "hello" – Guenon

And I did not make this log, I think it is a system log from the library, so the recognition does start for a brief second, but then I get this error. Here is the log message of the error: gist.github.com/anonymous/923ec5c132485d26d2f5 – Guenon

The log message also says shutting down VM before the error...I don't know why. – Guenon

I've done some research and I see that this user had the same error as me: #23773220 – Guenon

I think that this warning in the stack trace may be the problem: thread exiting with uncaught exception (group=0x4168be00). – Guenon

Why don't you use apache common io. Just add compile 'commons-io:commons-io:2.4' to the module build.graddle. Or make sure the command.gram exits and has the expected text. – Zeigler

Wait, I got it! I just needed to add the assets.xml into my app directory! But, now I am getting a problem in onPartialResult...Hypothesis hypothesis is always null. Please let me know. Thanks, Ruchir :) – Guenon

try hello /1e-10/ the lower the value the easier for the recognizer to pick up the word but also easy to have false positive. – Zeigler

what to do if i want to recognize a word out of a set of words – Diathermic