Speech Recognition on Kinect

Question

I have a project for recognition. It works, but if I use this project how a class and call its methods from other class I have a problem with exception on line:

sre = new SpeechRecognitionEngine(ri.Id);

The error being:

No recognizer of the required ID found.

The code:

KinectAudioSource source = kinectSensor.AudioSource;
source.EchoCancellationMode = EchoCancellationMode.None; // No AEC for this sample
source.AutomaticGainControlEnabled = false; // Important to turn this off for speech recognition
//  source.SystemMode = SystemMode.OptibeamArrayOnly;
speechRecognizer = CreateSpeechRecognizer();

using (Stream s = source.Start())
 {
   speechRecognizer.SetInputToAudioStream(s, new SpeechAudioFormatInfo(EncodingFormat.Pcm, 16000, 16, 1, 32000, 2, null));
   Console.WriteLine("Recognizing speech. Say: 'purple', 'green' or 'blue'. Press ENTER to stop");
   speechRecognizer.RecognizeAsync(RecognizeMode.Multiple);
   Console.ReadLine();
   Console.WriteLine("Stopping recognizer ...");
   speechRecognizer.RecognizeAsyncStop();
  }

 private static SpeechRecognitionEngine CreateSpeechRecognizer()
 {
   RecognizerInfo ri = GetKinectRecognizer();

   SpeechRecognitionEngine sre;
   //if (ri == null) return 0;
   sre = new SpeechRecognitionEngine(ri.Id);
   var colors = new Choices();
   colors.Add("red");
   colors.Add("green");
   colors.Add("blue");
   var gb = new GrammarBuilder { Culture = ri.Culture };
   gb.Append(colors);

   // Create the actual Grammar instance, and then load it into the speech recognizer.
   var g = new Grammar(gb);
   sre.LoadGrammar(g);
   sre.SpeechRecognized += SreSpeechRecognized;
   sre.SpeechHypothesized += SreSpeechHypothesized;
   sre.SpeechRecognitionRejected += SreSpeechRecognitionRejected;
   return sre;
  }
private static RecognizerInfo GetKinectRecognizer()
  {
   Func<RecognizerInfo, bool> matchingFunc = r =>
     {
      string value;
      r.AdditionalInfo.TryGetValue("Kinect", out value);
      return "True".Equals(value, StringComparison.InvariantCultureIgnoreCase) && "en-US".Equals(r.Culture.Name, StringComparison.InvariantCultureIgnoreCase);
       };
      return SpeechRecognitionEngine.InstalledRecognizers().Where(matchingFunc).FirstOrDefault(); 
    }

Eskisehir

debug the ID. Check if the value is correct – Xenogenesis

Make sure you install the required 32-bit speech engine to support Kinect. Follow the prerequisites listed on this page: channel9.msdn.com/coding4fun/kinect/… – Mizuki

what is the Platform Target set to? – Brady

Answer 1

I think your GetKinectRecognizer() method is not correct.

Doesn't TryGetValue() return a boolean if it is found and the value found as an out parameter? You aren't doing anything with the returned boolean from TryGetvalue().

Are you expecting the AdditionalInfo dictionary to have a key equal to "Kinect" and a string "True" or "False" value? That is what you code appears to look for.

Is this code based on an example you can point to. I don't really follow what your testing for in the matchingFunc. You are ignoring the return value from TryGetvalue, you are looking for an AdditionalInfo key named "Kinect" with a string value of "True", and a recognizer with culture of "en-US".

Why don't you just dump the contents of SpeechRecognitionEngine.InstalledRecognizers() and make sure it contains what you think it contains. This is old school, but useful:

foreach (RecognizerInfo ri in SpeechRecognitionEngine.InstalledRecognizers())
{
    Debug.WriteLine(String.Format("Id={0}, Name={1}, Description={2}, Culture={3}", ri.Id, ri.Name, ri.Description, ri.Culture));
    foreach(string key in ri.AdditionalInfo.Keys)
    {
        Debug.WriteLine(string.Format("{0} = {1}", key, ri.AdditionalInfo[key]));
    }
    Debug.WriteLine("-");
}

I don't have Kinect SDK installed, but on my Windows 7 machine it shows:

Id=MS-1033-80-DESK, Name=MS-1033-80-DESK, Description=Microsoft Speech Recognizer 8.0 for Windows (English - US), Culture=en-US
VendorPreferred = 
CommandAndControl = 
Version = 8.0
Language = 409;9
Desktop = 
SupportedLocales = 409;1009;3409;9
AudioFormats = 16;18;20;22;45;53;{6F50E21C-E30E-4B50-95E9-21E8F23D15BD}
SpeakingStyle = Discrete;Continuous
WildcardInCFG = Anywhere;Trailing
Dictation = 
Hypotheses = 
Alternates = CC;Dictation
windowsV6compatible = 
Name = MS-1033-80-DESK
DictationInCFG = Anywhere;Trailing
UPSPhoneSet = 
WordSequences = Anywhere;Trailing
Vendor = Microsoft
-
Id=MS-2057-80-DESK, Name=MS-2057-80-DESK, Description=Microsoft Speech Recognizer 8.0 for Windows (English - UK), Culture=en-GB
 = 
VendorPreferred = 
CommandAndControl = 
Version = 8.0
Language = 809
Desktop = 
SupportedLocales = 809;C09;1409;1809;1C09;2009;2409;2809;2C09;3009;4009;4409;4809;9
AudioFormats = 16;18;20;22;45;53;{6F50E21C-E30E-4B50-95E9-21E8F23D15BD}
SpeakingStyle = Discrete;Continuous
WildcardInCFG = Anywhere;Trailing
Dictation = 
Hypotheses = 
Alternates = CC;Dictation
windowsV6compatible = 
Name = MS-2057-80-DESK
DictationInCFG = Anywhere;Trailing
UPSPhoneSet = 
WordSequences = Anywhere;Trailing
Vendor = Microsoft
-
-

Make sure the values you are looking for in the AdditionalInfo dictionary are really there. Then craft your matchingFunc to check for it.

Wiseman

Answer 2

Try this. It works for me.

 private static RecognizerInfo GetKinectRecognizer()
        {
            foreach (RecognizerInfo recognizer in SpeechRecognitionEngine.InstalledRecognizers())
            {
                System.Diagnostics.Debug.Write(recognizer.Culture.Name+"\n\n");
                //string value;
                //recognizer.AdditionalInfo.TryGetValue("Kinect",out value);
                if ("en-US".Equals(recognizer.Culture.Name, StringComparison.OrdinalIgnoreCase))
                {
                    return recognizer;
                }

            }

            return null;
        }

Greaten

Worked for me too. Just for the record, I just installed the 64x verions of, Microsoft Speech Platform SDK v11.0, Microsoft Speech Platform Runtime v11.0 and Redistributable Language Packs v11.0 (in this order). All downloaded from here. – Montfort