Are Cortana APIs available for desktop applications?
Asked Answered
S

3

14

I want to develop a Windows application on Windows 10 using the new Cortana engine.

Unfortunately as far as I know, it seems to be available only on Windows Phone 8.1 project (for instance, I didn't find a way to access to the Windows.Media.SpeechRecognition namespace from a different type of Visual Studio project).

Also I wasn't able to find a good API documentation, only some very simple examples.

Edit:

Based on Peter Torr answer I've wrote some code. I've been able to recognize some word but the engine seems to struggle when it tried to recognize some simple words like "Hello", while Cortana recognized it successfully.

Am I doing something wrong?

public static class SpeechSynthetizerManager
{
    private static readonly SpeechSynthesizer synth = new SpeechSynthesizer();
    private static readonly SpeechRecognitionEngine speechRecognitionEngine = new SpeechRecognitionEngine();

    public static event EventHandler<SpeechRecognizedEventArgs> SpeechRecognized
    {
        add { speechRecognitionEngine.SpeechRecognized += value; }
        remove { speechRecognitionEngine.SpeechRecognized -= value; }
    }

    public static event EventHandler<RecognizeCompletedEventArgs> RecognizeCompleted
    {
        add { speechRecognitionEngine.RecognizeCompleted += value; }
        remove { speechRecognitionEngine.RecognizeCompleted -= value; }
    }

    static SpeechSynthetizerManager()
    {
        synth.SelectVoiceByHints(VoiceGender.Female);

        speechRecognitionEngine.LoadGrammar(new DictationGrammar());

        speechRecognitionEngine.SetInputToDefaultAudioDevice();
    }

    public static void Speak(string message)
    {
        synth.Speak(message);
    }

    public static void Listen()
    {
        speechRecognitionEngine.RecognizeAsync();
    }
}
Soares answered 10/8, 2015 at 22:51 Comment(0)
B
21

Strictly speaking, the Cortana APIs are the ones in the Windows.ApplicationModel.VoiceCommands namespace. These are not available to Classic ("Desktop") apps, but are available to Universal Windows apps on Windows 10. The reason Classic apps can't use the APIs is because they rely on concepts such as Background Tasks and App Identity that don't apply to Classic apps.

The types in the Windows.Media.SpeechRecognition namespace are also unavailable to Classic apps, but I'm not sure what the limitation is there.

Note: As @Andrew Pilley mentions, you might be able to get these types to work in a Desktop app but that's not explicitly supported at the moment.

If you just want speech recognition in a .NET app, you can use the System.Speech.Recognition namespace, which uses the same underlying technology.

Barbaraanne answered 10/8, 2015 at 23:17 Comment(4)
Hi, thank you very much for your reply. I've tried to create a class library project but in the list of available references there is no Windows.Media (or similar) assembly. The only DLLs with that name exist in path like C:\Windows\WinSxS\amd64_microsoft-windows-runtime-windows-media_31bf3856ad364e35_10.0.10240.16384_none_4f69ddc0692a98bf and they are COM dlls. Honestly, I feel a bit confused. I would have expect the dll to be part of .Net Framework (or windows Sdk) and to be a C# library.Soares
I think you missed this part of the answer: "The types in the Windows.Media.SpeechRecognition namespace are also unavailable to Classic apps". I think you need to be looking at System.Speech.Recognition.Ladin
Yes Brendan, you're right, I've misread. Next I should go to bed earlier :) By the way, I've wrote some code based on Peter reply and included it into the original question. I achieved some speech recognition, but it's still not good enough.Soares
I ended up developing a new Windows Universal App and it seems is using the Cortana speech recognition engine in the end. This is the sample that I followed: projectoxford.ai/doc/speech/Get-Started/windowsSoares
I
5

So, while Peter Torr is right about Cortana (the Windows.ApplicationModel.VoiceCommands API), the new Speech Recognition WinRT APIs (Windows.Media.SpeechRecognition) can be used in a classic C# app, if you're willing to use the WinRT Interop libraries.

I've detailed how to set that up in this answer to a stack overflow question, although the person who asked that question is having some trouble replicating what I've gotten to work locally.

Indignation answered 12/8, 2015 at 23:20 Comment(2)
Oh that looks promising! I'll try to implement it as soon as I can :)Soares
Be careful - the APIs might "work" but they are not explicitly tested for Desktop use (it doesn't have the DualApiPartitionAttribute or MuseAttribute in the WinMD).Barbaraanne
B
0

You can activate Cortana's "WIN+C" event by just calling this "uri" like a new Process(), even when the setting is turned off.

ms-cortana://StartMode=Reactive&ListeningMode=True&QuerySource=VoiceShortcutKey&Positioner=desktop&SpeechTextInputHeight=40&ColorPrevalenceEnabled=true&PersonaOnlySupported=true&TaskbarOrientation=3&SearchBoxInTaskbar=false&AppsUseLightTheme=false&HideFileExtensions=false

Bowker answered 17/12, 2017 at 9:8 Comment(0)

© 2022 - 2024 — McMap. All rights reserved.