iOS Swift 5 - How to implement Siri VoIP command "Call Person using the Example app" support without having to add shortcuts?
Asked Answered
D

2

6

Goal

I want to be able to trigger a VoIP call with Siri saying "Call Emily using the Next app", without adding shortcuts.

Emily is a contact I added to my contacts, which holds my own phone number. I test the example app on my daily driver phone, so it has a SIM card for calling.

Issue breakdown

  • 1 type of scenario works: if the bundle (display) name of the app is identical to what I say, e.g. "Next", Siri will properly open the app and initiate the call code, fantastic!
  • However, the app bundle name must remain Nexxt.
  • If I launch the app with the app name Nexxt and say "Call Emily using the Next app", Siri will respond with:

I don't see a "Emily" in your Contacts.

  • I have added alternative app names, namely CFBundleSpokenName, aka Accessibility Bundle Name and INAlternativeAppNames:
    • Next
    • Dog
    • Cat
  • The alternative app names work! If I say "Open the Next app", "Open the Dog app" or "Open the Cat app" even when the actual app name is Nexxt, the app will be opened.
  • However, if I say "Call Emily using the Dog app", Siri will respond with:

I don't see an app for that. You'll need to download one. Search the App Store

  • I can't seem to make it work for my specific goal. Considering the alternative app names work, I figured something may be fundamentally wrong or missing in my "INStartCallIntent" implementation.

Note, if you have trouble with changing the Display Name. Select project, change the display name, then click away from the project (to any file) and select the project again. Run the app and the name should update.

Code

Here's the code. It works for "Call Emily using the Next app" if my Display Name is Next. It also works for "Call Emily using the Dog app" if my Display Name is Dog.

The example app is written in SwiftUI code with a minimal setup to test the Siri feature.

TestSiriSimple -> TestSiriSimpleIntents -> IntentHandler:

import Intents

class IntentHandler: INExtension {
    
    override func handler(for intent: INIntent) -> Any {
        if intent is INStartCallIntent {
            return StartCallIntentHandler()
        }

        return self
    }
}

TestSiriSimple -> Shared -> StartCallIntentHandler:

import Foundation
import Intents

class StartCallIntentHandler: NSObject, INStartCallIntentHandling {
    
    func confirm(intent: INStartCallIntent) async -> INStartCallIntentResponse {
        let userActivity = NSUserActivity(activityType: String(describing: INStartCallIntent.self))
        return INStartCallIntentResponse(code: .continueInApp, userActivity: userActivity)
    }

    func handle(intent: INStartCallIntent, completion: @escaping (INStartCallIntentResponse) -> Void) {
        let response: INStartCallIntentResponse
        defer {
            completion(response)
        }
        
        let userActivity = NSUserActivity(activityType: String(describing: INStartCallIntent.self))
        response = INStartCallIntentResponse(code: .continueInApp, userActivity: userActivity)
        completion(response)
    }
    
    func resolveContacts(for intent: INStartCallIntent) async -> [INStartCallContactResolutionResult] {
        guard let contacts = intent.contacts, contacts.count > 0 else {
            return []
        }
        
        return [INStartCallContactResolutionResult.success(with: contacts[0])]
    }
    
    func resolveCallCapability(for intent: INStartCallIntent) async -> INStartCallCallCapabilityResolutionResult {
        INStartCallCallCapabilityResolutionResult(callCapabilityResolutionResult: .success(with: intent.callCapability))
    }
    
    func resolveDestinationType(for intent: INStartCallIntent) async -> INCallDestinationTypeResolutionResult {
        INCallDestinationTypeResolutionResult.success(with: .normal)
    }
}

The root app class is unchanged. TestSiriSimple -> Shared -> ContentView:

import SwiftUI
import Intents

struct ContentView: View {
    @State private var status: INSiriAuthorizationStatus = .notDetermined
    
    var body: some View {
        Text("Hello, world! Siri status: \(status.readableDescription)")
            .padding()
            .onAppear {
                requestSiri()
            }
            .onContinueUserActivity(NSStringFromClass(INStartCallIntent.self)) { userActivity in
                continueUserActivity(userActivity)
            }
    }
    
    private func requestSiri() {
        INPreferences.requestSiriAuthorization { status in
            self.status = status
        }
    }
    
    private func continueUserActivity(_ userActivity: NSUserActivity) {
        if let intent = userActivity.interaction?.intent as? INStartCallIntent {
            // Find person from contacts or create INPerson from app specific contacts.
            // Execute VoIP code.
            // I consider it a success if Siri responds with "Calling Now", opens the app and reaches this code.
        }
    }
}

extension INSiriAuthorizationStatus {
    var readableDescription: String {
        switch self {
        case .authorized:
            return "Authorized"
        case .denied:
            return "Denied"
        case .notDetermined:
            return "Not determined"
        case .restricted:
            return "Restricted"
        default:
            return "Unknown"
        }
    }
}
Details

TestSiriSimple -> (Main) Info.plist

TestSiriSimpleIntents -> Info.plist

Privacy - Siri Usage Description = Siri wants to let you start calls in this app.

TestSiriSimpleIntents target has INStartCallIntent as a supported intent

If you have any ideas, they are more than welcome!

I'm willing to share a zip of my example code if you could show me how I would go about that in StackOverflow. If any other other info would help, don't hesitate to comment!

Diligent answered 18/8, 2022 at 15:35 Comment(1)
I submitted a Technical Support Incident with Apple developer.apple.com/contact/technical/#!/request/form. I'll update this post/add an answer with future findings!Diligent
D
1

There's a better phrase than Call Emily using the Nexxt app, try the following:

Call Emily on Nexxt

See documentation: https://developer.apple.com/documentation/sirikit/instartaudiocallintent

Example phrase

Diligent answered 6/9, 2022 at 11:35 Comment(1)
Confirmed that this works, solving the scope of my question. Instead of diving deep into providing alternative pronunciations (see intent phrases), we'll inform our users the correct way of using the Siri command through our design.Diligent
D
0

Update 1: Currently resolving the following feedback. Posted here now so others may resolve similar issues. Will update with results.

Here's the response I got from the Apple TSI:

Hello Lex, Thank you for the sample project.

Given the testing you’ve done so far, it’s possible that this comes down to a recognition issue, where Siri needs to improve its recognition of “Nexxt” to associate your app and route requests to it. Before we get to that ultimate conclusion however, I want to try the following things to better round out your testing, in case they make a difference.

  1. Add an app vocabulary file with the IntentPhrases dictionary. Since you are building on the system intent, INStartCallItent, and not a custom intent, you’ll need this file when submitting to the App Store. This file will contain some suggested invocation phrases you expect customers to use, like those shown in the INStartAudioCallIntent documentation.

https://developer.apple.com/documentation/sirikit/registering_custom_vocabulary_with_sirikit/global_vocabulary_reference https://developer.apple.com/documentation/sirikit/instartaudiocallintent

To see a sample app with this file added, take a look at the workaround intent domain. https://developer.apple.com/documentation/sirikit/workouts/handling_workout_requests_with_sirikit

  1. Donate contact information to the system. Your notes show you testing with a contact named Emily, but it’s not clear how Emily is resolved to a person to contact. If there are contacts in the Contacts app that you’d like to associate with your app, you should donate as much info about your app’s relation to the contact as possible, as discussed by this article: https://developer.apple.com/documentation/foundation/app_extension_support/supporting_suggestions_in_your_app_s_share_extension

While that article is about sharing, the same mechanism underpins a broad range of areas where the system needs to associate actions between a contact and an app.

If you manage your own contact information outside of the system Contacts, then you should be donating the contact information to the system as user vocabulary, described by this article: https://developer.apple.com/documentation/sirikit/registering_custom_vocabulary_with_sirikit

  1. Try the Speakerbox sample code for VoIP apps. While this sample doesn’t integrate 1 and 2, you can run additional tests with this sample project by integrating items 1 and 2 into it, along with your real app name Nexxt (and the alternative name hints in the Info.plist file), and see how things behave there with our example of how a VoIP app should integrate with the system. https://developer.apple.com/documentation/callkit/making_and_receiving_voip_calls_with_callkit

Please let me know how adding those recommended best practices go, and if they change the results of your testing.

Ed Ford  DTS Engineer

Diligent answered 5/9, 2022 at 12:23 Comment(0)

© 2022 - 2024 — McMap. All rights reserved.