Detect Siri Remote swipe in SwiftUI

Question

How can you recognise Siri Remote swipe gestures from SwiftUI.

I seems like it has not yet been implemented, so how can I get around that?

Smallclothes

Answer 1

the other (more reliable but hacky) way is to use the GameController low-level x/y values for dPad control,

the Siri remote is considered a Game Controller as well, and it is the first one set to the connected game controllers of the apple tv,

so onAppear of a SwiftUI view you can do something like this:

import SwiftUI
import GameController

struct SwipeTestView: View
{
    var body: some View
    {
        Text("This can be some full screen image or what not")
            .onAppear(perform: {
                let gcController = GCController.controllers().first
                let microGamepad = gcController!.microGamepad
                microGamepad!.reportsAbsoluteDpadValues = true
                microGamepad!.dpad.valueChangedHandler = { pad, x, y in
                    let fingerDistanceFromSiriRemoteCenter: Float = 0.7
                    let swipeValues: String = "x: \(x), y: \(y), pad.left: \(pad.left), pad.right: \(pad.right), pad.down: \(pad.down), pad.up: \(pad.up), pad.xAxis: \(pad.xAxis), pad.yAxis: \(pad.yAxis)"
                    
                    if y > fingerDistanceFromSiriRemoteCenter
                    {
                        print(">>> up \(swipeValues)")
                    }
                    else if y < -fingerDistanceFromSiriRemoteCenter
                    {
                        print(">>> down \(swipeValues)")
                    }
                    else if x < -fingerDistanceFromSiriRemoteCenter
                    {
                        print(">>> left \(swipeValues)")
                    }
                    else if x > fingerDistanceFromSiriRemoteCenter
                    {
                        print(">>> right \(swipeValues)")
                    }
                    else
                    {
                        //print(">>> tap \(swipeValues)")
                    }
                }
            })

            .focusable() // <-- this is required only if you want to capture 'press' and 'LongPress'
        
            .onLongPressGesture(minimumDuration: 1, perform: { // on press action
                print(">>> Long press")
            })

            .onLongPressGesture(minimumDuration: 0.01, perform: { // on press action
                print(">>> press")
            })
    }
}

this is a far more reliable solution and works every time, all you have to do is swipe finger from the center of the Siri remote outwards to your desired swipe direction (up / down / left / right),

you could also implement this way up+left, up+right, down+left, down+right, circular-clockwise swipe or circular-counter-clockwise and what ever you want.

You even might be able to implement magnification gesture and alike using the simultaneousGesture()

• Note: [12.SEP.2021] if you intend to run this code on a simulator know that for now the simulator does not support the controller as a GameController yet and the line: GCController.controllers().first will return nil, you need a real hardware to try it, see this answer

I wrote several extensions based on that and tested (tvOS 14.7), here is one that you can use as SwipeGesture for tvOS:

import SwiftUI
import GameController

// MARK: - View+swipeGestures
struct SwipeGestureActions: ViewModifier
{
    // swipeDistance is how much x/y values needs to be acumelated by a gesture in order to consider a swipe (the distance the finger must travel)
    let swipeDistance: Float = 0.7
    // how much pause in milliseconds should be between gestures in order for a gesture to be considered a new gesture and not a remenat x/y values from the previous gesture
    let secondsBetweenInteractions: Double = 0.2
    
    // the closures to execute when up/down/left/right gesture are detected
    var onUp: () -> Void = {}
    var onDown: () -> Void = {}
    var onRight: () -> Void = {}
    var onLeft: () -> Void = {}

    @State var lastY: Float = 0
    @State var lastX: Float = 0
    @State var totalYSwipeDistance: Float = 0
    @State var totalXSwipeDistance: Float = 0
    @State var lastInteractionTimeInterval: TimeInterval = Date().timeIntervalSince1970
    @State var isNewSwipe: Bool = true
    
    func resetCounters(x: Float, y: Float)
    {
        isNewSwipe = true
        lastY = y // start counting from the y point the finger is touching
        totalYSwipeDistance = 0
        lastX = x // start counting from the x point the finger is touching
        totalXSwipeDistance = 0
    }

    func body(content: Content) -> some View
    {
        content
            .onAppear(perform: {
                let gcController = GCController.controllers().first
                let microGamepad = gcController!.microGamepad
                microGamepad!.reportsAbsoluteDpadValues = false // assumes the location where the user first touches the pad is the origin value (0.0,0.0)
                let currentHandler = microGamepad!.dpad.valueChangedHandler
                microGamepad!.dpad.valueChangedHandler = { pad, x, y in
                    // if there is already a hendler set - execute it as well
                    if currentHandler != nil {
                        currentHandler!(pad, x, y)
                    }
                    
                    /* check how much time passed since the last interaction on the siri remote,
                     * if enough time has passed - reset counters and consider these coming values as a new gesture values
                     */
                    let nowTimestamp = Date().timeIntervalSince1970
                    let elapsedNanoSinceLastInteraction = nowTimestamp - lastInteractionTimeInterval
                    lastInteractionTimeInterval = nowTimestamp // update the last interaction interval
                    if elapsedNanoSinceLastInteraction > secondsBetweenInteractions
                    {
                        resetCounters(x: x, y: y)
                    }
                    
                    /* accumelate the Y axis swipe travel distance */
                    let currentYSwipeDistance = y - lastY
                    lastY = y
                    totalYSwipeDistance = totalYSwipeDistance + currentYSwipeDistance
                    
                    /* accumelate the X axis swipe travel distance */
                    let currentXSwipeDistance = x - lastX
                    lastX = x
                    totalXSwipeDistance = totalXSwipeDistance + currentXSwipeDistance
                    
//                    print("y: \(y), x: \(x), totalY: \(totalYSwipeDistance) totalX: \(totalXSwipeDistance)")
                    
                    /* check if swipe travel goal has been reached in one of the directions (up/down/left/right)
                     * as long as it is consedered a new swipe (and not a swipe that was already detected and executed
                     * and waiting for a few milliseconds stop between interactions)
                     */
                    if (isNewSwipe)
                    {
                        if totalYSwipeDistance > swipeDistance && totalYSwipeDistance > 0 // swipe up detected
                        {
                            isNewSwipe = false // lock so next values will be disregarded until a few milliseconds of 'remote silence' achieved
                            onUp() // execute the appropriate closure for this detected swipe
                        }
                        else if totalYSwipeDistance < -swipeDistance && totalYSwipeDistance < 0 // swipe down detected
                        {
                            isNewSwipe = false
                            onDown()
                        }
                        else if totalXSwipeDistance > swipeDistance && totalXSwipeDistance > 0 // swipe right detected
                        {
                            isNewSwipe = false
                            onRight()
                        }
                        else if totalXSwipeDistance < -swipeDistance && totalXSwipeDistance < 0 // swipe left detected
                        {
                            isNewSwipe = false
                            onLeft()
                        }
                        else
                        {
                            //print(">>> tap")
                        }
                    }
                }
            })
    }
}

extension View
{
    func swipeGestures(onUp: @escaping () -> Void = {},
                       onDown: @escaping () -> Void = {},
                       onRight: @escaping () -> Void = {},
                       onLeft: @escaping () -> Void = {}) -> some View
    {
        self.modifier(SwipeGestureActions(onUp: onUp,
                                          onDown: onDown,
                                          onRight: onRight,
                                          onLeft: onLeft))
    }
}

and you can use it like this:

struct TVOSSwipeTestView: View
{
    @State var markerX: CGFloat = UIScreen.main.nativeBounds.size.width / 2
    @State var markerY: CGFloat = UIScreen.main.nativeBounds.size.height / 2
    
    var body: some View
    {
        VStack
        {
            Circle()
                .stroke(Color.white, lineWidth: 5)
                .frame(width: 40, height: 40)
                .position(x: markerX, y: markerY)
                .animation(.easeInOut(duration: 0.5), value: markerX)
                .animation(.easeInOut(duration: 0.5), value: markerY)
        }
            .background(Color.blue)
            .ignoresSafeArea(.all)
            .edgesIgnoringSafeArea(.all)
            .swipeGestures(onUp: {
                print("onUp()")
                markerY = markerY - 40
            },
                           onDown: {
                print("onDown()")
                markerY = markerY + 40
            },
                           onRight: {
                print("onRight()")
                markerX = markerX + 40
            },
                           onLeft: {
                print("onLeft()")
                markerX = markerX - 40
            })
        
            .focusable() // <-- this is required only if you want to capture 'press' and 'LongPress'
        
            .onLongPressGesture(minimumDuration: 1, perform: { // on press action
                print(">>> Long press")
            })

            .onLongPressGesture(minimumDuration: 0.01, perform: { // on press action go to middle of the screen
                markerX = UIScreen.main.nativeBounds.size.width / 2
                markerY = UIScreen.main.nativeBounds.size.height / 2
            })
    }
}

Kenric

Trying this, and it results in the (onLeft, onRight, etc) closures still being called even after I close the page of this View. Unless I set valueChangedHandler to nil after. – Ataman

Does this work in the tvOS - Simulator too? For me it works only on a real device. – Lachellelaches

Answer 2

i have 2 answers for this question, i will answer them separately and let you guys decide which is best,

first is the apple way (which obviously doesn't work allways, there are more clicks captured then swipes):

import SwiftUI

struct SwipeTestView: View
{
    var body: some View
    {
        Text("This can be some full screen image or what not")
            .focusable() // <-- this is a must
            .onMoveCommand { direction in  // <-- this $#!* can't tell a move swipe from a touch (direction is of type: MoveCommandDirection)
                print("Direction: \(direction)")
                if direction == .left { print(">>> left swipe detected") }
                if direction == .right { print(">>> right swipe detected") }
                if direction == .up { print(">>> up swipe detected") }
                if direction == .down { print(">>> down swipe detected") }
            }

    }
}

you really (and i can't emphasize this enough) have to swipe on the very edge of the Siri-remote or the iPhone Siri remote widget,

so try to swipe on these yellow areas and try not to tap you'r finger and then swipe, rather gently swipe outwards and let you'r finger get outside of the remote edge completely

Result expected:

i literally tried 100+ times before successfully captured a swipe (clearly not something for production), hopefully tvOS 15 and higher will fix this

Kenric

onMoveCommand doesn't capture swipes in tvOS 16 – Outtalk

shame, was good while it lasted – Kenric