How to detect key press and release?

Question

Not much to say other than the title. I want to be able to take action in a SwiftUI view when a key is pressed and when it is released (on macOS). Is there any good way to do this in SwiftUI, and if not, is there any workaround?

Herophilus

Answer 1

Unfortunately keyboard event handling is one of those areas where it's painfully obvious that SwiftUI was designed first and foremost for iOS, with macOS being an afterthought.

If the key you're trying to detect is a modifier to a mouse click, such as cmd, option, or shift, you can use the .modifiers with onTapGesture to distinguish it from an unmodified onTapGesture. In that case, my experience with it is that you want the .onTapGesture call that uses .modifiers to precede the unmodified one.

Handling general key events for arbitrary views requires going outside of SwiftUI.

If you just need it for one View, one possibility is to implement that view with AppKit so you can receive the keyboard events via the ordinary Cocoa firstResponder mechanism, and then wrap that view in SwiftUI's NSViewRepresentable. In that case your wrapped NSView would update some @State property in NSViewRespresentable. A lot of developers using SwiftUI for macOS do it this way. While this is fine for a small number of views, if it turns out that you have to implement a lot of views in AppKit to make them usable in SwiftUI, then you're kind of defeating the point of using SwiftUI anyway. In that case, just make it an ordinary Cocoa app.

But there is another way...

You could use another thread that uses CGEventSource to poll the keyboard state actively in conjunction with a SwiftUI @EnvironmentObject or @StateObject to communicate keyboard state changes to the SwiftUI Views that are interested in them.

Let's say you want to detect when the up-arrow is pressed. To detect the key, I use an extension on CGKeyCode.

import CoreGraphics

extension CGKeyCode
{
    // Define whatever key codes you want to detect here
    static let kVK_UpArrow: CGKeyCode = 0x7E

    var isPressed: Bool {
        CGEventSource.keyState(.combinedSessionState, key: self)
    }
}

Of course, you have to use the right key codes. I have a gist containing all of the old key codes. Rename them to be more Swifty if you like. The names listed go back to classic MacOS and were defined in Inside Macintosh.

With that extension defined, you can test if a key is pressed anytime you like:

if CGKeyCode.kVK_UpArrow.isPressed { 
    // Do something in response to the key press.
}

Note these are not key-up or key-down events. It's simply a boolean detecting if the key is pressed when you perform the check. To behave more like events, you'll need to do that part yourself by keeping track of key state changes.

There are multiple ways of doing this, and the following code is not meant to imply that this is the "best" way. It is simply a way. In any case, something like the following code would go (or be called from) wherever you do global initialization when you app starts.

// These will handle sending the "event" and will be fleshed 
// out further down
func dispatchKeyDown(_ key: CGKeyCode) {...}
func dispatchKeyUp(_ key: CGKeyCode) {...}

fileprivate var keyStates: [CGKeyCode: Bool] =
[
    .kVK_UpArrow: false,
    // populate with other key codes you're interested in
]

fileprivate let sleepSem = DispatchSemaphore(value: 0)
fileprivate let someConcurrentQueue = DispatchQueue(label: "polling", attributes: .concurrent)

someConcurrentQueue.async 
{
    while true
    {
        for (code, wasPressed) in keyStates
        {
            if code.isPressed 
            {
                if !wasPressed 
                {
                    dispatchKeyDown(code)
                    keyStates[code] = true
                }
            }
            else if wasPressed 
            {
                dispatchKeyUp(code)
                keyStates[code] = false
            }
        }
        
        // Sleep long enough to avoid wasting CPU cycles, but 
        // not so long that you miss key presses.  You may 
        // need to experiment with the .milliseconds value.
        let_ = sleepSem.wait(timeout: .now() + .milliseconds(50))
    }
}

The idea is just to have some code that periodically polls key states, compares them with previous states, dispatches an appropriate "event" when they change, and updates the previous states. The code above does that by running an infinite loop in a concurrent task. It requires creating a DispatchQueue with the .concurrent attribute. You can't use it on DispatchQueue.main because that queue is serial not concurrent, so the infinite loop would block the main thread, and the program would become unresponsive. If you already have a concurrent DispatchQueue you use for other reasons, you can just use that one instead of creating one just for polling.

However, any code that accomplishes the basic goal of periodic polling will do, so if you don't already have a concurrent DispatchQueue and would prefer not to create one just to poll for keyboard states, which would be a reasonable objection, here's an alternate version that uses DispatchQueue.main with a technique called "async chaining" to avoid blocking/sleeping:

fileprivate var keyStates: [CGKeyCode: Bool] =
[
    .kVK_UpArrow: false,
    // populate with other key codes you're interested in
]

fileprivate func pollKeyStates()
{
    for (code, wasPressed) in keyStates
    {
        if code.isPressed 
        {
            if !wasPressed 
            {
                dispatchKeyDown(code)
                keyStates[code] = true
            }
        }
        else if wasPressed 
        {
            dispatchKeyUp(code)
            keyStates[code] = false
        }
    }

    DispatchQueue.main.asyncAfter(deadline: .now() + .milliseconds(50))
    {
        // The infinite loop from previous code is replaced by
        // infinite chaining.
        pollKeyStates()
    }
}

// Start up key state polling
DispatchQueue.main.asyncAfter(deadline: .now() + .milliseconds(50)) {
    pollKeyStates()
}

With code in place to detect when keys are pressed, you now need a way to communicate that to your SwiftUI Views. Again, there's more than one way to skin that cat. Here's an overly simplistic one that will update a View whenever the up-arrow is pressed, but you'll probably want to implement something a bit more sophisticated... probably something that allows views to specify what keys they're interested in responding to.

class UpArrowDetector: ObservableObject
{
    @Published var isPressed: Bool = false
}

let upArrowDetector = UpArrowDetector()

func dispatchKeyDown(_ key: CGKeyCode) 
{
    if key == .kVK_UpArrow {
        upArrowDetector.isPressed = true
    }
}

func dispatchKeyUp(_ key: CGKeyCode) {
    if key == .kVK_UpArrow {
        upArrowDetector.isPressed = false
    }
}

// Now we hook it into SwiftUI
struct UpArrowDetectorView: View
{
    @StateObject var detector: UpArrowDetector

    var body: some View
    {
        Text(
            detector.isPressed 
                ? "Up-Arrow is pressed" 
                : "Up-Arrow is NOT pressed"
        )
    }
}

// Use the .environmentObject() method of `View` to inject the
// `upArrowDetector`
struct ContentView: View
{
    var body: some View
    {
        UpArrowDetectorView()
            .environmentObject(upArrowDetector)
    }
}

I've put a full, compilable, and working example at this gist patterned on code you linked to in comments. It's slightly refactored from the above code, but all the parts are there, including starting up the polling code.

I hope this points you in a useful direction.

Coronado

This is incredibly helpful! Thank you so much for the detailed response! I really hope apple takes more interest in macOS and adds a way to do this in swiftUI, but this is a great solution for now. Thanks! – Herophilus

I'm working on implementing this now and I have one question. I've never worked with concurrency before and I wasn't sure what I'm supposed to do on the line someConcurrentQueue.async {. Can someConcurrentQueue just be whatever I want, like keyboardManagerQueue? Sorry if this is a dumb question. Thanks! – Herophilus

someConcurrentQueue is just a stand-in for a DispatchQueue that was created with the .concurrent attribute (let someConcurrentQueue = DispatchQueue(label: "queueNameGoesHere", attributes: .concurrent)). You could use an existing queue as long as it's not a serial queue, which means you can't use DispatchQueue.main with my code example. – Coronado

BTW - there are ways you could use DispatchQueue.main. Instead of an finite loop in the async closure, you could use "async chaining". To do that put the code from the closure into a named function, and then use DispatchQueue.main.async in the function to call it self instead of an infinite loop. In that case, you don't want to sleep, because that would block the main thread. As I said, there are multiple ways to do it. – Coronado

@CameronDelong, I updated my answer both to include the creation of the concurrent queue, and with an alternate implementation that can be used with DispatchQueue.main. – Coronado

Thanks, that makes sense! I'm also having trouble figuring out where to put some of the code. When you say "where I do global initiation", I don't think I have globals? So I'm not sure where that would be. It that just somewhere at the top level? And does most of the code after that go in the same place? – Herophilus

Also one more question I have is, while I was having trouble figuring out this method, I tried to implement the way with NSViewRepresentable, but I had the problem that I have a NagivationView, and the destination of that navigationView is where I need the keyboard input. So keyboard input would either not work when the NavigationView was focused, or cause the NavigationView to lose focus instantly after selecting an item on it, leading to it just flashing blue before turning to a darker gray. Do you know if this approach with CGEventSource would have the same problem? – Herophilus

Regarding where your initialization code should go... Wherever your ContentView is created. That used to be in AppDelegate.didFinishLaunching() but a lot of projects use WindowGroup. If you don't have an obvious place to put it, you could put it in a static function and use static properties instead of globals for keyStates and upArrowDetector (or you version of it). Make make sure the code that polls for the key states is started before ContentView.body is called. You will still need to inject the upArrowDetector (or your equivalent) as instance properties though. – Coronado

As for the other question, the reason you had problems with the NSViewRepresentable solution has to do with the way NavigationView steals/sets the firstResponder behind the scenes. The good and bad thing about using CGEventSource to detect keys is that it's global... meaning it doesn't care about firstResponder, but that means if you use it in multiple Views, you will need to add some code to manage how the "events" are dispatched that makes sense for your app. – Coronado

BTW - I mentioned there are other ways to deal handle it keyboard events. I have a SwiftPackage called MacMenuBar that handles the menu bar in a SwiftUI way (another pain point for SwiftUI on macOS), and I provide a Sudoku app to demonstrate how to use it. I wanted the Sudoku puzzle to respond to key events so I took another approach that intercepts events from the AppKit event loop before SwiftUI gets its grubby paws on them. – Coronado

github.com/chipjarred/MacMenuBar . NSGameWindow.swift, NSGameHostingView.swift in Cocoa-BasedTypes folder, and in Views/KeyboardHacks there is KeyResponder.swift and KeyResponderGroup.swift. It's all very special case for that particular app, but it may give you ideas. The purpose was to demonstrate MacMenuBar. Keyboard handling was just an incidental thing. I'm not sure I'd do it that way again. – Coronado

I tried to take the approach of using static vars, and I can't quite get it to work. I've created a file called KeyboardManager, with a class KeyboardManager to store the static values, and in my case a SpaceDetector class for now (as well as the CGKeyCode extension): pastebin.com/WCgHwyRJ. I then added the StateObject to the View I needed the input in, and the text to test. Xcode throws an error when I call that view, Missing argument for parameter 'detector' in call. I tried passing KeyboardManager.spaceDetector, which got rid of the error. – Herophilus

It still didn't work which I assume was because I was never starting the loop to get the key states? But I'm not sure where to run that code. I tried KeyboardManager.pollKeyStates() in Window Group, but WindowGroup throws Type '()' cannot conform to 'View'. – Herophilus

Where is your main view created? ie, ContentView unless you changed it. I don't mean where it is defined. I mean there is some place where an instance of it is created. Do you have it in a repo on Github I can take a look? – Coronado

OK, so I just looked over the code from your pastebin link. I think the easiest way would be to put a static var isInitialized = false in your KeyboardManager. Rename your spaceDetector property and make it private. Add a public computed property that calls pollKeyStates() if isInitialized is false, sets it to true and returns the private spaceDetector – Coronado

Also if you want to start it up in Window Group, you probably have to explicitly return the view. Adding non-View code messes up View's result builder. You can run into similar issues putting non-View code in View.body. – Coronado

I'll work on adding that when I have a chance. However I'm still having trouble running the code in WindowGroup. I just tried using print() and returning ContentView but I still get an error, now its Cannot use explicit 'return' statement in the body of result builder 'ViewBuilder'. Here's code: pastebin.com/UGgxt1Nc. Also tysm for all the help so far, I would've been stuck ages ago without it – Herophilus

I made this gist for you. I roughly patterned it after the code from your pastebin link so hopefully it's more obvious how to initialize it. – Coronado

Just got it working, thank you so much! Sorry if I was a bit incompetent, I’m fairly new to swift so I wasn’t sure exactly what I was doing. This worked perfectly for what I needed though, and now I can finally get on to other parts of my app! Thanks!! – Herophilus

Awesome! I'm glad I was able to help – Coronado

I can't add an answer (closed), but I was able to get an onKeyDown event firing by subclassing NSTextView with override func keyDown(...) method override. For some reason, the keyDown handler doesn't fire with NSTextField but it does with NSTextView. It seems to be the least hacky solution that I've come across (e.g. doesn't require polling). – Continence

@Continence Thanks for the solution, however I covered using Cocoa (AppKit) in the fourth paragraph of my answer. The problem is the question is about capturing key presses in SwiftUI apps, not Cocoa apps, which means all such views have to be wrapped by NSViewRepresentable which might be objectionable when there many different View types that need the functionality, and though I didn’t originally mention it, it also doesn’t address if it’s needed globally, not just in certain views. – Coronado

@Continence another approach is intercept CGEvent by subclassing (or swizzling) NSHostingView, but that means having to intercept/forward to/restore firstResponder. That’s too much messing with Cocoa innards to be a clean solution. – Coronado

@Dan I'm glad you found it useful. Apple is improving SwiftUI support for macOS, but they still have a long way to go to achieve parity with Cocoa apps, and maybe they never will. I kind of expect that we'll probably always have to reach down into lower levels for some behavior or other. – Coronado

Thank you so much. It took forever to find an answer that didn't require some super hacky way to do things. :) – Tavish

Answer 2

in the meanwhile (Xcode 15, iOS 17, macOS 14) we have some support for detecting key press events in SwiftUI, that we can use, when developing for macOS. There is a new view modifier .onKeyPress, that helps in such cases.

Here is an example code, how to implement:

TextField("new name of the player", text: $newName)
   .focused($isTextFieldFocussed)
   .autocorrectionDisabled()
   .onSubmit { confirmRenaming() }   // <RETURN> key pressed
   .onKeyPress(.escape) {            // <ESCAPE> key tracking
       cancelRenaming()
       return .handled 
   }

The return options are .handeled OR .ignored and tells the system, that the key input is already fully handled or not, which means the system will still take care for the key input (and do whatever with it).

Try to clean and rebuild because the canvas view sometimes does not really work with the key events.

Atalayah

.onKeyPress(.escape) on a TextField doesn't work for me on macOS 14.5 (Sonoma). It never enters the block. – Almatadema

Answer 3

I'm building a complex macOS(13|14) app and the 'default' .keyboardShortcut() was not triggering my button action as explained in the headers.

I wanted to display a custom view in a .sheet modifier and there I would dismiss the sheet by hitting the .esc key or hitting .return key.

.keyboardShortcut() was not working consistently and often would beep. The beep in Cocoa is indicative of system events not properly handled through the responder chain. Well yes there is still NSResponder under the hood on SwiftUI.

The following code allows you to get key downs into your view. Since this solved my issue I wanted to post it here as another simpler mechanism for handling any key down in any view. This is obviously macOS code and relatively brief.

At the heart I'm using 2 old Cocoa APIs NSEvent.addLocalMonitorForEvents and NSEvent.removeMonitor

MyView, will now be able to distinguish between a .mouseClick or .return and call whatever logic.

struct Demo {
    // Convenience
    // https://gist.github.com/rdev/627a254417687a90c493528639465943
    //
    extension UInt16 {
        // Layout-independent Keys
        // eg.These key codes are always the same key on all layouts.
        static let returnKey: UInt16 = 0x24
        static let enter: UInt16 = 0x4C
        static let space: UInt16 = 0x31
    }

    struct MyView: View {
        @State var monitorID: Any?

        func handleAction(_ type: String) {
            // do your work here
        }

        var body: some View {
            VStack {
                Button(action: {
                    handleAction(".mouseClick")
                }) {
                    Text("Hit return")
                }
            }
            .onAppear(perform: {
                self.monitorID = NSEvent.addLocalMonitorForEvents(matching: .keyDown) { event in
                    if event.keyCode == .returnKey || event.keyCode == .enter {
                        handleAction(".return")
                        return nil
                    } else if event.keyCode == .space {
                        handleAction(".space")
                        return nil
                    }
                    // allow others to handle this event
                    return event
                }
            })
            .onDisappear(perform: {
                if let id = self.monitorID {
                    // clean up or else we are called with older captured values
                    NSEvent.removeMonitor(id)
                }
            })
        }
    }
}

Excusatory