security / codesign in Sierra: Keychain ignores access control settings and UI-prompts for permission

Question

Starting with macOS Sierra, I can't import a codesign-identity into a keychain with /usr/bin/security any more without usr/bin/codesign UI-prompting for access when using this identity. This breaks the packaging scripts of build server. There seems to be no workaround. This affects custom created keychains, but also the login.keychain.

Steps to Reproduce: Execute the following commands in Terminal (requires a signing identity to be available to import):

security create-keychain -p test buildagent.keychain
security unlock-keychain -p test buildagent.keychain

security list-keychains -d user -s buildagent.keychain
security default-keychain -s buildagent.keychain

security import identity.p12 -k buildagent.keychain -P password -T /usr/bin/codesign

codesign -vfs '$IDENTITY' '${PRODUCT}' --keychain 'buildagent.keychain'

Result: macOS shows a UI-prompt asking for permission to access the previously imported private key.

I have tried many workarounds, but nothing seems to work:

• Using the new .keychain-db extension when specifying the keychain-name
• Using the login.keychain instead of the custom one
• Importing the p12 with -A ('Allow any application to access the imported key')
• Importing the Cert und Key separately (being extracted from the p12 before with openssl pkcs12)

Importing the identity definitely works, I can see the cert and key when displaying the contents of the keychain in the Keychain Access application. The access control setting for the private key is also correctly configured (with the desired codesign exception rule).

How can I avoid the UI prompt from Sierra?

Newsmagazine

Currently I found I had to bring out the UI once and click the Always allow then the CLI works well. – Rem

Hi jayatubi, that's a viable solution if you work with only a few Codesign Identities and the same keychain (e.g. login.keychain). But my company has to manage dozens of different Codesign Identities and prefers to use per-build-keychains, as this eliminates the risk of breaking the build with multiple similarly-named identities being available in the same keychain. – Newsmagazine

I've noticed one strange thing. When the UI prompt it ask for the permission of private XXX. However, the XXX is not the name of the private key but the name of the p12 file. If I have more than one keys in the same p12 file no matter which key I use it always show the name of the p12 file. And if I click always allow all the keys with in the same p12 will be fine to codesign. – Rem

In my case issue was that I was doing it via SSH – Bozcaada

Answer 1

The command you need to use is as follows:

security set-key-partition-list -S apple-tool:,apple: -s -k keychainPass keychainName

Please have in mind that this command line tool works like the list-keychains's way of modification. If you execute set-key-partition-list with a single value it will overwrite all partitionIDs in the certificates. It won't validate the values passed.

What this command does is that it sets the PartitionIDs (items after -S separated by comma) for keys that can sign (-s) for a specific keychain. The actual partitionID that allows the codesigning is apple:.

I am not aware what apple-tool: is doing as it is not documented, but it was there after importing the key with security import so I'm keeping it in order to avoid breaking people who copy-paste the command.

This change was introduced with Mac OS Sierra and is not documented (or at least I could not find documentation). As of Oct 16 the man page for security still doesn't list this command.

For more information you can refer to this bug report - http://www.openradar.me/28524119

Bemused

Thank you very much, this works perfectly fine! How did you come up with this? When googling for set-key-partition-list, the only result is the this site and the referenced radar :) – Newsmagazine

I reverse engineered the security command line tool, keychain UI tool and the securityd. Additionally I used the acltool from the Apple Mac OS testing team in order to print the current partitionIDs set by their tool. AclTool's source code can be found here - opensource.apple.com/source/Security/Security-57337.60.2/… – Bemused

Wow what an effort ;) I just found out that set-key-partition-list is an unknown command for security on El Capitan, so it must have been added with Sierra. – Newsmagazine

Is there a get-partition-list to see what things were currently set to beforehand? – Botello

This didn't work for me. It seems to just dump my keychain. When I tried to test it with codesign --force --sign <identity_hash> $(mktemp), I still get the approval dialog – Whitish

Does this only work on temporary keychains? I'm trying to do it with my login keychain. – Whitish

I didn't include my keychain password when I initially imported the private key, and that caused set-key-partition-list not to work. https://mcmap.net/q/246632/-after-set-key-partition-list-codesign-still-prompts-for-key-access – Whitish

If you created a self signed certificate for test signing and it is not found after running this command, you need to update the Trust settings in the keychain, like this: cloud.githubusercontent.com/assets/350686/15832978/… – Giovannagiovanni

What is "keychainName"? I'm not being able to identify which kind of information should I use to replace "keychainName". – Tourmaline

Any idea if this approach still works in High Sierra? Seems like a new tactic is require for every update... – Strobila

@ElliotNelson working the same way in High Sierra. Finished a new setup just today. – Cichocki

@ElliotNelson working the same way in Mojave. Finished a new setup just today. – Sardis

This worked perfectly in CircleCI workflow which was hanging at the code-sign step, despite all keychain and certificates confirmed to be installed and correct. Thank you @IlianIliev what an amazing effort - reverse engineering the security tool! – Elbring

This just spontaneously (?) stopped working on one of our build servers, saying "SecKeychainItemCopyAccess: The specified item is no longer valid. It may have been deleted from the keychain". Obviously, I'm not sure which thing is is saying is not valid. – Intrusive

This solution solved the archive issue when building on Jenkins. However I'd need to run this as part of the script each time, and exposing the password in the script seems unsecure.. – Englacial

@Englacial yes it seems insecure, there is no secure design from Apple for this. – Intrusive

@O'Rooney Actually I was able to mask the password using 'Inject password to the build as environment variables'. Cheers! – Englacial

Answer 2

The command from this answer only unlocked the keychain for me, but I still had the UI-prompt asking whether the current application could use the key.

I prevented the prompt like this:

Go to the keychain in Keychain Access, double click on all the keys there, and in the tab Access Control, check 'Allow all applications to access this item'.

I was able to upload the new keychain file then to my Jenkins build server, where it is unlocked by the Keychains and Provisioning Profiles Plugin. The build now succeeds signing.

Try

This answer deserves more upvotes. All of the other solutions still won't work unless the access control is set to Allow all applications. – Sartorius

Answer 3

For those who are having this issue with Travis or other CI, you have to add codesign in the application id list.

security set-key-partition-list -S apple-tool:,apple:,codesign: -s -k keychainPass keychainName

P.S: I'm using keychainName.keychain (adding .keychain)

Stabilize

This didn't work for me. It seems to just dump my keychain. When I tried to test it with codesign --force --sign <identity_hash> $(mktemp), I still get the approval dialog – Whitish

Does this only work with temporary keychains? I am trying to do it with my login keychain. – Whitish

I did not include my keychain password when I initially imported the private key, and that caused set-key-partition-list not to work. https://mcmap.net/q/246632/-after-set-key-partition-list-codesign-still-prompts-for-key-access – Whitish

This did not work for me with CircleCI. The code-sign command hangs (as the os is asking for a password). What worked was Ilian Iliev's answer above. – Elbring

Answer 4

For some reason the security set-key-partition-list did not work for me.

I solved it by using the -A option when importing the certificate in the keychain:

security import ${P12_FILE} -k ${KEYCHAIN_PATH} -P ${P12_PASSWORD} -A

There is no need to use the security set-key-partition-list afterwards.

This option allows any application to access the imported key without warning. Hence, it prevents the prompt from showing up. Note that it is insecure as the key is not protected but depending on your build context it might help.

On top of that the keychain must be added to the search list:

security list-keychains -s ${KEYCHAIN_PATH}

Then the keychain should be unlocked. Otherwise a prompt asking for the keychain password will be displayed:

security unlock-keychain -p ${KEYCHAIN_PASSWORD} ${KEYCHAIN_PATH}

Eventually the auto-lock timeout should be disabled. This is in case the build is quite long and the keychain re-locks itself:

security set-keychain-settings ${KEYCHAIN_PATH}

Enrichetta

Interesting. This didn't work for me back then in Oct '16 with Sierra. On which OS did you test this? – Newsmagazine

It was tested on a Jenkins slave running Sierra 10.12.6 – Enrichetta

Answer 5

I spent a couple of days looking for a solution. This didn't help

security import ${P12_FILE} -k ${KEYCHAIN_PATH} -P ${P12_PASSWORD} -A

but when I listed the apps explicitly - it worked (on Catalina at least)!

security import ${P12_FILE} -k ${KEYCHAIN_PATH} -P ${P12_PASSWORD} -T /usr/bin/codesign -T /usr/bin/productsign

Subteen

Answer 6

Next to using

security set-key-partition-list -S apple-tool:,apple: -s -k keychainPass keychainName

I also had to change the settings for my keychain to "no timeout" used by

security set-keychain-settings keychainName

(Documentation available at https://ss64.com/osx/security-keychain-settings.html)

Gusher

Answer 7

None of the above worked, but I suspect it does work on an empty keychain. The problem is that if the private key already exists in the keychain, then the access control modifiers of the pkcs12 import statement -A and -T have no effect. The access list of the existing key is not overridden.

When importing a new certificate into a keychain for a CI server, we use the following script to ensure the import is successful and codesign is on the access control list:

#!/bin/zsh
KC_FILE=keychain-path
KC_PASSWORD=keychain-password
P12_STORE=pkcs12-path
P12_PASSWORD=pkcs12-password

security unlock-keychain -p $KC_PASSWORD $KC_FILE

CERT_SHA1=`openssl pkcs12 -in $P12_STORE -nodes -passin pass:"$P12_PASSWORD" 2> /dev/null |openssl x509 -noout -fingerprint | grep Fingerprint | sed '/Fingerprint/s/^.*=//;s/://g'`

security delete-identity -Z $CERT_SHA1 $KC_FILE

security import $P12_STORE -t cert -f pkcs12 -P "$P12_PASSWORD" -T /usr/bin/codesign -k $KC_FILE

security set-key-partition-list -S apple-tool:,apple:,codesign: -s -k $KC_PASSWORD $KC_FILE

Thereafter

Answer 8

After trying many different solutions, what worked for me was simply changing the password of my keychain.

• Finder > Go > Utilities
• Open the Keychain Access utility.
• Not sure if I needed to do this step: In the left sidebar of the Keychain Access utility, click on My Certificates. Look at the Keychain column to confirm which Keychain your apple developer certificate is in. In my case it was in the "login" keychain.
• Change the password for the keychain from the previous step. You might want to trying locking it then unlocking it, if it's locked. You change the password by clicking on the relevant keychain ("login", in my case) and then selecting "Change Password..." from the Edit menu of the Keychain Access utility.
• The next time I ran the archive step in Xcode (in the Product menu) I was eventually prompted for a keychain password and I entered the password for my "login" keychain. Then it worked. When it finished I saw an Archives screen with my app listed in it.

Dripps