Java applet with self-signed certificate on OS X Mountain Lion

Question

We have a Java applet that needs to run with full trust.

While developing and during pre-release tests we sign it using a self-signed certificate (the production version is signed with a real code signing certificte).

But when we try to start the self-signed applet on the prerelases of OS X 10.8, we can no longer choose to allow it to run. The "Allow"-button is simply disabled:

If I press "Show Details..." I can choose to "Always trust" the certificate, but this makes no difference:

It works with the same version of the Java JRE on OS X Lion 10.7, so I suspect it is an issue with the OS and not the JRE.

Are there any workarounds?

I would prefer not to use a real code signing certificate for testing: signing with a real code signing certificate means that my company asserts that the applet is secure and should be trusted. We can hardly assert that before we have tested it.

Dermatologist

"I would prefer not to use a real code signing certificate for testing." I would prefer if people offered reasons for bizarre & seemingly arbitrary requirements. – Oxa

@Andrew: I have added the reasoning for this requirement. – Dermatologist

Thanks, that reason is not as bizarre as I'd imagined. Good question. +1 BTW: 1) Have you checked the 'bug database' for that JRE? 2) Does the applet interact with JavaScript? (Appleteer can run applets in simple HTML with no JS - it has no security sand-box). 3) Can you run another JRE on the OS for testing applets? – Oxa

This has some suspicious similarities (OS X, signed Jars, security failure) and some differences (self signed vs. purely OS version based & different versions of OS X) to java and signed jars on Mac – Oxa

I am not sure if this works the way I think it does, but.. Try to create a second certificate based on your verified certificate. Change all the details to TESTING ONLY & DO NOT TRUST & other such shouting and waving about of hands. My understanding is that 2nd certificate should itself behave exactly like the fully verified certificate. One hitch is that the 'tick' to always trust would be selected by default. Could that approach work for your use-case? I am just musing that Apple made a security update that forbids all untrusted/untrustworthy Java code.(?) – Oxa

@Andrew Thompson: Yes, it would surely work to buy a new code-signing-certificate with "TESTING ONLY" in the common name. Java shouts a bit more (on the platforms where it works), but it would be workable. I just find it strange that there is no workaround - a code-signing certificate costs around 100$ per year, so this is a hurdle for amateurs developing on OS X. – Dermatologist

"buy a new code-signing-certificate" That is not what I meant. I meant that (my understanding is) you can generate new valid certificate(s) using the SDK, based on your paid one. Please try it and get back to me. I do not feel that 'buying a new certificate' either based on your current one (in the key chain) or entirely new one is a real answer to your question. – Oxa

Answer 1

It is new security feature in Mac OS X, by default only apps from Mac Store & from trusted developers are allowed to run there. Fortunatelly, it is easy to change, you have to allow this in Mac OS X preferences.

Go to Preferences -> Security & Privacy and click on padlock to allow changes.

Then in "Allow appications downloaded from" select "Anywhere".

After that, the button in Java dialog will be enabled.

Deposal

When I enable the "AnyWhere" option, the behaviour changes, but I still get an error dialog stating: "Application Blocked by Security Settings" and "Your Security settings have blocked a self-signed application from running". The app is one of the JavaFX sample apps. On Mac OS X 10.9 Mavericks, Java 8. – Wreckage

what about in the mac? – Bergwall

Answer 2

If you get the "Application Blocked by Security Settings" message, you need to go into System Preferences: Java: Security and either add your site to the exception list or reduce the security level to Medium.

Cardona

Answer 3

OS X Lion you can manually add the certificate as a trusted root certificate using the built-in Keychain Access tool. I don't have access to OS X Mountain Lion yet so I don't know if it will work in Mountain Lion, but it seems worth a try. The steps in Lion are:

• Open Keychain Access (located in /Applications/Utilities)
• Click File | Import Items...
• Change the Destination Keychain to System
• Find your certificate file and click Open
• It will say "Do you want your computer to trust certificates sign by [...] from now on?". Click Always Trust.

If you generated your certificate directly in a Java keystore then you might not have a standalone certificate file. You can easily export one using this guide from Oracle.

I note that your screenshot says "This certificate is marked as trusted for this account", which is curious because I'd expect that to be acceptable. Maybe in OS X Mountain Lion user-added root certificates somehow have a lower status than official ones distributed by Apple? If you find that the above steps don't work, you could try adding your certificate to the official root keychain. Keychain Access won't let you do that, but I believe you can use the builtin certtool to manually edit the keychain located at /System/Library/Keychains/SystemRootCertificates.keychain to achieve that.

Newly

Wow... I really hope he isn't deploying this to 1000 beta testers. – Squatness

I have this problem with one of the JavaFX sample apps. On Mac OS X 10.9 Mavericks, Java 8. I can't seem to find the key it is using to "self-sign" the app, or where the keystore might be to export the key and import to keychain. Any ideas? Meanwhile I'll browse through all the ant build scripts..... – Wreckage