How can I get SecKeyRef from DER/PEM file

Question

I need to integrate my iPhone app with a system, and they require to encrypt data by a given public key, there are 3 files in 3 different format .xml .der and .pem, I have researched and found some articles about getting SecKeyRef from DER/PEM, but they are always return nil. Below is my code:

NSString *pkFilePath = [[NSBundle mainBundle] pathForResource:@"PKFile" ofType:@"der"];
NSData *pkData = [NSData dataWithContentsOfFile:pkFilePath]; 

SecCertificateRef   cert; 
cert = SecCertificateCreateWithData(NULL, (CFDataRef) pkData);
assert(cert != NULL);

OSStatus err;

    if (cert != NULL) {
        err = SecItemAdd(
                         (CFDictionaryRef) [NSDictionary dictionaryWithObjectsAndKeys:
                                            (id) kSecClassCertificate,  kSecClass, 
                                            (id) cert,                  kSecValueRef,
                                            nil
                                            ], 
                         NULL
                         );
        if ( (err == errSecSuccess) || (err == errSecDuplicateItem) ) {
            CFArrayRef certs = CFArrayCreate(kCFAllocatorDefault, (const void **) &cert, 1, NULL); 
            SecPolicyRef policy = SecPolicyCreateBasicX509();
            SecTrustRef trust;
            SecTrustCreateWithCertificates(certs, policy, &trust);
            SecTrustResultType trustResult;
            SecTrustEvaluate(trust, &trustResult);
            if (certs) {
                CFRelease(certs);
            }
            if (trust) {
                CFRelease(trust);
            }
            return SecTrustCopyPublicKey(trust);
        }
    }
return NULL;

Problem happens at SecCertificateCreateWithData, it always return nil even through read file is ok. Anybody has done this please help me, thanks!

EDIT: The cert file was MD5 signature.

Sind

I think that you'll find your answer here: #1595513 – Extended

Answer 1

I struggled a lot with the same problem and finally found a solution. My problem was that I needed to use both an external private and public key for encrypting/decrypting data in an iOS app and didn't want to use the keychain. It turns out you also need a signed certificate for the iOS security library to be able to read the key data and of course the files have to be in the correct format. The procedure is basically as follows:

Say you have a private key in PEM format (with the -----BEGIN RSA PRIVATE KEY----- and -----END RSA PRIVATE KEY----- markers): rsaPrivate.pem

//Create a certificate signing request with the private key
openssl req -new -key rsaPrivate.pem -out rsaCertReq.csr

//Create a self-signed certificate with the private key and signing request
openssl x509 -req -days 3650 -in rsaCertReq.csr -signkey rsaPrivate.pem -out rsaCert.crt

//Convert the certificate to DER format: the certificate contains the public key
openssl x509 -outform der -in rsaCert.crt -out rsaCert.der

//Export the private key and certificate to p12 file
openssl pkcs12 -export -out rsaPrivate.p12 -inkey rsaPrivate.pem -in rsaCert.crt

Now you have two files which are compatible with the iOS security framework: rsaCert.der (public key) and rsaPrivate.p12 (private key). The code below reads in the public key assuming the file is added to your bundle:

- (SecKeyRef)getPublicKeyRef {

    NSString *resourcePath = [[NSBundle mainBundle] pathForResource:@"rsaCert" ofType:@"der"];
    NSData *certData = [NSData dataWithContentsOfFile:resourcePath];
    SecCertificateRef cert = SecCertificateCreateWithData(NULL, (CFDataRef)certData);
    SecKeyRef key = NULL;
    SecTrustRef trust = NULL;
    SecPolicyRef policy = NULL;

    if (cert != NULL) {
        policy = SecPolicyCreateBasicX509();
        if (policy) {
            if (SecTrustCreateWithCertificates((CFTypeRef)cert, policy, &trust) == noErr) {
                SecTrustResultType result;
                OSStatus res = SecTrustEvaluate(trust, &result);

                //Check the result of the trust evaluation rather than the result of the API invocation.
                if (result == kSecTrustResultProceed || result == kSecTrustResultUnspecified) {
                    key = SecTrustCopyPublicKey(trust);
                }
            }
        }
    }
    if (policy) CFRelease(policy);
    if (trust) CFRelease(trust);
    if (cert) CFRelease(cert);
    return key;
}

To read in the private key use the following code:

SecKeyRef getPrivateKeyRef() {
    NSString *resourcePath = [[NSBundle mainBundle] pathForResource:@"rsaPrivate" ofType:@"p12"];
    NSData *p12Data = [NSData dataWithContentsOfFile:resourcePath];

    NSMutableDictionary * options = [[NSMutableDictionary alloc] init];

    SecKeyRef privateKeyRef = NULL;

    //change to the actual password you used here
    [options setObject:@"password_for_the_key" forKey:(id)kSecImportExportPassphrase];

    CFArrayRef items = CFArrayCreate(NULL, 0, 0, NULL);

    OSStatus securityError = SecPKCS12Import((CFDataRef) p12Data,
                                             (CFDictionaryRef)options, &items);

    if (securityError == noErr && CFArrayGetCount(items) > 0) {
        CFDictionaryRef identityDict = CFArrayGetValueAtIndex(items, 0);
        SecIdentityRef identityApp =
        (SecIdentityRef)CFDictionaryGetValue(identityDict,
                                             kSecImportItemIdentity);

        securityError = SecIdentityCopyPrivateKey(identityApp, &privateKeyRef);
        if (securityError != noErr) {
            privateKeyRef = NULL;
        }
    }
    [options release];
    CFRelease(items);
    return privateKeyRef;
}

Holoenzyme

The first solution I found that manages to do it without using the keychain. Excellent. – Duleba

This is incorrect now. iOS 9.3 now supports bringing in PEM keys directly. Although it may require the use of the keychain you can immediately delete it out of the keychain. – Neuman

@Neuman - How does one do this since 9.3? – Ridiculous

I couldn't get the private key export to work. SecPKCS12Import worked fine with no errors, but SecIdentityCopyPrivateKey would crash (in CFDataGetLength). After many hours debugging it, on a whim I regenerated the .p12 file with the exact same command line (bash history) and then everything worked perfectly. Untouched priv key & cert files. Really wish I'd kept the old .p12 file to prove my sanity. – Actinochemistry

Answer 2

Starting with iOS 10, it is actually possible to import PEM private keys w/o converting them to PKCS#12 (which is a very universal container format for everything related to cryptography) and thus also w/o using OpenSSL on command line or statically linking apps with it. On macOS it's even possible since 10.7 using a different function than the ones mentioned here (but so far it doesn't exist for iOS). Exactly the way described below will also work on macOS 10.12 and later, though.

To import a certificate, it's enough to just strip the

-----BEGIN CERTIFICATE-----

and

-----END CERTIFICATE-----

lines, then run base64 decoding over the data left, the result is a certificate in standard DER format, which can just be fed to SecCertificateCreateWithData() to get a SecCertificateRef. This has always been working, also prior to iOS 10.

To import a private key, a little bit of extra work may be required. If the private key is wrapped with

-----BEGIN RSA PRIVATE KEY-----

then it is very easy. Again, the first and last line needs to be stripped, the remaining data needs to be base64 decoded and the result is a RSA key in PKCS#1 format. This format can only hold RSA keys and it is directly readable, just feed the decoded data into SecKeyCreateWithData() to obtain a SecKeyRef. The attributes dictionary just need the following key/value pairs:

• kSecAttrKeyType: kSecAttrKeyTypeRSA
• kSecAttrKeyClass: kSecAttrKeyClassPrivate
• kSecAttrKeySizeInBits: CFNumberRef with then number of bits in the key (e.g. 1024, 2048, etc.) If not known, this information can actually be read from the raw key data, which is ASN.1 data (it's a bit beyond the scope of this answer, but I will provide some helpful links below about how to parse that format). This value is maybe optional! In my tests it was actually not necessary to set this value; if absent, the API determined the value on its own and it was always set correctly later on.

In case the private key is wrapped by -----BEGIN PRIVATE KEY-----, then the base64 encoded data is not in PKCS#1 format but in PKCS#8 format, however, this is a just a more generic container that can also hold non-RSA keys but for RSA keys the inner data of that container is equal to PKCS#1, so one could say for RSA keys PKCS#8 is PKCS#1 with an extra header and all you need to do is stripping that extra header. Just strip the first 26 bytes of the base64 decoded data and you have PKCS#1 again. Yes, it's really that simple.

To learn more about PKCS#x formats in PEM encodings, have a look at this site. To learn more about ASN.1 format, here's a good site for that. And if you need a simple, yet powerful and interactive online ASN.1 parser to play around with different formats, one that can directly read PEM data, as well as ASN.1 in base64 and hexdump, try this site.

Very important: When adding a private key to keychain, that you created as above, please be aware that such a private key doesn't contain a public key hash, yet a public key hash is important for they keychain API to form an identity (SecIdentityRef), as using the public key hash is how the API finds the correct private key that belongs to an imported certificate (a SecIdentityRef is just a SecKeyRef of a private key and a SecCertificateRef of a cert forming a combined object and it's the public key hash, that binds them together). So when you plan to add the private key to keychain, be sure to set a public key hash manually, otherwise you won't ever be able to get an identity for it and without that you cannot use keychain API for tasks like signing or decrypting data. The public key hash must be stored in an attribute named kSecAttrApplicationLabel (stupid name, I know, but it's really not a label and nothing the user can ever see, check out the documentation). E.g.:

OSStatus error = SecItemAdd(
    (__bridge CFDictionaryRef)@{
        (__bridge NSString *)kSecClass: 
            (__bridge NSString *)kSecClassKey,
        (__bridge NSString *)kSecAttrApplicationLabel: 
             hashOfPublicKey, // hashOfPublicKey is NSData *
#if TARGET_OS_IPHONE
        (__bridge NSString *)kSecValueRef: 
            (__bridge id)privateKeyToAdd, // privateKeyToAdd is SecKeyRef
#else
        (__bridge NSString *)kSecUseItemList: 
              @[(__bridge id)privateKeyToAdd], // privateKeyToAdd is SecKeyRef
              // @[ ... ] wraps it into a NSArray object,
              // as kSecUseItemList expects an array of items
#endif
     },
     &outReference // Can also be NULL,
                   // otherwise reference to added keychain entry
                   // that must be released with CFRelease()
);

Ferruginous

In addition to stripping -----BEGIN and -----END lines newline characters should also be stripped out before base64 decoding. – Millda

@Millda Every standard conform base64 decoder must be able to ignore newlines as that's required by the standard, otherwise it couldn't even decode it's own encoding output that very often contains newlines. When using initWithBase64EncodedString:options: just use the option NSDataBase64DecodingIgnoreUnknownCharacters. Most 3rd party base64 decoders ignore newlines by default and that can only be turned of by an option. – Ferruginous

you say its enough to just strip out BEGIN CERTIFICATE and END... But how do I get that cert to do that? – Chrischrism

@Chrischrism You load the cert as a string and you remove these two substrings. What is left over is the base64 encoded cert data. PEM is just human reable text and a PEM file can be loaded as a string. – Ferruginous

ok so I guess my next question is how do I get/create the cert...I was confused by the beinging "To import To import a certificate" – Chrischrism

@Chrischrism The question was how to get a SecKeyRef from a PEM file. The answer is: You load the PEM file to a string, you strip the begin/end markers from the string, you decode the rest using a Base64 decoder and you feed the data into SecKeyCreateWithData() together withe appropriate attributes and then you have SekKeyRef of the key that was stored in the PEM file. – Ferruginous

what to set for the hashOfPublicKey? It says NSData * there, but I don't get it... – Watercraft

@Watercraft The hash of the public key is the SHA1 hash of the public key data. If you have the public key as PEM, you most likely also have the private one as PEM. Decode the public key PEM (also base64) and feed that data into SHA1, the result are 20 byte of data. And yes, it's still SHA1 as even though SHA1 has been broken and shouldn't be used for cryptography anymore, it isn't used for cryptography in this case, it is just used as an identifier value. – Ferruginous

Answer 3

After hours of effort researching online with the help of this post, I finally get it working perfectly. Here is the notes with working Swift code of the most current version. I hope it can help someone!

1. Received a certificate in the base64 encoded string sandwiched between header and tail like this (PEM format):

   -----BEGIN CERTIFICATE-----
   -----END CERTIFICATE-----
   

2. strip out the header and the tail, such as

   // remove the header string  
   let offset = ("-----BEGIN CERTIFICATE-----").characters.count  
   let index = certStr.index(cerStr.startIndex, offsetBy: offset+1)  
   cerStr = cerStr.substring(from: index)  
   
   // remove the tail string 
   let tailWord = "-----END CERTIFICATE-----"   
   if let lowerBound = cerStr.range(of: tailWord)?.lowerBound {  
   cerStr = cerStr.substring(to: lowerBound)  
   }
   

3. decode base64 string to NSData:

   let data = NSData(base64Encoded: cerStr, 
      options:NSData.Base64DecodingOptions.ignoreUnknownCharacters)!  
   

4. Convert it from NSdata format to SecCertificate:

   let cert = SecCertificateCreateWithData(kCFAllocatorDefault, data)
   

5. Now, this cert can be used to compare with the certificate received from the urlSession trust:

   certificateFromUrl = SecTrustGetCertificateAtIndex(...)
   if cert == certificate {
   }
   

Kahn

What if the PEM is a chain of multiple keys, i.e. multiple "BEGIN / END"s? – Forename

NSData.Base64DecodingOptions.ignoreUnknownCharacters seems to have fixed my conversion issues. – Gmt

how do you receive that certificate? – Chrischrism

nice one. also, remember that you may have base64 in URI format where you need to replace '-' to '+' and '_' to '/'. on iOS you also need to add padding with '='. String have to be divisible by 4 without fraction. Otherwise you end up with wrong Data object. – Mcclinton