I'm trying to develop a license verification solution. Licenses are encoded on server using OpenSSL's RSA_private_encrypt
function.
For Mac OX X I use RSA_public_decrypt
and it works like a charm. On Windows I must use very tiny bit of code, so I can not link with OpenSSL or other lib AND I have to use MS Crypto API.
I have spent several days trying to figure out what is wrong, but with no luck. I can successfully import public key, but here my success ends. I'm aware that I need to reverse byte order with CAPI so this might not be the issue.
I have tried everything, including CryptVerifyMessageSignatureWithKey
and CryptDecodeObject
to load the blob with different params, but still no luck.
It always ends up with GetLastError() == CRYPT_E_ASN1_BADTAG
, which I assume means that the BLOB is not ASN1 formatted... Google does not tell anything on the output format of RSA_private_encrypt... so I am completely lost here.
Here is the OS X code based on OpenSSL:
void cr_license_init(const char* lic) {
__cr_license_ = lic;
unsigned char lic_encoded[CR_LIC_LEN];
BIO* b64 = BIO_new(BIO_f_base64());
BIO* licIn = BIO_new_mem_buf((void*)lic, -1);
licIn = BIO_push(b64, licIn);
if(BIO_read(licIn, lic_encoded, CR_LIC_LEN) == CR_LIC_LEN) {
const unsigned char* key_data = license_pub_der;
RSA* r = d2i_RSA_PUBKEY(NULL, &key_data, sizeof(license_pub_der));
if(r != NULL) {
if(__cr_license_data_ != NULL) {
free((void*)__cr_license_data_);
}
__cr_license_data_ = malloc(CR_LIC_LEN);
if(RSA_public_decrypt(CR_LIC_LEN, lic_encoded,
(unsigned char*)__cr_license_data_, r, RSA_PKCS1_PADDING) <= 0) {
free((void*)__cr_license_data_);
__cr_license_data_ = NULL;
}
RSA_free(r);
}
}
BIO_free_all(licIn);
}
This part of code on windows works well, so I assume public key is not an issue.
__cr_license_ = lic;
unsigned char lic_encoded[CR_LIC_LEN];
DWORD dwSize;
if(CryptStringToBinaryA(__cr_license_, 0/*autocalculate*/, CRYPT_STRING_BASE64, lic_encoded, &dwSize, NULL, NULL) && dwSize == CR_LIC_LEN) {
HCRYPTPROV hProv;
if(CryptAcquireContext(&hProv, NULL, MS_ENHANCED_PROV, PROV_RSA_FULL, CRYPT_VERIFYCONTEXT)) {
PCERT_PUBLIC_KEY_INFO pki = NULL;
DWORD dwKeySize;
if(CryptDecodeObjectEx(X509_ASN_ENCODING, X509_PUBLIC_KEY_INFO, license_pub_der, sizeof(license_pub_der), CRYPT_ENCODE_ALLOC_FLAG, NULL, &pki, &dwKeySize)) {
HCRYPTKEY hKey = 0;
if(CryptImportPublicKeyInfo( hProv, X509_ASN_ENCODING, pki, &hKey)) {
But after that anything I try to do with message leads to CRYPT_E_ASN1_BADTAG
. I tried CryptMsgOpenToDecode
with CryptMsgUpdate
, CryptDecodeObject
, CryptVerifyMessageSignatureWithKey
- nothing works.
Basically I think that the problem is in pkcs1 and pkcs7 incompatibility as owlstead mentioned. Does anyone has experience working with pkcs1 format importing/converting/etc with MS CAPI?
Any help or even a clue is appreciated a lot! Thanks in advance!
CRYPT_E_ASN1_BADTAG
error you are refering to. – GaglianoCryptVerifyMessageSignatureWithKey
? For some reason it is not specified by the API (not that I'm surprised, Microsoft regularly underspecifies) – Mutineer