You have to create an EVP_PKEY with domain parameters first if you use custom prime and generator.
// Create the OSSL_PARAM_BLD.
OSSL_PARAM_BLD* paramBuild = OSSL_PARAM_BLD_new();
if (!paramBuild) {
// report the error
}
// Set the prime and generator.
if (!OSSL_PARAM_BLD_push_BN(paramBuild, OSSL_PKEY_PARAM_FFC_P, prime) ||
!OSSL_PARAM_BLD_push_BN(paramBuild, OSSL_PKEY_PARAM_FFC_G, generator)) {
// report the error
}
// Convert to OSSL_PARAM.
OSSL_PARAM* param = OSSL_PARAM_BLD_to_param(paramBuild);
if (!param) {
// report the error
}
// Create the context. The name is DHX not DH!!!
EVP_PKEY_CTX* domainParamKeyCtx = EVP_PKEY_CTX_new_from_name(nullptr, "DHX", nullptr);
if (!domainParamKeyCtx) {
// report the error
}
// Initialize the context.
if (EVP_PKEY_fromdata_init(domainParamKeyCtx) <= 0) {
// report the error
}
// Create the domain parameter key.
EVP_PKEY* domainParamKey = nullptr;
if (EVP_PKEY_fromdata(domainParamKeyCtx, &domainParamKey,
EVP_PKEY_KEY_PARAMETERS, param) <= 0) {
// report the error
}
The domain parameter domainParamKey is ready! Now, use it to create a key pair.
EVP_PKEY_CTX* keyGenerationCtx = EVP_PKEY_CTX_new_from_pkey(nullptr, domainParamKey, nullptr);
if (!keyGenCtx) {
// report the error
}
if (EVP_PKEY_keygen_init(keyGenerationCtx) <= 0) {
// report the error
}
EVP_PKEY* keyPair = nullptr;
if (EVP_PKEY_generate(keyGenerationCtx, &keyPair) <= 0) {
// report the error
}
The key pair keyPair is ready!
See this project on GitHub: https://github.com/eugen15/diffie-hellman-cpp. It shows how to generate key pairs, set and get Diffie-Hellman parameters. Plus, it has some custom Diffie-Hellman implementations.
See the following files for an OpenSSL 3 example:
- diffie-hellman-openssl.h
- diffie-hellman-openssl.cpp
The following files is a LibreSSL example which is basically OpenSSL before version 3. This is to be sure you have backward compatibility.
- diffie-hellman-libressl-dh.h
- diffie-hellman-libressl-dh.cpp
More things to consider
FIRST
If you used a custom prime before OpenSSL 3, then it is better to move to a predefined safe prime. You can see an example here: https://www.openssl.org/docs/manmaster/man7/EVP_PKEY-DH.html
int priv_len = 2 * 112;
OSSL_PARAM params[3];
EVP_PKEY *pkey = NULL;
EVP_PKEY_CTX *pctx = EVP_PKEY_CTX_new_from_name(NULL, "DH", NULL);
params[0] = OSSL_PARAM_construct_utf8_string("group", "ffdhe2048", 0);
/* "priv_len" is optional */
params[1] = OSSL_PARAM_construct_int("priv_len", &priv_len);
params[2] = OSSL_PARAM_construct_end();
EVP_PKEY_keygen_init(pctx);
EVP_PKEY_CTX_set_params(pctx, params);
EVP_PKEY_generate(pctx, &pkey);
...
EVP_PKEY_free(pkey);
EVP_PKEY_CTX_free(pctx);
Alice executes the above code (taken from the OpenSSL manual). Then you get the prime and generator:
EVP_PKEY_get_bn_param(pkey, OSSL_PKEY_PARAM_FFC_P, &prime);
EVP_PKEY_get_bn_param(pkey, OSSL_PKEY_PARAM_FFC_G, &generator);
Then send it to Bob and use my code above to create the key pair.
SECOND
Use Elliptic-Curve Diffie-Hellman (ECDH) key exchange for new projects. It is much faster. Leave your DH as a fallback. See examples here.