Listing [Python.Docs]: hashlib - Secure hashes and message digests.
I want to start with a note: it is possible to have multiple instances of the "same" library loaded in a process, as shown in [SO]: Independent CDLL Library Instances with Ctypes (@CristiFati's answer).
Check [SO]: How to enable FIPS mode for libcrypto and libssl packaged with Python? (@CristiFati's answer).
It's a duplicate of the referenced (closed) question at the end (Update #0 section).
What happens:
Default LibCrypto (if found) is loaded by import hashlib
You load "your own" LibCrypto version, where you set FIPS mode on
HashLib on the other hand, is still using the default instance (where FIPS is not enabled)
I tested with OpenSSL 3.0.8.
Output - Preliminary (note that I'll be reusing this console):
[cfati@cfati-5510-0:/mnt/e/Work/Dev/StackExchange/StackOverflow/q075913071]> ~/sopr.sh
### Set shorter prompt to better fit when pasted in StackOverflow (or other) pages ###
[064bit prompt]> # Test OpenSSL
[064bit prompt]> /usr/local/pc064/openssl/openssl/3.0.8/bin/openssl md5 ./code00.py
FATAL: Startup failure (dev note: apps_startup()) for /usr/local/pc064/openssl/openssl/3.0.8/bin/openssl
80E2A959297F0000:error:80000002:system library:process_include:No such file or directory:crypto/conf/conf_def.c:805:calling stat(fipsmodule.cnf)
80E2A959297F0000:error:07800069:common libcrypto routines:provider_conf_load:provider section error:crypto/provider_conf.c:156:section=fips_sect not found
80E2A959297F0000:error:0700006D:configuration file routines:module_run:module initialization error:crypto/conf/conf_mod.c:270:module=providers, value=provider_sect retcode=-1
[064bit prompt]>
[064bit prompt]> # Local configuration stuff
[064bit prompt]> OPENSSL_CONF_INCLUDE=/usr/local/pc064/openssl/openssl/3.0.8/ssl /usr/local/pc064/openssl/openssl/3.0.8/bin/openssl md5 ./code00.py
Error setting digest
80123BCEDC7F0000:error:0308010C:digital envelope routines:inner_evp_generic_fetch:unsupported:crypto/evp/evp_fetch.c:373:Global default library context, Algorithm (MD5 : 102), Properties ()
80123BCEDC7F0000:error:03000086:digital envelope routines:evp_md_init_internal:initialization error:crypto/evp/digest.c:254:
Check [GitHub]: openssl/openssl - openssl.cnf failed to load fispmodules.cnf if use without full path in 3.0.1 (@CristiFati's comment) for details regarding OPENSSL_CONF_INCLUDE.
Since OpenSSL build (with FIPS support) is valid, moving to Python stuff. Here's an updated version of your code:
code00.py:
#!/usr/bin/env python
import ctypes as cts
import hashlib
import os
import sys
#LIBCRYPTO = "/usr/local/ssl/lib/libcrypto.so.3"
CUSTOM_PATH = "/tmp/fips_files"
LIBCRYPTO = os.path.join(CUSTOM_PATH, "libcrypto.so.3")
CUSTOM_PATH = "/usr/local/pc064/openssl/openssl/3.0.8" # @TODO - cfati: Custom build path
LIBCRYPTO = os.path.join(CUSTOM_PATH, "lib", "libcrypto.so") # @TODO - cfati: Custom build lib path
POSSL_LIB_CTX = cts.c_void_p
POSSL_PROVIDER = cts.c_void_p
def main(*argv):
libcrypto = cts.CDLL(LIBCRYPTO)
OSSL_PROVIDER_set_default_search_path = libcrypto.OSSL_PROVIDER_set_default_search_path
OSSL_PROVIDER_set_default_search_path.argtypes = (POSSL_LIB_CTX, cts.c_char_p)
OSSL_PROVIDER_set_default_search_path.restype = cts.c_int
OSSL_LIB_CTX_load_config = libcrypto.OSSL_LIB_CTX_load_config
OSSL_LIB_CTX_load_config.argtypes = (POSSL_LIB_CTX, cts.c_char_p)
OSSL_LIB_CTX_load_config.restype = cts.c_int
OSSL_PROVIDER_load = libcrypto.OSSL_PROVIDER_load
OSSL_PROVIDER_load.argtypes = (POSSL_LIB_CTX, cts.c_char_p)
OSSL_PROVIDER_load.restype = POSSL_PROVIDER
EVP_default_properties_is_fips_enabled = libcrypto.EVP_default_properties_is_fips_enabled
EVP_default_properties_is_fips_enabled.argtypes = (POSSL_LIB_CTX,)
EVP_default_properties_is_fips_enabled.restype = cts.c_int
EVP_default_properties_enable_fips = libcrypto.EVP_default_properties_enable_fips
EVP_default_properties_enable_fips.argtypes = (POSSL_LIB_CTX, cts.c_int)
EVP_default_properties_enable_fips.restype = cts.c_int
provider_search_path = CUSTOM_PATH.encode()
provider_search_path = os.path.join(CUSTOM_PATH, "lib", "ossl-modules").encode() # @TODO - cfati: Custom mod location
res = OSSL_PROVIDER_set_default_search_path(None, provider_search_path)
if res == 0:
print("OSSL_PROVIDER_set_default_search_path error")
return 1
cfg_search_path = provider_search_path
cfg_search_path = os.path.join(CUSTOM_PATH, "ssl").encode() # @TODO - cfati: Custom cfg location
res = OSSL_LIB_CTX_load_config(None, os.path.join(cfg_search_path, b"openssl.cnf"))
if res == 0:
print("OSSL_LIB_CTX_load_config error")
return 1
base_provider = OSSL_PROVIDER_load(None, b"base")
fips_provider = OSSL_PROVIDER_load(None, b"fips")
print(base_provider, fips_provider)
fips_status = EVP_default_properties_is_fips_enabled(None)
print("EVP_default_properties_is_fips_enabled: {:d}".format(fips_status))
res = EVP_default_properties_enable_fips(None, 1)
print ("EVP_default_properties_enable_fips: {:d}".format(res))
fips_status = EVP_default_properties_is_fips_enabled(None)
print("EVP_default_properties_is_fips_enabled: {:d}".format(fips_status))
print("HL mod: {:}".format(getattr(hashlib, "_hashlib", None)))
print("SHA1: {:s}".format(hashlib.sha1("test_str".encode('utf-8')).hexdigest()))
print("MD5: {:s}".format(hashlib.md5("test_str".encode('utf-8')).hexdigest()))
if __name__ == "__main__":
print("Python {:s} {:03d}bit on {:s}\n".format(" ".join(elem.strip() for elem in sys.version.split("\n")),
64 if sys.maxsize > 0x100000000 else 32, sys.platform))
rc = main(*sys.argv[1:])
print("\nDone.\n")
sys.exit(rc)
Output:
[064bit prompt]> # Attempt #0.
[064bit prompt]> OPENSSL_CONF_INCLUDE=/usr/local/pc064/openssl/openssl/3.0.8/ssl python3.9 ./code00.py
Python 3.9.16 (main, Dec 7 2022, 01:11:51) [GCC 9.4.0] 064bit on linux
20919264 20902112
EVP_default_properties_is_fips_enabled: 0
EVP_default_properties_enable_fips: 1
EVP_default_properties_is_fips_enabled: 1
HL mod: <module '_hashlib' from '/usr/lib/python3.9/lib-dynload/_hashlib.cpython-39-x86_64-linux-gnu.so'>
SHA1: f9a90e7c1ff51236191623b84267d110c617118a
MD5: 74e710825309d622d0b920390ef03edf
Done.
[064bit prompt]>
[064bit prompt]> # Attempt #1.
[064bit prompt]> LD_LIBRARY_PATH=${LD_LIBRARY_PATH}:/usr/local/pc064/openssl/openssl/3.0.8/lib OPENSSL_CONF_INCLUDE=/usr/local/pc064/openssl/openssl/3.0.8/ssl python3.9 ./code00.py
Python 3.9.16 (main, Dec 7 2022, 01:11:51) [GCC 9.4.0] 064bit on linux
36180960 36163808
EVP_default_properties_is_fips_enabled: 0
EVP_default_properties_enable_fips: 1
EVP_default_properties_is_fips_enabled: 1
HL mod: <module '_hashlib' from '/usr/lib/python3.9/lib-dynload/_hashlib.cpython-39-x86_64-linux-gnu.so'>
SHA1: f9a90e7c1ff51236191623b84267d110c617118a
MD5: 74e710825309d622d0b920390ef03edf
Done.
[064bit prompt]>
[064bit prompt]> ldd /usr/lib/python3.9/lib-dynload/_hashlib.cpython-39-x86_64-linux-gnu.so
linux-vdso.so.1 (0x00007ffe72fe9000)
libcrypto.so.1.1 => /lib/x86_64-linux-gnu/libcrypto.so.1.1 (0x00007ff3fb0f9000)
libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007ff3faf07000)
libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007ff3faf01000)
libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007ff3faede000)
/lib64/ld-linux-x86-64.so.2 (0x00007ff3fb405000)
[064bit prompt]>
[064bit prompt]> # Attempt #2.
[064bit prompt]> ln -s /usr/local/pc064/openssl/openssl/3.0.8/lib/libcrypto.so ./libcrypto.so.1.1
[064bit prompt]> LD_LIBRARY_PATH=.:${LD_LIBRARY_PATH}:/usr/local/pc064/openssl/openssl/3.0.8/lib OPENSSL_CONF_INCLUDE=/usr/local/pc064/openssl/openssl/3.0.8/ssl python3.9 ./code00.py
Python 3.9.16 (main, Dec 7 2022, 01:11:51) [GCC 9.4.0] 064bit on linux
43861088 43843936
EVP_default_properties_is_fips_enabled: 0
EVP_default_properties_enable_fips: 1
EVP_default_properties_is_fips_enabled: 1
HL mod: None
SHA1: f9a90e7c1ff51236191623b84267d110c617118a
MD5: 74e710825309d622d0b920390ef03edf
Done.
Since (default) Python's HashLib is built with OpenSSL 1.1.*, and there's a big API / ABI difference between that and v3.*, it doesn't work (as seen, in the 3rd attempt I tried a (lame) workaround (gainarie), but HashLib bindings to OpenSSL failed to load, and default implementations were used).
In order to get past this, you need to build Python with OpenSSL 3.* (not sure if it works OOTB). Check [SO]: How to compile python3 on RHEL with SSL? SSL cannot be imported (@CristiFati's answer) for details (although versions differ).
Anyway, setting LD_LIBRARY_PATH should do the trick. In your case, something like:
LD_LIBRARY_PATH=/tmp/fips_files:${LD_LIBRARY_PATH} python ./code00.py
As I stated in your other question ([SO]: Access OpenSSL FIPS APIs from python ctypes not working (@CristiFati's answer)), things might be easier if using some specialized Python modules (OpenSSL wrappers).
