How to get the signed content from a PKCS7 envelop with M2Crypto?

Question

I need to get the digest of a PKCS#7 envelop to manually check it.

Usually when you want to validate the signature of a pkcs#7 envelop you do that:

from M2Crypto import SMIME, X509, BIO

sm_obj = SMIME.SMIME()
x509 = X509.load_cert(join(PATH, 'QualifiedChain.crt'))
sk = X509.X509_Stack()
sk.push(x509)
sm_obj.set_x509_stack(sk)

st = X509.X509_Store()

st.load_info(join(PATH, 'QualifiedChain.crt'))

sm_obj.set_x509_store(st)

# re-wrap signature so that it fits base64 standards
cooked_sig = '\n'.join(raw_sig[pos:pos + 76] for pos in
                       xrange(0, len(raw_sig), 76))

# now, wrap the signature in a PKCS7 block
sig = "-----BEGIN PKCS7-----\n%s\n-----END PKCS7-----\n" % cooked_sig


# and load it into an SMIME p7 object through the BIO I/O buffer:
buf = BIO.MemoryBuffer(sig)
p7 = SMIME.load_pkcs7_bio(buf)

signers = p7.get0_signers(sk)
certificat = signers[0]
data_bio = BIO.MemoryBuffer(MSG)
sm_obj.verify(p7, data_bio)  # This is the line that count.

But in my case, the digest type is md5sha1 that is not recognized by openssl:

$ openssl list-message-digest-commands
md4
md5
rmd160
sha
sha1

What I need to do I to get the pkcs#7 signedContent and to manually check it.

What I need is a Python equivalent to org.bouncycastle.cms.CMSSignedDataParser.

How can I get the digest to be able to validate it manually without having to use sm_obj.verify?

Jedthus

I've never heard of the "md5sha1" digest, googling doesn't seem to yield anything either. Perhaps it is as it reads? A SHA1'd MD5? Where are you getting that the digest is md5sha1? – Regress

It is a custom digest for some Signing Server it is why I need to validate it mannually. The md5sha1 is just the concat of bash hashes. – Jedthus

Have a look at v13.gr/blog/?p=303. That might be an alternative to getting the content and the signature out of a PKCS7 envelope to then manually verify it. – England

@England Looks really good. Do you mind creating a answer for this post and copy the code in it? – Jedthus

@England This is checking for a cert to be signed by a cacert not to be able to check if a pkcs7 is signed by a certficate will try to modify it. – Jedthus

@England This could help to : qistoph.blogspot.fr/2012/01/… – Jedthus

Can you post some example data you are trying to verify somewhere? – Chilung

Basically it is just a nonce value. – Jedthus

i.e: ZMTgLYpPUpNkx50aQchYXKClz7qfoqY4 – Jedthus

Answer 1

A PKCS7 has a collection of SignerInfos. Each SignerInfo may have a different Message Digest Algorithm.

See https://github.com/erny/pyx509. This needs pyasn1 and pyasn1-modules.

./pkcs_parse <pkcs7 signature in DER format>

This extracts the digest algorithm for each Signer Info.

Gentlewoman

Can you help me in signing the data (I mean the opposite process) with m2crypto or cryptography? – Convoluted

Answer 2

Ok so I was able to do it in bash:

#!/bin/bash

if [ $# -ne 1 ]; then
    echo "USAGE: $0 sig.pkcs7.pem"
    exit 1
fi

rm -fr /tmp/pkcs7tosignature
mkdir /tmp/pkcs7tosignature
cp "$1" /tmp/pkcs7tosignature/sig.pkcs7
cd /tmp/pkcs7tosignature/

# Convert PEM pkcs7 to DER
openssl pkcs7 -in sig.pkcs7 -inform PEM -out sig.der -outform DER

# Extract x509 certificate
openssl pkcs7 -in sig.pkcs7 -inform PEM -print_certs > cert.pem

# Look for signed signature offset
offset=$(openssl asn1parse -inform der -in sig.der | python -c "import sys; l = sys.stdin.readlines()[-1]; print int(l.split(':')[0]) + int(l.split('hl=')[1].split()[0])")
count=$(openssl asn1parse -inform der -in sig.der | python -c "import sys; l = sys.stdin.readlines()[-1]; print int(l.split('hl=')[1].split('l=')[1].split()[0])")

# Copy signed signature
dd if=sig.der of=signed-sha1.bin bs=1 skip=$[ $offset ] count=$count 2>/dev/null

# Extract public key from certificate
openssl x509 -inform pem -in cert.pem -noout -pubkey > pubkey.pem

# Decrypt signed signature
openssl rsautl -verify -pubin -inkey pubkey.pem < signed-sha1.bin > verified.bin

# Print pkcs7 algorithm
openssl asn1parse -inform der -in verified.bin | python -c "import sys; l = sys.stdin.read(); print l.split('OBJECT')[1].split('\n')[0].split(':')[1].strip()"

# Print pkcs7 signature
openssl asn1parse -inform der -in verified.bin | python -c "import sys; l = sys.stdin.read(); print l.split('[HEX DUMP]:')[1].split('\n')[0].strip()"

Just need to convert it in Python now.

Jedthus

Just use Popen calls in python to all of those commands and pipe the output to the appropriate objects :) – Regress

Sign, I just want to do the reverse. I need to generate a pkcs#7 signature from raw text and certificate.... Do you have any clue – Orvie

@Orvie The Sign paragraph of this HowTo should help you: proj.badc.rl.ac.uk/svn/ndg-security/branches/Dependencies/… – Jedthus

@Jedthus I've seen this before, but I need to do the samething as sbsign tool does. I still can't figure out what is the difference. github.com/CyanogenMod/android_external_sbsigntool – Orvie

It looks like you need to sign using SHA256 — github.com/CyanogenMod/android_external_sbsigntool/blob/… – Jedthus

Answer 3

Try the following, heavily inspired by this blog post:

import OpenSSL
from Crypto.Util import asn1

c=OpenSSL.crypto

certbuffer = # get your PEM here, e.g. by cert = open("/path/to/file.pkcs7.pem", 'rb').read()

# This is the certificate to validate
# an OpenSSL.crypto.X509 object
cert=c.load_certificate(crypto.FILETYPE_PEM, certbuffer)

# Get the signing algorithm
algo=cert.get_signature_algorithm()

# Get the ASN1 format of the certificate
cert_asn1=c.dump_certificate(c.FILETYPE_ASN1, cert)

# Decode the certificate
der=asn1.DerSequence()
der.decode(cert_asn1)

# The certificate has three parts:
# - certificate
# - signature algorithm
# - signature
# http://usefulfor.com/nothing/2009/06/10/x509-certificate-basics/
der_cert=der[0]
der_algo=der[1]
der_sig=der[2]

# The signature is a BIT STRING (Type 3)
# Decode that as well
der_sig_in=asn1.DerObject()
der_sig_in.decode(der_sig)

# Get the payload
sig0=der_sig_in.payload

# Do the following to see a validation error for tests
# der_cert=der_cert[:20]+'1'+der_cert[21:]

# First byte is the number of unused bits. This should be 0
# http://msdn.microsoft.com/en-us/library/windows/desktop/bb540792(v=vs.85).aspx
if sig0[0]!='\x00':
    raise Exception('Number of unused bits is strange')

# Now get the signature itself
sig=sig0[1:]

# and do the actual verification against your custom hash
# ...

England

I cannot verify this as I don't have a PEM with this structure available. If @Jedthus you can provide a test file, I'm willing to try and improve, if necessary. – England

The problem is that your code is for the certificate signature not for the pkcs7 signature. Apparently pyOpenSSL doesn't have the bidding for PKCS7. – Jedthus

@Jedthus Do you have a test file? I'd like to try to extract the signature from it using M2Crypto. – England

Any clues? Where did you arrived? – Jedthus

@Jedthus Thanks for asking. I tried again, really hard, to get this to work. After reading through the whole source code (!) of M2Crypto, I strongly believe that it is not possible to get a certificate out of a PKCS7 object. This is mainly for writing a PKCS7 envelope, and perhaps signing it. pyOpenSSL is certainly not up to the task for that, I also studied it intensively. If you also wanna try, I'd go for pkcs7=m2.SMIME.load_pkcs7(), der = m2.BIO.MemoryBuffer() and pkcs7.write_der(der). Unfortunately, I cannot get the certificate out of the der object then, and also not parse it. – England

I'm going to remove this answer in a few days, as it doesn't really solve any problems. If anybody is interested in that avenue, the link is still in the comments to the original question. Thanks for providing a sample certificate. Sorry it did not help anything though :( – England

@Jedthus I could have easily done it in languages that I'm more familiar with, so I just couldn't believe it was not possible in Python. I took that as a challenge though ;) Sorry for not having been of more help. – England