If the accepted answer doesn't apply to you (e.g., because you created the CSR from the same Mac, and should have the private key on this Mac), here's one more possibility:
When you happen to have selected the wrong keychain (like if you accidentally had single clicked on the System keychain rather than Login keychain), you may run into the same problem. If the cert signing request came from that machine, it would associate the private key with the Login keychain (at least, that seems to be the default). So if you somehow then downloaded and imported the certificate into another keychain like the System keychain (or iCloud keychain, as some comments have mentioned), Keychain Access would not find the matching private key and would not put the cert under "My certificates", and so it cannot export a p12. It would be similar to the case of importing to the keychain on a different Mac.
Make sure the import is into the keychain associated with the private key. Often this would be the Login keychain.