Loading a RSA private key from memory using libxmlsec
It takes binary data, a size, a format string and several PEM-callback related parameters.
When I call the function, it just stucks, uses 100% of the CPU time and never returns. Quite annoying, because I have no way of finding out what is wrong.
Here is my code:
d_xmlsec_dsig_context->signKey = xmlSecCryptoAppKeyLoadMemory( reinterpret_cast<const xmlSecByte*>(data), static_cast<xmlSecSize>(datalen), xmlSecKeyDataFormatBinary, NULL, NULL, NULL );
data is a const char* pointing to the raw bytes of my RSA key (using i2d_RSAPrivateKey(), from OpenSSL) and datalen the size of data.
My test private key doesn't have a passphrase so I decided not to use the callbacks for the time being.
Has someone already done something similar ? Do you guys see anything that I could change/test to get forward on this problem ?
I just discovered the library on yesterday, so I might miss something obvious here; I just can't see it.
Thank you very much for your help.
I changed the format of data to PEM, using the OpenSSL function PEM_write_bio_RSAPrivateKey() and changed the third argument to the call to xmlSecCryptoAppKeyLoadMemory() so it matches the new format.
The new code is:
d_xmlsec_dsig_context->signKey = xmlSecCryptoAppKeyLoadMemory( reinterpret_cast<const xmlSecByte*>(data), // data is now in PEM format static_cast<xmlSecSize>(datalen), xmlSecKeyDataFormatPem, // Updated NULL, NULL, NULL );
And since then, everything works: the call does no longer get stuck.