Salt length check fails during handshake on Windows with tpm2 on server side (on some TPM modules) #75
Comments
|
Hmmm. There may be something wrong in the public key or the certificate. Would you please be able to compare the certificate used when it works vs when it doesn't work ( |
|
What makes me wonder is the fact that the RSA-PSS code seems to be used, the certificate is RSA though. Might this be the problem? Here is the output for the certificates I am using. tpm2 certificate: regular certificate: The metadata is the same, only the creation date and the actual certificate entries are different. The regular keypair works in the handshare, the tpm2 does not. |
|
The handshake actually only fails on real tpm2 hardware, it works now in a virtual machine with a simulated tpm2 module. Could there be some functionality missing on my tpm2 hardware module? And I also debugged the specific function a little deeper and it actually goes the same way with the tpm2 certificate and the regular certificate. The XOR of EM and DB fails, in the screenshot you can see that, i.e. the first byte on each side (32 and 33) will not XOR to 0, which it expects later:
The check is done on client side, which does not use the tpm2 provider. So I assume the server side (with tpm2) sends wrong data? Where would be the best place to look at what actually goes wrong here? |
philippun1
changed the title
|
I was able to test it on other TPM hardware and it works on 1 machine and does not work on 2 other machines (in addition to it working in a VM with a simulated TPM module). So in general it works, which is good. The question now would be, how to determine which TPM module works and which do not? @gotthardp Do you maybe have an idea? |
As seen in the debug window TPM is generating signature with salt length = 222 (which is the max value possible with your key and digest). But OpenSSL would want to see 32 (same as digest length) as per TLS 1.3. This is likely because TPM on your PC is following the old standard (see this link too: https://learn.microsoft.com/en-us/answers/questions/467673/windows-10-tpm-2-0-client-authentication-in-tls-1). |
|
Hi philippun1, May I ask do you get the root cause? I have the same issue as yours. My tpm2 chip is SLB 9665 TPM2.0 which probably an old one. |
Hi,
if I try to perform a TLS handshake with tpm2 provider being used on the server side on Windows, I get the following error:
This command is being used to create the certificate:
The actual error happens in the
rsa_pss.cin the functionRSA_verify_PKCS1_PSS_mgf1. The Autos tab shows the mismatching sizes:Although the error happens inside of OpenSSL code, I assume the error is tpm2 provider related. Everything works as expected if I do not load the provider and use a regular certificate.
If it is of any help, I can also test the code on a Linux VM and see if it works there.
Any help is appreciated, even if it is only a hint on where to look at so I can debug into this myself. Thanks.
The text was updated successfully, but these errors were encountered: