self generated cert has invalid signature

[RebootExpert]

using 3.5.0 and NANO module.
I am doing SSH R&D, and I started with example SSH/sshServerUserAuth. It build and runs fine.
In the example it call to generate a cert and key if there isn't one available for use. So I add a dummy html and enable HTTPS web server in the project. the certificate I download from the web browser indicate the certificate that generated has an invalid digital signature.
then I build the example, SSL/SslOnboardCertGeneration/Simple without any modification. It's the same result.

invalidcert.jpg (34.38 KiB) Viewed 1270 times

for a comparison, this is how it looks when I generate a self signed cert using OpenSSL, you're seeing the the cert is not trusted that's because I've not install the cert into the CA store yet.

valid.jpg (37.74 KiB) Viewed 1270 times

[RebootExpert]

also if I add a compile server cert and key into the SSH example instead of using the onboard generated cert and key, and that cause the SshAccept function fail and return an error code 304 which is SSH_ERROR_BAD_KEY.
The cert and key are created by OpenSSL and compile into cpp file using makeserver.sh file. I doubt that's an issue of the key

[RebootExpert]

This reminds me when I first play with SSL a few years ago that having the same problem with 2.9.3. I forget how it get solved, but I think it has to do with subject alternate name.

same to the second problem in the post below
viewtopic.php?t=3055

[RebootExpert]

with further troubleshoot, I found that was caused by the signature algorithm mismatch with the public key algorithm in the certificate.
Since it's a self signed certificate, the two algorithms in the cert need to be the same, either RSA or ECC.
If a cert was signed by a CA, the signature algorithm in the cert need to match with the CA encryption algorithm.

The fix I make to SSL_CreateNewSelfSignedCert(CertGenData & GenData) in \nburn\libraries\crypto\NetBurner\NbCertGen.cpp

Code: Select all

#if defined(SSL_KEY_ECC)
    gNewCert->sigType = CTC_SHA256wECDSA;
#elif defined(SSL_KEY_RSA)
    gNewCert->sigType = CTC_SHA256wRSA;
#endif

Certificate:
Data:
Version: 3 (0x2)
Serial Number:
0e:f4:68:e3:a2:d7:6b:6d:70:1d:32:28:ca:0a:02:66
Signature Algorithm: sha256WithRSAEncryption
Issuer: C = US, ST = CA, L = San Diego, O = NetBurner, OU = CodeDemo, CN = MyNetburner, emailAddress = Sales@NetBurner.com
Validity
Not Before: Jul 5 15:55:31 2023 GMT
Not After : Jul 3 15:55:31 2033 GMT
Subject: C = US, ST = CA, L = San Diego, O = NetBurner, OU = CodeDemo, CN = MyNetburner, emailAddress = Sales@NetBurner.com
Subject Public Key Info:
Public Key Algorithm: id-ecPublicKey
Public-Key: (384 bit)
pub:
04:fe:77:01:69:ea:08:f2:16:1a:ff:fd:1b:4d:a2:
1c:6b:53:41:27:30:f2:78:18:32:5a:9b:1c:0d:80:
86:f1:f9:1c:4d:54:c5:93:a2:a0:f6:38:07:81:df:
a3:92:7e:f4:0c:fa:ee:76:60:0d:ca:1d:1b:9b:1b:
e9:24:4c:ff:6f:cc:6d:e7:13:f4:43:c1:6e:ad:3b:
c5:a1:57:41:ec:f4:06:cf:bd:af:2a:74:ce:74:2b:
96:8c:ed:3e:84:8d:79
ASN1 OID: secp384r1
NIST CURVE: P-384
X509v3 extensions:
X509v3 Subject Alternative Name:
IP Address:10.250.5.80, DNS:10.250.5.80
Signature Algorithm: sha256WithRSAEncryption
30:64:02:30:0a:0a:29:12:16:c5:be:50:ac:d20e:c3:d3:
2d:af:4d:d9:1c:4a:11:d6:c3:09:fd:5f:6c:40:b3:6d:79:52:
acd3:b2:0f:8e:4f:28:4a:6e:0a:e0:da:b6:da:01:02:30:
2e:31:65:c2:75:13:a2:51:9a:95:27:32:aa:3d:80:a4:b3:46:
76:99:f0:ba:3a:a8:4e:4d:3ded:01:1e:55:bb:bb:80:b9:
d3:6c:95:cc:f3:87:73:82:9e:b9:68:67

[TomNB]

Hello,

That is some great troubleshooting, and thank you very much for sharing the results. We have been checking the behavior and any fixes from 3.5.0 to the latest 3.5.3, and will post back to you as soon as we know more. Have a great weekend and thank you again.

[Forrest]

Hello,

I examined the onboard certification generation code and agree with your assessment and fix. I am working on improving the selection logic around selecting a DEFAULT_KEY_TYPE to be used, instead of relying on a plethora of defines. This should arrive with the next release. But in the short term, your solution is both valid and will be used. Any fix put in to place around selecting the key type will be backwards compatible.

I'm curious, what method did you use to set RSA as your preferred key type?

with further troubleshoot, I found that was caused by the signature algorithm mismatch with the public key algorithm in the certificate.
Since it's a self signed certificate, the two algorithms in the cert need to be the same, either RSA or ECC.
If a cert was signed by a CA, the signature algorithm in the cert need to match with the CA encryption algorithm.

The fix I make to SSL_CreateNewSelfSignedCert(CertGenData & GenData) in \nburn\libraries\crypto\NetBurner\NbCertGen.cpp

Code: Select all

#if defined(SSL_KEY_ECC)
    gNewCert->sigType = CTC_SHA256wECDSA;
#elif defined(SSL_KEY_RSA)
    gNewCert->sigType = CTC_SHA256wRSA;
#endif

[RebootExpert]

I use the default ECC. However there's a HAVE_ECC defined in the \nburn\libraries\include\crypto\platform\NANO54415, my guess is if you commented it out, it will default to RSA. But I didn't test with RSA