AIA returning PEM-encoded PKCS7

[fredericoschardong]

Dear @MatthiasValvekens,

There is a CA that is using a rogue endpoint for Authority Information Access. The URL returns an application/octet-stream content type with a PEM-encoded PKCS7 file with the certificate chain. Neither application/octet-stream nor PEM-encoded PKCS7 can be handled by pyhanko_certvalidator's fetchers/common_utils.py.

The first step for a PR is to add application/octet-stream to ACCEPTABLE_CERT_PEM_ALIASES, however the second is not clear to me. We require the functionality of elif permit pem and (content type in ACCEPTABLE CERT PEM ALIASES):, which accepts PEM, as well as elif content type == 'application/pkcs7-mime':, which reads CMS.

Would you be willing to embrace a PR that addresses the aforementioned issues? If so, could you please tell me how you want me to organize the solution?

You can see this in action with the attached files files.zip and python3 -m pyhanko --verbose sign validate --pretty-print signed.pdf --trust ac-raiz-v3.cer --trust tsa_cert.pem.

Thank you in advance.

[MatthiasValvekens]

Hi @fredericoschardong,

Sure thing! From what I remember, this would be legal if the PKCS#7 payload were not PEM-encoded (and the content type were correct...). As such, most of the handling code should already be in place. If that is indeed the case, all that needs to happen is (a) add application/octet-stream to the set of admissible content types, and (b) make the PEM processor a bit smarter. I'd be happy to accept a PR that does that :).

I've put some implementation notes below, but if/when you make your PR, please branch off the 0.19.6 release tag instead of master. That should make sure the change can be applied to the current master branch and the next bugfix/patch release as well.

[Reason: The master branch in pyhanko-certvalidator contains a number of breaking changes that are not in "mainline" pyHanko yet. The corresponding changes in pyHanko are on the feature/ades-validation branch, which I originally intended to merge half a year ago, but that one turned out to be way more complicated than I anticipated... Judgment error on my part.]

---

Currently, there's this piece of logic in the response handler:

    elif permit_pem and (content_type in ACCEPTABLE_CERT_PEM_ALIASES):
        # technically, PEM is not allowed here, but of course some people don't
        # bother following the rules
        if pem.detect(response_data):
            for _, _, data in pem.unarmor(response_data, multiple=True):
                yield x509.Certificate.load(data)

The for _, _, data in pem.unarmor(...) dummies out two variables, but IIRC those contain information from the PEM header. You should be able to use that info to decide whether or not we're processing a cert or a PKCS#7/CMS payload. Not ideal, but we're in "out-of-spec" territory anyway, so I guess it'll do.

Other than that, I'd factor out the common extraction logic from the elif content_type == 'application/pkcs7-mime' branch to keep the code clean, but that's it.

[MatthiasValvekens]

FYI: the documentation for pem.unarmor(...) can be found over in the asn1crypto repo.