BoringSSL-based binary packages

[ilammy]

Is your feature request related to a problem? Please describe.
Current packaging options for Themis provide only packages that rely on system's OpenSSL. This is convenient on general-purpose server platforms. However, more and more deployments forgo TLS support entirely, leaving it to edge machines, with data-processing ones not having a need to include OpenSSL. Embedded systems may not fit the entire OpenSSL library too.

Describe the solution you'd like to see
It would be nice for Themis to ship with prebuilt packages that don't depend on system OpenSSL, using embedded BoringSSL.

Describe alternatives you've considered
Current alternative is to build Themis from source to use it with applications that already use BoringSSL and do not rely on systems's OpenSSL. This is more involved that installing it from the package, but should work.

[Lagovas]

You want to install themis which is statically compiled with some backend (openssl/boringssl) to not install openssl/boringssl to your system? Or you want some API to configure themis at runtime from your app with already initialized backend engine (like some "engine" object from openssl)?

[ilammy]

You want to install themis which is statically compiled with some backend (openssl/boringssl) to not install openssl/boringssl to your system?

This.

It's about providing libthemis-boringssl0 packages that ship Themis with statically linked BoringSSL, along with the current alternative of libthemis0 that are linked against the system's OpenSSL.

[popaaaandrei]

You want to install themis which is statically compiled with some backend (openssl/boringssl) to not install openssl/boringssl to your system?

I would even go as far as remove the statically compiled openssl/boringssl at all. My suspicion is that from the whole openssl API, soter uses a very limited set of crypto operations, which could be just embedded (cherry-picked) into the soter code directly. openssl comes also with an extended X.509 API that it's not needed.

IMHO there are a number of advantages of not depending on any underlying SSL library.

• avoiding going to openssl-dependency hell
• decoupling the Themis API from OpenSSL thus accepting all kinds of combinations; it seems that all openssls are equal, but some are more equal than the others and the idea is to have one platform-independent package
• smaller library
• simpler DevOps
• being able to better interface with packages and framework such as SwiftNIO SSL that come with their own implementation of TLS
• potential use for embedded devices and sensors with low resources available; the hypethesis here is that they do not have the resources needed to load the complete openssl library into memory
• on the server-side we see that TLS needs are being pushed to the edges to the ingress controllers and east-west traffic being upgrated by TLS though proxies (service mesh pattern)
• sometimes you just need to send a Secure Message

@gene-eu @vixentael what's your take on this?
Andrei

[ilammy]

openssl comes also with an extended X.509 API that it's not needed

Modern linkers have a dead code elimination pass and are able to remove whatever code is unused during static linkage. Admittedly, it's coarse because OpenSSL encryption API uses EVP abstraction and we're using it through that, so it's not like linker will remove everything besides exactly the code Soter uses. But it should remove the libcrypto bits that Soter definitely does not use, such as ASN.1, PKCS, PEM, X.509 parsers, probably most of the BIO code, etc. It will leave everything transitively used from generic EVP API though. OpenSSL has algorithm-specific API so it should probably be possible to avoid including even more code, such as unneeded algorithm implementations, if that's really necessary. And just for completeness, Soter does not use libssl or link against it, so there will be no TLS-related code if static linkage is used.

[ilammy]

To put some numbers to where my mouth is, here are sizes that Themis currently has with typical builds on a typical x86_64 machine with Linux.

Crypto backend linkage	On-disk size	In-memory size
Dynamic OpenSSL *	0.16 MB (+ OpenSSL)	2.92 MB
Static OpenSSL	2.50 MB total	2.49 MB
Static BoringSSL	0.97 MB total	0.96 MB

^{* This is what is shipped in packaged binary form, everything else are custom builds.}

That said, it seems I have been overoptimistic about linker capabilities, but I'm still fairly sure that we can do something about size optimizations (they were never a priority).

How I did the measurements

Shared library dynamically linked against system OpenSSL

Build:

make
strip build/lib*.so

On-disk sizes can be measured with ls -l.

Library	Size, bytes
libcrypto.so.1.1	2925408
libsoter.so.0	111392
libthemis.so.0	55112

The application package will bundle 162.6 KB of libraries. Additionally, 2.79 MB of libcrypto will be installed by dependency manager, plus whatever else is in the package since it typically includes libssl too (for another 571.7 KB). OpenSSL disk usage is going to be shared by applications that need it, same goes for Themis.

In-memory size can be estimated with size, adding up text and data sections which are going to be loaded into memory. This will give approximate virtual memory usage (excluding zero-initialized data from BSS), not actual resident set that depends on run-time behavior. However, it gives a good perspective of how much of the file is loaded, and how much is just ancillary data that stays on disk.

Library	Total, bytes	Text sections	Data sections
libcrypto.so.1.1	2914610	2730666	183944
libsoter.so.0	101995	68555	33440
libthemis.so.0	50352	48944	1408

So the appliction process will load 148.7 KB of Themis into memory, followed by 2.78 MB of OpenSSL, for a total of 2.92 MB of virtual memory usage (plus some 13 KB of zero pages).

Shared library statically linked against system OpenSSL

There is no easy option to do this, but it can be hacked in the following way:

make CRYPTO_ENGINE_LDFLAGS="/lib/x86_64-linux-gnu/libcrypto.a -ldl -pthread -Wl,--exclude-libs,ALL -flto -Wl,--gc-sections"
strip build/lib*.so

We currently don't really support static builds so some additional flags have to be manually added to enable appropriate size optimizations.

On-disk sizes:

Library	Size, bytes
libsoter.so.0	2567920
libthemis.so.0	55112

That's not much, to be honest: 2.5 MB installation size. Probably because of the way OpenSSL is compiled, the linker was not able to remove some code (like, X.509 parsing is still there).

In-memory sizes:

Library	Total, bytes	Text sections	Data sections
libsoter.so.0	2556342	2348382	207960
libthemis.so.0	50352	48944	1408

No significant change here either, 2.49 MB of virual memory, give or take.

Shared library statically linked against vendored BoringSSL

This is supported better in the build system, but some flags are still necessary:

LDFLAGS="-Wl,--exclude-libs,ALL -flto -Wl,--gc-sections" make ENGINE=boringssl
strip build/lib*.so

On-disk size:

Library	Size, bytes
libsoter.so.0	959344
libthemis.so.0	55112

This is a bit better, as you can see: 0.97 MB installation size. There is still some room for optimization as I see some unnecessary symbols in the binaries before stripping.

In-memory size:

Library	Total, bytes	Text sections	Data sections
libsoter.so.0	952071	917239	34832
libthemis.so.0	49670	48270	1400

Memory image is also a bit smaller, for a total of 0.96 MB.

If memory is premium and static linkage is used for Themis as well, it's possible to shave off even more because the linker will throw out parts of Themis/Soter that are not used. For example, if the application does not use Secure Session or Comparator, they won't be included along with ed25519 handling code that they need. It won't throw things like RSA support even if it's not needed though, but there is some room for tailored builds if someone really needs to be minimal.

[vixentael]

Install themis which is statically compiled with some backend (openssl/boringssl)
It's about providing libthemis-boringssl0 packages that ship Themis with statically linked BoringSSL, along with the current alternative of libthemis0 that are linked against the system's OpenSSL.

1. This changes responsibility to maintain/update crypto-engine from users to us. We don't follow Open/Boring/LibraSSL updates so carefully.

2. Also, I imagine situations, where app has 3 dependencies that use crypto-engine, if all of them are using static libs, it means 3 versions of crypto-engine will be installed (↑ app size, ↑ duplicated symbols?).

3. I'm not against shipping Themis with statically linked BoringSSL (libthemis-boringssl0), but we are jumping into updating dependencies more often water. Especially with BoringSSL, as even Google doesn't support versioning and doesn't recommend to link to master.

which could be just embedded (cherry-picked) into the soter code directly

4. Cherrypicking code from crypto-engine to Soter makes us responsible for all the hacks/bugs/backdoors. We are not. Themis doesn't implement own crypto, Themis relies on crypto-engine.

potential use for embedded devices and sensors with low resources available

5. They have their own SSL, like BearSSL, WolfSSL or SSL-less solutions, like E4. Themis is not a lib for low energy / weak CPU devices, it was never designed and tested like this. Themis is tested and proved to work on ARM x32 and above.

being able to better interface with packages and framework such as SwiftNIO SSL that come with their own implementation of TLS

6. SwiftNIO links against BoringSSL. I imagine that it's OK to use Themis and SwiftNIO in the same app, when BoringSSL package is dynamically linked. Will it be possible if Themis links BoringSSL as static lib?

---

TLDR:

a. I'm completely against cherry-picking openssl/boringssl code into Soter and maintaining it on our own. Due to numerous reasons: security, "do not implement your own crypto", maintainability and speed of fixes.

b. I'm not against shipping yet-another-Themis with statically linked BoringSSL package, but it should solve some real world use-cases. What use-case are we trying to solve here?

[ilammy]

libthemis-boringssl is probably going to be a thing starting with Themis 0.14. It is already implemented in #683, but it definitely needs more testing.