Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support Binary Serialization #47

Open
L1lith opened this issue Jun 14, 2022 · 0 comments
Open

Support Binary Serialization #47

L1lith opened this issue Jun 14, 2022 · 0 comments

Comments

@L1lith
Copy link
Owner

L1lith commented Jun 14, 2022

By piggybacking onto the API suggested here we could easily define a methodology for converting data into Binary. We could also take advantage of the type information in the format to create significant size reductions. For example we could account for custom string character sets (like only ASCII or a-z or lowercase etc) in order to represent the data in less bits. By doing this we could create a BSON like encoding that largely doesn't need any type information at all, and it can be optimized at compile time so we can create lightning fast binary serializers.

We could take a format here

[{_: Number, {min: 0, max: 16}]

Here we know at compile time we can represent this number using only 4 bits (2^4=16). The only dynamic part here would be the array length itself.

Before our encoder maybe it was in JSON which requires either a ton of Unicode (very wasteful) or BSON (pretty slow and can't optimize based on the data structure to my knowledge), or using a custom serializer which is generally very time consuming and error-prone.

@L1lith L1lith moved this to Todo in Sandhands 2.0 Jun 14, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
Status: Todo
Development

No branches or pull requests

1 participant