Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Inline the binary protos #25

Merged
merged 3 commits into from
Apr 26, 2024
Merged

Conversation

jchaffraix-slack
Copy link
Collaborator

protoc-gen-hack/plugin.go was generating the gzipped binary ProtoDescriptorProto and returning it to protoc so we could save it to a file. protoc was warning us about having those file's content being invalid UTF-8.

Unfortunately newer protoc binary reject files with such a ayload. We do want to keep the binary FileDescriptorProto logic for servers that can send the binary descriptor to their clients but we can inline it in each generated file.

While we don't need to gzip the content anymore, I kept this logic as we were generating large binary strings in Hacklang.

@@ -101,6 +101,7 @@ public function Name(): string {
}

public function FileDescriptorProtoBytes(): string {
return (string)\gzuncompress(\file_get_contents(\realpath(\dirname(__FILE__)) . '/any_file_descriptor.pb.bin.gz'));
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Anything in the generated/ directory is autogenerated through:

bazel run //:gen

The diffs are:

  1. All the files ending in .pb.bin.gz were removed.
  2. We inlined their content in the associated proto.php.

Comment on lines -235 to -238
// The protoc compiler will complain that this contains invalid UTF-8 data,
// but we will ignore that for now.
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is the reason we need this PR: the complain is turning into an error in newer version of the protoc.

binaryProtoStr := ""
for i := 0; i < len(binaryProto); i++ {
c := binaryProto[i]
binaryProtoStr += fmt.Sprintf("\\x%x", c)
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

%x prints a single byte in hexadecimal (perfect for the \x?? format in Hacklang)

protoc-gen-hack/plugin.go was generating the gzipped binary
ProtoDescriptorProto and returning it to protoc so we could
save it to a file. protoc was warning us about having those
file's content being invalid UTF-8.

Unfortunately newer protoc binary reject files with such a
ayload. We do want to keep the binary FileDescriptorProto
logic for servers that can send the binary descriptor to
their clients but we can inline it in each generated file.

While we don't need to gzip the content anymore, I kept
this logic as we were generating large binary strings
in Hacklang.
Copy link

@junoatwork junoatwork left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the inline comments!

@jchaffraix-slack jchaffraix-slack merged commit 864f0fb into master Apr 26, 2024
2 checks passed
@jchaffraix-slack jchaffraix-slack deleted the julien_inline_binary_proto branch April 26, 2024 12:13
jchaffraix-slack added a commit that referenced this pull request Apr 26, 2024
Inlining the protobufs in
#25
unblocked migrating to newer version of Protobuf
as the proto compiler (protoc) error'd.
jchaffraix-slack added a commit that referenced this pull request Apr 26, 2024
Inlining the protobufs in
#25
unblocked migrating to newer version of Protobuf
as the proto compiler (protoc) error'd.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants