Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

GH-43946: [C++][Parquet] Guard against use of cleared decryptor/encryptor #43947

Merged
merged 2 commits into from
Sep 5, 2024
Merged

GH-43946: [C++][Parquet] Guard against use of cleared decryptor/encryptor #43947

merged 2 commits into from
Sep 5, 2024

Conversation

pitrou
Copy link
Member

@pitrou pitrou commented Sep 4, 2024 •
edited
Loading

This is to get a clearer error rather than an obscure crash, see #43057 for an example.

@pitrou pitrou requested a review from wgtmac as a code owner September 4, 2024 09:38
@github-actions GitHub Actions
Copy link

github-actions bot commented Sep 4, 2024

⚠️ GitHub issue #43946 has been automatically assigned in GitHub to PR creator.

@pitrou pitrou changed the title GH-43946: [C++][Parquet] Guard against use of decryptor/encryptor after wipeout GH-43946: [C++][Parquet] Guard against use of cleared decryptor/encryptor Sep 4, 2024
@pitrou
Copy link
Member Author

pitrou commented Sep 4, 2024

@github-actions crossbow submit -g cpp

@github-actions GitHub Actions

This comment was marked as outdated.

Sign in to view
@pitrou
Copy link
Member Author

pitrou commented Sep 4, 2024

@adamreeve Does this look good to you?

@pitrou
Copy link
Member Author

pitrou commented Sep 4, 2024

We have a nice example of the guard being triggered here:
https://github.com/ursacomputing/crossbow/actions/runs/10699204038/job/29660230356#step:7:3575

This was a sporadic crash before:
#43057

@github-actions github-actions bot added awaiting committer review Awaiting committer review and removed awaiting review Awaiting review labels Sep 4, 2024
@adamreeve
Copy link
Contributor

This looks good to me, impressive work tracking down the cause of the segfault!

@wgtmac
Copy link
Member

wgtmac commented Sep 5, 2024

We have a nice example of the guard being triggered here: https://github.com/ursacomputing/crossbow/actions/runs/10699204038/job/29660230356#step:7:3575

This was a sporadic crash before: #43057

Does it mean that DatasetEncryptionTest.WriteReadDatasetWithEncryption test case does not run in the regular CI? Or is it nondeterministic when the decryptor is cleared?

wgtmac
wgtmac approved these changes Sep 5, 2024
View reviewed changes
cpp/src/parquet/encryption/encryption_internal.cc
@@ -413,6 +423,12 @@ class AesDecryptor::AesDecryptorImpl {
}

private:
void CheckValid() {
if (ctx_ == nullptr) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
if (ctx_ == nullptr) {
if (ARROW_PREDICT_FALSE(ctx_ == nullptr)) {

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I was under the impression that compilers automatically treated branches that throw an exception to be unlikely, but I can't find a reference. @felipecrv Do you know about that?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok, judging from https://www.youtube.com/watch?v=T84swS6DCRo (at 29:00), the gcc compiler already uses such a heuristic.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

CheckValid should be const?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good point @mapleFU

cpp/src/parquet/encryption/encryption_internal.cc
@@ -89,6 +89,12 @@ class AesEncryptor::AesEncryptorImpl {
}

private:
void CheckValid() {
if (ctx_ == nullptr) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
if (ctx_ == nullptr) {
if (ARROW_PREDICT_FALSE(ctx_ == nullptr)) {

@pitrou
Copy link
Member Author

pitrou commented Sep 5, 2024

Does it mean that DatasetEncryptionTest.WriteReadDatasetWithEncryption test case does not run in the regular CI? Or is it nondeterministic when the decryptor is cleared?

It seems to be non-deterministic in this particular test, as the dataset reading is multi-threaded (you can see this in the stack traces above).

@pitrou
Copy link
Member Author

pitrou commented Sep 5, 2024

@github-actions crossbow submit -g cpp

@github-actions GitHub Actions
Copy link

github-actions bot commented Sep 5, 2024

Revision: 0fb9ee6

Submitted crossbow builds: ursacomputing/crossbow @ actions-0c32dece8e

Task Status
test-alpine-linux-cpp GitHub Actions
test-build-cpp-fuzz GitHub Actions
test-conda-cpp GitHub Actions
test-conda-cpp-valgrind GitHub Actions
test-cuda-cpp GitHub Actions
test-debian-12-cpp-amd64 GitHub Actions
test-debian-12-cpp-i386 GitHub Actions
test-fedora-39-cpp GitHub Actions
test-ubuntu-20.04-cpp GitHub Actions
test-ubuntu-20.04-cpp-bundled GitHub Actions
test-ubuntu-20.04-cpp-minimal-with-formats GitHub Actions
test-ubuntu-20.04-cpp-thread-sanitizer GitHub Actions
test-ubuntu-22.04-cpp GitHub Actions
test-ubuntu-22.04-cpp-20 GitHub Actions
test-ubuntu-22.04-cpp-emscripten GitHub Actions
test-ubuntu-22.04-cpp-no-threading GitHub Actions
test-ubuntu-24.04-cpp GitHub Actions
test-ubuntu-24.04-cpp-gcc-13-bundled GitHub Actions
test-ubuntu-24.04-cpp-gcc-14 GitHub Actions

@pitrou pitrou merged commit 032e6a4 into apache:main Sep 5, 2024
34 of 35 checks passed
@pitrou pitrou removed the awaiting committer review Awaiting committer review label Sep 5, 2024
@pitrou pitrou mentioned this pull request Sep 5, 2024
Closed
@pitrou pitrou deleted the gh43946-decryptor-wipeout branch September 5, 2024 09:47
@conbench-apache-arrow conbench-apache-arrow
Copy link

After merging your PR, Conbench analyzed the 4 benchmarking runs that have been run so far on merge-commit 032e6a4.

There were no benchmark performance regressions. 🎉

The full Conbench report has more details. It also includes information about 3 possible false positives for unstable benchmarks that are known to sometimes produce them.

zanmato1984 pushed a commit to zanmato1984/arrow that referenced this pull request Sep 6, 2024
@pitrou @zanmato1984
apacheGH-43946: [C++][Parquet] Guard against use of cleared decryptor…
967ab83 
…/encryptor (apache#43947)

This is to get a clearer error rather than an obscure crash, see apache#43057 for an example.

* GitHub Issue: apache#43946

Authored-by: Antoine Pitrou <antoine@python.org>
Signed-off-by: Antoine Pitrou <antoine@python.org>
khwilson pushed a commit to khwilson/arrow that referenced this pull request Sep 14, 2024
@pitrou @khwilson
apacheGH-43946: [C++][Parquet] Guard against use of cleared decryptor…
981b841 
…/encryptor (apache#43947)

This is to get a clearer error rather than an obscure crash, see apache#43057 for an example.

* GitHub Issue: apache#43946

Authored-by: Antoine Pitrou <antoine@python.org>
Signed-off-by: Antoine Pitrou <antoine@python.org>
@EnricoMi EnricoMi mentioned this pull request Nov 20, 2024
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@mapleFU mapleFU mapleFU left review comments

@wgtmac wgtmac wgtmac approved these changes

@adamreeve adamreeve adamreeve approved these changes

@felipecrv felipecrv Awaiting requested review from felipecrv

Assignees
No one assigned
Labels
Component: C++ Component: Parquet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
4 participants
@pitrou @adamreeve @wgtmac @mapleFU