Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Minor build performance improvements #40

Merged
merged 2 commits into from
Oct 27, 2024
Merged

Minor build performance improvements #40

merged 2 commits into from
Oct 27, 2024

Conversation

zeux
Copy link
Owner

@zeux zeux commented Oct 27, 2024

  • Use memchr in skipByLines to optimize scanning
  • Avoid redundant file copy when the file is newly allocated

On Linux kernel source tree with 16 workers, this gets total wall clock time for qgrep build down from 1.36s to 1.21s, although timing is noisy; both operations are done on the main thread so this effectively reduces the serialization.

While there's some overhead for the function call, we end up getting
SIMD memchr for free. We could write our own SIMD function here but
memchr ends up being very cheap.
When buildAppendFilePart is called with chunk data as the source we need
to copy it as the lifetimes are decoupled; however for changed files,
we've already allocated a vector for it so it's a waste to copy it as we
can move instead.
@zeux zeux merged commit 8f8b127 into master Oct 27, 2024
8 checks passed
@zeux zeux deleted the perf branch October 27, 2024 17:02
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant