Add support for reading several rntuple files #708
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
tmadlener
reviewed
Nov 19, 2024
include/podio/RNTupleReader.h
Outdated
Comment on lines
151
to
153
| // Map category to a vector that contains at how many entries each reader starts | ||
| // For example, if we have 3 readers and the first one has 10 entries, the second one 20 and the third one 30 | ||
| // then the vector will be {0, 10, 30, 60} |
There was a problem hiding this comment.
I am not sure I fully understand why we need to store the final sum of entries here? Is it because of the upper_bound below?
There was a problem hiding this comment.
It doesn't have to be stored (and it isn't) but since it has to be computed anyway I think the code looks cleaner like it is (computing it and storing it in the vector and later removing it) than having to check if it's the last entry every time and not add it in that case. For the comment I can remove the last number
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
BEGINRELEASENOTES
ENDRELEASENOTES
Now it will work correctly for multiple files. For TTrees reading multiple files with random access (needed because we can read any entry) is provided through
TChainandTBranch. ForRNTuplesthere is not such thing, there is aRNTupleProcessorbut it is intended for iterative access and not random access:https://root.cern.ch/doc/master/classROOT_1_1Experimental_1_1RNTupleProcessor.html
So the idea is to save a vector with how many entries each file has, and then every time an entry is read do a lookup in the vector to find out which file we are at and get the corresponding reader for that file. Scales as log N in the number of files, which is probably OK for most cases.
Requires C++20 for
std::ranges::upper_bound.