Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Over-Read request sent to origin when issued a pgRead #13

Open
djw8605 opened this issue Oct 25, 2024 · 0 comments
Open

Over-Read request sent to origin when issued a pgRead #13

djw8605 opened this issue Oct 25, 2024 · 0 comments

Comments

@djw8605
Copy link

djw8605 commented Oct 25, 2024

When XRootD sends a pgRead to the pelican plugin, the last block may request more bytes than the size of the file, such that offset + request_size > file_size. The built-in http plugin catches this case and restricts requests to the file_size maximum file size: XrdClHttpFilePlugin.cc

When the requested bytes range is larger than the size of the file, the xrootd origin will return the response headers, then close the connection without sending the data. This causes the curl library to return error code 18: transfer closed with outstanding read data remaining. I have replicated this error with multiple production pelican caches:

curl --output /dev/null http://osg-chicago-stashcache.nrp.internet2.edu:8443/icecube/PUBLIC/groups/arasoft/raytrace_timing_tables/arrivaltimes_station_100_icemodel_40_radius_300.00_angle_1.00_solution_0.root
curl --output /dev/null https://sdsc-cache.nationalresearchplatform.org:8443//icecube/PUBLIC/groups/arasoft/raytrace_timing_tables/arrivaltimes_station_100_icemodel_40_radius_300.00_angle_1.00_solution_0.root
curl --output /dev/null https://osg-sunnyvale-stashcache.nrp.internet2.edu:8443//icecube/PUBLIC/groups/arasoft/raytrace_timing_tables/arrivaltimes_station_100_icemodel_40_radius_300.00_angle_1.00_solution_0.root

The pelican platform should restrict the maximum size requested from the origin to the file size, similar to how the built-in http plugin.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant