Improve error reporting and add support for older files

[tamasgal]

This PR is a bit crowded but will eventually solve a few error reporting problems and hopefully issues like #235 with old versions of ROOT classes (TTree 5, TBranch 8, ...)

[codecov]

Codecov Report

Patch coverage: 41.57% and project coverage change: -1.72 ⚠️

Comparison is base (5f9ae4f) 89.35% compared to head (eb9a0f9) 87.64%.

Additional details and impacted files

@@            Coverage Diff             @@
##           master     #236      +/-   ##
==========================================
- Coverage   89.35%   87.64%   -1.72%     
==========================================
  Files          18       18              
  Lines        2104     2266     +162     
==========================================
+ Hits         1880     1986     +106     
- Misses        224      280      +56     

Impacted Files	Coverage Δ
src/bootstrap.jl	86.78% <38.75%> (-9.84%)	⬇️
src/root.jl	94.30% <50.00%> (-0.53%)	⬇️
src/streamers.jl	90.13% <80.00%> (+0.78%)	⬆️

... and 14 files with indirect coverage changes

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

☔ View full report in Codecov by Sentry.
📢 Do you have feedback about the report comment? Let us know in this issue.

[tamasgal]

OK, after debugging the whole day, I got stuck at a runaway pointer. I need a break... ;)

julia> using UnROOT

julia> f = ROOTFile("/Users/tamasgal/Downloads/gears.root")
cname = "TStreamerInfo"
version = 1
(start, Const.kMapOffset) = (71, 2)
cname = "TObjArray"
version = 1
(start, Const.kMapOffset) = (132, 2)
cname = "TStreamerBasicType"
version = 1
(start, Const.kMapOffset) = (175, 2)
cname = "TStreamerBase"
version = 1
(start, Const.kMapOffset) = (495, 2)
cname = "TStreamerString"
version = 1
(start, Const.kMapOffset) = (616, 2)
cname = "TStreamerObjectPointer"
version = 1
(start, Const.kMapOffset) = (1287, 2)
cname = "TStreamerBasicPointer"
version = 1
(start, Const.kMapOffset) = (4963, 2)
cname = "TStreamerSTL"
version = 1
(start, Const.kMapOffset) = (5844, 2)
cname = "TStreamerObject"
version = 1
(start, Const.kMapOffset) = (11891, 2)
cname = "TStreamerObjectAny"
version = 1
(start, Const.kMapOffset) = (12150, 2)
ROOTFile with 1 entry and 53 streamers.
/Users/tamasgal/Downloads/gears.root
cname = "TBranch"
version = 1
(start, Const.kMapOffset) = (221, 2)
cname = "TLeafI"
version = 1
(start, Const.kMapOffset) = (379, 2)
cname = "TBasket"
version = 1
(start, Const.kMapOffset) = (501, 2)
RecoveredTBasket
io = IOBuffer(data=UInt8[...], readable=true, writable=false, seekable=true, append=false, size=349965, maxsize=Inf, ptr=461, mark=-1)
fNbytes = 65
fNbytes = 16121


julia> f["t/n"]
cname = "TBranch"
version = 1
(start, Const.kMapOffset) = (221, 2)
cname = "TLeafI"
version = 1
(start, Const.kMapOffset) = (379, 2)
cname = "TBasket"
version = 1
(start, Const.kMapOffset) = (501, 2)
RecoveredTBasket
io = IOBuffer(data=UInt8[...], readable=true, writable=false, seekable=true, append=false, size=349965, maxsize=Inf, ptr=461, mark=-1)
fNbytes = 65
fNbytes = 16121
ERROR: Invalid class reference.
Stacktrace:
  [1] error(s::String)
    @ Base ./error.jl:35
  [2] readobjany!(io::IOBuffer, tkey::UnROOT.TKey32, refs::Dict{Int32, Any})
    @ UnROOT ~/Dev/UnROOT.jl/src/streamers.jl:265
  [3] unpack(io::IOBuffer, tkey::UnROOT.TKey32, refs::Dict{Int32, Any}, T::Type{UnROOT.TObjArray})
    @ UnROOT ~/Dev/UnROOT.jl/src/streamers.jl:333
  [4] UnROOT.TTree(io::UnROOT.MmapStream, tkey::UnROOT.TKey32, refs::Dict{Int32, Any}; top::Bool)
    @ UnROOT ~/Dev/UnROOT.jl/src/bootstrap.jl:1082
  [5] UnROOT.TTree(io::UnROOT.MmapStream, tkey::UnROOT.TKey32, refs::Dict{Int32, Any})
    @ UnROOT ~/Dev/UnROOT.jl/src/bootstrap.jl:1048
  [6] macro expansion
    @ ~/Dev/UnROOT.jl/src/root.jl:172 [inlined]
  [7] (::UnROOT.var"##getter#313#148"{ROOTFile, SubString{String}})()
    @ UnROOT ~/.julia/packages/Memoization/xgAKx/src/Memoization.jl:207
  [8] get!(default::UnROOT.var"##getter#313#148"{ROOTFile, SubString{String}}, lru::LRUCache.LRU{Any, Any}, key::Tuple{Tuple{ROOTFile, SubString{String}}, Tuple{}})
    @ LRUCache ~/.julia/packages/LRUCache/8li9I/src/LRUCache.jl:116
  [9] _get!
    @ ~/.julia/packages/Memoization/xgAKx/src/Memoization.jl:214 [inlined]
 [10] _getindex(f::ROOTFile, s::SubString{String})
    @ UnROOT ~/.julia/packages/Memoization/xgAKx/src/Memoization.jl:209
 [11] getindex
    @ ~/Dev/UnROOT.jl/src/root.jl:157 [inlined]
 [12] macro expansion
    @ ~/Dev/UnROOT.jl/src/root.jl:165 [inlined]
 [13] (::UnROOT.var"##getter#313#148"{ROOTFile, String})()
    @ UnROOT ~/.julia/packages/Memoization/xgAKx/src/Memoization.jl:207
 [14] get!(default::UnROOT.var"##getter#313#148"{ROOTFile, String}, lru::LRUCache.LRU{Any, Any}, key::Tuple{Tuple{ROOTFile, String}, Tuple{}})
    @ LRUCache ~/.julia/packages/LRUCache/8li9I/src/LRUCache.jl:116
 [15] _get!
    @ ~/.julia/packages/Memoization/xgAKx/src/Memoization.jl:214 [inlined]
 [16] _getindex(f::ROOTFile, s::String)
    @ UnROOT ~/.julia/packages/Memoization/xgAKx/src/Memoization.jl:209
 [17] getindex(f::ROOTFile, s::String)
    @ UnROOT ~/Dev/UnROOT.jl/src/root.jl:157
 [18] top-level scope
    @ REPL[3]:1

[oschulz]

OK, after debugging the whole day, I got stuck at a runaway pointer. I need a break... ;)

Next time you're in Munich, you so are going to drink for free! :-)

[tamasgal]

😆 better wait until I solved it 🙈

[oschulz]

better wait until I solved it

I have faith :-)

[tamasgal]

Ouff, that's tough.. I probably need to reorganise a few things. Baskets in TTree is a bit "awkward"

[tamasgal]

Just encountered a specific type of ROOT files in KM3NeT which also have TBaskets in a TTree... the pressure is increasing 🙈

[oschulz]

Sometimes ROOT is just ... well ...

[tamasgal]

OK, I am still struggling. Something with our KM3NeT internal files is a bit different. I now have added a warning if TBaskets are discovered in a TTree so that the user knows what's going on.

I'll still merge this since 1. at least now we are able to read the type of KM3NeT ROOT files which are related to this and 2. the error reporting is definitely improved + the support for older files is also extended.

I'll open an other PR to work on the recovered TBaskets separately.

[Moelf]

btw does this help:

• ERROR: TBasket not defined #14 ?

[tamasgal]

Yep, together with scikit-hep/uproot3#401 (comment)

Thanks!