Trawl Data Storage Tidying

[adamkemberling]

Nearly every year we get a new "pull" of the groundfish data. Typically they have the same name, and you would expect the same structure, but that is not the case. In going through the catalog we have found inconsistencies within the data that were not planned, in addition to some pulls that specifically omit certain data. Currently the folder organization is confusing, and there isn't clarity on what is the most current, and how to send/share the data internally and externally.

[aallyn]

It would be great if we could get away from the email data passing to using existing functions and directly querying the trawl data database -- much like we did with the DFO data. Do we know if this is possible? I feel like I remember seeing some type of survdat package or suite of functions by someone at the science center -- might be something we could ask about on Openscapes slack channel? I don't think we want to be responsible for passing along the data (mostly thinking about if there are any issues with it, then we would need to be the ones notifying any downstream users about the issues). Though, we could still certainly pass along our code that work from the central database? This would also likely help clean up some of the "processing" scripts and we could do some work to make them more application specific (e.g., size analyses, no size group species, etc). I think we've already done a lot of that, too, both with the size spectrum analysis and the sdm workflow project.