Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Validate signature of input files #153

Open
gaow opened this issue Sep 7, 2018 · 0 comments
Open

Validate signature of input files #153

gaow opened this issue Sep 7, 2018 · 0 comments

Comments

@gaow
Copy link
Member

gaow commented Sep 7, 2018

It took @KaiqianZhang and I some time this morning to figure out a weird bug. Turns out Kaiqian has revised part of the input data, and not knowing that DSC does not track file status she did not remove cached output files. It apparently caused troubles.

Ordinary workflow systems do track input and output files explicitly. But in the context of DSC the files tracked are files of data flow, an abstracted concept users do not have to worry about. In other words we have not introduced the notion that users have to worry about any file status.

Tracking input script file status is easy and was done. We do not have mechanism tracking arbitrary input file status. Proposed solutions:

  1. Add something like path("/path/to/file") to explicitly mark what we want to track. But it is one additional parameter and more importantly a new concept we'll have to explain to users.
  2. For any string type of input I check if it is a file (if it exists on disk) and track its status if it does. A bit more work for the DSC parser but should not be too bad.

I'll go for solution 2. Any objections?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

1 participant