corrupted specification_local_data.json after update to 0.8.5

[kTelaar]

Hello everyone,

I recently tried the then new spinetoolbox 0.8.5 as ZIP bundle with an existing project.
After testing a bit I wanted to go back to my old config with spinetoolbox 0.7.0 but received an error message nearly identical to Manuel's old issue in 2022 #1808 while the Toolbox tried to load the project's specification endlessly without success:

$ spinetoolbox
Traceback (most recent call last):
  File "C:\spineProjects\git\steam\myenv\lib\site-packages\spinetoolbox\widgets\custom_menus.py", line 156, in <lambda>
    lambda checked=False, filepath=filepath: self.call_open_project(checked, filepath),
  File "C:\spineProjects\git\steam\myenv\lib\site-packages\spinetoolbox\widgets\custom_menus.py", line 190, in call_open_project
    if not self._parent.open_project(p):
  File "C:\spineProjects\git\steam\myenv\lib\site-packages\spinetoolbox\ui_main.py", line 575, in open_project
    return self.restore_project(load_dir)
  File "C:\spineProjects\git\steam\myenv\lib\site-packages\spinetoolbox\ui_main.py", line 606, in restore_project
    success = self._project.load(self._item_specification_factories, self.item_factories)
  File "C:\spineProjects\git\steam\myenv\lib\site-packages\spinetoolbox\project.py", line 309, in load
    spec = load_specification_from_file(
  File "C:\spineProjects\git\steam\myenv\lib\site-packages\spinetoolbox\helpers.py", line 1112, in load_specification_from_file
    spec = specification_from_dict(spec_dict, local_data_dict, spec_factories, app_settings, logger)
  File "C:\spineProjects\git\steam\myenv\lib\site-packages\spinetoolbox\helpers.py", line 1139, in specification_from_dict
    return spec_factory.make_specification(spec_dict, app_settings, logger)
  File "C:\spineProjects\git\steam\myenv\lib\site-packages\spine_items\tool\specification_factory.py", line 32, in make_specification
    return make_specification(definition, app_settings, logger)
  File "C:\spineProjects\git\steam\myenv\lib\site-packages\spine_items\tool\tool_specifications.py", line 75, in make_specification
    spec = JuliaTool.load(path, definition, app_settings, logger)
  File "C:\spineProjects\git\steam\myenv\lib\site-packages\spine_items\tool\tool_specifications.py", line 454, in load
    return JuliaTool(path=path, settings=settings, logger=logger, **kwargs)
TypeError: __init__() got an unexpected keyword argument 'execution_settings'

After some insightful discussions with @tokorte we found this old discussion indicating that some files are ignored by default looking suspiciously similar to our issue: #1734
After manually deleting all the entries for the julia items in the specification_local_data.json in the .spinetoolbox/local folder we were able to load the project successfully again in 0.7.0.

This was a bit of a struggle because we were not aware of the .gitignore default settings in the .spinetoolbox folder so git was never able to revert really ALL the changes that were done in the process of updating the projects files to 0.8.5 while we thought that we did revert ALL the changes.

few ideas for making it easier to spot this issue:

• remove the gitignore setting of the local folder (which may be not desireable)
• make it clear that deleting the non-tracked file specification_local_data.json can resolve this issue (e.g. via a error message print)
• inform the user what upgrading the project means and what they might be missing: not only changes to the project.json and the .sqlite-DB's but also the specification_local_data.json

Cheers from Bochum
Konrad

[soininen]

We backup the project.json on update. We should perhaps backup the files in the .spinetoolbox/local/ folder as well. We could also add a 'Troubleshooting' section to User guide that instructs users about the existence of the backups.

If you project is not huge, you can always just duplicate the project directory for testing new Toolbox versions - this way you keep you original intact.

remove the gitignore setting of the local folder

The files in local/ may contain database credentials - it is not recommended to allow these files to get committed to any repository.

[kTelaar]

We should perhaps backup the files in the .spinetoolbox/local/ folder as well. We could also add a 'Troubleshooting' section to User guide that instructs users about the existence of the backups.

That sounds good.

duplicate the project directory for testing new Toolbox versions

will do it in the future. I was just under the false impression that all project related files were getting committed so I thought I can easily revert these changes. Now I know better and maybe when someone makes the same mistake, they will find this thread and know how to solve it.

What about adding a new pop-up that informs the user which files will be upgraded?
You already added this for the sqlite-DB's and the project.json. The only thing missing seems to be the specification_local_data.json. It would have helped me tremendously to know where exactly I should be looking to resolve the issues.