Skip to content

Scribe-Data 3.0.0

Compare
Choose a tag to compare
@andrewtavis andrewtavis released this 19 Apr 00:35
· 1525 commits to main since this release

✨ Features

  • Scribe-Data now has the ability to generate SQLite databases from formatted language data.
    • data_to_sqlite.py is used to read available JSON files and input their information into the databases.
  • These databases are now sent to Scribe apps via defined paths.
    • send_dbs_to_scribe.py finds all available language databases and copies them.
    • Separating this step from the data update is in preparation for data import in the future where this will be an individual step.
  • Scribe-Data now also creates autocomplete lexicons for each language within data_to_sqlite.py.
  • JSON data is no longer able to be uploaded to Scribe app directories directly, with the SQLite directories now being exported instead.
  • Emojis of singular nouns are now also linked to their plural counterparts if the plural isn't present in the emoji keyword outputs.
  • The emoji process also now updates a column to the data_table.txt file for sharing on readmes with update_data.py maintaining it in the data update process.

♻️ Code Refactoring

  • The Jupyter notebooks for autosuggestions and emojis as well as update_data.py were moved to the extract_transform directory given that they're not used to load data anymore.
    • Their code was refactored to reflect their new locations.
  • Massive amounts of refactoring happened to achieve the shift in the data export method:
    • format_WORD_TYPE.py files export to a formatted_data directory within extract_transform.
    • Copies of all data JSONs that were originally in Scribe apps are now in the formatted_data directories.
    • Functions in update_utils.py were switched given that data is no longer uploaded into a Data directory within the language keyboard directories within Scribe apps.
    • Lots of functions and variables were renamed to make them more understandable.
    • Code to derive appropriate export locations within format_WORD_TYPE.py files was removed in favor of a language formatted_data directory.
    • regex was added as a dependency.
    • pylint comments were removed.
  • Verb SPARQL query scripts for Spanish and Italian were simplified to remove unneeded repeat conditions.

🐞 Bug Fixes

  • The statements in translation files have been fixed as they were improperly defined after a file was moved.