Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Adds type hints throughout Hatchet and integrate checks with mypy #153

Open
wants to merge 5 commits into
base: develop
Choose a base branch
from

Conversation

ilumsden
Copy link
Collaborator

@ilumsden ilumsden commented Nov 14, 2024

As several of us (particularly myself and @pearce8) have mentioned on numerous occasions, Python's duck typing system can be a pain to deal with.

To help with this, this PR adds extensive type hinting throughout Hatchet to help with understanding what is valid input/output for functions. Additionally, this PR adds support for mypy: the most widely used static type checker for Python. mypy makes use of the type hints added in this PR to determine whether code is written correctly, similar to how a compiler works for languages like C and C++.

Besides type checking, this PR corrects numerous bugs that mypy revealed, ranging from minor typing issues due to reusing variables too much to functions not returning when they are supposed to (this one may actually be part of the cause of the squash issues you were having @slabasan). One important set of things that mypy pointed out is logic for Python 2. Since we are now officially supporting Python 3.5+, this logic is no longer needed, so I have removed any such logic that mypy detected.

Finally, this PR adds mypy-based static type checking to our CI. Like flake8 and black, this will only run for Python 3.9.

Edit:

I got fed up with Black giving different formatting with different versions, so I added a devcontainer to Hatchet. This lets developers who user VSCode work in a Docker-based environment with the exact right versions of tools like black, flake8, and mypy.

@ilumsden ilumsden added area-ci Issues and PRs related to continuous integration processes used by Hatchet developers area-external Issues and PRs related to external libraries used by Hatchet area-graphframe Issues and PRs involving Hatchet's core GraphFrame datastructure and associated classes area-query-lang Issues and PRs related to Hatchet's query language area-readers Issues and PRs involving Hatchet's data readers area-utils Issues and PRs related to Hatchets high-level API and other utility libraries area-writers Issues and PRs involving Hatchet's data writers priority-normal Normal priority issues and PRs status-work-in-progress PR is currently being worked on type-internal-cleanup PR or issues related to the structure of the codebase, directories and refactors labels Nov 14, 2024
@ilumsden ilumsden self-assigned this Nov 14, 2024
@ilumsden
Copy link
Collaborator Author

Also, as a small add-on to this PR, I've updated the .gitignore using the template from GitHub: https://github.com/github/gitignore/blob/main/Python.gitignore

@ilumsden ilumsden closed this Nov 14, 2024
@ilumsden ilumsden reopened this Nov 14, 2024
@ilumsden ilumsden added status-ready-for-review This PR is ready to be reviewed by assigned reviewers and removed status-work-in-progress PR is currently being worked on labels Nov 14, 2024
@ilumsden ilumsden marked this pull request as ready for review November 14, 2024 21:35
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area-ci Issues and PRs related to continuous integration processes used by Hatchet developers area-external Issues and PRs related to external libraries used by Hatchet area-graphframe Issues and PRs involving Hatchet's core GraphFrame datastructure and associated classes area-query-lang Issues and PRs related to Hatchet's query language area-readers Issues and PRs involving Hatchet's data readers area-utils Issues and PRs related to Hatchets high-level API and other utility libraries area-writers Issues and PRs involving Hatchet's data writers priority-normal Normal priority issues and PRs status-ready-for-review This PR is ready to be reviewed by assigned reviewers type-internal-cleanup PR or issues related to the structure of the codebase, directories and refactors
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant