Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

feat: testing framework in python and microsoft fabric support with expression grammar and evaluation support #30

Merged
merged 131 commits into from
Dec 8, 2023
Merged

feat: testing framework in python and microsoft fabric support with expression grammar and evaluation support #30

merged 131 commits into from
Dec 8, 2023

Conversation

arjendev
Copy link
Collaborator

@arjendev arjendev commented Nov 15, 2023 •
edited
Loading

This PR transforms our framework from dotnet to python with an insane amount of new features like:

  1. Support for Fabric and Azure Data Factory with a unified model to support any activity and be resilient for future changes in activities
  2. Expression parsing with lark-parser with our own grammar definition and evaluation
  3. 100% expression function call coverage with a few missing cases around xml / date formats)
  4. All iteration activities are implemented (e.g. switch, foreach, filter, set-variable etc..)
  5. Child activity evaluation and pipelineReturnValue support

And a lot more...

arjendev and others added 30 commits November 8, 2023 10:37
@arjendev
feat: introduce initial python code base and patching script to fix a…
354b349 
…zure-sdk datafactory models
@arjendev
fix: fix imports of scripts
82058f9
@arjendev
chore: migrate to pytest
3a1a016
@arjendev
feat: model and data factory repositories, dynamically patching gener…
149d995 
…ated models
@arjendev
feat: testframework pipeline evaluation logic without iteration activ…
ea7a209 
…ities
@arjendev
chore: move examples to example folder
29c9ff6
@arjendev
feat: foreach support and refactor model patching
314a1f4
@arjendev
refactor: automatically patching of models
bbc159a
@arjendev
feat: if condition activity
d731665
@arjendev
feat: until activity
d85fa3a
@arjendev
feat: setVariable activity
036c370
@arjendev
feat: pipeline activity
4a5f63f
@arjendev
feat: function_call, function_argument and expression_parsing
a01458d
@arjendev
feat: function argument expression evaluation
9b2b2f3
@arjendev
feat: function argument activity output evaluation
e83afbb
@arjendev
feat: function_call evaluation so far
386c7b6
@arjendev
feat: function_call evaluation, optional parameters, apply leonards f…
f3e8c47 
…eedback and bunch of improvements
@arjendev
test: add function_call tests
5fe72b0
@arjendev
fix: function_call error argument naming
853db41
@LeonardHd
chore(python): add ruff linting
b8ac613
@LeonardHd
style(python): apply ruff auto fixes
30c7178
@LeonardHd
style(python): apply ruff unsafe auto fixes
b8ddb34
@LeonardHd
style(python): ruff fix E501 and ruff formatting
d4eb7ac
@LeonardHd
style(python): ruff fixes ANN001,ANN201,ANN401
9e9354e
@LeonardHd
style(python): fix new line formatting
30c5f44
@LeonardHd
style(python): format test files
efa809a
@LeonardHd
style(python): ruff E741
85dc7c4
@LeonardHd
fix(python): import docstring parse
aa0921b
@LeonardHd
style(python): fix N806
3c3ab64
@LeonardHd
style(python): various ruff fixes on style
62d74e3
@arjendev arjendev self-assigned this Dec 1, 2023
@arjendev arjendev added the enhancement New feature or request label Dec 1, 2023
@arjendev arjendev added this to the 2023-12 milestone Dec 1, 2023
This was linked to issues Dec 1, 2023
Implement all functions in functions_repository in python version #31
Closed
Support system variables #6
Closed
Evaluate child pipelines #7
Closed
jayachithra
jayachithra previously approved these changes Dec 8, 2023
View reviewed changes
.github/workflows/ci-cd.yml Outdated Show resolved Hide resolved
@arjendev arjendev merged commit 44e3a5f into main Dec 8, 2023
2 checks passed
@arjendev arjendev linked an issue Dec 11, 2023 that may be closed by this pull request
Publish alpha release for pyPI #32
Closed
@arjendev arjendev deleted the feature/python-codebase branch December 24, 2023 13:20
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@jayachithra jayachithra jayachithra approved these changes

@sreedhar-guda sreedhar-guda Awaiting requested review from sreedhar-guda

@ydaponte ydaponte Awaiting requested review from ydaponte

Assignees

@arjendev arjendev

Labels
enhancement New feature or request
Projects
Azure Data Factory Testing Framework
Status: Done
Milestone
2023-12
3 participants
@arjendev @jayachithra @LeonardHd