feat: initial commit introducing initial iteration of testing framework

README.md

@@ -1,14 +1,145 @@
-# Project
+# Azure Data Factory v2 - Unit Testing Framework
 
-> This repo has been populated by an initial template to help get you started. Please
-> make sure to update the content to build a great experience for community-building.
+A unit test framework that allows you to write unit and functional tests for Azure Data Factory v2 against the git integrated json resource files.
 
-As the maintainer of this project, please make a few updates:
+## Disclaimer
 
-- Improving this README.MD file to provide a great experience
-- Updating SUPPORT.MD with content about this project's support experience
-- Understanding the security reporting process in SECURITY.MD
-- Remove this section from the README
+This unit test framework is not officially supported. It is currently in experimental state and has not been tested with every single data factory resource. It should support all data factory resources, but has not been thoroughly tested, please report any issues in the issues section and include an example of the data factory pipeline that is not working as expected.
+
+If there's a lot of interest in this framework, then I will continue to improve it and move it to a production ready state. 
+
+## Features
+
+1. Evaluate the outcome of any data factory resource result given a set of input parameters. The framework will evaluate parameters, globalParameters, variables, activityOutputs and their expressions, so that the final result can be asserted.
+2. Simulate a pipeline run and evaluate the execution flow and outcome of each activity.
+3. Automatically parse any data factory resource into the correct typed class (1500+ classes available).
+4. Evaluate expressions, but not all functions are supported yet. You can always easily register your own custom functions.
+
+## Why
+
+Azure Data Factory does not support unit testing out of the box. The only way to validate your changes is through manual testing or running e2e tests against a deployed data factory. These tests are great to have, but miss the following benefits that unit tests, like using this unit test framework, provides:
+
+* Shift left with immediate feedback on changes - Evaluate any individual data factory resource (pipelines, activities, triggers, datasets, linkedServices etc..), including (complex) expressions
+* Allows testing individual resources (e.g. activity) for many different input values to cover more scenarios.
+* Less issues in production - due to the fast nature of writing and running unit tests, you will write more tests in less time and therefore have a higher test coverage. This means more confidence in new changes, less risks in breaking existing features (regression tests) and thus far less issues in production.
+
+> Even though Azure Data Factory is a UI-driven tool and writing unit tests might not be in the nature of it. How can you be confident that your changes will work as expected, and existing pipelines will not break, without writing unit tests?
+
+## Features - Examples
+
+The samples seen below is the _only_ code that you need to write! The framework will take care of the rest. 
+
+1. Evaluate expressions
+
+    ```csharp
+        // Arrange 
+        var expression = FunctionPart.Parse("concat('https://example.com/jobs/', '123', concat('&', 'abc'))");
+
+        // Act
+        var evaluated = expression.Evaluate();
+
+        // Assert
+        Assert.Equal("https://example.com/jobs/123&abc", evaluated);
+    ``` 
+
+2. Evaluate activities (e.g. a WebActivity that calls Azure Batch API), LinkedServices, Datasets and Triggers
+
+    ```csharp
+      // Arrange
+      var pipeline = PipelineFactory.ParseFromFile("Example/example-pipeline.json");
+      var activity = pipeline.GetActivityByName("Trigger Azure Batch Job") as WebHookActivity;
+
+      _state.Parameters.Add(new RunParameter(ParameterType.Global, "BaseUrl", "https://example.com"));
+      _state.Parameters.Add(new RunParameter(ParameterType.Parameter, "JobId", "123"));
+      _state.Variables.Add(new PipelineRunVariable("JobName", "Job-123"));
+      _state.AddActivityResult(new TestActivityResult("Get version", new
+      {
+        Version = "version1"
+      }));
+
+      // Act
+      activity.Evaluate(_state);
+
+      // Assert
+      Assert.Equal("https://example.com/jobs", activity.Uri);
+      Assert.Equal("POST", activity.Method);
+      Assert.Equal("{ \n    \"JobId\": \"123\",\n    \"JobName\": \"Job-123\",\n    \"Version\": \"version1\",\n}", activity.Body);
+
+    ```
+
+3. Evaluate Pipelines and test the flow of activities given a specific input
+
+    ```csharp
+    var pipeline = PipelineFactory.ParseFromFile(pipelineFileName);
+    Assert.Equal("example-pipeline", pipeline.Name);
+    Assert.Equal(6, pipeline.Activities.Count);
+
+    // Runs the pipeline with the provided parameters
+    var activities = pipeline.EvaluateWithActivityEnumerator(new List<RunParameter>
+        {
+            new(ParameterType.Parameter, "JobId", "123"),
+            new(ParameterType.Parameter, "ContainerName", "test-container"),
+            new(ParameterType.Global, "BaseUrl", "https://example.com"),
+        });
+
+    var setVariableActivity = activities.GetNext() as SetVariableActivity;
+    Assert.NotNull(setVariableActivity);
+    Assert.Equal("Set JobName", setVariableActivity.Name);
+    Assert.Equal("JobName", setVariableActivity.VariableName);
+    Assert.Equal("Job-123", setVariableActivity.Value);
+
+    var getVersionActivity = activities.GetNext() as WebActivity;
+    Assert.NotNull(getVersionActivity);
+    Assert.Equal("Get version", getVersionActivity.Name);
+    Assert.Equal("https://example.com/version", getVersionActivity.Uri);
+    Assert.Equal("GET", getVersionActivity.Method);
+    Assert.Null(getVersionActivity.Body);
+    getVersionActivity.SetResult(DependencyCondition.Succeeded, new {
+        Version = "version1"
+    });
+
+    var createBatchActivity = activities.GetNext() as WebHookActivity;
+    Assert.NotNull(createBatchActivity);
+    Assert.Equal("Trigger Azure Batch Job", createBatchActivity.Name);
+    Assert.Equal("https://example.com/jobs", createBatchActivity.Uri);
+    Assert.Equal("POST", createBatchActivity.Method);
+    Assert.Equal("{ \n    \"JobId\": \"123\",\n    \"JobName\": \"Job-123\",\n    \"Version\": \"version1\",\n}", createBatchActivity.Body);
+    createBatchActivity.SetResult(DependencyCondition.Succeeded, "OK");
+
+    Assert.Throws<ActivityEnumeratorException>(() => activities.GetNext());
+    ```
+
+> See AzureDataFactory.TestingFramework.Example project for more samples
+
+## Registering missing expression functions
+
+As the framework is interpreting expressions containing functions, these functions need to be implemented in C#. The goal is to start supporting more and more functions, but if a function is not supported, then the following code can be used to register a missing function:
+
+```csharp
+   FunctionsRepository.Register("concat", (IEnumerable<object> arguments) => string.Concat(arguments));
+   FunctionsRepository.Register("trim", (string text, string trimArgument) => text.Trim(trimArgument[0]));
+``` 
+
+On runtime when evaluating expressions, the framework will try to find a matching function and assert the expected amount of arguments are supplied. If no matching function is found, then an exception will be thrown.
+
+> Feel free to add a pull request with your own custom functions, so that they can be added to the framework and enjoyed by everyone.
+
+## Tips
+
+1. After parsing a data factory resource file, you can use the debugger to easily discover which classes are actually initialized so that you can cast them to the correct type.
+
+## Recommended development workflow
+
+* Use ADF Git integration
+* Use UI to create feature branch, build initial pipeline and save to feature branch
+* Pull feature branch locally
+* Start writing tests unit and functional tests, run them locally for immediate feedback and fix bugs
+* Push changes to feature branch
+* Test the new features manually through the UI in sandbox environment
+* Create PR, which will run the tests in the CI pipeline
+* Approve PR
+* Merge to main and start deploying to dev/test/prd environments
+* Run e2e tests after each deployment to validate all happy flows work on that specific environment
 
 ## Contributing
 
@@ -26,8 +157,12 @@ contact [opencode@microsoft.com](mailto:opencode@microsoft.com) with any additio
 
 ## Trademarks
 
-This project may contain trademarks or logos for projects, products, or services. Authorized use of Microsoft 
-trademarks or logos is subject to and must follow 
+This project may contain trademarks or logos for projects, products, or services. Authorized use of Microsoft
+trademarks or logos is subject to and must follow
 [Microsoft's Trademark & Brand Guidelines](https://www.microsoft.com/en-us/legal/intellectualproperty/trademarks/usage/general).
 Use of Microsoft trademarks or logos in modified versions of this project must not cause confusion or imply Microsoft sponsorship.
 Any use of third-party trademarks or logos are subject to those third-party's policies.
+
+
+
+

SUPPORT.md

@@ -1,13 +1,3 @@
-# TODO: The maintainer of this repo has not yet edited this file
-
-**REPO OWNER**: Do you want Customer Service & Support (CSS) support for this product/project?
-
-- **No CSS support:** Fill out this template with information about how to file issues and get help.
-- **Yes CSS support:** Fill out an intake form at [aka.ms/onboardsupport](https://aka.ms/onboardsupport). CSS will work with/help you to determine next steps.
-- **Not sure?** Fill out an intake as though the answer were "Yes". CSS will help you decide.
-
-*Then remove this first heading from this SUPPORT.MD file before publishing your repo.*
-
 # Support
 
 ## How to file issues and get help  
@@ -16,9 +6,7 @@ This project uses GitHub Issues to track bugs and feature requests. Please searc
 issues before filing new issues to avoid duplicates.  For new issues, file your bug or 
 feature request as a new Issue.
 
-For help and questions about using this project, please **REPO MAINTAINER: INSERT INSTRUCTIONS HERE 
-FOR HOW TO ENGAGE REPO OWNERS OR COMMUNITY FOR HELP. COULD BE A STACK OVERFLOW TAG OR OTHER
-CHANNEL. WHERE WILL YOU HELP PEOPLE?**.
+For help and questions about using this project, feel free ask your question via GitHub issues. In the future we will think about other channels.
 
 ## Microsoft Support Policy  
 

src/.editorconfig

@@ -0,0 +1 @@
+dotnet_naming_rule.public_members_must_be_capitalized.severity = suggestion

src/AzureDataFactory.TestingFramework.Example/AzureDataFactory.TestingFramework.Example.csproj

@@ -0,0 +1,35 @@
+<Project Sdk="Microsoft.NET.Sdk">
+
+    <PropertyGroup>
+        <TargetFramework>net7.0</TargetFramework>
+        <ImplicitUsings>enable</ImplicitUsings>
+        <Nullable>enable</Nullable>
+
+        <IsPackable>false</IsPackable>
+        <IsTestProject>true</IsTestProject>
+    </PropertyGroup>
+
+    <ItemGroup>
+        <PackageReference Include="Microsoft.NET.Test.Sdk" Version="17.5.0" />
+        <PackageReference Include="xunit" Version="2.4.2" />
+        <PackageReference Include="xunit.runner.visualstudio" Version="2.4.5">
+            <IncludeAssets>runtime; build; native; contentfiles; analyzers; buildtransitive</IncludeAssets>
+            <PrivateAssets>all</PrivateAssets>
+        </PackageReference>
+        <PackageReference Include="coverlet.collector" Version="3.2.0">
+            <IncludeAssets>runtime; build; native; contentfiles; analyzers; buildtransitive</IncludeAssets>
+            <PrivateAssets>all</PrivateAssets>
+        </PackageReference>
+    </ItemGroup>
+
+    <ItemGroup>
+      <None Update="BatchJob\pipeline.json">
+        <CopyToOutputDirectory>PreserveNewest</CopyToOutputDirectory>
+      </None>
+    </ItemGroup>
+
+    <ItemGroup>
+      <ProjectReference Include="..\AzureDataFactory.TestingFramework\AzureDataFactory.TestingFramework.csproj" />
+    </ItemGroup>
+
+</Project>