Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Pull write #16

Open
wants to merge 143 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
143 commits
Select commit Hold shift + click to select a range
46d9c88
Take4
Oct 1, 2023
cb3b3a0
working action
Oct 9, 2023
407c205
new action
Oct 9, 2023
1103c81
changing a root directory for build
Oct 9, 2023
b7361a5
commit to testing system, instead of master
Oct 9, 2023
9ad8841
working directory - ./modelCsharp
Oct 9, 2023
dec0b36
Clean up redundant code
Oct 9, 2023
98d450e
check if it is still working
Oct 10, 2023
9c90eca
second try
Oct 10, 2023
357b8f0
third
Oct 10, 2023
9195f2a
Checking before start on config/fertil
Oct 11, 2023
9d49297
push to master
Oct 11, 2023
9aba6cd
no specified working directory
Oct 11, 2023
e7d6686
back to Testing system
Oct 11, 2023
76858ef
try on testing system
Oct 11, 2023
92f6228
testingSystem now
Oct 11, 2023
e0357fa
changed to master
Oct 11, 2023
3a07190
Action with Artifact
Oct 12, 2023
7df088a
ls folder
Oct 12, 2023
3e8257c
ls graphs
Oct 12, 2023
d79a487
mkdir graphs is moved above
Oct 12, 2023
d4909a7
path: ./modelCsharp/graphs
Oct 12, 2023
87486cc
Removing DLLs
Oct 31, 2023
0f4972e
Preview of graph in HTML
Oct 31, 2023
e571e38
Path is correct
Oct 31, 2023
41ab67e
Chanhed setting to gh-pages in SVS
Oct 31, 2023
6c0f296
Write permission exists
Oct 31, 2023
fbfa07c
permissions: contents: read,pages: write,id-token: write
Oct 31, 2023
97be7b1
permissions write-all
Oct 31, 2023
4d1f674
Sequence of jobs in action is changed
Oct 31, 2023
3d3ca1d
Added blank space just to rub GitHub page
Nov 10, 2023
0b2b34f
Fertiliser input.
Nov 12, 2023
c8d2d01
Whitespace added just to run action
Nov 12, 2023
c4f0652
First dates has been changed
Nov 12, 2023
3013f76
new format for dates/fertiliser
Nov 13, 2023
4f8e4a6
Python.exe is commented. Run on action now
Nov 13, 2023
c612d3f
Proper settings for Build are fixed
Nov 13, 2023
a9c44ac
No tests, looking for a csv file
Nov 13, 2023
63009a8
PYthon is commented out
Nov 13, 2023
ef2b54f
Find a csv files
Nov 13, 2023
2d8e7c0
find a csv file
Nov 13, 2023
1f10bd4
home directory
Nov 13, 2023
fac4acc
rooot diretory
Nov 13, 2023
f5b25d5
ls/OutputFiles
Nov 13, 2023
36d1819
ls OutputFiles
Nov 13, 2023
0f8f646
csv files are saved in folder
Nov 13, 2023
9ccb6a4
list folder - OutputFiles
Nov 13, 2023
6cbbe12
Full path - hard coded
Nov 13, 2023
af7242d
Allocating csv file/run
Nov 14, 2023
e72bd30
ls is fixed
Nov 14, 2023
1a179a7
permissions are set
Nov 14, 2023
993454a
env:
Nov 14, 2023
01e6d66
contents: write
Nov 14, 2023
84a3bc1
double permission
Nov 14, 2023
3533c31
contents: write
Nov 14, 2023
07fbf11
${{ secrets.GITHUB_TOKEN }}
Nov 14, 2023
7942003
'write'
Nov 14, 2023
b4d90f0
new permission has been added
Nov 14, 2023
89b0391
action - write
Nov 14, 2023
6f92006
give write permission to pull requests
HamishBrownPFR Nov 14, 2023
78a47b0
try different santax
HamishBrownPFR Nov 14, 2023
4ad03e9
remove permissions
HamishBrownPFR Nov 14, 2023
ae57dfb
try
HamishBrownPFR Nov 14, 2023
2dea623
give write permission to pull requests
HamishBrownPFR Nov 14, 2023
f68c802
try different santax
HamishBrownPFR Nov 14, 2023
f1d32b4
No GitHub secret
Nov 14, 2023
7ad965b
Take4
Oct 1, 2023
621bcd1
working action
Oct 9, 2023
f47f39b
new action
Oct 9, 2023
16ca6eb
changing a root directory for build
Oct 9, 2023
8a38f01
commit to testing system, instead of master
Oct 9, 2023
7bd8582
working directory - ./modelCsharp
Oct 9, 2023
8d83836
Clean up redundant code
Oct 9, 2023
285fa26
check if it is still working
Oct 10, 2023
f5e9c67
second try
Oct 10, 2023
0e5cac9
third
Oct 10, 2023
db96709
Checking before start on config/fertil
Oct 11, 2023
fc60675
push to master
Oct 11, 2023
de4f8e3
no specified working directory
Oct 11, 2023
04ddd1d
back to Testing system
Oct 11, 2023
01f211a
try on testing system
Oct 11, 2023
cd13f3b
testingSystem now
Oct 11, 2023
8b12d91
changed to master
Oct 11, 2023
9a2f985
Action with Artifact
Oct 12, 2023
bfff3d1
ls folder
Oct 12, 2023
8541301
ls graphs
Oct 12, 2023
93f5e50
mkdir graphs is moved above
Oct 12, 2023
a4e7eda
path: ./modelCsharp/graphs
Oct 12, 2023
fb5fee2
Removing DLLs
Oct 31, 2023
62e6385
Preview of graph in HTML
Oct 31, 2023
eda9cb3
Path is correct
Oct 31, 2023
779355b
Chanhed setting to gh-pages in SVS
Oct 31, 2023
5ed3b4e
Write permission exists
Oct 31, 2023
d500ff6
permissions: contents: read,pages: write,id-token: write
Oct 31, 2023
f0ab443
permissions write-all
Oct 31, 2023
08dbf2e
Sequence of jobs in action is changed
Oct 31, 2023
6091f51
Added blank space just to rub GitHub page
Nov 10, 2023
26ac67a
Fertiliser input.
Nov 12, 2023
9c1a67b
Whitespace added just to run action
Nov 12, 2023
68af7ba
First dates has been changed
Nov 12, 2023
9842c08
new format for dates/fertiliser
Nov 13, 2023
f89b6e3
Python.exe is commented. Run on action now
Nov 13, 2023
8f21a2f
Proper settings for Build are fixed
Nov 13, 2023
e4b5c47
No tests, looking for a csv file
Nov 13, 2023
1284792
PYthon is commented out
Nov 13, 2023
114c87b
Find a csv files
Nov 13, 2023
d8aea76
find a csv file
Nov 13, 2023
e4f99a5
home directory
Nov 13, 2023
3ecb29e
rooot diretory
Nov 13, 2023
affdb1c
ls/OutputFiles
Nov 13, 2023
691dad3
ls OutputFiles
Nov 13, 2023
8f62707
csv files are saved in folder
Nov 13, 2023
b49e53c
list folder - OutputFiles
Nov 13, 2023
1d7b5ab
Full path - hard coded
Nov 13, 2023
4001ffb
Allocating csv file/run
Nov 14, 2023
0e48b8e
ls is fixed
Nov 14, 2023
a8a50c7
permissions are set
Nov 14, 2023
aa9a90b
env:
Nov 14, 2023
2e66703
contents: write
Nov 14, 2023
4e17751
double permission
Nov 14, 2023
b92bca7
contents: write
Nov 14, 2023
0756f7e
${{ secrets.GITHUB_TOKEN }}
Nov 14, 2023
33434d1
'write'
Nov 14, 2023
afcaa86
new permission has been added
Nov 14, 2023
dac1cb1
action - write
Nov 14, 2023
cfe4fcd
give write permission to pull requests
HamishBrownPFR Nov 14, 2023
4b9e472
try different santax
HamishBrownPFR Nov 14, 2023
776eed8
remove permissions
HamishBrownPFR Nov 14, 2023
3b4e398
try
HamishBrownPFR Nov 14, 2023
bbf6e90
give write permission to pull requests
HamishBrownPFR Nov 14, 2023
81f44f6
try different santax
HamishBrownPFR Nov 14, 2023
002615d
Merge remote-tracking branch 'Hamish/PullWrite' into TestingSystem
HamishBrownPFR Nov 14, 2023
f99432d
try different trigger
HamishBrownPFR Nov 15, 2023
f2627bc
change back
HamishBrownPFR Nov 15, 2023
1b284d6
Merge remote-tracking branch 'Sasha/TestingSystem' into TestingSystem
HamishBrownPFR Nov 15, 2023
9a2a574
target
HamishBrownPFR Nov 15, 2023
2caf26e
all permissions
HamishBrownPFR Nov 15, 2023
01b3b7c
remove target
HamishBrownPFR Nov 15, 2023
8895de0
pull_request_target: + new permissions
Nov 15, 2023
683430c
more white space
Nov 15, 2023
dac0bb2
pull request no target
Nov 15, 2023
62b8867
back to request
Nov 15, 2023
2f0e0b9
Merge remote-tracking branch 'Sasha/TestingSystem' into TestingSystem
HamishBrownPFR Nov 15, 2023
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
31 changes: 0 additions & 31 deletions .github/workflows/build-solution.yml

This file was deleted.

98 changes: 98 additions & 0 deletions .github/workflows/ubuntodotnet.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,98 @@
# This workflow will build a .NET project
# For more information see: https://docs.github.com/en/actions/automating-builds-and-tests/building-and-testing-net

name: .NET

on:
push:
branches: [ "master" ]
pull_request_target:
branches: [ "master" ]

jobs:
build:

runs-on: ubuntu-latest


permissions:
actions: write
pages: write
id-token: write
contents: write
pull-requests: write
deployments: write

environment:
name: github-pages
url: ${{ steps.change-page-url.outputs.new_page_url }}

defaults:
run:
working-directory: ./modelCsharp

steps:
- uses: actions/checkout@v3
- name: Setup .NET
uses: actions/setup-dotnet@v3
with:
dotnet-version: 6.0.x

- name: Install Python 3
uses: actions/setup-python@v1
with:
python-version: 3.11.6

#installing packages
- name: Install Python pandas
run: python -m pip install --upgrade pip pandas
- name: Install Python aspose
run: python -m pip install --upgrade pip aspose.words
- name: Install Python matplotlib
run: python -m pip install --upgrade pip matplotlib
- name: Install Python pyinstaller
run: python -m pip install --upgrade pip datetime
- name: Install Python datetime
run: pip install pyinstaller

- name: Create a temporary artifact downloads folder for graphs
run: mkdir graphs

- name: Restore dependencies
run: dotnet restore

- name: Build
run: dotnet build --no-restore

- name: run
run: dotnet run --project ConsoleAppSVS-test/ConsoleAppSVS-test.csproj

- name: run python
run: python TestModel/testGraph/testGraph/testGraph.py

- name: moving artfacts to temp folder
run: |
mv index.html graphs
mv index.001.png graphs

- name: Step 3 - Use the Upload Artifact GitHub Action
uses: actions/upload-artifact@v2
with:
name: folder for graphs
path: ./modelCsharp/graphs

- name: Deploy 🚀
uses: actions/upload-pages-artifact@v2
with:
path: ./modelCsharp/graphs

- name: Deploy to GitHub Pages
id: deployment
uses: actions/deploy-pages@v2
with:
repo-token: ${{ secrets.GITHUB_TOKEN }}

- name: Override page_url
id: change-page-url
run: echo "new_page_url=${{ steps.deployment.outputs.page_url }}index.html" >> $GITHUB_OUTPUT

268 changes: 268 additions & 0 deletions Untitled1.ipynb

Large diffs are not rendered by default.

4 changes: 4 additions & 0 deletions modelCsharp/ConsoleAppSVS-test/ConsoleAppSVS-test.csproj
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,10 @@
<Nullable>enable</Nullable>
</PropertyGroup>

<ItemGroup>
<PackageReference Include="CommandLineParser" Version="2.9.1" />
</ItemGroup>

<ItemGroup>
<ProjectReference Include="..\SVSModel\SVSModel.csproj" />
<ProjectReference Include="..\TestModel\TestModel\TestModel.csproj" />
Expand Down
18 changes: 15 additions & 3 deletions modelCsharp/ConsoleAppSVS-test/Program.cs
Original file line number Diff line number Diff line change
@@ -1,4 +1,16 @@

using TestModel;
using TestModel;

Test.RunTests(TestConfigData.configDict);
namespace ConsoleApp1
{

internal class Program
{
/// <summary>
/// The main entry point for the application.
/// </summary>
static void Main(string[] args)
{
Test.RunTests(TestConfigData.configDict);
}
}
}
1 change: 1 addition & 0 deletions modelCsharp/SVSModel.Excel/SVSModel.Excel.csproj
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,7 @@
</ItemGroup>

<ItemGroup>
<PackageReference Include="CommandLineParser" Version="2.9.1" />
<PackageReference Include="ExcelDna.AddIn" Version="1.7.0-rc1" />
</ItemGroup>

Expand Down
1 change: 1 addition & 0 deletions modelCsharp/SVSModel/SVSModel.csproj
Original file line number Diff line number Diff line change
Expand Up @@ -114,6 +114,7 @@
</ItemGroup>

<ItemGroup>
<PackageReference Include="CommandLineParser" Version="2.9.1" />
<PackageReference Include="CsvHelper" Version="30.0.1" />
<PackageReference Include="Microsoft.Data.Analysis" Version="0.20.0" />
</ItemGroup>
Expand Down
Binary file added modelCsharp/TestModel/Observed/SoilMineralN.xlsx
Binary file not shown.
23 changes: 23 additions & 0 deletions modelCsharp/TestModel/TestModel/FertiliserData.csv
Original file line number Diff line number Diff line change
@@ -0,0 +1,23 @@
Site,Date,Nitrogen
8,2020-09-24,46
8,2020-09-24,22
8,2020-11-15,22
8,2020-11-15,16
8,2020-11-21,24
8,2020-11-20,0.75
8,2020-11-24,33.28
8,2020-11-26,54
8,2020-11-26,0.75
8,2020-12-01,24
8,2020-12-04,32
8,2020-12-12,38.4
8,2020-12-12,0.75
8,2021-04-21,8.4
8,2021-07-03,4.2
8,2021-09-10,76
8,2021-09-26,76
8,2021-10-11,76
8,2022-09-29,0.125
8,2022-09-29,0.63
8,2022-11-30,0.39
8,2022-11-30,0.1
89 changes: 73 additions & 16 deletions modelCsharp/TestModel/TestModel/Test.cs
Original file line number Diff line number Diff line change
Expand Up @@ -12,48 +12,62 @@
using System;
using static System.Net.Mime.MediaTypeNames;
using System.Text.Json.Serialization;
//using ServiceStack;
//using ServiceStack.Text;
using System.Collections.Generic;
//using Nancy.Routing.Constraints;
using IronPython.Hosting;
using Microsoft.Scripting.Hosting;
using Microsoft.Scripting;
using static IronPython.Modules._ast;
using System.IO;
using CommandLine;
using static IronPython.Modules.PythonDateTime;
using System.Text.RegularExpressions;
using System.Net.Http.Json;
using Microsoft.VisualBasic;
using static IronPython.SQLite.PythonSQLite;


namespace TestModel
{
public class Test
{
private static void runPythonScript()
/* private static void runPythonScript()
{
//that is new with james
//string progToRun = dir + @"/../TestModel/testGraph/testGraph/testGraph.py";
string dir = Directory.GetCurrentDirectory();
string newPath = Path.GetFullPath(Path.Combine(dir, @"..\..\..\..\"));
string progToRun = newPath+ @"TestModel\testGraph\testGraph\testGraph.py";
string progToRun = newPath + @"TestModel\testGraph\testGraph\testGraph.py";

Process proc = new Process();
proc.StartInfo.FileName = "python.exe";
proc.StartInfo.RedirectStandardOutput = true;
proc.StartInfo.UseShellExecute = false;
proc.StartInfo.Arguments =progToRun;
proc.StartInfo.UseShellExecute = false;
proc.StartInfo.Arguments = progToRun;
proc.Start();
StreamReader sReader = proc.StandardOutput;
proc.WaitForExit();
Console.ReadLine();

}
}*/
public static void RunTests(Dictionary<string, object> _configDict)

{
string dir = Directory.GetCurrentDirectory();
//DataFrame testConfigs = Crop.LoadCoefficients("SVSModel.Data.TestConfig.csv");
string resourceName = "TestModel.actualDataConfig.csv";

// new line
string resourceFertiliser = "TestModel.FertiliserData.csv";

string resourceName = "TestModel.TestConfig.csv";
var assembly = Assembly.GetExecutingAssembly();
Stream csv = assembly.GetManifestResourceStream(resourceName);

//new line
Stream csvFertiliser = assembly.GetManifestResourceStream(resourceFertiliser);

DataFrame allTests = DataFrame.LoadCsv(csv);

//new line
DataFrame fertiliser = DataFrame.LoadCsv(csvFertiliser);

List<string> Tests = new List<string>();

foreach (DataFrameRow row in allTests.Rows)
Expand All @@ -69,6 +83,40 @@ public static void RunTests(Dictionary<string, object> _configDict)

Dictionary<DateTime, double> testResults = new Dictionary<DateTime, double>();
Dictionary<DateTime, double> nApplied = new Dictionary<DateTime, double>();

int siteNumber = Int32.Parse(Regex.Match(test, @"\d+").Value);

foreach (DataFrameRow row in fertiliser.Rows)
{

int firstElement = Int32.Parse(row[0].ToString());

DateTime fertDate = Convert.ToDateTime(row[1].ToString());

if (firstElement == siteNumber)

if ((fertDate > _config.Current.EstablishDate) && (fertDate <= _config.Current.HarvestDate))

try
{

if (!nApplied.ContainsKey((DateTime)row[1]))
{
nApplied.Add(fertDate, (Single)row[2]);
}
else
{
nApplied[fertDate]+= (Single)row[2];
}

}

catch (Exception e)
{
Trace.WriteLine(e+"something is not working properly in fertiliser department");

}
}

string weatherStation = allTests["WeatherStation"][testRow].ToString();

Expand Down Expand Up @@ -102,14 +150,24 @@ public static void RunTests(Dictionary<string, object> _configDict)
}
newDataframe.Append(nextRow, true);
}
string folderName = "OutputFiles";

string fertiliserFolder = "NitrogenApplied";

if (!Directory.Exists(folderName) && !Directory.Exists(fertiliserFolder))
{
System.IO.Directory.CreateDirectory("OutputFiles");

}

DataFrame.SaveCsv(newDataframe, dir + "\\OutputFiles\\" + test + ".csv");
//DataFrame.SaveCsv(newDataframe, dir + "\\OutputFiles\\" + test + ".csv");
DataFrame.SaveCsv(newDataframe, test + ".csv");

}
runPythonScript();
// uncomment it if run is on local machine
//runPythonScript();

}

public static SVSModel.Configuration.Config SetConfigFromDataFrame(string test, DataFrame allTests)
{
int testRow = getTestRow(test, allTests);
Expand Down Expand Up @@ -184,7 +242,6 @@ public static SVSModel.Configuration.Config SetConfigFromDataFrame(string test,

return ret;
}

private static int getTestRow(string test, DataFrame allTests)
{
int testRow = 0;
Expand Down
4 changes: 0 additions & 4 deletions modelCsharp/TestModel/TestModel/TestConfig.csv

This file was deleted.

9 changes: 8 additions & 1 deletion modelCsharp/TestModel/TestModel/TestModel.csproj
Original file line number Diff line number Diff line change
Expand Up @@ -8,11 +8,16 @@
</PropertyGroup>

<ItemGroup>
<None Remove="actualDataConfig.csv" />
<None Remove="fert.csv" />
<None Remove="Fertiliser.csv" />
<None Remove="FertiliserData.csv" />
<None Remove="TestConfig.csv" />
</ItemGroup>

<ItemGroup>
<EmbeddedResource Include="TestConfig.csv" />
<EmbeddedResource Include="actualDataConfig.csv" />
<EmbeddedResource Include="FertiliserData.csv" />
</ItemGroup>


Expand All @@ -21,7 +26,9 @@
</ItemGroup>

<ItemGroup>
<PackageReference Include="CommandLineParser" Version="2.9.1" />
<PackageReference Include="IronPython" Version="3.4.1" />
<PackageReference Include="Newtonsoft.Json" Version="13.0.3" />
</ItemGroup>

<ItemGroup>
Expand Down
4 changes: 4 additions & 0 deletions modelCsharp/TestModel/TestModel/actualDataConfig.csv
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
Property,InitialN,SoilOrder,SampleDepth,BulkDensity,PMNtype,PMN,Trigger,Efficiency,Splits,AWC,PrePlantRain,InCropRain,Irrigation,PriorCropNameFull,PriorSaleableYield,PriorFieldLoss,PriorDressingLoss,PriorMoistureContent,PriorEstablishDate,PriorEstablishYear,PriorEstablishMonth,PriorEstablishDay,PriorEstablishStage,PriorHarvestDate,PriorHarvestYear,PriorHarvestMonth,PriorHarvestDay,PriorHarvestStage,PriorResidueRemoval,PriorResidueIncorporation,CurrentCropNameFull,CurrentSaleableYield,CurrentFieldLoss,CurrentDressingLoss,CurrentMoistureContent,CurrentEstablishDate,CurrentEstablishYear,CurrentEstablishMonth,CurrentEstablishDay,CurrentEstablishStage,CurrentHarvestDate,CurrentHarvestYear,CurrentHarvestMonth,CurrentHarvestDay,CurrentHarvestStage,CurrentResidueRemoval,CurrentResidueIncorporation,FollowingCropNameFull,FollowingSaleableYield,FollowingFieldLoss,FollowingDressingLoss,FollowingMoistureContent,FollowingEstablishDate,FollowingEstablishYear,FollowingEstablishMonth,FollowingEstablishDay,FollowingEstablishStage,FollowingHarvestDate,FollowingHarvestYear,FollowingHarvestMonth,FollowingHarvestDay,FollowingHarvestStage,FollowingResidueRemoval,FollowingResidueIncorporation,WeatherStation
8Wheat,50,Immature Pallic,0-30cm,1.07,PMN,160,30,80,3,148,Typical,Very Dry,Full,Potato Vegetable General,88.4,0,0,30,24/09/2020,2020,9,24,Seed,17/03/2021,2021,3,21,LateReproductive,None removed,Full (Plough),Wheat Grain General,12.7,0,0,30,21/04/2021,2021,4,21,Seed,26/01/2022,2022,1,26,Maturity,Baled,Full (Plough),Pea Grain General,4.7,0,0,30,27/09/2022,2022,9,27,Seed,25/01/2023,2023,1,25,Maturity,None removed,Full (Plough),ashburton
8Peas,50,Immature Pallic,0-30cm,1.22,PMN,120,30,80,3,148,Typical,Very Dry,Full,Wheat Grain General,12.7,0,0,30,21/04/2021,2021,4,21,Seed,26/01/2022,2022,1,26,Maturity,Baled,Full (Plough),Pea Grain General,4.7,0,0,30,27/09/2022,2022,9,27,Seed,25/01/2023,2023,1,25,Maturity,None removed,Full (Plough),Oat Fodder General,10,0,0,40,10/04/2023,2023,4,10,Seed,4/08/2023,2023,8,4,Maturity,None removed,Full (Plough),ashburton
8Oat,50,Immature Pallic,0-30cm,1.31,PMN,80,30,80,3,148,Typical,Very Dry,Full,Pea Grain General,4.7,0,0,30,27/09/2022,2022,9,27,Seed,25/01/2023,2023,1,25,Maturity,None removed,Full (Plough),Oat Fodder General,10,0,0,40,10/04/2023,2023,4,10,Seed,4/08/2023,2023,8,4,Maturity,None removed,Full (Plough),Oat Fodder General,10,0,0,0,1/09/2023,2023,9,1,Seed,1/10/2023,2023,10,1,EarlyReproductive,None removed,Full (Plough),ashburton
Loading