[WIP] tkg test adaptation on tkg-windows #3040

rajguptavm · 2024-09-11T09:37:42Z

What this PR does / why we need it:
testcase adaptation to support Windows container in Unified TKG

Which issue this PR fixes (optional, in fixes #<issue number>(, fixes #<issue_number>, ...) format, will close that issue when PR gets merged): fixes #
testcase adaptation to support Windows container in Unified TKG

Testing done:
In Progress

k8s-ci-robot · 2024-09-11T09:37:47Z

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: rajguptavm
Once this PR has been reviewed and has the lgtm label, please assign sipriyaa for approval. For more information see the Kubernetes Code Review Process.

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

tests/e2e/OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

k8s-ci-robot · 2024-09-11T09:37:52Z

Hi @rajguptavm. Thanks for your PR.

I'm waiting for a kubernetes-sigs member to verify that this patch is reasonable to test. If it is, they should reply with /ok-to-test on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work. Regular contributors should join the org to skip this step.

Once the patch is verified, the new status will be reflected by the ok-to-test label.

I understand the commands that are listed here.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

svcbot-qecnsdp · 2024-09-11T17:46:44Z

Started GC block pre-checkin pipeline... Build Number: 52

svcbot-qecnsdp · 2024-09-11T17:49:13Z

Build ID: 52
GC build status: FAILURE 
Stage before exit: e2e-tests 
Jenkins E2E Test Results: 
FAIL! -- 0 Passed | 1 Failed | 0 Pending | 881 Skipped
--- FAIL: TestE2E (4.25s)
FAIL

Ginkgo ran 1 suite in 1m8.731279172s

Test Suite Failed
make: Leaving directory `/home/worker/workspace/gc-csi-precheckin-Raj-tkg-windows/Results/52/vsphere-csi-driver`

svcbot-qecnsdp · 2024-09-11T17:54:23Z

Started GC block pre-checkin pipeline... Build Number: 53

svcbot-qecnsdp · 2024-09-11T18:02:53Z

Build ID: 53
GC build status: FAILURE 
Stage before exit: e2e-tests 
Jenkins E2E Test Results: 
FAIL! -- 0 Passed | 1 Failed | 0 Pending | 881 Skipped
--- FAIL: TestE2E (336.80s)
FAIL

Ginkgo ran 1 suite in 6m39.228501102s

Test Suite Failed
make: Leaving directory `/home/worker/workspace/gc-csi-precheckin-Raj-tkg-windows/Results/53/vsphere-csi-driver`

Aishwarya-Hebbar · 2024-11-22T09:29:01Z

tests/e2e/csi_static_provisioning_basic.go

@@ -212,7 +213,8 @@ var _ = ginkgo.Describe("Basic Static Provisioning", func() {
 		framework.Logf("storageclass name :%s", storageclass.GetName())

 		ginkgo.By("create resource quota")
-		setStoragePolicyQuota(ctx, restConfig, storagePolicyName, namespace, rqLimit)
+		// setStoragePolicyQuota(ctx, restConfig, storagePolicyName, namespace, rqLimit)


Remove unnecessary log and commented lines

Aishwarya-Hebbar · 2024-11-22T09:30:19Z

tests/e2e/csi_snapshot_utils.go

-			"echo 'Hello message from test into Pod1' >> /mnt/volume1/Pod1.html"}
+		var wrtiecmd []string
+		if windowsEnv {
+			wrtiecmd = []string{"exec", pod.Name, "--namespace=" + namespace, "powershell.exe", "Add-Content /mnt/volume1/Pod1.html 'Hello message from test into Pod1'"}


We can use filePathPod1 variable instead of /mnt/volume1/Pod1.html

Aishwarya-Hebbar · 2024-11-22T09:32:14Z

tests/e2e/gc_block_resize_retain_policy.go

@@ -435,7 +452,11 @@ var _ = ginkgo.Describe("[csi-guest] Volume Expansion Tests with reclaimation po
 		_ = getPVCFromSupervisorCluster(svcPVCName)

 		scParameters := make(map[string]string)
-		scParameters[scParamFsType] = ext4FSType
+		if windowsEnv {


This is already present in BeforeEach, no need here

Aishwarya-Hebbar · 2024-11-22T09:32:57Z

tests/e2e/gc_block_resize_retain_policy.go

@@ -122,8 +126,11 @@ var _ = ginkgo.Describe("[csi-guest] Volume Expansion Tests with reclaimation po
 		pvDeleted = false

 		// Replace second element with pod.Name.
-		cmd = []string{"exec", "", fmt.Sprintf("--namespace=%v", namespace),
-			"--", "/bin/sh", "-c", "df -Tkm | grep /mnt/volume1"}
+		if windowsEnv {


We need to pass a podName here right? Otherwise we can remove above comment here

Aishwarya-Hebbar · 2024-11-22T09:37:24Z

tests/e2e/gc_block_volume_expansion.go

+		defer func() {
+			// Delete Pod.
+			ginkgo.By(fmt.Sprintf("Deleting the pod %s in namespace %s", pod.Name, namespace))
+			err := fpod.DeletePodWithWait(ctx, client, pod)


Please also verify that pod gets detached from volume

Aishwarya-Hebbar · 2024-11-22T09:49:05Z

tests/e2e/gc_block_volume_expansion.go

-		_ = e2ekubectl.RunKubectlOrDie(namespace, "cp", testdataFile,
-			fmt.Sprintf("%v/%v:/mnt/volume1/testdata", namespace, pod.Name))
+		if windowsEnv {
+			cmdTestData := []string{"exec", pod.Name, "--namespace=" + namespace, "powershell.exe", "$out = New-Object byte[] 536870912; (New-Object Random).NextBytes($out); [System.IO.File]::WriteAllBytes('/mnt/volume1/testdata2.txt', $out)"}


Please add some comments here

Aishwarya-Hebbar · 2024-11-22T09:51:37Z

tests/e2e/tkgs_ha_utils.go

+	var testdataFile string
+	var op []byte
+	var err error
+	if !windowsEnv {


We can put an if else conditios instead of 2 if conditions

Aishwarya-Hebbar · 2024-11-22T09:54:52Z

tests/e2e/util.go

+// execCommandOnGcWorker logs into gc worker node using ssh private key and executes command
+func execCommandOnGcWorker(sshClientConfig *ssh.ClientConfig, svcMasterIP string, gcWorkerIp string,
+	svcNamespace string, cmd string) (fssh.Result, error) {
+	result := fssh.Result{Host: gcWorkerIp, Cmd: cmd}


Is this line required?

Aishwarya-Hebbar · 2024-11-22T09:57:34Z

tests/e2e/volume_health_test.go

@@ -2275,7 +2285,9 @@ var _ = ginkgo.Describe("Volume health check", func() {
 		replicas := *(statefulset.Spec.Replicas)
 		// Waiting for pods status to be Ready.
 		fss.WaitForStatusReadyReplicas(ctx, client, statefulset, replicas)
-		gomega.Expect(fss.CheckMount(ctx, client, statefulset, mountPath)).NotTo(gomega.HaveOccurred())
+		if !windowsEnv {


Don't we need to check mountPath in case of windows environments?

Aishwarya-Hebbar · 2024-11-22T09:59:02Z

tests/e2e/vsphere_volume_expansion.go

@@ -3779,7 +3806,11 @@ func invokeTestForExpandVolumeMultipleTimes(f *framework.Framework, client clien
 	defer cancel()
 	ginkgo.By("Invoking Test to verify Multiple Volume Expansions on the same volume")
 	scParameters := make(map[string]string)
-	scParameters[scParamFsType] = ext4FSType
+	if windowsEnv {


We can put this in BeforeEach

kavyashree-r · 2024-11-22T11:07:27Z

tests/e2e/csi_snapshot_utils.go

-		cmd := []string{"exec", pod.Name, "--namespace=" + namespace, "--", "/bin/sh", "-c",
-			"cat /mnt/volume1/Pod1.html "}
+		if windowsEnv {
+			cmd = []string{"exec", pod.Name, "--namespace=" + namespace, "powershell.exe", "cat /mnt/volume1/Pod1.html"}


use filePathPod1 instead /mnt/volume1/Pod1.html in all the places

kavyashree-r · 2024-11-22T11:14:50Z

tests/e2e/policy_driven_vol_allocation.go

-			svcClient, svNamespace := getSvcClientAndNamespace()
-			setResourceQuota(svcClient, svNamespace, rqLimitScaleTest)
-		}
+		// if guestCluster {


remove these lines if not needed

kavyashree-r · 2024-11-22T11:15:56Z

tests/e2e/policy_driven_vol_allocation.go

-			_, err = e2eoutput.LookForStringInPodExec(namespace, newPods[0].Name,
-				[]string{"/bin/cat", "/mnt/volume1/fstype"}, "", time.Minute)
-		}
+		// covered in this method createMultiplePods to check fss type


Remove commented code

kavyashree-r · 2024-11-22T11:19:57Z

tests/e2e/tkgs_ha_utils.go

-	gomega.Expect(err).NotTo(gomega.HaveOccurred())
-	gomega.Expect(len(op)).To(gomega.BeZero())
+	if windowsEnv {
+		cmdTestData := []string{"exec", pod.Name, "--namespace=" + namespace, "powershell.exe", "((Get-FileHash '/mnt/volume1/testdata2.txt' -Algorithm SHA256).Hash -eq (Get-FileHash '/mnt/volume1/testdata2_pod.txt' -Algorithm SHA256).Hash)"}


Since this path is used in many places /mnt/volume1/testdata2.txt declare it in e2e_common.

k8s-ci-robot · 2024-11-22T20:29:39Z

PR needs rebase.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

k8s-ci-robot added the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Sep 11, 2024

k8s-ci-robot requested review from kavyashree-r and sipriyaa September 11, 2024 09:37

k8s-ci-robot added cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. needs-ok-to-test Indicates a PR that requires an org member to verify it is safe to test. labels Sep 11, 2024

k8s-ci-robot added the size/XL Denotes a PR that changes 500-999 lines, ignoring generated files. label Sep 11, 2024

rajguptavm force-pushed the tkg-windows-automation branch from 481e064 to 6e23936 Compare September 20, 2024 09:22

k8s-ci-robot added size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files. and removed size/XL Denotes a PR that changes 500-999 lines, ignoring generated files. labels Oct 1, 2024

k8s-ci-robot added the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Oct 24, 2024

rajguptavm force-pushed the tkg-windows-automation branch from a4b3093 to 16c089e Compare November 4, 2024 16:02

k8s-ci-robot added size/XL Denotes a PR that changes 500-999 lines, ignoring generated files. and removed needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files. labels Nov 4, 2024

tkg test adaptation on windows

90dfbf8

rajguptavm force-pushed the tkg-windows-automation branch from 16c089e to 90dfbf8 Compare November 4, 2024 16:06

tkgs-ha changes for windows

d0adfa6

k8s-ci-robot added size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files. and removed size/XL Denotes a PR that changes 500-999 lines, ignoring generated files. labels Nov 7, 2024

Aishwarya-Hebbar reviewed Nov 22, 2024

View reviewed changes

kavyashree-r reviewed Nov 22, 2024

View reviewed changes

k8s-ci-robot added the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Nov 22, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[WIP] tkg test adaptation on tkg-windows #3040

[WIP] tkg test adaptation on tkg-windows #3040

rajguptavm commented Sep 11, 2024

k8s-ci-robot commented Sep 11, 2024

k8s-ci-robot commented Sep 11, 2024

svcbot-qecnsdp commented Sep 11, 2024

svcbot-qecnsdp commented Sep 11, 2024

svcbot-qecnsdp commented Sep 11, 2024

svcbot-qecnsdp commented Sep 11, 2024

Aishwarya-Hebbar Nov 22, 2024

Aishwarya-Hebbar Nov 22, 2024

Aishwarya-Hebbar Nov 22, 2024

Aishwarya-Hebbar Nov 22, 2024

Aishwarya-Hebbar Nov 22, 2024

Aishwarya-Hebbar Nov 22, 2024

Aishwarya-Hebbar Nov 22, 2024

Aishwarya-Hebbar Nov 22, 2024

Aishwarya-Hebbar Nov 22, 2024

Aishwarya-Hebbar Nov 22, 2024

kavyashree-r Nov 22, 2024

kavyashree-r Nov 22, 2024

kavyashree-r Nov 22, 2024

kavyashree-r Nov 22, 2024

k8s-ci-robot commented Nov 22, 2024

[WIP] tkg test adaptation on tkg-windows #3040

Are you sure you want to change the base?

[WIP] tkg test adaptation on tkg-windows #3040

Conversation

rajguptavm commented Sep 11, 2024

k8s-ci-robot commented Sep 11, 2024

k8s-ci-robot commented Sep 11, 2024

svcbot-qecnsdp commented Sep 11, 2024

svcbot-qecnsdp commented Sep 11, 2024

svcbot-qecnsdp commented Sep 11, 2024

svcbot-qecnsdp commented Sep 11, 2024

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

k8s-ci-robot commented Nov 22, 2024