storage policy quota #3027

kavyashree-r · 2024-09-04T07:40:35Z

What this PR does / why we need it:

Which issue this PR fixes (optional, in fixes #<issue number>(, fixes #<issue_number>, ...) format, will close that issue when PR gets merged): fixes #

Testing done:
A PR must be marked "[WIP]", if no test result is provided. A WIP PR won't be reviewed, nor merged.
The requester can determine a sufficient test, e.g. build for a cosmetic change, E2E test in a predeployed setup, etc.
For new features, new tests should be done, in addition to regression tests.
If jtest is used to trigger precheckin tests, paste the result after jtest completes and remove [WIP] in the PR subject.
The review cycle will start, only after "[WIP]" is removed from the PR subject.

Special notes for your reviewer:

Release note:

svcbot-qecnsdp · 2024-09-05T07:42:55Z

Started GC block pipeline...

Aishwarya-Hebbar · 2024-11-21T06:05:50Z

tests/e2e/csi_snapshot_utils.go

 	if vanillaCluster || supervisorCluster {
 		snapshotHandle = *snapshotContent.Status.SnapshotHandle
 		snapshotID = strings.Split(snapshotHandle, "+")[1]
+		framework.Logf("********** snapshotHandle :%v", snapshotHandle)


Please remove unnecessary log and commented lines across all files

Aishwarya-Hebbar · 2024-11-21T06:06:32Z

tests/e2e/csi_static_provisioning_basic.go

@@ -140,6 +142,9 @@ var _ = ginkgo.Describe("Basic Static Provisioning", func() {
 		} else {
 			fullSyncWaitTime = defaultFullSyncWaitTime
 		}
+		vcAddress := e2eVSphere.Config.Global.VCenterHostname + ":" + sshdPort
+		isStorageQuotaFSSEnabled = isFssEnabled(ctx, vcAddress, "STORAGE_QUOTA_M2")


Please add a comment to remove this once the FSS is enabled

Aishwarya-Hebbar · 2024-11-21T06:10:19Z

tests/e2e/improved_csi_idempotency.go

+			storageclass.Name, namespace, volExtensionName)
+		framework.Logf("storagepolicyquota_pvc_after :%v", storagepolicyquota_pvc_after)
+
+		storagepolicy_usage_pvc_after, _ = getStoragePolicyUsageForSpecificResourceType(ctx, restConfig,


Please use camelCase to name your variables

Aishwarya-Hebbar · 2024-11-21T06:10:57Z

tests/e2e/util.go

+		storagepolicyUsage_reserved_Quota.String() == "0"
+
+}
+


Add comments to this specific method

Aishwarya-Hebbar · 2024-11-21T06:12:19Z

tests/e2e/util.go

+}
+
+// stopCSIPods function stops all the running csi pods
+func stopKubeSystemPods(ctx context.Context, client clientset.Interface, namespace string) (bool, error) {


Please rename this method to appropriate name and add a comment accordingly

Aishwarya-Hebbar · 2024-11-21T06:13:28Z

tests/e2e/improved_csi_idempotency.go

+		6. Delete pvcs and SC
+		7. Verify no orphan volumes are left
+	*/
+	ginkgo.It("create volume when storage-quota-weebhook goes down", func() {


This testcase will run for vanilla as well, we can add a tag to just run for supervisor suite

Also please correct the test steps accordingly

Aishwarya-Hebbar · 2024-11-21T06:30:15Z

tests/e2e/csi_snapshot_basic.go

@@ -1624,6 +1750,31 @@ var _ = ginkgo.Describe("Volume Snapshot Basic Test", func() {
 		snapshotCreated, snapshotContentCreated, err = deleteVolumeSnapshot(ctx, snapc, namespace,
 			volumeSnapshot, pandoraSyncWaitTime, volHandle, snapshotId, true)
 		gomega.Expect(err).NotTo(gomega.HaveOccurred())
+
+		if isStorageQuotaFSSEnabled && supervisorCluster {


Please check if this can be seperated into 3 different functions:

For volume and snapshot quota and usage before

For volume and snapshot quota and usage after

For cleanup of volumes and snapshots

rajguptavm · 2024-11-22T09:36:24Z

tests/e2e/csi_static_provisioning_basic.go

+
+			storagepolicyquota_pvc_before, _ = getStoragePolicyQuotaForSpecificResourceType(ctx, restConfig,
+				storagePolicyName, namespace, volExtensionName)
+			framework.Logf("volume ********** storagepolicyquota_pvc_before :%v ", storagepolicyquota_pvc_before)


Add some proper comment

rajguptavm · 2024-11-22T09:36:34Z

tests/e2e/csi_static_provisioning_basic.go

+
+			storagepolicy_usage_pvc_before, _ = getStoragePolicyUsageForSpecificResourceType(ctx, restConfig,
+				storagePolicyName, namespace, pvcUsage)
+			framework.Logf("volume ********** storagepolicy_usage_pvc_before :%v", storagepolicy_usage_pvc_before)


Here as well

rajguptavm · 2024-11-22T09:36:54Z

tests/e2e/csi_static_provisioning_basic.go

+
+			storagepolicyquota_pvc_after, _ = getStoragePolicyQuotaForSpecificResourceType(ctx, restConfig,
+				storagePolicyName, namespace, volExtensionName)
+			framework.Logf("********** storagepolicyquota_pvc_after :%v", storagepolicyquota_pvc_after)


rajguptavm · 2024-11-22T09:37:03Z

tests/e2e/csi_static_provisioning_basic.go

+
+			storagepolicy_usage_pvc_after, _ = getStoragePolicyUsageForSpecificResourceType(ctx, restConfig,
+				storagePolicyName, namespace, pvcUsage)
+			framework.Logf("********** pvc_Usage_Quota_After :%v", storagepolicy_usage_pvc_after)


rajguptavm · 2024-11-22T09:38:09Z

tests/e2e/csi_static_provisioning_basic.go

@@ -2272,7 +2427,7 @@ var _ = ginkgo.Describe("Basic Static Provisioning", func() {
 	})

 	/*
-		VMDK is deleted from datastore but CNS volume is still present
+		VMDK is deleted from datastore but CNS volume is still presentx


modify this

rajguptavm · 2024-11-22T09:48:01Z

tests/e2e/statefulsets.go

+
+			storagepolicyquota_pvc_before, _ = getStoragePolicyQuotaForSpecificResourceType(ctx, restConfig,
+				storagePolicyName, namespace, volExtensionName)
+			framework.Logf("volume ********** storagepolicyquota_pvc_before :%v ", storagepolicyquota_pvc_before)


add meaningful log

rajguptavm · 2024-11-22T09:48:11Z

tests/e2e/statefulsets.go

+
+			storagepolicy_usage_pvc_before, _ = getStoragePolicyUsageForSpecificResourceType(ctx, restConfig,
+				storagePolicyName, namespace, pvcUsage)
+			framework.Logf("volume ********** storagepolicy_usage_pvc_before :%v", storagepolicy_usage_pvc_before)


here as well

rajguptavm · 2024-11-22T09:52:15Z

tests/e2e/util.go

+	cnsOperatorClient, err := k8s.NewClientForGroup(ctx, restClientConfig, cnsoperatorv1alpha1.GroupName)
+	gomega.Expect(err).NotTo(gomega.HaveOccurred())
+
+	//spq := &storagepolicyv1alpha2.StoragePolicyQuota{}


remove this comment

rajguptavm · 2024-11-22T09:58:11Z

tests/e2e/util.go

+		if string(suffix1) == "Mi" {
+			var bytes int64 = quotaBefore
+			// Convert to Gi
+			//kibibytes := float64(bytes) / 1024


remove all commented code if not required

rajguptavm · 2024-11-22T10:34:36Z

tests/e2e/csi_snapshot_basic.go

@@ -284,6 +290,45 @@ var _ = ginkgo.Describe("Volume Snapshot Basic Test", func() {
 			gomega.Expect(err).NotTo(gomega.HaveOccurred())
 		}()

+		if isStorageQuotaFSSEnabled {
+			totalquota_used_before, _ = getTotalQuotaConsumedByStoragePolicy(ctx, restConfig, storageclass.Name, namespace)
+			framework.Logf("************ totalUsedQuota_Before :%v", totalquota_used_before)


add meanningful comment

rajguptavm

left some comment

k8s-ci-robot · 2024-11-22T10:36:51Z

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: kavyashree-r, rajguptavm

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

~~tests/e2e/OWNERS~~ [kavyashree-r]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

k8s-ci-robot · 2024-11-22T20:29:41Z

PR needs rebase.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

k8s-ci-robot requested review from shalini-b and sipriyaa September 4, 2024 07:40

k8s-ci-robot added cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. approved Indicates a PR has been approved by an approver from all required OWNERS files. size/L Denotes a PR that changes 100-499 lines, ignoring generated files. labels Sep 4, 2024

kavyashree-r force-pushed the storageQuota1 branch from 0440d5f to 7dc7cac Compare September 4, 2024 07:42

k8s-ci-robot added size/XL Denotes a PR that changes 500-999 lines, ignoring generated files. and removed size/L Denotes a PR that changes 100-499 lines, ignoring generated files. labels Oct 7, 2024

kavyashree-r force-pushed the storageQuota1 branch 3 times, most recently from ac647e0 to 1f9abaf Compare October 11, 2024 20:03

k8s-ci-robot added size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files. and removed size/XL Denotes a PR that changes 500-999 lines, ignoring generated files. labels Oct 11, 2024

kavyashree-r force-pushed the storageQuota1 branch 3 times, most recently from 8f39aa0 to 6bbac3f Compare October 17, 2024 11:36

k8s-ci-robot added the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Oct 24, 2024

kavyashree-r force-pushed the storageQuota1 branch from 6bbac3f to 6ad92d7 Compare November 18, 2024 12:41

k8s-ci-robot removed the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Nov 18, 2024

kavyashree-r changed the title ~~[WIP] storage policy quota - 1st set~~ storage policy quota Nov 19, 2024

kavyashree-r force-pushed the storageQuota1 branch from 6ad92d7 to 4b79833 Compare November 20, 2024 10:55

storage policy quota - 1st set

ad62563

kavyashree-r force-pushed the storageQuota1 branch from 4b79833 to ad62563 Compare November 20, 2024 11:29

Aishwarya-Hebbar reviewed Nov 21, 2024

View reviewed changes

rajguptavm reviewed Nov 22, 2024

View reviewed changes

rajguptavm approved these changes Nov 22, 2024

View reviewed changes

k8s-ci-robot added the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Nov 22, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

storage policy quota #3027

storage policy quota #3027

kavyashree-r commented Sep 4, 2024

svcbot-qecnsdp commented Sep 5, 2024

Aishwarya-Hebbar Nov 21, 2024

Aishwarya-Hebbar Nov 21, 2024

Aishwarya-Hebbar Nov 21, 2024

Aishwarya-Hebbar Nov 21, 2024

Aishwarya-Hebbar Nov 21, 2024

Aishwarya-Hebbar Nov 21, 2024

Aishwarya-Hebbar Nov 21, 2024

Aishwarya-Hebbar Nov 21, 2024

rajguptavm Nov 22, 2024

rajguptavm Nov 22, 2024

rajguptavm Nov 22, 2024

rajguptavm Nov 22, 2024

rajguptavm Nov 22, 2024

rajguptavm Nov 22, 2024

rajguptavm Nov 22, 2024

rajguptavm Nov 22, 2024

rajguptavm Nov 22, 2024

rajguptavm Nov 22, 2024

rajguptavm left a comment

k8s-ci-robot commented Nov 22, 2024

k8s-ci-robot commented Nov 22, 2024

storage policy quota #3027

Are you sure you want to change the base?

storage policy quota #3027

Conversation

kavyashree-r commented Sep 4, 2024

svcbot-qecnsdp commented Sep 5, 2024

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

rajguptavm left a comment

Choose a reason for hiding this comment

k8s-ci-robot commented Nov 22, 2024

k8s-ci-robot commented Nov 22, 2024