Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

DNM: POC - Override affinity policy for each GlanceAPI #656

Open
wants to merge 2 commits into
base: main
Choose a base branch
from
Open

DNM: POC - Override affinity policy for each GlanceAPI #656

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
664ebc0
Consume the Affinity interface provided by lib-common
fmount Nov 18, 2024
845c1b3
Expose lib-common AffinityOverrides for each GlanceAPI
fmount Nov 19, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
65 changes: 65 additions & 0 deletions api/bases/glance.openstack.org_glanceapis.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -574,6 +574,71 @@ spec:
type: object
override:
properties:
scheduling:
properties:
affinity:
properties:
preferred:
properties:
selectorKey:
type: string
selectorValues:
items:
type: string
type: array
topologyKey:
type: string
weight:
format: int32
type: integer
type: object
required:
properties:
selectorKey:
type: string
selectorValues:
items:
type: string
type: array
topologyKey:
type: string
weight:
format: int32
type: integer
type: object
type: object
antiAffinity:
properties:
preferred:
properties:
selectorKey:
type: string
selectorValues:
items:
type: string
type: array
topologyKey:
type: string
weight:
format: int32
type: integer
type: object
required:
properties:
selectorKey:
type: string
selectorValues:
items:
type: string
type: array
topologyKey:
type: string
weight:
format: int32
type: integer
type: object
type: object
type: object
service:
additionalProperties:
properties:
Expand Down
65 changes: 65 additions & 0 deletions api/bases/glance.openstack.org_glances.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -584,6 +584,71 @@ spec:
type: object
override:
properties:
scheduling:
properties:
affinity:
properties:
preferred:
properties:
selectorKey:
type: string
selectorValues:
items:
type: string
type: array
topologyKey:
type: string
weight:
format: int32
type: integer
type: object
required:
properties:
selectorKey:
type: string
selectorValues:
items:
type: string
type: array
topologyKey:
type: string
weight:
format: int32
type: integer
type: object
type: object
antiAffinity:
properties:
preferred:
properties:
selectorKey:
type: string
selectorValues:
items:
type: string
type: array
topologyKey:
type: string
weight:
format: int32
type: integer
type: object
required:
properties:
selectorKey:
type: string
selectorValues:
items:
type: string
type: array
topologyKey:
type: string
weight:
format: int32
type: integer
type: object
type: object
type: object
service:
additionalProperties:
properties:
Expand Down
2 changes: 2 additions & 0 deletions api/go.mod
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -70,3 +70,5 @@ require (
// mschuppert: map to latest commit from release-4.16 tag
// must consistent within modules and service operators
replace github.com/openshift/api => github.com/openshift/api v0.0.0-20240830023148-b7d0481c9094 //allow-merging

replace github.com/openstack-k8s-operators/lib-common/modules/common => github.com/fmount/lib-common/modules/common v0.0.0-20241121121123-92d5e021a60f
4 changes: 2 additions & 2 deletions api/go.sum
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,8 @@ github.com/evanphx/json-patch v5.7.0+incompatible h1:vgGkfT/9f8zE6tvSCe74nfpAVDQ
github.com/evanphx/json-patch v5.7.0+incompatible/go.mod h1:50XU6AFN0ol/bzJsmQLiYLvXMP4fmwYFNcr97nuDLSk=
github.com/evanphx/json-patch/v5 v5.9.0 h1:kcBlZQbplgElYIlo/n1hJbls2z/1awpXxpRi0/FOJfg=
github.com/evanphx/json-patch/v5 v5.9.0/go.mod h1:VNkHZ/282BpEyt/tObQO8s5CMPmYYq14uClGH4abBuQ=
github.com/fmount/lib-common/modules/common v0.0.0-20241121121123-92d5e021a60f h1:QggMF3xkvFcqRxJ7tca4lbwH0H3amP+3oy3Y9N6aeu4=
github.com/fmount/lib-common/modules/common v0.0.0-20241121121123-92d5e021a60f/go.mod h1:YpNTuJhDWhbXM50O3qBkhO7M+OOyRmWkNVmJ4y3cyFs=
github.com/fsnotify/fsnotify v1.7.0 h1:8JEhPFa5W2WU7YfeZzPNqzMP6Lwt7L2715Ggo0nosvA=
github.com/fsnotify/fsnotify v1.7.0/go.mod h1:40Bi/Hjc2AVfZrqy+aj+yEI+/bRxZnMJyTJwOpGvigM=
github.com/go-logr/logr v1.4.2 h1:6pFjapn8bFcIbiKo3XT4j/BhANplGihG6tvd+8rYgrY=
Expand Down Expand Up @@ -75,8 +77,6 @@ github.com/onsi/gomega v1.34.1 h1:EUMJIKUjM8sKjYbtxQI9A4z2o+rruxnzNvpknOXie6k=
github.com/onsi/gomega v1.34.1/go.mod h1:kU1QgUvBDLXBJq618Xvm2LUX6rSAfRaFRTcdOeDLwwY=
github.com/openshift/api v0.0.0-20240830023148-b7d0481c9094 h1:J1wuGhVxpsHykZBa6Beb1gQ96Ptej9AE/BvwCBiRj1E=
github.com/openshift/api v0.0.0-20240830023148-b7d0481c9094/go.mod h1:CxgbWAlvu2iQB0UmKTtRu1YfepRg1/vJ64n2DlIEVz4=
github.com/openstack-k8s-operators/lib-common/modules/common v0.5.1-0.20241114091812-6dc9fd0961dc h1:Ufa/q/nC9wmKblvsc0kJppsXHOJoY4fbUamb3ItWCOk=
github.com/openstack-k8s-operators/lib-common/modules/common v0.5.1-0.20241114091812-6dc9fd0961dc/go.mod h1:YpNTuJhDWhbXM50O3qBkhO7M+OOyRmWkNVmJ4y3cyFs=
github.com/openstack-k8s-operators/lib-common/modules/storage v0.5.1-0.20241114091812-6dc9fd0961dc h1:J5Kr0/ST3KqMzSRhcubr0fF9/vWzma+U63P9kfmgIA8=
github.com/openstack-k8s-operators/lib-common/modules/storage v0.5.1-0.20241114091812-6dc9fd0961dc/go.mod h1:tfgBeLRqmlH/NQkLPe7396rj+t0whv2wPuMb8Ttvh8w=
github.com/pkg/errors v0.9.1 h1:FEBLx1zS214owpjy7qsBeixbURkuhQAwrK5UwLGTwt4=
Expand Down
6 changes: 6 additions & 0 deletions api/v1beta1/common_types.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,7 @@ package v1beta1
import (
"strings"

"github.com/openstack-k8s-operators/lib-common/modules/common/affinity"
"github.com/openstack-k8s-operators/lib-common/modules/common/service"
"github.com/openstack-k8s-operators/lib-common/modules/common/tls"
"github.com/openstack-k8s-operators/lib-common/modules/common/util"
Expand Down Expand Up @@ -142,6 +143,11 @@ type APIOverrideSpec struct {
// Override configuration for the Service created to serve traffic to the cluster.
// The key must be the endpoint type (public, internal)
Service map[service.Endpoint]service.RoutedOverrideSpec `json:"service,omitempty"`

// APIAffinity exposes PodAffinity and PodAntiaffinity overrides that are applied
// to the StatefulSet
// +optional
APIAffinity affinity.Overrides `json:"scheduling,omitempty"`
}

// SetupDefaults - initializes any CRD field defaults based on environment variables (the defaulting mechanism itself is implemented via webhooks)
Expand Down
2 changes: 1 addition & 1 deletion api/v1beta1/conditions.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -32,7 +32,7 @@ const (
// GlanceAPIReadyCondition Status=True condition which indicates if the GlanceAPI is configured and operational
GlanceAPIReadyCondition condition.Type = "GlanceAPIReady"
// CinderCondition
CinderCondition= "CinderReady"
CinderCondition = "CinderReady"
// GlanceLayoutUpdateErrorMessage
GlanceLayoutUpdateErrorMessage = "The GlanceAPI layout (type) cannot be modified. To proceed, please add a new API with the desired layout and then decommission the previous API"
// KeystoneEndpointErrorMessage
Expand Down
6 changes: 3 additions & 3 deletions api/v1beta1/glanceapi_webhook.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -17,14 +17,14 @@ limitations under the License.
package v1beta1

import (
"fmt"
"github.com/google/go-cmp/cmp"
apierrors "k8s.io/apimachinery/pkg/api/errors"
"k8s.io/apimachinery/pkg/runtime"
ctrl "sigs.k8s.io/controller-runtime"
logf "sigs.k8s.io/controller-runtime/pkg/log"
"sigs.k8s.io/controller-runtime/pkg/webhook"
"fmt"
"sigs.k8s.io/controller-runtime/pkg/webhook/admission"
"github.com/google/go-cmp/cmp"
apierrors "k8s.io/apimachinery/pkg/api/errors"
)

// GlanceAPIDefaults -
Expand Down
1 change: 1 addition & 0 deletions api/v1beta1/zz_generated.deepcopy.go
View file
Open in desktop

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

65 changes: 65 additions & 0 deletions config/crd/bases/glance.openstack.org_glanceapis.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -574,6 +574,71 @@ spec:
type: object
override:
properties:
scheduling:
properties:
affinity:
properties:
preferred:
properties:
selectorKey:
type: string
selectorValues:
items:
type: string
type: array
topologyKey:
type: string
weight:
format: int32
type: integer
type: object
required:
properties:
selectorKey:
type: string
selectorValues:
items:
type: string
type: array
topologyKey:
type: string
weight:
format: int32
type: integer
type: object
type: object
antiAffinity:
properties:
preferred:
properties:
selectorKey:
type: string
selectorValues:
items:
type: string
type: array
topologyKey:
type: string
weight:
format: int32
type: integer
type: object
required:
properties:
selectorKey:
type: string
selectorValues:
items:
type: string
type: array
topologyKey:
type: string
weight:
format: int32
type: integer
type: object
type: object
type: object
service:
additionalProperties:
properties:
Expand Down
65 changes: 65 additions & 0 deletions config/crd/bases/glance.openstack.org_glances.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -584,6 +584,71 @@ spec:
type: object
override:
properties:
scheduling:
properties:
affinity:
properties:
preferred:
properties:
selectorKey:
type: string
selectorValues:
items:
type: string
type: array
topologyKey:
type: string
weight:
format: int32
type: integer
type: object
required:
properties:
selectorKey:
type: string
selectorValues:
items:
type: string
type: array
topologyKey:
type: string
weight:
format: int32
type: integer
type: object
type: object
antiAffinity:
properties:
preferred:
properties:
selectorKey:
type: string
selectorValues:
items:
type: string
type: array
topologyKey:
type: string
weight:
format: int32
type: integer
type: object
required:
properties:
selectorKey:
type: string
selectorValues:
items:
type: string
type: array
topologyKey:
type: string
weight:
format: int32
type: integer
type: object
type: object
type: object
service:
additionalProperties:
properties:
Expand Down
11 changes: 11 additions & 0 deletions config/samples/layout/single/glance_v1beta1_glance.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,17 @@ spec:
debug = true
glanceAPIs:
default:
overrides:
affinity:
spec:
requiredDuringSchedulingIgnoredDuringExecution:
- labelSelector:
matchExpressions:
- key: security
operator: In
values:
- S1
topologyKey: failure-domain.beta.kubernetes.io/zone
type: single
preserveJobs: false
replicas: 1
Expand Down
1 change: 1 addition & 0 deletions controllers/glanceapi_controller.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -815,6 +815,7 @@ func (r *GlanceAPIReconciler) reconcileNormal(
GetServiceLabels(instance),
serviceAnnotations,
privileged,
&instance.Spec.Override.APIAffinity,
)
if err != nil {
return ctrlResult, err
Expand Down
2 changes: 2 additions & 0 deletions go.mod
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -89,3 +89,5 @@ replace github.com/openstack-k8s-operators/glance-operator/api => ./api
// mschuppert: map to latest commit from release-4.16 tag
// must consistent within modules and service operators
replace github.com/openshift/api => github.com/openshift/api v0.0.0-20240830023148-b7d0481c9094 //allow-merging

replace github.com/openstack-k8s-operators/lib-common/modules/common => github.com/fmount/lib-common/modules/common v0.0.0-20241121121123-92d5e021a60f
Loading
Loading