-
Notifications
You must be signed in to change notification settings - Fork 36
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Do not create keystone endpoints when Glance is not available #633
Conversation
Unlike services like CinderVolume, where replicas:0 still sees an available api and scheduler, if the default GlanceAPI has replicas:0, - which is something suggested at day1 when the storage backend is still not available -, a keystone endpoint is created anyway in the catalog. That means that the glance service is discovered, the endpoint can be reached from an OpenShift point of view, but it fails returning a 503 if a request is made. This behavior might result confusing for the human operator, who thinks that there's a problem with the service, ignoring that is should still be configured. This patch fixes this problem by **not** creating the endpoints at all if no Glance replica is available. Signed-off-by: Francesco Pantano <fpantano@redhat.com>
[APPROVALNOTIFIER] This PR is APPROVED This pull-request has been approved by: fmount The full list of commands accepted by this bot can be found here. The pull request process is described here
Needs approval from an approver in each of these files:
Approvers can indicate their approval by writing |
maybe I read it wrong, but this will still make glance to be in |
Correct, it reconciles (like before) and reaches |
ack, I guess we need to revisit what we discussed a couple of months back if we need a condition which reflects the actual functional state in addition the the |
Do you want me to hold this change? It doesn't change the problem and the current state of things, but I'm ok to hold in case we need to make it part of a broader discussion. |
No, if @dprince agrees that this is a better then the current behavior I am fine to get it in and we follow up with the discussion on general improvement to reflect |
Would we then remove it if replicas get scaled down to 0. My issue with replicas wasn't that the endpoint existed, it was that I expected the default glance-api to be 1. |
Right, when |
@stuggi @dprince Note that Glance is a "special" component here: both API and the underlying engine are up or down at the same time, you can't have API up and running, but not accepting data/image upload. |
technically, I'd say the route behaves correct as it has no knowledge of what kind of service it servers. If there is no service running it can just return the 503.
Returning information to the user on what is missing would be beneficial. Not registering the endpoint probably also does not point to that the config is missing, right? But as you said in other discussion to proceed e.g. with ceph hci deployment, at least right now, we first need a ctlplane deployment, then do the ceph deployment and go back to the ctlplane and configure it as backend.
|
maybe then we could default to 1? |
I'm not sure about this: my concern is that you're writing this data in the database, and I don't think we have a way to clean it up automatically when we redeploy (and rolling out a new config doesn't restart everything from scratch). Also, the human operator is allowed to mess up w/ the |
I think it would just be wrong to create/delete the endpoint when there is a scale down to 0 and up again. And also the response does not point to that there is a config missing. What I did was not to add something to the DB, its just a static response from the custom deployment. for this I was just using an nginx deployment which returned
|
I agree that creating/deleting keystone endpoints might be seen as a workaround other than a real fix, and 503 is the Ingress default behavior that is does not depend on the application. |
I think my preference might be to just always create the endpoint. Seems simpler that way |
Closing this change as we gathered two main things:
We're going to follow up with a different patch, but thanks everyone involved for the suggestions and feedback! |
Unlike services like
CinderVolume
wherereplicas:0
still sees an availableapi
andscheduler
, if the defaultGlanceAPI
hasreplicas:0
, - which is something suggested at day1 when the storage backend is still not available -, akeystone
endpoint is created anyway in the catalog.This means that the
Glance
service can be discovered, the endpoint can be reached from anOpenShift
point of view, and it fails returning a503
if a request is made.This behavior might result confusing for the human operator, who thinks that there's a problem with the service, ignoring that is should still be configured.
This patch fixes this problem by not creating the endpoints at all if no
Glance
replica is available.