SCIM users endpoint

[fflorent]

Context

As an IT asset administrator, I can create account of my users using a centralized solution like an SSO so they can log on Grist. It's quite convenient because I don't have to worry about creating accounts specifically for them.

Also Grist handles the update of Users when reconnect.

There are things the administrator cannot do though:

• assign users to Groups (like the owners of an team site);
• change immediately user information;
• delete the user when they are removed from the SSO;
• get the list of users or groups in a normalized way;
• ...

Proposed solution

SCIM is a standard proposed by the IETF through RFC7644 and RFC7643 which aims to through a simple Rest API provide solution for the above use cases.

Here is the abstract of the RFC7644 which introduces SCIM:

The System for Cross-domain Identity Management (SCIM) specification is an HTTP-based protocol that makes managing identities in multi-domain scenarios easier to support via a standardized service.
Examples include, but are not limited to, enterprise-to-cloud service providers and inter-cloud scenarios. The specification suite seeks to build upon experience with existing schemas and deployments, placing specific emphasis on simplicity of development and integration, while applying existing authentication, authorization, and privacy models. SCIM's intent is to reduce the cost and complexity of user management operations by providing a common user schema, an extension model, and a service protocol defined by this document.

This PR provides the implementation of SCIM for Users Resources (Group will come in a future PR), and supports:

• All the basic actions (POST /Users/, PUT /Users/:id, GET /Users/, GET /Users/:id and DELETE /Users/:id).
• The /Schemas, /ServiceProviderConfig, /ResourceTypes endpoints;
• The /Me endpoint (it takes advantage of the id you returned in the authentication middleware);
• The POST /Bulk endpoint
• The POST /Resources/.search by using the Filters (actually to use them, you must have to fetch all the Resources from the DB, the filtering is done in JS, which is probably fine for small projects, I would just be cautious when using big databases + an ORM);
• There are some error utilities to help you;
• The PATCH /Resources/:id endpoint! It reads a resource using the egress method, applies the asked changes, and calls the ingress method to update the record ;
• The pagination

To do that, I take advantage of two libraries: scimmy and scimmy-routers. Scimmy is lightweight (0 dependency), and scimmy-routers will also be dependency-free be in a future version (reported in that issue and already fixed).

Two variables are introduced:

• GRIST_ENABLE_SCIM to let the administrator enable the scim API (defaults to false);
• GRIST_SCIM_EMAIL to let the administrator specify a user that is allowed, they are granted rights to do any operation using SCIM (just like the administrators of GRIST_DEFAULT_EMAIL and GRIST_SUPPORT_EMAIL);

Assumption regarding the SCIM implementation

• the ID is the technical ID in the database;
• SCIM's userName corresponds to the normalized email (logins.email), the SCIM emails corresponds to the displayEmail;
• I don't allow more than an email to be passed (as the Grist code requires currently IIRC);
• Anonymous users cannot call any SCIM endpoint;
• Authenticated non-admin and non-GRIST_SCIM_EMAIL users can only request these endpoints;

How to test manually?

I can document the API in grist-help upon request (otherwise I will do that after this PR is merged).

You may:

1. run a local Grist server setting either the GRIST_DEFAULT_EMAIL, GRIST_SUPPORT_EMAIL or GRIST_SCIM_EMAIL env variable without omitting to enable SCIM using GRIST_ENABLE_SCIM:

GRIST_SCIM_EMAIL="you@example.com" GRIST_ENABLE_SCIM=1 yarn start

2. Generate a bearer for you@example.com
3. then you may start using SCIM:

$ export BEARER=<paste the bearer here>
$ curl -H 'Content-Type: application/scim+json' -H "Authorization: Bearer $BEARER" -X POST -d '{"schemas": ["urn:ietf:params:scim:api:messages:2.0:SearchRequest"], "sortBy": "userName", "sortOrder": "descending"}' https://localhost:8484/api/scim/v2/Users/.search

I described some examples of the SCIM API usage here (they need to be adaptated for the context of Grist): https://github.com/scimmyjs/scimmy-routers/blob/8ffa2221b542054c3f0cfb765ea6957f29ebe5e1/example/README.md#play-with-the-scim-server

Limitations of the current implementation

• The user bearer not renewed automatically, so it does not comply with the request of limiting their lifetime (source);
• Only an administrator (with the GRIST_DEFAULT_EMAIL or the support account) or the user with GRIST_SCIM_EMAIL are allowed to make operations on resources (other user are limited to use /Me).
• A dedicated account (like GRIST_SCIM_EMAIL) is required to have access to the endpoints, which should have their API key generated. I considered instead having a bearer directly set in an env variable, but I rejected the idea because it would have been rejected by the Authorizer.
• The /Me endpoint implementation seems partial (issue);
• I forgot to add tests for the pagination… I am noting to do that;
• The SCIMMY and scimmy-routers libraries lack of typing support, so you may see many any types or some casts until that is fixed (issue and issue);
• [now fixed] The Content-Type must be application/scim+json, currently application/json is not supported (will be fixed in the next scimmy-routers release)

Documentation

I opened this PR in draft to start documenting SCIM: gristlabs/grist-help#434

It can be previewed here:

• the documentation (including an intro): https://deploy-preview-434--grist-help-preview.netlify.app/install/scim/
• the API reference: https://deploy-preview-434--grist-help-preview.netlify.app/api/#tag/scim

Related issues

It partly implements #870 (Users resource only for now).

Has this been tested?

• 👍 yes, I added tests to the test suite
• 💭 no, because this PR is a draft and still needs work
• 🙅 no, because this is not relevant here
• 🙋 no, because I need help

[fflorent]

It feels like a bit dirty to me to create a fake Scope, but I fear it is hard to get rid of it from UsersManager.deleteUser, especially because the deleteOrg needs it.

Any opinion/idea on that?

[hexaltation]

Thanks for this huge job.
I think It could be great to have a documentation/scim.md file documenting at least :

• What part of the SCIM RFCs are not yet implemented,
• What part of the RFCs are "adapted" to work with grist current logic

It could be a huge time saver for administrators of instances when doing the interconnection.

[hexaltation]

Is there a clear policy about keeping or removing trailing new-lines in the project ?

[fflorent]

The linter says nothing 🤷

[fflorent]

Thanks @hexaltation for you feedback!

Thanks for this huge job. I think It could be great to have a documentation/scim.md file documenting at least :

* What part of the SCIM RFCs are not yet implemented,

* What part of the RFCs are "adapted" to work with grist current logic

It could be a huge time saver for administrators of instances when doing the interconnection.

I am rather planing on documenting the API in the grist-help repository and probably add a new page to explain how to setup SCIM and what is currently supported.

[fflorent]

I think we should not authorize the update and the deletion of Special Ids.

There also is a edge case for GRIST_DEFAULT_EMAIL, should we allow the IdP administrator propagate a change of their email (which would the owner of that email would lose the installation admin right)? I guess it's the responsibility of the IdP admin to be sure of the impacts, also it is easy to fix.

Do you have any thoughts about these cases?

[fflorent]

Same problem regarding the Special Ids

[fflorent]

I tested it against the SCIM plugin for Keycloak (currently being developed). Of course you can have users with a userName and an email different.

A look at the RFCs have not revealed me that the userName is required to be taken into consideration by the service provider, despite being mandatory. I would probably just log a warning telling that both values are different and that the userName is ignored in such case.

[fflorent]

I could test my development against an existing IdP with SCIM support, which raised some interesting questions. I expect them to be not very hard to fix.