Add redirect for e.g. /mapr/gene/?value=PAX7

[will-moore]

This provides support for URLs that correspond to omero-mapr URLs containing key and value of the form:

/mapr/gene/?value=PAX7

and also the "home" page for each key, e.g.

/mapr/gene/

We handle them in this PR at e.g.

/idr-gallery/mapr/gene/?value=PAX7
/idr-gallery/mapr/gene/

But, since /idr-gallery/ is deployed for IDR as the "root" app (so that the /idr-gallery/ home page is /) the /idr-gallery/mapr/gene/?value=PAX7 page is actually at /mapr/gene/?value=PAX7 and replaces mapr at the same URL (no redirect needed).

This requires that idr-gallery is listed in omero.web.apps BEFORE omero-mapr.

changes made to idr-deployment (to be copied to playbooks):

I have deployed this change (ordering of omero.web.apps) on idr-testing.
For each omeroreadonly-1-4 (and omeroreadwrite):

$ sudo vi /opt/omero/web/config/omero-web-apps.omero
$ sudo service omero-web restart

omero-web-apps.omero:

# add application
config append -- omero.web.apps '"omero_iviewer"'
config append -- omero.web.apps '"idr_gallery"'
config append -- omero.web.apps '"omero_figure"'
config append -- omero.web.apps '"omero_mapr"'
...

Also needed to disable mapr caching on the idr-testing proxy server

$ sudo vi /etc/nginx/conf.d/proxy-cache.conf
# comment out line: "~mapr/*" omeromapr;
$ sudo systemctl reload nginx

Alternatively, we can use /mapr2gallery/ instead of /mapr/ to avoid uncertainty about caching, since the same behaviour is provided at both URLs. E.g.

/mapr2gallery/gene/?value=PAX7
/mapr2gallery/gene/

In these examples, the key is an identifier of a mapr config: e.g. gene config is:

{"menu": "gene", "config": {
  "default": ["Gene Symbol"], "all": ["Gene Symbol", "Gene Identifier"],
  "ns": ["openmicroscopy.org/mapr/gene"], "wildcard": {"enabled": true},
  "case_sensitive": "true", "label": "Gene"
}}'

We need to know the correct Key to use for the Search engine. In this case there are 2 possible keys: "Gene Symbol", "Gene Identifier" so we use the searchengine to find keys that match the Value e.g. https://idr.openmicroscopy.org/searchengine/api/v1/resources/image/searchvalues/?value=PAX7
This allows us to pick "Gene Symbol" as the correct key.

Then we redirect to idr_gallery/search/?key=Gene+Symbol&value=PAX7&operator=equals using the equals operator since we want to show only exact matches.

To test: See testing sheet (Sheet 2) at https://docs.google.com/spreadsheets/d/1cPtsMSMY-5Rd-ob8xA9XCJrmogO6glGt5kM9_8ISONY/edit?gid=1203223646#gid=1203223646

• Pick a mapr category at idr-testing e.g. "Gene" from the top link. If it doesn't open in idr-gallery search page, then add a cache-busting query. e.g. https://idr-testing.openmicroscopy.org/mapr/gene/?_=_
• This should open e.g. /search/?key=Gene+Symbol&operator=contains with the primary attribute e.g. Gene Symbol selected.
• Perform any search to find images with that key-value pair in the right panel.
• Copy the URL(s) E.g. gene Identifier /mapr/gene/?value=YFL009W and Gene Symbol /mapr/gene/?value=CDC4 and open in a new tab (or just click on the link)
• If you see a mapr page instead of the search page, then replace /mapr/ in the URL with /mapr2gallery/ or add e.g. &test=abc123.
• Should now get redirected to the search page that shows the same results that mapr does (in IDR).

Previously tested:

• /mapr/gene/?value=Cbx5
• /mapr/gene/?value=ENSMUSG00000009575 (needed bc52d14)
• /mapr/organism/?value=Mus%20musculus - minor numbers mismatch E.g. compare https://idr.openmicroscopy.org/search/?key=Organism&value=Mus+musculus&operator=equals Mus musculus found 135159 images in 34 experiments and https://idr.openmicroscopy.org/mapr/organism/?value=Mus%20musculus Mus musculus (134972) [34]
• /mapr/cellline/?value=C127
• /mapr/compound/?value=MLN8054
• /mapr/phenotype/?value=CMPO_0000278
• /mapr/phenotype/?value=motile%20lamellae
• /mapr/sirna/?value=ACTB
• /mapr/sirna/?value=ACTB%20pool

[dominikl]

Tested a few too. Seems to work fine 👍 With that PR we could redirect /mapr/ to /mapr2gallery/ without the need for "or" queries via /search/.

[will-moore]

In order to remove mapr searches, we also need to redirect mapr API requests (direct users to searchengine page at https://idr-testing.openmicroscopy.org/searchengine/apidocs/#/ )
Seb suggested the appropriate response is 410 - see https://developer.mozilla.org/en-US/docs/Web/HTTP/Status/410

Looking at existing nginx config... - everything under /etc/nginx/conf.d/ is included...

$ grep -r "include" /etc/nginx/
/etc/nginx/nginx.conf:    include       /etc/nginx/mime.types;
/etc/nginx/nginx.conf:    include /etc/nginx/conf.d/*.conf;

https://superuser.com/questions/1256843/nginx-return-custom-410-page-with-map
https://serverfault.com/questions/1135572/how-to-setup-410-permanently-deleted-on-nginx
https://blog.avirtualhome.com/create-custom-410-error-page-nginx/

less /etc/nginx/conf.d/proxy-default.conf

server {
 ...
    location = /50x.html {
        root   /usr/share/nginx/html;
    }

According to link above, this "works"

   error_page 410 /410.html;

    # works
    location ~^/gone-location-1 {
        return 410;
    }

So with https://idr.openmicroscopy.org/mapr/api/gene/
let's add a new file at /etc/nginx/conf.d/proxy-gone.conf

server {
    error_page 410 /410.html;
    location = /410.html {
        root   /usr/share/nginx/html;
    }
    location ^~ /mapr/api/gene {
        return 410;
    }
}

Create a 410.html...

$ sudo vi /usr/share/nginx/html/410.html

<html>
<body>
mapr api is gone. Use searchengine instead https://idr.openmicroscopy.org/searchengine/apidocs/#/
</body>
</html>

After reload that didn't work..
Testing redirect instead:
sudo vi /etc/nginx/conf.d/proxy-redirect.conf

# Nginx redirection map
map $request_uri $redirect_uri
{
    default /;
    /mito/webclient/?show=screen-1 /webclient/?show=screen-1100;
...
    /mapr/api/antibody/ /searchengine/apidocs/
}

sudo systemctl reload nginx
This had no effect, https://idr-testing.openmicroscopy.org/mapr/api/antibody/ same JSON response as before.

Seb "try restart" instead of reload...

$ sudo systemctl restart nginx
Job for nginx.service failed because the control process exited with error code.
See "systemctl status nginx.service" and "journalctl -xeu nginx.service" for details.

Oh :(

[will-moore]

$ sudo systemctl status nginx
× nginx.service - nginx - high performance web server
     Loaded: loaded (/usr/lib/systemd/system/nginx.service; enabled; preset: disabled)
     Active: failed (Result: exit-code) since Thu 2024-11-14 14:19:04 UTC; 2min 4s ago
   Duration: 3d 3h 36min 36.373s
       Docs: http://nginx.org/en/docs/
    Process: 53345 ExecStart=/usr/sbin/nginx -c /etc/nginx/nginx.conf (code=exited, status=1/FAILURE)
        CPU: 10ms

Nov 14 14:19:04 test125-proxy.novalocal systemd[1]: Starting nginx - high performance web server...
Nov 14 14:19:04 test125-proxy.novalocal nginx[53345]: nginx: [emerg] unexpected "}" in /etc/nginx/conf.d/proxy-redirect.conf:18

Fixed missing ; in /etc/nginx/conf.d/proxy-redirect.conf
Restarted nginx - OK!

Editing sudo vi /etc/nginx/conf.d/proxy-redirect.conf e.g. this line: works OK

/mito/webclient/?show=screen-1 /webclient/?show=screen-1100;

But new line added doesnt

/mapr/api/antibody/ /searchengine/apidocs/

because it's not included in...

[sbesson@test125-proxy ~]$ grep -B 1 -A 1 redirect_uri /etc/nginx/conf.d/proxy-default.conf 
    location ~ ^/(mito|tara|pgpc|study)($|/) {
        return 302 $redirect_uri;
    }

Let's add mapr to that list...
$ sudo vi /etc/nginx/conf.d/proxy-default.conf

location ~ ^/(mito|tara|pgpc|study|mapr/api)($|/) {
        return 302 $redirect_uri;
    }

Now https://idr-testing.openmicroscopy.org/mapr/api/antibody/ redirects to https://idr-testing.openmicroscopy.org/searchengine/apidocs/ but all other mapr/api urls redirect to https://idr-testing.openmicroscopy.org/ (need /mapr/api/config to NOT be redirected.

So, undo the |mapr/api addition to redirect above and add new block below in /etc/nginx/conf.d/proxy-default.conf

    location ~ ^/mapr/api/(antibody|gene|compound|sirna|omap|phenotype|organism|orf|cellline|protein)($|/) {
        return 302 /searchengine/apidocs/;
    }

After reststart, this is working for all those /mapr/api/ urls.

[will-moore]

Instead of adding proxy-gone.conf which got ignored, lets add that to proxy-default.conf
replacing the redirect

    error_page 410 /410.html;
    location = /410.html {
        root   /usr/share/nginx/html;
    }
    location ~ ^/mapr/api/(antibody|gene|compound|sirna|omap|phenotype|organism|orf|cellline|protein)($|/) {
        return 410;
    }

It works!
Let's improve the page at
sudo vi /usr/share/nginx/html/410.html (based on the 50x.html)

<!DOCTYPE html>
<html>
<head>
<title>Error</title>
<style>
html { color-scheme: light dark; }
body { width: 35em; margin: 0 auto;
font-family: Tahoma, Verdana, Arial, sans-serif; }
</style>
</head>
<body>
<p>mapr/api pages have been removed.</p>
<p>
  Please use searchengine instead. See
  <a href="https://idr.openmicroscopy.org/searchengine/apidocs/">https://idr.openmicroscopy.org/searchengine/apidocs/</a>
</p>
</body>
</html>

[will-moore]

We want to recognise ALL /mapr/api/ URLs for 410 response except /mapr/api/config/ which is still used elsewhere for now (e.g. rendering popup panel in idr-gallery).

E.g. /mapr/api/autocomplete/... should be included, as well as /mapr/api/autocomplete/gene/ etc.

This should work...

sudo vi /etc/nginx/conf.d/proxy-default.conf

    location ~ ^/mapr/api/(?!config).*($|/) {
        return 410;
    }

sudo systemctl restart nginx

This seems to be working.
https://idr-testing.openmicroscopy.org/mapr/api/gene/ is 410 response and so is https://idr-testing.openmicroscopy.org/mapr/api/autocomplete/gene/

But https://idr-testing.openmicroscopy.org/mapr/api/config/ still works!

[will-moore]

Deployed on idr-testing and tested and approved as part of #47