Merge pull request #585 from Ouranosinc/geoserver-combined

.readthedocs.yml

@@ -11,7 +11,7 @@ formats:
   #- pdf
   #- epub
 python:
-  version: "3.7"
+  version: "3.8"
   install:
     - requirements: requirements-sys.txt
     - requirements: requirements-doc.txt
@@ -21,4 +21,3 @@ python:
 #      path: .
 #      extra_requirements:
 #        - docs
-  system_packages: true

CHANGES.rst

@@ -1,4 +1,4 @@
-.. explicit references must be used in this file (not references.rst) to ensure they are directly rendered on Github
+    .. explicit references must be used in this file (not references.rst) to ensure they are directly rendered on Github
 .. :changelog:
 
 Changes
@@ -9,7 +9,21 @@ Changes
 `Unreleased <https://github.com/Ouranosinc/Magpie/tree/master>`_ (latest)
 ------------------------------------------------------------------------------------
 
-* Nothing new for the moment.
+Features / Changes
+~~~~~~~~~~~~~~~~~~~~~
+* Add support of RESTful API endpoints (i.e.: ``ServiceAPI``) under ``ServiceGeoserver`` using ``Route`` resources
+  (fixes `#584 <https://github.com/Ouranosinc/Magpie/issues/584>`_).
+  Requires the `Service` to be configured either with the default ``configuration``,
+  or by explicitly setting ``api: true``. When a HTTP request is sent toward a `Service` typed ``ServiceGeoserver``,
+  any non-`OWS` request (i.e.: `WFS`, `WMS`, `WPS`) will default to the resolution handling of typical ``ServiceAPI``.
+  This can be used notably to access the ``/web`` and ``/ogc`` endpoints of a `GeoServer` instance.
+
+Bug Fixes
+~~~~~~~~~~~~~~~~~~~~~
+* Fix `UI` rendering of the `Permission` label list under a `Service` edition page when a large amount of possible
+  permissions is applicable. This was notably the case of ``ServiceGeoserver`` that combines permissions of multiple
+  `OWS`-based services, which where going out of bound of the UI page.
+* Fix `UI` scrollbars going over the `Permission` titles in the `User` and `Group` permission edition pages.
 
 .. _changes_3.34.0:
 

Makefile

@@ -564,12 +564,18 @@ check-security-code-only: mkdir-reports  ## run security checks on source code
 .PHONY: check-docs-only
 check-docs-only: check-doc8-only check-docf-only check-links-only	## run every code documentation checks
 
+# FIXME: temporary workaround (https://github.com/PyCQA/doc8/issues/145 and https://github.com/PyCQA/doc8/issues/147)
+# 		configuration somehow not picked up directly from setup.cfg in python 3.11
+#		setting 'ignore-path-errors' not working without the full path (relative 'docs/changes.rst' fails)
+CHECK_DOC8_XARGS := --ignore-path-errors "$(APP_ROOT)/docs/changes.rst;D000"
+
 .PHONY: check-doc8-only
 check-doc8-only: mkdir-reports		## run PEP8 documentation style checks
 	@echo "Running PEP8 doc style checks..."
 	@-rm -fr "$(REPORTS_DIR)/check-doc8.txt"
 	@bash -c '$(CONDA_CMD) \
 		doc8 --config "$(APP_ROOT)/setup.cfg" "$(APP_ROOT)/docs" \
+			$(CHECK_DOC8_XARGS) \
 		1> >(tee "$(REPORTS_DIR)/check-doc8.txt")'
 
 # FIXME: move parameters to setup.cfg when implemented (https://github.com/myint/docformatter/issues/10)

config/providers.cfg

@@ -10,15 +10,42 @@
 #
 # Parameters:
 # -----------
-#   url:        private URL of the service to be created
-#   title:      pretty name of the service (real name is the section key)
-#   public:     parameter passed down to Phoenix for service registration
-#   c4i:        parameter passed down to Phoenix for service registration
-#   type:       service type to use for creation, must be one of the known Magpie service types
-#               (see: magpie.services.SERVICE_TYPE_DICT)
-#   sync_type:  service synchronization type, must be one of the known Magpie service sync-types,
-#               often equals to 'type' (see: magpie.cli.SYNC_SERVICES_TYPES)
-#   hooks:      list of request processing hooks for the service
+#   url:            private URL of the service to be created
+#   title:          pretty name of the service (real name is the section key)
+#   public:         parameter passed down to Phoenix for service registration
+#   c4i:            parameter passed down to Phoenix for service registration
+#   type:           service type to use for creation, must be one of the known Magpie service types
+#                   (see: magpie.services.SERVICE_TYPE_DICT)
+#   sync_type:      service synchronization type, must be one of the known Magpie service sync-types,
+#                   often equals to 'type' (see: magpie.cli.sync_services.SYNC_SERVICES_TYPES)
+#   configuration:  advanced custom configuration for service type that support it (see details in section below)
+#   hooks:          list of request processing hooks for the service (see details in section below)
+#
+# Configuration (requires Magpie>=3.21.0 minimally, or more recent version as relevant for specific features)
+# -------------
+#   Some services allow custom configuration settings to slightly modify their behaviour for handling requests.
+#   This can be used to enable/disable a subset of functionalities offered by a given service, or to control special
+#   logic of request properties for the service. The specific configuration format differs for each service type, and
+#   is only supported for certain cases listed below. More configuration details are provided in the documentation.
+#
+#   - type: geoserver   (see also: https://pavics-magpie.readthedocs.io/en/latest/services.html#servicegeoserver)
+#     configuration:
+#       wfs: true|false   # allows use of OWS WFS requests with Workspaces and Layers
+#       wms: true|false   # allows use of OWS WMS requests with Workspaces and Layers
+#       wps: true|false   # allows use of OWS WPS requests with Workspaces and Processes
+#       api: true|false   # allows use of REST API requests with OGC API and Web UI endpoints
+#
+#   - type: thredds    (see also: https://pavics-magpie.readthedocs.io/en/latest/services.html#servicethredds)
+#     configuration:
+#       skip_prefix: "<path>/<sub>"   # path prefix to skip (strip) before processing the rest of the request path
+#       file_patterns:                # patterns to map different path variations to a same file resource
+#         - "<regex>"
+#       metadata_type:                #  path prefix to resources to consider as BROWSE-able metadata
+#         prefixes:
+#           - "<regex|path>"
+#       data_type:                    #  path prefix to resources to consider as READ-able data
+#         prefixes:
+#           - "<regex|path>"
 #
 # Hooks: (requires Magpie>=3.25.0, Twitcher>=0.7.0)
 # ------
@@ -112,7 +139,7 @@ providers:
     title: geoserver
     public: true
     c4i: false
-    type: wfs
+    type: geoserver
     sync_type: wfs
 
   geoserver-web:

docs/conf.py

@@ -215,7 +215,8 @@ def ignore_down_providers():
     "https://pcmdi.llnl.gov/",  # works, but very often causes false-positive 'broken' links
 ] + ignore_down_providers()
 linkcheck_anchors_ignore = [
-    r".*issuecomment.*"   # GitHub issue comment anchors not resolved
+    r".*issuecomment.*",  # GitHub issue comment anchors not resolved
+    "defusedxmllxml",  # not found because of GitHub dynamic links
 ]
 
 # Add any paths that contain templates here, relative to this directory.

docs/services.rst

@@ -571,16 +571,18 @@ administrator intends to only make use (for the moment) of :term:`WFS` functiona
         url: http://localhost:1234
         type: geoserver
 
-        # customizable configuration (enable desired OWS request handlers)
-        # all OWS are enabled by default if no configuration is provided
+        # customizable configuration (enable desired OWS/REST request handlers)
+        # all OWS/REST services are enabled by default if no configuration is provided
         configuration:
           wfs: true
           wms: false
           wps: false
+          api: false
 
-This would make sure that request parsing and access to :term:`WMS` and :term:`WPS` endpoints is disabled, but leaves
-the :term:`Resource` definitions available for use at a later time if the administrator decides to eventually make use
-of them. For example, the administrator could decide to start using :term:`WMS` as well without any further change
+This would make sure that request parsing and access to :term:`WMS`, :term:`WPS` and any REST :term:`API` endpoints
+are disabled, but leaves the :term:`Resource` definitions available for use at a later time if the administrator
+decides to eventually make use of them.
+For example, the administrator could decide to start using :term:`WMS` as well without any further change
 needed other than updating this :term:`Service` custom configuration and applying :term:`Permissions <Permission>`
 specific only to :term:`WMS`.
 All other :term:`Applied Permissions <Applied Permission>` to existing :term:`User`, :term:`Group` and :term:`Resource`
@@ -590,7 +592,8 @@ the :term:`WFS` to :term:`WMS` request handlers.
 .. note::
     Custom configuration can be provided With either the `providers.cfg`_ (as presented above), in
     a :ref:`config_file` as described in greater lengths within the :ref:`configuration` chapter,
-    or by providing the ``configuration`` field directly within the API request body during :term:`Service` creation.
+    or by providing the ``configuration`` field directly within the :term:`API` request body during
+    :term:`Service` creation.
 
 
 Service Synchronization

magpie/owsrequest.py

@@ -48,7 +48,6 @@ def ows_parser_factory(request):
 
 
 class OWSParser(object):
-
     def __init__(self, request):
         self.request = request
         self.params = {}
@@ -94,7 +93,6 @@ def _get_param_value(self, param):
 
 
 class OWSPostParser(OWSParser):
-
     def __init__(self, request):
         super(OWSPostParser, self).__init__(request)
         self.document = xml_util.fromstring(self.request.body)