Skip to content

Commit

Permalink
paper.md fixes
Browse files Browse the repository at this point in the history
  • Loading branch information
trchudley committed May 11, 2024
1 parent 5b3ac35 commit c732b9e
Show file tree
Hide file tree
Showing 4 changed files with 15 additions and 12 deletions.
2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# pDEMtools

__Conveniently search, download, and preprocess ArcticDEM and REMA products__
__Conveniently search, download, and process ArcticDEM and REMA products__


[![conda-forge version](https://anaconda.org/conda-forge/pdemtools/badges/version.svg)](https://anaconda.org/conda-forge/pdemtools/badges) [![PyPI version](https://badge.fury.io/py/pdemtools.svg)](https://pypi.org/project/pdemtools/) [![Documentation Status](https://readthedocs.org/projects/pdemtools/badge/?version=latest)](https://pdemtools.readthedocs.io/en/latest/?badge=latest) [![Unit Tests](https://github.com/trchudley/pdemtools/actions/workflows/unit_test.yml/badge.svg)](https://github.com/trchudley/pdemtools/actions/workflows/unit_test.yml)
Expand Down
2 changes: 1 addition & 1 deletion docs/index.rst
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@ pDEMtools
.. :width: 1600
.. :height: 349
**Conveniently search, download, and preprocess ArcticDEM and REMA products.**
**Conveniently search, download, and process ArcticDEM and REMA products.**

.. figure:: _static/arcticdem_header.jpg
:width: 800
Expand Down
6 changes: 3 additions & 3 deletions paper/paper.bib
Original file line number Diff line number Diff line change
Expand Up @@ -209,7 +209,7 @@ @misc{morlighem_icebridge_2020
language = {en},
urldate = {2024-05-09},
publisher = {NASA National Snow and Ice Data Center Distributed Active Archive Center},
author = {Morlighem, M},
author = {Morlighem, Mathieu},
month = oct,
year = {2020},
file = {Snapshot:/Users/tom/Zotero/storage/79I9VV36/5.html:text/html},
Expand Down Expand Up @@ -242,7 +242,7 @@ @misc{morlighem_measures_2020
language = {en},
urldate = {2024-05-09},
publisher = {NASA National Snow and Ice Data Center Distributed Active Archive Center},
author = {Morlighem, M},
author = {Morlighem, Mathieu},
month = oct,
year = {2020},
file = {Snapshot:/Users/tom/Zotero/storage/XUEID76L/3.html:text/html},
Expand Down Expand Up @@ -307,7 +307,7 @@ @article{how_pypromice_2023
file = {Full Text PDF:/Users/tom/Zotero/storage/TCIQ56U9/How et al. - 2023 - pypromice A Python package for processing automat.pdf:application/pdf},
}

@article{nuth_co-registration_2011,
@article{nuth_coregistration_2011,
title = {Co-registration and bias corrections of satellite elevation data sets for quantifying glacier thickness change},
volume = {5},
issn = {1994-0416},
Expand Down
17 changes: 10 additions & 7 deletions paper/paper.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
---
title: 'pDEMtools: conveniently search, download, and preprocess ArcticDEM and REMA products'
title: 'pDEMtools: conveniently search, download, and process ArcticDEM and REMA products'
tags:
- Python
- digital-elevation-models
Expand Down Expand Up @@ -30,22 +30,24 @@ bibliography: paper.bib

---


# Summary

`pdemtools` is a Python package designed for accessing, processing, and handling high-resolution Digital Elevation Models (DEMs) of the polar regions from the ArcticDEM and Reference Elevation Model of Antarctica (REMA) projects. Tools are provided to search, filter, and download ArcticDEM and REMA data, as well as to fulfill common preprocessing requirements such as geoid correction, coregistration, and calculating terrain attributes. The aim is to allow users to quickly move beyond basic DEM data management into their own analyses.


# Statement of need

ArcicDEM and REMA are high-resolution, time-stamped 2-metre-resolution DEMs of the polar regions provided by the Polar Geospatial Center (PGC). They are extracted by applying stereo auto-correlation techniques [@noh_surface_2017] to pairs of submetre Maxar satellite imagery, including Worldview-1, Worldview-2, Worldview-3, and GeoEye-1, beginning in 2007 (ArcticDEM) or 2009 (REMA) and ongoing to the present day. Products are available as tens of thousands of time-stamped 'strips' [@porter_arcticdem_2022; howat_reference_2022-1] constructed from individual scene-pairs, or as a single mosaic [@porter_arcticdem_2023; howat_reference_2022] compiled from the combined stack of strips. Strips allow users to perform change detection by comparing data from different seasons or years, whilst mosaics provide a consistent and comprehensive product over the entire polar regions.
ArcicDEM and REMA are high-resolution, time-stamped 2-metre-resolution DEMs of the polar regions provided by the Polar Geospatial Center (PGC). They are extracted by applying stereo auto-correlation techniques [@noh_surface_2017] to pairs of submetre Maxar satellite imagery, including Worldview-1, Worldview-2, Worldview-3, and GeoEye-1, beginning in 2007 (ArcticDEM) or 2009 (REMA) and ongoing to the present day. Products are available as tens of thousands of time-stamped 'strips' [@porter_arcticdem_2022; @howat_reference_2022-1] constructed from individual scene-pairs, or as a single mosaic [@porter_arcticdem_2023; @howat_reference_2022] compiled from the combined stack of strips. Strips allow users to perform change detection by comparing data from different seasons or years, whilst mosaics provide a consistent and comprehensive product over the entire polar regions.

As Earth Science has moved into the 'big data' era, increasing amounts of Arctic- and Antarctic-focused resources are available as public, cloud-optimised datasets. New approaches are providing Python tools to act as combined API and processing tools, such as `icepyx` [@scheick_icepyx_2023] or `pypromice` [@how_pypromice_2023]. From 2022 (ArcticDEM v4.1 and REMA v2), the PGC DEM products are hosted as Cloud Optimised GeoTIFFs (CoGs) in a SpatioTemporal Asset Catalog (STAC) [@pgc_stac_2024], a standardised structure for cataloguing spatiotemporal data . However, the PGC STAC is not currently a dynamic STAC, which defines a RESTful API interface for searching and downloading datasets in response to user queries. This limits the ability of users to programmatically interact with ArcticDEM and REMA data in a quick and efficient manner. The `pdemtools` package has two aims: the first is to provide a Python-focussed alternative for searching and downloading ArcticDEM and REMA data, emulating dynamic STAC query tools such as `pystac` [@radiant_pystac_2024]; whilst the second is to provide commonly used processing functions specific to the needs of ArcticDEM and REMA users (a focus on ice sheet and cryosphere work), as well as the particular strengths of the ArcticDEM and REMA datasets (high-resolution and multitemporal).

The `pdemtools` `search()` tool and `load` module allow for convenient access to the ArcticDEM and REMA datasets. Mosaics can be downloaded from a one-line `tools.mosaic()` function, whilst the `search()` function allows for convenient filtering of a locally downloading ArcticDEM/REMA strip index according to variables such as date, region of interest, spatial coverage, temporal baseline, source sensors, accuracy, and cross-track data. Elevation models are returned as `xarray` DataArrays [@hoyer-xarray-2017] with geospatial metadata via the `rioxarray` extension - a standard format for storing and processing n-dimensional geospatial data within the Python community. By utilising standardised formats, the aim is to allow the user to quickly move beyond `pdemtools` into their own analysis in whatever format they desire, be that `xarray`, `numpy` or `dask` datasets, DEM analysis Python packages such as `xdem` [@glaciohack_xdem_2020] for advanced coregistration or `richdem` [@barnes_richdem_2016] for flow analysis, or exporting to geospatial file formats for analysis beyond Python.
The `pdemtools` `search()` tool and `load` module allow for convenient access to the ArcticDEM and REMA datasets. Mosaics can be downloaded from a one-line `tools.mosaic()` function, whilst the `search()` function allows for convenient filtering of a locally downloading ArcticDEM/REMA strip index according to variables such as date, region of interest, spatial coverage, temporal baseline, source sensors, accuracy, and cross-track data. Elevation models are returned as `xarray` DataArrays [@hoyer_xarray_2017] with geospatial metadata via the `rioxarray` extension - a standard format for storing and processing n-dimensional geospatial data within the Python community. By utilising standardised formats, the aim is to allow the user to quickly move beyond `pdemtools` into their own analysis in whatever format they desire, be that `xarray`, `numpy` or `dask` datasets, DEM analysis Python packages such as `xdem` [@glaciohack_xdem_2020] for advanced coregistration or `richdem` [@barnes_richdem_2016] for flow analysis, or exporting to geospatial file formats for analysis beyond Python.

After download, there exist a number of (pre-)processing steps that are near universally common in topographic analyses. These include geoid-correction, co-registration of time-series data, and/or the construction of terrain parameters such as hillshade, slope, aspect, and curvature. `pdemtools` contains pre-built functions to accelerate this processing, customised to ArcticDEM and REMA use cases. For instance, we include functions to extract geoid and bedrock masks directly from local versions of the Greenland and Antarctic BedMachine datasets [@morlighem_bedmachine_2017; @morlighem_icebridge_2020], although options for integrating other data are provided. Additionally, partial deriviatives of the surface used to calculate terrain parameters ($\frac{\partial z}{\partial x}$, $\frac{\partial z}{\partial y}$, $\frac{\partial^2 z}{\partial x^2}$, $\frac{\partial^2 z}{\partial y^2}$, $\frac{\partial^2 z}{\partial x \partial y}$) are calculated following @florinsky_computation_2009, as opposed to more common methods such as @zevenbergen_quantitative_1987.
The newer approach computes partial derivatives of elevation based on fitting a third-order polynomial, by the least-squares approach, to a 5 $\times$ 5 window as opposed to the more common 3 $\times$ 3 window. This is more appropriate for high-resolution DEMs: curvature over a 10 m window for the 2 m resolution ArcticDEM/REMA strips will lead to a local denoising effect that limits the impact of noise common in high-resolution photogrammetric products. These methods are also adapted into our co-registration routine, which otherwise follows the commonly used approach of [@registration_2011].
After download, there exist a number of (pre-)processing steps that are near universally common in topographic analyses. These include geoid-correction, co-registration of time-series data, and/or the construction of terrain parameters such as hillshade, slope, aspect, and curvature. `pdemtools` contains pre-built functions to accelerate this processing, customised to ArcticDEM and REMA use cases. For instance, we include functions to extract geoid and bedrock masks directly from local versions of the Greenland and Antarctic BedMachine datasets [@morlighem_bedmachine_2017; @morlighem_icebridge_2020, @morlighem_measures_2020], although options for integrating other data are provided. Additionally, partial deriviatives of the surface used to calculate terrain parameters ($\frac{\partial z}{\partial x}$, $\frac{\partial z}{\partial y}$, $\frac{\partial^2 z}{\partial x^2}$, $\frac{\partial^2 z}{\partial y^2}$, $\frac{\partial^2 z}{\partial x \partial y}$) are calculated following @florinsky_computation_2009, as opposed to more common methods such as @zevenbergen_quantitative_1987.
The newer approach computes partial derivatives of elevation based on fitting a third-order polynomial, by the least-squares approach, to a 5 $\times$ 5 window as opposed to the more common 3 $\times$ 3 window. This is more appropriate for high-resolution DEMs: curvature over a 10 m window for the 2 m resolution ArcticDEM/REMA strips will lead to a local denoising effect that limits the impact of noise common in high-resolution photogrammetric products. These methods are also adapted into our co-registration routine, which otherwise follows the commonly used approach of [@nuth_coregistration_2011].

Finally, we aim to grow `pdemtools` by implementing new methods developed by the ArcticDEM and REMA research community, specific to analysis of the Cryosphere. For instance, we currently include sea-level-filtering and iceberg detection routines outlined by [@shiggins_automated_2023], and invite community contributions or requests of other routines that will be of use to users of `pdemtools`. Ongoing research projects making use of `pdemtools` are applying it to the mapping of crevasses, ice cliff heights, and subglacial lakes, as well is the initiation of ice sheet models. It has also been used within training exercises at the 2024 Polar Geospatial Center Data Workshop, contributing to a growing international network of `pdemtools` users.
We aim to grow `pdemtools` by implementing new methods developed by the ArcticDEM and REMA research community. For instance, we currently include sea-level-filtering and iceberg detection routines outlined by [@shiggins_automated_2023], and invite community contributions or requests of other routines that will be of use to users of `pdemtools`. Ongoing research projects making use of `pdemtools` are applying it to the mapping of crevasses, ice cliff heights, and subglacial lakes, as well is the initiation of ice sheet models. It has also been used within training exercises at the 2024 Polar Geospatial Center Data Workshop, contributing to a growing international network of `pdemtools` users.


# Documentation
Expand All @@ -55,6 +57,7 @@ Package documentation is available at the `pdemtools` [readthedocs](https://pdem

# Acknowledgements

Chudley is supported by a Leverhulme Early Career Fellowship (ECF-2022-589). ArcticDEM data are provided by the Polar Geospatial Center under NSF-OPP awards 1043681, 1559691, and 1542736. REMA data are provided by the Byrd Polar and Climate Research Center and the Polar Geospatial Center under NSF-OPP awards 1543501, 1810976, 1542736, 1559691, 1043681, 1541332, 0753663, 1548562, 1238993 and NASA award NNX10AN61G. Computer time provided through a Blue Waters Innovation Initiative. DEMs produced using data from Maxar. We are grateful to the PGC for making index strips in the parquet format available from their website; to Connor Shiggins for his input on the iceberg detection function; to Andrew Sole and Adrien Wehrle for identifying and helping to resolve bugs in early versions of `pdemtools`; and to the many colleagues who helped to test the package in the early phase.
Chudley is supported by a Leverhulme Early Career Fellowship (ECF-2022-589). ArcticDEM data are provided by the Polar Geospatial Center under NSF-OPP awards 1043681, 1559691, and 1542736. REMA data are provided by the Byrd Polar and Climate Research Center and the Polar Geospatial Center under NSF-OPP awards 1543501, 1810976, 1542736, 1559691, 1043681, 1541332, 0753663, 1548562, 1238993 and NASA award NNX10AN61G. Computer time provided through a Blue Waters Innovation Initiative. DEMs produced using data from Maxar. We are grateful to the PGC for making index strips in the parquet format available from their website; to Connor Shiggins for his input on the iceberg detection function; to Andrew Sole and Adrien Wehrle for identifying and helping to resolve bugs in early versions of `pdemtools`; and to the many colleagues who helped to test this package in the early phases of development.


# References

0 comments on commit c732b9e

Please sign in to comment.