Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update narrative of the course #175

Open
wants to merge 105 commits into
base: main
Choose a base branch
from

Conversation

fnattino
Copy link
Contributor

@fnattino fnattino commented Oct 2, 2024

As promised during discussions offline, here is the PR that updates the material to include the changes that we have introduced in our fork of the lesson (for reference, that version of the material is deployed here).

More in the specific, this PR introduces the following changes:

  • We have updated the narrative of the course, focusing on the detection of wildfires and changing the area of interest from the Netherlands to Rhodes, Greece. This is to make the lesson more engaging. All episodes now are working towards the ultimate goal of detecting wildfires and estimating damages.
  • As vector data, we are now using infrastructure data from Open Street Map and administrative boundaries from GADM. The Dutch governmental data that we had previously introduced was too Netherlands-specific (and headers were in Dutch!).
  • We found that starting with the STAC data-access episode was not always ideal. Some of the participants we had in our workshops were interested in using Python to work with raster datasets, but not necessarily with satellite images. Also, we saw that working with APIs was too advanced for some of them. So we decided to make the chapter optional and make it possible to teach it as an overview/demo episode at the end of the workshop. We have thus included the raster datasets to the lesson figshare repository and updated the material accordingly, so that the default now is to work with raster data downloaded locally (and not streamed from AWS).
  • We have added the first version of a final episode on using ODC-STAC to create and work with data cubes (also optional and to be taught with “advanced” participants - thus in principle also addressing Stackstac episode #82 and consider teaching odc-stac for wide problems #105).

@rbavery very sorry for the huge PR, I have tried to split it in smaller chunks by cherry-picking updates on individual episodes. However, various commits seems to have covered multiple files, so it is really not possible to create episode-specific branches at this stage. Of course feel free to submit reviews in parts, if it helps, and please let me/us know if we can do anything else to simplify the review process..

Also cc'ing @Morrizzzzz and @rogerkuou

fnattino and others added 24 commits June 22, 2024 15:42
Co-authored-by: Maurice de Kleijn <mtm.de.kleijn@gmail.com>
Co-authored-by: Maurice de Kleijn <mtm.de.kleijn@gmail.com>
Co-authored-by: Maurice de Kleijn <mtm.de.kleijn@gmail.com>
Co-authored-by: Maurice de Kleijn <mtm.de.kleijn@gmail.com>
Co-authored-by: Maurice de Kleijn <mtm.de.kleijn@gmail.com>
Co-authored-by: Maurice de Kleijn <mtm.de.kleijn@gmail.com>
Co-authored-by: Maurice de Kleijn <mtm.de.kleijn@gmail.com>
Co-authored-by: Maurice de Kleijn <mtm.de.kleijn@gmail.com>
Co-authored-by: Maurice de Kleijn <mtm.de.kleijn@gmail.com>
Co-authored-by: Maurice de Kleijn <mtm.de.kleijn@gmail.com>
Co-authored-by: Maurice de Kleijn <mtm.de.kleijn@gmail.com>
…2024

Updating the material to the new narrative
…pring_2024

minor updates in epi 7, 8 and 10. Also exercise added in epi 8
…on_epi5_intro

added intro text about placement epi 5
@fnattino fnattino requested a review from rbavery October 2, 2024 20:57
Copy link

github-actions bot commented Oct 2, 2024

Thank you!

Thank you for your pull request 😃

🤖 This automated message can help you check the rendered files in your submission for clarity. If you have any questions, please feel free to open an issue in {sandpaper}.

If you have files that automatically render output (e.g. R Markdown), then you should check for the following:

  • 🎯 correct output
  • 🖼️ correct figures
  • ❓ new warnings
  • ‼️ new errors

Rendered Changes

🔍 Inspect the changes: https://github.com/carpentries-incubator/geospatial-python/compare/md-outputs..md-outputs-PR-175

The following changes were observed in the rendered markdown documents:

 01-intro-raster-data.md                            |  63 +--
 02-intro-vector-data.md                            |  30 +-
 03-crs.md                                          |  20 +-
 04-geo-landscape.md                                |  24 +-
 05-access-data.md                                  | 311 +++++++++----
 06-raster-intro.md                                 | 418 +++++++++++-------
 07-vector-data-in-python.md                        | 482 ++++++++++-----------
 08-crop-raster-data.md                             | 317 +++++++-------
 09-raster-calculations.md                          | 352 +++++++--------
 10-zonal-statistics.md                             | 168 +++----
 11-parallel-raster-computations.md                 | 213 +++------
 12-data-cube.md (new)                              | 195 +++++++++
 fig/E05/STAC-s2-preview-after.jpg (new)            | Bin 0 -> 70832 bytes
 fig/E05/STAC-s2-preview-before.jpg (new)           | Bin 0 -> 75077 bytes
 fig/E06/rhodes_multiband_80.png (new)              | Bin 0 -> 1033117 bytes
 fig/E06/rhodes_multiband_80_equal_aspect.png (new) | Bin 0 -> 861267 bytes
 fig/E06/rhodes_red_80_B04.png (new)                | Bin 0 -> 590536 bytes
 fig/E06/rhodes_red_80_B04_robust.png (new)         | Bin 0 -> 946323 bytes
 fig/E06/rhodes_red_80_B04_robust_nan.png (new)     | Bin 0 -> 865948 bytes
 .../rhodes_red_80_B04_vmin100_vmax2000.png (new)   | Bin 0 -> 880764 bytes
 fig/E06/rhodes_red_B04.png (new)                   | Bin 0 -> 801039 bytes
 fig/E07/greece_administration_areas.png (new)      | Bin 0 -> 64055 bytes
 fig/E07/greece_highways.png (new)                  | Bin 0 -> 26479 bytes
 fig/E07/rhodes_administration_areas.png (new)      | Bin 0 -> 17351 bytes
 fig/E07/rhodes_assets.png (new)                    | Bin 0 -> 45400 bytes
 fig/E07/rhodes_builtup_buffer.png (new)            | Bin 0 -> 18396 bytes
 fig/E07/rhodes_highways.png (new)                  | Bin 0 -> 80495 bytes
 fig/E07/rhodes_infra_highways.png (new)            | Bin 0 -> 46913 bytes
 fig/E08/dem.png (new)                              | Bin 0 -> 98203 bytes
 fig/E08/dem_matched.png (new)                      | Bin 0 -> 122237 bytes
 fig/E08/solution_exercise.png (new)                | Bin 0 -> 374927 bytes
 fig/E08/visual_clip.png (new)                      | Bin 0 -> 77146 bytes
 fig/E08/visual_clip_box.png (new)                  | Bin 0 -> 374262 bytes
 fig/E08/visual_large.png (new)                     | Bin 0 -> 197716 bytes
 fig/E09/NDVI-hist.png                              | Bin 7916 -> 14496 bytes
 fig/E09/NDVI-map.png                               | Bin 163222 -> 332276 bytes
 fig/E09/NDWI.png (new)                             | Bin 0 -> 314396 bytes
 fig/E09/NIR-band.png                               | Bin 167850 -> 276002 bytes
 fig/E09/custom-index.png (new)                     | Bin 0 -> 429445 bytes
 fig/E09/red-band.png                               | Bin 162461 -> 265509 bytes
 fig/E09/visual-burned-index.png (new)              | Bin 0 -> 357171 bytes
 fig/E10/zones_rasterized_xarray.png (new)          | Bin 0 -> 31334 bytes
 fig/E11/dask-graph.png                             | Bin 2311171 -> 94585 bytes
 fig/E12/NDVI-after.png (new)                       | Bin 0 -> 201397 bytes
 fig/E12/NDVI-after_point.png (new)                 | Bin 0 -> 201432 bytes
 fig/E12/NDVI-before.png (new)                      | Bin 0 -> 206500 bytes
 fig/E12/NDVI-time-series.png (new)                 | Bin 0 -> 25906 bytes
 index.md                                           |  44 +-
 md5sum.txt                                         |  27 +-
 setup.md                                           |  27 +-
 50 files changed, 1466 insertions(+), 1225 deletions(-)
What does this mean?

If you have source files that require output and figures to be generated (e.g. R Markdown), then it is important to make sure the generated figures and output are reproducible.

This output provides a way for you to inspect the output in a diff-friendly manner so that it's easy to see the changes that occur due to new software versions or randomisation.

⏱️ Updated at 2024-10-02 20:58:38 +0000

github-actions bot pushed a commit that referenced this pull request Oct 2, 2024
@rbavery
Copy link
Collaborator

rbavery commented Oct 3, 2024

@fnattino All good! I'll go episode by episode and submit reviews for each here. Thanks for the amazing work on this, excited to step through the new episodes.


```python
import pystac
items_loaded = pystac.ItemCollection.from_file("../data/stac_json/rhodes_sentinel-2.json")
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

would prefer this to point to the file that users downloaded in the preceding cell. since it's a fast cell to run and users who come across this first coding episode online will in all likelihood not be downloading the data beforehand.

Could we call out when users are expected to do that prior to a cell that requires imagery in the data folder instead?

Suggested change
items_loaded = pystac.ItemCollection.from_file("../data/stac_json/rhodes_sentinel-2.json")
items_loaded = pystac.ItemCollection.from_file("rhodes_sentinel-2.json")

We can visualize the calculated NDVI for the AoI at two given dates (before and after the wildfires) by selecting the date:

```python
ndvi_before = ndvi.sel(time="2023-07-13")
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this cell that triggers the computation presents a warning

/Users/ryanavery/test-dask-on-ray/.venv/lib/python3.11/site-packages/rasterio/warp.py:387: NotGeoreferencedWarning: Dataset has no geotransform, gcps, or rpcs. The identity matrix will be returned.
  dest = _reproject(

we should diagnose why georeferencing is not preserved and try to preserve it in this example so that geospatial metadata can be roundtripped when saving and loading the ndvi raster

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

though I'm testing this in an env called "dask-on-ray" this is just with vanilla dask and xarray. here are my versions

    "odc-stac>=0.3.10",
    "pystac>=1.11.0",
    "pystac-client>=0.8.3",
    "ray>=2.37.0",
    "rioxarray>=0.17.0",
    "torch>=2.4.1",
    "xarray>=2024.9.0",
    "leafmap>=0.38.3",
    "geopandas>=1.0.1",

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants