Rendering assets in mdBook

[BilalM04]

After playing around with different ways of getting the assets rendered, I noticed a couple things. Turns out assets do in fact work the way it is currently structured. On the local server that mdBook launches, the assets will not be rendered as it is out of scope of the server. However, when deployed to GitHub pages they will render properly.

The reason assets don't render locally is because mdbook serve launches the server in the book directory; thus, everything outside of it is out of scope. So when we try to link to something like ../../../, it cannot access anything outside of the book directory.

However, since we deploy the entire github-pages branch, which includes all the assets, the mdBook project is able to render those assets as they are hosted together. I tested this by downloading a zip of the github-pages branch, manually adding in an mdBook SRS for one of the examples, and hosting it locally. It seemed to work without the need for any symbolic links, copies of assets, or changing the file paths.

Although that works when the mdBook projects are hosted with the required assets, the down side of this approach is that the assets will not render when locally hosted using the command that mdBook provides (as the server is restricted to the book directory). @balacij mentioned that leaving it the way it is, is not feasible as we need to be able to render the assets locally for testing purposes, to which I agree.

What Didn't Work

mdBook has support for custom preprocesses before building the book, allowing you to run your own commands before the book is built. Drasil has the knowledge of where the assets are located, and where they should be located in order to be rendered. The idea behind this approach is that each example's book.toml would call a generic script to create symbolic links by passing in the file paths as parameters.

When the script was run manually, everything ran perfectly. However, when added to the book.toml as a custom preprocess, it was very broken, creating random hidden directories and just simply not working. I also tried by adding a simple command to create a symbolic link directly inside the book.toml, without the use of any scripts, which still did not work. There is also a lack of support online regarding custom preprocesses, so @balacij and I agreed to avoid them entirely.

Possible Solutions

Here are some ideas for possible approaches @balacij and I came up with.

Script to create symbolic links inside src

This approach involves writing a script, which will be called by the makefile, which goes through the datafiles and traceygraphs directories and creates symbolic links for all the .png/.jpg/.svg inside of the respective mdBook projects. The downside of this is that it would only work for Drasil examples. For a Drasil user generating their own example, they would have to manually go in and add the assets into the src directory.

Each example generates a CSV file

This approach involves each example generating a CSV (or some sort of text file) containing information of the original file paths of the assets, and where they should be located inside src. Then in the makefile, we would call a generic script passing in the CSV file as an argument to create the symbolic links. The problem here again is that a user using Drasil would either need to have this generic script, or manually add the assets.

Launch our own local server

The problem of assets not being rendered locally is due to the way mdBook launches the server using the mdbook serve command, which is detailed above. Instead of using mdbook serve in the makefile, we can rather just run mdbook build then use python or something else to launch the server. I tested with python, and it seems to work on both WSL Ubuntu and native Windows Powershell, rendering the assets locally. This was the command:

python3 -m http.server 8080 -d /path/to/your/directory

[JacquesCarette]

Symbolic links AFAIK are not entirely portable. Launching our own server seems like overkill. So the second solution seems best.

[balacij]

As discussed in our meeting on July 8th, 2024, this is the solution we'll go with, for now at least.

[smiths]

During the meeting (#3834) we also discussed generating a Makefile with the mdbook server command (so that users don't have to remember the specific syntax). @BilalM04 can you create an issue for this, if you haven't already?

[BilalM04]

I have developed two prototypes showcasing possible solutions for rendering assets in mdBook. One being the CSV approach that that we discussed in #3834. The other being a Makefile approach which leverages the features from #3841.

CSV Approach

This approach is prototyped in #3848. In this approach, a CSV file is generated by each example that maps the actual location of the assets (specified in SRSDecl and extracted from the ChunkDB) to a location which can be rendered locally by mdBook. A generic script is called by the main Makefile in order to copy the assets to the desired location specified by the generated CSV file.

Makefile Approach

This approach is prototyped in #3847. From #3841, each generated mdBook project will also generate a makefile to both build the project and launch the server. Since we are already generating a makefile, we can easily add the copying of assets into this generated makefile, rather than going through the whole CSV and generic script ordeal. Since the paths to the assets are also relative to the location of the makefile, this removes the hassle of cd-ing into directories in the scripts/makefiles. This approach does not include the CSV and generic script, the main Makefile can simply cd into the project directory and use the generated makefile to build the examples.

Combination of Both

We can also always do a combination both. We can generate both a CSV file and include the copying of assets in the generated makefiles. We will then have to decide which method to use in our main Makefile which builds all the examples.

---

I personally like the Makefile approach. With the CSV approach, the generated makefiles will just build and launch the server without the assets. With the Makefile approach, copying over the assets will be a dependency for building and launching the server, ensuring assets are always rendered when locally hosted (if done through the generated makefile).

[smiths]

@BilalM04, I didn't realize you were going to copy the assets to the folder for each example. Was that always the plan? I thought we were trying to come up with an approach that didn't require us to copy the assets to multiple locations. I'm sure I missed something in the discussion.

[BilalM04]

@JacquesCarette mentioned in #3835 (comment) that symbolic links are not entirely portable, so I avoided them. The copy commands can always be replaced with commands to create symbolic links instead. At the moment, either copying or creating symbolic links seem to be the only two feasible options.

Additionally, the copies/symbolic links will only be created when building and launching the server of the mdBook project. They will not appear in stable or anywhere else.

[smiths]

Thank you for the quick reply @BilalM04.

Yes, that makes sense. Not adding the copies to stable is a key that I was missing. They only have to be locally copied if someone wants to view the mdbook.

I agree that we don't want symbolic links.

[BilalM04]

In #3842, we decided we will proceed with the CSV approach.