November 2023 guide site maintenance

[thibaudcolas]

👋 Are there any specific backend improvements to the site you would like to see happen? Please comment with your ideas / top priorities on how to make the site better.

---

@Tijani-Dia and I are going to be working on the site’s backend over the next few days. I managed to secure a bit of his time to help us keep the site in tip-top shape, in particular:

1. Maintenance – Keeping up with upgrades
2. Security – Making sure we can devolve editing on the site to as many people as possible
3. Versioning – Possible improvements to make this smoother
4. Translations – Same as above

Any suggestions welcome in those four areas, and any other suggestion for this site to keep being a good foundation for our work on documentation.

[thibaudcolas]

My top 3:

1. Wagtail 4.0 -> 5.1 upgrade + 5.2 #357
2. Automate guide versioning steps when creating a new release #305
3. Roadmap from a 100-page site to 100,000 pages site (10x more content, in 10x more languages, across 10x more versions)

[laymonage]

Search improvements

I'd like to suggest some improvements to the search mechanism. Right now the search logic requires complete words before suggesting results:

It would be nice if the search uses autocomplete or fuzzy matching instead. This might be as simple as changing .search() to .autocomplete() in the views, but I'm not very sure:

guide/apps/search/views.py

Line 18 in 4fb0682

Page.objects.live().filter(locale=Locale.get_active()).search(search_query)

guide/apps/search/views.py

Line 69 in 4fb0682

Page.objects.live().filter(locale=Locale.get_active()).search(search_query)

Also the non-AJAX version of the search view (accessible by pressing Enter in the search dialog) could use some styling improvements:

Not sure if we want to keep the view though.

Translations

Nothing in particular, but I remember reading some reports about some translation changes getting lost or desynced from the page tree. Not sure if that's been solved, Sævar might know more about this. It might be worth testing the translations (e.g. creating new pages, translating them to multiple languages and versions) to see if we find any issue. If not then it's probably safe to ignore.

---

No further thoughts at the moment, but I'll add more later if I find anything else.

[thibaudcolas]

Maintenance

• @allcaps top priority: maintenance
• Keep up with open issues and PRs
• Package upgrades
• Demo environment(s) / PR previews (Squash, GitPod)

Translations

• More translators involved, lower barrier to entry for translators (lots of interest, but hard with the current setup to get those people going)
• Automatic translation has gotten a lot better
  • Is it good enough to justify the effort?
  • @allcaps investigate simple_translation rather than localise intermediary table?
    • Translations instead submitted as revision data like any content change
    • Quick PoC / @allcaps project to explore this – perhaps with GSoC internship or similar in sight?
• @saevarom – On localise – fundamental issue w/ translation workflow when SF blocks change
  • No awareness from source language authors of the consequences of their editing actions
  • Translators lose a lot of work because of it
• Localise UX: hard to figure out which pages need translation work (report doesn’t seem accurate?)
• Overcast experience: modeltranslation
• Need wider Wagtail investment in localisation features?
  • And-or feedback to wagtail-localise to improve the UX, logic

Takeaway: PoC on new translation approach with more automation, lower barrier for entry

Versioning

• Initial plan we had with this package makes sense.
• Does it make sense with more content? Unsure.
• Important for us to preserve content because of links from within the CMS.
• Parallel with Read the Docs where updating past versions’ content is impossible.
• Images’ revisions / optimise process? Coen’s prior art on automation

---

Future

• GSoC project? Is this substantial enough?
• Improving translations in Wagtail in general
• Mentors: @saevarom "possibly", @allcaps "same", @Tijani-Dia in general yeah
• Involvement in core team

[lb-]

Wagtail development documentation links to the guide

This is not strictly maintainance on the Guide itself but more promotion of and getting more use out of the Guide for developers.

Currently we intentionally limit screenshots in Wagtail development docs both because it's Dev focused and for maintenance.

This means that sometimes it's hard to understand some features until you get them running locally while developing or load up.the bakery demo site.

However, we could add links in the Dev docs to the relevant sections in the Guide.

Here's the Wagtail issue with this suggestion.

wagtail/wagtail#11156

Any feedback welcome.

[Tijani-Dia]

I've been working on the following this week:

Wagtail, Django, Python upgrades
The website is now running with Wagtail 5.0, Django 4.2 and Python 3.11. As part of this, I've also upgraded wagtail-localize to 1.6.

Search improvements
We've made improvements on search, namely partial matches matches in AJAX forms as Sage suggested and full text search fixing already existing issues.

Custom 404
I've submitted a PR aiming to handle 404 more graciously by suggesting fallback links for the same page if it's available in other locales.

[lb-]

This is great news. Thank you.

[thibaudcolas]

Last few updates from Tidiane and I’s work:

• We now have a CSP set up (Images and third party scripts are not loading (CSP) #384 😓 )
• Tidiane researched performance issues with the site coming from wagtail-localize
• DebugBear says the site is fast as far as front-end performance.

The specific performance bottleneck Tidiane investigated is page translations (takes about 10s per page when doing them in bulk). He said:

There is a high number of queries (N + 1 and duplicates...) on that route but I'm having mixed signals so far. Some inefficiencies seem to come from Wagtail itself, while some other come from wagtail-localize. I'd need more time to really know what is the actual bottleneck.
there are various places to improve (query-wise) on wagtail-localize and it's tedious to investigate all of them manually

Working on tweaks to https://github.com/Tijani-Dia/dj-tracker to make this easier in the future