Modern Font Atlases

[colincornaby]

Plasma has issues around fonts in plTextFont. Specifically the way it renders font glyphs into a single texture doesn't leave room for the extended characters we'd need for more broad UTF support.

Modern games still typically use font atlases - but they've augmented them taking advantage of a few things:

• You don't need to preload every single glyph from a font into the font atlas. You only need to load the glyphs you actually need to render a frame.
• However - Once a glyph has been loaded into the font atlas, the chances of it being needed again is very high.
• You can allocate multiple font textures as needed.

Font atlas pages

Modern games use a linked list or array of font atlas textures. The textures are populated on the fly as characters are needed. There is a performance hit every time new glyphs are needed - but as the textures fill out with characters, this should happen less and less often.

This approach is also adaptive. A player playing in English will have English characters cached. A player playing in Japanese will have Japanese characters cached. And if languages are mixed - both character sets can be active simultaneously.

By not pre-rendering every character - the requirements for display of different languages is also simplified. Plasma doesn't need to pre-render every localized character - just the ones necessary to render a given frame.

If textures are lazily allocated - it can also result in performance and memory usage nearly identical to the existing client. A player who continues to only encounter ASCII characters will still only have the memory overhead of the ASCII character set.

Recycling pages

One complication of this approach is that if a player was exposed to too many characters during a play session, the number of atlas pages could grow unbounded and consume too much memory.

A simple approach could be to simply rank textures by how recently they have been used - and release the texture with the oldest access time when a bounds condition is hit. The number of textures could be bounded by the worst foreseeable case of how many unique characters would be needed to render a given frame. For example - if we believed that 1000 unique characters in a given frame represented our worst case - we'd only need to maintain storage for 1000 characters.

It would be extremely unlikely that a play session would also constantly churn the texture atlas pages in back to back frames. Each frame successively would need a large set of completely new and unique characters to force complete map regeneration between frames.

A more nuanced approach could be to release individual glyphs and recycle the regions of individual textures that contained those glyphs. This can be complicated because not all glyphs are the same width. Navigating the textures looking for a spot to insert a glyph would be more difficult (and becomes more analogous to a memory allocator looking for a location suitable for allocation a buffer of a certain size.)

Another complication of re-using textures themselves is that newer APIs like Metal/Vulkan/DX12 require explicit texture management for double/triple buffering. If a texture is changed in the current render frame, it will also affect any other previous frame currently in the render pipeline. This could make removing characters from font textures more tricky.

There are other trivial ways of forcing font atlas cleanup. When the player loads a new age - that could be used as a chance to purge all existing font atlas pages.

Example implementations

Because this is a common approach to UTF rendering in games - there are other example implementations to work off of. It would be ideal if we could handover text rendering to one of these libraries - but I haven't seen any so far that work across multiple rendering backends, or don't come with a whole lot of extra baggage.

• FTGL (GL2/Fixed function)
• FTGL-ES
• FreeTypeGL (Seems to use a single resizable texture - unsure if textures that large would work in DX9)

Alternative approaches

Really big textures

If we assume there is a maximum bound on text size and the number of characters needed simultaneously - we could just use a really large texture. When the texture runs out of room - the glyph renderer could simply reset back to the beginning of the texture and begin writing the oldest glyphs.

DX9 might be a limitation here - I'm unsure what the largest texture size is in DX9 for the hardware Plasma supports.

IMGUI takes the really large texture approach - but seems to also ask the developer to load entire character sets at once.

SDF

Signed distance field rendering allows one texture atlas to render at any font size. This would reduce the number of textures needed even further and allow more memory for more characters at once.

Signed distance fields would also allow the rendered font size to be considerably smaller - making for even more efficient use of texture memory to add more characters.

A Metal approach to SDF rendering is described here:
https://metalbyexample.com/rendering-text-in-metal-with-signed-distance-fields/

Real time creation of an SDF map could be more complicated. Most games opt to ship pre-rendered SDF map images. It could be too expensive to generate an SDF map in real time.

SDF might make it more possible to pre-render all characters in the unicode set needed instead of doing real time rendering because of the size efficiency - and not needing to ship pre-rendered files for each font size.

Bonus tasks

Better texture organization

There are a lot of algorithms out there for better texture packing with characters. A risk with dynamically adding characters naively is that we might pack them into a texture inefficiently and leave unused space.

I don't think this is a high priority for Plasma. And it could complicate recycling parts of the texture for new glyphs. But it could be worth looking into at a future point.

Feedback?

I'd like to get feedback on this issue. Particularly - the more I think about SDF, the more I wonder if we shouldn't be real time generating glyphs at all. There could be an alternative approach where we pre-ship a bunch of font atlases pre-sorted by character set, and we load them from disk in real time. Then Plasma would only be responsible for layout.

There are tools to generate SDF atlases for any Unicode range - like this on here:
https://github.com/astiopin/sdf_atlas

(Though it also notes it is still efficient enough to generate SDF at runtime.)

[dpogue]

hmmm, the current p2f fonts used for non-debug text are already stored as bitmap characters. I wonder if we could store SDF using the same format, which would allow us to stop needing one p2f file per font size.

Acknowledging however, that the p2f fonts are not an atlas, so building an atlas from the bitmaps might still be necessary.

[colincornaby]

One issue with internationalization is I don't known if the ttfs we use have international characters. I'm not sure if Trebuchet MS Bold contains Japanese characters. SDF files might make it easier to bake a combination of characters we need from multiple font sets.

I don't know what size of encoded SDF file we'd end up with though.

[dgelessus]

Thanks for writing this down. What you're suggesting for font atlases makes sense IMO - we definitely can't keep rendering everything eagerly if we expand the character set further (especially once we get to scripts with complex ligatures, where you have many more glyphs than characters).

The main issue right now is that plTextFont is practically not very relevant, because normal players only see it in exactly two places: the loading screen and avatar name tooltips. The KI and GUIs all use plFont instead, which is still completely unaccelerated and restricted to pre-rendered bitmap fonts. Before we invest too much work into improving the hardware-accelerated vector font rendering, we should figure out plFont can benefit from those improvements as well, because otherwise they won't make any practical difference.

Honestly, I'm still skeptical that we need hardware-accelerated text rendering at all, considering that the unaccelerated plFont has worked fine for KI chat for the last 20 years (even on much worse hardware than we have today). Personally, I would rather get rid of the hardware-accelerated rendering, because that needs special support in every rendering backend... But my impression might be wrong, especially at higher resolutions and refresh rates. It would be helpful to run some small performance tests to see how much of an improvement hardware-accelerated rendering makes.

One issue with internationalization is I don't known if the ttfs we use have international characters. I'm not sure if Trebuchet MS Bold contains Japanese characters.

Yes, this will be an issue once we go beyond Latin-based languages. This Microsoft documentation has some useful info on how to handle that. Unfortunately, it seems that Windows doesn't help much with choosing a fallback font - not sure how it is for other OSes.

[colincornaby]

Honestly, I'm still skeptical that we need hardware-accelerated text rendering at all, considering that the unaccelerated plFont has worked fine for KI chat for the last 20 years (even on much worse hardware than we have today). Personally, I would rather get rid of the hardware-accelerated rendering, because that needs special support in every rendering backend... But my impression might be wrong, especially at higher resolutions and refresh rates. It would be helpful to run some small performance tests to see how much of an improvement hardware-accelerated rendering makes.

The Metal/SDF tutorial makes a mention of software fonts, and it has an interesting bit that might get at the performance question:

One of the most flexible approaches to text rendering is dynamic rasterization, in which strings are rasterized on the CPU, and the resulting bitmap is uploaded as a texture to the GPU for drawing. This is the approach taken by libraries such as stb_truetype.

The disadvantage of dynamic rasterization is the computational cost of redrawing the glyphs whenever the text string changes. Even though much of the cost of text rendering occurs in the layout phase, rasterizing glyphs is nontrivial in its demand on the CPU, and there is no extant GPU implementation of font rasterization on iOS. This technique also requires an amount of texture memory proportional to the font size and the length of the string being rendered. Finally, when magnified, rasterized text tends to become blurry or blocky, depending on the magnification filter.

Plasma currently sidesteps this issue by using font maps so it doesn't have to re-rasterize individual glyphs. Doing the typesetting on the CPU might not be ideal - but it could be that rasterizing glyphs on the fly is really what they wanted to avoid.

This also may be a Windows/Mac difference. On Windows, text rasterization is hardware accelerated through DirectWrite. On macOS, there is no GPU font rendering (as the tutorial notes in the text above.)