Producing fonts from FontLab Vfj sources using Fontmake

[vv-monsalve]

One of the requirements to publish a font in Google Fonts has to do with having a scalable font production, which implies the use of repeatable build scripts to produce the fonts using Fontmake.
As currently it only works with Glyphs or Ufo sources, for projects built in Fontlab, like Castoro, we have created new Ufo sources (exporting them from FontLab) and a build script to produce the new fonts. The build also includes a ttfautohint and some post-production processes to prevent some common Fontbakery fails.
The build is still a work in progress. I still need to solve the inclusion of the Open Type Layout currently on .vtp.

The following posts reproduce some ideas discussed in a private conversation with @tiroj, @khaledhosny, @twardoch, @kateliev, @tphinney, @davelab6 and @m4rc1e mainly on how would be best to work from FontLab files (but also on the sideway, on how to include Open Type Layout from VOLT to the process)

[vv-monsalve]

@tiroj:

With regard to the inclusion of OTL, there are a couple of options I can think of.

One would be to use Volto to convert from the .vtp sources to .fea files. The output should be pretty straightforward for fonts like Castoro:
https://github.com/TiroTypeworks/TiroTools

The other, clunkier option, would be to do TTX dump of the compiled OTL tables and fuse them into the fonts as part of the build process.

Of course, the best option would be to be able to consume .vtp files natively in the fontmake process, which would not only mean not having to convert and potentially maintain two sets of OTL sources for the same project, but would also enable us to bypass some limitations of .fea based tools for more complex projects requiring e.g. contextual GPOS.

[vv-monsalve]

@khaledhosny:

There is currently no VTP compilers other than VOLT itself (though I think it should be possible to write one on top of FontTools which already have a VTP parser), so to build with fontmake VTP sources will need to be converted to FEA as John mentioned.

It should be possible to also make fontmake work with VFJ files directly (I almost did this as a try, but couldn’t figure the syntax of glyph outlines in VFJ and couldn’t find any documentation).

[vv-monsalve]

@davelab6:

Adam any links for vfj docs?
Marc any thoughts about how to handle the build expectations for this project?

[vv-monsalve]

@twardoch:

Unfortunately, the VFJ format is not yet documented. The glyph layers can potentially have a very complex structure if one uses groups, element references, and various live filters. Documenting it will take some effort.

Vassil Kateliev (adding in cc) has written a handy lib
https://github.com/kateliev/vfjLib that provides basic object-oriented access to VFJ structures, and I think we should make efforts to extend it (I’d encourage Khaled & John that the TiroTools scripts switch to using it).

vfjLib is only a "formal" parser now, but it could potentially serve as a higher-level transformer at least for some simpler VFJs that don't use the advanced stuff (this is kind of similar as the notion of smart components and other non-plain structures in glyphsLib).

I've added two scripts to the Lato repo

https://github.com/latofonts/lato-source/tree/master/tools

One is a .vfpy that exports a series of VFJs into DesignSpace+UFO. When you call the FontLab app with a .vfpy in command-line, or open the .vfpy in FontLab, FontLab executes the script and optionally quits. Note that this must be Py 2.7 for now.

The other script is a Py 3 script that uses fontmake to make a series of fonts.

So — the export can be automated, but it requires FontLab (running on macOS, Windows or Linux with Wine) for .vfpy to run.

There is now also a new documentation for the FontLab Python API at

https://fontlabcom.github.io/fontlab-python-docs/

Overall, it's better to work with TypeRig as it's more high-level (rather similar to the now-archived
https://github.com/trufont/tfont — do you know why this was abandoned ? ).

Much of the logic of the lower-level FL Py API is a bit convoluted, but Vasdil is tirelessly working on his TypeRig proxy wrappers that make it more pythonic.

I do have lots of VFJs and if you have suggestions as to how to create a JSON schema from a bunch of them, and then annotate it for documentation, I'll be much-obliged :)

[vv-monsalve]

@khaledhosny:

Unfortunately, the VFJ format is not yet documented. The glyph layers can potentially have a very complex structure if one uses groups, element references, and various live filters. Documenting it will take some effort.

I’m currently concerned only about the representation of contour nodes, I don’t know how to map it to UFO points or to draw it to a FontTools pen.

I see things like:

"301 -8 s",
"337 -8 358 16 358 48 s",
"358 79 337 107 302 107 s",
"265 107 243 80 243 48 s",
"243 15 262 -8 c”

But I don’t what s and c stand for, and what it means when they are not present. I need to convert this to the equivalent UFO points. I guess I can convert to UFO in FL and then try to reverse engineer this, but authoritative description would be definitely better.

The other complex structures aren’t an immediate issue for me, since the fonts I’m working with don’t use any (which isn’t dissimilar from glyphsLib not supporting smart components, for instance, as it didn’t prevent people from using it).

Vassil Kateliev (adding in cc) has written a handy lib
https://github.com/kateliev/vfjLib that provides basic object-oriented access to VFJ structures, and I think we should make efforts to extend it (I’d encourage Khaled & John that the TiroTools scripts switch to using it).

I gave it a look, but it did not provide much on top of parsing the json data into objects, so couldn’t justify the extra complication of an additional dependency. My code does some parsing, but the important part is the manipulation it does on top of it.

So — the export can be automated, but it requires FontLab (running on macOS, Windows or Linux with Wine) for .vfpy to run.

That is interesting, but the requirement here is something that can be automated with fontmake, so requiring FL is not an option.

[vv-monsalve]

@twardoch:

I understand, and I'd be keen to figure out how documentation can be provided.

I'm pointing out that you can have a predictable scripted export from VFJ to DS+UFO. This export could be associated with git pushes. In other words, the author reliably produces "dual sources", one in a proprietary format (VFJ), one in a standardized format (DS+UFO).

This is exactly what some projects do. As long as the DS+UFO sources are reliably published alongside the VFJ sources, you can consider them a "interchangable cache", which then build systems can consume.

After all, this is exactly what UFO was designed for. As long as I can get from .sfd, .glyphs or .vfj to DS+UFO, I can use the benefits of fontmake and automated building. No?

But of course it'd be great to be able to have an alternative GUI-less implementation of a more-less full VFJ parser and "compiler" (bridge).

[vv-monsalve]

@davelab6:

Automated with a fontlab dependency is fine.

[vv-monsalve]

@tiroj:

Automated with a fontlab dependency is fine.

For Castoro, I am working on getting everything we need for the build, including OTL, into the VFJ sources, so whether the UFO files are manually exported or automated the content should be the same. One flag to raise is that Viviana says the designspace file generated by FL7 for Castoro needs post-editing, so I wouldn't rely on full automation with .vfpy yet.

For some other projects, things get more complicated because of the nature of the GPOS complexities once we start dealing with features using anchor attachments, contextual GPOS, and other things for which I rely on VOLT. I can sometimes convert to .fea format with Volto if I take care not to include mechanisms that .fea can't handle, but I can't make the VFJ files the nominal source for OTL in the same way as I can with a simple font like Castoro, because a) editing the complex lookups in .fea code is inhumane, and b) the OTL panel in FL7 only permits one set of features and lookups per font, and I need them per master. So as soon as I have a variable project with complex OTL, I can't bring the latter via VFJ/UFO ... hence the path we're having to use for STIX Two Text. [Longer term solution for this is the kind of extension of the FL7 anchors model that Yuri mentioned recently on Twitter and/or VOLT-like interface for complex GPOS editing.]
The interesting case will be the Tiro Indics. For editing purposes, the OTL work has to be done in VOLT, and I'm going to publish those project files regardless of the build path, because they're also the best way for someone to examine the OTL and understand how it has been made. But because the Indic fonts are not variable, and only have a single weight plus italic for each family, I can with some care make .fea sources with Volto and enable a VFJ->UFO->ttf/otf scripted build very similar to what I am not working on for Castoro.

[vv-monsalve]

@twardoch:

Incidentally, let me explain what my Lato .vfpy+.py does:

1. The VFJ has 2 masters
2. .vfpy exports a DS+UFO (2 masters)
3. Then it exports all instances as UFO into the same folder, overwriting the UFO masters.
4. There is another DS treated as source, pre-made. It uses 3 masters, which it takes from the instances, because I want Regular neutral which is not a source master. In fact there is yet another 2-master DS.
5. .py calls fontmake several times to export a 2-master VF, a 3-master VF, OTF instances and TTF instances. The cubic to quad conversion is by fontmake (cu2qu) which is always compatible

--
Basically I used the exported DS once and then modified it and I treat it as an extra source now. But I could just as easily read the DS in my. py, postprocess it and continue with fontmake

[vv-monsalve]

@twardoch:

BTW, I found that when I assign GDEF classes in FL7 to only some bases and some marks in Glyph panel, then make a simple

table GDEF {
} GDEF;

in FEA, then the mark feature writer from fontmake (ufo2ft) only makes mark/mkmk for those glyphs. So while Lato has tons of anchors also on precomposed letters, I’m only building mark/mkmk for the base letters. This is not to bloat the size and also because the anchors on the precomposed letters are not perfect.

Overall, I made that build path for Lato but I’ll “universalize” it a bit, so others can use it.

[davelab6]

Thanks for posting this @vv-monsalve