Whistler Houdini1

Agenda

• Specifically, we’re very interested in being able to implement some less-critical layout features in JavaScript in Servo and would like to chat with you about good candidate features and the practical subset of proposed Houdini features that we might want to prototype along the way. Two features ideas that we were thinking of are grid and SVG. We assume that we would at least have to implement some of the proposed CSS parser extensions and possibly some of the extensions allowing us to run JS during layout, but we’d like to have a deeper conversation before we invest any time in it.

What should we implement?

• larsberg: Idea is to find some subset of the houdini specs we can implement that would let us JS-implement some longer-tail features. e.g., Grid
• Simon: drafts: http://dev.w3.org/houdini/ [http://dev.w3.org/houdini/css-properties-values-api/ is the only spec with anything in it]
• wiki: https://wiki.css-houdini.org/
• Simon: At least CSS Properties and values and CSS Layout (which is the core of grid). Not sure the Box Tree API is relevant to Grid.
• jack: Is box tree the one that's basically a formalization of one browser's implementation.
• simon: No, it's read-only.
• jack: What can you get?
• larsberg: Actual margins, etc.
• simon: Anonymous boxes, sizing, etc. Less limited than getBoundingClientRects.
• jack: Requirements draft or set of references being used to develop these specs?
• simon: Lots of informal discussions, but no requirements document yet.
• mbrubeck: Brian Kardell gave some high-level motivations at https://dev.opera.com/articles/houdini/
• jack: What caused each of these specs to happen?
• larsberg: Simon's notes have the meeting minutes - http://www.mail-archive.com/dev-servo@lists.mozilla.org/msg01131.html
• simon: In Sydney, we had conversations about each of these specs. Regardless, for Grid layout we probably don't need Box Tree. Just Custom Layout and CSS Properties and values.
• larsberg: Not CSS Parser?
• simon: Entirely in JS requires Properties and values. But since we're doing this internally, we can do some parts in Rust and some in JS instead. In particular, properties parsing in the rust code is not hard, so maybe don't need that API for now.
• jack: So just implement the properties directly for Grid?
• SimonSapin: Why would we want to implement something like Grid in JS instead of Rust?
• larsberg: 1) To let people work on them who wouldn't otherwise, and 2) To manage complexity by modularizing features into JavaScript code, and try to limit the layout engine complexity to what we have today even as specs proliferate. The first point argues more for standardization. The second one could be beneficial for Servo even if it never becomes a web standard.
• jack: Blink implements some CSS properties in JS already, for example.
• SimonSapin: If you want to be able to do this completely outside the layout engine (no lines of Rust code), we need the CSS properties and values.

Does this mean script on layout?

• gw: Would we need a JS engine running on the layout thread?
• jack: We need to figure that out. It's not clear what they're envisioning.
• SimonSapin: Most of the JS APIs are planning to use some kind of worker.
• jack: But when does the layout worker start working? How does it coordinate efforts with the Rust layout code?
• SimonSapin: There's no concrete proposal for any of that yet.
• jack: The dream API for Servo would be that you write a traversal, and Servo schedules and calls your traversal functions. bubble_widths, assign_widths, etc.
• gw: Or not necessarily those; could allow arbitrary traversals and you just specify preorder, postorder, etc.
• jack: But we don't want to encourage people to use ones that don't parallelize.
• nrc: If LayoutWorker doesn't have access to the DOM, what does it lay out?
• pcwalton: API that allows you to lay out the flow.
• simon: Access to the read-only DOM, perhaps, as input.

GPU painting

• pcwalton: I had an interesting idea about painting... you could have a geometry shader that paints CSS borders.
• gw: And outputs triangles?
• pcwalton: Yeah.
• gw: Geometry shaders are still slow on mobile, though.

Next steps?

• larsberg: I don't see any show-stoppers here.
• jack: First thing: expose some version of our three functions to JS for callbacks and see how that goes. Then, we can integrate it into our current scheduler. Probably have to do some magic DOM stuff.
• simon: Definitely not a single callback from the Microsoft people. Definitely a few (compute intrinisic width; etc.).
• jack: Could have as many passes as we need.
• pcwalton: Each pass has overhead, though, to discover all your childern or otherwise just walk the whole tree (blowing the whole cache).
• jack: Sure, just pointing out that our infra handles it.
• larsberg: So, these callbacks are NOT called sequentially/blocking from our layout, right?
• jack: Yes. The callbacks are implemented in LayoutWorker threads, and are async.
• simon: Hard to be general enough with this API to allow any kind of element but parallelizable.
• jack: Still have to compute your height and width of the box.
• pcwalton: Impression I got is that you can have an API called top-down and bottom-up but as long as you can access your descendents by simply ignoring all of your descendents but the root. Hard case here is the Cassowary constraint-solver - it's a big dynamic programming algorithm to do layout.
• jack: Imagine there's lots of work you need to do. The inside of your box is layout code in Rust. So, bubble widths may just use the default pass.
• gw: Cassowary's incremental, though, right?
• mbrubeck: In the case where it is parallelizable, do we spin up one LayoutWorker per core?
• jack: Same thread pool we schedule work on now. Instead of calling Rust, call JS.
• mbrubeck: So, can you parallelize those? In JS, you can't have two threads accessing the same data structure, though.
• jack: The methods have to be similar to what we do in Rust; can't ask for children in top-down.
• pcwalton: If we have one JSContext per layout worker thread. Then, suppose you have a flow and the JS layout sets an expando property on it for internal state. Now, the kids, if they are on different threads, can't all access those same JS objects on different JSContexts.
• jack: But the flow tree is thread-safe!
• pcwalton: Flow tree is Rust and has specific threads. You can't transplant JS objects to other threads except via "brain transplant." If you store random properties on flows...
• jack: The question isn't if you can set the property; it's that you can't access them from other threads. Nobody else should be able to see the properties ...
• larsberg: The nodes you've completed are visible - in top-down you have problems.
• pcwalton: Children need to be able to read the info from parents and may get scheduled on other threads. Then you can't touch them from multiple threads. Maybe brain transplant will help? But I can't imagine we want to do that. And we would be breaking the GC and JIT constraints.
• simonsapin: Maybe SharedArray, but that’s only chunks of bytes, not objects...
• pcwalton: I think we need a simpler API.
• jack: Maybe TypedObject and have everything defined in advance.
• mbrubeck: How do you package up data to pass up the tree in the JS case?
• jack: It happens behind the scenes, because as you traverse the flow tree you can access the children.
• mbrubeck: But, if you are doing work, each time you process a child, you compute a width, getting a number, and at the root you want that info.
• jack: Each node of the flow tree will have a pointer to it.
• pcwalton: You probably also need a treewalking API to traverse your kids and access their data.
• jack: Your own void* (but safe!) data. With restrictions on how you can access parents or children.
• mbrubeck: So, you can't do JS-y stuff like appending to a shared array that gets passed around?
• jack: Right, you're passing stuff into the flow tree.

Overhead

• gw: Minimum overhead on a JSRuntime?
• jack: Too high because they don't do this in Gecko
• gw: Don't they have workers?
• jack: In Portland, the SM team said they plan to work on the memory usage

Restrictions

• jack: So, how do we restrict what code is allowed to write?
• mbrubeck: WebWorkers can only do postMessage with POD
• larsberg: How about just require the JS code serialize the results back into the Rust instance?
• mbrubeck: Exactly what I'm suggesting. They do JSON.serialize.
• jack: So TypedObjects similarly have requirements that you can't store arbitrary JS objects in?
• pcwalton: At worst, a string, but I think there are some other things we can do. Could do something like adoptNode, though I'd prefer not to go that route.

Something simpler than grid to do first

• larsberg: What's a smaller thing we could do to try out this infrastructure work? Marquee?
• jack: Images... hrm, no layout.
• mbrubeck: Want something with a tree. Maybe select or other form controls?
• simon: Are we also doing shadow dom?
• pcwalton: No. This and web components are probably not where we want to go right now.
• jack: Similar targets (create new tags), but not clear where shadow dom or web components are really going right now.
• pcwalton: Let’s do simple first-steps on Grid. Worried about doing something else.
• simon: Worried that we won’t test the whole API.
• larsberg: Just want something to check the architecture first. Can expand, but I’d like to have something that we can do in a quarter to check whether the architectural ideas even work.

Others to talk with?

• larsberg: Whom else do we need?
• pcwalton: bobby holley for some of this SM stuff.
• jack: Can we get support for these changes from other engines?
• larsberg: It sounds like it from the editors. Edge is interested in working on this; the others are not as active yet. We should include some of the editors in reviewing or implementing this work.

Any other next steps

• pcwalton: Probably factoring the script stuff so that layout can also spin up an engine. Just getting hello world appearing in layout will be a lot of technical work. After that, running layout algorithms is just mashing bits.