General state of AD

[rowanG077]

I love auto differentation. And now I actually have a use case for it. So I really like to use the facilities in Futhark for this. But the documentation gave me pause:

Warning: This is an experimental feature. It has several known bugs. Documentation is very sparse. You probably don't want to use this unless you have the soul of a hero.

Since AD has now been in Futhark for a couple of years is this still true at this point?

[athas]

In retrospect it was probably a mistake to make it sound so scary, because it was never really all that fragile. It is quite well tested. It is certainly no longer "experimental" in that we want to take it out, and I really enjoy working on it, too. The most "experimental" part of it is that we do not systematically benchmark its performance in CI, but I've been meaning to do that. It also is not supported in the interpreter (which makes exploratory programming more annoying), but #2153 adds that, and needs more or less only cosmetic fixes in order to merge.

I don't recall any very serious bugs anymore. The main restriction is more or less the same as also applies to flattening, in that sequential loops with very dynamic memory behaviour, which are also nested inside parallel operations, will probably not produce code that works with the GPU backends.

Documentation is still very sparse, but this is partly because I don't actually know what should be documented. I'm reluctant to write an entire basic guide to AD, and I'm uncertain what Futhark-specific quirks are worth mentioning.

I don't think you need to be a hero to use Futhark's AD anymore, but you should probably still expect a certain roughness. I think polish will not happen until it sees use, however, and then that warning has been somewhat counterproductive.

[athas]

AD now works in the interpreter, we improved the documentation, and I started work on a utility library intended to make it more approachable: https://github.com/diku-dk/autodiff

[rowanG077]

Awesome! That helps me a lot with rapid development!

[athas]

Please let me know what you want to use it for, and how it can be improved. I find AD to be very fun to work on, but I am very much at the level of "forging a hammer" rather than "having that many nails to pound in".

[rowanG077]

I'm writing a variant of ICP (not really ICP anymore but it's very close to the concept) which tries to find objects in pointclouds using custom "point-to-model" functions instead of doing point to point matching. The model can have many parameters. I use AD to automatically find the derivatives of each specific "point-to-model" function for the ICP loop. The idea beeing that these point-to-model functions are much faster then large kdtree lookups.