@gdalle wrote in #389 (comment)
In terms of terminology, I would change this to pushforward, mirroring DI conventions and your choice of pullback in reverse mode.
Essentially:
|
general case |
special case |
| forward mode |
pushforward |
derivative1 (pushforward of 1) |
| reverse mode |
pullback |
gradient (pullback of 1) |
@willtebbutt replies
Ah, right. I chose this quite intentionally to be consistent with the docs. They discuss Frechet derivatives and their adjoints, and never use / define pushforward / pullback. I should probably actually change all uses of gradient and informal uses of the term pullback (eg. in rules in Mooncake) to be adjoint.
Agreed -- it makes sense to make the comparison clear. We've got the as-yet-to-be-written Mooncake.jl-API section of the tutorial in the docs, which will be the perfect home for it.
@gdalle wrote in #389 (comment)
@willtebbutt replies
Footnotes
maybe not necessary in Mooncake itself, since DI takes care of it ↩