[R] Automatic differentiation in R

[Gabor Grothendieck]

On Thu, Jul 23, 2009 at 11:07 AM, Ben Bolker<bolker at ufl.edu> wrote:
>
>
>
> nashjc wrote:
>>
>>  >> Gabor G. wrote
>>  >>  "R does not currently have AD (except for the Ryacas package
>>  >>  which can do true AD for certain simple one line functions, i..e.
>>  >>  input the function and output a function representing its
>>  >>  derivative); however, for specific problems one can get close
>>  >>  using deriv and associated functions or the approach explained
>>  >>  below using rSymPy:
>>  >> ...
>>
>> As the instigator of Finlay's participation in this work, I probably
>> didn't express clearly enough the contribution Gabor has made to get as
>> far as he has with Ryacas and rSympy, which may show another pathway for
>> AD/Symbolic diff. development. At UseR all conversations seemed more
>> rushed than I'd like.
>>
>> Gabor showed Ravi Varadhan and I a way to get some derivatives via his
>> tools that "worked". We need to play with this a bit more to see how
>> general it could be -- Gabor is very fair in his post that some work is
>> needed for each instance. On the other hand, if analytic gradients were
>> straightforward, we wouldn't be exchanging posts about them.
>>
>> The clear issue in my mind is that users who need gradients/Jacobians
>> for R want to be able to send a function X to some process that will
>> return another function gradX or JacX that computes analytic
>> derivatives. This has to be "easy", which implies a very simple command
>> or GUI interface. I am pretty certain the users have almost no interest
>> in the mechanism, as long as it works. Currently, most use numerical
>> derivatives, not realizing the very large time penalty and quite large
>> loss in accuracy that can compromise some optimization and differential
>> equation codes. I'll try to prepare a few examples to illustrate this
>> and post them somewhere in the next few weeks. Time, as always, ...
>>
>> However, the topic does appear to be on the table.
>>
>> JN
>>
>>
>
> On my wish list for the bbmle package (providing "mle2", which extends
> stats:::mle
> in various ways) is symbolic (not automatic) differentiation in the subset
> of cases
> where users specify the model as a formula (and the guts of the formula are
> susceptible to simple differentiation by deriv()).  For example, if someone
> specifies
>
>  mle2(cover~dbeta(shape1=exp(a*rain),shape2=exp(b*rain)),
> start=list(a=1,b=1))
>
> (I'm not sure this actually makes much sense as a statistical model,
> but whatever) and I have provided information to R about the formula
> for the Beta distribution in terms of shape1 and shape2, then it is
> "straightforward"
> (i.e. it would take me a while to write some non-horrible code, but I'm sure
> it's
> doable) to use the chain rule to generate a function that computes the
> derivatives
> symbolically (still not as efficient as auto-differentiation, but a lot
> better than
> finite differences ...)

Note that its not true that AD is more efficient on all problems.