[R] BFGS versus L-BFGS-B

[Ben Bolker]

Brian Tsai <btsai00 <at> gmail.com> writes:

> 
> Hi all,
> 
> I'm trying to figure out the effective differences between BFGS and L-BFGS-B
> are, besides the obvious that L-BFGS-B should be using a lot less memory,
> and the user can provide box constraints.
> 
> 1) Why would you ever want to use BFGS, if L-BFGS-B does the same thing but
> use less memory?

  L-BFGS-B is a bit more finicky: for example, it does not allow
non-finite (infinite or NA) return values from the objective function,
while BFGS does (although neither does during the initial function evaluation).
I don't know offhand of other differences, although speed may differ.

> 2) If i'm optimizing with respect to a variable x that must be non-negative,
> a common approach is to do a change of variables x = exp(y), and optimize
> unconstrained with respect to y.  Is optimization using box constraints on
> x, likely to produce as good a result as unconstrained optimization on y?

  It depends.  If the optimal solution is on the boundary (i.e. x=0)
then optimization on the transformed variable (I think you mean y=exp(x)
above?) will work very badly. On the other hand, if the solution is in 
the interior then transforming sometimes works even better -- for example,
the goodness-of-fit surface may be closer to quadratic (which sometimes
has advantages in terms of inference) with the transformed than the
untransformed parameter.

  Ben Bolker