[R] nonlinear model: pseudo-design matrix

[Spencer Graves]

Hi, Murray:

	  I just got 54 hits from RSiteSearch("numerical differentiation"), the 
first of which mentioned a function "numericDeriv" WITH a warning 
(http://finzi.psych.upenn.edu/R/Rhelp02a/archive/55462.html).

	  hope this helps.
	  spencer graves
###################
Hi Spencer,

I will try some of the ways you suggest and thank you for the 
suggestions. I still think that performing a score test is a sensible 
thing to do in my situation, though.

Murray

Spencer Graves wrote:

> Hi, Murray:
> 
>       When I have problems with nonconvergence of nls, I often
> move the problem to "optim(..., hessian=TRUE)".  Even if the larger 
> model is overparameterized and the hessian is singular, I optim usually 
> returns an answer from which I can then compute 2*log(likelihood ratio). 
>  Moreover, the hessian will help me diagnose the problem.  If it were my 
> problem today, I'd try the following:
> 
>       (1) If there are substantial differences in the diagonal elements 
> of the hessian, it suggests the scaling should be adjusted.  Not too 
> long ago, someone else suggested that this could be done within optim 
> via the argument control = list(parscale=...).  I have yet to try that, 
> but I think it should work fine.
> 
>       (2) If the diagonal elements of the hessian do not differ by more 
> than a couple orders of magnitude, then I'd try eigen(fit$hessian, 
> symmetric=TRUE).  The relative magnitudes of the eigenvalues will expose 
> the effective numer of paramaters that can be estimated, and the 
> eigenvectors associated with the smallest eigenvalues can help one 
> diagnose the problem.
> 
>       hope this helps.
>       spencer graves
> 
> Murray Jorgensen wrote:
> 
>> Hi Spencer,
>>
>> you were the only one to reply. Yes I am aware of the intrinsic / 
>> parameter effects distinction and the advantages of LR tests and 
>> profiling over Wald tests based on the local curvature of the 
>> loglikelihood surface at the larger of two models being compared. My 
>> situation is that I am comparing two nested models both of which have 
>> uncomfortably many parameters for the amount of data available. I am 
>> able to fit the smaller of the two models but not the larger. In this 
>> situation neither the the Wald nor the LR test is available to me but 
>> the score test (a.k.a. the Lagrange Multiplier test) is available to 
>> me because it is based on the loglikelihood gradient at the smaller 
>> model.
>>
>> I have been able to carry out the test by extracting
>>
>> X <- smaller.nls$m$gradient()
>>
>> and obtaining the extra columns of X for the parameters in larger but 
>> not in smaller by numerical differentiation. It seems that there 
>> should be some way of obtaining the extra columns without recourse to 
>> numerical differentiation, though.
>>
>> Cheers,  Murray Jorgensen
>>
>> Spencer Graves wrote:
>>
>>>       There doubtless is a way to extract the gradient information 
>>> you desire, but have you considered profiling instead?  Are you 
>>> familiar with the distinction between intrinsic and parameter effects 
>>> curvature?  In brief, part of the nonlinearities involved in 
>>> nonlinear least squares are intrinsic to the problem, and part are 
>>> due to the how the problem is parameterized.  If you change the 
>>> parameterization, you change the parameter effects curvature, but the 
>>> intrinsic curvature remains unchanged.  Roughly 30 years ago, Doug 
>>> Bates and Don Watts reanalized a few dozen published nonlinear 
>>> regression fits, and found that in all but perhaps one or two, the 
>>> parameter effects were dominant and the intrinsic curvature was 
>>> negligible.  See Bates and Watts (1988) Nonlinear Regression Analysis 
>>> and Its Applications (Wiley) or Seber and Wild (1989) Nonlinear 
>>> Regression (Wiley).
>>>
>>>       Bottom line:
>>>
>>>       1.  You will always get more accurate answers from profiling 
>>> than from the Wald "pseudodesign matrix" approach.  Moreover, often 
>>> the differences are dramatic.
>>>
>>>       2.  I just did RSiteSearch("profiling with nls").  The first 
>>> hit was 
>>> "http://finzi.psych.upenn.edu/R/library/stats/html/profile.nls.html". 
>>> If this is not satisfactory, please explain why.
>>>
>>>       hope this helps.
>>>       spencer graves
>>>
>>> Murray Jorgensen wrote:
>>>
>>>> Given a nonlinear model formula and a set of values for all the
>>>> parameters defining a point in parameter space, is there a neat way to
>>>> extract the pseudodesign matrix of the model at the point? That is the
>>>> matrix of partial derivatives of the fitted values w.r.t. the 
>>>> parameters
>>>> evaluated at the point.
>>>>
>>>> (I have figured out how to extract the gradient information from an 
>>>> nls fitted model using the nlsModel part, but I wish to implement a 
>>>> score test, so I need to be able to extract the information at 
>>>> points other than the mle.)
>>>>
>>>> Thanks, Murray Jorgensen
>>
>>
>>
>