[R] Likelihood Function for Multinomial Logistic Regression and its partial derivatives

[nikolay12]

Thanks a lot. The info about computing the gradient will be helpful.

I admit that I am somewhat confused about the likelihood function itself. It
is often said that you need to set a reference category. However, I found
two different implementations in Matlab for which setting the reference
category is not really obvious. Thus I wanted to find a single, indisputable
implementation of the likelihood function for multinomous logistic
regression in R. Can't believe there is no such available. 

Nick


Ravi Varadhan wrote:
> 
> Hi,
> 
> Providing the gradient function is generally a good idea in optimization;
> however, it is not necessary.  Almost all optimization routines will
> compute this using a simple finite-difference approximation, if they are
> not user-specified. If your function is very complicated, then you are
> more likely to make a mistake in computing analytic gradient, although
> many optimization routines also provide a check to see if the gradient is
> correctly specified or not.  But you can do this yourself using the `grad'
> function in "numDeriv" package.
> 
> Hope this is helpful,
> Ravi.
> 
> ____________________________________________________________________
> 
> Ravi Varadhan, Ph.D.
> Assistant Professor,
> Division of Geriatric Medicine and Gerontology
> School of Medicine
> Johns Hopkins University
> 
> Ph. (410) 502-2619
> email: rvaradhan at jhmi.edu
> 
> 
> ----- Original Message -----
> From: nikolay12 <nikolay12 at gmail.com>
> Date: Sunday, August 2, 2009 3:04 am
> Subject: [R] Likelihood Function for Multinomial Logistic Regression and
> its partial derivatives
> To: r-help at r-project.org
> 
> 
>>  Hi,
>>  
>>  I would like to apply the L-BFGS optimization algorithm to compute 
>> the MLE
>>  of a multilevel multinomial Logistic Regression. 
>>  
>>  The likelihood formula for this model has as one of the summands the 
>> formula
>>  for computing the likelihood of an ordinary (single-level) 
>> multinomial logit
>>  regression. So I would basically need the R implementation for this
>> formula.
>>  The L-BFGS algorithm also requires computing the partial derivatives 
>> of that
>>  formula in respect to all parameters. I would appreciate if you can 
>> point me
>>  to existing implementations that can do the above.
>>  
>>  Nick
>>  
>>  PS. The long story for the above:
>>  
>>  My data is as follows: 
>>  
>>  - a vector of observed values (lenght = D) of the dependent multinomial
>>  variable each element belonging to one of N levels of that variable
>>  
>>  - a matrix of corresponding observed values (O x P) of the independent
>>  variables (P in total, most of them are binary but also a few are
>>  integer-valued)
>>  
>>  - a vector of current estimates (or starting values) for the Beta
>>  coefficients of the independent variables (length = P).
>>  
>>  This data is available for 4 different pools. The partially-pooled model
>>  that I want to compute has as a likelihood function a sum of several
>>  elements, one being the classical likelihood function of a 
>> multinomial logit
>>  regression for each of the 4 pools.
>>  
>>  This is the same model as in Finkel and Manning "Hierarchical Bayesian
>>  Domain Adaptation" (2009).
>>  
>>  -- 
>>  View this message in context: 
>>  Sent from the R help mailing list archive at Nabble.com.
>>  
>>  ______________________________________________
>>  R-help at r-project.org mailing list
>>  
>>  PLEASE do read the posting guide 
>>  and provide commented, minimal, self-contained, reproducible code.
> 
> ______________________________________________
> R-help at r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide
> http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.
> 
> 

-- 
View this message in context: http://www.nabble.com/Likelihood-Function-for-Multinomial-Logistic-Regression-and-its-partial-derivatives-tp24772731p24781298.html
Sent from the R help mailing list archive at Nabble.com.