[R] Linear Regression with Constraints

[Bill.Venables at csiro.au]

You can accommodate the constraints by, e.g., putting

c2 = pnorm(theta2)
c3 = pnorm(theta3)

x1 has a known coefficient (unity) so it becomes an offset.  Essentially your problem can be written

y1 = y-x1 = c1 + pnorm(theta2)*x2 - pnorm(theta3)*x3 + error

This is then a (pretty simple) non-linear regression which could be fitted using, e.g. nls

If you could not rule out the possibility that the solution is on the boundary, you could put c2 = (cos(theta2))^2, and the fitting procedure could take you there.  The solution is not unique, but the original coefficients, c2,c3, would be unique, of course.

With just 6 observations and 4 parameters to estimate, you will need the model to be an exceptionally close fitting one for the fit to have any credibility at all.

Bill Venables.
________________________________________
From: r-help-bounces at r-project.org [r-help-bounces at r-project.org] On Behalf Of Emmanuel Charpentier [charpent at bacbuc.dyndns.org]
Sent: 27 May 2009 17:05
To: r-help at stat.math.ethz.ch
Subject: Re: [R] Linear Regression with Constraints

Le mardi 26 mai 2009 à 14:11 -0400, Stu @ AGS a écrit :
> Hi!
> I am a bit new to R.
> I am looking for the right function to use for a multiple regression problem
> of the form:
>
> y = c1 + x1 + (c2 * x2) - (c3 * x3)
>
> Where c1, c2, and c3 are the desired regression coefficients that are
> subject to the following constraints:
>
> 0.0 < c2 < 1.0, and
> 0.0 < c3 < 1.0

Sounds rather like an in-the-closet Bayesian problem (with a very
strange prior...). Did you consider to submit it to WinBUGS (or JAGS) ?

If you still want a direct optimization, you could have started :

RSiteSearch("optimization constraint")

Which would have quickly led you to ask :

? constrOptim

> y, x1, x2, and x3 are observed data.
> I have a total of 6 rows of data in a data set.

??? I that's real-life data, I wonder what kind of situation forces you
to estimate 3+1 parameters (c1, c2, c3 and the residual, which is not
really a parameter) with 6 data points ? Your problem can be written as
a system of 6 linear equations with 3 unknowns (c1, c2, c3), leaving you
room to search in (a small piece of) R^3 (the residual is another way to
express your objective function, not an independent parameter).

Of course, if it's homework, get lost !

                                        Emmanuel Charpentier

> Is "optim" in the stats package the right function to use?
> Also, I can't quite figure out how to specify the constraints.
> Thank you!
>
> -Stu
>

______________________________________________
R-help at r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.