[R] Evaluating/comparing dynamic linear model

[Erb Philipp (erbp)]

Apologies for my overhasty and thus inaccurate reply. I of course assumed that model2 is a submodel of model1. But thanks to Giovanni: a closer look at your equations shows that this is only the case if at == bt (and of course same innovations/ disturbances et, etat) and xt e.g. something like xt == theta0 * zt + 1. In that case you are allowed to perform a likelihood ratio test to test the hypothesis theta0 == 0. If model2 is included in model1 likelihood ratio test might be more convenient than comparing the predictive performance.


-----Original Message-----
From: Giovanni Petris [mailto:GPetris at uark.edu]
Sent: Thu 10/8/2009 3:55 PM
To: Erb Philipp (erbp)
Cc: rhelpacc at gmail.com; r-help at r-project.org
Subject: Re: [R] Evaluating/comparing dynamic linear model
 

The standard asymptotic theory of likelihood ratio tests assumes that
you are testing a submodel, which is not the case here. Moreover, even
when testing submodels, there are other assumptions that often are not
met in the case of DLMs - the typical example being hypothesised
values on the boundary of the parameter space. 

An informal, but commonly used approach to compare DLMs is to compare
their predictive performance. There are many ways to measure it. You
can find some of them in the book "dynamic linear models with R". 

BTW, in the package dlm you find functions for MLE, forecasting,
one-step-ahead forecast errors (which is what you need to compute
predictive performance), and more...

HTH,
Giovanni


> Date: Thu, 08 Oct 2009 08:32:21 +0200
> From: "Erb Philipp (erbp)" <erbp at zhaw.ch>
> Sender: r-help-bounces at r-project.org
> Precedence: list
> Thread-topic: [R-SIG-Finance] Evaluating/comparing dynamic linear model
> Thread-index: AcpHsn7VAX05YB13ShmB34Y7FCJyDAALTuuA
> 
> What kind of filter are you using? Since your models are expressed
> in state space form I suggest that you fit your models by maximizing
> the log likelihood function of the Kalman filter output (see
> e.g. FKF-package). Using the obtained log likelihood values you
> might perform a likelihood ratio test to test the hypothesis whether
> model 1 explains yt "better" than model 2. 
> 
> HTH, Phil
> 
> 
> -----Ursprüngliche Nachricht-----
> Von: r-sig-finance-bounces at stat.math.ethz.ch [mailto:r-sig-finance-bounces at stat.math.ethz.ch] Im Auftrag von R_help Help
> Gesendet: Donnerstag, 8. Oktober 2009 02:55
> An: r-sig-finance at stat.math.ethz.ch; r-help at r-project.org
> Betreff: [R-SIG-Finance] Evaluating/comparing dynamic linear model
> 
> Hi,
> 
> I have two DLM model specifications (x[t] and y[t] are univariate):
> 
> MODEL1:
> y[t] = b[t]x[t]+e[t], e[t] ~ N(0,v1^2)
> b[t] = b[t-1]+eta[t], eta[t] ~ N(0,w1^2)
> 
> MODEL2:
> y[t] = a[t]+e[t], e[t] ~ N(0,v2^2)
> a[t] = a[t-1]+eta[t], eta[t] ~ N(0,w2^2)
> 
> I run the filter through data recursively to obtain state variables
> for each model. However, how do I know if b[t]x[t] in MODEL1 is
> different from MODEL2? In other words, how do I know if x[t] makes a
> difference in explaining dynamic of y[t]?
> 
> Another question is that how do I compare MODEL1 and MODEL2? From
> model specification point of view, how can one say that MODEL1 is
> better than MODEL2? Any suggestion/reference would be greatly
> appreciated. Thank you.
> 
> ac
> 
> _______________________________________________
> R-SIG-Finance at stat.math.ethz.ch mailing list
> https://stat.ethz.ch/mailman/listinfo/r-sig-finance
> -- Subscriber-posting only.
> -- If you want to post, subscribe first.
> 
> ______________________________________________
> R-help at r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.
> 
> 
>