[R] Calculating loess value

[Noah Silverman]

Hello,

I'm attempting to evaluate the accuracy of the probability predictions 
for my model.  As previously discussed here, the AUC is not a good 
measure as I'm not concerned with classification accuracy but 
probability accurcy.

It was suggested to me that the loess function would be a good measure 
to look at.

I can see some libraries (Design) will plot the loess function as a 
curve of the resulting model.  That's nice to look at, but I need a way 
to feed in a test set of data and measure the accuracy of my predicted 
probabilities.

I read the help page for loess and it looks like a learner of its own. I 
already have a model trained with SVM and can apply the test data for a 
result of output that is predicted probabilities and true labels.  How 
can I feed this to the loess function to both see a curve and get the 
mean absolute error among other measures.

Ultimately what I am trying to do is develop a model with high accuracy 
of predicted probabilities.  I'm testing many different learning 
functions, kernels, parameters, etc.  I'mm looking of a single 
"performance measure" that summarized the probability accuracy for a 
given model.  That way I can track my experiment's results and pick the 
best one.

Any suggestions along these lines would be greatly appreciated.

Thank You,

-Noah