[R] mathematical model/equations for neural network in library(nnet)

[jude.ryan at ubs.com]

Hi All,

 

I am trying to manually extract the scoring equations for a neural
network so that I can score clients on a system that does not have R
(mainframe using COBOL).

Using the example in Modern Applied Statistics with S (MASS), by
Venables and Ripley (VR), pages 246 and 247, I ran the following neural
network. The code is the same as in VR pages 246 and 247 except I have
skip = F. The equation will have 3 more terms if skip = T.

 

library(nnet)

attach(rock)

area1 <- area/10000; peri1 <- peri/10000

rock1 <- data.frame(perm, area = area1, peri = peri1, shape)

# skip = F

rock2.nn <- nnet(log(perm) ~ area + peri + shape, rock1, size=3,
decay=1e-3, linout=T, skip=F, maxit=1000, Hess=T)

 

# weights:  16

initial  value 1420.968942 

iter  10 value 96.823665

iter  20 value 32.177295

iter  30 value 25.012430

iter  40 value 23.109650

iter  50 value 20.981236

iter  60 value 15.019016

iter  70 value 14.082190

iter  80 value 14.042717

iter  90 value 13.931124

iter 100 value 13.883691

iter 110 value 13.877307

iter 120 value 13.875051

iter 130 value 13.873667

final  value 13.873634 

converged

 

summary(rock2.nn)

 

The output from summary(rock2.nn) is:

 

a 3-3-1 network with 16 weights

options were - linear output units  decay=0.001

 b->h1 i1->h1 i2->h1 i3->h1 

 10.65  -8.90 -14.63   6.17 

 b->h2 i1->h2 i2->h2 i3->h2 

 -0.72  11.76 -17.17  -1.56 

 b->h3 i1->h3 i2->h3 i3->h3 

  2.96  -9.03  -8.07  -2.54 

  b->o  h1->o  h2->o  h3->o 

 -6.91   2.45  11.53   9.22

 

Following the mathematical model / equations shown in VR (pages 243 to
247) and another book on neural networks, I extracted the neural network
equations manually, and scored the dataset rock1, and compared the
manual scores I obtained with the scores from predict(). They were
totally different, and I am not sure what I am doing wrong. If anyone
can give me some pointers I would appreciate it.

 

The mathematical model/equations I come up with from the weights are:

 

# manual calculate neural network predictions based on neural network
equations

rock1$h1 <- 10.65 - 8.9 * rock1$area - 14.63 * rock1$peri + 6.17 *
rock1$shape

rock1$logistic_h1 <- exp(rock1$h1) / (1 + exp(rock1$h1))

rock1$h2 <- -0.72 + 11.76 * rock1$area - 11.17 * rock1$peri - 1.56 *
rock1$shape

rock1$logistic_h2 <- exp(rock1$h2) / (1 + exp(rock1$h2))

rock1$h3 <- 2.96 - 9.03 * rock1$area - 8.07 * rock1$peri - 2.54 *
rock1$shape

rock1$logistic_h3 <- exp(rock1$h3) / (1 + exp(rock1$h3))

# predictions based on manual scoring

rock1$pred_perm <- -6.91 + 2.45 * rock1$logistic_h1 + 11.53 *
rock1$logistic_h2 + 9.22 * rock1$logistic_h3

# predictions using predict() and object that has the output of the
neural network

rock1$nn_pred_perm <- predict(rock2.nn)

rock1$log_perm <- log(rock1$perm)

head(rock1)

 

  perm   area     peri     shape         h1 logistic_h1       h2
logistic_h2        h3  logistic_h3 pred_perm nn_pred_perm log_perm

1  6.3 0.4990 0.279190 0.0903296  2.6816839   0.9359372 1.888774
0.8686156 -4.028470 0.0174901847  5.559444     1.920348 1.840550

2  6.3 0.7002 0.389260 0.1486220 -0.3596561   0.4110428 2.934467
0.9495242 -6.881634 0.0010254128  5.054524     1.546815 1.840550

3  6.3 0.7558 0.393066 0.1833120 -0.6961405   0.3326685 3.491694
0.9704505 -7.502529 0.0005513831  5.099416     2.630932 1.840550

4  6.3 0.7352 0.386932 0.1170630 -0.8318165   0.3032611 3.421303
0.9683637 -7.098737 0.0008254655  5.005834     2.489565 1.840550

5 17.1 0.7943 0.394854 0.1224170 -1.4406711   0.1914414 4.019478
0.9823546 -7.709940 0.0004481475  4.889712     3.235397 2.839078

6 17.1 0.7979 0.401015 0.1670450 -1.2874918   0.2162777 3.923376
0.9806092 -7.905522 0.0003685659  4.929703     3.078584 2.839078

 

sum((log(perm) - rock1$nn_pred_perm)^2)

[1] 12.55929

 

sum((log(perm) - rock1$pred_perm)^2)

[1] 82.63254

 

Thanks in advance,

 

Jude

 

 

___________________________________________
Jude Ryan
Director, Client Analytical Services
Strategy & Business Development
UBS Financial Services Inc.
1200 Harbor Boulevard, 4th Floor
Weehawken, NJ 07086-6791
Tel. 201-352-1935
Fax 201-272-2914
Email: jude.ryan at ubs.com



-------------- next part --------------
Please do not transmit orders or instructions regarding a UBS 
account electronically, including but not limited to e-mail, 
fax, text or instant messaging. The information provided in 
this e-mail or any attachments is not an official transaction 
confirmation or account statement. For your protection, do not 
include account numbers, Social Security numbers, credit card 
numbers, passwords or other non-public information in your e-mail. 
Because the information contained in this message may be privileged, 
confidential, proprietary or otherwise protected from disclosure, 
please notify us immediately by replying to this message and 
deleting it from your computer if you have received this 
communication in error. Thank you. 

UBS Financial Services Inc. 
UBS International Inc. 
UBS Financial Services Incorporated of Puerto Rico 
UBS AG

 
UBS reserves the right to retain all messages. Messages are protected
and accessed only in legally justified cases.