| Prev | Next | Start of Chapter | End of Chapter | Contents | Index | (2 out of 6)

Introduction

 The Neural Network blocks are the heart of NeurOn-Line.

You can find the Neural Networks palette under the Neural Networks submenu of the Palettes menu:


Saving and Loading Network Weights

All the Neural Network blocks let you save and load what they have learned so far. NeurOn-Line saves the information to text files, which you can examine yourself. This section describes the formats used for these files.

Backpropagation and Autoassociative Network File Format

The text format for saving and loading BPNs from files consists of the following lines.

Note: All lists are comma-separated, and are in order beginning with the item for the first layer. A comment begins with a semicolon and continues to the end of the line.
  1. The version number. For this version of NeurOn-Line, it is 1.

  2. The number of layers in the network.

  3. A list of the number of nodes in each layer.

  4. A list of the transfer functions for each layer. The number 0 stands for linear, and the number 1 stands for sigmoid.

  5. The weight of each node in the network. Each weight is on a separate line and is followed by a comment that identifies which nodes it is for. The convention used to identify the nodes is described below. Note that the bias node is included as an extra node in each layer, except the output layer.

The weights are listed in order. The first is for the connection from the first node in the first layer to the first node in the second layer. The last is for the connection from the last node in the second-to-last layer to the last node in the last layer.

Each weight is followed by a comment that identifies which nodes it is for. The comment contains three numbers, as follows.

This is the weight from node i in layer k to node j in layer k+1. For example, the following weight is for the second node in the third layer to the first node in the fourth layer.

Here is an example of a file for a BPN, with 3 layers, 2 input layer nodes, 3 hidden layer nodes, and 1 output layer node:

1; Version of this file save/restore protocol for BPNs
3 ; Number of layers
2, 3, 1 ; Layer sizes of BPN.
0, 1, 0 ; Transfer functions of BPN.
-0.32507146396688 ; 1 1 1
-0.82195669379450 ; 1 2 1
0.19680059179068 ; 1 3 1
-0.97116809332961 ; 2 1 1
0.61150472166297 ; 2 2 1
-0.12016215756566 ; 2 3 1
0.84987859399967 ; 3 1 1
-0.74007775586113 ; 3 2 1
-0.63971444152242 ; 3 3 1
0.84097431197944 ; 1 1 2
-0.79330091884975 ; 2 1 2
-0.01523408110297 ; 3 1 2
0.43417445464837 ; 4 1 2

Radial Basis Function and Rho Network File Format

The text format for saving and loading RBFNs from files consists of the following lines.

Note: All lists are comma-separated. A comment begins with a semicolon and continues to the end of the line.
  1. The version number. For this version of NeurOn-Line, it is 1.

  2. A list of the number of nodes in each layer.

  3. The unit overlap.

  4. Whether the network uses spherical or elliptical units. The number 0 stands for spherical units, and the number 1 stands for elliptical units.

  5. The locations of the sphere or ellipse centers. The locations for each row are on a separate line. Each line contains as many numbers as there are elements in the input vector. The number of location lines is the same as the number of hidden units.

  6. The shapes of the units. If you are using spherical units, there is one line for each hidden unit, and each line contains the width for the unit. If you are using elliptical units, there are N*H lines, and each line contains N values, where N is the number of input values and H is the number of hidden units. The first N lines represent the inverse covariance matrix of radial unit 1, the next N lines represent the inverse covariance of the second radial unit, etc.

  7. The weights for the output layer. There is one line for each node in the hidden layer, and each line contains the weights from the hidden node to the node in the output layer. If this is a RBFN, the weights for the bias node are on an extra line at the end.

Here is an example of a file for an RBFN with spherical units.

1; Version of this file save/restore protocol for RBFNs
3, 6, 1 ; Layer sizes of RBFN.
2 ; Unit overlap parameter
0 ; Spherical unit shape
1 ; Bias on
10.8896000000, 5.0603000000, 5.1376100000 ; Unit centers row 1
11.4327000000, 5.5737400000, 5.6030400000 ; Unit centers row 2
7.8846400000, 5.0246100000, 5.0246100000 ; Unit centers row 3
9.4227900000, 5.1608600000, 5.1608600000 ; Unit centers row 4
10.1503000000, 5.5839500000, 5.2830700000 ; Unit centers row 5
6.7058600000, 5.0688600000, 4.9504500000 ; Unit centers row 6
0.8992220000 ; Unit shapes row 1
1.1230300000 ; Unit shapes row 2
1.3784100000 ; Unit shapes row 3
1.2011400000 ; Unit shapes row 4
0.8846430000 ; Unit shapes row 5
2.1013600000 ; Unit shapes row 6
-2.3439900000 ; Second layer weights row 1
-0.3743570000 ; Second layer weights row 2
-0.3669410000 ; Second layer weights row 3
-3.9031200000 ; Second layer weights row 4
0.6760480000 ; Second layer weights row 5
-0.9269620000 ; Second layer weights row 6
11.1002000000 ; Second layer weights row 7

Here is a file for an RBFN that has the same basic architecture as the one above, but that uses elliptical units.

1; Version of this file save/restore protocol for RBFNs
3, 6, 1 ; Layer sizes of RBFN.
2 ; Unit overlap parameter
1 ; Elliptical unit shape
1 ; Bias on
6.6357800000, 5.2808800000, 4.9493200000 ; Unit centers row 1
7.2610900000, 5.1641900000, 5.1641900000 ; Unit centers row 2
10.3036000000, 5.3620800000, 5.1868500000 ; Unit centers row 3
8.2783000000, 5.0083000000, 5.0083000000 ; Unit centers row 4
6.6201000000, 4.8137500000, 4.8137500000 ; Unit centers row 5
11.2850000000, 5.3348900000, 5.4306300000 ; Unit centers row 6
5.3577500000, 0.1998100000, -4.0963000000 ; Unit shapes row 1
0.1998100000, 2.8251900000, -1.3811100000 ; Unit shapes row 2
-4.0963000000, -1.3811100000, 7.7281600000 ; Unit shapes row 3
1.0171300000, 0., -1.2055800000 ; Unit shapes row 4
0.7242780000, 9.4219700000, -7.0106700000 ; Unit shapes row 5
-1.2055800000, -7.0106700000, 11.3346000000 ; Unit shapes row 6
0.3653470000, 0.2290360000, -0.7771570000 ; Unit shapes row 7
0.2290360000, 4.0050800000, -3.5144000000 ; Unit shapes row 8
-0.7771570000, -3.5144000000, 7.2356000000 ; Unit shapes row 9
0.3881750000, -0.1173250000, -0.4112650000 ; Unit shapes row 10
-0.1173250000, 17.4052000000, -15.6234000000 ; Unit shapes row 11
-0.4112650000, -15.6234000000, 25.1755000000 ; Unit shapes row 12
4.1952000000, 0.2980240000, -3.6536000000 ; Unit shapes row 13
0.2980240000, 5.1403200000, -4.1436400000 ; Unit shapes row 14
-3.6536000000, -4.1436400000, 11.2164000000 ; Unit shapes row 15
0.2617710000, 0.1799830000, -0.6148770000 ; Unit shapes row 16
0.1799830000, 3.1725600000, -2.8314600000 ; Unit shapes row 17
-0.6148770000, -2.8314600000, 5.4504000000 ; Unit shapes row 18
3.7661200000 ; Second layer weights row 1
0.4220330000 ; Second layer weights row 2
-0.8155410000 ; Second layer weights row 3
0.9443360000 ; Second layer weights row 4
-0.5184290000 ; Second layer weights row 5
1.9278200000 ; Second layer weights row 6
7.8096200000 ; Second layer weights row 7

Backpropagation and Autoassociative Networks

Both the Backpropagation Net (BPN) block and the Autoassociative Net block are feed-forward networks with multiple layers. The Autoassociative Network is a type of Backpropagation Network with a specific architecture, especially good for handling certain types of problems, including sensor validation.

Radial Basis Function and Rho Networks

Both the Radial Basis Function Net (RBFN) block and the Rho Net block are 3- layer, feed-forward networks, whose middle layers use a multivariate Gaussian function. Both are especially good at handling classification problems. Their biggest difference is what their output values are. The Radial Basis Function Network can return any type of number. The Rho Network passes a number between 0.0 and 1.0, which represents the probability that the input value is in a particular class.

| Prev | Next | Start of Chapter | End of Chapter | Contents | Index | (2 out of 6)

Copyright © 1996, Gensym Corporation, Inc.