| Prev | Next | Start of Chapter | End of Chapter | Contents | Index | (3 out of 6)

Backpropagation Net (BPN)


The Backpropagation Network, or BPN, is a feed-forward, layered network. Each node in a layer is connected to all other nodes in the layer before it and the layer after it. It is especially useful for modeling multivariate functions.

The first layer and the input vector must be the same size. The last layer and the output vector must have the same size. The hidden or intermediate layers (layers between the first and last layers) can be any size. You can have up to three hidden layers, for a total of up to five layers. In general, a network has one hidden layer. The number of nodes depends on the complexity of the function that the network has to model. The more complex the function, the more nodes needed.

You can choose whether a layer uses the sigmoidal or linear function for its nodes. In general, the input and output layers use the linear function, and at least one of the hidden layers use a sigmoidal function.

Note: NeurOn-Line does not use G2 objects to represent the nodes and connections in a BPN. Instead, NeurOn-Line stores the network internally and lets you change the network's architecture with the configure menu choice.
Before you can pass data through a network, you must train the network. For more information, see "Training Blocks"." The number of data pairs in the training data set should be greater than the number of weights over the number of outputs. For example, if a network has 5 inputs, 10 hidden nodes and 3 outputs, its training data set should have at least this many data pairs.
In practice, several times this number is recommended.

When you pass a vector to a network, it calculates the value for its output vector by passing the input vector's data through the layers of its network. Passing data through a network does not change the values of its weights.

Configuring

To set the number of layers, number of nodes, and the transfer functions, you configure the BPN block.

When you select configure on the block, NeurOn-Line displays this dialog:


To set the number of layers, use the arrows to the right of the Number of Layers attribute. To increase the number, click the up arrow. To decrease the number, click the down arrow. You can select 2 to 5 layers.

The rows below Number of Layers let you set the number of nodes and transfer function for each input layer. If the network contains fewer than five layers, some of the fields will be inactive. You specify the nodes and transfer functions for the output layers.

To set the number of nodes for a layer, enter a number in the Nodes attribute for each layer. To set the transfer function for a layer, select the linear or sigmoid button for the Transfer Function attribute for each layer. Selecting the button toggles the button between linear and sigmoid.

Caution: If you change the architecture for a trained network by reducing the size of any layer, you must retrain the network.

Adjusting Weights

 When you first clone a BPN off the palette, all its weights are set to zero. However, the network needs to contain small random weights to train properly. To fill the network with weights appropriate for training, select the randomize weights. . . button. NeurOn-Line overwrites the block`s current weights with new random weights.

Caution: When you click the Randomize Weights button, NeurOn-Line immediately commits your changes. Clicking the Cancel button does not discard them.
NeurOn-Line displays the dialog below, which lets you specify an absolute amount to randomize by, a percentage to randomize by, or both:


This is the formula that the block uses to jiggle the weights, where P is the percentage you entered, A is the absolute amount you entered, and R1 and R2 are random numbers from -1.0 to 1.0.

When you are training a network and it seems to be stuck in a local minimum, slightly changing the weights can help push the network back onto the right track.

Saving and Loading Weights

You can save the network's weights to a text file so you can load them later. The file format for saved weights is described in "Saving and Loading Network Weights".

To save or load network data, select the file operations. . . menu choice to display this dialog:


Edit the File Pathname attribute to specify the filename. To save the weights, select the Save to File button. To load weights, overwriting the weights that are currently in the network, select the Load from File button.

Making Values Permanent

When you choose make permanent from the block's menu, it saves the network's internal configuration and weight so that resetting G2 has no effect on their values.

Examples

The following configure dialog is for a BPN that is being trained with a data set of angles and their sines. It contains three layers: the input layer, the output layer, and one hidden layer. Both the input and output layers have 1 node and use the linear function. The hidden layer has 4 nodes and uses the sigmoid function.


Below is a simple diagram for training a network. The data set on the left is filled with a sample of angles and their sines. The BPN is configured as described above. To train this network, choose configure from the BPN's menu and click Randomize Weights, then choose evaluate from the Training block's menu.


This diagram uses a trained BPN to estimate the sine for an angle of 45 degrees.


See Also

For more information on how to use this block, see the pages below.

Click here for more information...
Basic Block Behavior
Saving a Block's Data After Resetting G2

| Prev | Next | Start of Chapter | End of Chapter | Contents | Index | (3 out of 6)

Copyright © 1996, Gensym Corporation, Inc.