| Prev | Next | Start of Chapter | End of Chapter | Contents | Index | (5 out of 7)

Train and Test


The Train and Test block helps you determine whether you have chosen the best neural network configuration for your problem. You can use it to test a data set against different types of neural networks or different configurations of the same type of neural network. It trains and tests your network a specified number of times, each time randomly splitting the data set into two subsets: training and testing.

Note: The Train and Test block is an encapsulation block that contains a NeurOn- Line diagram on its subworkspace. For more information on what the subworkspace contains, see "The Train and Test Block's Subworkspace".
Attach the Train and Test block's top action link to your data set and the bottom action link to your neural network. You cannot attach more than one neural network or data set to the block.

To configure the Train and Test block, choose configure from its menu. It displays three dialogs, one on top of the other. The topmost is for the Train and Test Block. It is described in "Configuring". The other dialogs are for the Training and Fit Tester blocks that are on the Train and Test block's subworkspace. For more information on how to configure them, see "Trainer" and "Fit Tester".

To evaluate the Train and Test block, you must pass it a control signal or choose evaluate from the menu. The block trains and tests your neural network a specified number times. Each time it randomly splits the attached data set into two different subsets using a proportion you specify: one subset for training and one for testing.

To cancel training, choose Controls > Remote Process > Kill and then reset the block. The neural network's weights and other internal parameters might be changed if the Train and Test completed one or more iterations before it cancels training. The Train and Test block outputs nothing. If you cancel training, you must restart the remote process.

When the block has finished all the iterations, it passes a control signal and two scalar values. The scalar value from its top output port is the median value of the results from the Trainer block. The scalar value from its bottom port is the median value of the results from the Fit Tester block. The meaning of these values depends on how you configure the Trainer and Fit Tester blocks. For more information on these values, see "Trainer" and "Fit Tester".

In each case, the training error is an indicator of how well the network fits the training data. The error will generally decrease if you expand the network architecture, even when the network is overfitting. Therefore, do not use the training error to select the optimum network architecture.

The test error indicates how well the network fits data not used in training. In general, you should select the smallest network architecture that minimizes the test error. This value will increase or stay nearly constant when the network is too large.

Configuring

To configure the Train and Test block, choose configure from its menu. It displays three dialogs, one on top of the other. The topmost dialog is shown below.


To specify the number of times to train and test your network, enter a number in the attribute Resampling Trials. To specify how much of the data set to use for training and how much for testing, enter the proportion for training in the attribute Fraction to Training Subset. The block uses the rest of the data for testing.

You must also configure the dialog for the Trainer block on the subworkspace of the Train and Test block. For more information on configuring this block, see "Configuring".

Finally, you must configure the dialog for the Fit Tester on the subworkspace of the Train and Test block. For more information on configuring this block, see "Configuring":

The Train and Test Block's Subworkspace

The Train and Test Block has a subworkspace with the following diagram, which you can access by using the view diagram menu choice.


For the number of times that you specified in the configuration panel, the block follows this procedure:

  1. Clear the neural network to prepare it for a new training session.

  2. Randomly separate the data set into two smaller data sets: one for training and one for testing. The training set contains the fraction of the data set that you specified in the configuration panel. The testing set contains the rest.

  3. Train the network with the training set.

  4. Test the network with the testing set.

Note: Do not configure blocks on the subworkspace directly, because this can cause inconsistencies. Use the configuration panels instead.

Example

 This example uses Train and Test blocks to see which of three neural network configurations is the best. All networks use exactly the same data set. To determine which configuration fits the data best, look at the number passed from the test output of the Train and Test block. The lower the number, the better the fit for data not seen in the training process.


See Also

For more information on how to use this block, see the pages below.

Click here for more information...
Basic Block Behavior
"Neural Network Blocks" Chapter
Data Set
Trainer
Fit Tester
Five Fold CV

| Prev | Next | Start of Chapter | End of Chapter | Contents | Index | (5 out of 7)

Copyright © 1996, Gensym Corporation, Inc.