--OO--
                      
                           Magnus Norgaard's
                       
           NEURAL NETWORK BASED SYSTEM IDENTIFICATION TOOLBOX

                             Version 1.1
                         
                       Department of Automation
                           Building 326
                         2800 Lyngby, Denmark
                        
                   Technical University of Denmark

                                --OO--


The toolbox contains a number of m-files for training and evaluation of
multi-layer perceptron type neural networks. There are functions for 
working ordinary feed-forward networks as well as for identification of 
nonlinear dynamic systems and time-series analysis.

Not all functions have been thoroughly tested yet and even if I 
haven't been able to find any bugs, I can't guarantee that you won't encounter 
any problems. 

Below an overview of the files contained in this directory is given along
with a brief description of what they do. The on-line help facility explains
how to call the different functions. You simply write help <function-name>
in the MATLAB command window.

Along with the m- and c-files in this directory you will find a manual for the
toolbox. Start by printing this out and read the release notes.

A number of simple demonstration programs have been made to illustrate how most
of the functions work. Run these to get an idea of what the toolbox
provides. Be careful with the demos, though. Due to the existence of several
local minimas, the results will often vary a great deal from run to run.


Enjoy!
- MN


                                --OO--
                                
                                
                                

CURRENTLY WORKING FUNCTIONS IN THIS DIRECTORY:
---------------------------------------------

FUNCTIONS FOR TRAINING NETWORKS:
batbp    : Batch version of the back-propagation algorithm.
incbp    : Recursive (/incremental) version of back-propagation.
igls     : Iterated Generalized Least Squares training of multi-output nets.
marq     : Levenberg-Marquardt method.
marqlm   : Memory-saving implementation of the Levenberg-Marquardt method.
rpe      : Recursive prediction error method.


FUNCTIONS FOR PRETREATING THE DATA:
dscale   : Scale data to zero mean and variance one.


FUNCTIONS FOR TRAINING NETWORKS TO MODEL DYNAMIC SYSTEMS:
lipschit : Determine the lag space.
nnarmax1 : Identify a Neural Network ARMAX (or ARMA) model (Linear MA filter).
nnarmax2 : Identify a Neural Network ARMAX (or ARMA) model.
nnarx    : Identify a Neural Network ARX (or AR) model.
nnarxm   : Identify a multi output Neural Network ARX (or AR) model.
nnigls   : Iterated Generalized LS training of multi-output NNARX models.
nniol    : Identify a Neural Network model suited for I-O linearization control.
nnoe     : Identify a Neural Network Output Error model.
nnrarmx1 : Recursive counterpart to NNARMAX1.
nnrarmx2 : Recursive counterpart to NNARMAX2.
nnrarx   : Recursive counterpart to NNARX.
nnssif   : Identify a NN State Space Innovations form model.


FUNCTIONS FOR PRUNING NETWORKS:
netstruc : Extract weight matrices from matrix of parameter vectors.
nnprune  : Prune models of dynamic systems with Optimal Brain Surgeon (OBS).
obdprune : Prune feed-forward networks with Optimal Brain Damage (OBD).
obsprune : Prune feed-forward networks with Optimal Brain Surgeon (OBS).


FUNCTIONS FOR EVALUATING TRAINED NETWORKS:
fpe      : FPE estimate of the generalization error for feed-forward nets.
ifvalid  : Validation of models generated by NNSSIF.
ioleval  : Validation of models generated by NNIOL.
kpredict : k-step ahead prediction of dynamic systems.
loo      : Leave-One-Out estimate of generalization error for feed-forward nets.
nneval   : Validation of feed-forward networks (trained by marq,rpe,bp).
nnfpe    : FPE for I/O models of dynamic systems.
nnloo    : Leave-One-Out estimate for NNARX models.
nnsimul  : Simulate model of dynamic system from control sequence alone.
nnvalid  : Validation of I/O models of dynamic systems.
wrescale : Rescale weights of trained network.
xcorrel  : Calculates high-order cross-correlation functions


MISCELLANOUS FUNCTIONS:
README   : This file.
RELEASE  : Release notes.
Contents : Contents file.
drawnet  : Draws a two layer neural network.
getgrad  : Derivative of network outputs w.r.t. the weights.
pmntanh  : Fast tanh function.


DEMOS:
test1    : Demonstrates different training methods on a curve fitting example.
test2    : Demonstrates the NNARX function.
test3    : Demonstrates the NNARMAX2 function.
test4    : Demonstrates the NNSSIF function.
test5    : Demonstrates the NNOE function.
test6    : Demonstrates the effect of regularization by weight decay.
test7    : Demonstrates pruning by OBS on the sunspot benchmark problem.


OTHER FILES IN DIRECTORY
pmnshow, test6mat, test7mat, and solplet.asc are used by the test programs.

manual1.ps, manual2.ps contains the manual for the toolbox in Postscript format.

Makefile, marq.c, nnarmax2.c, nnoe.c, nnssif.c, matrix.c, matrix2.h, nnmisc.c,
nnmisc.h contains the source code necessary for generating CMEX versions of
the functions marq, nnarmax2, nnoe, and nnssif.