NARX Model Types

The following model types are available for the NARX Structure modeling method:

• ASC GP-Spectrum (default)

• ASC GP-SCS

• ASC GP

• ASC Compressed
• MLP

• Linear

For the Recurrent Neural Network modeling method, see Model Predictions with Recurrent Neural Networks (RNN).

Note
For an overview of all model types and their best use, see Overview: Model Type Descriptions.

ASC GP-Spectrum Model

The ASCMO Gaussian Process Spectrum (ASC GP-Spectrum) is the standard type.

The ASC GP-Spectrum modeling algorithm represents the recent improvement of the standard ASC algorithm to cope with large datasets.

ASC GP-Spectrum requires a number of basis functions (n_basis) to be set by the user. During the model training, the information contained within the entire dataset is transformed to a set of n_basis virtual basis points. The bigger n_basis, the better the modeling result, but this goes along with an increase in modeling time. The recommended range is 50 < n_basis < 200.

You can define further model parameters for ASC GP-Spectrum:

No. Basis Functions

For a faster convergence, use a smaller number than in the actual model training (e.g., 20).

Number of Iterations

Enter the number of iterations to be performed during model training. If the model performance does not improve within 10 iterations on the validation data, the training will be aborted. In deep learning this is often referred to as number of epochs.

Plot Likelihood

If deactivated, the probability of mapping the input to class 1 will be plotted.

If activated, the probability values larger or equal to the threshold will be mapped to class 1. The class membership will be plotted.

Output Transformation

Select the transformation type of the output. Using a transformation can improve the model prediction. Not all transformations are available if the training data has negative or zero values.

You can select from the following choices:

• none: no transformation
• log(y): logarithm

• Bounded: limited to lower and upper bound

  Click Edit to view the automatically selected bounds or to define the lower and upper bounds manually. To define them manually, deactivate the Automatic checkbox. The bounds must be in the range of the training data.

  

• log(y+c): logarithm plus constant

  Click Edit to view the automatically selected log shift or to define a manual shift value. To define it manually, deactivate the Automatic checkbox.

  

Multistart

Enter the number of times to run the optimizer with different starting values during model training. A higher number means a higher probability of finding the optimal model, but it takes more time.

Note
Models of type ASC GP-Spectrum can be exported to all available export formats.

ASC GP-SCS Model

ASC GP Model

The ASCMO Gaussian Process (ASC GP) model type can be used for data sets with up to 4000 training data points with 15 inputs. This is the default model type. Training time and memory consumption scale especially with the number of data points. For many more data points, other model types may be more suitable.

The following model parameters can be specified:

Automatic Transformation

Activate this checkbox if you want the optimal Box-Cox transformation to be determined automatically during model training.

Output Transformation

Select the transformation type of the output. Using a transformation can improve the model prediction. If the Automatic Transformation checkbox is activated, the transformation type is selected automatically. Not all transformations are available if the training data has negative or zero values.

You can select from the following choices:

• none: no transformation
• 1/y: inversion
• 1/sqrt(y): inverted square root
• log(y): logarithm
• sqrt(y): square root

• Bounded: limited to lower and upper bound

  Click Edit to view the automatically selected bounds or to define the lower and upper bounds manually. To define them manually, deactivate the Automatic checkbox. The bounds must be in the range of the training data.

  

• log(y+c): logarithm plus constant

  Click Edit to view the automatically selected log shift or to define a manual shift value. To define it manually, deactivate the Automatic checkbox.

  

Advanced

If the advanced settings are enabled (File > Options > Advanced Settings), additional settings for the model training of the output can be selected here.

Advanced parameters for the ASC GP model

Number of Iterations

Enter the number of iterations to be performed during model training. If the model performance does not improve within 10 iterations on the validation data, the training will be aborted. In deep learning this is often referred to as number of epochs.

Number of Multistarts

Enter the number of training repetitions with different starting values. A higher value can improve the model quality, but the model training then takes more time. The default value is 3.

Plot Likelihood

Activate if you want the values of the logarithmic Likelihood function and the Leave-One-Out error to be displayed as a function of the runs during model training.

Max No. of Basis Functions

Enter a value as maximum number of basis functions for the model training. A higher value can improve the model quality, but the model training then takes more time. The default value is 4000. The smaller value of either training data points or subset size is used as the number of basic functions in the model. If more training data points are available, all data is used for training, but the resulting model is constrained to 4000 basis functions. A slightly different training algorithm is used in this case, while model predictions are done in the same way.

Sparse Subset Selection Method

Select the method you want to use to reduce the training data points to a spare subset when there are more than "Max No. of Basis Functions" data points.

• Random (Default): Random selection of a subset without specific selection criteria.

• GP-SCS like Selection using Gaussian process regression to iteratively add relevant features to a subset based on their impact on predictive performance.

Kernel

Select the kernel function to be used during model training.

• Squared Exponential (ARD) (Default): Trains models with softer curve characteristics.

• Matern (ARD) Trains models with harder curve characteristics. Provides sharper resolution of strong nonlinear effects with small noise. This could lead to overfitting.

Automatic Noise

Activate if you want the optimization of the noise level parameter to be performed automatically.

Noise Level

Enter a value for the maximum noise level to be tolerated. The input field is only accessible if the "Automatic Noise" checkbox is deactivated.

Lower Bound Noise

Enter a value as minimum for the noise level parameter. The input field is only accessible if the "Automatic Noise" checkbox is activated.

Linear Extrapolation

Activate if you want a supplementary linear model to be used, which learns the basic tendency of the data and shows this tendency outside the measured area.

Modeling Criterion

Select the modeling criterion to be used during model training.

• Default: Based on likelihood.

• Relative Error: Is the quotient of the error and the actual value (measured data - predicted data / measured data * 100). During the optimization, a visualization window pops up where you can manually stop the optimization and use the best result so far.

  

Input Length Scale Squared

Click Edit to open the Input Length Scale Squared window. You can edit the hyperparameter for each input. If Automatic is activated for an input, the respective hyperparameter is set automatically. Otherwise, you can edit the values manually. The hyperparameter per input dimension is the core width of the Gaussian bell. The length scale is r in the following equation, so a smaller value has greater relevance:

Rounding Threshold

Enter an anomaly prediction threshold below which the model prediction is automatically set to 0, and 1 otherwise. The threshold is displayed in the Receiver Operating Characteristic (Model > Anomaly Detection: Receiver Operating Characteristic).

ASC Compressed Model

This model type allows you to limit the number of basis functions within the model (see Model Compression). The following model parameters can be specified:

Model Size

Enter the number of basis functions of the compressed model.

Number of Iterations 1/2

Enter the number of iterations to be performed during model training. If the model performance does not improve within 10 iterations on the validation data, the training will be aborted. In deep learning this is often referred to as number of epochs.

Multistart

Enter the number of times to run the optimizer with different starting values during model training. A higher number means a higher probability of finding the optimal model, but it takes more time.

Plot Error

Activate the checkbox to display model error information during model training.

To save the RMSE as a bitmap graphic use View > Save as Bitmap in the main window.

Output Transformation

Select the transformation type of the output. Using a transformation can improve the model prediction. Not all transformations are available if the training data has negative or zero values.

You can select from the following choices:

• none: no transformation
• 1/y: inversion
• 1/sqrt(y): inverted square root
• log(y): logarithm
• sqrt(y): square root

• Bounded: limited to lower and upper bound

  Click Edit to view the automatically selected bounds or to define the lower and upper bounds manually. To define them manually, deactivate the Automatic checkbox. The bounds must be in the range of the training data.

  

• log(y+c): logarithm plus constant

  Click Edit to view the automatically selected log shift or to define a manual shift value. To define it manually, deactivate the Automatic checkbox.

  

Advanced

Click Edit to edit the advanced parameters.

MLP Model

MLP (Multilayer Perceptron) with possibility to export it to flatbuffer format for Bosch ECU.

The following model parameters can be specified:

Automatic Transformation

Activate this checkbox if you want the optimal Box-Cox transformation to be determined automatically during model training.

Output Transformation

Select the transformation type of the output. Using a transformation can improve the model prediction. If the Automatic Transformation checkbox is activated, the transformation type is selected automatically. Not all transformations are available if the training data has negative or zero values.

You can select from the following choices:

• none: no transformation
• 1/y: inversion
• 1/sqrt(y): inverted square root
• log(y): logarithm
• sqrt(y): square root

• Bounded: limited to lower and upper bound

  Click Edit to view the automatically selected bounds or to define the lower and upper bounds manually. To define them manually, deactivate the Automatic checkbox. The bounds must be in the range of the training data.

  

• log(y+c): logarithm plus constant

  Click Edit to view the automatically selected log shift or to define a manual shift value. To define it manually, deactivate the Automatic checkbox.

  

Iterations

Enter the number of iterations to be performed during model training. If the model performance does not improve within 10 iterations on the validation data, the training will be aborted. In deep learning this is often referred to as number of epochs.

Multistart

Enter the number of times to run the optimizer with different starting values during model training. A higher number means a higher probability of finding the optimal model, but it takes more time.

Validations Ratio [%]

Enter the relative number, in percent, of validation samples to be randomly selected from the training data.

Use Test Data as Validation Data

Activate the checkbox if you want to use the test data as validation data.

Continue Training

Activate the checkbox to continue with existing model training and iterations, if possible, instead of starting a new training. You can change the training properties and continue. For example, train with a complex activation function, then switch to a more efficient one (for the ECU) and continue training seamlessly.

Layers

Configure the layers of the multilayer perceptron. There is always at least one hidden layer, usually 1-3, and exactly one output layer.

Select an activation function from the list:

• Linear: y = x

• ReLU: y = max(0, x)

• LeakyReLU: y = max(0.01 * x, x)

• Sigmoid: y = 1 / (1 + exp(-x))

• PrecTanh: y = 2 / (1 + exp(-2 * x)) - 1

• Elliotsig: y = x / (1 + abs(x))

Insert: Click to insert a hidden layer.

Delete: Select one or more layers and delete them.

Linear Model

If the model type is Linear, you can define the following parameters:

Output Transformation

Select the transformation type of the output. Using a transformation can improve the model prediction. Not all transformations are available if the training data has negative or zero values.

You can select from the following choices:

• none: no transformation
• log(y): logarithm

• Bounded: limited to lower and upper bound

  Click Edit to view the automatically selected bounds or to define the lower and upper bounds manually. To define them manually, deactivate the Automatic checkbox. The bounds must be in the range of the training data.

  

• log(y+c): logarithm plus constant

  Click Edit to view the automatically selected log shift or to define a manual shift value. To define it manually, deactivate the Automatic checkbox.

  

Note
Models of type Linear cannot be exported.

See also

Enabling the Advanced Settings

Model Improvement Through Transformation of Output Variables

Model Predictions with Recurrent Neural Networks (RNN)

Overview: Model Type Descriptions