Evaluating the Model Quality

To evaluate the quality of the model and recognize any existing outliers, the Model menu in the ISP view contains the following submenus:

• Error (Leave-One-Out)
• Error (Test Data)
• Error (Training Data)

These submenus can be used to perform an analysis based on different data.

For the analysis based on leave-one-out error, a model is formed on all training data except for one measurement. In this case, this omitted measuring point represents the test data with which the model prediction is tested.

This process is repeated as long as there are training data. In each case, another measuring point is omitted and the model analysis is performed on this test point. With 100 training data, a model with 99 training data is formed 100 times, and the comparison is performed between model prediction and the individual omitted test data point.

This allows performing an representative test for the model quality without providing a large quantity of test data for this purpose since all the training data are also used as test data here.

For the ASC GP model type, this complex calculation is not performed since the algorithm also automatically outputs the corresponding information – for this reason, the calculation time is very short. For other model types, it may take relatively long for large data volumes.

For the analysis based on test data, the model outputs are compared with the test data. Test data are data which the model did not use for training. This makes it possible, e.g. to assess the capability of the model for generalization.

The third option is the analysis based on training data. In the process, the deviation between the measured values at all training data from the model output is compared at the corresponding location and graphically displayed.