SKEDSOFT

Six Sigma

Neural Nets for Regression Type Problems (cont.):

  • One additional complication is the number of runs selected for the so-called “test set”. These runs are set aside and not used for estimating the weights in the minimization of the sum of squares error.
  • In the context of welding parameter development from planned experiments, it seems reasonable to assume that the number of runs is typically small by the standards discussed in the neural net literature. Therefore, the ad hoc selection of five random runs for the test set was proposed because this is perhaps the smallest number that could reasonably be expected to provide an independent and reliable estimate of the prediction errors.
  • A final complication is the so-called “termination criterion” for the minimization of the sum of squares error. In the hopes of avoiding over-fitting inaccuracies illustrated in Figure below, many neural net users do not attempt to solve the sum of squares minimization problem for the coefficients (“weights”) to global optimality. Instead they terminate the minimization algorithm before its completion based on nontrivial rules deriving from inspection of the test set errors.
  • For simplicity, these complications were ignored, and the Excel solver was permitted to attempt to select the weights that globally minimize the sum of squares error.