Wavelet Neural NetworksWith Applications in Financial Engineering, Chaos, and ClassificationContentsPrefacexiii1Machine Learning and Financial Engineering1Financial Engineering / 2 Financial Engineering and Related Research Areas / 3 Functions of Financial Engineering / 5 Applications of Machine Learning in Finance / 6 From Neural to Wavelet Networks / 8 Wavelet Analysis / 8 Extending the Fourier Transform: The Wavelet Analysis Paradigm / 10 Neural Networks / 17 Wavelet Neural Networks / 19 Applications of Wavelet Neural Networks in Financial Engineering, Chaos, and Classification / 21 Building Wavelet Networks / 23 Variable Selection / 23 Model Selection / 24 Model Adequacy Testing / 25 Book Outline / 25 References / 27viiviiiCONTENTS2Neural Networks35Parallel Processing / 36 Processing Units / 37 Activation Status and Activation Rules / 37 Connectivity Model / 39 Perceptron / 41 The Approximation Theorem / 42 The Delta Rule / 42 Backpropagation Neural Networks / 44 Multilayer Feedforward Networks / 44 The Generalized Delta Rule / 45 Backpropagation in Practice / 49 Training with Backpropagation / 51 Network Paralysis / 54 Local Minima / 54 Nonunique Solutions / 56 Configuration Reference / 56 Conclusions / 59 References / 593Wavelet Neural Networks61Wavelet Neural Networks for Multivariate Process Modeling / 62 Structure of a Wavelet Neural Network / 62 Initialization of the Parameters of the Wavelet Network / 64 Training a Wavelet Network with Backpropagation / 69 Stopping Conditions for Training / 72 Evaluating the Initialization Methods / 73 Conclusions / 77 References / 784Model Selection: Selecting the Architecture of the Network81The Usual Practice / 82 Early Stopping / 82 Regularization / 83 Pruning / 84 Minimum Prediction Risk / 86CONTENTSixEstimating the Prediction Risk Using Information Criteria / 87 Estimating the Prediction Risk Using Sampling Techniques / 89 Bootstrapping / 91 Cross-Validation / 94 Model Selection Without Training / 95 Evaluating the Model Selection Algorithm / 97 Case 1: Sinusoid and Noise with Decreasing Variance / 98 Case 2: Sum of Sinusoids and Cauchy Noise / 100 Adaptive Networks and Online Synthesis / 103 Conclusions / 104 References / 1055Variable Selection: Determining the Explanatory Variables107Existing Algorithms / 108 Sensitivity Criteria / 110 Model Fitness Criteria / 112 Algorithm for Selecting the Significant Variables / 114 Resampling Methods for the Estimation of Empirical Distributions / 116 Evaluating the Variable Significance Criteria / 117 Case 1: Sinusoid and Noise with Decreasing Variance / 117 Case 2: Sum of Sinusoids and Cauchy Noise / 120 Conclusions / 123 References / 1236Model Adequacy: Determining a Network s Future Performance125Testing the residuals / 126 Testing for Serial Correlation in the Residuals / 127 Evaluation Criteria for the Prediction Ability of the Wavelet Network / 129 Measuring the Accuracy of the Predictions / 129 Scatter Plots / 131 Linear Regression Between Forecasts and Targets / 132 Measuring the Ability to Predict the Change in Direction / 136 Two Simulated Cases / 137 Case 1: Sinusoid and Noise with Decreasing Variance / 137 Case 2: Sum of Sinusoids and Cauchy Noise / 142xCONTENTSClassification / 146 Assumptions and Objectives of Discriminant Analysis / 146 Validation of the Discriminant Function / 148 Evaluating the Classification Ability of a Wavelet Network / 150 Case 3: Classification Example on Bankruptcy / 153 Conclusions / 156 References / 1567Modeling Uncertainty: From Point Estimates to Prediction Intervals159The Usual Practice / 160 Confidence and Prediction Intervals / 161 Constructing Confidence Intervals / 164 The Bagging Method / 164 The Balancing Method / 165 Constructing Prediction Intervals / 166 The Bagging Method / 167 The Balancing Method / 168 Evaluating the Methods for Constructing Confidence and Prediction Intervals / 168 Conclusions / 170 References / 1718Modeling Financial Temperature Derivatives173Weather Derivatives / 174 Pricing and Modeling Methods / 175 Data Description and Preprocessing / 176 Data Examination / 176 Model for the Daily Average Temperature: Gaussian OrnsteinUhlenbeck Process with Lags and Time-Varying Mean Reversion / 179 Estimation Using Wavelet Networks / 183 Variable Selection / 183 Model Selection / 187 Initialization and Training / 187 Confidence and Prediction Intervals / 189 Out-of-Sample Comparison / 189 Conclusions / 191 References / 192CONTENTSxi9Modeling Financial Wind Derivatives197Modeling the Daily Average Wind Speed / 199 Linear ARMA Model / 202 Wavelet Networks for Wind Speed Modeling / 206 Variable Selection / 206 Model Selection / 209 Initialization and Training / 209 Model Adequacy / 209 Speed of Mean Reversion and Seasonal Variance / 211 Forecasting Daily Average Wind Speeds / 212 Conclusions / 215 References / 21610Predicting Chaotic Time Series219MackeyGlass Equation / 220 Model Selection / 221 Initialization and Training / 221 Model Adequacy / 222 Predi