1.12 Exercises

1.1 (a) Compute by hand the linear regressor for the following data:

X= {-0.5, -0.2, -0.1, 0.3,0.4, 0.5, 0.7}

D= {-1, 1, 2, 3.2, 3.5, 5, 6}

(b) Compute the correlation coefficient.

(c) Estimate D for the values x=0 and x=1 . Which d can you trust the most? Justify your answer.

(d) Use NeuroSolutions to confirm your results.

1.2 For the data of Problem 1.1, compute the performance surface and plot it. Estimate the gradient at the point w =2.

1.3 (a) Compute the linear regression for the following data:

X1= {-0.5, -0.2, -0.1, 0.3,0.4, 0.5, 0.7}

X2= {3, 3, 2.5, 2, -1, -1, -4}

D= {-3, -1, 0, 1.2, 1.5, 3, 4}

(b) Use NeuroSolutions to confirm your results.

1.4 Find the eigenvalues and eigenvectors of the following matrix

1.5 Show that when you apply Eq. 1.5 to data that is Gaussian distributed, effectively you obtain a set of linear equations.

1.6 Rerun Example 1.8 with white noise that has a mean value of m =0.5 and explain the results.

1.7 Show mathematically that Eq. 1.14 will always give a value of |g|<1.

1.8 For the data set in Problem 1.3, estimate the largest step size, the critically damped step size, and the time constant of adaptation for each step size. Run NeuroSolutions and verify your results by plotting the weight tracks. Also verify the time constant of adaptation. For which value does the system adapt the fastest?

1.9 If you know the eigenvalues of the input autocorrelation function, would you schedule the step size for fast adaptation? Justify your answer. Run NeuroSolutions to confirm your conclusion.

1.10 Verify in the data of Problem 1.3 the statement of Section 1.6.5 that when the step size is very large in on-line learning the MSE in NeuroSolutions can decrease below the theoretical limit. Plot the weight tracks and comment on the validity of the model.

1.11 Compute the largest eigenvalue for the data of Problem 1.3, but now use a regressor with bias. Show all the work. What can you conclude about the largest step size for the bias?

1.12 Apply Newton's search to the data of Problem 1.3, and compare the speed of adaptation with LMS. You should also use NeuroSolutions.

1.13 Go to the Data directory and open the folder Auto MPG. Read the instructions and determine the regression equation to estimate the MPG of a car. Use NeuroSolutions. Make sure the system is converging. What is the estimated misadjustment for the solution? How would you obtain a better linear fit? Compute also the correlation coefficient and conclude about the applicability of your model.

1.14 Repeat Problem 1.13 with the data found in the folder Abalone.

1.15 Construct the Breadboard for Problem 1.13 from scratch, i.e. one component at a time.

Go to next section