| Download a free evaluation copy of NeuroSolutions to discover how to apply neural network technology to your artificial intelligence application. |
1.11 Conclusions
In this first chapter we introduced ideas that are very important for the rest of the book. Probably the most important was the concept of adaptive systems. Instead of designing the system through specifications, we let the system learn from the input data. To achieve this, the system has to be augmented with an external cost criterion that measures how well the model fits the data and with an algorithm that will adapt the system parameters so that the minimum of the cost can be reached. We will use this concept throughout this book.
We covered much more in this chapter. We described an extremely simple and elegant algorithm that is able to minimize the external cost function by using local information available to the system parameters. The principle is to search the performance surface in the opposite direction of the gradient. The name of the algorithm is least means squares (LMS) and in just two multiplications per weight and data sample it is able to move the system parameters towards the neighborhood of the optimal values. Gradient descent is a powerful concept that we will also use throughout this book.
When we apply the LMS to the linear network, we end up with a system called the linear regressor or Adaline that can fit hyperplanes to data. The solution is identical to least squares.
We quantified the properties of the LMS algorithm, and we showed the fundamental trade-off of adaptation: the compromise between speed of adaptation and precision in the final solution. We defined the learning curve, which we called the thermometer of learning, that we will also use over and over. Therefore this chapter covers the basic concepts for the intriguing adventure of designing systems that learn directly from data.
We have also provided a project to help you understand the power of adaptive systems. The applications of the Adaline are bounded by our imagination and the data we can find to train it. Thus, knowing how to get data from the Web and how to use it in NeuroSolutions is of great value.