Greetings from
NeuroDimension!
Makers of NeuroSolutions,
the Neural Network Simulation Environment.
This issue of the
newsletter discusses the use of adaptive step sizes and introduces several
offerings from NeuroDimension, Inc.
In this issue you’ll find:
What’s News?
* Neural Network Courses May 8-12 in Orlando
Designing Neural
Networks
* Training with Adaptive Step Sizes
* Porosity, permeability and TOC prediction from well logs
Products and Events
of Interest
Note: You are receiving this newsletter because you requested to stay informed concerning new developments at NeuroDimension. If you would like to stop receiving these newsletters, please see the bottom of this newsletter for removal instructions.
=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=
What’s News?
Now is your last chance to sign up for this year’s May neural network courses in Orlando. The courses will take place May 8-12, 2000 in Orlando, Florida at the Doubletree Guest Suites Hotel in the Walt Disney World Resort.
Our course format allows both novice and advanced users to find a suitable course. Offered courses include: "Introduction to NeuroSolutions", "Fundamentals of Neural Networks and NeuroSolutions", and "Advanced NeuroSolutions". In addition, class sizes are kept small enough to allow for individualized instruction and problem analysis.
The May courses are filling up quickly and the next courses are set to be scheduled in November, so be sure to contact us soon if you are interested in participating.
For details on the May courses, or to sign-up from the internet, see http://www.nd.com/course/may.htm
For general ND course information, see http://www.nd.com/course
We have made a recent addition to the list of products available for purchase from our web site – a data preprocessor created by the Safit company. This comprehensive set of tools is designed to provide analysis and processing on your data before using it within a neural network. This preprocessing will often make it much easier for the neural network to learn the data, which can dramatically improve the network's performance.
To obtain more information and download a free evaluation copy, please visit: http://www.nd.com/products/safit.htm
Note: NeuroDimension is only a reseller of this third-party product. Any technical support (other than basic installation problems) should be directed to the Safit company: support@safit.kiev.ua , phone: (38044) 443-34-16
=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=
Designing Neural Networks
Neural networks are trained with gradient descent learning. The step size determines how fast the algorithm moves down the gradient towards the optimal value of the parameters or weights. Too large a step size and the algorithm may diverge, too small and the algorithm will take a long time to reach the optimal position. Determining the best step size is highly dependent upon the data in your specific problem and it is largely a trial and error effort to set the step size appropriately.
Adaptive step size algorithms adjust the step size each weight update, essentially searching for the best step size. For example, the delta-bar-delta algorithm increases the step size when the error is consistently of the same sign, meaning that the training process is converging consistently (and slowly) in the same direction. If the error is consistently changing its sign, the training process is rattling or unstable so the algorithm reduces the step size. Because of the potential for losing everything learned thus far, the delta-bar-delta algorithm reduces the step size geometrically (i.e. it is multiplied by a constant). It increases the algorithm arithmetically (i.e. it adds a small value). An interesting and powerful feature of NeuroSolutions is that adaptive step size algorithms can change the step size on each individual weight independently (not just by layers).
To use the delta-bar-delta gradient descent rule, simply change the “learning rule” in the Neural Wizard from Momentum to DeltaBarDelta. If you have already created the breadboard, you can drag-and-drop the gradient descent components individually or change them all at once using the inspector of the backprop controller. Simply click the “remove” button on the right hand side of the “backpropagation” page. Then select DeltaBarDelta instead of Momentum and click the “add” button. There are two parameters in the DeltaBarDelta component that must be adjusted. First, because the step sizes are adjustable, you should be somewhat conservative in your initial choice of the step size. For example, start at 0.01 instead of 0.1. On the “Delta Bar Delta” page, you should set the “additive” box to be between 10 and 100 times lower than the initial step size (e.g. 0.001). This is your additive increment to the step size. The multiplicative scale factor is fine at 0.1, meaning that you will decrease the step size by a factor of 10 when the algorithm is unstable.
In many cases, the delta-bar-delta algorithm can converge much quicker than momentum training. Another example of an adaptive step size algorithm is by Professor Almeida and is implemented in a DLL that can be downloaded from: http://www.nd.com/dll/AdaptiveStep.htm
This DLL is also showcased in a DLL demo in NeuroSolutions. To see the Adaptive Step Size DLL in action, please run the NeuroSolutions Demos (within the Help menu), select "Dynamic Link Library Demos", click "Run", then select "Adaptive Step Size" and click "Run".
For more information on adaptive
step sizes, see the interactive book, Section 4.3, Other Search
Procedures.
=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=
Alpana Bhatt,1,2
and Hans B. Helle1
Norsk Hydro, E&P
Research Centre, N-5020 Bergen, Norway
Norwegian University of Science and Technology
Dept. of Petroleum Engineering and Applied Geophysics
N-7034 Trondheim, Norway
Abstract. Apart from their obvious impact on reservoir flow patterns, porosity and permeability are the key variables for characterizing the sediments in a basin-scale fluid migration analysis. For basin modeling involving quantification of hydrocarbon generation and its migration pathways, the organic content of the source rock is one of the key inputs.
Several relationships have been offered which can relate porosity to wireline readings such as the sonic transit time and density logs. However, the conversion from density and transit time to equivalent porosity values is not trivial. The common conversion formulas contain terms and factors that depend on the individual location and lithology; e.g. grain density and grain transit time for the conversion from density and sonic log, respectively, that in general are unknown and remain to be determined.
Permeability is also recognized as a complex function of several interrelated factors such as lithology, pore fluid composition and porosity. Thus, estimates from well logs often rely upon porosity; e.g. through the Kozeny-Carman equation that also contains adjustable factors such as the Kozeny constant that varies within the range 5-100 depending on the reservoir rock and grain geometry (Rose and Bruce, 1949). Prediction of organic carbon (TOC) from wireline logs using the DlogR technique of Passey et al. (1990) is also tied to model parameters that include an exponent (LOM) that characterize the source rock maturity.
Motivated by the results of artificial neural network (ANN) modeling by Huang and Williamson (1995, 1997) applied to porosity, permeability and TOC prediction from wireline data offshore Canada, we applied this new approach to data from the North Sea. Our results clearly indicate that from a high-quality standard log suite, the prediction of formation properties by ANN is simple to implement and, moreover, implies accuracy exceeding that of conventional conversion techniques based on semi-empirical formulas or linear regression analysis.
NOTE: This is just an abstract of the application summary. The entire summary is available at:
http://www.nd.com/application%20summaries/appsum-sed.htm
Want to have your solutions spotlighted? We strongly encourage our customers to send their 1-2 page application summaries to submissions@nd.com so that we may post them on our web site at: http://www.nd.com/applicationsum.htm. We frequently spotlight solutions in our newsletters and include a link for people to get more information.
=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=
NeuroDimension supports the 2001 WSES International Conference on Neural Networks and Applications (NNA'01) that will be held in Puerto De La Cruz, Tenerife, Canary Islands -- February 11-15, 2001. Visit http://www.worldses.org/wses/nna or email to nna@worldses.org for more information.
Have an event or product you would like to announce? Contact us at submissions@nd.com.
=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=
We appreciate your feedback! Please send us your comments or
suggestions concerning this newsletter, our web site, or part of the
NeuroDimension product line. Write to us at: feedback@nd.com
Have questions about NeuroDimension products or training services? Send your questions to: info@nd.com
This issue and previous issues of this newsletter are available on the NeuroDimension web site at: http://www.nd.com/mailinglist.htm
If you would prefer not to receive these newsletters or subsequent product updates from NeuroDimension, please reply to this letter with the subject heading changed to the word REMOVE.
Thank you again for your support of NeuroDimension products!