Advantages and Limitations of Neural Networks

Keywords: advantages of neural networks, limitations of neural networks, neural networks analysis

There are many advantages and limitations to neural network analysis and to discuss this subject properly we would have to look at each individual type of network, which isn’t necessary for this general discussion. In reference to backpropagational networks however, there are some specific issues potential users should be aware of.

• Backpropagational neural networks (and many other types of networks) are in a sense the ultimate ‘black boxes’. Apart from defining the general archetecture of a network and perhaps initially seeding it with a random numbers, the user has no other role than to feed it input and watch it train and await the output. In fact, it has been said that with backpropagation, “you almost don’t know what you’re doing”. Some software freely available software packages (NevProp, bp, Mactivation) do allow the user to sample the networks ‘progress’ at regular time intervals, but the learning itself progresses on its own. The final product of this activity is a trained network that provides no equations or coefficients defining a relationship (as in regression) beyond it’s own internal mathematics. The network ‘IS’ the final equation of the relationship.
• Backpropagational networks also tend to be slower to train than other types of networks and sometimes require thousands of epochs. If run on a truly parallel computer system this issue is not really a problem, but if the BPNN is being simulated on a standard serial machine (i.e. a single SPARC, Mac or PC) training can take some time. This is because the machines CPU must compute the function of each node and connection separately, which can be problematic in very large networks with a large amount of data. However, the speed of most current machines is such that this is typically not much of an issue.

The advantage of neural networks over conventional programming lies on their ability to solve problems that do not have an algorithmic solution or the available solution is too complex to be found. Neural networks are well suited to tackle problems that people are good at solving, like prediction and pattern recognition (Keller). Neural networks have been applied within the medical domain for clinical diagnosis (Baxt:95), image analysis and interpretation (Miller:92, Miller:93), signal analysis and interpretation, and drug development (Weinstein:92). The classification of the applications presented below is simplified, since most of the examples lie in more than one category (e.g. diagnosis and image interpretation; diagnosis and signal interpretation). Depending on the nature of the application and the strength of the internal data patterns you can generally expect a network to train quite well. This applies to problems where the relationships may be quite dynamic or non-linear. ANNs provide an analytical alternative to conventional techniques which are often limited by strict assumptions of normality, linearity, variable independence etc. Because an ANN can capture many kinds of relationships it allows the user to quickly and relatively easily model phenomena which otherwise may have been very difficult or imposible to explain otherwise.

Future Enhancements

Because gazing into the future is somewhat like gazing into a crystal ball, so it is better to quote some “predictions”. Each prediction rests on some sort of evidence or established trend which, with extrapolation, clearly takes us into a new realm.

Prediction 1:

Neural Networks will fascinate user-specific systems for education, information processing, and entertainment. “Alternative ralities”, produced by comprehensive environments, are attractive in terms of their potential for systems control, education, and entertainment. This is not just a far-out research trend, but is something which is becoming an increasing part of our daily existence, as witnessed by the growing interest in comprehensive “entertainment centers” in each home.

This “programming” would require feedback from the user in order to be effective but simple and “passive” sensors (e.g fingertip sensors, gloves, or wristbands to sense pulse, blood pressure, skin ionisation, and so on), could provide effective feedback into a neural control system. This could be achieved, for example, with sensors that would detect pulse, blood pressure, skin ionisation, and other variables which the system could learn to correlate with a person’s response state.

Prediction 2:

Neural networks, integrated with other artificial intelligence technologies, methods for direct culture of nervous tissue, and other exotic technologies such as genetic engineering, will allow us to develop radical and exotic life-forms whether man, machine, or hybrid.

Prediction 3:

Neural networks will allow us to explore new realms of human capability realms previously available only with extensive training and personal discipline. So a specific state of consciously induced neurophysiologically observable awareness is necessary in order to facilitate a man machine system interface.

Recommendations

The major issues of concern today are the scalability problem, testing, verification, and integration of neural network systems into the modern environment. Neural network programs sometimes become unstable when applied to larger problems. The defence, nuclear and space industries are concerned about the issue of testing and verification. The mathematical theories used to guarantee the performance of an applied neural network are still under development. The solution for the time being may be to train and test these intelligent systems much as we do for humans. Also there are some more practical problems like:

• the operational problem encountered when attempting to simulate the parallelism of neural networks. Since the majority of neural networks are simulated on sequential machines, giving rise to a very rapid increase in processing time requirements as size of the problem expands. Solution: implement neural networks directly in hardware, but these need a lot of development still.
• “¢ instability to explain any results that they obtain. Networks function as “black boxes” whose rules of operation are completely unknown.

There are many advantages and limitations to neural network analysis and to discuss this subject properly we would have to look at each individual type of network, which isn’t necessary for this general discussion. In reference to backpropagational networks however, there are some specific issues potential users should be aware of.

• “¢ Backpropagational neural networks (and many other types of networks) are in a sense the ultimate ‘black boxes’. Apart from defining the general archetecture of a network and perhaps initially seeding it with a random numbers, the user has no other role than to feed it input and watch it train and await the output. In fact, it has been said that with backpropagation, “you almost don’t know what you’re doing”. Some software freely available software packages (NevProp, bp, Mactivation) do allow the user to sample the networks ‘progress’ at regular time intervals, but the learning itself progresses on its own. The final product of this activity is a trained network that provides no equations or coefficients defining a relationship (as in regression) beyond it’s own internal mathematics. The network ‘IS’ the final equation of the relationship.
• “¢ Backpropagational networks also tend to be slower to train than other types of networks and sometimes require thousands of epochs. If run on a truly parallel computer system this issue is not really a problem, but if the BPNN is being simulated on a standard serial machine (i.e. a single SPARC, Mac or PC) training can take some time. This is because the machines CPU must compute the function of each node and connection separately, which can be problematic in very large networks with a large amount of data. However, the speed of most current machines is such that this is typically not much of an issue.

Conclusion

In this paper, we have presented a system for recognizing handwritten English characters. An experimental result shows that backpropagation network yields good recognition accuracy of 85%. We have demonstrated the application of MLP network to the handwritten character recognition problem. The skeletonized and normalized binary pixels of these characters were used as the inputs of the MLP network. In our further research work, we would like to improve the recognition accuracy of network for character recognition by using more training samples written by one person and by using a good feature extraction system. The training time may be reduced by using a good feature extraction technique and instead of using global input, we may use the feature input along with other neural network classifier.

The computing world has a lot to gain from neural networks. Their ability to learn by example makes them very flexible and powerful. Furthermore there is no need to devise an algorithm in order to perform a specific task; i.e. there is no need to understand the internal mechanisms of that task. They are also very well suited for real time systems because of their fast response and computational times which are due to their parallel architecture.

Neural networks also contribute to other areas of research such as neurology and psychology. They are regularly used to model parts of living organisms and to investigate the internal mechanisms of the brain.

Perhaps the most exciting aspect of neural networks is the possibility that some day ‘conscious’ networks might be produced. There are a number of scientists arguing that consciousness is a ‘mechanical’ property and that ‘conscious’ neural networks are a realistic possibility.

Finally, I would like to state that even though neural networks have a huge potential we will only get the best of them when they are integrated with computing, AI, fuzzy logic and related subjects