He received a PhD from the University of Rouen in 1993 in the area of collaboration in classification and neural networks for pattern recognition applications. The method studied in Section 3 achieves the first step of this goal by determining the clusters in the learning set: this clustering provides a «natural» decomposition of the problem. The principles of the cooperative scheme in the distributed and modular neural classifier are also presented in Section 4.
The designer must consider all these parameters in the combination scheme to obtain an optimal behavior regarding the classification performances. This therefore implies determining the number and composition of the groups in the learning data set. The main problem with these methods is that in practice, the number of clusters is required in advance to obtain a good representation of the data.
This choice has a large impact on the representational capabilities of the hierarchy, and subsequently, on the clustering capabilities. One of the problems that can arise is a large difference in node heights due to significant density changes in the data. The configuration in Fig.2 shows that (i) the data must first be considered from a global perspective and (ii) that some dense (and well-structured) parts must be detailed to obtain a reliable representation of reality.
This criterion, called the coefficient of variation in the literature [38] is used as a measure of the homogeneity of a population.
Construction of Cooperative Modular Classifier
An example of such a function is given in [32] and allows us to evaluate the clustering quality in a supervised sense. More details and discussion of this method are also given in [32], and Fig.4 gives the obtained result of the example of Fig.2 without any parameter estimation (α is arbitrarily fixed to 1). Indeed, some of the elements are located close to the boundaries, so they will not appear in a clean subtree.
The percentage of learning database elements associated with an island can therefore also vary widely. As a result, neural networks do not learn the entire database, but only its most reliable parts. Non-parametric methods seem to be the most suitable as they do not require any actual learning phase and their computational cost can be greatly reduced by the collaborative process.
Moreover, this remaining part of the training set seems to belong to unreliable regions (inhomogeneous clusters, overlapping regions..). The collaboration with neural networks was carried out as follows: we use all neural networks for the unknown sample; if only one of them recognizes the element, it takes that decision, otherwise it takes the K-NN decision. This combination strategy is basically simple, knowing that there are many other and often more sophisticated methods available.
Such an option was indeed made in order to emphasize the resulting influence in the performance improvement and in the simplification design phase by the fact that the data topology was taken into account in the distribution task. Other more sophisticated and complex combination methods can be used and there remain opportunities for future developments and improvements of this work. This hierarchical architecture of the neural classifier is, in principle, close to a decision tree [9,29].
In fact, it can be seen as a special decision tree where the nodes and leaves are the neural network. The main difference with the latter is that in our case the assignment of tasks of different modules (neural networks) is based on cluster analysis.
Experimental results : a handwritten digit recognition problem
An average of 120 islets of more than P=15 elements were detected, and 76% of the training set was assigned to an islet. So it can be said that, as expected, learning an island is a fairly simple problem. These statistics show that the modular classifier always provides better performances than the single MLP.
The performance curve of the distributed classifier is close to that of K-NN, except for low error rates. So for a 0% error rate, the distributed classifier's recognition rate is 14% higher than that of K-NN, while in this configuration only 41% of the decisions are made by the networks. By training neural networks on islands, they can recognize an element from one class, while the 50 or 55 nearest neighbors are from another class.
Boundaries generated by islet learning (even if a simple network-building rule is followed) are therefore particularly efficient. It can be noted that few neural networks rarely implement such boundaries, as there is no explicit learning rule to find them.
Conclusion
Further experiments should lead to a better characterization of these limits to provide clear rules for high-performance network construction. 1] Alimoglu F., alpaydin E., Combining Multiple Representations and Classifiers for Handwriting Based Digit Recognition, In Proceedings of ICDAR'1997, IAPR- International Conference on Document Analysis and Recognition, IEEE Computer Society pp7 ], U., Smieja F, Learning from examples, teams of agents and the concept of reflection, International Journal of Pattern Recognition and Artificial Intelligence.
Combining Neural and Statistical Algorithms for Supervised Remote Sensing Image Classification, Pattern Recognition Letters 21, p. 16] Hebert J.F., Parizeau M., Ghazzali N., A new hybrid ANN architecture for active and incremental learning: a self-organizing perceptron (SOP) Network. 35] Serpico S.B., Bruzzone L., Roli F., Experimental comparison of neural and statistical nonparametric algorithms for supervised classification of remote sensing images, Pattern Recognition Letters 17, 1996, p. 1331-1341.
36] Vuurpijl L., Schomaker L., Finding structure in diversity: a hierarchical clustering method for the categorization of allographs in handwriting, Proceedings of ICDAR’97, 4th International Conference on Document Analysis and Recognition, vol. Tillidskombinationsmetoder i multiekspertsystemer, Lecture Notes in Computer Science, N° 1876, Advances in Pattern Recognition, Proceedings of joint international workshops SSPR.
