Multimedia Devices Laboratory

Qiangfu Zhao Professor	Yong Liu Associate Professor

The main stream in our lab is related to computational intelligence. So far we have focused our study on three key words: recognition, learning and understanding. The goal of our research is to develop some learning models that are flexible enough to adapt changing environment, and also simple enough to be realized, interpreted and re-used. The ultimate goal is to design a system that can think, and decide what to do and how to grow-up based on its own thinking. For this purpose, many approaches have been studied - e.g., neuro-computation, evolutionary computation, reinforcement learning, and so on. Of course, results proposed in conventional symbol based artificial intelligence are also included. So far we have used or proposed the following learning models: Neural network trees (NNTrees), Nearest neighbor classifier trees (NNC-Trees), Neural network ensembles, Modular neural networks, Cellular automata, and Recurrent neural networks. Based on the above learning models, we have proposed many new algorithms. Examples include: IEA: individual evolutionary algorithm (also called the R4-rule), CoopCEA: cooperative co-evolutionary algorithms, EPNet: evolutionary programming neural net, and Evolutionary design of neural network trees. Induction of NNC-Trees with the R4-rule. To verify and to improve the models and learning algorithms proposed so far, we have being studying on-line growing of neural network trees, evolution of neural network ensemble, evolutionary design of decision trees, and so on. Currently, we are very interested in applying our models and algorithms to solving practical problems related to producing a "safe, secure and healthy" society. Examples include: face detection, face/expression recognition, image compression and protection, medical diagnosis, and data mining.

Refereed Journal Papers

[qf-zhao-01:200]	Qiangfu Zhao. Inducing NNC-Trees with the R4-rule. IEEE Trans. on Systems, Man, and Cybernetics - Part B: Cybernetics, 36(3):520.533, 2006.
	An NNC-Tree is a decision tree (DT) with each non-terminal nodecontaining a nearest neighbor classifier (NNC). Compared with theconventional axis-parallel DTs (APDTs), the NNC-Trees can be moreefficient because the decision boundary made by an NNC is morecomplex than an axis-parallel hyperplane. Compared with single layerNNCs, the NNC-Trees can classify given data in a hierarchicalstructure which is often useful for many applications. In thispaper, we propose an algorithm for inducing NNC-Trees based on theR4-rule, which was proposed by the author for finding thesmallest NN-MLPs (nearest neighbor based multilayer perceptrons).There are mainly two contributions here. First, a heuristic buteffective method is given to define the teacher signals (grouplabels) for the data assigned to each non-terminal node. Second, theR4-rule is modified so that an NNC with proper size can bedesigned automatically in each non-terminal node. Experiments withseveral public databases show that the proposed algorithm canproduce NNC-Trees effectively and efficiently.
[qf-zhao-02:2006]	Hazem M. El-Bakry and Q. F. Zhao. Fast Normalized Neural Processors For Pattern Detection Based on CrossCorrelation Implemented in the Frequency Domain. Journal of Research and Practice in Information Technology, 38(2):151.170, 2006.
	Neural networks have shown good results for detecting acertain patterns in a given image. In this paper, fast neural networksfor pattern detection are presented. Such neural processors aredesigned based on cross correlation in the frequency domain betweenthe input image and the weights of neural networks. New generalformulas for fast cross correlation as well as an accurate speed upratio are given. Also, commutative cross correlation isachieved. Furthermore, the principle of divide and conquer is appliedthrough image decomposition. Compared with conventional and fastneural networks, experimental results show that a further speedup isachieved using the proposed method. This paper also proves thatcentering of sub-images is equivalent to centering of the weightsmatrices of the neural networks. Therefore, the pre-normalized neuralnetworks are illumination independent to some extent

Refereed Proceeding Papers

[qf-zhao-03:2006]	H. Hayashi and Q. F. Zhao. Evolving NNTrees more efficiently. In IEEE, editor, Proc. IEEE International Congress on Evolutionary Computation (CEC06), pages 2638.2643, Vancouver, July 2006. IEEE, IEEE.
	Neural network tree (NNTree) is a decision tree (DT)with each non-terminal node containing an expert neural network (ENN).Generally speaking, NNTrees can outperform standard axis-parallel DTsbecause the ENNs can extract more complex features. However, inductionof multivariate DTs is very difficult. Even if each non-terminal nodecontains a simple oblique hyperplane, finding the optimal testfunction is an NP-complete problem. To solve this problem, we havestudied two evolutionary algorithms (EAs). One is to induce theNNTrees by applying the genetic algorithm (GA) recursively, andanother is to evolve the NNTrees directly. These two algorithms
[qf-zhao-04:2006]	K. Sakamoto and Q. F. Zhao. A study on generating good environment patterns for evolving robot navigators. In IEEE, editor, Proc. IEEE International Conference on Systems, Man and Cybernetics (SMC06), pages 3280.3285, Taipei, Oct. 2006. IEEE, IEEE.
	To evolve robot navigators that generalize well, weshould evaluate the navigators using as many environment patterns aspossible during evolution. To reduce the computational cost, however
[qf-zhao-05:2006]	Q. F. Zhao. Inducing NNC-Trees quickly. In IEEE, editor, Proc. IEEE International Conference on Systems, Man and Cybernetics (SMC06), pages 2784.2789, Taipei, Oct. 2006. IEEE, IEEE.
	An NNC-Tree is a decision tree (DT) with eachnon-terminal node containing a nearest neighbor classifier (NNC).Compared with the axis-parallel decision trees (APDTs), NNC-Trees aremore comprehensible for large problems, because the decision rulescorresponding to the trees are simpler. Currently, the author hasproposed an algorithm for inducing NNC-Trees based on theR4-rule. However, compared with C4.5, which is a popular program forinducing APDTs, the computation of our algorithm is relativelyexpensive. This paper proposes two methods for reducing thecomputational cost. The efficiency of the proposed methods isverified through experiments on three public databases.
[qf-zhao-06:2006]	S. Kondo and Q. F. Zhao. A novel steganographic technique based on image morphing. In J. Ma et al, editor, Lecture Notes in Computer Science 4159, Springer, pages 806.815, Wuhan, Sept. 2006. Huazhong University of Science and Technology, Springer.
	Steganography is the technology of hiding messages insuch a way that no one except the authorized recipient knows theexistence of the messages. In steganography, a message is hidden insome cover message. The larger the cover message is relative to thehidden message, the easier it is to hide the latter. When the hiddenmessage is an image, it is difficult to hide the message into anotherimage unless the size (in number of bits) of the hidden image is muchsmaller than that of the cover image. To solve this problem, thispaper proposes a novel steganographic technique based on imagemorphing. The basic idea is to transform the hidden image into amorphing image, and use the morphing image as the stego message. Theauthorized recipient can recover the hidden image from the morphingimage through demorphing. The morphing image can also be used directlyfor certain purposes.
[yliu-01:2006]	Y. Liu. Create Stable Neural Networks by Cross-Validation. In Proceedings of the IEEE World Cpngress onComputational Intelligence (WCCIf06, pages 7656.7659. IEEE Computational Intelligence Society, IEEE Press, 2006.
	This paper studies how to learn a stable neural networkthrough the use of cross-validation. Cross-validationhas been widely used for estimating the performanceof neural networks and early stopping of training.Although crossvalidation could give a good estimate ofthe generalisation errors of the trained neural networks.
[yliu-02:2006]	Y. Liu. Evolving Neural Network Ensembles by Fitness Sharing. In Proceedings of the IEEE World Cpngress onComputational Intelligence (WCCIf06, pages 11058.11062. IEEE Computational Intelligence Society, IEEE Press, 2006.
	The difference between evolving neural networks and evolvingneural network ensembles is that the solution of evolvingneural networks is an evolved neural network while the solutionof evolving neural network ensemble is an evolved population ofneural networks. In the practice of evolving neural networkensemble, it is common that each individual rather the wholepopulation is evaluated. During the evolution, the solutionof evolving neural networks would be better and betterwhile it might not be the case for the solution of evolvingneural network ensembles. It suggests that the final evolvedpopulation might be worse so that it is not wise to choosethe final population as a solution. Through experimental studies

Book

[yliu-03:2006]	T. Higuchi, Y. Liu, and X. Yao. Evolvable Hardware. Springer, 2006.
[yliu-04:2006]	L. Kang, Y. Liu, and S. Zeng. Intelligence Computation and Applications. Number 4683 in Lecture Notes in Computer Science. Springer, 2007.
[yliu-05:2006]	L. Kang, Y. Liu, and S. Zeng. Evolvable Systems: From Biology to Hardware. Number 4684 in Lecture Notes in Computer Science. Springer, 2007.

Chapters in Book

[yliu-06:2006]	T. Higuchi, Y. Liu, and X. Yao. Introduction to Evolvable Hardware, page 18. Evolvable Hardware. Springer, 2006.
[yliu-07:2006]	Y. Liu. How to Generate Different Neural Networks, page 16. Trends in Neural Computation, Lecture Notes in Computer Science. Springer, 2006.

Academic Activities

[yliu-8:2006]	Y. Liu, 2004-. Editor, International Journal of Hybrid Intelligent Systems
[yliu-9:2006]	Y. Liu, 2005-2006. One of editors of the book Evolvable Hardware by Springer
[yliu-10:2006]	Y. Liu, 2006-2007. Program Chair of the 7th InternationalConference on Evolvable Systems: From Biology to Hardware(ICES2007)
[yliu-11:2006]	Y. Liu, 2006-2007. General Co-Chair of the 2nd International Symposium onIntelligence Computation and Applications (ISICA2007)

Ph.D, Master and Graduation Theses

[qf-zhao-07:2006]	Hazem Mokhtar Mokhtar El-Bakry. Doctor Thesis: A fast searching algorithm for data detection using neural networks, University of Aizu, 2006. Thesis Advisor: Qiangfu Zhao
[qf-zhao-08:2006]	Tomohiro Tsuchiya. Graduation Thesis: Learning inverse system of robot arm by neural network, University of Aizu, 2006. Thesis Advisor: Qiangfu Zhao
[qf-zhao-09:2006]	Chuanfeng Lv. Doctor Thesis: Non-linear PCA approaches for semi-universal image compression, University of Aizu, 2006. Thesis Advisor: Qiangfu Zhao
[qf-zhao-10:2006]	Ryohei Shindow. Graduation Thesis: Spam e-mail detection based on Bayesian filter, University of Aizu, 2006. Thesis Advisor: Qiangfu Zhao
[qf-zhao-11:2006]	Naoki Tominaga. Graduation Thesis: Fast induction of NNCTrees based on dimensionality reduction, University of Aizu, 2006. Thesis Advisor: Qiangfu Zhao
[qf-zhao-12:2006]	Kei Sato. Graduation Thesis: Frontal face recognition based on PCA, University of Aizu, 2006. Thesis Advisor: Qiangfu Zhao
[qf-zhao-13:2006]	Hiroyuki Kobayashi. Master Thesis: Fact detection with clusteringbased LDA, University of Aizu, 2006. Thesis Advisor: Qiangfu Zhao
[qf-zhao-14:2006]	Akihiko Tamura. Master Thesis: A study on face recognition based on linear approaches, University of Aizu, 2006. Thesis Advisor: Qiangfu Zhao
[qf-zhao-15:2006]	Yoshihiko Watanabe. Graduation Thesis: A study on the efficacy of R4-rule based NNC-Tree design, University of Aizu, 2006. Thesis Advisor: Qiangfu Zhao

2008 THE UNIVERSITY OF AIZU ALL RIGHTS RESERVED