3.6 Complex-valued deep Boltzmann machines
3.6.2 Experimental results
For each test vector, the lower bound is maximized with respect to the parametersµusing the mean-field equations above.
Figure 3.3: MNIST images generated by the real-valued (left) and complex-valued (right) DBMs, along with training images (center) [171]
Table 3.13: Experimental results for MNIST Hidden layer sizes Real Avg.
Log-Prob.
Complex Avg.
Log-Prob.
500,1000 −86.59 −86.37
1000,1000 −86.31 −86.16
500,1000,1000 −85.48 −85.23 1000,1000,1000 −85.16 −84.95 Hidden layer sizes Real Error Complex Error
500,1000 1.36% 1.23%
1000,1000 1.38% 1.32%
500,1000,1000 1.35% 1.21%
1000,1000,1000 1.31% 1.20%
Figure 3.4: FashionMNIST images generated by the real-valued (left) and complex-valued (right) DBMs, along with training images (center) [171]
Table 3.14: Experimental results for FashionMNIST Hidden layer sizes Real Avg.
Log-Prob.
Complex Avg.
Log-Prob.
500,1000 −236.95 −231.34 1000,1000 −228.43 −225.76 500,1000,1000 −221.76 −216.85 1000,1000,1000 −216.52 −213.63 Hidden layer sizes Real Error Complex Error
500,1000 10.01% 9.93%
1000,1000 9.86% 9.65%
500,1000,1000 10.13% 9.96%
1000,1000,1000 10.04% 9.87%
Dynamics of complex-valued neural networks (CVNNs)
Over the last few years, there has been an increasing interest in the domain of recurrent neu- ral networks, especially the following types of models: Hopfield [80, 81], Cohen-Grossberg [44], cellular [42, 43], and bidirectional associative memory neural networks [100], mainly be- cause of their applications in many areas such as classification, optimization, signal and image processing, solving nonlinear algebraic equations, pattern recognition, system identification, associative memories, cryptography, and so on. These applications are highly dependent on the dynamical properties of the networks. Thus, the analysis of the dynamical behavior is an important part in the design of the recurrent neural networks used in applications.
The study of the dynamics of recurrent neural networks has become a field of study in its own right, attracting many researchers. The review paper [238] lists more than 300 refer- ences only for the stability analysis of continuous-time recurrent neural networks, not taking into account the discrete-time ones, or other dynamical properties that can be studied such as bifurcation, attractivity, dissipativity, passivity, synchronization, and so on.
On the other hand, complex-valued neural networks have been proposed by Aizenberg [1], but have caught the attention of researchers in the past years, especially due to their applica- tions in physical systems dealing with electromagnetic, ultrasonic, quantum, and light waves, and also in filtering, imaging, optoelectronics, speech synthesis, computer vision, and so on (see, for example, [119, 77]). In these applications, the stability of the complex-valued neural networks plays a very important role. Moreover, they have more complicated properties than the real-valued neural networks because of their complex-valued states, connection weights, and activation functions. The activation functions cannot be a simple generalization of the real- valued ones, because, by Liouville’s theorem, it can be deduced that a bounded entire function is a constant, which makes the choice of such functions more difficult. As a consequence, the study of the dynamic behavior of the complex-valued recurrent neural networks has received increasing interest, especially in the last few years.
4.1 µ-Stability of neutral-type impulsive BAM CVNNs with leakage delay and unbounded time-varying delays
Since they were first introduced by Kosko in [100], bidirectional associative memories, an ex- tension of the unidirectional auto-associative Hopfield neural networks, were intensely studied, and have many applications in pattern recognition, signal and image processing, and automatic
73
control.
Time delays are known to appear in practical implementations of neural networks due to the finite switching speed of amplifiers, and can cause instability or chaotic behavior. Past deriva- tive information is also considered to influence the present state in neutral-type systems. These systems more accurately describe the properties of neural reaction processes that occur in the real world. The existence of neutral-type delays makes the study of these systems more compli- cated than that of the usual time-delayed models. This type of delays are relevant in automatic control, population dynamics, and vibrating masses attached to an elastic bar. Also, neutral delays may appear when implementing neural networks in VLSI circuits. On the other hand, impulsive effects express instantaneous changes that naturally occur in electronic networks, caused by switching phenomena, frequency changes, or noise. Taking the above analysis into account, the neutral-type impulsive complex-valued BAM neural networks with leakage de- lay and unbounded time-varying delays will be studied by giving sufficient conditions for the existence and uniqueness of the equilibrium point and for its globalµ-stability.
The following notations will be used in this section:Rdenotes the set of real numbers,Cthe set of complex numbers,Z+the set of positive integer numbers,Rndenotes then-dimensional Euclidean space, andCnthen-dimensional unitary space. Real matrices of dimensionn×mare denoted as Rn×m, and similarly complex matrices of dimensionn×mas Cn×m. AT denotes the transpose of matrix A, A∗ denotes the Hermitian transpose of matrix A, and ? denotes the symmetric or conjugate symmetric terms in a matrix. In denotes the identity matrix of dimension n. λmin(P) is defined as the smallest eigenvalue of positive definite matrix P. |z|
stands for the norm of the complex numberz. ||z||stands for the Euclidean norm of the complex vectorz. A >0(A < 0) means thatAis a positive definite (negative definite) matrix. ARand AI, denote, respectively, the real and imaginary parts of complex matrixA.
The presentation in this section follows that in the author’s paper [173].