A partition on the state set S of a system is a collection of sub- sub-sets of S such that each state in S belongs to one and only one such subset

104

2. Decomposition

Chapter II. Markov Chains

Definition 2.1: A set S' of states of a Markov system

A is

a persistent

Let C

=

(S,{C(o')}) be a Markovian system which is decomposable. It follows from Eq. (10) above and the remark after that the system must satisfy the following:

Lumpability condition: There exists a nontrivial partition on the state set S such that for any0', the sum of the columns of the matrixC(o')corresponding to any block of the partition, is a column having equal values in entries corre-sponding to the same block of the partition.

Remark:

One sees easily that the partition in the lumpability condition above is represented in Eq. (10) by the first part of the row [or column] double index, i.e., two states are in the same block if they have the samei in their row-ik index (or the same j in their column-j/index). Thus, summing up allCik,)I(O') for fixedj [i.e., in a given block] results in a value which depends on

i

[Le., on the corresponding block] but not on k.

It follows now from (12) that if the system (S,{C(O')})is decomposable, one must also have the following:

Condition of Separability: There exist two nontrivial partitions on the state set,1twith blocks1t)andl' with blocks 1'1 such that: (I) 11t)

n I'll <

^{I for all}

j

and /; (2) if1t)

n

1'/

= j/,

then for allikand all0'

I;

C/k,)/(O')

I;

Cik,)/(O')

=

cik,J/(O')

I )

Remark:

The partitions1tandl'in the separability condition are represented in Eq. (12) by the first and second part of the column [or row] index corre-spondingly. Thus two states are in the same block of1tif they have the same j and they are in the same block ofl'if they have the same / in theirj/-column index.

The previous considerations suggest the following:

Theorem

2.1:

A Markov system (S,{C(O')}) is decomposable if and only if it satisfies the conditions of lumpability and separability with the same1tpartition in both conditions.

Proof' Necessity has been proved already. It is easy to show that the con-ditions are also sufficient, for if a system (S,{C(O')}) satisfies the two conditions, then, by a proper reindexing of the entries of the matricesC(O') into double indices: C(O')

=

[cik,J/(O')]withi,j ranging over the blocks of1tand k, / rang-ing over the blocks of1',one can define the matrices A(O')andB(i,0')by way of the Eqs. (10) and (11). [If for some / andk,1t/

n

l'k

=

0 then this represents a "dno't care" condition and the corresponding entries in the B(i,0') matrices canbechosen at will.]

I

The decomposition procedure will be illustrated in the following example.

Example 13:

Let C

=

^(S,{C(O')}) be a Markov system such that S

=

{I, 2, 3, 4, 5} [for the sake ofsimplicity the states are identified with their index if no ambiguity results], 1:

=

^{{a, b},and}

ChapterII. Markov Chains

1

0 0 0 0

1 1

0 0 0

"2" "2"

C(b)

= 0

"2"¹

0

"2¹

0

1 1 1 1

0

"3" 6 "3" 6

i 0 i 0

{-i 0 i 0

{-C(a)

= {- i i

~

0,

o i 0 {- 0

o 0 {- {- 0 i 0

~

0 i

Consider the partitions1t= (1th 1t_z,1t_J) = ({I, 2},

p,

4}, {5}) and 'r = ('rio 'r_z) =

({I,

3,

5},

{2,

4}).

It is easy to verify that 1t satisfies the lumpability condition and 1t and 'r satisfy the separability condition. Using Eq. (10) we have

:E).x,

cij(a) = ak/(a),i E 1tk;k,1=

1, 2, 3,

or

106

[

^"4

I 1 IJ

^"4 "2"

A(a)

= i i ^0,

010 [ 10 0]

A(b)

= {- {- 0

1 1 1

"3" "3" "3"

Using Eq. (11) now we have

:E).T,

cda)=

bim,

a), if

k

E 'r/(11tm

=t= 0;

m= 1,2,3;i,/=

1,2. Ifm=3andi=2,then'r/(1'r_m = 0, and the values

bim, a)

can be chosen for this case in an arbitrary way subject to the condition that theB(i, a)matrices are stochastic. Choosing b_zl(3,a) = b_zz(3,a)= {- for

a

=

a,

bwe have

B(l, a)= [;

~J.

^B(2,a)=

^[~ tJ.

^{B(3,a) = [:}

:J

B(I, b) =

[~ ~J.

^B(2,b)=

^[~ ~J.

^{B(3,b) = [;}

~J

and the decomposition is completely defined.

Corollary2.2: Let (S, {C(a)}) be a Markov system such that there are two nontrivial partitions1tand 'r on its state set satisfying the following properties:

1. Both1tand 'r satisfy the lumpability cQndition.

2.

1tand 'r satisfy the separability condition.

Then the system is decomposable into a Kronecker product of two systems.

Proof' Consider again Eq. (11) and let bk/(i,a),bk/(j,a) be two different elements in its right-hand side with fixed k, I, a. bk/(i, a)is the sum of the ele-ments corresponding to the block 'r/ of 'r in a row corresponding to the block 1t1of1tin the matrixC(a)[to be more specific, the index of the rowis1t/(1'rk].

Similarly, bk/(j, a)is the sum of the elements in the row with index1t)(1 'rk

corresponding to the block 'r/ of 'r in c(a). It follows from the fact that 'r satisfies the lumpability condition that bk/(j, a) = bk/(i, a), the summation being over entries in the same block of 'r('r/) and the rows belonging to the same block of 'r('r_{k ).} Thus B(i, a) = BU, a) for all pairs i,j so that the B

system can be represented in the form (S", fB(a)}) and is independent on the state of the systemA, which proves the corollary.

I

Remarks:

a. Given a Markov system (S,fe(a)}) which satisfies the lumpability con-dition above one can still use Eq. (10) to define a new system(S', fA(a)})with

IS'I < lSI

and such that the original system is homomorphic to the new one [Le., there is a mapping

if>

from S to S' such that al}(x)

= I:'E

~-I(j)Ck'(X), k E if>-I(i) for all x E 1:*; the states in

S'

will be the blocks of 11: and if m E 11:₀ thenif>(m)

=

n]. The new system is; however, not isomorphic to the original one which cannot be recovered back from it. Some of the information on the transition probabilities from a particular state to another is lost in the lumping process and only the information about the transition probabilities from a block of states to another block is retained. [see Exercises 1,2 at the end of this section.]

b. The set of all partitions over a set of states, including the trivial parti-tions have a lattice structure. One can define a partial order

<

over partitions, where 11:< 't' means that each block of T is the union of one or more blocks of 11:. Thus if

S = {I,

2, 3, 4}, 11:

=

(fI, 2}, f3}, f4}) 't'

= ({I,

2, 3}, f4}), then 11:<'t'.

Let 1 be the partition with all the states in a single block and 0 the partition with each state in a separate block and, using the partial order defined above, define 11:

+

't' to be lub(11:, 't') and 11:' 't' to be glb(11:, T). Clearly 0 < 11: < 1 for any partition 11: and, as the lattice of partitions over a finite set in finite, 11:

+

^l'

and 11:' T always exist. Thus

({I,

2}, f3}, f4, 5, 6})

+ ^(P},

f2, 3} f4, 5}, f6}),

=

(fI, 2, 3}f4, 5, 6}) and (P}, f2}, f3}, f4, 5}, f6}) is the product of the above two partitions. In addition to the above properties, one can also prove the following:

Theorem 2.3: If 11: andl'are two partitions over the set of states S of a Markov system(S,fC(a)}) such that both partitions satisfy the lumpability condition and in addition

(13) then 11:' T is a partition satisfying the Iumpability condition.

Proof: Because of the Iumpability condition for both 11: andl'the sum

I:/EO.

cu<a) has the same value for alli E 11:, and the sum I:/E<.cl}(a) has the same value for all

i

E 't'gwhere 11:, and't'gare arbitrary blocks in 11: andl' respective-ly. It follows that the sum I:/E".M,CIj(a)has the same value for all i E 11:,nTg •

But 11:, n Tgand 11:_k n 't'Jare arbitrary blocks of 11:'l'and all the blocks of 11:'l'

have this form, which proves the theorem.

I

Using the algebra of partitions and the theorem above one can find all pos-sible pairs of partitions satisfying the necessary conditions for decomposition.

It is to be mentioned, however, that, in contrast to the deterministic case, there

108 Chapter II. Markov Chains

exist no clear-cut theory of decomposition for Markov systems. The conditions of lumpability and separability are restrictive and cannot be both satisfied in general.

c. Generalizations of the results in this section can be achieved through in-troducing combinatorial gates between the various parts in an interconnection of systems or through combinations of various types of decomposition. In ad-dition a decomposition can be carried through several steps leading to more than two component subsystems. Those possibilities have been mentioned before.

There may be also decompositions based on interconnections more general than the cascade type as will be shown later. [See exercises 9-12 at the end of this section.] Still another possibility is the possibility of state splitting. This will be illustrated now by the following:

Example14:

Let

(S,{C(u)})

be the 3-state system over

I:

=

fa, b}

with

C(a)=

l~ _itO : :1,

^{C(b)= [: : :]}

~

t t

An easy check will show that the above system is not decomposable. One can try, however, to split some state into two, to get another 4-state system which will be decomposable into two 2-state components. Suppose some state say

Si

is split into two (or more) states

s;,

and

Si"

i.e., the ith row in each matrix is duplicated and then the ith column is divided into two columns whose sum is equal to the original one. Trivially, the new system satisfies the lumpability condition for the partition which will merge the states

Sit

and

Si,

into a single block and leaving all the other states alone. The new system is therefore equivalent to the old one provided that the states

^Sit

and

Si,

are merged at its output, and a decomposition of the new system provides us, therefore, with a decomposition of a system which is externally equivalent to the original one.

In our example one may try to split the second state so as to have a 4-state system with matrices

and the

ail

and

bij

will be determined by a series of equations requiring that:

(1) The sum of the two

a

columns and the two

b

columns equal to the

cor-responding columns in the original matrices

C(a)

and

C(b);

(2) there is a

parti-tion 1C say 1C = ((SIS2}, {S3S4}} which satisfies the lumpability condition; and (3)

there is a partition

'r

say

'r

= ({Sl S3}. {S2S4}}, such that 1C and

'r

satisfy the

separability condition. Formulation of these equations is an easy matter to do and is left as an exercise. The resulting matrices are

C'(a)

= l~

^{2 4 1 2 '~}

:

^~

^~:1

^{~ ~ C'(b)}

⁼ l: "; ::l

^{~2 1 4 2~ ~ ~}

! 0 t O t t t t

A decomposition of the system is now obtained in the same way as in Example 13. The resulting decomposition (for1tand 't'as specified above) is

A(a)

= [; :}

A(b)

= [: ;J

B(a, 1)

= [~~J.

^{B(b, 1)}

^{= [:} :J

B(a,2)=

[~~J.

^{B(b,2)= [;}

:J

d. In deterministic machine theory, it has been proved that, by properly splitting the states of an n-state machine one can always decompose an external-ly equivalent machine, in a cascade form, into two component machines, one of them having a set of transition matrices which are either permutation or reset matrices and the other having onlyn-1 states. This fact has served as a _bas~

for the classical theorem of Krohn and Rhodes (1963) showing that every de-terministic machine can be "embedded" into a cascade interconnection of a sequence of machines of a certain simple and cannonical form. Unifortunately, it seems reasonable to assume that the Krohn-Rhodes theorem does not carry over, in its original form, to the stochastic

case.

One of the reasons for this is that even if state splitting is allowed the conditions for cascade decomposability seem to be restrictive for stochastic systems and cannot always be met. Note, however, that a cascade interconnection of a sequence of systems A_lo

A

_2,••• ,

Ak has the property that the next state of a system AIin the interconnection depends on the present input, on its present state and on the present state of all other systemsAj withj

<

^i, but does not depend on the present state of any systemAj withj

>

i. This means that the interconnectivity in the decom-position is not maximal a fact which has some advantage from the realiza-tion point of view. We will show now that if the interconnectivity is allowed to be maximal, then any n-state Markov system can be decomposed into a sequence of 2-state Markov systems.

Definition2.6: Let

A

=

(S, [A(CT,t)J"El;)

rET and

110 Chapter II. Markov Chains

be two Markov systems. The system (S x

T, (C(O')}Uq;)

is the maximal inter-connection of

A

and

B

if

C(O')

=

[Cst,.,,,(O')]

and

cst,•.,.{O')

=

a...(O', t)btt,(O', s) (14) ([a...(O',t)] = A(O', t); [briO',s)] = B(O',s».

Thus, in a maximal interconnection, the next state of each system depends on the present state of both systems and on the present input.

It

is easily proved that a maximal interconnection of two Markov systems is a Markov system. A maximal interconnection reduces to a cascade intercon-nection if all the matrices of one of the two component systems corresponding to the, same input,

0',

are equal. Once the maximal interconnection of two systems is formed the resulting system can be further maximally interconnected with a third system and so on. The resulting system will be called a maximal interconnection of the sequence of systems involved. Definition 2.6 is illustrated in Figure 15.

\---1

I I

I

A

I I

I I

-~-+----l

I

I

I ^I

I

L -l

I

C

Figure 15. Graphical representation of a maximal interconnection of Markov systems.

We are now able to state the following:

Theorem 2.4:

For each n-state Markov system

A

=

(S, (A(O')))

there exist two

No documento Computer Mathematics (páginas 122-128)