12:6 Specification and Implementation of Replicated List: The Jupiter Protocol Revisited
H. Wei, Y. Huang, and J. Lu 12:7
u
u
′v
v
′op′ op
op⟨op′
⟩ op′⟨op⟩
Figure 3Illustration of an OT of two operations op, op0 in both then- ary ordered state space of CJupiter and the 2D state space of Jupiter:
(ophop0i, op0hopi) = OT(op, op0). In the CJupiter and Jupiter protocols (and Examples 4 and 13), op corres-
ponds to the new incoming operation to be transformed.
0ϵ
1x
1:Ins(x,0)
12ϵ 13
ax
123a
2:Del
(x,0) 3:
Ins (a,
0)
3:Ins (a,0)
2:Del (x,1)
124 b
4:Ins(b,0)
1234 ba
3:Ins(a,1) 4:Ins
(b,0)
14 xb
4:Ins(b,1)
2:Del (x,0) S, C1
C2 C3
Figure 4The same finaln-ary ordered state space (thus for CSSs and each CSSci) constructed by CJupiter for each replica under the schedule of Figure 1. Each replica behavior (i.e., the sequence of state transitions) corresponds to a path going through this state space.
Local Processing Part. When a client receives an operationo∈ Ofrom a user, it 1. applieso locally, obtaining a new listval∈Val;
2. generates op ∈ Op by attaching to o a unique operation identifier and the operation context S.cur.oids, representing the set of operations that are causally beforeop;
3. creates a vertex v withv.oids=S.cur.oids∪ {op.oid}, appendsv toS by linking it to S.cur via an edge labeled withop, and updatescur to bev;
4. sendsopto the server asynchronously and returns valto the user.
Server Processing Part. To establish the total order ‘≺s’ on operations, the server maintains the setsoidsof operations it has executed. When the server receives an operationop∈Op from clientci, it
1. updates op.sctxto besoidsand updatessoidsto includeop.oid;
2. transformsopwith an operation sequence inS to obtainop0 by callingS.xForm(op) (see below), and applies op0 (specifically,op0.o) locally;
3. sendsop(instead of op0) to other clients asynchronously.
Remote Processing Part. When a client receives an operationop∈Opfrom the server, it transformsopwith an operation sequence inS to obtainop0 by calling S.xForm(op) (see below), and appliesop0 (specifically, op0.o) locally.
O P O D I S 2 0 1 8
12:8 Specification and Implementation of Replicated List: The Jupiter Protocol Revisited
CSS0c3
0ϵ
CSS1c3
0ϵ
1x
1:Ins(x,0)
CSS2c
3
0ϵ
1 x
1:Ins(x,0)
14 xb
4:
Ins (b,
1)
CSS3c3 0ϵ
1x
1:Ins(x,0)
12ϵ 14
xb
4:Ins (b,1) 2:Del
(x,0)
124 b
2:Del (x,0) 4:Ins
(b,0)
CSS4c
3
0ϵ
1x
1:Ins(x,0)
12ϵ 13
ax
123a
2:Del
(x,0) 3:
Ins (a,
0)
3:Ins (a,0)
2:Del (x,1)
14 xb
4:Ins(b,1)
124 b
2:Del (x,0) 4:Ins(b,0)
1234 ba
3:Ins (a,1) 4:
Ins (b,0) C3
Figure 5Illustration of clientc3 in CJupiter under the schedule of Figure 1. Its behavior (i.e., the sequence of state transitions) is indicated by the path in CSS4c3. (Please refer to Figure B.1 of [25] for the illustration of clientsc1andc2 and the servers.)
OTs in CJupiter. The procedure S.xForm(op : Op) transforms op with an operation sequence in ann-ary ordered state spaceS. Specifically, it
1. locates the vertexuwhoseoidsmatches thectxofop, i.e.,u.oids=op.ctx4, and creates a vertexvwith v.oids=u.oids∪ {op.oid};
2. iteratively transformsopwith an operation sequence consisting of operations along the first edges fromuto the final vertexcurofS (Figure 3):
a. obtains the vertexu0 and the operationop0 associated with the first edge ofu;
b. transformsopwithop0 to obtainophop0iandop0hopi;
c. creates a vertexv0 withv0.oids=v.oids∪ {op0.oid};
d. linksv0 to vvia an edge labeled with op0hopiandv touvia an edge labeled withop;
e. updatesu,v, andopto beu0,v0, andophop0i, respectively;
3. whenuis the final vertexcurofS, linksv touvia an edge labeled withop, updatescur to bev, and returns the last transformed operation op.
To keep track of the construction of then-ary ordered state spaces in CJupiter, for each state space, we introduce a superscriptkto refer to the one after the k-th step (i.e., after processingkoperations), counting from 0. For instance, the state space CSSci (resp. CSSs) after thek-th step maintained by clientci (resp. the servers) is denoted by CSSkci (resp.
CSSks). This notational convention also applies to Jupiter (reviewed in Section 4.1).
IExample 4 (Illustration of CJupiter). Figure 5 illustrates clientc3in CJupiter under the schedule of Figure 1. For convenience, we denote, for instance, a vertexv with v.oids= {o1, o4} by v14 and an operation o3 with o3.ctx = {o1, o2} by o3{o1, o2}. We have also mixed the notations of operations of typesOandOp when no confusion arises. We map various vertices and operations in this example to the ones (i.e.,u, u0, v, v0, op, op0) used in the description of the CJupiter protocol.
After receiving and applyingo1=Ins(x,0) of clientc1from the server, clientc3generates o4=Ins(b,1). It applieso4 locally, creates a new vertexv14, and appends it to CSS1c3 via an edge fromv1 labeled witho4{o1}. Then,o4{o1} is propagated to the server.
4 The vertexuexists due to the FIFO communication between the clients and the server.
H. Wei, Y. Huang, and J. Lu 12:9
Next, clientc3receiveso2=Del(x,0) of clientc1from the server. The operation context ofo2is{o1}, matching theoidsofv1(u). ByxForm,o2{o1}(op) is transformed witho4{o1} (op0): OT
o2{o1}=Del(x,0), o4{o1}=Ins(b,1)
=
o2{o1, o4}=Del(x,0), o4{o1, o2}= Ins(b,0)
. As a result,v124 (v0) is created and is linked tov12 (v) andv14 (u0) via the edges labeled witho4{o1, o2}ando2{o1, o4}, respectively. Becauseo2is unaware ofo4 at the server (o4.sctx=∅now), the edge fromv1 tov12 is ordered before (to the left of) that fromv1 to
v14 in CSS3c3.
Finally, clientc3 receiveso3{o1}=Ins(a,0) of client c2 from the server. The operation context ofo3is {o1}, matching theoidsof v1 (u). ByxForm,o3{o1} will be transformed with the operation sequence consisting of operations along thefirst edges fromv1to the final vertexv124 of CSS3c
3, namelyo2{o1} fromv1 ando4{o1, o2} fromv12. Specifically, o3{o1} (op) is first transformed witho2{o1} (op0): OT
o3{o1}=Ins(a,0), o2{o1}=Del(x,0)
= o3{o1, o2}=Ins(a,0), o2{o1, o3}=Del(x,1)
. Sinceo3is aware ofo2but unaware of o4 at the server, the new edge fromv1labeled witho3{o1}is placed before that witho4{o1}but after that witho2{o1}. Then,o3{o1, o2}(op) is transformed witho4{o1, o2} (op0), yielding v1234 ando3{o1, o2, o4}. Clientc3applieso3{o1, o2, o4}, obtaining the list content ba.
The choice of the “first” edges in OTs is necessary to establish equivalence between CJupiter and Jupiter, particularlyat the server side. First, the operation sequence along the first edges from a vertex of CSSsat the server admits a simple characterization.
I Lemma 5 (CJupiter’s “First” Rule). Let OP = hop1, op2, . . . , opmi (opi ∈ Op) be the operation sequence the server has currently processed in total order ‘≺s’. For any vertexv in the current CSSs, the path along thefirstedges fromv to the final vertex of CSSsconsists of the operations ofOP \v in total order ‘≺s’ (may be empty if v is the final vertex of CSSs), where
OP \v=n
op∈OP |op.oid∈ {op1.oid, op2.oid,· · · , opm.oid} \v.oidso .
IExample 6 (CJupiter’s “First” Rule). Consider CSSsat the server shown in Figure 4 under the schedule of Figure 1; see Figure B.1a of [25] for its construction. Suppose that the server has processed all four operations. That is, we takeOP =ho1, o2, o3, o4i in Lemma 5 (we mix operations of types O andOp). Then, the path along the first edges from vertex v1 (resp. v13) consists of the operationsOP \v1={o2, o3, o4} (resp. OP \v13={o2, o4}) in
total order ‘≺s’.
Based on Lemma 5, the operation sequence with which an operation transformsat the server can be characterized as follows, which is exactly the same with that for Jupiter [26].
ILemma 7 (CJupiter’s OT Sequence). In xForm of CJupiter, the operation sequence L (may be empty) with which an operationoptransformsat the serverconsists of the operations that are both totally ordered by ‘≺s’ before and concurrent by ‘k’with op. Furthermore, the operations inLare totally ordered by ‘≺s’.
IExample 8 (CJupiter’s OT Sequence). Consider the behavior of the server summarized in Figure 4 under the schedule of Figure 1. According to Lemma 5, the operation sequence with whichop=o4transforms consists of operationso2(i.e.,o2{o1}) from vertexv1 ando3
(i.e.,o3{o1, o2}) from vertexv12 in total order ‘≺s’, which are both totally ordered by ‘≺s’ before and concurrent by ‘k’ witho4.
O P O D I S 2 0 1 8
12:10 Specification and Implementation of Replicated List: The Jupiter Protocol Revisited
3.3 CJupiter is Compact
Although (n+ 1) n-ary ordered state spaces are maintained by CJupiter for a system with nclients, they are all the same. That is, at a high level, CJupiter maintains only a single n-ary ordered state space.
IProposition 9(n+ 1⇒1). In CJupiter, the replicas that have processed the same set of operations (in terms of their oids) have the samen-ary ordered state space.
Informally, this proposition holds because we have kept all “by-product” states/vertices of OTs in then-ary ordered state spaces, and each client is “synchronized” with the server.
Since all replicas will eventually process all operations, the finaln-ary ordered state spaces at all replicas are the same. The construction order may differ replica by replica.
IExample 10 (CJupiter is Compact). Figure 4 shows the same finaln-ary ordered state space constructed by CJupiter for each replica under the schedule of Figure 1. (Figure B.1 of [25] shows the step-by-step construction for each replica.) Each replica behavior (i.e., the sequence of state transitions) corresponds to a path going through this state space. As illustrated, the serversand client c1 go along the pathv0
o1
−→v1 o2
−→v12 o3
−→v123 o4
−→v1234, clientc2 goes along the pathv0 −→o1 v1−→o3 v13−→o2 v123−→o4 v1234, and clientc3 goes along the pathv0
o1
−→v1 o4
−→v14 o2
−→v124 o3
−→v1234.
Together with the fact that the OT functions satisfy CP1, Proposition 9 implies that ITheorem 11(CJupiter|=Acp). CJupitersatisfies the convergence propertyAcp.