<HTML><BODY><div><div><font face="Courier New"><font style="caret-color: rgb(0, 0, 0); color: rgb(0, 0, 0); font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-size-adjust: auto; -webkit-text-stroke-width: 0px; text-decoration: none;">The aim of this RFC is to address the following list of problems</font></font></div><div><font face="Courier New"><font style="caret-color: rgb(0, 0, 0); color: rgb(0, 0, 0); font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-size-adjust: auto; -webkit-text-stroke-width: 0px; text-decoration: none;">formulated at MRG planning meeting:</font></font></div><div><font face="Courier New"><font style="caret-color: rgb(0, 0, 0); color: rgb(0, 0, 0); font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-size-adjust: auto; -webkit-text-stroke-width: 0px; text-decoration: none;"> - protocol backward compatibility to enable cluster upgrade w/o</font></font></div><div><font face="Courier New"><font style="caret-color: rgb(0, 0, 0); color: rgb(0, 0, 0); font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-size-adjust: auto; -webkit-text-stroke-width: 0px; text-decoration: none;"> downtime<br> - consistency of data on replica and leader<br> - switch from leader to replica without data loss<br> - up to date replicas to run read-only requests<br> - ability to switch async replicas into sync ones<br> - guarantee of rollback on leader and sync replicas<br> - simplicity of cluster orchestration</font></font></div><div style="-webkit-text-stroke-width:0px; text-align:start; text-indent:0px"> </div><div style="-webkit-text-stroke-width:0px; text-align:start; text-indent:0px"><span style="font-size:12px"><span style="caret-color:#000000"><span style="color:#000000"><span style="font-family:Helvetica"><span style="font-style:normal"><span style="font-variant-caps:normal"><span style="font-weight:normal"><span style="letter-spacing:normal"><span style="orphans:auto"><span style="text-transform:none"><span style="white-space:normal"><span style="widows:auto"><span style="word-spacing:0px"><span style="-webkit-text-size-adjust:auto"><span style="text-decoration:none"><font face="Courier New">What this RFC is not:</font></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></div><div style="-webkit-text-stroke-width:0px; text-align:start; text-indent:0px"> </div><div style="-webkit-text-stroke-width:0px; text-align:start; text-indent:0px"><span style="font-size:12px"><span style="caret-color:#000000"><span style="color:#000000"><span style="font-family:Helvetica"><span style="font-style:normal"><span style="font-variant-caps:normal"><span style="font-weight:normal"><span style="letter-spacing:normal"><span style="orphans:auto"><span style="text-transform:none"><span style="white-space:normal"><span style="widows:auto"><span style="word-spacing:0px"><span style="-webkit-text-size-adjust:auto"><span style="text-decoration:none"><font face="Courier New"> - high availability (HA) solution with automated failover, roles</font></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></div><div style="-webkit-text-stroke-width:0px; text-align:start; text-indent:0px"><span style="font-size:12px"><span style="caret-color:#000000"><span style="color:#000000"><span style="font-family:Helvetica"><span style="font-style:normal"><span style="font-variant-caps:normal"><span style="font-weight:normal"><span style="letter-spacing:normal"><span style="orphans:auto"><span style="text-transform:none"><span style="white-space:normal"><span style="widows:auto"><span style="word-spacing:0px"><span style="-webkit-text-size-adjust:auto"><span style="text-decoration:none"><font face="Courier New"> assignments an so on</font></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></div><div style="-webkit-text-stroke-width:0px; text-align:start; text-indent:0px"><span style="font-size:12px"><span style="caret-color:#000000"><span style="color:#000000"><span style="font-family:Helvetica"><span style="font-style:normal"><span style="font-variant-caps:normal"><span style="font-weight:normal"><span style="letter-spacing:normal"><span style="orphans:auto"><span style="text-transform:none"><span style="white-space:normal"><span style="widows:auto"><span style="word-spacing:0px"><span style="-webkit-text-size-adjust:auto"><span style="text-decoration:none"><font face="Courier New"> - master-master configuration support</font></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></div><div style="-webkit-text-stroke-width:0px; text-align:start; text-indent:0px"> </div><div style="-webkit-text-stroke-width:0px; text-align:start; text-indent:0px"><span style="font-size:12px"><span style="caret-color:#000000"><span style="color:#000000"><span style="font-family:Helvetica"><span style="font-style:normal"><span style="font-variant-caps:normal"><span style="font-weight:normal"><span style="letter-spacing:normal"><span style="orphans:auto"><span style="text-transform:none"><span style="white-space:normal"><span style="widows:auto"><span style="word-spacing:0px"><span style="-webkit-text-size-adjust:auto"><span style="text-decoration:none"><font face="Courier New">Quorum-confirmed commit details</font></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></div><div style="-webkit-text-stroke-width:0px; text-align:start; text-indent:0px"> </div><div style="-webkit-text-stroke-width:0px; text-align:start; text-indent:0px"><span style="font-size:12px"><span style="caret-color:#000000"><span style="color:#000000"><span style="font-family:Helvetica"><span style="font-style:normal"><span style="font-variant-caps:normal"><span style="font-weight:normal"><span style="letter-spacing:normal"><span style="orphans:auto"><span style="text-transform:none"><span style="white-space:normal"><span style="widows:auto"><span style="word-spacing:0px"><span style="-webkit-text-size-adjust:auto"><span style="text-decoration:none"><font face="Courier New">The main idea behind the proposal is to reuse existent machinery as much <br>as possible. It will ensure the well-tested and proven functionality</font></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></div><div style="-webkit-text-stroke-width:0px; text-align:start; text-indent:0px"><span style="font-size:12px"><span style="caret-color:#000000"><span style="color:#000000"><span style="font-family:Helvetica"><span style="font-style:normal"><span style="font-variant-caps:normal"><span style="font-weight:normal"><span style="letter-spacing:normal"><span style="orphans:auto"><span style="text-transform:none"><span style="white-space:normal"><span style="widows:auto"><span style="word-spacing:0px"><span style="-webkit-text-size-adjust:auto"><span style="text-decoration:none"><font face="Courier New">across many instances in MRG and beyond is used. The transaction rollback</font></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></div><div style="-webkit-text-stroke-width:0px; text-align:start; text-indent:0px"><span style="font-size:12px"><span style="caret-color:#000000"><span style="color:#000000"><span style="font-family:Helvetica"><span style="font-style:normal"><span style="font-variant-caps:normal"><span style="font-weight:normal"><span style="letter-spacing:normal"><span style="orphans:auto"><span style="text-transform:none"><span style="white-space:normal"><span style="widows:auto"><span style="word-spacing:0px"><span style="-webkit-text-size-adjust:auto"><span style="text-decoration:none"><font face="Courier New">mechanism is in place and works for WAL write failure. If we substitute </font></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></div><div style="-webkit-text-stroke-width:0px; text-align:start; text-indent:0px"><span style="font-size:12px"><span style="caret-color:#000000"><span style="color:#000000"><span style="font-family:Helvetica"><span style="font-style:normal"><span style="font-variant-caps:normal"><span style="font-weight:normal"><span style="letter-spacing:normal"><span style="orphans:auto"><span style="text-transform:none"><span style="white-space:normal"><span style="widows:auto"><span style="word-spacing:0px"><span style="-webkit-text-size-adjust:auto"><span style="text-decoration:none"><font face="Courier New">the WAL success with a new situation which is named 'quorum’ later in </font></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></div><div style="-webkit-text-stroke-width:0px; text-align:start; text-indent:0px"><span style="font-size:12px"><span style="caret-color:#000000"><span style="color:#000000"><span style="font-family:Helvetica"><span style="font-style:normal"><span style="font-variant-caps:normal"><span style="font-weight:normal"><span style="letter-spacing:normal"><span style="orphans:auto"><span style="text-transform:none"><span style="white-space:normal"><span style="widows:auto"><span style="word-spacing:0px"><span style="-webkit-text-size-adjust:auto"><span style="text-decoration:none"><font face="Courier New">this </font></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span><span style="font-size:12px"><span style="caret-color:#000000"><span style="color:#000000"><span style="font-family:Helvetica"><span style="font-style:normal"><span style="font-variant-caps:normal"><span style="font-weight:normal"><span style="letter-spacing:normal"><span style="orphans:auto"><span style="text-transform:none"><span style="white-space:normal"><span style="widows:auto"><span style="word-spacing:0px"><span style="-webkit-text-size-adjust:auto"><span style="text-decoration:none"><font face="Courier New">document then no changes to the machinery is needed. The same is </font></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></div><div style="-webkit-text-stroke-width:0px; text-align:start; text-indent:0px"><span style="font-size:12px"><span style="caret-color:#000000"><span style="color:#000000"><span style="font-family:Helvetica"><span style="font-style:normal"><span style="font-variant-caps:normal"><span style="font-weight:normal"><span style="letter-spacing:normal"><span style="orphans:auto"><span style="text-transform:none"><span style="white-space:normal"><span style="widows:auto"><span style="word-spacing:0px"><span style="-webkit-text-size-adjust:auto"><span style="text-decoration:none"><font face="Courier New">true for </font></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span><span style="font-size:12px"><span style="caret-color:#000000"><span style="color:#000000"><span style="font-family:Helvetica"><span style="font-style:normal"><span style="font-variant-caps:normal"><span style="font-weight:normal"><span style="letter-spacing:normal"><span style="orphans:auto"><span style="text-transform:none"><span style="white-space:normal"><span style="widows:auto"><span style="word-spacing:0px"><span style="-webkit-text-size-adjust:auto"><span style="text-decoration:none"><font face="Courier New">snapshot machinery that allows to create a copy of the database</font></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></div><div style="-webkit-text-stroke-width:0px; text-align:start; text-indent:0px"><span style="font-size:12px"><span style="caret-color:#000000"><span style="color:#000000"><span style="font-family:Helvetica"><span style="font-style:normal"><span style="font-variant-caps:normal"><span style="font-weight:normal"><span style="letter-spacing:normal"><span style="orphans:auto"><span style="text-transform:none"><span style="white-space:normal"><span style="widows:auto"><span style="word-spacing:0px"><span style="-webkit-text-size-adjust:auto"><span style="text-decoration:none"><font face="Courier New">in memory for the whole period of snapshot file write. Adding quorum here </font></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></div><div style="-webkit-text-stroke-width:0px; text-align:start; text-indent:0px"><span style="font-size:12px"><span style="caret-color:#000000"><span style="color:#000000"><span style="font-family:Helvetica"><span style="font-style:normal"><span style="font-variant-caps:normal"><span style="font-weight:normal"><span style="letter-spacing:normal"><span style="orphans:auto"><span style="text-transform:none"><span style="white-space:normal"><span style="widows:auto"><span style="word-spacing:0px"><span style="-webkit-text-size-adjust:auto"><span style="text-decoration:none"><font face="Courier New">also minimizes changes.</font></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></div><div style="-webkit-text-stroke-width:0px; text-align:start; text-indent:0px"> </div><div style="-webkit-text-stroke-width:0px; text-align:start; text-indent:0px"><span style="font-size:12px"><span style="caret-color:#000000"><span style="color:#000000"><span style="font-family:Helvetica"><span style="font-style:normal"><span style="font-variant-caps:normal"><span style="font-weight:normal"><span style="letter-spacing:normal"><span style="orphans:auto"><span style="text-transform:none"><span style="white-space:normal"><span style="widows:auto"><span style="word-spacing:0px"><span style="-webkit-text-size-adjust:auto"><span style="text-decoration:none"><font face="Courier New">Currently replication represented by the following scheme:</font></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></div><div style="-webkit-text-stroke-width:0px; text-align:start; text-indent:0px"> </div><div style="-webkit-text-stroke-width:0px; text-align:start; text-indent:0px"><img src="cid:7znS@q1rxkkQF.W5kZohCt" style="width: 546px; height: 741px;"></div><div style="-webkit-text-stroke-width:0px; text-align:start; text-indent:0px"> </div><div style="-webkit-text-stroke-width:0px; text-align:start; text-indent:0px"><p class="p1"> </p><div class="p1"><span style="font-size:12px"><span style="caret-color:#000000"><span style="color:#000000"><span style="font-family:Helvetica"><span style="font-style:normal"><span style="font-variant-caps:normal"><span style="font-weight:normal"><span style="letter-spacing:normal"><span style="orphans:auto"><span style="text-transform:none"><span style="white-space:normal"><span style="widows:auto"><span style="word-spacing:0px"><span style="-webkit-text-size-adjust:auto"><span style="text-decoration:none"><font face="Courier New">To introduce the 'quorum' we have to receive confirmation from replicas</font></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></div><div class="p1"><span style="font-size:12px"><span style="caret-color:#000000"><span style="color:#000000"><span style="font-family:Helvetica"><span style="font-style:normal"><span style="font-variant-caps:normal"><span style="font-weight:normal"><span style="letter-spacing:normal"><span style="orphans:auto"><span style="text-transform:none"><span style="white-space:normal"><span style="widows:auto"><span style="word-spacing:0px"><span style="-webkit-text-size-adjust:auto"><span style="text-decoration:none"><font face="Courier New">to make a decision on whether the quorum is actually present. Leader </font></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></div><div class="p1"><span style="font-size:12px"><span style="caret-color:#000000"><span style="color:#000000"><span style="font-family:Helvetica"><span style="font-style:normal"><span style="font-variant-caps:normal"><span style="font-weight:normal"><span style="letter-spacing:normal"><span style="orphans:auto"><span style="text-transform:none"><span style="white-space:normal"><span style="widows:auto"><span style="word-spacing:0px"><span style="-webkit-text-size-adjust:auto"><span style="text-decoration:none"><font face="Courier New">collects necessary amount of replicas confirmation plus its own WAL </font></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></div><div class="p1"><span style="font-size:12px"><span style="caret-color:#000000"><span style="color:#000000"><span style="font-family:Helvetica"><span style="font-style:normal"><span style="font-variant-caps:normal"><span style="font-weight:normal"><span style="letter-spacing:normal"><span style="orphans:auto"><span style="text-transform:none"><span style="white-space:normal"><span style="widows:auto"><span style="word-spacing:0px"><span style="-webkit-text-size-adjust:auto"><span style="text-decoration:none"><font face="Courier New">success. This state is named 'quorum' and gives leader the right to </font></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></div><div class="p1"><span style="font-size:12px"><span style="caret-color:#000000"><span style="color:#000000"><span style="font-family:Helvetica"><span style="font-style:normal"><span style="font-variant-caps:normal"><span style="font-weight:normal"><span style="letter-spacing:normal"><span style="orphans:auto"><span style="text-transform:none"><span style="white-space:normal"><span style="widows:auto"><span style="word-spacing:0px"><span style="-webkit-text-size-adjust:auto"><span style="text-decoration:none"><font face="Courier New">complete the customers' request. So the picture will change to:</font></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></div><div class="p1"> </div><div class="p1"><span style="font-size:12px"><span style="caret-color:#000000"><span style="color:#000000"><span style="font-family:Helvetica"><span style="font-style:normal"><span style="font-variant-caps:normal"><span style="font-weight:normal"><span style="letter-spacing:normal"><span style="orphans:auto"><span style="text-transform:none"><span style="white-space:normal"><span style="widows:auto"><span style="word-spacing:0px"><span style="-webkit-text-size-adjust:auto"><span style="text-decoration:none"><font face="Courier New"><img src="cid:DBtA@5ueQJQxG.1wqtpoDs" style="width: 542px; height: 1019px;"></font></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></div><div class="p1"> </div><div class="p1"><span style="font-size:12px"><span style="caret-color:#000000"><span style="color:#000000"><span style="font-family:Helvetica"><span style="font-style:normal"><span style="font-variant-caps:normal"><span style="font-weight:normal"><span style="letter-spacing:normal"><span style="orphans:auto"><span style="text-transform:none"><span style="white-space:normal"><span style="widows:auto"><span style="word-spacing:0px"><span style="-webkit-text-size-adjust:auto"><span style="text-decoration:none"><font face="Courier New">The quorum should be collected as a table for a list of transactions </font></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></div><div class="p1"><span style="font-size:12px"><span style="caret-color:#000000"><span style="color:#000000"><span style="font-family:Helvetica"><span style="font-style:normal"><span style="font-variant-caps:normal"><span style="font-weight:normal"><span style="letter-spacing:normal"><span style="orphans:auto"><span style="text-transform:none"><span style="white-space:normal"><span style="widows:auto"><span style="word-spacing:0px"><span style="-webkit-text-size-adjust:auto"><span style="text-decoration:none"><font face="Courier New">waiting for quorum. The latest transaction that collects the quorum is</font></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></div><div class="p1"><span style="font-size:12px"><span style="caret-color:#000000"><span style="color:#000000"><span style="font-family:Helvetica"><span style="font-style:normal"><span style="font-variant-caps:normal"><span style="font-weight:normal"><span style="letter-spacing:normal"><span style="orphans:auto"><span style="text-transform:none"><span style="white-space:normal"><span style="widows:auto"><span style="word-spacing:0px"><span style="-webkit-text-size-adjust:auto"><span style="text-decoration:none"><font face="Courier New">considered as complete, as well as all transactions prior to it, since</font></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></div><div class="p1"><span style="font-size:12px"><span style="caret-color:#000000"><span style="color:#000000"><span style="font-family:Helvetica"><span style="font-style:normal"><span style="font-variant-caps:normal"><span style="font-weight:normal"><span style="letter-spacing:normal"><span style="orphans:auto"><span style="text-transform:none"><span style="white-space:normal"><span style="widows:auto"><span style="word-spacing:0px"><span style="-webkit-text-size-adjust:auto"><span style="text-decoration:none"><font face="Courier New">all transactions should be applied in order. Leader writes a ‘quorum’</font></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></div><div class="p1"><span style="font-size:12px"><span style="caret-color:#000000"><span style="color:#000000"><span style="font-family:Helvetica"><span style="font-style:normal"><span style="font-variant-caps:normal"><span style="font-weight:normal"><span style="letter-spacing:normal"><span style="orphans:auto"><span style="text-transform:none"><span style="white-space:normal"><span style="widows:auto"><span style="word-spacing:0px"><span style="-webkit-text-size-adjust:auto"><span style="text-decoration:none"><font face="Courier New">message to the WAL and it is delivered to Replicas.</font></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></div><div class="p1"> </div><div class="p1"><span style="font-size:12px"><span style="caret-color:#000000"><span style="color:#000000"><span style="font-family:Helvetica"><span style="font-style:normal"><span style="font-variant-caps:normal"><span style="font-weight:normal"><span style="letter-spacing:normal"><span style="orphans:auto"><span style="text-transform:none"><span style="white-space:normal"><span style="widows:auto"><span style="word-spacing:0px"><span style="-webkit-text-size-adjust:auto"><span style="text-decoration:none"><font face="Courier New">Replica should report a positive or a negative result </font></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span><span style="font-size:12px"><span style="caret-color:#000000"><span style="color:#000000"><span style="font-family:Helvetica"><span style="font-style:normal"><span style="font-variant-caps:normal"><span style="font-weight:normal"><span style="letter-spacing:normal"><span style="orphans:auto"><span style="text-transform:none"><span style="white-space:normal"><span style="widows:auto"><span style="word-spacing:0px"><span style="-webkit-text-size-adjust:auto"><span style="text-decoration:none"><font face="Courier New">of the TXN to the </font></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></div><div class="p1"><span style="font-size:12px"><span style="caret-color:#000000"><span style="color:#000000"><span style="font-family:Helvetica"><span style="font-style:normal"><span style="font-variant-caps:normal"><span style="font-weight:normal"><span style="letter-spacing:normal"><span style="orphans:auto"><span style="text-transform:none"><span style="white-space:normal"><span style="widows:auto"><span style="word-spacing:0px"><span style="-webkit-text-size-adjust:auto"><span style="text-decoration:none"><font face="Courier New">Leader via the IPROTO explicitly to allow Leader to collect the quorum </font></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></div><div class="p1"><span style="font-size:12px"><span style="caret-color:#000000"><span style="color:#000000"><span style="font-family:Helvetica"><span style="font-style:normal"><span style="font-variant-caps:normal"><span style="font-weight:normal"><span style="letter-spacing:normal"><span style="orphans:auto"><span style="text-transform:none"><span style="white-space:normal"><span style="widows:auto"><span style="word-spacing:0px"><span style="-webkit-text-size-adjust:auto"><span style="text-decoration:none"><font face="Courier New">or anti-quorum for the TXN. In case negative result for the TXN received</font></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></div><div class="p1"><span style="font-size:12px"><span style="caret-color:#000000"><span style="color:#000000"><span style="font-family:Helvetica"><span style="font-style:normal"><span style="font-variant-caps:normal"><span style="font-weight:normal"><span style="letter-spacing:normal"><span style="orphans:auto"><span style="text-transform:none"><span style="white-space:normal"><span style="widows:auto"><span style="word-spacing:0px"><span style="-webkit-text-size-adjust:auto"><span style="text-decoration:none"><font face="Courier New">from minor number of Replicas, then Leader has to send an error message</font></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></div><div class="p1"><span style="font-size:12px"><span style="caret-color:#000000"><span style="color:#000000"><span style="font-family:Helvetica"><span style="font-style:normal"><span style="font-variant-caps:normal"><span style="font-weight:normal"><span style="letter-spacing:normal"><span style="orphans:auto"><span style="text-transform:none"><span style="white-space:normal"><span style="widows:auto"><span style="word-spacing:0px"><span style="-webkit-text-size-adjust:auto"><span style="text-decoration:none"><font face="Courier New">to each Replica, which in turn has to disconnect from the replication</font></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></div><div class="p1"><span style="font-size:12px"><span style="caret-color:#000000"><span style="color:#000000"><span style="font-family:Helvetica"><span style="font-style:normal"><span style="font-variant-caps:normal"><span style="font-weight:normal"><span style="letter-spacing:normal"><span style="orphans:auto"><span style="text-transform:none"><span style="white-space:normal"><span style="widows:auto"><span style="word-spacing:0px"><span style="-webkit-text-size-adjust:auto"><span style="text-decoration:none"><font face="Courier New">the same way as it is done now in case of conflict.</font></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></div><div class="p1"> </div><div class="p1"><span style="font-size:12px"><span style="caret-color:#000000"><span style="color:#000000"><span style="font-family:Helvetica"><span style="font-style:normal"><span style="font-variant-caps:normal"><span style="font-weight:normal"><span style="letter-spacing:normal"><span style="orphans:auto"><span style="text-transform:none"><span style="white-space:normal"><span style="widows:auto"><span style="word-spacing:0px"><span style="-webkit-text-size-adjust:auto"><span style="text-decoration:none"><font face="Courier New">In case Leader receives enough error messages to do not achieve the </font></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></div><div class="p1"><span style="font-size:12px"><span style="caret-color:#000000"><span style="color:#000000"><span style="font-family:Helvetica"><span style="font-style:normal"><span style="font-variant-caps:normal"><span style="font-weight:normal"><span style="letter-spacing:normal"><span style="orphans:auto"><span style="text-transform:none"><span style="white-space:normal"><span style="widows:auto"><span style="word-spacing:0px"><span style="-webkit-text-size-adjust:auto"><span style="text-decoration:none"><font face="Courier New">quorum it should write the ‘rollback’ message </font></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span><span style="font-size:12px"><span style="caret-color:#000000"><span style="color:#000000"><span style="font-family:Helvetica"><span style="font-style:normal"><span style="font-variant-caps:normal"><span style="font-weight:normal"><span style="letter-spacing:normal"><span style="orphans:auto"><span style="text-transform:none"><span style="white-space:normal"><span style="widows:auto"><span style="word-spacing:0px"><span style="-webkit-text-size-adjust:auto"><span style="text-decoration:none"><font face="Courier New">in the WAL. After that </font></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></div><div class="p1"><span style="font-size:12px"><span style="caret-color:#000000"><span style="color:#000000"><span style="font-family:Helvetica"><span style="font-style:normal"><span style="font-variant-caps:normal"><span style="font-weight:normal"><span style="letter-spacing:normal"><span style="orphans:auto"><span style="text-transform:none"><span style="white-space:normal"><span style="widows:auto"><span style="word-spacing:0px"><span style="-webkit-text-size-adjust:auto"><span style="text-decoration:none"><font face="Courier New">Leader and Replicas will perform </font></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span><span style="font-size:12px"><span style="caret-color:#000000"><span style="color:#000000"><span style="font-family:Helvetica"><span style="font-style:normal"><span style="font-variant-caps:normal"><span style="font-weight:normal"><span style="letter-spacing:normal"><span style="orphans:auto"><span style="text-transform:none"><span style="white-space:normal"><span style="widows:auto"><span style="word-spacing:0px"><span style="-webkit-text-size-adjust:auto"><span style="text-decoration:none"><font face="Courier New">the rollback </font></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span><span style="font-size:12px"><span style="caret-color:#000000"><span style="color:#000000"><span style="font-family:Helvetica"><span style="font-style:normal"><span style="font-variant-caps:normal"><span style="font-weight:normal"><span style="letter-spacing:normal"><span style="orphans:auto"><span style="text-transform:none"><span style="white-space:normal"><span style="widows:auto"><span style="word-spacing:0px"><span style="-webkit-text-size-adjust:auto"><span style="text-decoration:none"><font face="Courier New">for all TXN that didn’t </font></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></div><div class="p1"><span style="font-size:12px"><span style="caret-color:#000000"><span style="color:#000000"><span style="font-family:Helvetica"><span style="font-style:normal"><span style="font-variant-caps:normal"><span style="font-weight:normal"><span style="letter-spacing:normal"><span style="orphans:auto"><span style="text-transform:none"><span style="white-space:normal"><span style="widows:auto"><span style="word-spacing:0px"><span style="-webkit-text-size-adjust:auto"><span style="text-decoration:none"><font face="Courier New">receive quorum.</font></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></div><div class="p1"> </div><div class="p1"><span style="font-size:12px"><span style="caret-color:#000000"><span style="color:#000000"><span style="font-family:Helvetica"><span style="font-style:normal"><span style="font-variant-caps:normal"><span style="font-weight:normal"><span style="letter-spacing:normal"><span style="orphans:auto"><span style="text-transform:none"><span style="white-space:normal"><span style="widows:auto"><span style="word-spacing:0px"><span style="-webkit-text-size-adjust:auto"><span style="text-decoration:none"><font face="Courier New">Recovery and failover.</font></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></div><div class="p1"> </div><div class="p1"><span style="font-size:12px"><span style="caret-color:#000000"><span style="color:#000000"><span style="font-family:Helvetica"><span style="font-style:normal"><span style="font-variant-caps:normal"><span style="font-weight:normal"><span style="letter-spacing:normal"><span style="orphans:auto"><span style="text-transform:none"><span style="white-space:normal"><span style="widows:auto"><span style="word-spacing:0px"><span style="-webkit-text-size-adjust:auto"><span style="text-decoration:none"><font face="Courier New">Tarantool instance during reading WAL should postpone the commit until </font></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></div><div class="p1"><span style="font-size:12px"><span style="caret-color:#000000"><span style="color:#000000"><span style="font-family:Helvetica"><span style="font-style:normal"><span style="font-variant-caps:normal"><span style="font-weight:normal"><span style="letter-spacing:normal"><span style="orphans:auto"><span style="text-transform:none"><span style="white-space:normal"><span style="widows:auto"><span style="word-spacing:0px"><span style="-webkit-text-size-adjust:auto"><span style="text-decoration:none"><font face="Courier New">the quorum is read. In case the WAL eof is achieved, the instance should </font></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></div><div class="p1"><span style="font-size:12px"><span style="caret-color:#000000"><span style="color:#000000"><span style="font-family:Helvetica"><span style="font-style:normal"><span style="font-variant-caps:normal"><span style="font-weight:normal"><span style="letter-spacing:normal"><span style="orphans:auto"><span style="text-transform:none"><span style="white-space:normal"><span style="widows:auto"><span style="word-spacing:0px"><span style="-webkit-text-size-adjust:auto"><span style="text-decoration:none"><font face="Courier New">keep rollback for all transactions that are waiting for a quorum entry </font></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></div><div class="p1"><span style="font-size:12px"><span style="caret-color:#000000"><span style="color:#000000"><span style="font-family:Helvetica"><span style="font-style:normal"><span style="font-variant-caps:normal"><span style="font-weight:normal"><span style="letter-spacing:normal"><span style="orphans:auto"><span style="text-transform:none"><span style="white-space:normal"><span style="widows:auto"><span style="word-spacing:0px"><span style="-webkit-text-size-adjust:auto"><span style="text-decoration:none"><font face="Courier New">until the role of the instance is set. In case this instance become a </font></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></div><div class="p1"><span style="font-size:12px"><span style="caret-color:#000000"><span style="color:#000000"><span style="font-family:Helvetica"><span style="font-style:normal"><span style="font-variant-caps:normal"><span style="font-weight:normal"><span style="letter-spacing:normal"><span style="orphans:auto"><span style="text-transform:none"><span style="white-space:normal"><span style="widows:auto"><span style="word-spacing:0px"><span style="-webkit-text-size-adjust:auto"><span style="text-decoration:none"><font face="Courier New">Replica there are no additional actions needed, sine all info about </font></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></div><div class="p1"><span style="font-size:12px"><span style="caret-color:#000000"><span style="color:#000000"><span style="font-family:Helvetica"><span style="font-style:normal"><span style="font-variant-caps:normal"><span style="font-weight:normal"><span style="letter-spacing:normal"><span style="orphans:auto"><span style="text-transform:none"><span style="white-space:normal"><span style="widows:auto"><span style="word-spacing:0px"><span style="-webkit-text-size-adjust:auto"><span style="text-decoration:none"><font face="Courier New">quorum/rollback will arrive via replication. In case this instance is </font></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></div><div class="p1"><span style="font-size:12px"><span style="caret-color:#000000"><span style="color:#000000"><span style="font-family:Helvetica"><span style="font-style:normal"><span style="font-variant-caps:normal"><span style="font-weight:normal"><span style="letter-spacing:normal"><span style="orphans:auto"><span style="text-transform:none"><span style="white-space:normal"><span style="widows:auto"><span style="word-spacing:0px"><span style="-webkit-text-size-adjust:auto"><span style="text-decoration:none"><font face="Courier New">assigned a Leader role, it should write ‘rollback’ in it’s WAL and </font></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></div><div class="p1"><span style="font-size:12px"><span style="caret-color:#000000"><span style="color:#000000"><span style="font-family:Helvetica"><span style="font-style:normal"><span style="font-variant-caps:normal"><span style="font-weight:normal"><span style="letter-spacing:normal"><span style="orphans:auto"><span style="text-transform:none"><span style="white-space:normal"><span style="widows:auto"><span style="word-spacing:0px"><span style="-webkit-text-size-adjust:auto"><span style="text-decoration:none"><font face="Courier New">perform rollback for all transactions waiting for a quorum.</font></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></div><div class="p1"> </div><div class="p1"><span style="font-size:12px"><span style="caret-color:#000000"><span style="color:#000000"><span style="font-family:Helvetica"><span style="font-style:normal"><span style="font-variant-caps:normal"><span style="font-weight:normal"><span style="letter-spacing:normal"><span style="orphans:auto"><span style="text-transform:none"><span style="white-space:normal"><span style="widows:auto"><span style="word-spacing:0px"><span style="-webkit-text-size-adjust:auto"><span style="text-decoration:none"><font face="Courier New">In case of a Leader failure a Replica with the biggest LSN with former </font></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></div><div class="p1"><span style="font-size:12px"><span style="caret-color:#000000"><span style="color:#000000"><span style="font-family:Helvetica"><span style="font-style:normal"><span style="font-variant-caps:normal"><span style="font-weight:normal"><span style="letter-spacing:normal"><span style="orphans:auto"><span style="text-transform:none"><span style="white-space:normal"><span style="widows:auto"><span style="word-spacing:0px"><span style="-webkit-text-size-adjust:auto"><span style="text-decoration:none"><font face="Courier New">leader’s ID is elected as a new leader. The replica should record </font></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></div><div class="p1"><span style="font-size:12px"><span style="caret-color:#000000"><span style="color:#000000"><span style="font-family:Helvetica"><span style="font-style:normal"><span style="font-variant-caps:normal"><span style="font-weight:normal"><span style="letter-spacing:normal"><span style="orphans:auto"><span style="text-transform:none"><span style="white-space:normal"><span style="widows:auto"><span style="word-spacing:0px"><span style="-webkit-text-size-adjust:auto"><span style="text-decoration:none"><font face="Courier New">‘rollback’ in its </font></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span><span style="font-size:12px"><span style="caret-color:#000000"><span style="color:#000000"><span style="font-family:Helvetica"><span style="font-style:normal"><span style="font-variant-caps:normal"><span style="font-weight:normal"><span style="letter-spacing:normal"><span style="orphans:auto"><span style="text-transform:none"><span style="white-space:normal"><span style="widows:auto"><span style="word-spacing:0px"><span style="-webkit-text-size-adjust:auto"><span style="text-decoration:none"><font face="Courier New">WAL which effectively means that all transactions </font></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></div><div class="p1"><span style="font-size:12px"><span style="caret-color:#000000"><span style="color:#000000"><span style="font-family:Helvetica"><span style="font-style:normal"><span style="font-variant-caps:normal"><span style="font-weight:normal"><span style="letter-spacing:normal"><span style="orphans:auto"><span style="text-transform:none"><span style="white-space:normal"><span style="widows:auto"><span style="word-spacing:0px"><span style="-webkit-text-size-adjust:auto"><span style="text-decoration:none"><font face="Courier New">without quorum should be rolled back. This rollback will be delivered to</font></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></div><div class="p1"><span style="font-size:12px"><span style="caret-color:#000000"><span style="color:#000000"><span style="font-family:Helvetica"><span style="font-style:normal"><span style="font-variant-caps:normal"><span style="font-weight:normal"><span style="letter-spacing:normal"><span style="orphans:auto"><span style="text-transform:none"><span style="white-space:normal"><span style="widows:auto"><span style="word-spacing:0px"><span style="-webkit-text-size-adjust:auto"><span style="text-decoration:none"><font face="Courier New">all replicas and they will perform rollbacks of all transactions waiting </font></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></div><div class="p1"><span style="font-size:12px"><span style="caret-color:#000000"><span style="color:#000000"><span style="font-family:Helvetica"><span style="font-style:normal"><span style="font-variant-caps:normal"><span style="font-weight:normal"><span style="letter-spacing:normal"><span style="orphans:auto"><span style="text-transform:none"><span style="white-space:normal"><span style="widows:auto"><span style="word-spacing:0px"><span style="-webkit-text-size-adjust:auto"><span style="text-decoration:none"><font face="Courier New">for quorum.</font></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></div><div class="p1"> </div><div class="p1"><span style="font-size:12px"><span style="caret-color:#000000"><span style="color:#000000"><span style="font-family:Helvetica"><span style="font-style:normal"><span style="font-variant-caps:normal"><span style="font-weight:normal"><span style="letter-spacing:normal"><span style="orphans:auto"><span style="text-transform:none"><span style="white-space:normal"><span style="widows:auto"><span style="word-spacing:0px"><span style="-webkit-text-size-adjust:auto"><span style="text-decoration:none"><font face="Courier New">Snapshot generation.</font></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></div><div class="p1"> </div><div class="p1"><span style="font-size:12px"><span style="caret-color:#000000"><span style="color:#000000"><span style="font-family:Helvetica"><span style="font-style:normal"><span style="font-variant-caps:normal"><span style="font-weight:normal"><span style="letter-spacing:normal"><span style="orphans:auto"><span style="text-transform:none"><span style="white-space:normal"><span style="widows:auto"><span style="word-spacing:0px"><span style="-webkit-text-size-adjust:auto"><span style="text-decoration:none"><font face="Courier New">We also can reuse current machinery of snapshot generation. Upon</font></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></div><div class="p1"><span style="font-size:12px"><span style="caret-color:#000000"><span style="color:#000000"><span style="font-family:Helvetica"><span style="font-style:normal"><span style="font-variant-caps:normal"><span style="font-weight:normal"><span style="letter-spacing:normal"><span style="orphans:auto"><span style="text-transform:none"><span style="white-space:normal"><span style="widows:auto"><span style="word-spacing:0px"><span style="-webkit-text-size-adjust:auto"><span style="text-decoration:none"><font face="Courier New">receiving a request to create a snapshot an instance should request a</font></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></div><div class="p1"><span style="font-size:12px"><span style="caret-color:#000000"><span style="color:#000000"><span style="font-family:Helvetica"><span style="font-style:normal"><span style="font-variant-caps:normal"><span style="font-weight:normal"><span style="letter-spacing:normal"><span style="orphans:auto"><span style="text-transform:none"><span style="white-space:normal"><span style="widows:auto"><span style="word-spacing:0px"><span style="-webkit-text-size-adjust:auto"><span style="text-decoration:none"><font face="Courier New">readview for the current commit operation. Although start of the</font></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></div><div class="p1"><span style="font-size:12px"><span style="caret-color:#000000"><span style="color:#000000"><span style="font-family:Helvetica"><span style="font-style:normal"><span style="font-variant-caps:normal"><span style="font-weight:normal"><span style="letter-spacing:normal"><span style="orphans:auto"><span style="text-transform:none"><span style="white-space:normal"><span style="widows:auto"><span style="word-spacing:0px"><span style="-webkit-text-size-adjust:auto"><span style="text-decoration:none"><font face="Courier New">snapshot generation should be postponed until this commit operation</font></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></div><div class="p1"><span style="font-size:12px"><span style="caret-color:#000000"><span style="color:#000000"><span style="font-family:Helvetica"><span style="font-style:normal"><span style="font-variant-caps:normal"><span style="font-weight:normal"><span style="letter-spacing:normal"><span style="orphans:auto"><span style="text-transform:none"><span style="white-space:normal"><span style="widows:auto"><span style="word-spacing:0px"><span style="-webkit-text-size-adjust:auto"><span style="text-decoration:none"><font face="Courier New">receives its quorum. In case operation is rolled back, the snapshot</font></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></div><div class="p1"><span style="font-size:12px"><span style="caret-color:#000000"><span style="color:#000000"><span style="font-family:Helvetica"><span style="font-style:normal"><span style="font-variant-caps:normal"><span style="font-weight:normal"><span style="letter-spacing:normal"><span style="orphans:auto"><span style="text-transform:none"><span style="white-space:normal"><span style="widows:auto"><span style="word-spacing:0px"><span style="-webkit-text-size-adjust:auto"><span style="text-decoration:none"><font face="Courier New">generation should be aborted and restarted using current transaction </font></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></div><div class="p1"><span style="font-size:12px"><span style="caret-color:#000000"><span style="color:#000000"><span style="font-family:Helvetica"><span style="font-style:normal"><span style="font-variant-caps:normal"><span style="font-weight:normal"><span style="letter-spacing:normal"><span style="orphans:auto"><span style="text-transform:none"><span style="white-space:normal"><span style="widows:auto"><span style="word-spacing:0px"><span style="-webkit-text-size-adjust:auto"><span style="text-decoration:none"><font face="Courier New">after rollback </font></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span><span style="font-size:12px"><span style="caret-color:#000000"><span style="color:#000000"><span style="font-family:Helvetica"><span style="font-style:normal"><span style="font-variant-caps:normal"><span style="font-weight:normal"><span style="letter-spacing:normal"><span style="orphans:auto"><span style="text-transform:none"><span style="white-space:normal"><span style="widows:auto"><span style="word-spacing:0px"><span style="-webkit-text-size-adjust:auto"><span style="text-decoration:none"><font face="Courier New">is complete.</font></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></div><div class="p1"> </div><div class="p1"><span style="font-size:12px"><span style="caret-color:#000000"><span style="color:#000000"><span style="font-family:Helvetica"><span style="font-style:normal"><span style="font-variant-caps:normal"><span style="font-weight:normal"><span style="letter-spacing:normal"><span style="orphans:auto"><span style="text-transform:none"><span style="white-space:normal"><span style="widows:auto"><span style="word-spacing:0px"><span style="-webkit-text-size-adjust:auto"><span style="text-decoration:none"><font face="Courier New">After snapshot is created the WAL should start from the first operation</font></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></div><div class="p1"><span style="font-size:12px"><span style="caret-color:#000000"><span style="color:#000000"><span style="font-family:Helvetica"><span style="font-style:normal"><span style="font-variant-caps:normal"><span style="font-weight:normal"><span style="letter-spacing:normal"><span style="orphans:auto"><span style="text-transform:none"><span style="white-space:normal"><span style="widows:auto"><span style="word-spacing:0px"><span style="-webkit-text-size-adjust:auto"><span style="text-decoration:none"><font face="Courier New">that follows the commit operation snapshot is generated for. That means</font></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></div><div class="p1"><span style="font-size:12px"><span style="caret-color:#000000"><span style="color:#000000"><span style="font-family:Helvetica"><span style="font-style:normal"><span style="font-variant-caps:normal"><span style="font-weight:normal"><span style="letter-spacing:normal"><span style="orphans:auto"><span style="text-transform:none"><span style="white-space:normal"><span style="widows:auto"><span style="word-spacing:0px"><span style="-webkit-text-size-adjust:auto"><span style="text-decoration:none"><font face="Courier New">WAL will contain a quorum message that refers to a transaction that is</font></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></div><div class="p1"><span style="font-size:12px"><span style="caret-color:#000000"><span style="color:#000000"><span style="font-family:Helvetica"><span style="font-style:normal"><span style="font-variant-caps:normal"><span style="font-weight:normal"><span style="letter-spacing:normal"><span style="orphans:auto"><span style="text-transform:none"><span style="white-space:normal"><span style="widows:auto"><span style="word-spacing:0px"><span style="-webkit-text-size-adjust:auto"><span style="text-decoration:none"><font face="Courier New">not present in the WAL. Apparently, we have to allow this for the case<br>quorum refers to a transaction with LSN less than the first entry in </font></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span><span style="font-size:12px"><span style="caret-color:#000000"><span style="color:#000000"><span style="font-family:Helvetica"><span style="font-style:normal"><span style="font-variant-caps:normal"><span style="font-weight:normal"><span style="letter-spacing:normal"><span style="orphans:auto"><span style="text-transform:none"><span style="white-space:normal"><span style="widows:auto"><span style="word-spacing:0px"><span style="-webkit-text-size-adjust:auto"><span style="text-decoration:none"><font face="Courier New">t</font></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span><span style="font-size:12px"><span style="caret-color:#000000"><span style="color:#000000"><span style="font-family:Helvetica"><span style="font-style:normal"><span style="font-variant-caps:normal"><span style="font-weight:normal"><span style="letter-spacing:normal"><span style="orphans:auto"><span style="text-transform:none"><span style="white-space:normal"><span style="widows:auto"><span style="word-spacing:0px"><span style="-webkit-text-size-adjust:auto"><span style="text-decoration:none"><font face="Courier New">he</font></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></div><div class="p1"><span style="font-size:12px"><span style="caret-color:#000000"><span style="color:#000000"><span style="font-family:Helvetica"><span style="font-style:normal"><span style="font-variant-caps:normal"><span style="font-weight:normal"><span style="letter-spacing:normal"><span style="orphans:auto"><span style="text-transform:none"><span style="white-space:normal"><span style="widows:auto"><span style="word-spacing:0px"><span style="-webkit-text-size-adjust:auto"><span style="text-decoration:none"><font face="Courier New">WAL and only once.</font></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></div><div class="p1"> </div><div class="p1"><span style="font-size:12px"><span style="caret-color:#000000"><span style="color:#000000"><span style="font-family:Helvetica"><span style="font-style:normal"><span style="font-variant-caps:normal"><span style="font-weight:normal"><span style="letter-spacing:normal"><span style="orphans:auto"><span style="text-transform:none"><span style="white-space:normal"><span style="widows:auto"><span style="word-spacing:0px"><span style="-webkit-text-size-adjust:auto"><span style="text-decoration:none"><font face="Courier New">Asynchronous replication.</font></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></div><div class="p1"> </div><div class="p1"><span style="font-size:12px"><span style="caret-color:#000000"><span style="color:#000000"><span style="font-family:Helvetica"><span style="font-style:normal"><span style="font-variant-caps:normal"><span style="font-weight:normal"><span style="letter-spacing:normal"><span style="orphans:auto"><span style="text-transform:none"><span style="white-space:normal"><span style="widows:auto"><span style="word-spacing:0px"><span style="-webkit-text-size-adjust:auto"><span style="text-decoration:none"><font face="Courier New">Along with synchronous Replicas the cluster can contain asynchronous</font></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></div><div class="p1"><span style="font-size:12px"><span style="caret-color:#000000"><span style="color:#000000"><span style="font-family:Helvetica"><span style="font-style:normal"><span style="font-variant-caps:normal"><span style="font-weight:normal"><span style="letter-spacing:normal"><span style="orphans:auto"><span style="text-transform:none"><span style="white-space:normal"><span style="widows:auto"><span style="word-spacing:0px"><span style="-webkit-text-size-adjust:auto"><span style="text-decoration:none"><font face="Courier New">Replicas. That means async Replica doesn’t reply </font></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span><span style="font-size:12px"><span style="caret-color:#000000"><span style="color:#000000"><span style="font-family:Helvetica"><span style="font-style:normal"><span style="font-variant-caps:normal"><span style="font-weight:normal"><span style="letter-spacing:normal"><span style="orphans:auto"><span style="text-transform:none"><span style="white-space:normal"><span style="widows:auto"><span style="word-spacing:0px"><span style="-webkit-text-size-adjust:auto"><span style="text-decoration:none"><font face="Courier New">to the Leader with </font></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></div><div class="p1"><span style="font-size:12px"><span style="caret-color:#000000"><span style="color:#000000"><span style="font-family:Helvetica"><span style="font-style:normal"><span style="font-variant-caps:normal"><span style="font-weight:normal"><span style="letter-spacing:normal"><span style="orphans:auto"><span style="text-transform:none"><span style="white-space:normal"><span style="widows:auto"><span style="word-spacing:0px"><span style="-webkit-text-size-adjust:auto"><span style="text-decoration:none"><font face="Courier New">errors since they’re not contributing into quorum. Still, async </font></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></div><div class="p1"><span style="font-size:12px"><span style="caret-color:#000000"><span style="color:#000000"><span style="font-family:Helvetica"><span style="font-style:normal"><span style="font-variant-caps:normal"><span style="font-weight:normal"><span style="letter-spacing:normal"><span style="orphans:auto"><span style="text-transform:none"><span style="white-space:normal"><span style="widows:auto"><span style="word-spacing:0px"><span style="-webkit-text-size-adjust:auto"><span style="text-decoration:none"><font face="Courier New">Replicas have to follow the new WAL operation, such as keep rollback</font></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></div><div class="p1"><span style="font-size:12px"><span style="caret-color:#000000"><span style="color:#000000"><span style="font-family:Helvetica"><span style="font-style:normal"><span style="font-variant-caps:normal"><span style="font-weight:normal"><span style="letter-spacing:normal"><span style="orphans:auto"><span style="text-transform:none"><span style="white-space:normal"><span style="widows:auto"><span style="word-spacing:0px"><span style="-webkit-text-size-adjust:auto"><span style="text-decoration:none"><font face="Courier New">info until ‘quorum’ message is received. This is essential for the case</font></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></div><div class="p1"><span style="font-size:12px"><span style="caret-color:#000000"><span style="color:#000000"><span style="font-family:Helvetica"><span style="font-style:normal"><span style="font-variant-caps:normal"><span style="font-weight:normal"><span style="letter-spacing:normal"><span style="orphans:auto"><span style="text-transform:none"><span style="white-space:normal"><span style="widows:auto"><span style="word-spacing:0px"><span style="-webkit-text-size-adjust:auto"><span style="text-decoration:none"><font face="Courier New">of ‘rollback’ message appearance in the WAL. This message assumes </font></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></div><div class="p1"><span style="font-size:12px"><span style="caret-color:#000000"><span style="color:#000000"><span style="font-family:Helvetica"><span style="font-style:normal"><span style="font-variant-caps:normal"><span style="font-weight:normal"><span style="letter-spacing:normal"><span style="orphans:auto"><span style="text-transform:none"><span style="white-space:normal"><span style="widows:auto"><span style="word-spacing:0px"><span style="-webkit-text-size-adjust:auto"><span style="text-decoration:none"><font face="Courier New">Replica is able to perform all necessary rollback by itself. </font></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span><span style="font-size:12px"><span style="caret-color:#000000"><span style="color:#000000"><span style="font-family:Helvetica"><span style="font-style:normal"><span style="font-variant-caps:normal"><span style="font-weight:normal"><span style="letter-spacing:normal"><span style="orphans:auto"><span style="text-transform:none"><span style="white-space:normal"><span style="widows:auto"><span style="word-spacing:0px"><span style="-webkit-text-size-adjust:auto"><span style="text-decoration:none"><font face="Courier New">Cluster </font></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></div><div class="p1"><span style="font-size:12px"><span style="caret-color:#000000"><span style="color:#000000"><span style="font-family:Helvetica"><span style="font-style:normal"><span style="font-variant-caps:normal"><span style="font-weight:normal"><span style="letter-spacing:normal"><span style="orphans:auto"><span style="text-transform:none"><span style="white-space:normal"><span style="widows:auto"><span style="word-spacing:0px"><span style="-webkit-text-size-adjust:auto"><span style="text-decoration:none"><font face="Courier New">information should contain explicit notification of each Replica</font></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></div><div class="p1"><span style="font-size:12px"><span style="caret-color:#000000"><span style="color:#000000"><span style="font-family:Helvetica"><span style="font-style:normal"><span style="font-variant-caps:normal"><span style="font-weight:normal"><span style="letter-spacing:normal"><span style="orphans:auto"><span style="text-transform:none"><span style="white-space:normal"><span style="widows:auto"><span style="word-spacing:0px"><span style="-webkit-text-size-adjust:auto"><span style="text-decoration:none"><font face="Courier New">operation mode. </font></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></div><div class="p1"> </div><div class="p1"><span style="font-size:12px"><span style="caret-color:#000000"><span style="color:#000000"><span style="font-family:Helvetica"><span style="font-style:normal"><span style="font-variant-caps:normal"><span style="font-weight:normal"><span style="letter-spacing:normal"><span style="orphans:auto"><span style="text-transform:none"><span style="white-space:normal"><span style="widows:auto"><span style="word-spacing:0px"><span style="-webkit-text-size-adjust:auto"><span style="text-decoration:none"><font face="Courier New">Synchronous replication enabling.</font></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></div><div class="p1"> </div><div class="p1"><span style="font-size:12px"><span style="caret-color:#000000"><span style="color:#000000"><span style="font-family:Helvetica"><span style="font-style:normal"><span style="font-variant-caps:normal"><span style="font-weight:normal"><span style="letter-spacing:normal"><span style="orphans:auto"><span style="text-transform:none"><span style="white-space:normal"><span style="widows:auto"><span style="word-spacing:0px"><span style="-webkit-text-size-adjust:auto"><span style="text-decoration:none"><font face="Courier New">Synchronous operation can be required for a set of spaces in the data</font></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></div><div class="p1"><span style="font-size:12px"><span style="caret-color:#000000"><span style="color:#000000"><span style="font-family:Helvetica"><span style="font-style:normal"><span style="font-variant-caps:normal"><span style="font-weight:normal"><span style="letter-spacing:normal"><span style="orphans:auto"><span style="text-transform:none"><span style="white-space:normal"><span style="widows:auto"><span style="word-spacing:0px"><span style="-webkit-text-size-adjust:auto"><span style="text-decoration:none"><font face="Courier New">scheme. That means only transactions that contain data modification for </font></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></div><div class="p1"><span style="font-size:12px"><span style="caret-color:#000000"><span style="color:#000000"><span style="font-family:Helvetica"><span style="font-style:normal"><span style="font-variant-caps:normal"><span style="font-weight:normal"><span style="letter-spacing:normal"><span style="orphans:auto"><span style="text-transform:none"><span style="white-space:normal"><span style="widows:auto"><span style="word-spacing:0px"><span style="-webkit-text-size-adjust:auto"><span style="text-decoration:none"><font face="Courier New">these </font></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span><span style="font-size:12px"><span style="caret-color:#000000"><span style="color:#000000"><span style="font-family:Helvetica"><span style="font-style:normal"><span style="font-variant-caps:normal"><span style="font-weight:normal"><span style="letter-spacing:normal"><span style="orphans:auto"><span style="text-transform:none"><span style="white-space:normal"><span style="widows:auto"><span style="word-spacing:0px"><span style="-webkit-text-size-adjust:auto"><span style="text-decoration:none"><font face="Courier New">spaces should require quorum. Such transactions named synchronous.</font></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></div><div class="p1"><span style="font-size:12px"><span style="caret-color:#000000"><span style="color:#000000"><span style="font-family:Helvetica"><span style="font-style:normal"><span style="font-variant-caps:normal"><span style="font-weight:normal"><span style="letter-spacing:normal"><span style="orphans:auto"><span style="text-transform:none"><span style="white-space:normal"><span style="widows:auto"><span style="word-spacing:0px"><span style="-webkit-text-size-adjust:auto"><span style="text-decoration:none"><font face="Courier New">As soon as last operation of synchronous transaction appeared in Leader’s</font></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></div><div class="p1"><span style="font-size:12px"><span style="caret-color:#000000"><span style="color:#000000"><span style="font-family:Helvetica"><span style="font-style:normal"><span style="font-variant-caps:normal"><span style="font-weight:normal"><span style="letter-spacing:normal"><span style="orphans:auto"><span style="text-transform:none"><span style="white-space:normal"><span style="widows:auto"><span style="word-spacing:0px"><span style="-webkit-text-size-adjust:auto"><span style="text-decoration:none"><font face="Courier New">WAL, it will cause all following transactions — no matter if they are</font></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></div><div class="p1"><span style="font-size:12px"><span style="caret-color:#000000"><span style="color:#000000"><span style="font-family:Helvetica"><span style="font-style:normal"><span style="font-variant-caps:normal"><span style="font-weight:normal"><span style="letter-spacing:normal"><span style="orphans:auto"><span style="text-transform:none"><span style="white-space:normal"><span style="widows:auto"><span style="word-spacing:0px"><span style="-webkit-text-size-adjust:auto"><span style="text-decoration:none"><font face="Courier New">synchronous or not — wait for the quorum. In case quorum is not achieved</font></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></div><div class="p1"><span style="font-size:12px"><span style="caret-color:#000000"><span style="color:#000000"><span style="font-family:Helvetica"><span style="font-style:normal"><span style="font-variant-caps:normal"><span style="font-weight:normal"><span style="letter-spacing:normal"><span style="orphans:auto"><span style="text-transform:none"><span style="white-space:normal"><span style="widows:auto"><span style="word-spacing:0px"><span style="-webkit-text-size-adjust:auto"><span style="text-decoration:none"><font face="Courier New">the ‘rollback’ operation will cause rollback of all transactions after</font></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></div><div class="p1"><span style="font-size:12px"><span style="caret-color:#000000"><span style="color:#000000"><span style="font-family:Helvetica"><span style="font-style:normal"><span style="font-variant-caps:normal"><span style="font-weight:normal"><span style="letter-spacing:normal"><span style="orphans:auto"><span style="text-transform:none"><span style="white-space:normal"><span style="widows:auto"><span style="word-spacing:0px"><span style="-webkit-text-size-adjust:auto"><span style="text-decoration:none"><font face="Courier New">the synchronous one. It will ensure the consistent state of the data both</font></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></div><div class="p1"><span style="font-size:12px"><span style="caret-color:#000000"><span style="color:#000000"><span style="font-family:Helvetica"><span style="font-style:normal"><span style="font-variant-caps:normal"><span style="font-weight:normal"><span style="letter-spacing:normal"><span style="orphans:auto"><span style="text-transform:none"><span style="white-space:normal"><span style="widows:auto"><span style="word-spacing:0px"><span style="-webkit-text-size-adjust:auto"><span style="text-decoration:none"><font face="Courier New">on Leader and Replicas. </font></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span><span style="font-size:12px"><span style="caret-color:#000000"><span style="color:#000000"><span style="font-family:Helvetica"><span style="font-style:normal"><span style="font-variant-caps:normal"><span style="font-weight:normal"><span style="letter-spacing:normal"><span style="orphans:auto"><span style="text-transform:none"><span style="white-space:normal"><span style="widows:auto"><span style="word-spacing:0px"><span style="-webkit-text-size-adjust:auto"><span style="text-decoration:none"><font face="Courier New">In case user doesn’t require synchronous operation</font></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></div><div class="p1"><span style="font-size:12px"><span style="caret-color:#000000"><span style="color:#000000"><span style="font-family:Helvetica"><span style="font-style:normal"><span style="font-variant-caps:normal"><span style="font-weight:normal"><span style="letter-spacing:normal"><span style="orphans:auto"><span style="text-transform:none"><span style="white-space:normal"><span style="widows:auto"><span style="word-spacing:0px"><span style="-webkit-text-size-adjust:auto"><span style="text-decoration:none"><font face="Courier New">for any space then </font></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span><span style="font-size:12px"><span style="caret-color:#000000"><span style="color:#000000"><span style="font-family:Helvetica"><span style="font-style:normal"><span style="font-variant-caps:normal"><span style="font-weight:normal"><span style="letter-spacing:normal"><span style="orphans:auto"><span style="text-transform:none"><span style="white-space:normal"><span style="widows:auto"><span style="word-spacing:0px"><span style="-webkit-text-size-adjust:auto"><span style="text-decoration:none"><font face="Courier New">no changes to the WAL generation and replication will</font></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></div><div class="p1"><span style="font-size:12px"><span style="caret-color:#000000"><span style="color:#000000"><span style="font-family:Helvetica"><span style="font-style:normal"><span style="font-variant-caps:normal"><span style="font-weight:normal"><span style="letter-spacing:normal"><span style="orphans:auto"><span style="text-transform:none"><span style="white-space:normal"><span style="widows:auto"><span style="word-spacing:0px"><span style="-webkit-text-size-adjust:auto"><span style="text-decoration:none"><font face="Courier New">appear.</font></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></div><div class="p1"> </div><div class="p1"><span style="font-size:12px"><span style="caret-color:#000000"><span style="color:#000000"><span style="font-family:Helvetica"><span style="font-style:normal"><span style="font-variant-caps:normal"><span style="font-weight:normal"><span style="letter-spacing:normal"><span style="orphans:auto"><span style="text-transform:none"><span style="white-space:normal"><span style="widows:auto"><span style="word-spacing:0px"><span style="-webkit-text-size-adjust:auto"><span style="text-decoration:none"><font face="Courier New">Cluster description should contain explicit attribute for each Replica</font></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></div><div class="p1"><span style="font-size:12px"><span style="caret-color:#000000"><span style="color:#000000"><span style="font-family:Helvetica"><span style="font-style:normal"><span style="font-variant-caps:normal"><span style="font-weight:normal"><span style="letter-spacing:normal"><span style="orphans:auto"><span style="text-transform:none"><span style="white-space:normal"><span style="widows:auto"><span style="word-spacing:0px"><span style="-webkit-text-size-adjust:auto"><span style="text-decoration:none"><font face="Courier New">to denote it participates in synchronous activities. Also the description</font></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></div><div class="p1"><span style="font-size:12px"><span style="caret-color:#000000"><span style="color:#000000"><span style="font-family:Helvetica"><span style="font-style:normal"><span style="font-variant-caps:normal"><span style="font-weight:normal"><span style="letter-spacing:normal"><span style="orphans:auto"><span style="text-transform:none"><span style="white-space:normal"><span style="widows:auto"><span style="word-spacing:0px"><span style="-webkit-text-size-adjust:auto"><span style="text-decoration:none"><font face="Courier New">should contain criterion on how many Replicas responses are </font></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span><span style="font-size:12px"><span style="caret-color:#000000"><span style="color:#000000"><span style="font-family:Helvetica"><span style="font-style:normal"><span style="font-variant-caps:normal"><span style="font-weight:normal"><span style="letter-spacing:normal"><span style="orphans:auto"><span style="text-transform:none"><span style="white-space:normal"><span style="widows:auto"><span style="word-spacing:0px"><span style="-webkit-text-size-adjust:auto"><span style="text-decoration:none"><font face="Courier New">needed to </font></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></div><div class="p1"><span style="font-size:12px"><span style="caret-color:#000000"><span style="color:#000000"><span style="font-family:Helvetica"><span style="font-style:normal"><span style="font-variant-caps:normal"><span style="font-weight:normal"><span style="letter-spacing:normal"><span style="orphans:auto"><span style="text-transform:none"><span style="white-space:normal"><span style="widows:auto"><span style="word-spacing:0px"><span style="-webkit-text-size-adjust:auto"><span style="text-decoration:none"><font face="Courier New">achieve the quorum. </font></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></div><div class="p1"> </div><div class="p1"> </div><div class="p1"><span style="font-size:12px"><span style="caret-color:#000000"><span style="color:#000000"><span style="font-family:Helvetica"><span style="font-style:normal"><span style="font-variant-caps:normal"><span style="font-weight:normal"><span style="letter-spacing:normal"><span style="orphans:auto"><span style="text-transform:none"><span style="white-space:normal"><span style="widows:auto"><span style="word-spacing:0px"><span style="-webkit-text-size-adjust:auto"><span style="text-decoration:none"><font face="Courier New"> </font></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span><span style="font-size:12px"><span style="caret-color:#000000"><span style="color:#000000"><span style="font-family:Helvetica"><span style="font-style:normal"><span style="font-variant-caps:normal"><span style="font-weight:normal"><span style="letter-spacing:normal"><span style="orphans:auto"><span style="text-transform:none"><span style="white-space:normal"><span style="widows:auto"><span style="word-spacing:0px"><span style="-webkit-text-size-adjust:auto"><span style="text-decoration:none"><font face="Courier New"> </font></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></div></div></div><div> </div><div data-signature-widget="container"><div data-signature-widget="content"><div> </div></div></div></BODY></HTML>