kirancodes.me
To Proof Maintenance & Beyond!
ASE 2023

RocketHA: A High Availability Design Paradigm for Distributed Log-Based Storage System

Juntao Ji, Rongtong Jin, Yubao Fu, Yinyou Gu, Tsung-Han Tsai, Qingshan Lin

DOI 10.1109/ase56229.2023.00123dblp

Abstract

As a team from Alibaba Cloud, we have developed and open-sourced RocketMQ, a cloud-native “messaging, eventing, streaming” real-time data processing platform that covers cloud-edge-device collaboration scenarios. During the development of RocketMQ, we also formulated RocketHA, a log-based storage high availability design theory that provides a robust solution for distributed log storage software used in industrial applications. RocketHA comprises six fundamental components that enable automatic cluster recovery from failures such as crashes and partitions. This design paradigm has been successfully implemented in the open-source RocketMQ. Our evaluation demonstrates that RocketHA ensures high availability, fast recovery, high throughput, and data loss prevention. We hope that RocketHA will inspire and guide the development of high-availability solutions for all log-based storage systems.

BibTeX
@inproceedings{Ji-al:ASE23,
  author    = {Juntao Ji and
               Rongtong Jin and
               Yubao Fu and
               Yinyou Gu and
               Tsung{-}Han Tsai and
               Qingshan Lin},
  title     = {{RocketHA:} A High Availability Design Paradigm for Distributed {Log-Based} Storage System},
  booktitle = {ASE},
  pages     = {1819--1824},
  publisher = {{IEEE}},
  year      = {2023},
}

Related papers