Paper 6

Optimizing Inter-Data-Center Large-scale Database Parallel Replication with Workload-driven Partitioning

Authors: Zhen Gao, Hong Min, Xiao Li, Jie Huang, Yi Jin , An Lei, Serge Bourbonnais, Miao Zheng, Gene Fuh

Volume 24 (2016)

Abstract

Geographically distributed data centers are deployed for non-stop business opera-tions by many enterprises. In case of disastrous events, ongoing workloads must be failed over from the current data center to another active one within just a few seconds to achieve continuous service availability. Software-based parallel data-base replication techniques are designed to meet very high throughput with near-real-time latency. Understanding workload characteristics is one of the key factors for improving replication performance. In this paper, we propose a workload-driven method to optimize database replication latency and minimize transaction splits with a minimum of parallel replication consistency groups. Our two-phased approach includes 1) a log-based mechanism for workload pattern discovery; 2) a history-based algorithm on pattern analysis, database partitioning and partition ad-justment. The experimental results from a real banking batch workload and a benchmark OLTP workload demonstrate the effectiveness of the solution even for partitioning 1000s of database tables in very large workloads. Finally, the algo-rithm to automate the cyclic flow of workload profile capturing and partitioning readjustment is developed and verified.