Coordinating Compaction between LSM-tree based Key-Value Stores for Edge Federation

초록

Edge computing environments increasingly demand real-time data processing, leading to the adoption of logstructured merge-tree based key-value stores (LSM-KVS) for efficient data handling. LSM-KVS periodically runs compaction operations in the background to manage the database. However compaction delays cause write stalls, which lead to degraded throughput of LSM-KVS and system performance on resourcelimited edge servers. An edge federation environment, which shares resources and tasks between edge servers, can alleviate the resource limitations. Such environments can leverage compaction offloading where another server performs CPU-intensive compaction operations instead. But coordinating compaction offloading is an important challenge, as the performance of the server performing the compaction can be degraded. In this paper, we propose EDGEPILOT. EDGEPILOT is scheduling mechanism of compaction offloading that is designed for LSM-KVS within edge federation. EDGEPILOT schedules where to reallocate compactions among the edge servers. This is achieved by considering the resource and computing power of each server. As a result, the overall resource efficiency and compaction throughput are increased. In addition, EDGEPILOT provides EDGECODE to determine the effectiveness of compaction offloading. EDGECODE is a mathematical modeling based on compaction processing data to approximate inter-server compaction processing times. EDGEPILOT is implemented on the prominent LSM-KVS, RocksDB v8.3.2, and demonstrates notable improvements compared to the conventional RocksDB. The overall write stall duration of the system is reduced by up to 71%, and throughput is increased by 17%.

키워드