Research Team Led by Professor Kim Young-jae of the Department of Computer Science and Engineering Has Paper Accepted for Regular Presentation at SC 26, a Top Conference in the HPC Field
A research team led by Professor Kim Young-jae of the Department of Computer Science and Engineering at this university (Ph.D. candidate Hwang Soon, M.S. Park Jun-hyuk, Ph.D. candidate Yoo Jeong-hyun, and M.S. Ahn Seong-hoon) has developed a computable object storage system capable of vertically partitioned query execution. This research proposes a hierarchical query execution technique that reduces data movement by up to 99.998% by utilizing computational resources distributed across multiple layers within the storage system.
The paper titled “OASIS: Hierarchical Query Execution over Multi-Layer Computation-Enabled Object Storage,” conducted in collaboration with SK hynix Memory Systems Research Lab and the Los Alamos National Laboratory in the U.S., was accepted as a full paper at SC 26 (The 38th International Conference for High Performance Computing, Networking, Storage, and Analysis), the world’s most prestigious conference in the field of high-performance computing. This year’s paper acceptance rate was 19.4%.
In large-scale scientific data analysis, the process of transferring tens of gigabytes of data from storage to analysis servers becomes a performance bottleneck in itself. To mitigate this, technologies have been used that perform simple operations, such as filtering, on the data before the storage system sends it out; however, this approach only reduces the amount of data leaving the storage system, while leaving intact the movement of data within the storage system itself, from the disk enclosures to the storage servers. The research team identified the core issue not as determining which operations should be performed in storage, but as deciding at which layer within the storage hierarchy query execution should be split. To address this placement problem, the team developed a system called OASIS. OASIS’s core algorithm, SODA, predicts the data reduction ratio of each operation using only statistics collected when the data is stored, without reading the actual data. Based on these predictions, it automatically identifies the optimal query split point that minimizes data movement between storage layers. In evaluations using real-world scientific workloads, OASIS reduced internal data movement from 20 GB to 76 KB and shortened end-to-end latency by up to 25.1% compared with existing state-of-the-art systems.
This research holds academic significance in that it redefines the existing dichotomous view of storage-side computation as a hierarchical placement optimization problem. In particular, by quantitatively demonstrating that offloading is not always beneficial and can even be detrimental under certain conditions, it provides design guidelines for future related research. From an industry perspective, the approach is practical for immediate application to large-scale data analysis infrastructure in data centers, as it can be implemented using only the resources of existing storage equipment without requiring separate dedicated hardware, and it maintains the standard S3 interface, ensuring full compatibility with existing analytics engines. Furthermore, reduced data movement directly translates to energy savings, and the approach is expected to contribute to the sustainability of AI and data infrastructure, where power consumption is rapidly increasing.
Hwang Soon, the study's first author and a Ph.D. candidate, said, "While the potential benefits of storage-side computation have long been recognized, storage systems have continued to underutilize their internal computational resources. Our team reframed the problem as one of determining where within the storage hierarchy query execution should be split. This approach enabled us to fully leverage computational resources at each layer, from storage enclosures to storage servers, and ultimately eliminate the previously overlooked bottleneck of internal data movement." He added, "I would like to express my sincere gratitude to Professor Kim Young-jae for his guidance, as well as to our collaborators at SK hynix and Los Alamos National Laboratory for their invaluable contributions to this research."
The International Conference for High Performance Computing, Networking, Storage, and Analysis (SC) is the world’s most prestigious international academic conference in the field of high-performance computing (HPC). Since its founding in 1988, it has brought together researchers, system designers, and industry experts to share the latest advancements in computing, storage, and data analysis. Recognized as a premier conference with a low acceptance rate due to its rigorous annual peer review process, the conference is scheduled to take place from November 15 to 20 at McCormick Place in Chicago, USA, and the accepted full research papers will be presented as part of the Technical Program.
[SEO Keyword]
SC 26 Computable Object Storage, Hierarchical Query Execution, OASIS Storage System
The paper titled “OASIS: Hierarchical Query Execution over Multi-Layer Computation-Enabled Object Storage,” conducted in collaboration with SK hynix Memory Systems Research Lab and the Los Alamos National Laboratory in the U.S., was accepted as a full paper at SC 26 (The 38th International Conference for High Performance Computing, Networking, Storage, and Analysis), the world’s most prestigious conference in the field of high-performance computing. This year’s paper acceptance rate was 19.4%.
In large-scale scientific data analysis, the process of transferring tens of gigabytes of data from storage to analysis servers becomes a performance bottleneck in itself. To mitigate this, technologies have been used that perform simple operations, such as filtering, on the data before the storage system sends it out; however, this approach only reduces the amount of data leaving the storage system, while leaving intact the movement of data within the storage system itself, from the disk enclosures to the storage servers. The research team identified the core issue not as determining which operations should be performed in storage, but as deciding at which layer within the storage hierarchy query execution should be split. To address this placement problem, the team developed a system called OASIS. OASIS’s core algorithm, SODA, predicts the data reduction ratio of each operation using only statistics collected when the data is stored, without reading the actual data. Based on these predictions, it automatically identifies the optimal query split point that minimizes data movement between storage layers. In evaluations using real-world scientific workloads, OASIS reduced internal data movement from 20 GB to 76 KB and shortened end-to-end latency by up to 25.1% compared with existing state-of-the-art systems.
This research holds academic significance in that it redefines the existing dichotomous view of storage-side computation as a hierarchical placement optimization problem. In particular, by quantitatively demonstrating that offloading is not always beneficial and can even be detrimental under certain conditions, it provides design guidelines for future related research. From an industry perspective, the approach is practical for immediate application to large-scale data analysis infrastructure in data centers, as it can be implemented using only the resources of existing storage equipment without requiring separate dedicated hardware, and it maintains the standard S3 interface, ensuring full compatibility with existing analytics engines. Furthermore, reduced data movement directly translates to energy savings, and the approach is expected to contribute to the sustainability of AI and data infrastructure, where power consumption is rapidly increasing.
Hwang Soon, the study's first author and a Ph.D. candidate, said, "While the potential benefits of storage-side computation have long been recognized, storage systems have continued to underutilize their internal computational resources. Our team reframed the problem as one of determining where within the storage hierarchy query execution should be split. This approach enabled us to fully leverage computational resources at each layer, from storage enclosures to storage servers, and ultimately eliminate the previously overlooked bottleneck of internal data movement." He added, "I would like to express my sincere gratitude to Professor Kim Young-jae for his guidance, as well as to our collaborators at SK hynix and Los Alamos National Laboratory for their invaluable contributions to this research."
The International Conference for High Performance Computing, Networking, Storage, and Analysis (SC) is the world’s most prestigious international academic conference in the field of high-performance computing (HPC). Since its founding in 1988, it has brought together researchers, system designers, and industry experts to share the latest advancements in computing, storage, and data analysis. Recognized as a premier conference with a low acceptance rate due to its rigorous annual peer review process, the conference is scheduled to take place from November 15 to 20 at McCormick Place in Chicago, USA, and the accepted full research papers will be presented as part of the Technical Program.
[SEO Keyword]
SC 26 Computable Object Storage, Hierarchical Query Execution, OASIS Storage System