Research Team Led by Professor Yi Young-min of the Department of Artificial Intelligence, Has Paper Accepted at ACM ASPLOS 2026, the Most Prestigious International Conference
A paper co-authored by Song Geun-soo, a master’s student in the Department of Artificial Intelligence (first author), and Professor Yi Young-min (corresponding author) has been accepted at the ACM International Conference on Architectural Support for Programming Languages and Operating Systems (ASPLOS) 2026, the most prestigious international conference in the field of system software and architecture. ASPLOS is recognized by the Korean Institute of Information Scientists and Engineers as a top-tier conference (BK-recognized IF=4) and is scheduled to be held in Pittsburgh, USA, from March 22 to 26, 2026.
Titled “oFFN: Outlier and Neuron-aware Structured FFN for Fast yet Accurate LLM Inference,” this research identifies the characteristic that outliers tend to cluster at specific positions in activation vectors and proposes oFFN, an LLM inference acceleration technique based on this observation. oFFN statically rearranges weights in the FFN (Feed-Forward Network) layer by simultaneously considering the frequency of output neuron activations and the frequency of outlier occurrences. By efficiently leveraging activation sparsity, it achieves high inference acceleration without sacrificing accuracy.
Inference acceleration using activation sparsity is a highly effective approach as it can resolve both computational and memory bottlenecks; however, accurately predicting which outputs are sparse remains a challenging problem. Additionally, there is a limitation where the acceleration effect diminishes as the batch size increases, leading to reduced structural sparsity. This study simultaneously mitigates these two issues by clustering outlier dimensions through FFN weight rearrangement and efficiently clustering neurons with similar sparsity. As a result, it achieved an acceleration of up to 5.46x for the FFN and up to 2.01x (theoretical upper bound: 2.18x) for total inference time, with virtually no loss in accuracy, and demonstrated a 13% improvement in inference speed compared to the existing SOTA.
Professor Yi Young-min stated, “While the use of activation sparsity in LLM inference acceleration is highly promising, there has been a limitation in that output activation sparsity is difficult to utilize accurately and efficiently. oFFN overcomes these existing limitations by rearranging FFN weights based on an intriguing analysis of LLM inference characteristics and by structurally clustering outlier dimensions and neurons. Furthermore, this study is significant in that it leverages GPU tensor cores and CUDA cores in a complementary manner to realize activation sparsity as practical inference acceleration not only in single-batch but also in multi-batch inference. We plan to continue advancing related follow-up research in the future.”
ASPLOS Conference: https://www.asplos-conference.org/
High-Performance AI Systems Lab: https://aisys.sogang.ac.kr
[SEO 키워드]
ACM ASPLOS 2026, LLM Inference Acceleration, Activation Sparsity
[Summary]
The research team secured a paper acceptance at ACM ASPLOS 2026 for developing oFFN, a novel framework that achieves significant LLM Inference Acceleration by leveraging Activation Sparsity and outlier-aware weight rearrangement.
Titled “oFFN: Outlier and Neuron-aware Structured FFN for Fast yet Accurate LLM Inference,” this research identifies the characteristic that outliers tend to cluster at specific positions in activation vectors and proposes oFFN, an LLM inference acceleration technique based on this observation. oFFN statically rearranges weights in the FFN (Feed-Forward Network) layer by simultaneously considering the frequency of output neuron activations and the frequency of outlier occurrences. By efficiently leveraging activation sparsity, it achieves high inference acceleration without sacrificing accuracy.
Inference acceleration using activation sparsity is a highly effective approach as it can resolve both computational and memory bottlenecks; however, accurately predicting which outputs are sparse remains a challenging problem. Additionally, there is a limitation where the acceleration effect diminishes as the batch size increases, leading to reduced structural sparsity. This study simultaneously mitigates these two issues by clustering outlier dimensions through FFN weight rearrangement and efficiently clustering neurons with similar sparsity. As a result, it achieved an acceleration of up to 5.46x for the FFN and up to 2.01x (theoretical upper bound: 2.18x) for total inference time, with virtually no loss in accuracy, and demonstrated a 13% improvement in inference speed compared to the existing SOTA.
Professor Yi Young-min stated, “While the use of activation sparsity in LLM inference acceleration is highly promising, there has been a limitation in that output activation sparsity is difficult to utilize accurately and efficiently. oFFN overcomes these existing limitations by rearranging FFN weights based on an intriguing analysis of LLM inference characteristics and by structurally clustering outlier dimensions and neurons. Furthermore, this study is significant in that it leverages GPU tensor cores and CUDA cores in a complementary manner to realize activation sparsity as practical inference acceleration not only in single-batch but also in multi-batch inference. We plan to continue advancing related follow-up research in the future.”
ASPLOS Conference: https://www.asplos-conference.org/
High-Performance AI Systems Lab: https://aisys.sogang.ac.kr
[SEO 키워드]
ACM ASPLOS 2026, LLM Inference Acceleration, Activation Sparsity
[Summary]
The research team secured a paper acceptance at ACM ASPLOS 2026 for developing oFFN, a novel framework that achieves significant LLM Inference Acceleration by leveraging Activation Sparsity and outlier-aware weight rearrangement.