Early Detection of Online Grooming with Language Models

초록

This study aimed to develop a language model for the early detection of grooming in Korean. Based on PAN12 Korean dataset, we conducted early detection experiments using BERT-based models and Large Language Models (LLMs). In place of the window method, which references consecutive previous sentences, we introduced a memory method that references previous sentences similar to the input sentence and confirmed that the memory method outperforms the window method. We used reference sizes of 3, 5, and 10 sentences for each conversation. As the number of previous sentences increased, the memory method showed improved performance. We evaluated performance using F1 score, accuracy, speed, and latency-weighted F1. To address the limitations of latency-weighted F1, we introduced a new metric, Human-Model-Ratio (HMR).

키워드