심층신경망으로 가는 통계 여행, 네 번째 여행: 확률적경사하강법 탐험하기

A statistical journey to DNN, the fourth trip: exploring stochastic gradient descent methods
  • Jang Kisuk; 
  • Han Jungmin; 
  • Lim Seung Min; 
  • Park Min; 
  • Lee Yoon Dong
Citations

WEB OF SCIENCE

0

초록

Stochastic Gradient Descent (SGD) has been fundamental to the success of deep neural networks. This paper provides a comprehensive review of SGD and its extensions. We begin by surveying a wide range of function optimization techniques from numerical analysis and statistics to reveal how modern SGD methods relate to traditional optimization methods. We then explore the characteristics of various SGD-based algorithms to present a systematic comparison of the major SGD optimizers used in deep learning within a unified theoretical framework.

키워드

심층신경망; 최적화; 확률적경사하강법; RMSProp; AdamW; deep neural network; optimization; stochastic gradient descent; RMSProp; AdamW
제목
심층신경망으로 가는 통계 여행, 네 번째 여행: 확률적경사하강법 탐험하기
제목 (타언어)
A statistical journey to DNN, the fourth trip: exploring stochastic gradient descent methods
저자
Jang Kisuk; Han Jungmin; Lim Seung Min; Park Min; Lee Yoon Dong
DOI
10.5351/KJAS.2025.38.6.721
발행일
2025-12
유형
Article
저널명
응용통계연구
권
38
호
6
페이지
721 ~ 737