Object Tracking-based Foveated Super-Resolution Convolutional Neural Network for Head Mounted Display

초록

Recently, the immersive virtual reality (VR) environment using the head mounted display (HMD) has attracted attention as a new growth market due to the reasonable consumer price and high accessibility compared to other VR devices. However, users feel the cognitive heterogeneity caused by low resolution images, and hence, it is difficult to use it for a long time. To solve it, transmission techniques based on image resolution conversion have studied. In this paper, we propose a novel foveated super-resolution convolutional neural network (SRCNN) for HMD using an object tracking algorithm to reduce computation load for rendering high resolution images. We implement the object tracking on the region to compensate for a frame processing speed of eye-tracking devices, relatively slow to apply the resolution conversion. SRCNN applies to cognitive regions, and typical interpolation applies to other regions to reduce the rendering cost. As a result, the computation is decreased by 90.4059%, and PSNR is higher than the conventional foveated rendering algorithm.

키워드