Multi-Frame Quality Enhancement for Compressed Video Ren Yang, Mai Xu,∗ Zulin Wang and Tianyi Li School of Electronic and Information Engineering, Beihang University, Beijing, China fyangren, maixu, wzulin,
[email protected] HEVC Our MFQE approach PQF Abstract PSNR (dB) PSNR 81 85 90 95 100 105 Frames The past few years have witnessed great success in ap- Compressed frame 93 Compressed frame 96 Compressed frame 97 plying deep learning to enhance the quality of compressed image/video. The existing approaches mainly focus on en- hancing the quality of a single frame, ignoring the similar- ity between consecutive frames. In this paper, we investi- gate that heavy quality fluctuation exists across compressed MF-CNN of our MFQE approach video frames, and thus low quality frames can be enhanced DS-CNN [42] using the neighboring high quality frames, seen as Multi- Frame Quality Enhancement (MFQE). Accordingly, this pa- per proposes an MFQE approach for compressed video, as a first attempt in this direction. In our approach, we Enhanced frame 96 Enhanced frame 96 Raw frame 96 firstly develop a Support Vector Machine (SVM) based de- Figure 1. Example for quality fluctuation (top) and quality enhancement tector to locate Peak Quality Frames (PQFs) in compressed performance (bottom). video. Then, a novel Multi-Frame Convolutional Neural Recently, there has been increasing interest in enhanc- Network (MF-CNN) is designed to enhance the quality of ing the visual quality of compressed image/video [27, 10, compressed video, in which the non-PQF and its nearest 18, 36, 6, 9, 38, 12, 24, 24, 42].