Scientia Iranica B (2015) 22(6), 2134{2141
Sharif University of Technology Scientia Iranica Transactions B: Mechanical Engineering www.scientiairanica.com
Anti-shake system for digital dynamic images
S.J. Chang and W.H. Cai
Department of Mechanical Engineering, National Yunlin University of Science and Technology, 123 University Road, Section 3, Douliou, Yunlin 64002, Taiwan, R.O.C.
Received 22 May 2014; received in revised form 16 April 2015; accepted 22 June 2015
KEYWORDS Abstract. This paper presents an anti-shake system to improve the quality of digital dynamic image sequences. The achievement of this study is to develop a digital dynamic Anti-shake; image stabilization system using basic image processes and a signal-smoothing algorithm. Dynamic image; The dynamic image stabilization method includes three steps: motion vector detection, Tremor; motion vector smoothing, and image sequence reconstruction. The rst step is to estimate Stabilization. the motion vector of dynamic image sequences using the three-step method. The second step is to separate involuntary motion vectors from motion vectors by the Fourier linear combiner algorithm. The last step is to compensate the unstable dynamic image sequences according to the involuntary motion vector and to build stable dynamic image sequence. The performance of the smoothing algorithm is also discussed in this paper. The Smooth Index (SI) is used to evaluate the performance of the anti-shake system presented in this paper. From the results of the experiments under some di erent conditions, the SI of the digital dynamic image sequences can reach 2.224 through the anti-shake system presented in this paper. © 2015 Sharif University of Technology. All rights reserved.
1. Introduction automatically be extended to increase exposure, and hand tremors will cause the photos to blur more In past years, tiny sized digital cameras were developed easily if the shutter release time is too long. Thus, vigorously. Digital cameras have become a basic people always get blurry pictures because their hands element in most portable devices, such as mobile shake involuntarily during the time interval of shutter phones and tablet computers. People take pictures release. Figure 1 shows a clear picture taken in an or dynamic videos and share them immediately on environment with sucient light and a blurry picture Facebook or Twitter in their daily life. Usually, taken in an environment without sucient light. The people use portable devices to take pictures and shoot hand tremor problem is even more serious when people videos single handedly since the size of portable devices shoot a dynamic video. The problem is getting tends towards miniaturization. Hand tremors always worse while the resolution of image sensors and the occur when people take pictures and shoot videos, magni ed optical zooming of the camera are getting which causes their low quality. Therefore, shooting better. It also costs a lot to apply optical anti- a stable video is a basic request when people use a shake technologies in thin portable devices because built-in cameras of portable device [1,2]. Moreover, it is usually dicult to miniaturize the volume of when people use the camera in a dim environment optical anti-shake modules. Thus, developing an without sucient light, the shutter release time will anti-shake algorithm for thin portable products to enhance video quality is very important. In this paper, *. Corresponding author. Tel.: 886-5-5342601 Ext. 4119; some image processing technologies will be applied Fax: 886-5-5342062 to estimate the motion vectors of dynamic image E-mail address: [email protected] (S.J. Chang) sequences. Then, the involuntary motion signals will S.J. Chang and W.H. Cai/Scientia Iranica, Transactions B: Mechanical Engineering 22 (2015) 2134{2141 2135
Figure 1. The pictures taken in the environments: (a) With sucient light; and (b) without sucient light. be separated from the motion vector by the signal separation algorithm. Finally, the dynamic image will be reconstructed by compensating the involuntary shaking of each frame to enhance the quality of the dynamic image. Figure 3. Search mode of motion estimation process.
2. Image stabilization system 2.1. Motion vector detection The search mode of motion vector estimation is shown The anti-shake system, also known as the image in Figure 3. Every frame will be compared with the stabilization system, is a desirable feature for digital previous frame to estimate motion between these two cameras and portable video cameras to obtain stable frames. For example, a M N image in the center pictures and videos. Most previous approaches focused region of the previous frame in the image is considered on using either motion vector estimation [3-6] or frame as a reference image. Di erent regions in the current position smoothing [7-15]. Some methods will cause frame are compared with the reference image, and the time delay or phase change to the system. In this most similar part in the current frame to the reference paper, an anti-shake algorithm was proposed to reduce image will be found after comparison. The motion the hand tremor e ect using the Fourier series, which vector between the current and previous frame can be is based on a dynamic sinusoidal model. The system obtained according to the most similar region in the consists of three di erent stages: motion vector esti- current frame. In the search process, the Direction of mation, motion vector separation (extract the tremor Minimum Distortion (DMD) is used to quantify the signal in the motion vector), and compensation of similarity between the current frame and the previous image sequences to reduce the hand tremor e ect. A frame. The DMD, de ned according to the sum of block diagram of the anti-shake system presented in absolute di erence (SAD), is shown as Eq. (1): this paper is shown in Figure 2. Xm Xn DMD[I(t+1);I(t)]= jIx;y(t+1) Ix;y(t)j : x=1 y=1 (1)
I(t) is the tth frame of the video, x and y are the coordinates of the pixel of the image, and m and n are the sizes of the image [16]. In every comparing step, nine DMD values will be obtained by the comparing processes of nine di erent regions. The minimum DMD value region is considered as the most similar image to the center region in the previous frame. Thus, the motion vector between the current and previous frame will be obtained. In Figure 3, the dashed and solid squares repre- sent the compared images and the referenced images, respectively. In the comparing process, the nine compared images are de ned by pixels of the o set in the x- and y-directions from the center region in the current frame. The center region in the current Figure 2. Block diagram of the anti-shake system. frame is also one of compared images. In every step 2136 S.J. Chang and W.H. Cai/Scientia Iranica, Transactions B: Mechanical Engineering 22 (2015) 2134{2141
Figure 5. The block diagram of the Fourier linear combiner.
to separate v or t from the equation. The majority of smoothing algorithms are devoted to voluntary and involuntary motion separation using the Fourier linear combiner. It is very similar to the standard adaptive Figure 4. Example of the three step searching method. lter. This algorithm uses a group of sine and cosine functions to estimate voluntary motion and involuntary of the comparing process, each compared image should tremor. The assumption is that a hand tremor can be be compared with the referenced image, as shown in described as a linear combination of sine and cosine Figure 3. Figure 4 shows an example of the three functions at di erent frequencies. The block diagram step searching method [17]. In the rst searching step, of the Fourier linear combiner is shown in Figure 5. the o set is set as 4 pixels. If the 5th region was the The input of the system, the linear combination matched region with minimum DMD value in the rst of sine and cosine functions, is independent of the input searching step, then the 5th region would be set as signal, S . The assumption is that voluntary hand the new center region of the compared image of the k motion, v , is independent of involuntary hand tremor, second searching step. In the second searching step, k t . The weights, w , are the Fourier coecients. The the o set is set as 2 pixels. If the 4th region was k k x are listed as Eq. (3): the matched region with minimum DMD value in the k second searching step, then the 4th region would be 8