Special Conference Edition , November, 2018 http://dx.doi.org/10.4314/bajopas.v11i1.1S

Bayero Journal of Pure and Applied Sciences , 11(1): 1 - 7 ISSN 2006 – 6996

SIMULATION OF THE EFFECT OF BIT ASSIGNMENT TO MULTI- HYPOTHESIS PICTURES PER GROUP FOR MOTION COMPE NSATED COMPRESSION (MCVC)

Busari, 1K.A., Adamu, 2I.D ., Ali, 3M.H., Busari, 4T. A., Afolabi, 5M.A. and Kashimbila, 6M.M. 1Department of Physics, Federal University, Gashua, Yobe State, Nigeria 2,3,6 Department of Physics, Bayero University, Kano, Nigeria 4Nigerian Tulip International College, Kano, Nigeria 5The light College, Kano, Nigeria Correspondence author:[email protected] , 08147929160

ABSTRACT This paper describes the Lagrange based formulation of bit -assignment for multi -hypothesis frames over number of pictures in a group for motion compensated video compression. Theoretical insight on bit assignment is accomplished by applying Lagrange multip lier and its cost function for a real-time computational analysis. Video signals are compressed at optimal bit allocation for qualitative reconstruction at the decoder (receiver) depending upon the number of hypothesis used in coding the signal. This has m ade multi -hypothesis (MHMC) a great process in the prediction of the actual signal through the combination of more than one motion compensated prediction (MCP). Optimal Bits were allocated to multi-hypothesis frames using MATLAB to opti mize the number of hypothesis frames per group of pictures (GOP) decoded for an improved signal compression. Simulation results show that an optimal bit assignment of the range 0.089 bpp to 0.159 bpp was realized with eight numbers of hypotheses frames per (GOP). This implies that varying number of hypotheses actualizes optimal bit assignment for motion compensation. This shows a qualitative signal compression performance analysis for high technology motion pictures in communication services and storage pro cesses. It also addresses a better prediction due to greater time correlation between hypotheses and helps in the improvement of coding and reconstruction quality of video signals. Key words: Bit-rate, frame, Group of Pictures (GOP), Motion Compensation , multi-hypothesis, Video Compression.

INTRODUCTION motion vector (UMV) mode, rate-constrained In recent years, video compression algorithms mode decision, entropy coding model have improved tremendously with applications adjustment (CABAC), advanced prediction mode in multimedia such as digital storage and (AP), multi-hypothesis motion estimation issues communication, for video games and internet and other negotiable options like error network systems etc. Multi-hypothesis motion robustness, Scalability Mode etc (Flierl and compensation (MHMC) has been greatly used in Girod, 2003). the prediction of the current frame from This made MHMC preferable for low bit rate previous frames with the application of motion applications. A n optimal hypothesis selection vectors and quantization error in coded bit and predictor coe fficient selection algorithm stream for a qualitative real -time video was proposed (Flierl et al., 1998). A rate- compression analysis. Therefore, bit assignment constrained mode decision algorithm was also (bit allocation) for multi-hypothesis frames is proposed in Flierl et al. (2002) . The collective important for video data storage and network utilization of MHMC (2hypotheses), variable bandwidth. Additionally, video compression block-size coding, and long -term memory (10 issues is addressed w ith improved coding reference frames) proved that 30% bit-rate for standard like H.263 and H.264, which utilizes the same video quality is achieved thr ough MHMC with the following codec characteristics: computational simulation (Flierl et al., 2002). 3D variable length coding (VLC) table, syntax Rate- distortion optimization (RDO) has been based arithmetic coding (SAC) with about 5% used to overcome the problem of mi nimal less bits at the same signal to noise ratio (SNR) distortion at high rate to improve the – Inter frames is approximately 3 to 4% while performance of codec in video compression Intra frames is approximately 10% , unrestricted system (Li et al., 2009). 1

Special Conference Edition, November, 2018 Outstandingly, mathematical models are now distortion model has also been addressed (Lin developed to characterize the error and Wang, 2002). Optimal bit allocation for propagation effect caused by packet losses in picture ordering has been presented in Sam MHMC coder (Kung et al., 2004a, 2004b). Error (2012). Lastly, bit allocation for MHMC has also resilience property of the 2HMCP, which been presented in closed form expression in presented a decoder distortion model (given Sam (2015). channel conditions) and a simplified encoder

BACKGROUND THEORY

Figure 1: Group of picture (GOP) structure

As shown in Figure (1), all standard hybrid examines the movement of objects in an image video systems like Motion Picture Expert Group sequence and tries to obtain vectors 1, 2 and 4 MPEG-1, MPEG-2, MPEG-4, representing the estimated motion. As a means International Telecommunication of exploiting the temporal redundancy, it is a recommendation ITU-T Rec. H. 263 and H.264 key part of video coding and compression. It encodes set of image signals to reduce spatial may relate to the whole image or specific parts and temporal redundancy by dividing the such as rectangular blocks, arbitrarily shaped frames of video signal into group of pictures patches or even per pixel. Motion vectors MVs (GOP). Thereafter, classifying and ordering may be represented by model or many other each frame into I, P or B frames (intra, inter models that can approximate the motion of a and Bi-directional frames). Image data in each real video camera, such as rotation and frame are split into regions (blocks) for the translations in all three dimensions (3D) and prediction of a block in the current frame, zoom. using reference frames as the previously decoded frames (last I or P-frames). M and N Methods for Motion Compensation represent the distance between core pictures Bi-directional Motion Compensation and the number of core pictures respectively. As shown in fig 2 the current frame is predicted Therefore, M×N=Q which means the total with reference pictures before and after. This number of pictures in a group of pictures. makes it unsuitable for interactive application due to the delay of several frames. It is an Multi-Hypothesis Motion Compensation example of multi-hypothesis motion- (MHMC) compensated prediction where two motion- Multi-hypothesis motion compensation simply compensated predicted signals are involves the application of motion vectors superimposed to reduce the bit-rate of a video (block bases) to an image to synthesize the codec. Therefore, different coding standards: transformation to the next image through Motion Picture Expert Group: MPEG-1 and efficient and accurate measurement of the MPEG-2 or ITU-T Rec.H.263 utilizes B-pictures displacement field between two frames for bi-directional motion compensation. Motion (spatial-temporal displacement). It has found estimation problem is solved with the iteration many applications in video coding such that of motion vectors MVs by independent search coders employ motion-compensated prediction method which includes minimization of the signals that are superimposed to predict the prediction error by fixing and searching the original frame for easy reconstruction at the forward and backward (MVs). So, for joint decoder. Therefore, motion vectors MVs are estimation of forward and backward motion applied to describe the transformation from vectors in Bi-directional prediction, a low- one 2-D image to another, usually from complexity iterative algorithm was introduced adjacent frames in a video sequence. to minimize the prediction error without Successive video frames may contain the same considering the rate constraint (Wu and Gersho, objects (still or moving). Therefore, it 1994).

2 Special Conference Edition, November, 2018

Figure 2: Bi-directional motion compensation with reference pictures used before and after the current frame

Overlapped Block Motion Compensation errors and well pronounce in robustness when (OBMC) compared to the single frame approach. But by Overlapped block motion compensation is randomized frame selection among multiple another example of multi-hypothesis motion previous frames, robustness is highly compensated prediction similar to bi- pronounced. Multiple frames for motion directional prediction (Orchard and Sullivan, compensation also provide a significant 1994). This is specifically aimed at reducing the improved prediction gain. Multi-frame motion- blocking artifacts caused by block-based motion compensated prediction is used to control error compensation. OBMC does not increase the propagation for video transmission over number of vectors per block unlike the bi- wireless channels. directional prediction, but uses more than one Long term memory motion-compensated motion vector for predicting the same pixel. prediction employs several reference frames Multi-frame Motion Compensation (several previously decoded frames) while As shown in figure 3, this method of motion standard motion-compensated prediction uses compensation utilizes multiple frames with one reference frame (Wiegand et al., 1999). additional reference picture parameter chosen Therefore, this is done by assigning a variable to predict the current frame. Therefore, multi- picture reference parameter to each block frame block motion compensation makes use of motion vector in order to overcome any redundancy that exist across multiple frames in restriction that a specific block should be typical video conferencing sequences to chosen from a certain reference frame, which achieve additional compression over that consequently helps in the improvement of the obtained by single frame block motion compression efficiency of motion-compensated compensation. prediction. Hence the concept is incorporated Similarly, the multi-frame approach has an into the ITU-T Rec.H.264 and also the annex of inherent ability to overcome some transmission ITU-T Rec.H.263.

Figure 3: A multi-frame motion compensation with a reference frame chosen by an additional picture reference parameter.

MATHEMATICAL ANALYSIS OF OPTIMAL BIT ASSIGNMENT By minimizing the average reconstruction error variance [11] as a distortion we allocate bit-rate utilizing the Lagrange multiplier (λ) method (Li et al., 2009; (Flierl et al., 1998) with a Lagrange cost function j given by = + λ j D Rc (1)

Where, D and R c are distortion and constant bit-rate constraints respectively Taking into account the constant bit-rate constraints given by = + ( − ) − Rc RI Q 1 RMHP QR , (2) Where, min is the minimum reconstruction error variance given by 1 2 min σ 2 = [σ 2 + ()Q −1 σ 2 ] (3) r Q ,Ir MHP The target function becomes   = 1 []σ 2 + ()− σ 2 + λ[]+ ()− − min j min  ,Ir Q 1 MHP RI Q 1 RMHP QR  (4) Q  Where, min j is the minimum Lagrange cost function 3 Special Conference Edition, November, 2018 By relating equation (1) and equation (4) the derivative of the function j is given by dj dD = + λ = 0 (5) dR dR Since 1 ()Rj , R ,λ = [σ 2 + ()Q −1 σ 2 ]+ λ[]R + ()Q −1 R − QR ( Flierl et al., 2002). MHP I Q ,Ir MHP I MHP Then ∂j 1− Q σ 2  = log e + λ()Q −1 = ,0 ∂ 2 σ 2  RMHP 2Q  s  1 σ 2  λ = log  e , )6( 2 σ 2 2Q  s  Since = + λ RMHP R . (7) Substituting equation (6) into equation (7) we have 1 σ2  R =R+ log  e . )8( MHP 2Q 2 σ2  s  Hence, where and are the quantization performance factor and bit rates for the multi- hypothesis frames respectively and R is the overall bit rate. we now also obtain that for the cases σ 2  of P-frame, 2, 3, 4 and 8 hypotheses respectively given  e  in (Sam, 2015): σ 2  s 

1 ∆ ∆ R = R+ log [2−2ρ r ρ t ], )9( Pframe 2Q 2 s t   = + 1 3 − ρ∆r ρ∆t + 1 ρ∆h R2MHP R log 2  2 s t s , )10( 2Q 2 2    = + 1 4 − ρ∆r ρ∆t + 2ρ 3∆r R3MHP R log 2  2 s t s , )11( 2Q 3 3    = + 1 5 − ρ∆r ρ∆t + 1 ρ 2∆r + 1 ρ2∆r R4MHP R log 2  2 s t s s , )12( 2Q 4 2 4    = + 1 9 − ρ∆r ρ∆t + 1 ρ 2∆r + 1ρ2∆r + 1 ρ 22 ∆− r + 1 ρ 22 ∆+ r R8MHP R log 2  2 s t s s s s  )13(, 2Q 8 4 8 4 4 

MATERIALS AND METHODS A script is then used to run a log function In this work, bit-rates for Multi-hypothesis defined on the command window for a (equation 9-13) were analysed and optimized reasonable optimization. • with MATLAB in decoding signals of 100 pictures In the second case, , , ∆, ∆, were per group of pictures (GOP). Editor’s script and set at 0.8, 0.93, 0.3, 0.64, 1.0 and 1-100 command window of MATLAB were used to run respectively. A script is then used to run a log the analysis with the following simulation function defined on the command window for a procedures reasonable optimization. • Initially, output functions (bit-rate of multi-• In the third case, we also set hypothesis frames) such as RP, R2MHP, R3MHP, , , ∆, ∆, at 0.95, 0.93, 1.0, 0.50, R4MHP, R8MHP and input 1.0 and 1-100 respectively. A script is then used parameters , , ∆, ∆, were used. to run a log function defined on the command • Also, we generated script for the output window for a reasonable optimization. • functions in the editor’s script environment of Lastly, setting , , ∆, ∆, at 0.95, the MATLAB by defining a log-function for the 0.93, 0.3, 0.95, 1.0 and 1-100 respectively, a input parameters as [RP, R2MHP, R3MHP, script is then used to run a log function defined R4MHP, R8MHP]= log (, , ∆, ∆, ) on the command window for a reasonable • In the first case, we set , , ∆, ∆, at optimization. 0.8, 0.93, 1.0, 0.54, 1.0 and 1-100 respectively. 4 Special Conference Edition, November, 2018 • The output values of the optimal bit-rates for • Finally, the display of the 2-D graphics plot multi-hypothesis frames were obtained with the shows the performance limit of bit assignment given values of the input parameters on the for multi-hypothesis frames in motion command window due to the generated script compensated signal compression. in each case. • Hence, we plotted a two dimensional (2-D) RESULTS AND DISCUSSION graph of bit-rates for multi-hypothesis frames Setting the values of parameters as given in against number of pictures per group for each Figure 4, bit-rate of 8MHP frames is optimal at case using plot command which contains grid, 0.948 bpp while that of P frames is optimal at legend of strings and colors with x and y label 0.747 bpp. Therefore, an optimal bit-rate of for Q and the output functions R respectively 0.201 bpp is collectively obtained for 8MHP as, plot(Q,RP,'m.',Q,R2MHP,'b--',Q,R3MHP,'k- frames over P frames. .',Q,R4MHP,'k-',Q,R8MHP,'r:');

1.2

1

0.8 RPframes [bpp] 0.6 R2MHPframes [bpp] []bpp] R3MHPframes [bpp] 0.4 R4MHPframes [bpp] 0.2 R8MHPframes [bpp]

Bit-Rate forBit-Rate Multi-Hypothesis Frames 0 0 50 100 150 Number of pictures in GOP

Figure 4: Bit-rates for multi-hypothesis with , , ∆, ∆,, .,.,., .,., − respectively. Setting the values of parameters as given in Figure 5, bit-rate of 3MHP frames is optimal at 0.964 bpp while that of P frames is optimal at 0.852 bpp. Therefore, an optimal bit-rate of 0.112 bpp is collectively obtained for 3MHP frames over P frames.

1.2

1

0.8 RPframes [bpp] 0.6 R2MHPframes [bpp] R3MHPframes [bpp] 0.4 R4MHPframes [bpp] 0.2 R8MHPframes [bpp] 0 Bit-Rate forBit-Rate Multi-hypothesis [bpp] 0 20 40 60 80 100 120 Number of Pictures in GOP

Figure 5: Bit-rates for multi-hypothesis with , , ∆, ∆,, .,.,.,.,., − respectively. Setting the values of parameters as given in Figure 6, bit-rate of 8MHP frames is optimal at 1.638 bpp while that of P frames is optimal at 1.197 bpp. Therefore, an optimal bit-rate of 0.441 bpp is collectively obtained for 8MHP frames over P frames. 5 Special Conference Edition, November, 2018 1.2 1 0.8 0.6 RPframes [bpp] 0.4 R2MHPframes [bpp] 0.2

[bpp] R3MHPframes [bpp] 0 R4MHPframes [bpp] -0.2 0 20 40 60 80 100 120 R8MHPframes [bpp] -0.4 -0.6 Bit-Rate forBit-Rate Multi-hypothesis Frames -0.8 Number of Pictues in GOP

Figure 6: Bit-rates for multi-hypothesis with , , ∆, ∆,, .,.,.,.,., − respectively. Setting the values of parameters as given in Figure 7, bit-rate of 3MHP frames is optimal at 1.706 bpp while that of P frames is optimal at 1.424 bpp. Therefore, an optimal bit-rate of 0.282 bpp is collectively obtained for 3MHP frames over P frames.

1.2 1 0.8 0.6 RPframes [bpp] 0.4 0.2 R2MHPframes [bpp] 0 R3MHPframes [bpp] -0.2 0 20 40 60 80 100 120 Frames[bpp] R4MHPframe [bpp]s -0.4 R8MHPframes [bpp] -0.6

Bite-Rates forBite-Rates Multi-Hypothesis -0.8 -1 Number of Pictures in GOP

Figure 7: Bit-rates for multi-hypothesis with , , ∆, ∆,, .,.,.,.,., − respectively.

DISCUSSION displacement ranging from 0.3 to 1.0. An The Figures above (4, 5, 6 and 7) demonstrate optimal bit-rate is obtained with three the effect of the number of pictures in a group hypotheses from the Figures (4, 5, 6 and 7) over the optimal bit-rates of multi-hypothesis which signify a better prediction and frames (MHP-frames) using inter-frame video reconstruction of frames using up to eight coding with the values of the parameters such hypotheses in order to achieve improved as time correlation coefficient () , radial compression of video signal transmission. displacement (∆), time distance between In Figures 4 and 5, an optimal bit-rate of 0.201 pictures ( ∆) and spatial correlation coefficient bpp and 0.112 bpp is obtained and allocated for () suggested by video sequence (Ohta and 8MHP frames and 3MHP frames respectively Katto, 1995). A MATLAB program is used for the over P frames when 100 pictures in a group are simulation of the parametric equation of considered. Therefore bit-rate decreases for optimal bit-rates for the multi-hypothesis about 0.089 bpp when time correlation frames (9 to 13) over numbers of pictures in a coefficient () = 0.8 and radial displacement group of pictures (GOP). The optimal bit-rates (∆) ranges from 1.0 to 0.3 at a greater time for multi-hypothesis frames were varied over interval ( ∆) of 0.54 to 0.64. 100 pictures per group with the radial

6 Special Conference Edition, November, 2018

In Figures 6 and 7, an optimal bit-rate of 0.441 better video compression for digital storage and bpp and 0.282 bpp is obtained and allocated for transmission. 8MHP frames and 3MHP frames respectively CONCLUSION over P frames when 100 pictures in a group are Bit-rate for multi-hypothesis frames has been considered. Therefore bit-rate decreases for theoretically and computationally analyzed in about 0.159 bpp when time correlation order to compensate their assignment over coefficient () = 0.95 and radial displacement number of pictures per group for an achievable (∆) ranges from 1.0 to 0.3 at a greater time video signal compression quality. An optimal interval ( ∆) of 0.5 to 0.95. Also, a greater time bit-rate of the range 0.089 bpp to 0.159 bpp correlation coefficient ( ) of the range 0.8- was achieved with up to eight hypotheses. This 0.95 between picture elements with optimal bit shows a qualitative signal compression assignment of 0.089 bpp-0.159 bpp implies a performance analysis for high technology motion compensated digital video codec in communication services and storage processes.

REFERENCES Flierl, M. and Girod, B. (2003) “Generalized B Orchard, M. T. and G. J. Sullivan, (1994). pictures and the draft H.264/AVC "Overlapped Block Motion video-compression standard,” IEEE Compensation: An Estimation-Theoretic Trans.Circuits Syst. Video Technol., Approach," vol. 3 no. 5, pp. 693–699 vol. 13, no. 7, pp. 587-597,. Sam, A. (2012). “ANALYSIS OF CODING GAIN Flierl, M., Wiegand, T. and Girod, B. (1998). “A AND OPTIMAL BIT ALLOCATION IN Locally Optimal Design Algorithm for MOTION – COMPENSATED VIDEO Motion-Compensated Prediction,” Proc. COMPRESSION, ” Journal of Electrical Data Compression Conf. , pp. 239–248. Engineering,” vol. 63, no. 2, pp. 129– Flierl, M., T. Wiegand, and B. Girod, (2002). 132. “Rate-constrained multi-hypothesis Sam, A. (2015). “Mathematical modeling of prediction for motion-compensated coding gain and rate-distortion function video compression,” IEEE Trans. in multihypothesis motion Circuits Syst. Video Technol., vol. 12, compensation for video signals,” Int. J. no. 11, pp. 957-969,. Electron. Commun. ( AEÜ ) , vol. 69, pp. Li, X., N. Oertel, A. Hutter, and A. Kaup, 487–491,. (2009). "Laplace distribution based Wiegand, T. X. Zhang, and B. Girod, (1999). Lagrangian rate-distortion optimization "Long-Term Memory Motion- for hybrid video coding," IEEE Trans Compensated Prediction. IEEE Circuits Syst. Video Technol. vol. 19, Transactions on Circuits and Systems no. 2 pp. 193–205,. for Video Technology , vol 9 no. 1, Kung, W.Y., C.S. Kim, and C.C. J. Kuo, (2004). pp.70–84. “Analysis of multi-hypothesis motion Wu, S.-W. and A. Gersho, (1994). Joint compensated prediction for robust Estimation of Forward and Backward video transmission,” in Proc. IEEE Motion Vectors for Interpolative ISCAS, Vancouver, Canada , vol. 3, pp. Prediction of Video. IEEE Transactions 761-764,. on Image Processing, vol. 3 no.5, pp. Kung, W.Y., C.S. Kim, and C.C. J. Kuo, (2004). 684–687. “Error resilience analysis of multi- hypothesis motion compensated prediction for video coding with multi- hypothesis motion compensated prediction,” in Proc. IEEE ICIP, Singapore pp. 821-824, Lin, S. and Y. Wang, (2002). “Error resilience property of multi-hypothesis motion- compensated prediction,” in Proc. IEEEICIP, Rochester, NY vol. 3, pp. 545-548,. Ohta, M. and Katto, J. (1995). Mathematical analysis of MPEG compression capability and its application to rate control. Int. Conf. Image Process. Vol. 2, pp 555 - 558. 7