Error Control Techniques for Interactive Low-bit Rate Video Transmission over the Internet Injong Rhee Department of Computer Science North Carolina State University Raleigh, NC 27695-7534 E-mail: [email protected] Abstract The e ect of packet loss is also far-reaching. Packet loss degrades not only the quality of the frames con- tained in lost packets, but also the quality of their sub- A new retransmission-based error control technique is sequent frames. This is b ecause of motion comp ensation presented that does not incur any additional latency in employed by video co decs. Motion-comp ensated video frame playout times, and hence are suitable for inter- co decs remove temp oral redundancy in a video stream active applications. It takes advantage of the motion by enco ding only pixel value di erence prediction er- prediction loop employed in most motion compensation- ror b etween an image to b e enco ded inter-frame and a basedcodecs. By correcting errors in a referenceframe previously transmitted image reference frame. Packet causedbyearlier packet loss, it prevents error propaga- loss can intro duce an error in a reference frame, which tion. The technique rearranges the temporal dependency can b e propagated to its subsequent frames and get am- of frames so that a displayedframe is referenced for the pli ed as more packets are lost. decoding of its succeeding dependent frames much later than its display time. Thus, the delay in repairing lost Error propagation can be controlled by more fre- packets can be e ectively masked out. The developed quently adding intra frames which are co ded temp o- technique is combined with layered videocoding to main- rally indep endently. However, the ratio of the com- tain consistently good video quality even under heavy pression eciency of an intra-frame over an inter-frame packet loss. Through the results of extensive Internet is as large as 3 to 6 times. Increasing the frequency experiments, the paper shows that layeredcoding can be of intra-frames could increase the bandwidth require- very e ective when combined with the retransmission- menttoomuch for video transmission over a bandwidth- based error control technique for low-bit rate transmis- constraint network. Nonetheless, the severe degradation sion over best e ort networks where no network-level of image quality due to error propagation has forced mechanism exists for protecting high priority data from several p opular video conferencing to ols, suchas nv[7 ], packet loss. vic[13 ] and CU-SeeMe[6 ], to adopt an even more drastic approach. Using a technique called conditional replen- ishment, these to ols lter out the blo cks that have not 1 Intro duction changed much from the previous frame and intra-co de the remaining blo cks. Since all the co ded blo cks are temp orally indep endent, packet loss a ects only those Transmitting interactive video over packet switching frames contained in lost packets. However, this en- networks is time-constrained. If a packet containing a hanced error resilience comes at the cost of low com- video frame is not delivered b efore its scheduled play- pression eciency. Additional compression can always out time, and the frame cannot be displayed and ac- be obtained if temp oral redundancy is removed from cordingly, the overall quality of video degrades. The each co ded blo ck i.e., by co ding only their prediction diculty of meeting this time constraint lies mainly in error. frequent o ccurrences of packet loss which are commonly caused by network congestion. The recovery of lost The goal is to develop an error recovery scheme packets b efore their display times is not always feasi- that solves the error propagation problem without los- ble b ecause of the delay asso ciated with detecting and ing much compression eciency. Retransmission-based repairing the lost packets. and forward error correction schemes are the two ma jor error control schemes found in the literature. Retransmission-based error recovery REC can pro- vide go o d error resilience without incurring much band- width overhead b ecause packets are retransmitted only when some indications exist that they are lost. However, retransmission always involves additional transmission delay, and thus has b een widely known ine ective for in- teractive real-time video applications such as video con- frames temp orally dep end only on the essential signals ferencing. Many researchers prop osed use of extended of their reference frames. This dep endency ensures that control or playout times to allow retransmitted packets the distortion of the enhancement signals of a frame to arrive in time for display [21 , 18 , 12 , 23 ]. This implies do es not a ect the quality of its succeeding frames. that the playout time of a frame is delayed by at least Since essential signals are protected by FEC, this dep en- three one-way trip times after its initial transmission dency will signi cantly reduce error propagation. How- two for packet transmissions and one for a retransmis- ever, QAL has its own limitations. First, since frames sion request. Under the currentInternet environment, are motion-comp ensated only to the essential signals of this delaywould b e intolerable for interactive video ap- their reference frames, the temp oral redundancy present plications, and can severely reduce the interactivityof in the enhancement signals are not exploited at all, re- the application. sulting in low compression eciency. Second, under heavy packet loss, even essential signals can be lost, Retransmission, however, can be still a very e ec- causing error propagation. tive technique to improve error resilience in interactive real-time video conferencing. In this pap er, we study a These limitations, however, can b e greatly reduced REC scheme, called periodic temporal dependency dis- when QAL is combined with PTDD. In this pap er, we tance PTDD, that can be used for interactive video show that a motion-comp ensated layered co ding tech- applications. In particular, the schemes do not require nique such as QAL can b e very e ective for transmis- any arti cial extension of control time and play-out de- sion over a lossy packet-switched network even when no lays, and thus are suitable for interactive applications. network-level mechanism to protect a prioritized stream In the scheme, frames are simply displayed at their nor- exists. When used with PTDD, the layering technique mal playout times without any delay, as they are de- yields reasonably go o d compression eciency since p e- co ded. Thus, if a packet arrives after the playout time rio dic frames temp orally dep ends on b oth the essential of its frame, the frame will be displayed with errors. and enhancement signals of their reference frames. This However, this \late" packet can b e used to remove error dep endency is safe b ecause retransmission can recover propagation. Rather than discarding the late packet, lost packets containing b oth signals. In addition, p eri- PTDD uses it to restore its frame although the frame o dic frames reduce error propagation b ecause their im- is displayed. Because the frame is used as a reference mediately succeeding non-p erio dic frames use them as frame for its succeeding frames, restoring the reference a reference frame. frame stops error propagation. We conducted b oth transatlantic and transpaci c In- To allow enough time for retransmitted packets to ternet transmission tests from the East Coast of the arrive b efore their frames are referenced for the recon- US to show the utility of the REC-based techniques for struction of their dep endent frames, the PTDD scheme transmission over a long distance network. The exp er- extends the temporal dependency distance TDD of imental results indicated that the prop osed technique frames. The TDD of a frame is de ned to b e the min- achieve go o d error resilience and compression eciency. imum numb er of frame intervals or inter-frame delay In Section 2, we describ e the related work, and in between that frame and another frame on which it is Section 3 we describ e PTDD and QAL schemes. Section temp orally dep endent. In the PTDD scheme, every p- 4 contains a discussion of the exp erimental results, and th frame whichwe call periodic frame has an extended Section 5 contains the conclusion. TDD while the other frames have TDD 1. Note that the extension of TDD do es not a ect the playout times of frames since the playout times of frames are deter- 2 Related Work mined solely byinter-frame delays and the initial con- trol time which is used to reduce delay jitter. In the PTDD scheme, the TDD of p erio dic frames is deter- We rst discuss the related work on retransmission- mined by the estimated delaybetween the sender and based techniques for video transmission over packet- the receiver. switched networks and then discuss related work on lay- ered video co ding. In PTDD, non-p erio dic frames with TDD 1 are not protected by retransmission. If packet loss o ccurs for a Retransmission-based error recovery. Dempsey et al.[5 ] non-p erio dic frame, the frame will b e displayed with er- applied retransmission for the recovery of audio packets. rors which can b e propagated to its subsequent frames They showed that by adding some delay b efore the play- until the next p erio dic frame is received. The receiv- out of each received audio packet, retransmission can ing video quality will p erio dically uctuate. To remedy b e used to protect audio data from packet loss.