We didn’t evaluate HDTV formats. It’s Tape Formats Compared difficult to compare a 4:3 image to a 16:9 image. More important, the vast majority of current video production and viewing How do DV formats still occurs in 4:3. We evalua ted Digital Betacam, Digital- S , DVCA M , DVCP RO, Betacam SP,Hi8 , and measure up with 3/ 4 - i nch U-Matic VTR s . To ensure consis- tent results , we use d top- of - t he- l ine VTR s igital Betacam looks better than DV. to evalua te each forma t.We wanted VTR s Betacam SP That’s no surprise. But just how much with the highest- qu a lity components in Dbetter does it look? How does DV order to minimiz e errant measurements . compare to Betacam SP? What about the Where possible, we use d VTRs with SDI and 601? 50Mb (megabit) DV-based formats, I/O to carry the 601 test video to the VTR DVCPRO50 and Digital-S? And how does and to carry the processe d video out of the Digital Video Magazine each format’s image quality VTR . The specif i c VTR models are listed compare to ITU-R BT.601 video? on page 42.We also tested an Accom DDR. evaluates the image Although anecdotes and informal Together these devices cover a large range quality of major video evalu ations abound, Digital Video of current production format s . Magazine wanted to compare image- There are several formats we didn’t production tape formats. qua lity diff erences among leading digital evaluate. We didn’t test DVCPRO50, but by Jim Feeley and important analog video formats. we expected that the results would match That’s what we do here. those of Digital-S,a format based on the In our evaluation, we used a 525-line sa me 50Mb DV data stream as DVCP RO5 0 . 601 signal as our ultimate reference. We didn’t include S-VHS or VHS VT R s.

36 DV May 1999 www.dv.com We expected S-VHS would generate results peak signal- to - n oise ratio (PSNR) and the similar to those of Hi8. We assumed VHS While abstract Picture Qua lity Rating (PQR) measurement. would perform even worse. The PSNR measures diff erences in the We didn’t include a standard DVVTR “codec buster”test luminance component of the reference and because we needed SDI I/O for our core the encoded video. PS NR gives every diff er- evaluation, and we didn’t have an SDI- ence equa l weight, without regard to the to-1394 converter. However, we assumed signals and scenes huma n visua l system. With the PQA-2 0 0 , that standard DV would perform similarly we determined PSNR by finding the ratio to the other 25Mb formats, DVCAM have value for engi- of the peak signal to Root Mean Squa red and DVCPRO. (RMS) noise betw een the reference video We tested a DV- bas ed form at — neers designing and the encoded test video. The PSNR values DVCA M — w ith SDI input and Y/C output, for each field are averaged to give an overall and again with Y/C input and Y/C output. P SNR measurement for each test video These results will interest those who shoot video systems,we s equence on each VT R. The higher the or edit DV-b ase d video who don’t own a PSN R , the more signal and the less noise you 13 9 4 / F ireW ire or SDI- e quipped NLE yet. wanted to know have.A device that copies and plays back the We also didn’t perform multigenerational 601 reference without changing the values of tests .We measured only first- g eneration tapes how the tape any pix el will have an infi nite PSNR . ma de from 601 source video.With the advent of pure digital clones,generation loss is less of PQR measurements a factor in digital video production. Testing formats perform The PQA-200 makes picture quality mea- fi rst- g eneration video let us determine each surements using the Just Noticeable Dif- forma t’s best- p ossible video qua lity. in the real world. ference (JND) algorithm developed by the We conducted our tests in the online suite Sarnoff Corporation. Sarnoff designed at the Bay Area Video Coalition (BAVC ; Many other factors play a role in the JND to accurately emulate the perceptions ww w. b avc.o rg). All of the decks had been final image quality of a video: camera and of the human visual system.(See “How exp ertly maintained and had moderate lighting, possible transcoding errors in To Measure Video Quality” on page 28 drum hours. The DV-b ase d VTRs all had bringing video into an NLE that may not for more on the PQA-200 and JND.) under 700 hours on their recording drums. natively support the same production tape Tektronix’s PQR rating uses the JND The Digital Betacam and analog decks had format, and even different quantization model to correlate the quality ratings given more (but still acceptable) hours. tables for the same video format in a VTR by observers viewing the scenes from a dis- Drum wear can affect performance, but and NLE. However, our tests shed light on tance four times that of the screen heights. that effect is minimal if the VTR is carefully one key factor affecting the overall image For each sequence, the PQA-200 makes maintained and drum wear isn’t excessive. quality of digital video production. a series of ima ge measurements and estab- Drum wear didn’t play a significant factor In each of our im age qu ality test s, lishes a PQR base d on the JND measure- in our results. the Accom DDR, Digital Betacam VT R, ments of four fields, starting with field The analog VTRs didn’t have SDI input and Digital-S VTR finished first, s econd, four of the test sequence (see Figures 1 or output, so we converted between the and third, as you would ex pect. 3 / 4 – inch through 3). For every subse quent PQR SDI signal and analog with several devices. and Hi8 consistently trailed behind. measurement, the system ignores the JND Depending on the format, we chose a com- The middle of the pack is where it values for the oldest field and adds those of bination of an AJA Video D10C SDI-to- gets interesting. the next . The overall PQR rating for a scene analog Component converter, an AJA is an aggregate of the sequentia l PQR values. Video D10A analog Component-to-SDI BEHIND OUR TESTS The PQR luminance measurement ana- converter, a DPS-210 Transcoder, and a lyz es only a video signal’s brightness infor- Prime Image HR600+ time-base corrector. The measurements ma tion; the PQR luminance and chroma We also relied on our eyes and our years Our measurements reveal each tape forma t’s measurement measures errors in both the of experience viewing video. fi delity when reproduci ng several 601 refer- brightness and color channels. Shifts in While abstract “codec buster”test signals ence video sequences.We made our mea- luminance have a greater impact on a scene’s and scenes have value for engineers design- surements with a Tektronix PQA-200 perceived ima ge qua lity,and thus play a big- ing and debugging systems, we wanted to system. The system compares a reference ger role in establishing a device’s PQR score. know how the tape forma ts perform in 601 sequence to the same sequence as The lower the PQR, the fewer observers the real world.We chose video sequences encoded by the test VTR . The PQA-2 0 0 will notice the difference between the with content, detail, and motion similar measures horiz ontal and vertical shifts processed sequence and the 601 reference. to that encountered in everyda y video in luma and chroma , as well as gain and A device with perfect 601 capture and play- production. Spec if ically, we chose three level changes in luma and chroma . It also back will generate a PQR of 0.00.PQR standa rd test sequences: CCIR 36 Ferris measures luma cropping. ratings of 1.00 indicate degradations with Wheel, CCIR 30 Mobile and Calenda r, We measured many parameters on each a small perceptual impact. PQR ratings of and CCIR 16 Susie. VTR . The most signifi cant of these were the 3.00 indicate mild (but observable) image➣ www.dv.com DV May 1999 37 degradation.PQR ratings above 10.00 indi- cate clearly observable image degradation.

Test analysis The Ferris Wheel test sequence —CC I R 36 — h as an abunda nce of luminance and color detail in a scene with high motion. The cars in the video whip around, but most of the tape forma ts handled the motion well. The DDR transparently captured and played back the 601 test video sequence, resulting in an infi nite signal- to - n oise ratio and no ima ge degrada tion. Among the compresse d tape format s ,Digital Betacam’s 1.6:1 compression introduced the least ima ge degrada tion, with aggregate PQR scores

Figure 1—Hi8 PQR luminance map. below 1.00 for both luma , and luma and chroma measurements . Digital- S , with its 50 M b ps (megabits per second) data rate and 3.3:1 compression (nearly twice the compres- sion ratio of Digital Betacam) , produced PQR sc ores about twice those of Digital Betacam. The 25Mb DV-based formats— DVCAM and DVCPRO—both compress the 601 signal twice as much as Digital-S does, and, predictably, had PQR scores twice those of Digital-S.With this particu- lar video sequence, DVCAM produced a PQR score slightly better than that of DVCPRO, but the small difference wasn’t visible to our observers. With the test video, Betacam SP pro- duced images essentially equal in quality to those of DVCPRO and DVCAM, even Figure 2—DVCPRO PQR luminance map. though its PSNR was lower. DVCAM with Y/C output performed only about 0.5 PQR worse than DVCAM and DVCPRO with SDI out—an impressive result, suggesting little added degradation. The non-Component analog formats— 3/4-inch and Hi8, as well as DVCAM when receiving and outputting a Y/C signal— produced the most visible degradation. Hi8’s video was clearly impaired compared to the 601 source. Using this high - motion video, all the digital form ats (except DVCAM with S - Video I/O) and Betacam SP produced very acceptable video. The Mobile and Calen d ar test se quence—C CIR 30—provides detailed, Figure 3—DDR PQR luminance map. colorful objects and backgrounds. Several of the objects move in different directions, For each device we tested, the PQA-200 generated a series of PQR maps.These maps graphically show visible differences in brightness in the encoded test video when compared to the 601 reference video and the camera moves in a slow pan. This (the system generates a separate series of maps for the luma and chroma measurement).The more visi- test proved the hardest for all the tape for- ble the difference to a human observer, the brighter the PQR map.These maps from the Ferris Wheel mats to capture, with the non-Component test sequence reveal that Hi8 (Figure 1) shows significant image degradation,while DVCPRO (Figure 2) shows less degradation. The pure-black DDR PQR map (Figure 3) sho ws that the DDR imparted no image suffering visible image degradation. The changes to our 601 test signal. DDR introduced no image degradation. ➣

38 DV May 1999 www.dv.com 100 Infinite 90

80

70

60 51.85 50 44.37 39.05 39.55 39.12 40 36.1 30 28.2 29.05 26.24 20

10

0 The Ferris Wheel test sequence—CCIR 36—provides color detail and Peak signal-to-noise ratio.Longer bars are better. fast motion.

20 20

18 18

16 16

14 14 12.48 12 12 9.78 10 9.79 10 8.08 8 8 6.28 6.56 6 6 4 4 3.02 2.71 2.53 2.28 2.26 2.38 2.42 2.43 2 2 1.24 1.27 0.68 0.71 0 0 0 0 PQR-Luma. Shorter bars are better. PQR-Luma & Chroma. Shorter bars are better.

Digital Betacam was again the tape for- The Susie test sequence —C CIR 16— formats, DVCAM and DVCPRO. However, ma t with the least- d egraded ima ge, though plays a talking head shot with slow,smooth both Betacam SP and the 25Mb DV for- its (and every other forma t’s) PQR scores motion as the subject leans forward. The mats produced their best images when were higher (thus worse) than on the close up of a huma n face, along with the skin encoding this talking head shot. All the Ferris Wheel test. Digital- S , DVCA M , and and hair detail ,m ake an ima ge that dema nds above formats produced slight to no per- DVCP RO each produced PQR scores about careful compression. Several, but not all, of ceptual image degradation. tw ice as high as their Ferris Wheel scores, the tape forma ts produced their best ima ges When DVCAM output a Y/C signal with DVCAM and DVCP RO showing when encoding this test video. (after receiving an SDI signal), the degrada- impairments that were noticeable, but slight. Digital Betacam and Digital- S , the two tion was noticeable, but not annoying. DVCAM had PQR scores slightly lower digital tape forma ts that apply the least com- 3/4-inch and Hi8 both produced video than DVCPRO; but again, the differences pression, produced the best PQR scores for with significantly impaired image quality. were insignificant. tape. However, these scores and the ima ge For this talking head video, all the digi- The Mobile and Calendar’s highly degrada tion weren’t as low as their Ferris tal formats and Betacam SP produced detailed content proved to be less of a chal- Wheel scores. acceptable images. lenge for Betacam SP,which yielded PQR DVCAM and DVCPRO both scored scores approximately one point better than within one-half PQR point of Digital-S The winner is . .. those of DVCPRO and DVCAM. and within one PQR point of Digital The results of our image quality tests For this highly detailed and colored Betacam—impressive results considering closely matched our expectations. Less video, Betacam SP produced slightly, but their lower data rates and higher compres- compression is better than more compres- visibly, superior results to the 25Mb digital sion. In this test, DVCPRO produced sion. Component video is better than Y/C formats. The non-Component formats— slightly better scores than DVCAM; and Composite video. But beyond these Hi8 and 3/4-inch — showed strongly but as in the other tests, the differences obvious conclusions, we made some degraded ima ges. Only the uncompresse d between these two weren’t noticeable. other observations. DDR produced the same results it did in Betacam SP produced good images, The more detailed and complex the the easier Ferris Wheel test. but not as good as the 25Mb DV-based image we played, the greater the benefit o➣f 40 DV May 1999 www.dv.com 100 Infinite 90

80

70

60

50 47.05

40 35.68 30.28 29.25 29.85 30.36 30 22.24 21.58 22.19 20

10

0

The Mobile and Calendar test sequence—CCIR 36—has lots of detail and Peak signal-to-noise ratio.Longer bars are better. motion. It proved the most difficult challenge.

20 20 19.76 18 18

16 15.65 16 15.17 14 14 12.97 12 12

10 10 8.75 9.12 8 8

6 6 5.15 4.72 5.66 4.66 4.92 4.97 3.78 3.92 4 4 2.5 2.55 2 2 1.1 1.14 0 0 0 0 PQR-Luma. Shorter bars are better. PQR-Luma & Chroma. Shorter bars are better.

Tape Format 3/4 Hi8 Betacam DVCAM DVCAM DVCAM DVCPRO Digital-S Digital 601 SP Y/C I/O Y/C out Betacam DDR Test Source Ferris Wheel CCIR 36 PSNR 28.2 26.24 36.1 29.05 39.05 39.55 39.12 44.37 51.85 Infinite PQR Luma 8.08 9.79 2.28 6.28 2.71 2.26 2.38 1.24 0.68 0.00 PQR Luma & Chroma 9.78 12.48 2.42 6.56 3.02 2.43 2.53 1.27 0.71 0.00 Mobile/Calendar CCIR 30 PSNR 22.24 21.58 30.28 22.19 29.25 29.85 30.36 35.68 47.05 Infinite PQR Luma 12.97 15.65 3.78 8.75 5.15 4.66 4.72 2.5 1.1 0.00 PQR Luma &Chroma 15.17 19.76 3.92 9.12 5.66 4.92 4.97 2.55 1.14 0.00 Susie CCIR 16 PSNR 31.57 32.67 40.07 33.6 42.75 44.79 44.75 48.84 51.65 Infinite PQR Luma 7.61 8.25 2.52 4.93 2.46 1.83 1.75 1.34 0.92 0.00 PQR Luma & Chroma 9.16 13.66 2.66 5.07 2.69 1.92 1.84 1.36 0.99 0.00 Format/VTR Specs Color Sampling Y/C Y/C 4:2:2 4:1:1 4:1:1 4:1:1 4:1:1 4:2:2 4:2:2 4:2:2 Compression Ratio N/A N/A N/A 5:1 5:1 5:1 5:1 3.3:1 1.6:1 1:1 Compression System analog analog analog DV-based DV-based DV-based DV-based DV-based DCT-based uncompressed VTR Model VO-9850 EVO-9850 CVR-70 DSR-30 DSR-80 DSR-80 AJ-D640 BR-D92U DVW-500 WSD/2Xtreme VTR Manufacturer Sony Sony Ampex/Sony Sony Sony Sony Panasonic JVC Sony Accom VTR List Price $13,425 $7900 $39,000 $4800 $9495 $9495 $8495 $20,950 $39,000 $12,900 ➣ 42 DV May 1999 www.dv.com 100 Infinite 90

80

70

60 51.85 50 48.84 44.79 44.75 40.07 42.75 40 31.57 32.67 33.6 30

20

10

0

The Susie test sequence—CCIR 16—emphasizes skin and hair detail. Peak signal-to-noise ratio.Longer bars are better.

20 20

18 18

16 16

14 14 13.66 12 12

10 10 9.16 7.61 8.25 8 8

6 6 4.93 5.07 4 4 2.66 2.69 2.52 2.46 1.83 1.75 1.92 1.84 2 0.92 2 1.34 1.36 0.99 0 0 0 0 PQR-Luma. Shorter bars are better. PQR-Luma & Chroma. Shorter bars are better. mild compression. More formats produced DV-based formats look very good. The two 50Mb DV-based format. For many uses, PQR scores under 2.00 with the Susie format’s PQR scores were consistently the DV formats deliver similar or superior footage than with the Ferris Wheel or within one-tenth of a point of each other. images for the same or less money. Mobile and Calendar footage. We expect standard DV to be capable of Hi8 and 3/4? Well, each was good in its The perceptua l diff erence betw een Digital images equal to those of DVCAM and da y,but those days passe d a few years ago. Betacam and Digital-S is noticeable; but with DVCPRO, if you can get the signal to and So what format should you choose? the talking head and high- m otion ima ge from tape without reducing it to Y/C video. Many factors play a role in answering that content, the diff erence isn ’t very noticeable. Perhaps our most interesting finding question. Camera quality,VTR features, With the highly detailed and complex ima ges involves the 25Mb DV-b ase d format s . If you and compatibility with your current and in the Mobile and Calenda r sequence, the can record to a DV-b ase d forma t without a future equipment must be considered. diff erence is greater; but Digital-S still turns Y/C conversion (for exa mple, by recording Unless you run a close d shop where your in very good- l ooking video. straight from camera to tape) , you can often team acquires, edits , and outputs everything As the data rates suggest, 50Mb DV- get a good ima ge off tape even if you output in house , you must be able to play from and base d forma ts show about half the ima ge through an analog Y/C connection .2 5 M b record to the forma ts that your production degrada tion as 25Mb DV format s .We use d a DV with Y/C out looks good.You will suff er partners and clients require. Digital-S VTR for our 50Mb DV forma t, but an ima ge- qu a lity penalty compared to More money can buy higher image we exp ected that DVCP RO50 would exh ibit Component digital output, but the loss is quality. But our tests show that excellent the same ima ge qua lity improvement over slight. That’s good news if your NLE has image quality is no longer the exclusive DVCAM and DVCP RO,tw o 25Mb format s . neither an SDI nor a 1394/FireW ire input. province of just one tape format. DV DVCAM and DVCPRO perform essen- Betacam SP still holds its own against tially the same. Extremely detailed and the low-cost digital upstarts. If you’re satis- Jim Fe e l ey is DV’s senior edito r. Reach him at complex scenes provide a challenge for fied with Beta’s image quality, there’s no j i m @ dv. co m . Jim would like to thank Sue Fo l ey, these formats, yielding visibly, but not reason to abandon the format. But based Don Bi s h o p, and Michael Weitz of Te k t ro n i x , strongly, impaired video. But with talking solely on perceptible image quality,there’s Heather We aver of BAVC , and Lu ke Hones fo r head and fast-motion video, the 25Mb little reason not to consider a 25Mb or their assistance in pre p a ring this art i c l e.

44 DV May 1999 www.dv.com