Friend or Foe? High End Designers Reassess the Importance of Audio Measurements BY MYLES B. ASTOR

evolution was in the air. Rebelling against the establishment was the in thing to do in the ‘60s. It was a time of social tur- Rmoil and the birth of new ideas. A time for distrust of the establishment—and in the case of audio, of test measurements. And there were good reasons back then not to trust the measurements. This was an era when J. Gordon Holt, the founder of Stereophile magazine recalls, “Everyone sell- ing audio equipment lied about their mea- surements. Published specs were not accu- rate.” Eventually this resulted in magazines being entrusted with corroborating manu- facturer-supplied test data and, in the long run, the standardization of measurements. Holt led the revolt against using only audio measurements to evaluate audio equipment. Serving as chief equipment tester for High Fidelity magazine, he was becoming increasingly frustrated with its editorial policy. Holt believed that High Fidelity was only paying lip service to the sound of components. After all, audio equipment couldn’t reproduce life-like larities between the two publications, how- naught distortion amplifiers of the ‘70s. music; so why even care? Holt was dis- ever, ended there. Reviews in the Hirsch- Why is it that tube gear, with decidedly turbed in particular by his observation that Houck newsletter were based entirely upon poorer measurements than transistor- “two amplifiers that measured close to measurements and, as JGH observed, “ real- based equipment, is growing in popularity? each other sounded dramatically different.” ly didn’t tell us anything we didn’t already Even more recently, there’s the digital The realization that measurements have know.” audio debacle. little relationship to sound quality didn’t Stereophile was far from an overnight Yet today’s leading high-end audio come as “an epiphany,” Holt recalls, but success. Holt recalls the initial “lukewarm” designers haven’t given up hope that they rather something he had experienced over reaction to the newsletter. There were few will eventually find a set of measurements the years. So as with most revolutions, letters of congratulation (like you see regu- that correlates with what we hear to aid in changes were more of a gradual evolution. larly published in today’s magazines). the design of audio equipment. As Holt Holt eventually left High Fidelity maga- Audiophiles discovered the newsletter over noted in the ‘60s, “Observations have gone zine and joined Weathers, where he was time. And so the subjectivist school of audio far beyond our ability to explain,” and, “We entrusted with writing their dealer’s reviewing was born—and the battle haven’t found any measurements to newsletter. Over time, the demand for between objectivist (lovingly known as describe what we are hearing.” Forty years Weathers’ newsletter by customers out- “obs”) and subjectivist (“subs”) camps was born. later, we’re still struggling with these issues. stripped that of dealers. So in 1963, Holt, Like in the ‘60s, there’s still an intense Will the next millennium bring us any clos- with a background in journalism and a love sense of distrust between the two camps. er to the answer? We’ve approached a of audio (and music) since age 14, decided Part of this is the realization that today’s cross-section of high-end audio’s leading to take the plunge and launch his own standard static bench tests were selected to electronic, speaker and cable designers to newsletter titled Stereophile. make audio equipment look its best. find out what they think; we asked them: Holt was quick to point out that the Probably the most infamous examples of ● What technical measurement comes the Hirsch-Houck newsletter was the first measurements leading us down the garden closest to predicting the sound of a piece of

audio newsletter, not Stereophile. All simi- path were the horrible sounding, triple electronics or speakers, and why? ILLUSTRATION BY STUART GOLDBERG

12 • ULTIMATEAUDIO,SPRING 1999 ● At what point in the design of a compo- Correlating measurements to anomalies nent do you switch from using measure- is easy, if that is all that is desired. ments to listening? However, having a measurement for ● Finally, do you think we will ever see the all the different sounds we hear day when we have a set of measurements would be like trying to establish a that will accurately predict the sound of a measurement for all that we component? smell or see. We have a good set Here’s the initial installment of their of tools now, if you can work thoughts on the subject of whether there’s with a TEF-20 and Jon Risch’s been any progress in correlating test mea- program or really understand a surements with sound quality of audio Tektronix TDS 350 or other pro- equipment. grammable scope. With these tools we can predict that a George Cardas, CEO, component will sound bad or Cardas Audio possibly even “not so bad.” As A cable’s ability to replicate a waveform, far as a measurement for “awe- such as a square or triangle, is one measure some” or “state-of-the-art” or I use. High resolution programmable oscil- “this is the best system I have ever loscopes can find the “thumb print” of heard”—listen to the music! with gross frequency response variations. many audible anomalies. FFT analysis and Bio Notes: George Cardas has a gift for math- One problem we encountered early in harmonic distortion testing can also be a ematics and electronics, and a love of music. the development of our VTPH-1 phono predictor of anomalous behavior. There is a He holds several U.S. Patents for cable stage was that seemingly identical units very interesting AES paper presented by Jon stranding, pressure differential microphones exhibited a variation in unit-to-unit charac- Risch at the latest AES convention, in and connector designs. George loves working teristic sound. To find the cause of this which he explains a system for measuring with other designers, “helping musicians variation we designed a piece of test equip- cable distortion using a TEF-20 and a PHI reproduce their music.” ment capable of measuring the RIAA series of tones (AES Preprint #4803, “A New response of each unit to an accuracy of one Class of In-Band Multitone Test Signals,” Keith Herron, President, millibel (0.01 dB). Graphs were made of a AES website, http://www.aes.org). It clearly Herron Audio number of units and compared to listener shows differences between cable and Two fellows I knew in the late ‘60s used an evaluations done in a number of audio sys- cabling systems, such as bi-wiring. His sys- oscilloscope kit, built with 10% tolerance tems after each listening session. The most tem plots harmonic distortion in cable cur- parts at best, to inform a prominent local highly rated units were always the ones rent transfer. I am now working on a com- musician that she was off-pitch. Although with the most accurate RIAA response. mercial implementation for this system. the words she used to rebuke them may Listener ratings of units could be reversed Correlating measurements with audible have been off-color, she was likely not off- by altering the response curves between performance is part of a continuing, evolu- pitch. I have often marveled at the fact that units. Listeners often heard differences on tionary process. My designs are well-formed two pieces of audio equipment with similar the order of 3 millibels (0.03 dB). internally before the prototype runs. Usually frequency response specifications, as pro- After reviewing these results, it the results are very close to expectations. In vided by the manufacturer, could sound becomes clear that specifications of plus the final analysis, I find that I switch back totally different. One might sound warm and minus 0.5 dB have very little meaning and forth constantly, but listening is always and lush, and the other cold and analytical. when comparing audible colorations the final determining factor. This type of dichotomy would lead many to between various pieces of audio gear. It believe that measurements in general, also comes as no surprise that a frequency unless gross differences were evident, have response variation on the order of decibels little meaning. Possibly the real problem is, at 100 kHz would be audible, since it usual- however, that many specifications are like ly means that there is a response variation trying to measure the width of a human of several millibels in the 5, 10 and 20 kHz hair with a yardstick. (audible) range. Maybe the sensitivity of our ears to Another often confusing measurement frequency response has been drastically is distortion. Why is it that tube amplifiers underestimated. We recognize the voices often have much higher distortion figures, of people we know by often subtle fre- yet often sound cleaner than many solid quency variations in their voices, even over stage designs? Many triode tube circuits the telephone. This suggests that we are produce second harmonic distortion that keenly sensitive to relative frequency rises percentage-wise in a way that is lin- response variation. This could also explain ear with output. As the output signal our ability to hear minute frequency increases, so does the distortion. Odd order response differences between audio com- distortions exhibited by these configura- ponents when compared using speakers tions, although minute at low level,

ULTIMATEAUDIO,SPRING 1999 • 13 Measurements ment for Ampeg, Crate, and Audio Centron. have audible effects. On the other hand, I increase by the square of the output level. He combines a strong background in electron- am convinced that phase shift is very The bottom line is that the distortion can ics, mathematics, physics and music with a important at the low frequency end of the diminish to infinitesimal levels at very keen ear and an open-minded analysis of spectrum. small signal levels, and that’s where the audio equipment design. He describes himself Keep in mind that I have merely detail and life of the signal is. Many bipolar as an audiophile, a lover of music, and a “for- touched upon a few aspects of frequency transistor solid-state designs, on the other mer” musician. response measurements that give an indi- hand, have distortion percentage levels cation about the final sound of a speaker in that are seemingly small but constant from Siegfried Linkwitz, VP, R&D, your listening room. Many other important very small signal levels to near full output. Audio Artistry measurements require careful execution If distortion measurements are made at I would have to say that measurement of and interpretation during the design of a near full power for both tube and transistor the speaker’s frequency response is the great speaker. For instance there’s equipment, the higher distortion figures of technical measurement that comes closest measurements of non-linear distortion in the tube equipment will be misleading to predicting the sound of a loudspeaker. If drivers, cone excursion capability, reso- since most of the “life” in real and recorded you see a graph of a 20 to 20,000 kHz ±2 dB nances in drivers and cabinets, and the music takes place at very low levels, where on-axis, anechoic response, you can be fair- critical room/speaker interaction. the distortion is very low. ly sure that the speaker has a smooth After all of these factors have been eval- After hearing this brief discussion on sound characteristic. However, this infor- uated in light of their audible significance, the importance of frequency response and mation alone does not necessarily imply and after inadequate design parameters distortion, one might think that flat fre- that the sound is neutral, transparent and have been modified wherever possible, quency response and low distortion are all dynamic, and that it would display those then it is time to listen. I listen in my living that are necessary in a good sounding characteristics in your room. room at great length and with a wide vari- audio design. I wish it were that simple. First of all, it takes more than the ane- ety of source materials, including my own Designing a product with flat frequency choic on-axis response to describe the DAT recordings. It is difficult to find gen- response and low distortion figures is a sound you hear. When you listen in a room, uinely neutral sound reference material- good place to start, but these parameters your ears not only receive the on-axis sound and I consider it mandatory to regularly don’t guarantee very much. Measurements, but also, with a slight delay, the sound that attend live concerts to hear unamplified as we are currently doing them on the one has bounced around in your room and been instruments. The listening sessions may millibel level, fail to tell us why capacitors, modified by it. Thus, the anechoic response lead to touch-ups or refinements of the resistors, wire, power supplies, board lay- must also be measured at angles up to at fundamental speaker design, but the corre- outs and circuit configurations sound dif- least ±60 degrees horizontally and ±15 lation between the set of measurements ferent. Although we have a number of the- degrees vertically off-axis, in order to assess that I use and the audible results is very ories and models on the sonic behavior and the contribution of sound radiation in these strong. I am also constantly working on nature of individual parts, there is no guar- directions to the overall sound, when the refining those measurements, based on a anteed measurement beyond tolerance and speaker is placed in a room. Simply averag- study of the psychoacoustics of hearing known quantities such as dialectric absorp- ing the different anechoic responses over a and my own perceptions. tion to predict a sonic fingerprint. These “listening window” or normalizing them to Bio Notes: Siegfried Linkwitz designs dipole- aspects comprise the bulk of the design the on-axis response is not as relevant a type loudspeakers for Audio Artistry, pursuing work and are currently based on experi- description as is plotting the actual frequen- a love for music and realistic sound reproduc- ence and a lot of experimentation. cy response curves at different angles on the tion that started as a hobby more than 30 I suspect we will need to measure at same graph. years ago. He recently retired from Hewlett- much much higher resolution—possibly The other half of the equation, which is Packard Co. after working for 37 years in using an electron microscope rather than a also related to a speaker’s frequency research and development of test equipment yardstick—in order to predict the sonic per- response, is its phase response. The phase for radio frequency and microwave measure- sonality of individual and combined parts. response can be grossly different for two ments. Although there has been considerable speakers, even though the magnitude of improvement in the area of measurement their corresponding frequency response Howard Mandell, President, equipment and technique with some very may be similar. Interpreting the audible Altis Audio useful results, it is hard to imagine a mea- significance of a system’s phase response I think the single most important measure- surement procedure that will disclose the curve is even more difficult than assessing ment in any digital processor or drive is the true audible value of a piece of audio gear. the subjective impact of a particular ampli- total amount and type of jitter in the sig- Possibly it will occur at the same time that tude response. Instead of looking just at nal. Jitter causes noise which infiltrates the a camera is contrived capable of measuring the phase response, you can look at a playback and can cause very deleterious the perceived value of a painting. speaker’s step response. It highlights the effects. It can manifest itself as white Bio Notes: Keith Herron is the owner and prin- effects of phase shift, but primarily in the noise, harshness, bright highs or just sim- cipal designer at Herron Audio. A graduate of higher frequency range. However, I am not ply bad timing of the music. This means Iowa State University and a Registered convinced that the typical amounts of that the pace and timing of the perfor- Professional Electrical Engineer, he formerly phase shift due to crossovers that are mance could, in fact, be compromised. We served as director of research and develop- observed in these higher frequency regions feel that jitter should be kept to well below

ULTIMATEAUDIO,SPRING 1999 • 15 Measurements ing to measurement until eventually we uncover the actual performance potentials 2 picoseconds. decide to ship it. At that point, we hope we of complete audio systems. We stop designing by measurements have gotten it completely right, and occa- The speed and processing abilities of once we have completed the first layout sionally we have. current computer-interfaced testing equip- and completed board. Then we begin to lis- That is a matter of opinion. I have seen ment enable Jeff Rowland Design Group ten. We listen to different filter coefficients, pieces in Stereo Review and elsewhere stat- (JRDG) to perform tests which were impos- different timing signals and different ing outright that measurements have sible a few short years ago. Among the capacitors, just to name a few criteria. We already adequately defined performance most important tests for musical reproduc- also listen to make sure the layout of the and that the subjectivists are fooling them- tion accuracy is the Fastest procedure PC boards has been fully optimized. This selves. The opposite viewpoint is stated just developed by Audio Precision, Inc. This test could result in the moving of one or more as dogmatically. Both sides are emotional, will uncover wide band, dynamic non-lin- components for better sound. For example, and neither side is particularly reasonable. earities in DUT’s (devices under test) previ- the crystal oscillator might have to be Obviously, reality sits somewhere in ously unexposed by twin-tone IMD (inter- moved to keep the noise of this device from between. Some measurements do tell you modulation distortion) and THD + N (total infiltrating the analog signal. Things like something about the sound, but not very harmonic distortion plus noise) testing pro- this have to be studied carefully. reliably, and there are clearly some phe- cedures. A composite of either 32 or 64 dis- Personally, I don’t think we will ever be nomena going on that are not being mea- crete tones, non-harmonically spaced able to design using only measurements. I sured. On the other hand, I have witnessed throughout the 20 to 20 kHz bandwidth, is think they can only take you so far. They blind tests where the participants could introduced to the DUT inputs. The DUT can get you close but, the final realization of not hear a difference, or heard wild differ- outputs are routed back into the computer any design is in the listening. We take our ences that could not have existed. interfaced analyzer, which sharply attenu- units to many people to hear them in many Me, I don’t care that much; in fact, I find ates each of the 32 or 64 original tones. The systems before the product ever sees the the subject kind of boring. We build ampli- resulting intermodulation (sum and differ- light of day. I think the products that sound fiers that sound good and measure reason- ence) tones of the original tones are then like music are ones that not only make use ably well and don’t break. If you want to get integrated and displayed for analysis of good measurements, but also make use a machine to listen to them for you, be my and/or continuing product development. of the human experience. After all, listening guest! Note that this test signal more accurately to music is a human experience. Bio Notes: Nelson Pass received his graduate represents a musical signal due to its high Bio Notes: Howard Mandel has been involved in degree in physics from the University of content of discrete frequencies. Any non- the design and manufacture of digital products California-Davis and founded Threshold linearities present in the DUT directly cre- for many years. He founded Altis Audio eight Electronics in 1974. He sold Threshold ate a multitude of spurious difference fre- years ago in an attempt to build a more realistic Electronics in 1987 and founded PASS Labs in quencies, which fall throughout the lower sounding digital processor than was available 1991. Perhaps best known for his Class A amplitude ranges of the entire audio spec- at the time. The company has grown and now design amplifiers and single-ended transistor trum, significantly limiting and compress- serves audiophiles worldwide. amplifiers, Pass holds eight U.S. patents on ing the usable dynamic range. A poor result audio circuits. on this test correlates to the common lis- Nelson Pass, President, tening experience of dynamic range com- PASS Laboratories Jeff Rowland, President and pression, congestion, loss of detail and The standard bench measurements (fre- Chief Designer, Jeff Rowland obscuration of the silence and harmonic quency response, harmonic distortion con- Design Group integrity in music as the material becomes tent, speed, damping factor, etc.) all provide There is definitely a close relationship increasingly complex. useful feedback in the design process. Their between test measurement (specifications) Another overlooked area of testing is importance occurs in the context of a given and subjective sound quality. However, cur- evaluation of the DUT in a radio frequency topology and hardware, where the num- rent testing procedures commonly used in interference (RFI) environment. Thorough bers help you dial in the “sweet spot” and audio development, manufacturing and the testing and analysis of RFI, immunity in speed the process of subjective evaluation. audio press are inadequate to properly cor- audio equipment is increasingly important It is nevertheless possible to have a relate and relate measurements with sub- due to the proliferation of telecommunica- product that measures well but doesn’t jective experience. tion devices and computers throughout the sound so good. It is still a mystery as to Many well-meaning critics hold that so world. This testing involves injecting how this could be, but there it is. My expe- far as an audio component measures “per- minute amounts of continuous or multi- rience is that there is a reasonable correla- fect” in certain areas, the audio component tone RF signals into the input, output, AC tion between sound and measurement for will not have a “sound of its own,” and it mains, and chassis of the DUT. The output simple Class A circuitry with minimal or no will be indistinguishable from a component of the DUT is analyzed much the same as feedback. This relationship seems to disap- with similar measurements. The measure- above. Poorly designed equipment which pear when the circuit becomes complex or ment criteria usually involve flat frequency under common lab tests can measure fine, has a lot of non-linearity corrected by feed- response, insignificant static and dynamic will escape this particular scrutiny. In a back. non-linearities, high input impedances, low metropolitan environment, for example, Most audio design is a long, iterative output impedances, low noise and crosstalk multiple RF signals can be demodulated process. We go back and forth from listen- levels, etc. Yet these common notions fail to within the input circuitry of poorly

16 • ULTIMATEAUDIO,SPRING 1999 designed audio equipment. The resulting design, manufacturing and reviewing preting the overall octave-to-octave bal- sum and difference frequencies will mani- stages within a product life cycle, definite ance from measurement is error prone, fest throughout the audio frequency range correlations could be made between tech- since response imperfections are usually and create an obscuring effect upon the nical specifications and subjective cri- on the order of 1 or 2 dB, whereas the ear musical signal much in the same manner tiques. However, this is not generally the can detect errors of a fraction of a dB if over as mentioned above. case today. The result is confusion and a an octave or more. The act of transferring an audio signal distrustful attitude towards technical tests Listening is a necessary method of eval- from one system component to another altogether. Our hope is that more designers uating speakers because of the difficulties without audible degradation is very difficult. and reviewers will explore many of the test of interpreting measurements. The ear and The existence of significant ground voltage and measurement possibilities that are brain can sort out the different sonic differences and the resulting noise currents now possible. effects to determine whether a measured that flow between system components For a detailed discussion on this topic, effect is a major or minor problem, and must be considered in the system as a see the JRDG website, www.jeffrowland. can also untangle all the imperfections to whole. Carefully designed and executed sys- com. determine the tonal balance. The ear’s tem grounding schemes can reduce these Bio Notes: Jeff Rowland’s involvement in con- great sensitivity makes it an indispensable problems but not eliminate them. Audio sumer electronics began in 1980, when he tool for evaluation. equipment that may display excellent test designed and manufactured a line of audio I usually develop a product by first results in isolation can fail miserably in a power amplifiers, preamplifiers and electronic engineering everything possible from cal- typical audio system configuration. crossovers, under the company name of culations, simulations and measurements. The degree of isolation and immunity Rowland Research. Recognition of his unique After the product is developed to the point from these undesirable effects and their approach to audio amplifier design led to that it would seem to be finished as far as resulting problems is indicated by the worldwide acceptance and distribution of a measurements are concerned, I begin the CMRR (common-mode-rejection-ratio). complete line of amplifiers and preamplifiers design phase that includes listening. On (Note that balanced interconnection in 1986 under the current name of Jeff first listening the product might sound schemes are referred to in this discussion. Rowland Design Group, Inc. Mr. Rowland has good, but it often has significant problems, Single-ended interconnected systems, by provided innovative solutions to design prob- and it never sounds great. This phase of their inherent design limitations, can never lems, such as his pioneering implementation identifying imperfections—by listening overcome the problems discussed here.) of balanced circuitry (1986), transimpedence and solving them with the help of mea- Very few manufacturers give specifications topologies (1988), preamp remote control surements-involves a great deal of listen- relating to CMRR; the audio press fails to (1988), low-resonance chassis design (1992), ing time and is a critical design phase for test for this important test parameter alto- and battery power supplies for audio applica- determining whether the final product will gether. To make matters worse, most audio tions (1992), transformer interfacing (1995), be truly exceptional. designers who do test for CMRR do so by and high power integrated circuit applications The personal computer has allowed driving the opposite phase inputs shorted (1997). He is currently involved in the design great improvements in speaker measure- together, which is both unrealistic and mis- of ultra-low distortion, high efficiency, modu- ments during the last 20 years. Although leading. Tests which account for slight lar amplifiers for stereo and multichannel we now have much more information easi- source impedance mismatching between audio applications. ly available to us, the improvements have opposite signal phases, a reality in all audio not facilitated prediction of sonic charac- systems, are rarely done. Noise rejection in Jim Thiel, President, Thiel Audio ter. Extrapolating this trend, I don’t predict a balanced system has nothing to do with No single measurement predicts the sound that measurements will replace listening signal amplitude symmetry, erroneously of a loudspeaker, because there are several as an evaluative tool in the foreseeable regarded as an asset. Noise rejection has separate aspects of speaker performance. I future. everything to do with the balance of com- think the most important measurement is Bio Notes: Jim Thiel is a co-founder, co-owner, mon-mode impedances. Unfortunately, frequency response which, in principle, can and product design engineer at Thiel Audio. this is rarely accommodated for in equip- predict the tonal character of a speaker, Thiel pioneered the principle of time and ment design or mentioned in performance including overall balance and lack of col- phase accuracy in loudspeakers with the use specifications. orations. Keeping in mind that it cannot of sloped baffles, coaxial driver mounting, Audio hardware and interconnect predict the other performance aspects of and phase coherent crossover network design. cables are sensitive to mechanical vibra- imaging, clarity and dynamics, frequency His loudspeaker designs have been frequently tion (microphonics). Sound energy, trans- response can predict many important honored by the audio industry and interna- ferred through structural and air medi- attributes of speaker performance. tional press, earning 12 “Speaker of the Year” ums, can significantly impact the perfor- There is, however, a big difference awards and 15 International Consumer mance of all audio equipment in many between predicting performance in theory Electronics Show “Design and Engineering” domestic environments. Again, equipment and doing so in practice. Interpreting fre- awards over Thiel’s 20-year history. sensitivity to this condition is rarely tested quency response measurements is tricky in manufacturing or considered during the and complicated. For example, it can be dif- Next issue, Part II: More insights into the design process. ficult to differentiate between a relatively subject of audio measurements, and an In conclusion, if more thorough testing benign diffraction effect and a resonance examination of the tools at the designer’s procedures would be implemented at the effect that is quite deleterious. Even inter- beckoning. t

ULTIMATEAUDIO,SPRING 1999 • 17