The Cardiac Output from Blood Pressure Algorithms Trial*

The cardiac output from blood pressure algorithms trial*

James X. Sun, MEng; Andrew T. Reisner, MD; Mohammed Saeed, MD, PhD; Thomas Heldt, PhD; Roger G. Mark, MD, PhD

Objective: The value of different algorithms that estimate better than mean arterial pressure (MAP) at estimating direction cardiac output (CO) by analysis of a peripheral arterial blood changes in COTD. Only the formula proposed by Liljestrand and pressure (ABP) waveform has not been definitively identified. Zander in 1928 was a significantly better quantitative estimator of

In this investigation, we developed a testing data set contain- COTD compared with MAP (95% limits-of-agreement with COTD: > ing a large number of radial ABP waveform segments and ؊1.76/؉1.41 L/min versus ؊2.20/؉1.82 L/min, respectively; p contemporaneous reference CO by thermodilution measure- 0.001, per the Kolmogorov-Smirnov test). The Liljestrand method ments, collected in an intensive care unit (ICU) patient popu- was even more accurate when applied to the cleanest ABP wave- lation during routine clinical operations. We employed this forms. Other investigational algorithms were not signiﬁcantly data set to evaluate a set of investigational algorithms, and to superior to MAP as quantitative estimators of CO. establish a public resource for the meaningful comparison of Conclusions: Based on ABP data recorded during routine in- alternative CO-from-ABP algorithms. tensive care unit (ICU) operations, the Liljestrand and Zander

Design: A retrospective comparative analysis of eight method is a better estimator of COTD than MAP alone. Our at- investigational CO-from-ABP algorithms using the Multiparam- tempts to fully replicate commercially-available methods were eter Intelligent Monitoring in Intensive Care II database. unsuccessful, and these methods could not be evaluated. How- Setting: Mixed medical/surgical ICU of a university hospital. ever, the data set is publicly and freely available, and developers Patients: A total of 120 cases. and vendors of CO-from-ABP algorithms are invited to test their Interventions: None. methods using these data. (Crit Care Med 2009; 37:72–80) Measurements: CO estimated by eight investigational CO- KEY WORDS: cardiac output; non-invasive monitoring; pulse con- from-ABP algorithms, and COTD as a reference. tour; database; arterial blood pressure; thermodilution Main Results: All investigational methods were signiﬁcantly

ardiac output (CO) is a cardinal For more than a century (1), the pathology and useful in optimizing thera- parameter of cardiovascular premise that relative changes in CO could pies such as volume resuscitation and cat- state, and a fundamental deter- be estimated by analysis of the arterial echolamine infusions. Today, several com- minant of global oxygen deliv- blood pressure (ABP) waveform has cap- mercially-available methods offer Cery. Historically, routine clinical measure- tured the attention of many investigators. competing algorithms that derive CO from ment of CO has been limited to critically-ill Today, peripheral ABP is routinely available the ABP waveform (4–6). Other algorithms patients, often with invasive indicator- in intensive care unit (ICU) patients, and have been proposed in the medical litera- dilution methods such as thermodilution noninvasive devices exist to measure pe- ture but not incorporated into commercial (COTD). Alternative CO measurement strat- ripheral ABP in noncritically-ill popula- products (7–13). There are major chal- egies have not replaced indicator-dilution tions (2,3). Tracking changes in CO contin- lenges that each of these algorithms must methods in critical care, and outside the uously via ABP waveform analysis, without confront to accurately estimate global intensive care unit, imprecise metrics (e.g., the risks of central catheterization, may be blood flow from a peripheral blood pressure blood pressure, urine output, mental sta- valuable both within and beyond the ICU waveform. For instance, the relationship tus, etc.) are used to assess CO and circu- setting: such a “vital sign” might be a sen- between arterial pressure and volume (i.e., latory adequacy. sitive and specific indicator of circulatory compliance) varies from person to person, and for any given individual, compliance also varies as a nonlinear function of ABP and adrenergic state. Furthermore, the *See also page 337. Imaging and Bioengineering, by support from Philips From the Division of Health Sciences and Technology Medical Systems and Philips Research North America, pressure pulse represents the superposition (JXS), Massachusetts Institute of Technology, Massachusetts and by the Center for the Integration of Medicine and of antegrade waves that drive forward flow General Hospital, Boston, MA; Department of Emergency Innovative Technology. as well as retrograde reflected waves that Medicine (ATR), Division of Health Sciences and Technology, The authors have not disclosed any potential con- retard forward flow. To date, the relative Boston, MA; Division of Health Sciences and Technology flicts of interest. (MS), Massachusetts Institute of Technology, Philips Medical For information regarding this article, E-mail: capability of the various algorithms is diffi- Systems, Boston, MA; Lab for Electromagnetic & Electronic [email protected] cult to ascertain, because published evalu- Systems (TH), Massachusetts Institute of Technology, Bos- Copyright © 2008 by the Society of Critical Care ations are performed in different sets of ton, MA; Division of Health Sciences and Technology (RGM), Medicine and Lippincott Williams & Wilkins patients with different physiologic ranges Massachusetts Institute of Technology, Boston, MA. DOI: 10.1097/CCM.0b013e3181930174 This research was supported by grant R01 and different pathologies, making direct EB001659 from the National Institute of Biomedical comparisons between studies problematic.

72 Crit Care Med 2009 Vol. 37, No. 1 In this investigation, we used a subset able for free public use (http://www.physionet. unreliable predictor of CO. ABP waveform of the Multiparameter Intelligent Moni- org/physiotools/cardiac-output/). analysis assumes that other features in the toring in Intensive Care II (MIMIC II) ABP Signal Processing. Within each waveform are less affected by confounders database (14) containing radial artery minute-long ABP segment, individual heart such as PVR, and are thus more reliable cor- waveform data and contemporaneous ref- beats were identified using a Matlab (Math- relates of CO. MAP serves as our control works, Natick, MA) implementation of an al- method against which eight investigational erence COTD measurements to evaluate gorithm by Zong (17). The waveform quality CO-from-ABP methods are compared. eight CO-from-ABP algorithms previ- of each ABP pulse was assessed automatically (A) Pulse Pressure. In 1904, Erlanger and ously described in the literature. The test using a signal abnormality index (SAI) algo- Hooker suggested that the pulse pressure is a data were collected in an ICU patient pop- rithm (18). A set of features from each ABP surrogate of stroke volume (7). This notion ulation during routine clinical operation pulse was computed, including the peak (sys- naturally arises from a basic Windkessel model tolic blood pressure, [SBP]), trough (diastolic and contain “real-world” (e.g., not artifi- of the arterial tree, in which the arterial sys- blood pressure [DBP]), mean arterial pressure cially pristine) ICU physiologic data that tem is considered a single elastic tank, with (MAP) and pulse pressure (SBP minus DBP); can be fairly applied to any number of flow exiting through a distal resistive element. see Figure 1. Each ABP pulse’s average of disparate CO-from-ABP algorithms, es- Assuming that cardiac ejection is near- negative slopes was computed, providing a tablishing a resource for their meaning- instantaneous, then the product of pulse pres- metric of spiky, nonphysiologic noise in the ful comparison. Furthermore, we are sure and heart rate is a predictor of CO. ABP pulse waveform. After computing the pre- (B) Liljestrand and Zander. The compli- making this data set publicly available to ceding features for an ABP pulse, the SAI al- ance of the arterial tree varies with blood support the evaluation and further devel- gorithm checked that all were within normal opment of CO-from-ABP techniques limits (18). The SAI also checked that the pressure. The Liljestrand algorithm accounts (available at http://www.physionet.org/ features’ variations from one ABP pulse to the for the dependence of arterial compliance on physiotools/cardiac-output/). The MIMIC next were within normal limits. The SAI algo- arterial pressure by scaling its CO estimate to the reciprocal of MAP (8). II CO /ABP data set is in this sense an rithm reported a binary ‘normal’ or ‘abnormal’ TD (C) Systolic Area. A number of methods analog of the public access arrhythmia rating for each ABP pulse, depending if all the normality criteria were met (18). Any abnor- treat the arterial tree as a long viscoelastic databases that have played an indispens- tube, a “transmission line” model. Within a able role in the development, refinement, mal beat was excluded from further analysis. If a given minute-long ABP segment contained transmission line, pressure gradients acceler- and—ultimately—widespread accep- more than 40% of abnormal beats, the entire ate or decelerate flow. By assuming that ret- tance of automated algorithms for elec- rograde (reflected) pressure waves are negligi- segment (and its corresponding COTD) was trocardiogram analysis (15). excluded from further analysis. ble during systole, it is possible to estimate the In addition, we estimated the duration of pressure gradient and the forward flow from an ABP waveform. Specifically, stroke volume MATERIALS AND METHODS each entire beat and its systolic interval. There is no single widely accepted method to identify is proportional to the area under the systolic the systolic interval in a peripheral ABP pulse portion of the ABP pulse (9, 10). Database Development. Our COTD/ABP data set was extracted from the MIMIC II da- (in contrast to a central ABP pulse, the di- (D) Kouchoukos Correction. A potential tabase (14). The MIMIC II database includes crotic notch—if present—in a peripheral ABP source of error is the assumption that cardiac physiologic and wide-ranging clinical data pulse does not indicate closure of the aortic ejection is so rapid that no blood flows out of from over 2,500 ICU patients (medical ICU, valve). Therefore, we chose two alternative cri- the arterial tree during systole (“run-off”). critical care unit, and surgical ICU) hospital- teria to identify the end of systole. First, we Kouchoukos proposed a simple correction fac- ized at the Beth Israel Deaconess Medical Cen- computed a heuristic estimate of systolic du- tor, related to the ratio of systolic-to-diastolic ter, Boston, USA between 2001 and 2005. Ra- ration, (0.3 ⅐ͱbeat_period), originally sug- duration (11); this was a variation of an earlier dial ABP waveform data from the M1006B gested as an approximation of the QT interval method proposed by Warner (12). (E) Diastolic Decay. Bourgeois developed invasive pressure module and COTD data (tem- (19). Second, we identified the point after SBP porally resolved to the nearest minute) were with the lowest nonnegative slope, as shown an algorithm to quantify systolic run-off (13). originally sourced from Philips CMS bedside in Figure 1. In practice, this method located This method leads to an estimation of PVR patient monitors (Philips Medical Systems, the trough of the dicrotic notch, or any rela- (CO can then computed from MAP/PVR, as- Andover, MA), so both shared the same elec- tive plateau which persisted for two or more suming venous pressure is negligible). Bour- tronic time reference. Waveforms were sam- ABP samples. geois’ method is based on a constant compli- pled at 125 Hz with 8 bit resolution. The ance Windkessel model. In such an idealized patients’ gender and age were input by the Investigational CO-from-ABP model, a mono-exponential diastolic decay nursing staff as part of routine clinical opera- (due to the arterial run-off) is expected in the tions, using the Philips CareVue system, and Algorithms ABP pulse waveform, and that diastolic curve changes solely as a function of PVR these archived data were another component The investigational algorithms are sum- (20). Our diastolic decay method adapts the of the MIMIC II database. Additional details marized in Table 1. Most algorithms predict about the MIMIC II database are available in original Bourgeois method to the radial ABP stroke volume, and CO is taken as the product (14). The data were collected and analyzed (and thus its performance is likely to be of median stroke volume and median heart with institutional approval by the local IRB. degraded significantly due to the loss of the rate over the one-minute window. Many of the We identified and extracted MIMIC II cases dicrotic notch). We fit a monoexponential algorithms were initially intended for a central with COTD measurements and one-minute- curve to just two points of each ABP pulse, long segments of radial ABP waveform that aortic ABP waveform; in this investigation, we taking the peak of systole as the onset of a explored their application to a peripheral ra- immediately preceded the COTD measurement mono-exponential decay, and the trough of dial ABP. (up to and including the time of the COTD diastole as its end (21). measurement). All algorithms used in our MAP is positively but imperfectly corre- (F) Herd. Systolic blood pressure may be analyses were implemented in Matlab (Math- lated with CO. Of course, variable degrees of prone to amplification due to early reflected works, Natick, MA). These algorithms, and the systemic vasoconstriction or dilation (which waves. Herd et al proposed an empirical

MIMIC II COTD/ABP data set, have been con- affect peripheral vascular resistance [PVR]), as method, the difference between mean and di- tributed to PhysioToolkit (16) and are avail- well as variable venous pressure, make MAP an astolic pressure, as a predictor of stroke vol-

Crit Care Med 2009 Vol. 37, No. 1 73 ume that would be less confounded by this 100 effect (22). 80 (G) Corrected Impedance. Wesseling’s Cor- 60 rected Impedance method provides an empir- 40 ical correction to the systolic area-under-the- 0 200 400 600 800 1000 1200 ABP curve approach, to account for some of the sources of error described above (23). 120 (H) AC Power. Reportedly the commercial LiDCO Plus PulseCO method of pulse power 100 analysis (LiDCO Ltd., London, England) makes 80 use of the power of the ABP signal, deriving the 60 “beat power factor (r.m.s.—root mean square) 0 200 400 600 800 1000 1200 which is proportional to the nominal stroke volume ejected into the aorta” (4). This 100 method also entails additional processing 80 steps that are not comprehensively described; 60 therefore we could not independently replicate their methodology. Rather we assessed how 40 well the stroke volume could be estimated 0 200 400 600 800 1000 1200 with just the computed ABP waveform root- 140 mean-square, which is one component of the 120 LiDCO Plus PulseCO method. 100 We attempted to evaluate two additional 80 methods that are distributed commercially, 60 Modelﬂow (Finapres Medical Systems, 40 Amsterdam, The Netherlands) and PiCCO 0 200 400 600 800 1000 1200 (PULSION Medical Systems, Munich, Ger- 100 many), by developing new software routines that were consistent with published details 80 about these algorithms. However, the results 60 of our implementations were unsatisfactory and we decided to exclude these algorithms 40 from our trial. This is considered further in 0 200 400 600 800 1000 1200 the Discussion section. Figure 1. Five examples of arterial blood pressure waveforms and their key features as identiﬁed by our automated algorithms. The horizontal axis is sample number, with 125 samples ϭ one second. Onset point ⅐ ͱ of each beat is indicated by an asterisk “*”; end of systole, estimated by 0.3 beat_period, is indicated by Calibration “X”; end of systole, estimated by the ‘lowest nonnegative slope’ method, is indicated by a “0”. We applied the investigational algorithms described above (and summarized in Table 1) to the data of subjects who had at least two

paired measurements of COTD and a contemporaneous, minute-long segment of ABP waveform of sufficient quality. Each algorithm Table 1. Investigational CO-from-ABP algorithms was calibrated to each patient, using two dif- (CO ϭ Stroke Volume ϫ HR) ferent methods. First, the “best-possible calibration factor” was computed, C1 (Fig. 2). C1 Pulse pressure (7) Stroke volume ϭ k ϫ ͑SBP Ϫ DBP͒ was selected to minimize the root-mean- k ϫ ͑SBP Ϫ DBP͒ square of the differences of each pairing of Liljestrand (8) Stroke volume ϭ ͑SBP ϩ DBP͒ CO and CO-from-ABP. It provides the best Systolic area (9, 10) Stroke volume ϭ ϫ ͵ ͑ ͒ TD k Systole ABP t dt accuracy that could be obtained with a single Systolic area with Kouchoukos Stroke volume ϭ calibration factor. Next, each algorithm was Duration correction (11) ϫ ͩ ϩ Systole ͪ ϫ ͵ ͑ ͒ calibrated to each patient using a different k 1 Systole ABP t dt DurationDiastole methodology, C2 (Fig. 2). C2 was calculated a Diastolic decay Solves for beat-to-beat peripheral vascular resistance, fitting only from the first pairing of the CO estimate a monoexponential curve to each ABP pulse’s peak of and COTD. C2 describes an algorithm’s lower systole and trough of diastole, where: limits of performance, if caregivers rely on just P ϭ P ϫ eϪk ϫ time/PVR Diastolic Systole one initial pairing of CO and CO-from-ABP Herd (22) Stroke volume ϭ k ϫ ͑MAP Ϫ DBP͒ TD to calibrate the algorithm. All investigational Corrected impedance (23) Stroke volume ϭ algorithms were compared against MAP as k ϫ ͑163 ϩ HR Ϫ 0.48 ⅐ MAP͒ ϫ ͵ ABP͑t͒dt Systole predictors of CO. MAP was calibrated exactly 1 AC power (root-mean-square) Stroke volume ϭ k ϫ ͱ ͵͑ABP͑t͒ Ϫ MAP͒2dt like other algorithms, i.e., C1 and C2. T T where: T ϭ duration of heart beat Statistical Analysis aAdopted from Circ Res 1976; 39:15–24.

CO, cardiac output; ABP, arterial blood pressure; HR, heart rate; SBP, systolic blood pressure; DBP, Each paired CO-from-ABP and COTD had diastolic blood pressure; MAP, mean arterial pressure; AC, alternating current. an identiﬁable “error” (their difference). The

74 Crit Care Med 2009 Vol. 37, No. 1 7 ments per patient. On average, each sub- Ϯ 6.5 jects’ COTD varied by 46%, PVR varied by Ϯ50%, and MAP varied by Ϯ32%. 6 We excluded 13.7% of the available 5.5 minute-long ABP data segments (and

5 their paired COTD measurements) which did not pass our data quality criteria CO [L/min] CO 4.5 (those segments contained more than 4 40% of abnormal ABP pulses). Table 3, 3.5 tabulates the 95% limits-of-agreement for eight of our investigational algorithms 3 0 5 10 15 20 25 30 35 40 45 50 using both the C1 and C2 calibration meth- time [hours] ods. The Liljestrand algorithm performed Figure 2. We studied two methods of calibrating the algorithms for a subject. In C1 the “best-possible the best, and was statistically superior to calibration factor” was computed, minimizing the root-mean-square of the difference of each pairing calibrated MAP by the Kolmogorov-Smir- of cardiac output thermadilution (CO ) (stem plots) and the corresponding CO-from-arterial blood TD nov test. Figure 3 plots the differences (“er- pressure estimations (black line). In C2, the first pairing of the CO-from-arterial blood pressure rors”) between the Liljestrand method and estimate (gray line) and COTD was used to establish the calibration factor, which was then used for all subsequent CO estimation. COTD, and subplots in Figure 3 show error plotted as a function of various physiologic Table 2. Characteristics and physiologic variation of the study population parameters. One notable trend is that the Liljestrand error grew larger as PVR less- Units Mean Ϯ SD N ened. In 79 cases the largest magnitude Study population change in CO was an increase (ranging Ϯ TD Age yr 69 12 120 from ϩ5% to ϩ192%, with an average of Stay duration day 2.3 Ϯ 2.2 120 Ϯ ϩ65%), and in 29 cases the largest mag- COTD measurements per patient 10 8 120 Ϯ COTD range per patient L/min 2.3 1.2 120 nitude change was a decrease (ranging Mean arterial pressure range per patient mm Hg 24 Ϯ 10 120 from Ϫ10% to Ϫ56%, with an average of ⅐ Ϯ Peripheral vascular resistance range per patient mm Hg s/ml 0.5 0.3 120 Ϫ32%). For these largest magnitude Pooled data Ϯ changes, all eight of the reported algo- COTD L/min 5 2 1164 Mean arterial pressure mm Hg 75 Ϯ 10 1164 rithms showed similar frequencies of di- Ϯ Heart rate bpm 88 17 1164 rectional agreement with COTD, and all Peripheral vascular resistance mm Hg⅐s/ml 1 Ϯ 0.4 1164 were significantly different from calibrated MAP. See Table 3. CO , cardiac output thermadilution. TD The results in Table 3 and Figure 3 are exclusive of the minute-long ABP segments with Ͼ40% abnormal beats. After recom- ject and a given algorithm. Grouping all the distribution of errors for each investigational puting the Liljestrand 95% limits-of- algorithm was computed, for both the C1 and r.m.s. values for a given algorithm, we applied a agreement for all data segments (i.e., re- C2 calibration methods. From these error dis- paired Student’s t test versus calibrated MAP. This statistical methodology is quite conserva- gardless of data quality) the Liljestrand tributions, 95% limits-of-agreement were Ϫ ϩ computed for each CO-from-ABP algorithm, tive, treating repeated observations in a given limits-of-agreement grew to 1.88/ 1.57 per Bland-Altman methodology (24). subject as only one datum (one r.m.s. value). L/min. By contrast, applying more strin- We tested if the error distribution for an Finally, we computed the frequency with gent ABP quality criteria (analyzing investigational algorithm was statistically differ- which directional changes (i.e., increase or minute-long ABP waveform segments with ent from the error distribution of “calibrated decrease) of each CO-from-ABP algorithm no more than 5% of abnormal beats), the MAP” (using the C1 data), using the Kolmog- agreed with COTD. To avoid analysis of trivial limits-of-agreement were reduced to orov-Smirnov test (in Matlab). The Kolmogorov- changes in COTD we identified the single larg- Ϫ1.48/ϩ1.29 L/min, although this more Smirnov test can detect if one error distribution est magnitude percent change in each patient stringent ABP quality criteria excluded has a significantly wider limits-of-agreement. record (increase or decrease) and computed 40% of the available ABP data segments. We did not apply statistical methods such as the the incidence of algorithm/COTD directional Some of the algorithms required de- Student’s t test, repeated measures analysis of agreement. We tested if each algorithm was termining the end-of-systole in a radial significantly different from the performance of variance, or mixed-effects models, because these ABP pulse. The results in Table 3 were “calibrated MAP” using McNemar’s test. assess if the means of the distributions are dif- all based on our heuristic method ferent. Because we calibrated the investigational 0.3 ⅐ͱbeat_period. For select algo- methods to the reference method, the means of RESULTS these error distributions were expected to be rithms, superscripts in Table 3 report near-zero, i.e., minimal measurement biases, so Table 2 shows the characteristics of the the results for an alternative method of these tests would be nonsignificant. 120 subjects analyzed. Typical of an ICU identifying the end-of-systole (the “low- For each subject, we also computed the root- population, the subjects were older (age 69 est nonnegative slope” method), which Ϯ mean-square of the CO-from-ABP versus COTD yrs 12 SD), 67% were male. The average trended toward worse results. errors. This yielded a number that measures the length of stay in the ICU was slightly over 2 Grouping all the subjects’ r.m.s. error width of the error distribution, for a given sub- days with an average of ten COTD measure- values for each algorithm and for cali-

Crit Care Med 2009 Vol. 37, No. 1 75 Table 3. Agreement between thermodilution CO and investigational CO-from-ABP algorithms

“First Pairing “Best Possible Single Calibration” KS Test vs. Mean Calibration” (C2) 95% (C1) 95% Limits Agreement Arterial Pressure Limits Agreement % Agreement of Directional Ϯ Ϯ ⌬ Investigational Predictors of COTD ( L/min) (p) ( L/min) ’s vs. COTD

Liljestrand Ϫ1.76/ϩ1.41 0.0001 Ϫ2.81/ϩ2.04 78b Corrected impedance Ϫ1.91/ϩ1.57a Ͻ0.01 Ϫ3.39/ϩ2.28 78b Pulse pressure Ϫ2.07/ϩ1.73 Ͼ0.05 Ϫ3.05/ϩ2.76 74b Systolic area Ϫ2.07/ϩ1.73c Ͼ0.05 Ϫ2.85/ϩ3.05 77b Systolic area with Kouchoukos correction Ϫ2.08/ϩ1.71 Ͼ0.05 Ϫ3.20/ϩ2.89 78b AC power (root-mean-square) Ϫ2.09/ϩ1.73 Ͼ0.05 Ϫ3.12/ϩ2.78 79b Diastolic decay Ϫ2.23/ϩ1.77 Ͼ0.05 Ϫ3.22/ϩ2.57 78b Mean arterial pressure Ϫ2.20/ϩ1.82 — Ϫ3.19/ϩ3.42 56 Herd Ϫ2.66/ϩ1.89 Ͼ0.05 Ϫ3.65/ϩ3.16 78b

CO, cardiac output; ABP, arterial blood pressure; COTD, CO thermadilution; KS, Kolmogorov-Smirnov; AC, alternating current.

(one r.m.s. value). We found that the Liljestrand method was again signiﬁ- cantly better than calibrated MAP (p Ͻ 0.01). The Herd method yielded p ϭ 0.04. The other p values were greater than 0.05.

DISCUSSION In 1983, Wesseling pointed out that the ability to monitor CO continuously by analyzing the ABP waveform could pro- vide “an early warning signal if cardiac output would rise or fall suddenly, to adjust drug rates and infusion rates ͓...͔ to sense bleeding ͓...͔ to get a true mean cardiac output under arrhythmias, etc (23).” Yet the optimal method and clinical applicability of ABP waveform analysis have not been deﬁnitively identiﬁed. Per- haps this is in part because these algorithms haven’t been adequately evaluated in a comparative manner (25). In this investigation, we found that all eight of the investigational algorithms were superior to MAP as directional, qualitative indicators of major changes in

COTD. Although the methods offer similar information about major directional

changes in COTD (i.e., about 78%), they can differ drastically in magnitude. Only the Liljestrand method was a superior quantitative predictor of CO than calibrated MAP, which is essential for mean- Figure 3. A, Bland-Altman plot comparing cardiac output (CO) estimated by the algorithm (Liljestrand, ingful interpretation of directional using the C1 calibration methodology) with CO thermadilution (COTD). 95% limits-of-agreement for changes. The Liljestrand predictor may this algorithm and the other investigational algorithm are summarized in Table 3. B, Liljestrand be a useful parameter for intelligent algorithm estimation error as a function of several variables. Bins of equal sample sizes are illustrated. monitoring algorithms when a patient’s Rectangular bars represent 95% limits-of-agreement for each bin. For example, as shown, CO radial ABP is measured, providing more estimation error decreases as peripheral vascular resistance (PVR) increases. MAP, mean arterial pressure; HR, heart rate. information than MAP alone. As quantitative predictors of CO, the other investigational algorithms failed to surpass calibrated MAP, we compared the algo- This statistical methodology was more brated MAP, including several algorithms rithms’ distributions versus calibrated conservative, treating repeated observa- that we developed which incorporated pub- MAP, using the paired Student’s t test. tions in a given subject as only one datum lically-disclosed aspects of the Modelﬂow,

76 Crit Care Med 2009 Vol. 37, No. 1 innovative methods by caregivers, partic- ularly when some methods are proprietary and not fully disclosed to the public. The usefulness of ancillary algorithms for CO estimation (e.g., generalized transfer functions to estimate central aortic pressure, or ABP dampening detectors) can likewise be tested.

Because this COTD/ABP data set contains “real-world” ICU data, collected during routine operations, the reference

CO measurements were single COTD de- terminations. This reflects our ICU’s clinical practice, even though it is an imperfect CO reference method (26,27). As for quality control of the ABP signals, the ICU protocol calls for rezeroing and the flush test at least once per shift, although there was no explicit mechanism for us to Figure 4. Example of continuous cardiac output (CO)-from-arterial blood pressure (ABP) estimated by assess protocol compliance. Despite these the Liljestrand algorithm (gray line) versus episodic thermodilution CO measurements (stem plots) limitations, however, this large data set for a subject over a 50-hr time interval, using a single calibration factor (C1, see text for details). Pulse offers a fair basis for the relative compar- pressure (PP), mean arterial pressure (MAP) and heart rate (HR) through this same temporal window, isons of different CO-from-ABP algo- as computed by our algorithm, are also shown (gray lines) with stem plots illustrating their values rithms. Even if there are random errors each time CO thermadilution (CO ) was measured. C1 calibration minimizes the root-mean-square TD in some of the COTD measurements and of the difference of each pairing of COTD and the corresponding CO-from-ABP estimations. C2 random artifacts in some ABP waveforms, calibration uses the first pairing of the CO-from-ABP estimate and COTD only. For each investigational a superior algorithm should, on average, algorithm, the distribution of errors versus CO was compared with the distribution of errors of TD prove to be a better predictor of CO ‘calibrated MAP’ using the Kolmogorov-Smirnov test. *Significantly different from calibrated MAP TD than an inferior CO-from-ABP algorithm, (p Ͻ 0.001) per McNemar’s test. #When using the alternative “lowest nonnegative slope” method to estimate the systolic interval, the lower/upper limits are Ϫ1.94/ϩ1.54 L/min. Results in Table 3 given a data set of this size. (Such sources employ 0.3 ⅐ͱbeat_period to estimate systolic interval. ##When using the alternative “lowest nonnega- of error tend to be carefully minimized tive slope” method to estimate the systolic interval, the lower/upper limits are Ϫ2.40/ϩ1.97 L/min. during controlled clinical trials, and so such trials may or may not be applicable to routine clinical conditions). Arguably, PulseCO, and PiCCO methodologies. We and report their results. This database the fact that these are “real-world” data conclude that it is difficult to satisfactorily contains a large number of radial ABP (e.g., real-world incidence of improper implement proprietary methods. As we waveforms and paired measurements of transducer zeroing, damping, etc.) en- were unable to independently evaluate COTD (over 1000 paired data points from hances the validity of such relative com- these proprietary algorithms, it under- over 100 patients), archived during rou- parisons, since these measurements re- scores the need for a fair method for pub- tine clinical operations. We observe that flect the actual conditions under which licly comparing competing CO-from-ABP the typical record shows distinct intervals any useful algorithm must operate. On methodologies. of relative stability and other intervals of the other hand, this data set is problem- Such evaluation could be enabled by dynamic physiologic change, as in the atic for evaluating the absolute accuracy one or more publicly available testing da- example in Figure 4. The range of phys- of a given algorithm; the use of single tabases. In the 1970s, this laboratory iologic states in the overall database is COTD measurement, an imperfect refer- made the Beth Israel Hospital-Massachu- summarized in Table 2. The data quality ence method, will widen the overall lim- setts Institute of Technology Arrhythmia in this data set—motion artifacts, inci- its-of-agreement, because the investiga- database publicly available (15). That da- dence of dampened catheters, etc.—is tional algorithm and the CO reference tabase, together with other public access consistent with routine practice, rather will differ due to error in both the CO databases, e.g., the European ST-T data- than idealized research conditions. To algorithm and in COTD. base, promoted the development of auto- our knowledge, there is presently no If and when future investigators col- mated electrocardiogram (ECG) interpre- comparable public database. The perfor- lect additional data sets (perhaps using tation algorithms. Thirty years later, mance of a novel algorithm, including alternative CO reference methods as in computerized ECG arrhythmia analysis breakdown conditions or generally unsat- (28), or ABP measured from other ana- has evolved so that it is now standard in isfactory performances, can be identified tomical locations such as the femoral bedside monitors and even automated de- using this testing database. Furthermore, artery), these data sets can also be fibrillators. direct comparisons of different algo- freely posted on PhysioNet for public The Database. Academic and commer- rithms using a common testing database access, permitting further standardized cial developers can freely access and should breed healthy competition, and comparisons of different CO estimators. download the MIMIC II COTD/ABP data promote clear, iterative improvements. We suggest that it is beneficial to make set (www.physionet.org/physiotools/ Finally, credible evaluation using a stan- available the largest volume of data for cardiac-output/), apply their algorithms, dard database may encourage adoption of the widest range of populations and

Crit Care Med 2009 Vol. 37, No. 1 77 physiologic states, rather than ideal- numerator). If an automated algorithm for public inspection, to bring unreliable ized, smaller data sets. were able to detect or exclude slightly methods to light and accelerate accep- Investigational Algorithms. We report dampened ABP waveforms, or if the clinical tance of rigorous algorithms. However, results from eight CO-from-ABP algo- staff took special care to avoid dampened this is simply not standard practice for rithms, many of which were originally intra-arterial measurements, it is likely that commercial biomedical algorithms. Be- intended for use with a central ABP wave- the Liljestrand method, or indeed any of cause most commercial algorithms will form. One finding in this trial was the the investigational methods in Table 1, remain proverbial “black-boxes” to the superiority of the CO-from-ABP algo- would prove even more accurate. user community, and because it is chal- rithm described in 1928 by Liljestrand Presently, the best known CO-from- lenging to evaluate these algorithms in- and Zander, which is a modestly but sig- ABP algorithms are the commercially- dependently, publicly and freely available nificantly superior estimator of COTD available methods, such as PiCCO, Mod- testing databases are all the more essen- than MAP alone (95% limits-of-agree- elflow, and LiDCO Plus PulseCO (4–6), tial. Indeed, testing proprietary ECG ar- ment with COTD are 0.85 L/min smaller). which are procedurally more complex rhythmia detection algorithms using To the extent that COTD is a useful pa- than the simple formulas in Table 1. We standard testing databases is a mandatory rameter to monitor, the Liljestrand algo- attempted to implement algorithms step in obtaining United States Food and rithm may enhance standard vital signs. based on limited published descriptions Drug Administration approval (15). The Also, compared with MAP, the Liljestrand of the commercial systems. We examined clinical community might want to de- method offers additional (although im- the pulse power (r.m.s.), which is report- mand similar comparative evaluation of perfect) directional information about edly one component of the LiDCO Plus other types of diagnostic algorithms, major changes in COTD, 78% versus 56%, PulseCO method (4); the results are pro- such as CO-from-ABP algorithms, before respectively. Many of the other algo- vided in Table 3. In addition, we devel- relying on these methods for patient care. rithms offered very similar directional oped operational algorithms that were The MIMIC II COTD/ABP data set is now agreement with COTD, although failed to consistent with certain published details publicly and freely available as a first step track COTD changes quantitatively (rela- of the Modelflow and PiCCO methods. To toward providing such a database. We in- tive to MAP). We speculate that one rea- test the Modelflow method, we developed vite developers and vendors of CO-from- son why the algorithms do not exceed a nonlinear three element model consis- ABP algorithms to test their methods and 79% agreement with these major tent with (6). To test the PiCCO method, report their performance on the MIMIC II changes in COTD is because of noisy or we implemented an algorithm that uses COTD/ABP data set. Our best performing artifactual measurements in either ABP mathematical formulas reported in (29), algorithm, the Liljestrand method, may or COTD, which skew these results. The to estimate arterial compliance as a non- be a useful basis for comparison. former might be addressed in the future linear function of ABP and a calibration The modest performance of most of with improved preprocessing of the ABP factor, and subsequently, to estimate CO the investigational algorithms requires waveform. One exemplary case of CO es- as a function of ABP and arterial compli- discussion. As noted in the Introduction, timated continuously by the Liljestrand ance. The references described certain it is a theoretical challenge to rigorously algorithm versus COTD measured episod- details about the methods, but not nu- estimate global blood flow from a periph- ically is illustrated in Figure 4. merous additional details that are neces- eral pressure measurement, because of The performance of the Liljestrand sary for functional signal processing soft- arterial compliance changes, superposi- method is notable because the ABP data ware. Because these commercial methods tion of antegrade and retrograde pressure were collected during routine ICU clinical are not “open source” we had to rely waves, and other factors. In this investi- operations, during which some degree of extensively on our own software engi- gation, furthermore, additional factors motion artifact, catheter damping, improp- neering judgments to produce functional may have contributed to the poor perfor- erly calibrated transducers, etc. are inevita- data processing software, consistent with mances, including: use of real-world ABP ble. We employed lenient ABP quality cri- what we thought the commercial prod- waveform data rather than pristine re- teria (analyzing all data with Յ40% ucts might do. The performance of our search data (discussed above); use of ra- abnormal ABP pulses), which only excluded algorithms, however, was not signifi- dial ABP waveforms rather than central 13.7% of the noisiest minute-long ABP cantly better than calibrated MAP in ABP waveforms; one-time calibration waveform segments. We found that the terms of both 95% limits-of-agreement, rather than repeated recalibration; and 95% limits-of-agreement between the and frequency of directional agreement inclusion of all subjects regardless of car-

Liljestrand CO estimates and COTD are a with COTD for major changes in COTD.We diac valve function. function of ABP waveform quality: as the therefore chose to exclude these algo- Most of the investigational algorithms ABP quality criteria are made increasingly rithms from our study, in case they were originally intended for use with a stringent, the limits-of-agreement grow might be misconstrued as relating to the central ABP waveform, where the systolic tighter, although more data are excluded. actual commercial algorithms, rather interval of the ABP may have relatively Note that we did not explicitly exclude than our own unsuccessful software de- fewer retrograde components (i.e., re- dampened ABP waveforms, aside from re- velopment efforts. Our software imple- ﬂected waves). The PiCCO method has quiring that the pulse pressure was Ͼ20 mentations are available for review on the been applied primarily to femoral ABP mm Hg. Waveform damping (due to air Physionet website. We conclude that it is (e.g., 5). Yet measurement of a radial ABP bubbles, thrombus, partial lumen occlu- very difﬁcult to satisfactorily implement is a common clinical practice, and algo- sion, etc.) can subtly reduce the measured proprietary methods and independently rithms which perform suitably using a pulse pressure and so is a potentially seri- evaluate their capabilities. radial ABP may prove more valuable be- ous source of error for the Liljestrand algo- Ideally, vendors would make the cause radial ABP is more often available, rithm, which contains pulse pressure in its source code for their methods available and because noninvasive devices exist to

78 Crit Care Med 2009 Vol. 37, No. 1 measure distal extremity ABP. Therefore this subset of 36 subjects with docu- Colin radial artery tonometer measure the we feel that investigation of these algo- mented normal valve function. same blood pressure changes following defla- rithms applied to a radial ABP is war- tion of thigh cuffs? Physiol Meas 2003; 24(3): 653–660 ranted. There is precedent for applying an CONCLUSION algorithm intended for a central ABP on a 4. Rhodes A, Sunderland R. Arterial pulse All of the investigational algorithms power analysis: the LiDCO plus System. In: peripheral BP (6,28). Indeed, we discov- Pinsky MR, Payen D, eds. Functional Hemo- ered that the Liljestrand method per- were superior to MAP as directional, dynamic Monitoring. Berlin: Springer- forms well when applied to radial ABP. qualitative indicators of major changes in Verlag, 2005 Many pulse contour methods pre- COTD. However, only the Liljestrand 5. Godje O, Hoke K, Goetz AE, et al. Reliability scribe recalibration after each new ref- method was superior to calibrated MAP as of a new algorithm for continuous cardiac erence CO measurement, although we a quantitative predictor of CO (which is output determination by pulse-contour anal- did not use this methodology in our essential for meaningful interpretation of ysis during hemodynamic instability. Crit investigation. More frequent calibra- directional changes). The Liljestrand pre- Care Med 2002; 30(1):52–58 tions will certainly improve accuracy, dictor may be a useful parameter for intel- 6. Wesseling KH, Jansen JR, Settels JJ, et al. not only because this accounts for dy- ligent monitoring algorithms when a pa- Computation of aortic flow from pressure in humans using a nonlinear, three-element namic changes in arterial compliance, tient’s radial ABP is measured, providing model. J Appl Physiol 1993; 74(5):2566–2573 more information than MAP alone. In this but because each COTD is, on average, a 7. Erlanger J, Hooker DR: An experimental good predictor of the subsequent COTD. study, we developed complex algorithms study of blood pressure and of pulse-pressure Yet this does not reveal the key ques- that incorporated publicly-disclosed details in man. Johns Hopkins Hosp Rep 1904; (12): tion: when the patient’s cardiovascular of two commercial CO-from-ABP methods 145–378 state is changing but COTD is un- but their performances were unsatisfactory 8. Liljestrand G, Zander E: Vergleichen die known—and this is precisely when con- and we excluded these methods from study. bestimmungen des minutenvolumens des tinuous CO-from-ABP could prove use- We conclude that it is very difficult to sat- herzens beim menschen mittels der sti- ful—how accurate is the algorithm? isfactorily implement proprietary methods choxydulmethode und durch blutdruckmes- sung. Ztschr ges exper med 1928; (59): Taking a new measurement of COTD (for and independently evaluate their capabili- recalibration) defeats the need for CO ties. Since our testing data set is publicly 105–122 and freely available, investigators and 9. Verdouw PD, Beaune J, Roelandt J, et al. to be independently estimated via wave- Stroke volume from central aortic pressure? form analysis. Thus, in this study, we vendors of CO-from-ABP algorithms are A critical assessment of the various formulae compared how well each algorithm can invited to test their methods using these as to their clinical value. Basic Res Cardiol estimate CO using just a single calibra- data, offering a fair basis for comparison 1975; 70(4):377–389 tion factor. The calibration methods in of different CO-from-ABP algorithms un- 10. Jones WB, Hefner LL, Bancroft WH, Jr., et al. this study included C1, the “best possi- der real-world clinical conditions. The Velocity of blood flow and stroke volume ble calibration” (a retrospective con- clinical community might expect vendors obtained from the pressure pulse. J Clin In- struct, in which one optimal calibration to publically report how well their algo- vest 1959; 38:2087–2090 factor that minimizes the overall esti- rithms perform in public testing data- 11. Kouchoukos NT, Sheppard LC, McDonald mation error is employed); and C2,in bases before relying on the algorithms for DA. Estimation of stroke volume in the dog by a pulse contour method. Circ Res 1970; which only the first pairing of CO-from- patient care, and the MIMIC II CO /ABP TD 26(5):611–623 data set is a first step toward such a ABP and COTD are employed for calibra- 12. Warner HR, Swan HJ, Connolly DC, et al. tion, and subsequent pairings are exam- public resource. Quantitation of beat-to-beat changes in ined for accuracy. Presumably, real- stroke volume from the aortic pulse con- world performances (making some use ACKNOWLEDGMENTS tour in man. J Appl Physiol 1953; 5(9): of recalibration) will lie somewhere be- 495–507 tween the ideal of the C1 calibration This research was supported by 13. Bourgeois MJ, Gilbert BK, Von Bernuth G, method and the imprecision of the C2 grant R01 EB001659 from the National et al. Continuous determination of beat to method. Institute of Biomedical Imaging and beat stroke volume from aortic pressure We did not exclude subjects based on Bioengineering, by Philips Medical Sys- pulses in the dog. Circ Res 1976; 39(1): 15–24 heart valve function. Rather, we trusted tems and Philips Research North Amer- ica, and by the Center for the Integra- 14. Saeed M, Lieu C, Raber G, et al. MIMIC II: that the ICU staff would only measure a massive temporal ICU patient database to tion of Medicine and Innovative COTD in appropriate patients (e.g., with- support research in intelligent patient out significant tricuspid regurgitation), Technology. monitoring. Comput Cardiol 2002; 29: and that the ideal CO-from-ABP algo- 641–644 rithm would tolerate some aortic valve REFERENCES 15. Moody GB, Mark RG. The impact of the dysfunction. When we studied the subset MIT-BIH arrhythmia database. IEEE Eng of cases with documented normal tricus- 1. Osborn JJ, Beaumont J, De Lanerolle P. The Med Biol Mag 2001; 20(3):45–50 pid and aortic valve function, we did not measurement of relative stroke volume from 16. Goldberger AL, Amaral LA, Glass L, et al. find improvement in any of the algo- aortic pulse contour or pulse pressure. Vasc PhysioBank, PhysioToolkit, and PhysioNet: rithms’ performances. Echocardiograms Dis 1968; 5:165–177 components of a new research resource for 2. Schutte AE, Huisman HW, Van Rooyen JM, complex physiologic signals. Circulation were available in 56 subjects, and normal et al. Sensitivity of the Finometer device in 2000; 101(23):E215–220 cardiac valve function was found in 64% detecting acute and medium-term changes 17. Zong W, Heldt T, Moody GB, et al. An open- of them (36 subjects). For all eight inves- in cardiovascular function. Blood Press source algorithm to detect onset of arterial tigational algorithms, the 95% limits-of- Monit 2003; 8(5):195–201 blood pressure pulses. Comput Cardiol 2003; agreement with COTD were no better in 3. Birch AA, Morris SL. Do the Finapres and 30:259–262

Crit Care Med 2009 Vol. 37, No. 1 79 18. Sun JX, Reisner AT, Mark RG: A signal 22. Herd JA, Leclair NR, Simon W. Arterial pres- Intensive Care Med 1995; 21(8):691– abnormality index for arterial blood pressure pulse contours during hemorrhage in 697 sure waveforms. In: Computers in Cardiol- anesthetized dogs. J Appl Physiol 1966; 27. Nilsson LB, Nilsson JC, Skovgaard LT, et al. ogy: 2006; Valencia, Spain; 2006. (avail- 21(6):1864–1868 Thermodilution cardiac output–are three in- able at: http://cinc.mit.edu/archives/2006/ 23. Wesseling KH, de Wit B, Weber JAP, et al. A jections enough? Acta Anaesthesiol Scand pdf/0013.pdf) simple device for the continuous measure- 2004; 48(10):1322–1327 19. Bazett HC: An analysis of the time relations ment of cardiac output. Adv Cardiovasc Phys 28. Jellema WT, Wesseling KH, Groeneveld AB, of electrocardiograms. Heart 1920; (7): 1983; 5(2):16–52 et al: Continuous cardiac output in septic 353–370 24. Bland JM, Altman DG: Comparing two shock by simulating a model of the aortic 20. Cardiovascular Mechanics ͓http://ocw.mit. methods of clinical measurement: a per- input impedance: a comparison with bolus edu/OcwWeb/Health-Sciences-and-Technology/ sonal history. Int J Epidemiol 1995; 24 injection thermodilution. Anesthesiology HST-542JSpring-2004/Readings/͔ Suppl 1:S7–14 1999; 90(5):1317–1328. 21. Sun JX: Cardiac output estimation using ar- 25. Heldt T. Continuous blood pressure-derived 29. Joeken S, Fahle M, Pfeiffer UJ, inventors; terial blood pressure waveforms. Masters cardiac output monitoring–should we be PULSION Medical Systems AG, assignee. De- Thesis. Cambridge: Massachusetts Institute thinking long term? J Appl Physiol 2006; vices for in-vivo determination of the com- of Technology; 2006 ͓available at: http:// 101(2):373–374 pliance function and the systemic blood ﬂow www.physionet.org/physiotools/cardiac- 26. Jansen JR. The thermodilution method for of a living being. United States patent US output/doc/CO-from-ABP.pdf͔. the clinical assessment of cardiac output. 6315735. 2001 Nov 13

80 Crit Care Med 2009 Vol. 37, No. 1