1

2 Electronic Supplementary Material

3 This Electronic Supplimentary Material has been provided by the authors to give readers 4 additional information about the technical details of their work.

5 Supplement to:

6 Elliott JT, Wirth DJ, Davis SC, et al. Seeing the light: Improving the usability of fluorescence- 7 guided surgery by adding an optimized secondary light source.

8

9

10 11 Optimizing the Secondary Illuminant Filters and Intensity

12 Hyperspectral Imaging Instrumentation and Light Source Adapter

13 An in-house hyperspectral imaging system (HIS) enabled acquisition of spectrally resolved

14 images during reflectance and fluorescence illumination of the surgical field. The detection module

15 consisted of an achromatic lens that focused light transmitted from the accessory port, through a liquid

16 crystal tunable filter (LCTF; VariSpec VIS 450-750, Perkin-Elmer, Waltham, MA) with 7-nm bandwidth

17 onto a CMOS camera (Edge 4.2, PCO, Kelheim, Germany). Hyperspectral imaging data were acquired

18 every 100 ms while the LCTF was swept across wavelengths at 5 nm intervals to characterize the

19 spectrum within the 450-720 nm range, generating data (every 5 minutes) with dimensions 540 px x 640

20 px x 55 wavelengths. Simultaneously, the Zeiss camera acquired images and video during the reflectance

21 and fluorescence mode acquisitions..

22 Figure ESM1 shows the adapter that was used to mount a secondary illuminant onto the Pentero.

23

24 Figure ESM1. Bottom and side views of the secondary illuminant adapter, which attaches to the Zeiss OPMI via its 25 dove-tail plate adapter (not shown). An arc-shaped light baffle prevented glare from the primary light source. The 26 LED PCB and optics were placed in the adapter and interface via a M8 female receptacle connected to a 5-m long 27 M8 cable.

28 Simulating a Secondary Illuminant with Hyperspectral Imaging Data

29 Our hyperspectral imaging system (HIS) was calibrated using a NIST-traceable calibrated white light

30 source of known spectrum (SL1-CAL VIS-NIR Tungsten Halogen, StellarNet, Inc., Tampa, FL) so that

31 the response of each HIS band was known. The HIS was then used to capture the reflected spectrum of a

32 ColorChecker card illuminated by a standard Zeiss Pentero operating microscope (OPMI) tungsten-

33 halogen (xenon) arc lamp. 34 The HIS dataset comprising of images of the ColorChecker at each wavelength, after being

35 divided by the white-light source spectrum, was multiplied by modified source spectra simulating various

36 LPF and SPF combinations. These results were converted to RGB space by convolving the data with

37 standard RGB observer functions.1 The difference between the RGB values of each simulated

38 illumination and the RGB values obtained from the actual Pentero white-light illumination defined a

39 transformation, T, between these two colorspaces as:

40 CCmeas * T = CCstd (1)

41 where CCsi is the RGB values of ColorChecker squares in secondary illuminant (24 x 3

42 matrix) and CCstd is the RGB values of the ColorChecker squares under the standard Zeiss white light. To

43 simulate the addition of a secondary illuminant during -light illumination, the white-light image, IWL,

44 was converted to secondary illuminant colorspace, multiplied by a weighting scalar, w, and added to the

45 blue-light image. Specifically,

46 ISIM = w (IWL * T) + IBL \ max(w (IWL * T) + IBL ) (2)

47 where ISIM is the simulated, blended image, T is transformation between white-light and secondary-

48 illuminant colorspace, and the denominator readjusts the white point if necessary when R, G, and B

49 channels are saturated. The weighting scalar, w, corresponds to the intensity of the secondary illuminant

50 (from 10 to 100%). An example of simulated fluorescence of the ColorChecker with secondary illuminant

51 is presented in Figure 3.

A C D E F

B G H I J

52 53 Figure ESM2. Simulated fluorescence with secondary illuminant. Left-most images show the illumination of the

54 colorcard with white-light (A) and blue-light (B) only. Intensity of secondary illuminant relative to the area under

55 the curve of the blue light illuminant is varied from 2 to 9 (left to right C through J). LPF with cut-off of 480 nm and

56 SPF with cut-on at 595 were used. 57 Determination of

58 The ColorChecker card is a tool used by photographers to correct the color gamut of a photograph to

59 match the true reflectance spectra of objects in the scene. By taking one picture with the ColorChecker in

60 view, a linear transformation between the ColorChecker squares in the photograph, CCmeas, and a set of

61 standardized RGB values provided by the manufacturer, CCstd,, typically obtained with CIE Standard

62 Illuminant D65 (average midday light), can be defined using Eq. 1. This procedure enables the

63 photographer to preserve across subsequent images, assuming the camera settings and

64 illuminant are constant. Values can be defined for different standards, depending on the desired

65 illuminant. For the purpose of this study, CCstd was determined with the Pentero white-light source

66 following 30 minutes of warm-up time, set to an intensity of 30%.

67 When the ColorChecker is photographed under a non-, the difference in color

68 fidelity can be represented by the average distance, E, between the standard and observed values for

69 each of the 24 ColorChecker squares. Multiple approaches can be used to evaluate this fidelity. The most

70 straightforward calculates the overall accuracy in observed colors by defining the root square mean of

71 distances between measured and expected RGB values for each ColorChecker square. Another approach

72 involves the Color Rendering Index (CRI), which converts mean Euclidean distance between measured

73 and expected u,v values (in CIE 1960 color space) to an index with a perfect score of 100.2,3

74 This formulation is helpful for evaluating surgical lighting because the IEC 60601-2-41 guidelines

75 suggest a CRI of at least 85. Therefore, we chose CRI as an intuitive representation of error in color

76 fidelity obtained by the different lighting conditions evaluated in this paper.

77 Determination of Tumor-to-Background Color Contrast

78 In addition to optimizing the secondary illuminant on the basis of ColorChecker accuracy (evaluated via

79 CRI) and maximizing color rendering, preservation of tumor-to-background color contrast (TBCC) is

80 essential. To evaluate the influence of different combinations of filters and light intensity on these two

81 characteristics (CRI and TBCC), addition of the secondary illuminant was simulated on surgical images

82 acquired previously with the Zeiss Pentero camera in white-light and blue-light modes. ColorChecker HIS

83 datasets were used to simulate secondary illuminants with varying filter cut-on and cut-off wavelengths

84 and intensities, as described in the previous section. For each of these simulations, transformation, T,

85 from Equation 1 was calculated for each simulated HIS dataset, which allowed any RGB image acquired

86 under the standard surgical illuminant to be converted to the RGB space representing the color gamut of

87 the simulated ColorChecker dataset. Pairs of white-light and blue-light images acquired at the same 88 positions during 5-ALA guided tumor resection were obtained from ten patients. Secondary illuminants

89 corresponding to the LPF, SPF and intensity values identified through the ColorChecker fidelity analysis

90 were simulated by multiplying the white-light surgical image with the corresponding transformation and

91 then adding to the blue-light surgical image proportionally, according to intensity. From these simulated

92 images, TBCC and BFS were calculated and compared with values obtained from the blue-light and

93 white-light images, respectively.

94 Tumor-to-background color contrast was defined as the mean Euclidean distance in LAB

95 colorspace4 between pixels in the tumor region-of-interest and pixels in the surrounding cranial window.

96 Regions were defined manually based on the blue-light images. LAB colorspace was used because it is

97 designed to approximate human vision, and therefore, relative perceptual distance between any two colors

98 corresponds to the Euclidean distance between their points in Lab 3-dimensional space.5

99 Figure ESM3 is a condensed summary of the effect of short-pass filter and long-pass filter

100 selection on the color rendering index and the Tumor to Background Color Contrast for a fixed Intensity.

101 The crosses indicate the filter selection for the secondary illuminant that was ultimately built and used in

102 the intraoperative video and image panels in Figure 3.. 103 Effect of short-pass filter on CRI: 104 Effect of long-pass filter on CRI: Little impact is observed by reducing the LPF cut-off to less than 450 nm since this region is 105 well-covered by blue illumination, and not well-represented in the D65 standard to which CRI is compared. 106 Effect of short-pass filter on TBCC 107 Effect of long-pass filter on TBCC: 108

109 110 Figure ESM3. Relationship (left) between Color Rendering Index (CRI) and filter selection. Warm colors represent CRI > 30. 111 The white cross represents the filter set used in the clinical study. Relationship (right) between change in tumor to background 112 color contrast (TBCC) [compared with the unaltered blue light images] and filter selection. 113 References

114 1. Süsstrunk, S., Buckley, R. & Swen, S. Standard RGB color spaces. in Color and Imaging Conference

115 vol. 1999 127–134 (Society for Imaging Science and Technology, 1999).

116 2. Schanda, J. : understanding the CIE system. (John Wiley & Sons, 2007).

117 3. Wyszecki, G. & Stiles, W. S. Color science. vol. 8 (Wiley New York, 1982).

118 4. Commission Internationale De L’Eclairage. Colorimetry: Technical Report. CIE 15.3:2004.

119 5. Jain, A. K. Fundamentals of Digital Image Processing. (Prentice-Hall, Inc., 1989).

120

121