Research and applications Imaging informatics for consumer health: towards a radiology patient portal Corey W Arnold,1 Mary McNamara,1 Suzie El-Saden,2 Shawn Chen,1 Ricky K Taira,1 Alex A T Bui1

1Medical Imaging Informatics, ABSTRACT look up health information online verified it with Department of Radiological Objective With the increased routine use of advanced their physicians.7 Sciences, University of fi California–Los Angeles, imaging in clinical diagnosis and treatment, it has Several bene ts of tailored information within Los Angeles, California, USA become imperative to provide patients with a means to patient portal applications have been demon- – 2Department of Imaging view and understand their imaging studies. We illustrate strated,8 10 including equipping patients with Services, Greater Los Angeles, the feasibility of a patient portal that automatically vetted, higher quality information regarding their VA Medical Center, Los structures and integrates radiology reports with disease or condition; and facilitating access to their Angeles, California, USA corresponding imaging studies according to several underlying medical records. However, little work Correspondence to information orientations tailored for the layperson. has been done to make the full range of radiology Dr Corey W Arnold, Medical Methods The imaging patient portal is composed of content—imaging and text—available to patients in Imaging Informatics, an image processing module for the creation of a an understandable format. This lack is in spite of Department of Radiological Sciences, University of timeline that illustrates the progression of disease, a the fact that radiology reports and images consti- California–Los Angeles, 924 natural language processing module to extract salient tute a significant amount of the evidence used in Westwood Blvd Ste 420, Los concepts from radiology reports (73% accuracy, F1 score diagnosis and treatment assessment. Even though Angeles, CA 90024, USA; of 0.67), and an interactive user interface navigable by radiology test results are one of the most difficult [email protected] an imaging findings list. The portal was developed as a portions of the clinical record for lay people to 11 Received 1 November 2012 Java-based web application and is demonstrated for understand, they are one of the most frequently Revised 9 May 2013 patients with brain cancer. accessed pieces of information via patient portals Accepted 15 May 2013 Results and discussion The system was exhibited at when available.12 This suggests the need for new Published Online First an international radiology conference to solicit feedback methods of sharing radiology information with 5 June 2013 from a diverse group of healthcare professionals. There patients. was wide support for educating patients about their One possibility for bridging consumers’ under- imaging studies, and an appreciation for the informatics standing of illness with professional disease models tools used to simplify images and reports for consumer is the use of an ‘interpretive layer’ between interpretation. Primary concerns included the possibility clinically-generated information and consumer- of patients misunderstanding their results, as well as centric disease explanations. Such a layer would worries regarding accidental improper disclosure of potentially enable lay patients to construct more medical information. accurate mental models of health, form effective Conclusions Radiologic imaging composes a search queries, navigate medical information significant amount of the evidence used to make systems, understand the information found within diagnostic and treatment decisions, yet there are few health documents, and apply the information to tools for explaining this information to patients. The their personal situations appropriately.13 In this proposed radiology patient portal provides a framework work we utilize the concept of interpretive layers, for organizing radiologic results into several information and describe a methodology for automatically com- orientations to support patient . bining radiology data with educational information for the patient, presented through a web-accessible portal.

INTRODUCTION BACKGROUND AND SIGNIFICANCE The number of patients accessing health informa- Towards satisfying patients’ wishes for access to tion online continues to rise,1 and being diagnosed records and better knowledge resources, govern- with cancer has been shown to increase the amount ment policy has been developed to provide incen- – of time an individual searches for information.2 4 tives for institutions utilizing patient portals in However, the popularity of a website is not always order to promote usage.14 The US Department of indicative of its quality.5 The dearth of quality Health and Human Services believes that such material online is reflected in the Health portals will not only increase patient access to Information National Trends Survey (HINTS), information, but allow patients to become more which found that Americans feel that online cancer active in their care. This sentiment is also reflected information is inadequate. Of those surveyed, 69% in a recent Institute of Medicine Report, which did not have a website they especially liked for emphasizes the importance of patient portals in a cancer information, emphasizing the need for continuously healthcare system.15 With 6 To cite: Arnold CW, trusted information resources. With the quality of this additional motivation, patient portal deploy- McNamara M, El-Saden S, sources in question, patients thus often bring up ment and use is expected to become common- et al. J Am Med Inform information they find online with their doctor; one place.16 In point of fact, the (HL7) – Assoc 2013;20:1028 1036. study found that up to 90% of respondents who International Context-Aware Knowledge Retrieval

1028 Arnold CW, et al. J Am Med Inform Assoc 2013;20:1028–1036. doi:10.1136/amiajnl-2012-001457 Research and applications standard now provides a technical specification for integrating in radiologic interpretations31 32; a temporal orientation that electronic health records and personal health records with exter- shows the evolution of disease via imaging; and a source orien- nal information resources, and is increasingly being adopted by tation that allows patients to review an annotated version of – – vendors and information providers.17 19 their radiology reports.33 36 These three perspectives allow a Previous studies have found that despite the rising tendency user to navigate their radiologic information, allowing for the of patients to search for and access health information online, selective drilling down to the original image interpretations. they are often discouraged by the information they find as it is Figure 1 shows the four main components of our radiology frequently too general to elucidate the specifics of an indivi- portal interface: (1) a panel showing a patient’s ‘salient’ imaging – dual’s disease or treatment.1257 Notably, patients’ information findings, organized in reverse chronological order (figure 1A); needs are not limited to general knowledge, but also encompass (2) an information panel providing patient-oriented explana- access to their underlying medical records and the content tions of imaging techniques, disease concepts, and salient image within them.20 Indeed, receiving (accurate) information relevant findings (figure 1B); (3) an interactive panel showing only key to one’s cancer diagnosis has been shown to increase patient slices from patient imaging studies and associated extracted find- involvement in decision-making,8 and to enhance satisfaction ings from radiology reports (figure 1C); and (4) a study viewer with treatment options.9 Additionally, giving patients access to displaying the full image series with an annotated conclusion personalized health information can improve section from the corresponding report (figure 1D). Interactions between family members, and between patients and provi- with the portal are designed to be driven by the imaging find- ders.910The latter is especially important as it has been esti- ings list. From the list, a user may click on a finding of interest, mated that patients remember approximately only half of the which triggers the information panel to display a lay description information presented in a conversation with their physician.21 of the finding with an annotated illustration. In addition, click- Prior work shows that patient-oriented language is preferred ing a finding ‘activates’ imaging studies in a patient’s record by by patients when receiving abnormal radiology results,22 but graphically highlighting studies where the finding was noted by professional tools to explain medical concepts use expert lan- the radiologist. At any point, a user may click on a key slice guage, much of which patients do not understand.23 As such, from an imaging study to launch the study viewer. patients who do request copies of radiology reports and images generally receive this information with little or no additional System architecture and components explanatory material, and turn to their healthcare providers for The system architecture is shown in figure 2. Patients seen at the explanations. This scenario is sub-optimal in that some of the oncology clinic are pre-identified by a clinician, and on request, resultant questions could be answered with a suitable online our portal server fetches the required patient information information resource. Also, such an information resource could (images from the institutional picture archive and communica- be adapted to the specifics of a patient’s case, providing targeted tions system (PACS); reports from the radiology information details and lessening the cognitive burden on the patient to rec- system). The application is not intended to make new informa- oncile the content of his medical report with generalized infor- tion available to the patient before prior practitioner–patient mation resources designed for a broad spectrum of patients (eg, communication. Rather, it is meant to review information search engines, MedlinePlus, WebMD). For example, prior already disclosed to a patient by his healthcare provider, in research indicates that presenting medical information to order to limit both the stress of encountering new information patients accompanied by pictures can increase , recall, without professional guidance and that of attempting to recall and comprehension of medical concepts.24 25 This observation detailed information after talking with practitioners. The suggests that showing patients illustrations of imaging or disease retrieved data is fed into image and natural language processing concepts specifically related to their studies may provide them (NLP) modules, with the resultant analyses stored in a with an appreciation of the (causative) reasoning between their on the portal server. When a user accesses a given patient portal symptoms/sequelae and required treatments. While efforts exist page, a Java Server Pages application dynamically generates a set to create interfaces that support sharing radiologic imaging of HTML5 views from the raw and analyzed data, and serves across healthcare providers,26 current solutions are not designed up the web-based portal application. The modules and portal specifically to educate patients.27 28 components are now described in detail.

METHODS Salient findings panel We implemented an electronic portal for patients with primary The salient findings panel provides patients with a list of pertin- brain tumors (eg, gliomas, meningiomas, etc.), a population ent observations made over time and as documented through associated with a high degree of information needs and large radiologists’ interpretations. To define salient findings, a superset amounts of initial and follow-up radiologic imaging. Our system of candidate concepts was automatically extracted from the includes explanatory layers of information between the conclusion section of neuroradiology reports (brain MRI patient and the source clinical data, with each layer offering studies), using the Mayo Clinical Text Analysis and Knowledge a lay explanation and overview of the layer immediately Extraction System (cTAKES) NLP software operating with the below, forming a hierarchy of progressively more specific infor- Systemized Nomenclature of Medicine Clinical Terms mation views that ultimately link to individual source reports’ (SNoMED-CT) terminology.37 Negated concepts (eg, ‘no evi- findings and associated imaging studies. These layers help to dence of hydrocephalus’) were discarded using the cTAKES inte- mediate between professional and patient health perspectives, grated negation detector, which is based on NegEx.38 Although using concepts, illustrations, and key radiology images designed negated concepts can be particularly important (eg, ‘no edema for a consumer audience. Similar notions of augmenting present’), a design decision was made to focus only on concepts medical information have been discussed previously in the that were observed by the radiologist and therefore visible in literature.29 30The portal utilizes several information orienta- the imaging study. In total, 2883brain MRI reports from 277 tions, some of which have been previously explored in the lit- patients (based on all brain MRI studies and the related radi- erature, including: a problem orientation to summarize findings ology report, from all patients) were processed, resulting in the

Arnold CW, et al. J Am Med Inform Assoc 2013;20:1028–1036. doi:10.1136/amiajnl-2012-001457 1029 Research and applications

Figure 1 Annotated screenshot of the web-based patient radiology portal showing the different components of the application. (A) The imaging findings list displays salient concepts extracted from the conclusion section of radiology reports (eg, ‘edema’). (B) The information panel is used to display explanations of imaging findings and imaging techniques (eg, ‘What is edema?’ or ‘What is MRI?’). (C) The same key slice from each imaging study is displayed chronologically with the conclusion section from the corresponding radiology report to illustrate the individual’s response to treatment. (D) The interactive study viewer allows the user to view entire image series (ie, all slices) within the study and to read the conclusion section of the radiology report where complex terms (underlined words) have been annotated with mouse-over definitions (dark box overlaying the text).

extraction of 448 unique concepts. Although patients have a finding concepts, which were stored in a lookup table. The 15 desire to understand the significance of their radiology findings, most common concepts from this set are given in table 1. Using they lack the clinical expertise to define the set of specific the NLP module (see figure 2), concepts automatically extracted imaging concepts that fulfills this information need (eg, hydro- from a patient’s conclusion section were referenced in the cephalus, midline shift, necrosis, etc). Therefore, the selection lookup table; matches were retrieved and sorted in reverse of the concept subset was performed with guidance from: (1) chronological order for display in the salient findings panel. clinical experts, who have experience answering patient ques- To evaluate the automatic extraction of salient findings, we gen- tions; and (2) literature, which indicates that patients are con- erated a gold standard set and compared it to results from the cerned with understanding their different diagnoses (eg, NLP module. First, two annotators (the first and second author) glioblastoma), procedures (eg, craniotomy), and symptoms (eg, jointly annotated a random sample of 50 radiology conclusion sec- – edema).39 41 First, the investigators removed a large number of tions from our dataset under the guidance of a neuroradiologist concepts that were detected as a result of a radiologist’s (the third author). Next, the annotators separately annotated 150 mention of a patient’s historical disease or co-morbidity (ie, impression sections, which had an average length of 85 words. concepts without a visual representation in corresponding Following Hripcsak and Rothschild42 and Fleiss,43 we calculated imaging) from the set. Next, manual reconciliation of concepts the positive specific agreement between the annotators to be 0.91, that would be considered synonymous by a patient (eg, ‘malig- where the annotated spans for a term were required to overlap. nant neoplastic disease’ and ‘malignant neoplasm of brain’)was Discordances were adjudicated by a neuroradiologist to generate conducted, as was the removal of erroneous concepts from the the final gold standard. In total, 684 instances of the 52 salient set (eg, syncope, as in ‘to faint’, is often mapped, when a radi- terms were identified. Using this gold standard, the NLP module ologist mentions ‘faint contrast enhancement’). This process achieved an accuracy of 73% at identifying mentions of a concept, 44 resulted in 52 terms that comprised the final set of salient with an F1 score of 0.67 (precision 0.63, recall 0.73).

1030 Arnold CW, et al. J Am Med Inform Assoc 2013;20:1028–1036. doi:10.1136/amiajnl-2012-001457 Research and applications

Figure 2 System architecture showing natural language processing and imaging processing modules that generate several information orientations. The portal is implemented as a Java-based web application, with key slices in the imaging timeline shown as JPEGS, and an HTML Canvas image viewer that displays complete series using pixel data passed in a JSON object.

Information panel cerebral edema. These explanations were developed by our clin- Using a combination of graphical pictures and text, the informa- ical investigators based on their experience in providing such tion panel explains the salient concepts detected by NLP in the clarifications to patients in real life. When available, explana- conclusion section of radiology reports. We first attempted to tions were augmented with information from existing resources, employ existing concept explanations from the National such as the patient version of definitions from the National Institutes of Health (NIH) MedlinePlus repository. However, we Cancer Institute. Figure 3 shows an example explanation with found these definitions, as well as those present in the Unified an accompanying pictorial illustration. Medical Language System, to be too generic for use in this In addition to providing descriptions of disease concepts, the application. For example, when extracted from a brain magnetic information panel contains explanations of MRI pulse resonance (MR) report from a patient with brain cancer, there is sequences and contrast agents with the goal of helping patients a high degree of certainty as to the meaning of a radiologist’s understand why a given imaging study was performed, and how mention of ‘edema’ (ie, excess accumulation of water in the to interpret images in the portal. By way of illustration, figure 4 brain). Therefore, instead of using a general definition of shows the explanation for the function of contrast agents in edema, we present a more specific explanation to support the MRI brain tumor imaging. patient in understanding the relative importance and context of

Imaging studies panel For oncology patients, imaging is frequently used to assess Table 1 Fifteen most frequent concepts extracted from the response to treatment. For neuro-oncology, MR is the predom- conclusion sections of radiology reports inant modality, given its ability to highlight key pathophysiology UMLS Concept Unique Identifier (CUI) Term concomitant to the tumor’s progression/regression. Multiple variations of MR sequences are performed, with each series C0027651, C0006118 Brain neoplasms acquired to provide unique evidence on the state of a tumor: C0728940 Excision T1-weighted imaging highlights anatomy and therefore provides C0013604 Edema a good view of a tumor’s structure; whereas T2-weighted C0010280 Craniotomy images highlight water and are useful for observing edema sur- C1510420, C0333343, C1515091 Surgical resection cavity rounding a tumor (and potential fluid accumulation indicating C1627358 Contrast enhancement possible increasing intracranial pressure). Additional series, such C0017636 Glioblastoma as apparent diffusion coefficient maps (used for monitoring the C0229985 Surgical margins diffusion of water within a tumor), may also be acquired, but C0027540 Necrosis are not performed with the same regularity. C0543478 Residual tumor At our institution, each brain cancer patient typically receives C0019080 Hemorrhage a T1-weighted scan; a T1-weighted scan with contrast; a C0020255 Hydrocephalus T2-weighted scan; and a FLAIR (fluid attenuated inversion C0576481 Midline shift recovery) or PD (proton density) scan (FLAIR and PD scans are C0020564 Hypertrophy used to look for lesions and edema proximal to brain ventri- C0178874F Tumor progression cles). These scans are conducted every 4–6 weeks while receiv- Given the context of brain cancer imaging, some concepts were combined and ing chemotherapy, and then every few months if the cancer is in presented to the user in synonymous fashion. remission. The frequency of imaging and the fact that many UMLS, Unified Medical Language System. medical decisions are predicated on imaging results makes

Arnold CW, et al. J Am Med Inform Assoc 2013;20:1028–1036. doi:10.1136/amiajnl-2012-001457 1031 Research and applications

Figure 3 An example definition for the disease finding ‘edema’ used by the system, including textual and visual descriptions of the concept. patients naturally curious as to what their images mean. 2. Subsequent image normalization: When a new imaging study However, clinical MRI data is not ideal for unsupervised presen- is performed, it is retrieved from the institution’s PACS and tation to patients. First, it exists in DICOM (Digital Imaging automatically intensity standardized and registered to the ref- and in Medicine) format, which cannot be erence study selected by the neuroradiologist. Intensity stand- easily viewed without specialized software. Next, scans are ardization is performed using histogram matching.45 acquired as two-dimensional slices and viewed in stacks that Intra-subject registration is performed using a rigid transform- must be scrolled through to build a mental three-dimensional ation with nine degrees of freedom (three rotations, three view of the brain. Patients are not accustomed to interpreting translations, three scalings) using the FLIRT (Functional MRI images in this manner; nor will they have sufficient knowledge of the Brain Linear Image Registration Tool) package from to comprehend the neuro-anatomy seen in such studies. Finally, FSL (Functional MRI of the Brain Software Library).46 This as scans are acquired at different times, there are variations step aligns anatomy across studies, ensuring that the key slice across studies. For instance, patients’ heads may be tilted at dif- selected by the radiologist will match the same anatomical ferent angles in the scanner, the field of view may change, pixel slice in other studies, thereby automatically selecting the same intensity may vary, and the number of slices in a study is often key slice in all studies. The normalization process requires different. Ultimately, all of these factors confound the non- approximately 1 min per study, with preprocessed results expert in viewing and comparing imaging studies. stored on the portal server for subsequent . Following our paradigm of creating explanatory layers around 3. Key slice layout: The normalized key slices are then dis- such complex clinical data, we developed an image processing played on a timeline, side-by-side. This layout provides a module that generates key slices, which are displayed over time view of changes occurring within the brain as the result of in a single view (figure 1C). The process works as follows: disease progression and treatment. 1. Key slice selection: When an individual is first added to the The imaging studies panel presents these key slices accompan- patient portal system, one axial reference study is chosen by ied by the conclusion section of the corresponding radiology a radiologist, from which a key slice is selected. The key slice report, allowing a user to track changes visually and through the reflects as much of the radiologist’s description as possible. narrative of the radiologist. Figure 5 presents a sample result For example, if a patient has a tumor with midline shift from this processing module illustrating the collapse of a resec- (movement of the brain across the sagittal plane as a result tion cavity over time. of the space-occupying tumor), a key slice containing the tumor and the shifted brain ventricles will be selected. Study viewer Typically, a key diagnostic or post-surgical resection study The study viewer allows a user to peruse the original series data will be chosen. This key slice and study serves as a baseline from an imaging study alongside further educational materials reference point for subsequent imaging studies. In our specific to the conclusion section of the radiology report. The dataset, this process took approximately 3 min for a neurora- materials presented in this panel introduce concepts that are not diologist to complete. necessarily considered relevant in the specific context of a

Figure 4 An example definition for the imaging technique of ‘contrast’ used by the portal, including textual and visual descriptions of the concept.

1032 Arnold CW, et al. J Am Med Inform Assoc 2013;20:1028–1036. doi:10.1136/amiajnl-2012-001457 Research and applications

Figure 5 Key slices created by the imaging pipeline showing the temporal evolution of disease. In this example, a surgical resection cavity is collapsing over time, with the most recent image on the left. patient’s salient imaging findings list (eg, ‘lateral ventricles’), but Health Vocabulary (OAC-CHV) was used to classify concepts as they are core medical concepts that are not generally known found by cTAKES as ‘complex’ terms, which were then replaced by laypersons, they require explanation. We first tried to auto- with OAC-CHV lay definitions. Following Zeng-Treitler et al,33 matically translate the source text to a summary that a patient terms with a combination familiarity score lower than 0.6 in could understand. Necessarily, this approach required abstrac- OAC-CHV were deemed to be unfamiliar to the lay reader and tion and simplification, as it is not feasible to define such con- were replaced. This approach had several limitations. Though cepts to the same degree to which they are understood by OAC-CHV is under continual development, it does not contain clinicians. The Open-Access and Collaborative Consumer entries for many neuroradiology concepts (eg, ‘midline shift’)

Figure 6 Portal interaction illustrating the use of the image findings list to drive exploration of the radiology information. (1) Imaging findings may be clicked to display a definition in the information panel (2) and highlight relevant imaging studies (3). When a key image is clicked, the study viewer (4) is displayed.

Arnold CW, et al. J Am Med Inform Assoc 2013;20:1028–1036. doi:10.1136/amiajnl-2012-001457 1033 Research and applications

and includes many terms without lay explanations that have patient. Our future work includes exploring the integration uninformative complexity scores (eg, ‘vasogenic cerebral of domain-specific (eg, RadLex50) to improve the edema’); this latter point is likely due to the scarcity of the identification of findings and anatomical concepts, and creating concept in the corpus of query logs used to estimate complexity a gold standard to evaluate the performance of the NLP module in the OAC-CHV.47 Also, as a result of incorrect term detection at identifying terms for explanation in the study viewer. by NLP or improper lay definition insertion (eg, incorrectly Additionally, assessing the readability of the clinician-generated matching the tense of a sentence), we observed that the auto- definitions is a future goal, with previous work in the area using matic translation system introduced undesirable errors, if not cloze scoring and metrics specific to health-related content.51 52 misunderstandings. We therefore felt it was more appropriate to Finally, we note that as the NLP module targets only those con- leave the radiology conclusion unmodified and instead focused cepts that have been manually validated, it is possible that rele- on supplementing it with lay neuroradiology definitions. Similar vant concepts in unseen radiology conclusion sections may be to the creation of the salient image finding concept set, a set of missed. This problem is mitigated by having a large dataset on concepts was defined to augment the conclusion section by which to base concept selection, and may be further minimized manual review, a process that resulted in 247 concepts. by performing regular updates. However, if applied at a new Definitions for these concepts were written by the clinical inves- institution, a comprehensive review of radiology reports would tigators with a consumer audience in , an approach fol- likely be necessary to account for institution-specific reporting lowed in the creation of the OAC-CHV. This process resulted in practices. To receive feedback from a diverse group of health- a repository of explanations that are available to the portal to care professionals, the system was exhibited at a demonstration support the conclusion section of a radiology report when a booth at the Radiological Society of North America 2011 pertinent concept is identified by the NLP module. annual meeting. The application and its use were presented, with the authors available for support. Many people recognized DISCUSSION the portal’s ability to educate a patient on their disease state In contrast to previous work in patient portal development, through their record, provide a means by which a patient may which focuses on sharing text reports, medications, and labora- review diagnosis and treatment history, and allow a patient to tory results,12 31 32 48 49 our proposed portal displays and share their record with family members and other supporting attempts to explain both radiology imaging and reports, infor- individuals. However, there was also concern that despite the mation that is known to be difficult for patients to compre- mechanisms for structuring and explaining, the application hend.11 The presented portal view for neuro-oncology is just thrusts information onto the patient that may be past the one way information can be augmented to provide a (lay) average individual’s comprehension. Indeed, there are documen- patient with additional context pertinent to a particular disease. ted ‘mismatches’ between lay and professional definitions of In its most general form, this developed radiology portal frame- terms13 and furthermore, the language and concepts used by work can be used across a multitude of diseases and anatomies. patients is reflective of their ‘cultural, social, and experiential For instance, a view for lung cancer could also apply image knowledge’.53 Thus, no augmentation can allot for and correct registration to CT images to show changes related to disease all patient misconceptions. Ensuing misunderstandings could in progression, and the effects of interventions over time. One pos- turn result in extra work for the physician, who would be left sible addition to this imaging-centric view is the integration of with the burden of answering questions that would have not treatment information (eg, chemotherapy) concurrent with the otherwise arisen. Such viewpoints resulted in the suggestion that imaging timeline, providing a clearer picture of treatment effects the application be used as a tool during office visits, allowing as observed via radiologic imaging. for practitioners to be present while patients viewed the content Our preliminary results indicate the feasibility of the NLP in order to provide information support. module at identifying salient terms; however the problems of These critiques and comments were taken under consideration. using cTAKES’ annotators without customization are evident in The application is intended to facilitate the sharing our performance metrics. Common errors included the selection and explanation of radiology reports, providing users with of findings that are not negated, but are also not present in the new information in the form of increased comprehension of image. Representative instances of this type of error include, ‘… their medical imaging procedures and results. However, the following neurosurgical resection of the left lateral posterior portal is not designed to convey information beyond this frontal lobe mass…,’ and, ‘the pneumocephalus as well as the scope (eg, there is no personalized prognostic information layer of acute blood previously seen within the resection cavity offered). Furthermore, the system is designed to release test have resolved’. In the first example the mass has been resected results only after approval by a physician. For instance, a and therefore will not appear in the imaging. Similarly, in the neuro-oncologist may control when a patient has access to a second example pneumocephalus and acute blood have resolved study through the portal, which is likely only after an in-person from the resection cavity and therefore will not be visualized. clinic consultation. And while steps can be taken to prevent Such expressions illustrate the complexity of identifying the issues regarding patients misunderstanding and clinician work- findings from a radiology report that were actually observed in load increases, the potential for these issues cannot be completely the corresponding imaging study by the radiologist, and suggest eliminated.11 24 54 55 There are, and will always be, risks in allow- the need for training contextual sequence models of both words ing patients direct access with their records, but evidence indi- and reports over time. Further supporting the need for temporal cates that these potential risks are outweighed by the benefits models, there are few standards for reporting in the domain of provided by such a system to an engaged patient.9103356 neuroradiology and therefore a patient may observe inconsisten- cies in the coverage of concepts across reports over time due to CONCLUSION contrasting styles between radiologists, or reports that describe As part of having a health-literate patient population that is only incremental changes, rather than all findings. A sufficiently engaged and informed in its own care, it is of growing import- robust temporal model may be able to recognize such gaps in ance to establish educational portals that deliver customized, concept coverage and interpolate the missing findings for the understandable radiology content to patients. Despite concerns,

1034 Arnold CW, et al. J Am Med Inform Assoc 2013;20:1028–1036. doi:10.1136/amiajnl-2012-001457 Research and applications studies measuring the impact of patient portals have not found 16 Koch-Weser S, Bradshaw YS, Gualtieri L, et al. The internet as a health information an increase in consumer misconceptions about health informa- source: findings from the 2007 Health Information National Trends Survey and 57 58 implications for health communication. J Health Commun 2010;15:279–93. tion. In contrast, patients who reviewed their data via a 17 Kemper D, Del Fiol G, Hall L, et al. Getting patients to meaningful use: using the portal reported that it led them to more accurate information HL7 infobutton standard for information prescriptions. Healthwise, 2010. and better prepared them for upcoming clinical visits,32 as well 18 Fiol GD, Huser V, Strasberg HR, et al. Implementations of the HL7 context-aware as made them more able to cope with the anxiety associated knowledge retrieval (“Infobutton”) standard: challenges, strengths, limitations, and – with diagnoses when they receive information on disease pro- uptake. J Biomed Inform 2012;45:726 35. 10 48 49 56 19 Ma W, Dennis S, Lanka S, et al. MedlinePlus connect: linking health IT systems to gression and treatment. Moreover, increased access to consumer health information. IT Professional 2012;14:22–8. information for cancer patients, including their medical records, 20 Beckjord EB, Rechis R, Nutt S, et al. What do people affected by cancer think about has been shown to increase satisfaction with treatment choices, electronic health information exchange? Results from the 2010 LIVESTRONG increase confidence in care providers, and improve adherence to Electronic Health Information Exchange Survey and the 2008 Health Information 48 59 National Trends Survey. J Oncol Pract 2011;7:237–41. medical advice. Portal access also allows the consumer to 21 Ley P, Whitworth MA, Skilbeck CE, et al. Improving doctor-patient communication consolidate information that has historically been dispersed in general practice. J Royal Coll Gen Pract 1976;26:720–4. across sources, an important concern in medical imaging.60 The 22 Johnson AJ, Easterling D, Nelson R, et al. Access to radiologic reports via a patient presented system offers a novel solution to sharing radiology portal: clinical simulations to investigate patient preferences. J Am Coll Radiol – information with consumers, and is driven by imaging informat- 2012;9:256 63. 23 Zielstorff RD. Controlled vocabularies for consumer health. J Biomed Inform ics tools to transform clinically-generated information into edu- 2003;36:326–33. cational views that may be customized by patient and disease. 24 Zeng-Treitler Q, Kim H, Hunter M, edis. Improving patient comprehension and recall of discharge instructions by supplementing free texts with pictographs. American Contributors CWA and MM were responsible for all aspects of this work, Medical Informatics Association (AMIA) Annual Symposium Proceedings 2008; including conceptualization, design, implementation, and evaluation. SE-S, RKT, and 2008:849–53. AATB performed conceptualization and design. SC performed implementation and 25 Houts PS, Doak CC, Doak LG, et al. The role of pictures in improving health evaluation. CWA, MM, SE-S, and AATB participated in the writing and editing of the communication: a review of research on attention, comprehension, recall, and manuscript. adherence. Patient Educ Couns 2006;61:173–90. Funding This work was supported by NIH/NLM grant number R01 LM011333 and 26 RSNA Image Share Network Reaches First Patients. http://www.rsna.org/NewsDetail. NIH/NCI grant number R01 CA1575533. aspx?id=2409 27 lifeIMAGE. https://cloud.lifeimage.com Competing interests None. 28 Dell Unified Clinical Archive. http://content.dell.com/us/en/healthcare/healthcare-medical- Ethics approval UCLA IRB. archiving-unified-clinical-archive.aspx 29 Miller T, Leroy G, Wood E, eds. Dynamic generation of a table of contents with Provenance and peer review Not commissioned; externally peer reviewed. consumer-friendly labels. American Medical Informatics Association (AMIA) Annual Symposium Proceedings 2006; 2006:559–63. 30 Maddock C, Camporesi S, Lewis I, et al. Online information as a decision making REFERENCES aid for cancer patients: recommendations from the Eurocancercoms project. Eur J 1 Pew Internet and American Life Project. The Social Life of Health Information. Cancer 2011;48:1055–9. 2009. 31 Grant RW, Wald JS, Poon EG, et al. Design and implementation of a web-based 2 Bass S, Ruzek S, Gordon TF, et al. The relationship of internet health information patient portal linked to an ambulatory care : patient gateway use with patient behavior and self efficacy: experiences of newly diagnosed cancer for diabetes collaborative care. Diabetes Technol Ther 2006;8:576–86. patients who contact the National Cancer Institute’s Cancer Information Service. 32 Schnipper JL, Gandhi TK, Wald JS, et al. Design and implementation of a J Health Commun 2006;11:219–36. web-based patient portal linked to an electronic health record designed to improve 3 Castleton K, Fong T, Wang-Gillam A, et al. A survey of internet utilization among medication safety: the patient gateway medications module. Inform Prim Care patients with cancer. Support Care Cancer 2011;19:1183–90. 2008;16:147–55. 4 Metz JM, Devine P, DeNittis A, et al. A multi-institutional study of internet 33 Zeng-Treitler Q, Goryachev S, Kim H, et al. Making texts in electronic health records utilization by radiation oncology patients. Int J Radiat Oncol Biol Physiol comprehensible to consumers: a prototype translator. AMIA Annual Symposium 2003;56:1201–5. Proceedings. American Medical Informatics Association, 2007. 5 Meric F, Bernstam EV, Mirza NQ, et al. Breast cancer on the world wide web: cross 34 Cardillo E, Tamilin A, Serafini L. A methodology for knowledge acquisition in sectional survey of quality of information and popularity of websites. BMJ consumer-oriented healthcare. Knowledge discovery, knowledge engineering and 2002;324:577. knowledge management. Springer, 2011:249–61. 6 National Cancer Institute. Cancer Communication Health Information National 35 Brennan PF, Downs S, Casper G. Project HealthDesign: rethinking the power and Trends Survey 2003 and 2005. 2007; NIH Publication No. 07-6214. potential of personal health records. J Biomed Inform 2010;43:S3–5. 7 Schwartz KL, Roe T, Northrup J, et al. Family medicine patients’ use of the internet 36 Adnan M, Warren J, Orr M. Enhancing patient readability of discharge summaries for health information: a MetroNet Study. J Am Board Fam Med 2006;19:39–45. with automatically generated hyperlinks. and Informatics Review 8 Luker KA, Beaver K, Leinster SJ, et al. The information needs of women newly Online. 2009. diagnosed with breast cancer. J Adv Nurs 1995;22:134–41. 37 Savova GK, Masanz JJ, Ogren PV, et al. Mayo clinical text analysis and knowledge 9 Cawley M, Kostic J, Cappello C. Informational and psychosocial needs of women extraction system (cTAKES): architecture, component evaluation and applications. choosing conservative surgery/primary radiation for early stage breast cancer. Cancer J Am Med Inform Assoc 2010;17:507–13. Nurs 1990;13:90–4. 38 Chapman WW, Bridewell W, Hanbury P, et al. A simple for identifying negated 10 Gravis G, Protiere C, Eisinger F, et al. Full access to medical records does not findings and diseases in discharge summaries. J Biomed Inform 2001;34:301–10. modify anxiety in cancer patients: results of a randomized study. Cancer 39 Davidson JR, Brundage MD, Feldman-Stewart D. Lung cancer treatment decisions: 2011;117:4796–804. patients’ desires for participation and information. Psycho Oncol 1999;8:511–20. 11 Keselman A, Slaughter L, Arnott-Smith C, et al. editors. Towards consumer-friendly 40 Jenkins V, Fallowfield L, Saul J. Information needs of patients with cancer: results PHRs: patients’ experience with reviewing their health records. American Medical from a large study in UK cancer centres. Br J Cancer 2001;84:48. Informatics Association (AMIA) Annual Symposium Proceedings 2007. 41 Leydon GM, Boulton M, Moynihan C, et al. Cancer patients’ information needs and 2007;399–403. information seeking behaviour: in depth interview study. BMJ 2000;320:909–13. 12 Weingart SN, Rind D, Tofias Z, et al. Who uses the patient internet portal? The 42 Hripcsak G, Rothschild AS. Agreement, the f-measure, and reliability in information PatientSite experience. J Am Med Assoc 2006;13:91–5. retrieval. J Am Med Inform Assoc 2005;12:296–8. 13 Soergel D, Tse T, Slaughter L. Helping healthcare consumers understand: an 43 Fleiss JL. Measuring agreement between two judges on the presence or absence of ‘interpretive layer’ for finding and making sense of medical information. Int Med a trait. Biometrics 1975:651–9. Inform Assoc Medinfo Proc 2004;11(Pt 2):931–5. 44 Manning CD, Raghavan P, Schütze H. Introduction to information retrieval. Cambridge: 14 Office of the National Coordinator for Health Information Technology. Electronic Cambridge University Press, 2008. health records and meaningful use. 2012. http://healthit.hhs.gov/portal/server.pt? 45 Nyul LG, Udupa JK, Zhang X. New variants of a method of MRI scale standardization. open=512&objID=2996&mode=2 Med Imaging IEEE Trans 2000;19:143–50. 15 IOM (Institute of Medicine). Best care at lower cost: the path to continuously 46 Smith SM, Jenkinson M, Woolrich MW, et al. Advances in functional and structural learning health care in America. Washington, DC: National Academies Press, 2012. MR image analysis and implementation as FSL. Neuroimage 2004;23:S208–19.

Arnold CW, et al. J Am Med Inform Assoc 2013;20:1028–1036. doi:10.1136/amiajnl-2012-001457 1035 Research and applications

47 Zeng-Treitler Q, Tse T, Crowell J, et al. edn. Identifying consumer-friendly display 54 Ross SE, Lin CT. A randomized controlled trial of a patient accessible medical (CFD) names for health concepts. American Medical Informatics Association (AMIA) record. American Medical Informatics Association (AMIA) Annual Symposium Annual Symposium Proceedings 2005;2005:859–63. Proceedings 2003;2003:990. 48 Ross SE, Moore LA, Earnest MA, et al. Providing a web-based online 55 Detmer D, Bloomrosen M, Raymond B, et al. Integrated personal health records: with electronic communication capabilities to patients with congestive heart failure: transformative tools for consumer-centric care. BMC Med Inform Decis Mak randomized trial. J Med Internet Res 2004;6:e12. 2008;8:45. 49 Earnest MA, Ross SE, Wittevrongel L, et al. Use of a patient-accessible electronic 56 Meredith C, Symonds P, Webster L, et al. Information needs of cancer patients in medical record in a practice for congestive heart failure: patient and physician West Scotland: cross sectional survey of patients’ views. BMJ 1996;313:724–6. experiences. J Am Med Inform Assoc 2004;11:410–7. 57 Walker J, Leveille SG, Ngo L, et al. Inviting patients to read their doctors’ notes: 50 Langlotz CP. RadLex: a new method for indexing online educational materials1. patients and doctors look ahead. Ann Intern Med 2011;155:811–19. RadioGraphics 2006;26:1595–7. 58 Delbanco T, Walker J, Bell SK, et al. Inviting patients to read their doctors’ 51 Kim H, Goryachev S, Rosemblat G, et al. eds. Beyond surface characteristics: a new notes: a quasi-experimental study and a look ahead. Ann Intern Med health text-specific readability measurement. American Medical Informatics 2012;157:461–70. Association (AMIA) Annual Symposium Proceedings 2007; 2007;418–22. 59 Baldry M, Cheal C, Fisher B, et al. Giving patients their own records in 52 Taylor WL. “Cloze procedure”: a new tool for measuring readability. Journalism general practice: experience of patients and staff. BMJ (Clinical Res Ed) Quarterly 1953;30:415–33. 1986;292:596–98. 53 Keselman A, Logan R, Smith CA, et al. Developing informatics tools and strategies 60 Pratt W, Unruh K, Civan A, et al. Personal health information management: for consumer-centered health communication. J Am Med Inform Assoc integrating personal health information helps people manage their lives and actively 2008;15:473–83. participate in their own health care. Commun ACM 2006;49:51–5.

1036 Arnold CW, et al. J Am Med Inform Assoc 2013;20:1028–1036. doi:10.1136/amiajnl-2012-001457