The Unmet Data Visualization Needs of Decision Makers within Organizations Evanthia Dimara, Harry Zhang, Melanie Tory, Steven Franconeri To cite this version: Evanthia Dimara, Harry Zhang, Melanie Tory, Steven Franconeri. The Unmet Data Visualization Needs of Decision Makers within Organizations. IEEE Transactions on Visualization and Computer Graphics, Institute of Electrical and Electronics Engineers, 2021. hal-03199715 HAL Id: hal-03199715 https://hal.archives-ouvertes.fr/hal-03199715 Submitted on 15 Apr 2021 HAL is a multi-disciplinary open access L’archive ouverte pluridisciplinaire HAL, est archive for the deposit and dissemination of sci- destinée au dépôt et à la diffusion de documents entific research documents, whether they are pub- scientifiques de niveau recherche, publiés ou non, lished or not. The documents may come from émanant des établissements d’enseignement et de teaching and research institutions in France or recherche français ou étrangers, des laboratoires abroad, or from public or private research centers. publics ou privés. AUTHORS’ VERSION, 2021. TO APPEAR IN IEEE TRANSACTIONS ON VISUALIZATION AND COMPUTER GRAPHICS 1 The Unmet Data Visualization Needs of Decision Makers within Organizations Evanthia Dimara, Harry Zhang, Melanie Tory, and Steven Franconeri Abstract—When an organization chooses one course of action over alternatives, this task typically falls on a decision maker with relevant knowledge, experience, and understanding of context. Decision makers rely on data analysis, which is either delegated to analysts, or done on their own. Often the decision maker combines data, likely uncertain or incomplete, with non-formalized knowledge within a multi-objective problem space, weighing the recommendations of analysts within broader contexts and goals. As most past research in visual analytics has focused on understanding the needs and challenges of data analysts, less is known about the tasks and challenges of organizational decision makers, and how visualization support tools might help. Here we characterize the decision maker as a domain expert, review relevant literature in management theories, and report the results of an empirical survey and interviews with people who make organizational decisions. We identify challenges and opportunities for novel visualization tools, including trade-off overviews, scenario-based analysis, interrogation tools, flexible data input and collaboration support. Our findings stress the need to expand visualization design beyond data analysis into tools for information management. Index Terms—Decision making, visualization, interview, survey, organizations, management, business intelligence. F 1 INTRODUCTION ISUALIZATION varies in its goals, from testing data patrons, or temperature-sensitive catering operations? Other V veracity or confirming a suspected pattern, to open- factors are difficult or impossible to quantify. How does ended exploration in search of insight or enjoyment. Within she weigh financial factors against an improvement in the organizations, often these processes serve an end goal of organization’s reputation, or the abstract moral goal of making a decision that will affect the organization’s struc- decreasing negative environmental impacts? Sam does not ture, processes, or outcomes. For example, a homeless shel- fully trust the recommendations of the analysts because of ter might need to decide which services might provide max- this lack of context: To account for government tax incen- imum benefit for an individual, while balancing resource tive uncertainties, she dove into the financial data analysis distribution among many people. A university administra- herself, but gave up after wading through the consultant’s tor might need to compare retirement plan offerings for dozen disconnected spreadsheets. faculty and staff, while juggling an overwhelming list of The goal of the present study is to identify how visual- costs and benefits to many parties. izations can be embedded within the complex framework of We argue that this decision making step has received too organizational decision making (hereafter referred to simply little attention in the visualization research literature. Across as decision making). We identify themes and challenges, as a survey and interviews of organizational decision makers, well as opportunities for novel visualization tools to aid we identify challenges and opportunities for tools that can decision makers like Sam. These opportunities stress the better support them. need to expand visualization design beyond data analysis We summarize the challenges with an example ab- into tools for information management, including tools that stracted from our interviews. Sam is the CEO of a city facilitate trade-off overviews, scenario-based analysis, inter- convention center, facing the decision of whether to make rogation, more flexible data input, and collaborative work. a large investment in a greener power plant. Her decision is complex. Some important factors are quantifiable after hir- 2 RELATED WORK ing outside experts, such as a consultant who can estimate the tradeoff between initial capital costs against later savings We focus on decisions that influence the interpersonal, from higher efficiency and government tax incentives, or collaborative structures and processes of an organization engineers who can estimate the greener system’s slower cor- (not just micro-decisions related to the decision-maker’s rection in interior temperature in response to rapid weather own personal workflow) and aim to characterize organiza- changes. But even these quantities carry uncertainties, or tional decision makers as visualization users. To that end, rely on sparse or unreliable data. Will that government we discuss works at the intersection of visualization and tax incentive still exist after the next election? How much decision making and then literature investigating the use of will the slower correction time upset our temporarily chilly visualizations within organizations. 2.1 Visualization & Decision making • E. Dimara is with Utrecht University and University of Konstanz. E-mail: [email protected] Scholarly books on visualization emphasize that decision • H. Zhang and S. Franconeri are with Northwestern University making is the ultimate goal of data visualization [1], [2], [3], • M. Tory is with Tableau Software. [4], [5], [6], while the effective support of those decisions has AUTHORS’ VERSION, 2021. TO APPEAR IN IEEE TRANSACTIONS ON VISUALIZATION AND COMPUTER GRAPHICS 2 A TO WHAT EXTENT YOU CONSIDER "DECISION-MAKING" (i.e. the selection of a course of action over alternative actions) AS A PRIMARY TASK IN YOUR JOB? DECISIONDECI MAKING - DM LEVEL B JOB TITLE EXPERIENCE YEARS ORGANIZATION SIZE AGE I mmake decisions on a regular basis, primarily *across* multiple CEO > 20 5.000+ >60 departments or divisions 4 of my organization Executive 16-20 1000-4.999 51-60 Director 11-15 500-999 41-50 I make decisions on a regular basis, Senior Manager 100-499 31-40 primarily *within* a single 6-10 18-30 department or divisions 3 Junior Manager 3-5 25-99 of my organization Non-Entry Level E. 0-2 10-24 EDUCATION I make some decisions within Entry Level E. 2-9 PhD my organization, but 2 Msc not on a regular basis. Other PEOPLE I SUPERVISE 1 Prof >1000 BSc I am not directly responsible for LOCATION 201-1000 ORGANIZATION TYPE HS making decisions, but my job is to suggest 1 1 For profit recommended actions to decision-makers. 51-200 GENDER 10 31-50 Education Not at all. Apart from the micro- 20 Female 11-30 Non-profit decisions involved in my personal 30 workflow, my job is neither to make 0 1-10 Health Care Male decisions nor to suggest recommended Non-binary actions to decision-makers. 0 Government N/A TO WHAT EXTENT YOU CONSIDER "DATA ANALYSIS" AS A PRIMARY TASK IN YOUR JOB? FINDINGS: WHAT COULD DECISION MAKERS NEED? DATA ANALYSIS - DA LEVEL 122 decision makers C DM LEVEL >1 42 data analysts 0 1 2 3 Not at all. My role I do not conduct I conduct some My primary job DM LEVEL< 2 AND does not involve data data analysis, but data analysis when is to conduct DA LEVEL >1 analysis, and does not I do incorporate needed, but not data analysis on flexible interrogative scenario-based trade-off oriented ** 13 excluded with incorporate the work the work of data on a regular basis. a regular basis. data interfaces tools simulations overviews DM,DA < 2 (gray dots) of data-analysts analysts in my work. Fig. 1: A) Participants grouped as “decision makers” or “data analysts” based on their answers to the Decision Making (DM) and Data Analysis (DA) questions. B) Survey demographics. C) Emerging themes from our interview analysis. been identified as the core challenge of visual analytics [7]. Notably, the few studies that do assess decisions concern Decision-making is studied in domains such as psychology, cases of narrow complexity [25], such as binary decision economics, cognitive science and management, and each tasks [23], [26]. Here we attempt to understand more discipline has its own understanding of decision-making complex forms of decision making [27] by studying its processes and how to study them. Yet visualization research operational perspective within organizations. emphasizes building a unified cross-domain understanding of human decisions made with visualized data [8]. Numerous visualization tools can potentially support decision making activities. General-purpose tools typically 2.2 Visualization & Organizational Context support any multi-attribute