CRA-Industry Update

February 23, 2021

Vivek Sarkar Ben Zorn CRA-I Steering Committee (formed in Nov 2020)

Lorrie Cranor, CMU Fatma Ozcan, Google Chris Ramming, VMware Vivek Sarkar, Georgia Tech (Co-Chair) Divesh Srivastava, AT&T Gil Vandentop, Intel YY Zhou, UCSD Ben Zorn, Microsoft (Co-Chair)

2 Outline

• Update on CRA-I

• Breakout groups

• Wrap-up

3 CRA Strategic Initiatives Implementation Ownership Recommendation February 18, 2021

Indicates board or committee member Indicates staff Action Planning Suggested Implementation Initiatives (Phase 1) Participants Ownership A1a. In collaboration with the computing research Ann Schwartz Drobnis New Board Programmatic or Ad community, define tenets and practices for socially Peter Harsha Hoc Committee responsible computing Ran Libeskind-Hadas Ellen Zegura A2a. Engage the board in defining criteria for James Allan Executive Committee selection of activities to pursue Andy Bernat A2b. Develop governance and decision-making Erik Russell framework and cycle for selection of activities Ellen Zegura B1b. Broaden involvement of computing researchers Nancy Amato Staff, also see TIG from member organizations, beyond the member Khari Douglas recommendation as part of contacts Peter Harsha Governance report Brian Mosley Erik Russell Heather Wright Helen Wright Shar Steed B2a. Proactively influence government policy and Alex Aiken Government Affairs Committee procedures that affect the computing research Peter Harsha Context: CRA Strategic Initiativescommunity, in addition to responding to emerging Stephanie Forrest issues Ellen Zegura C2a. Continually gather and communicate examples of Khari Douglas Staff CRA Strategic Initiatives Implementation Ownership Recommendation the positive impact of computing research on society Peter Harsha Brian Mosley February 18, 2021 Erik Russell Shar Steed Indicates board or committee member Indicates staff Heather Wright Action Planning Suggested Implementation D1a. Meaningfully engage companies that conduct Ben Zorn CRA-I Committee Initiatives (Phase 1) Participants Ownership computing research Vivek Sarkar A1a. In collaboration with the computing research Ann Schwartz Drobnis New Board Programmatic or Ad Lorrie Cranor community, define tenets and practices for socially Peter Harsha Hoc Committee Fatma Özcan responsible computing Ran Libeskind-Hadas Chris Ramming Ellen Zegura Divesh Srivastava A2a. Engage the board in defining criteria for James Allan Executive Committee Gil Vandentop selection of activities to pursue Andy Bernat YY Zhou A2b. Develop governance and decision-making Erik Russell E2a. Enhance awareness of computing research Erik Russell CRA-WP or CRA-E framework and cycle for selection of activities Ellen Zegura opportunities to a broader student population Andrea Danyluk B1b. Broaden involvement of computing researchers Nancy Amato Staff, also see TIG Sandhya Dwarkada from member organizations, beyond the member Khari Douglas recommendation as part of Susanne Hambrusch contacts Peter Harsha Governance report Lori Pollock Brian Mosley E2b. Utilize best practices to create guidance for Erik Russell CRA-WP Erik Russell ensuring a supportive community for diverse students Burcin Tamer Heather Wright in computing research Ellen Zegura Helen Wright E3a. Expand CRA programming to enhance Susanne Hambrusch CRA-E Shar Steed preparation experiences for computing research Lori Pollock B2a. Proactively influence government policy and Alex Aiken Government Affairs Committee students Erik Russell procedures that affect the computing research Peter Harsha F1a. Institutionalize systemic, continuous, and targeted Khari Douglas Staff community, in addition to responding to emerging Stephanie Forrest communications Peter Harsha issues Ellen Zegura Brian Mosley C2a. Continually gather and communicate examples of Khari Douglas Staff Erik Russell the positive impact of computing research on society Peter Harsha Shar Steed Brian Mosley Heather Wright Erik Russell Helen Wright Shar Steed Heather Wright D1a. Meaningfully engage companies that conduct Ben Zorn CRA-I Committee computing research Vivek Sarkar Lorrie Cranor Fatma Özcan Chris Ramming 4 Divesh Srivastava Gil Vandentop YY Zhou E2a. Enhance awareness of computing research Erik Russell CRA-WP or CRA-E opportunities to a broader student population Andrea Danyluk Sandhya Dwarkada Susanne Hambrusch Lori Pollock E2b. Utilize best practices to create guidance for Erik Russell CRA-WP ensuring a supportive community for diverse students Burcin Tamer in computing research Ellen Zegura E3a. Expand CRA programming to enhance Susanne Hambrusch CRA-E preparation experiences for computing research Lori Pollock students Erik Russell F1a. Institutionalize systemic, continuous, and targeted Khari Douglas Staff communications Peter Harsha Brian Mosley Erik Russell Shar Steed Heather Wright Helen Wright

Mission and Vision Statements

• CRA Mission Statement: CRA’s mission is to enhance innovation by joining with industry, government and academia to strengthen research and advanced education in computing.

• CRA-I Mission Statement (draft): CRA-I’s mission is to create a computing research ecosystem that focuses on opportunities with industry to leverage the potential synergies among industry, government, and academia for mutual benefit and improved societal outcomes.

• CRA-I Vision: CRA-I is the one-stop portal for industry to engage with the computing research community across academia, government, and other companies

5 Background: Industry/Academia Ad Hoc Committee Report

(Presented in July 2020 Board Meeting)

https://cra.org/cra-committee-on-industry-academia-interactions-releases-report/ Vivek Sarkar (Chair), Nancy Amato, Susan Davidson, David Ebert, Mark Hill, Charles Isbell, Shwetak Patel, Chris Ramming, Divesh Srivastava, Eric de Sturler, Marvin Theimer, Ben Zorn Executive Summary (details in report) • Survey sent to 221 computing department chairs (special thanks to Betsy!) • 105 responses --- indicates strong interest by CRA member organizations • Over 60% also said that they were open to engaging further on this topic • Significant industry engagement is under way between faculty and industry, with an increasing trend

• Mix of positive and negative aspects of industry/academia engagements, with the positives outweighing the negatives

• Wealth of information from survey can help guide future CRA activities related to industry-academia engagements

• Overall: clear opportunity for CRA to add value to member institutions and

industry partners by facilitating and strengthening their interactions 7 Types of industry organizations involved in faculty engagements

8 1776 DC Critical Mass IBM Myomo SentiMetrix, Inc 3M Cyber Pack IBM Research National Security ServiceNow Accenture Ventures, Inc. ICSI Agency Siemens Technology Labs D.E. Shaw Group IDA NB Motors Siemens Adobe DAI Indeed NCR Corporation Corporation Adventium Lab Data Inflection Nervana SM&A Aerospace Dell EMC InfoSec New Scientist Snapchat CRA-I’s Initial Contact List Corporation Deloitte Consulting World Bank New Venture Social Tables Aetna Develop For Integral Mind Partners Splunk Airbnb Digital Intel NICT SRI Alphabet Inc. Manufacturing and Intel Labs Nimbis Services Stanley Black & Amazon Design Innovation Intercontinental Nissan Research Decker Amazon Web Institute (DMDII) Exchange -- ICE Nokia State Farm Services Disney Animation IonQ Norfolk Southern Stryker Corporation (to be extended) AMD DMI J.P. Morgan Northrop Grumman SunniBrown Inc. American Express Dow Chemical JETRO Nutanix Sunrise Analytics Dropbox John Deere NVIDIA Technologies, Inc. ANSER EarthLink Johnson & Johnson OIT Concepts LLC Synced Technology Apple Eccalon Kaiser Omada Health Co. Aptiv Education Advisory Kaiser Permanente OneTrust Synopsys, Inc. Asurion Board Center for Health Opaxis Tableau Software AT&T Research Educational Testing Research Orange Silicon The Home Depot • Initial source: CCC, Georgia Tech Automatic Data Service Keysight Valley The MITRE Processing, Inc. Efficient Cities Kimberly-Clark Orbital Insight Corporation (ADP) Envieta Koch Industries Oscar Health The QED Group, Automatic Labs EPIC KPMG OSIsoft LLC Bank of America Equifax KS TechPros Palantir Tipograph Law Bell Labs Research ESRI L3Harris Parkview Toyota Motor North • Additional sources: Company names Blackbaud, Inc. Expanse, Inc. Corporation Parkview Health America BlackRock F5 Networks Lambdalytics LLC PayPal Travelers Bloomberg Facebook Leidos Corp Pearson Trilateral Research Boeing Fetch Robotics Inc. Lewis-Burke Peloton Ltd provided in Industry/Academia survey Booz Allen Hamilton Formerly Oracle Associates LLC PennDOT Trivium Consulting Boston Scientific Friend Lockheed Martin Prognosys working with BT Americas Frog Lyft Protiviti HPCWire BuzzFeed News Gartner Macy's PwC Truist BWX Technologies GE Aviation Magic Leap Qualcomm, Inc. Twilio Capital One GE Digital Mailchimp QxBranch U.S. Xpress • Wide range of companies è need for a CAST, Inc. GE Global Research MarkLogic Raytheon Uber Center for Senior GE Power Corporation Redfin Union Pacific Independence Genentech Tech Mastercard RetailMeNot, Inc. UPS Chick-fil-a General Motors McAfee Revel Systems Urjanet “catalog” of options to engage with Cigital Georgia Pacific McGrath Analytics Revmetrix, AI tech Verizon Connect Cisco GlobalFoundries McKesson startup Verizon Media Cisco Systems, Inc. Good Research Measure Rieg and Associates Viasat CITADEL Google Measure UAS Inc. Rigetti Computing Visa universities Citi Gradiant Medtronic Riot Games Visa Research CIVILIAN Green Hills MeYouHealth Roblox VMware Civis Analytics Software, Inc. Micro Craft Inc. Rockwell VMware Research Clever GTRI Microsoft Salesforce Vox Media CloudMinds Inc HERE Corporation SambaNova Walmart Coca-Cola Hewlett Packard Microsoft Research Systems WarnerMedia • CRA-I can also help direct companies to ConocoPhillips Enterprise MindSpring Samsung WIOMAX Corporation Honeywell MorphoTrust Sandia National Workday Cooper Lighting HP Labs Mozilla Labs Yahoo Corning HubSpot MPH - a Porsche SAS Yu & Robinson other CRA committees and activities, as Create with Context i3solutions Company Zillow Group

appropriate

9 CRA-I Need Finding Process

● Initial set of interviews with the following people, representing a range of expertise across industry, academia, and government: ○ Ron Brachman, Karina Edmonds, Peter Harsha, Kim Hazelwood, Mark Hill, Jim Kurose, Margaret Martonosi, Raghu Ramakrishnan, Vijay Saraswat, Alfred Spector, Jaime Teevan, Marvin Theimer, Manuela Veloso ○ Many of our interviewees observed that there is a unique opportunity for CRA-I to fill a long-standing need with its bridging role.

● Pilot survey created to gain broader insight on CRA-I opportunities from industry partners: ○ Results to be collected, analyzed and shared in July 2021 Board meeting ○ Next slide has sample output from 12 responses

10

Q10 - In which of the following do you see value in what CRA-I could provide to your company? Select all that apply. (Sample output from 12 responses)

Interview Questions

We asked how CRA-I can support the following interactions:

Industry-Industry (I-I) Industry Academia Industry-Academia (I-A) Industry-Academia-Government (I-A-G) Government Top level questions: ● What role can CRA-I play in providing value? ● What topic areas are of particular interest and/or most challenging? ● What approaches to collaboration between the entities above should we consider?

12 CRA-I Observations and Synthesis from Interviews (1)

CRA-I’s Role

● Bring together industry partners & CRA for shared understanding of issues ○ Including identifying partners and creating social connections

● Help connect industry/academia/government in seeding partnerships

● Create a shared understanding of best practices working with academia and other industry researchers ○ Help make creating a corporate computing research culture easy and effective (e.g., a term suggested was “Research in a Box”) Emerging topics/challenges mentioned: Explainable AI, ethical AI, social responsibility, cloud

13 CRA-I Observations and Synthesis from Interviews (2) Collaboration ideas ● Create mingling opportunities and information sharing that connect companies and influencers who inform computing research ● Provide a central place for industry to develop a shared perspective on important social implications of tech, including community standards (e.g., privacy, fairness, etc.) ● How? Writing white papers, hosting workshops, etc.

● Provide guidance and support effective data sharing ● Programs that help faculty and industry researchers exchange places

14 Opportunities for Collaboration with other parts of CRA

CCC - Example: quantum readiness study - A first-time experiment in having industry input on CCC study topics - Resulted in one of the recent quadrennial papers - Company participants valued the experience and commented on how they could not imagine another neutral forum where the conversations would have been so forthright CRA-WP - CRA-I can introduce partners to opportunities with CRA-WP - CRA-I can learn from CRA-WP experience with companies CRA Government Affairs - CRA-I partners can contribute to possible GA events - GA can benefit from CRA-I input on new government initiatives

15 Outline

• Update on CRA-I

• Breakout groups

• Wrap-up

16 Break Out Group Discussions (11:45am hard stop)

1. Connection between CRA-I and the rest of CRA: We focus on what Board members have to say about how the CRA-I can effectively interact with other CRA committees (like CRA-WP, CERP, CCC, CRA Government Affairs, Awards, etc.) 2. Opportunities and Pain Points in Industry-Academia Interactions in CS departments - The goal is to collect input from board members on questions that CRA-I could focus on with respect to developing best practices (“design patterns”) on issues related to faculty, student, intellectual property, funding, etc. 3. Best practices on using cloud computing as research infrastructure - increasingly research is being conducted using cloud computing infrastructure. This discussion focuses on sharing best practices to leverage cloud infrastructure to accelerate the research process. 4. Data sharing practices - Data is critical for innovation in a data-driven society. What kinds of data can be shared that is of value? How do we share data while keeping rights to it (similar to patenting algorithms)? 5. Social justice – Brainstorm about how to involve industry partners with other CRA efforts on this topic. Consider questions such as: How can we leverage technology to close the injustice gaps? What role does industry and academia have to ensure equality of opportunity? 6. Responsible social media – Consider bringing partners to the table to consider questions such as: What are the technical obstacles to ensuring social media is not harmful? What roles do industry, academia and the government have in ensuring responsible social media? How do we ensure individuals’ rights of freedom of speech is not violated while preventing fake news? 7. Seeding new I/A/G alliances: Example: changing the nature of the semiconductor roadmap - For decades, the semiconductor industry has fueled Moore’s “law” with public private partnerships driven by the SRC. To maintain national leadership, the traditional semiconductor industry needs more research engagement from design and systems companies.

17 Outline

• Update on CRA-I

• Breakout groups

• Wrap-up

18 CRA-Industry Goals for 2021

- Initiate activities that require lower partner investment initially - Target: initiate “Computing Research Industry Pulse” round tables on topics of greatest interest to industry, possibly in partnership with other CRA committees - Initiate 1-2 task forces focusing on particular opportunities for industry/academic/government partnerships - Target: convene in-person meetings around these areas in Spring/Summer 2022 - Engage with companies as potential partners - Expand communications, outreach efforts beyond interviews - Target: populate web site, author “value” white papers by end-2021 - Target: recruit initial set of member companies by February 2022 - Target: coordinate activities with partners around Snowbird and other CRA meetings 19 Proposed Membership/Participants (from draft charter in 2020 report)

• The CRA-I steering committee serves to catalyze and track project-based activities that fulfill CRA-I’s vision and mission. • A CRA-I advisory board will be established, to which each member company will be allowed to nominate up to one board representative. The board is the voice of its constituent members to surface issues and help drive CRA opportunities at the intersection of industry, academia, and government. • Project teams. At the discretion of the steering committee, project teams will be established to execute on CRA-I projects, and can draw on a wider community of participants. Each project team must include at least two CRA-I board members.

20 Summary

• CRA-I has been launched – thanks for your support! • Our approach is start small in 2021 and build a constituency with concrete activities • We plan to start identifying potential champions, members and partners by end-2021 • Stars are lining up for CRA-I: industry needs, CCC activities, CRA Strategic Planning, NSF focus on technology transfer, … • Our ask for you --- let us know how you would like to participate in CRA-I to help achieve our vision • We will also build on input from today’s breakout groups

21