Does Stress Impact Technical Interview Performance?

Total Page:16

File Type:pdf, Size:1020Kb

Does Stress Impact Technical Interview Performance? Does Stress Impact Technical Interview Performance? Mahnaz Behroozi Shivani Shirolkar Titus Barik Chris Parnin [email protected] [email protected] [email protected] [email protected] NC State University NC State University Microsoft NC State University Raleigh, NC, USA Raleigh, NC, USA Redmond, WA, USA Raleigh, NC, USA ABSTRACT 1 INTRODUCTION Software engineering candidates commonly participate in white- Most companies in the software industry administer a technical board technical interviews as part of a hiring assessment. During interview as a procedure for hiring a software developer [3, 49]. these sessions, candidates write code while thinking aloud as they Companies believe technical interviews offer a “reasonably consis- work towards a solution, under the watchful eye of an interviewer. tent evaluation of problem-solving ability, communication skills, While technical interviews should allow for an unbiased and inclu- and preparedness” [75]. Technical interviews can also give visibility sive assessment of problem-solving ability, surprisingly, technical to the personality of the candidate, how they interact with their interviews may be instead a procedure for identifying candidates future colleagues and how much they pay attention to details—such who best handle and migrate stress solely caused by being examined as checking all possible test cases—and knowledge of programming by an interviewer (performance anxiety). languages. Companies also find it important to reduce unwanted To understand if coding interviews—as administered today—can stress [8] during a technical interview, as “not everyone does their induce stress that significantly hinders performance, we conducted best work in fast-paced, high-pressure situations” [52]. In principle, a randomized controlled trial with 48 Computer Science students, companies should expect the technical interview process to be a comparing them in private and public whiteboard settings. We sound and fair assessment of a candidate’s ability, and thus yield a found that performance is reduced by more than half, by simply steady stream of qualified candidates for positions that open upin being watched by an interviewer. We also observed that stress and the company. cognitive load were significantly higher in a traditional technical in- Technical interviews can also introduce other effects on can- terview when compared with our private interview. Consequently, didates who report unexpectedly “bombing” [50], “freezing” [67], interviewers may be filtering out qualified candidates by confound- or “choking” [59] during this critical hiring procedure. Through a ing assessment of problem-solving ability with unnecessary stress. happy accident, the software industry has seemingly reinvented We propose interview modifications to make problem-solving as- a crude yet effective instrument for reliably introducing stress in sessment more equitable and inclusive, such as through private subjects, which typically manifests as performance anxiety [78]. A focus sessions and retrospective think-aloud, allowing companies technical interview has an uncanny resemblance to the trier social to hire from a larger and diverse pool of talent. stress test [41], a procedure used for decades by psychologists and is the best known “gold standard” procedure [1] for the sole pur- CCS CONCEPTS pose of reliably inducing stress. The trier social stress test involves • Software and its engineering; • Human-centered comput- having a subject prepare and then deliver an interview-style pre- ing ! Empirical studies in HCI; sentation and perform mental arithmetic, all in front of an audience. Alone, none of these actions consistently induce stress in a subject; KEYWORDS however, the unique combination of cognitive-demanding tasks with a social-evaluative threat (essentially being watched) is consis- cognitive load, eye-tracking, program comprehension, technical tent and powerful. If a technical interview is essentially a de facto interviews version of a trier social stress test, then the implications can be ACM Reference Format: profound. Rather than measuring the few that answer correctly in a Mahnaz Behroozi, Shivani Shirolkar, Titus Barik, and Chris Parnin. 2020. timely manner, companies are most likely measuring the ability of Does Stress Impact Technical Interview Performance?. In Proceedings of the the few who perform well under stress [31]. Rather than measuring 28th ACM Joint European Software Engineering Conference and Symposium explanation skills, companies are most likely measuring the ability on the Foundations of Software Engineering (ESEC/FSE ’20), November 8– of candidates to handle or mitigate stress (e.g. through practice [58]). 13, 2020, Virtual Event, USA. ACM, New York, NY, USA, 12 pages. https: Finally, rather than avoiding unwanted stress, technical interviews //doi.org/10.1145/3368089.3409712 may be inadvertently designed with the sole purpose of inducing it. To understand how to maintain the desirable goals of a technical Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed interview (e.g. measure time and correctness), while mitigating for profit or commercial advantage and that copies bear this notice and the full citation the undesirable effects of stress, we created a design probe[77] on the first page. Copyrights for components of this work owned by others than the which removed the social-evaluative threat component of a techni- author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission cal interview. To this end, we designed an interview format where and/or a fee. Request permissions from [email protected]. participants privately solved a technical problem on a whiteboard, ESEC/FSE ’20, November 8–13, 2020, Virtual Event, USA without any monitoring or interaction with an interviewer. We © 2020 Copyright held by the owner/author(s). Publication rights licensed to ACM. ACM ISBN 978-1-4503-7043-1/20/11...$15.00 evaluated this format in a randomized controlled trial to under- https://doi.org/10.1145/3368089.3409712 stand the impact of stress during a technical interview and whether ESEC/FSE ’20, November 8–13, 2020, Virtual Event, USA Mahnaz Behroozi, Shivani Shirolkar, Titus Barik, and Chris Parnin we could isolate and dampen its influence. We then compared this constructing data structures, implementing search and sorting al- to a typical interview format, where a candidate used talk-aloud gorithms, and characterizing running time and space requirements. to explain their problem-solving in front of a proctor. Participants Furthermore, students represent potential technical interview can- wore specialized eye-tracking glasses, which allowed us to obtain didates, as many were actively engaged in their initial job search measurements associated with high cognitive load and stress. We prior to graduation. To use head mounted eye-trackers as a part of then compared the correctness, self-reported experiences, and cog- our study, participants had to have normal or corrected-to-normal nitive load and stress levels of participants across the interview vision. They received extra course credit through an experiment settings. pool and could end the experiment at any time. The study was We found that participants reported high levels of stress and approved by the local ethics board. The first and second authors of often had difficulty coping with someone else being present while the paper conducted the study. they attempted to solve a cognitively-demanding task. The impact To estimate our population size for our study, we used the results on performance was drastic: nearly twice as many participants of our pilot study to obtain estimators for effect size. Through a failed to solve the problem correctly, and the median correctness power analysis of 2-sample 1-sided test with 80% power and 5% score was cut more than half, when simply being watched. In con- Type I error [19], we found that approximately 18 participants trast, participants in the private setting reported feeling at ease, would be required in each group. having time to understand the problem and reflect on their solution. Finally, measures of cognitive load and stress were significantly 2.2 Pilots and Tasks lower in the private setting, and majority of participants solved We ran pilot studies at the authors’ institutions. During our pilots, the problem correctly. We also observed that no women success- we experimented with tasks that would satisfy the following criteria: fully solved the problem in the public setting, whereas all women be solvable within the time limits of the experiment, demonstrate solved it correctly in the private setting. Standard NASA-TLX pro- sufficient difficulty such that cognitive load can be induced andis cedure [35] and various cognitive metrics, provided evidence that not trivial to solve, and have ecological validity—the problem should higher rates of stress could causally explain these differences across be similar to those used in actual interviews. Based on feedback and groups. observations during the pilot studies, we found some tasks were We suggest that private interview settings have considerable too complex, where participants would quickly give up after a few advantages for candidates, that both reduce their stress and allow minutes, and some tasks too ambiguous,
Recommended publications
  • Google Interview Prep Guide Software Engineer
    Google Interview Prep Guide Software Engineer What’s a Software Engineer (SWE)? Software Engineers (referred to as “SWEs”) at Google develop the next-generation technologies that change how millions of users connect, explore and interact with information and one another. As a SWE, you’ll be responsible for the whole lifecycle of a project critical to Google’s needs, with opportunities to switch teams and projects as you and our fast-paced business grow and evolve. Depending upon the project you join, you could be involved in research, design, planning, architecture, development, test, implementation and release phases. You'll be working on products that handle information at a massive scale, bringing fresh ideas from all areas and tackling new problems across the full-stack as we continue to push technology forward. Why Google? Impact. Google is and always will be an engineering company. We hire people with a broad set of technical skills who are ready to tackle some of technology's greatest challenges and make an impact on millions, if not billions, of users. At Google, engineers not only revolutionize search, they routinely work on massive scalability and storage solutions, large-scale applications and develop entirely new platforms around the world. From AdWords to Chrome, Android to YouTube, Cloud to Maps, Google engineers are changing the world one technological achievement after another. General Interview Tips Explain: We want to understand how you think, so explain your thought process and decision making ​ throughout the interview. Remember we’re not only evaluating your technical ability, but also how you approach problems and try to solve them.
    [Show full text]
  • Founder/Ceo, Careercup
    GAYLE LAAKMANN MCDOWELL FOUNDER/CEO, CAREERCUP I can help you create or refine your hiring process. With my deep background in both software development and hiring, I have the technical and recruiting knowhow to work with (and earn the respect of) your technology, HR, and other teams. • Author of Cracking the Coding Interview (#1 best-selling interview book), Cracking the PM Career, Cracking the PM Raise the Bar & Interview, and Cracking the Tech Career. Hire More Tech Talent • Software engineer at Google, Microsoft, and Apple. Google hiring committee & top 1% of most active interviewers. My Philosophy • Acquisition interview consultant: coached dozens of companies through acquisition interview process. Hire Smart • BSE / MSE in Computer Science from University of Smart employees produce high Pennsylvania. MBA from the Wharton School. quality work, make good decisions, learn new skills quickly, and adapt My Goals for Your Company well to company growth and needs. High & Consistent Bar Happy Candidates Hire People Who Care When desperate to hire, Good candidate experiences Your best employees will care— some teams drop their bar a matter. If your process leaves about getting things done, about little or make excuses for a a bad taste in people’s creating a great product, about candidate. Those weak hires mouths, the strong doing the right thing. can cost you in the long run. candidates won’t take your offer and the weak Hire Good Employees, Not Adaptable Process candidates ones will tell their Good Candidates friends not to. Your process If your process isn’t adaptable I’ve coached many candidates. The should make candidates want to future roles, you need to more I can coach someone, the to become employees.
    [Show full text]
  • Google Interview Prep Guide Engineering Manager
    Google Interview Prep Guide Engineering Manager What’s an Engineering Manager (EM)? Engineering Managers not only have the technical leadership to roll up their sleeves and provide technical leadership to major projects, they also manage a team of engineers. As an EM, you’ll lead your team to shape and deliver Google products and not only optimize your own code but make sure engineers are able to optimize theirs. You’ll manage your project goals, contribute to product strategy, drive architecture decisions and design internet-scale solutions to resolve challenging technology problems for our customers, in an effort to help develop your team. With extensive technical and leadership expertise, you’ll manage engineers across multiple teams and locations, a large product budget and oversee the deployment of large-scale projects across multiple sites internationally. Operating with speed, our world-class software engineers are just getting started — and as a manager, you guide the way. Why Google? Impact. Google is and always will be an engineering company. We hire people with a broad set of technical skills who are ready to tackle some of technology's greatest challenges and make an impact on millions, if not billions, of users. At Google, engineers not only revolutionize search, they routinely work on massive scalability and storage solutions, large-scale applications and develop entirely new platforms around the world. From AdWords to Chrome, Android to YouTube, Cloud to Maps, Google engineers are changing the world one technological achievement after another. General Interview Tips Explain: We want to understand how you think, so explain your thought process and decision making ​ throughout the interview.
    [Show full text]
  • Interlibrary Loan Using the World Share Among Academic Libraries in Thailand by Using a Mixed Research Method
    สภาพและการประเมินความต้องการจ าเป็นต่อบริการยืมระหว่างห้องสมุดเวิลด์แชร์ ของห้องสมุดมหาวิทยาลัยในประเทศไทย THE CONDITIONS AND NEEDS ASSESSMENT FOR WORLDSHARE INTERLIBERY LOAN SERVICE OF ACADEMIC LIBRARIES IN THAILAND ์ ชยั สิทธิ องั คะ ปัญญาเดช บัณฑิตวิทยาลัยมหาวิทยาลัยศรีนครินทรวิโรฒ 2561 สภาพและการประเมินความต้องการจ าเป็นต่อบริการยืมระหว่างห้องสมุดเวิลด์แชร์ ของห้องสมุดมหาวิทยาลัยในประเทศไทย ์ ชยั สิทธิ องั คะ ปัญญาเดช ปริญญานิพนธ์นี้เป็นส่วนหนึ่งของการศึกษาตามหลักสูตร ศิลปศาสตรมหาบัณฑิต สาขาวิชาสารสนเทศศึกษา คณะมนุษยศาสตร์ มหาวิทยาลัยศรีนครินทรวิโรฒ ปีการศึกษา 2561 ์ ลิขสิทธิของมหาวิทยา ลยั ศรีนครินทรวิโรฒ THE CONDITIONS AND NEEDS ASSESSMENT FOR WORLDSHARE INTERLIBERY LOAN SERVICE OF ACADEMIC LIBRARIES IN THAILAND CHAIYASIT ANGKAPUNYADECH A Thesis Submitted in partial Fulfillment of Requirements for MASTER OF ARTS (Information Studies) Faculty of Humanities Srinakharinwirot University 2018 Copyright of Srinakharinwirot University ปริญญานิพนธ์ เรื่อง สภาพและการประเมินความต้องการจ าเป็นต่อบริการยืมระหว่างห้องสมุดเวิลด์แชร์ ของห้องสมุดมหาวิทยาลัยในประเทศไทย ของ ชยั สิทธิ์ องั คะปัญญาเดช ได้รับอนุมัติจากบัณฑิตวิทยาลัยให้นับเป็นส่วนหนึ่งของการศึกษาตามหลักสูตร ปริญญาศิลปศาสตรมหาบัณฑิต สาขาวิชาสารสนเทศศึกษา ของมหาวิทยาลัยศรีนครินทรวิโรฒ คณบดีบัณฑิตวิทยาลัย (ผู้ช่วยศาสตราจารย์ นายแพทย์ฉัตรชัย เอกปัญญาสกุล) คณะกรรมการสอบปากเปล่าปริญญานิพนธ์ .............................................. ที่ปรึกษาหลัก .............................................. ประธาน (รองศาสตราจารย์ ดร.แววตา เตชาทวีวรรณ) (รองศาสตราจารย์ ดร.จุฑารัตน์ ศราวณะวงศ์) .............................................
    [Show full text]
  • Cracking the Coding Interview
    CRACKING THE CODING INTERVIEW SPEAKER: Gayle Laakmann McDowell DATE: 21-01-2014, 20.00 hours VENUE: LT-4 CAMPUS: BITS- Pilani, K K Birla Goa Campus. ABSTRACT: The theme of this lecture was to make students aware of what the recruiters abroad actually expect during a coding interview. She also stressed upon the format and importance of a resume. And most of all she advised students upon getting hands on experiences, do projects and clear their basic concepts. This interactive session covered: 1. The basic aspects of an interview. 2. Expectations of an interviewer. 3. Resume making and format. 4. Interview questions and topics. 5. CGPA requirements and importance of projects. 6. Coding the right way. SPEAKER BIOGRAPHY: Gayle Laakmann McDowell is a post-graduate in Computer Science from University of Pennsylvania and did her MBA from the Wharton School of Business, where she specialized in entrepreneurship. An author of two bestselling books Cracking the Coding Interview and The Google Resume, she is also a founder of CareerCup and Seattle Unfreeze. She has worked as a software engineer and recruiter in many tech companies like Apple, Microsoft and Google. SUMMARY: The main motive of talk was to answer the common question that keeps bugging students, how to crack a coding interview? Having worked as a recruiter herself she could answer this question from the point of view of recruiter. She started the lecture telling how a resume should look like. According to her a resume should be short, organized and a grammatically correct one. She pointed out that becauseof difference in Indian GPA system and that in the foreign universities, it becomes difficult for the recruiters abroad to analyze and hence, she recommended that one must put try filling their resume with lots of experiences and also translate GPA.She also told that comparison between candidates is inevitable.
    [Show full text]
  • M.S. Career Opportunities & Job Search
    MS Career Opportunities and Job Search Gonzalo Ramos, Microsoft Research Jaime H. Moreno, IBM Research (…and thanks to authors of similar presentations from previous workshops!) 2020 CRA URMD Grad Cohort Workshop Agenda • Introductions • Possible career paths with an MS degree • Preparing for the job market • Career growth and advancement • Preparing for interviewing Gonzalo Ramos – [email protected] From Argentina Principal Researcher – Microsoft Research AI - Redmond before that… – Senior UX Scientist – Amazon Grand Challenges (2015-1016) – Senior Design Technologist – Amazon Concept Lab (2013-2015) – Scientist, Microsoft Live Labs; OSD (2007-2013) – MS, Ph.D. in CS, University of Toronto, Canada Jaime H Moreno – [email protected] • Originally from Chile • Current role – Distinguished Researcher, Senior Manager, IBM TJ Watson Research Center, NY – Recent completed project: IBM Summit and Sierra supercomputers (#1, #2 in Top500) – Current focus: Cloud Infrastructure • Previously (although long ago ..) – Faculty Member, University of Concepcion, Chile • Computer Science, PhD, MS, UCLA • Electrical Engineer, University of Concepcion, Chile PRELIMINARIES Who is in the audience? • How many currently in Master Degree program? – Course or Thesis program? • Who will be graduating this spring? – Do you already have plans for after graduation (other than a well-deserved vacation..??) Exercise: Turn And Talk To Your Neighbor A. What is your plan after graduation? - E.g., look for job in what area/industry, enroll in Ph.D., … B. What do you
    [Show full text]
  • CS 395: Technical Interview Preparation Fall 2021
    CS 395: Technical Interview Preparation Fall 2021 Contact Information Facilitators: Andi Motas - [email protected] Brenda Henriquez - [email protected] Faculty Advisor: Shvetha Soundararajan - [email protected] Course Description CS 395: Student Initiated Special Topics are 1-credit courses that cover special and emerging topics of interest to computer science undergraduates. Lectures are guided by student facilitators under faculty advisement. This course will prepare students to perform well in technical interviews for CS-related positions. We will review technical concepts such as Big O analysis, data structures, and search/sort algorithms with the context of solving interview code questions. This course will also go over preparation steps and soft skills for standing out as a candidate. Students will develop and evaluate their understanding through problem-solving exercises, assigned readings with quizzes, lecture, discussions, mock interviews, and a final exam. This is a 10 week course. Course Outcomes Upon completion of this course, students should: ● Be able to approach technical interview questions and articulate their thought process. ● Demonstrate a strong understanding of key data structures and algorithms that can be applied to solving technical interview problems. ● Be able to analyze and discuss runtime complexity and trade-offs. ● Be able to effectively communicate their experience and skills during a technical interview. ● Have confidence in technical interview performance from practicing in mock interviews. Prerequisite Required
    [Show full text]
  • Five Interview Phases to Identify the Best Software Developers
    WHITE PAPER Five Interview Phases To Identify The Best Software Developers by Gayle Laakmann McDowell Author of Amazon Bestseller “Cracking the Coding Interview” Consultant for Acquisitions & Tech Hiring Former Developer (Google, Microsoft, Apple) INTRODUCTION Want to know the There may not be a perfect set of interview questions to find A+ perfect interview programmers, but there is a basic flow you can use to maximize the questions for efficiency of your screening process and end up with developers? We do, too, but unfortunately top tier programming talent. I’ve approached this as a there is no such thing bottom-up approach, where you gradually screen out as “the perfect” set of interview questions. unqualified candidates quickly and easily, and then end up with a small set of great candidates. There The questions you are five phases in this bottom-up approach and ask will vary based on your organization, the ultimately the hiring manager can decide what need, and the role. phase they want to begin. FIVE INTERVIEW PHASES TO IDENTIFY THE BEST SOFTWARE DEVELOPERS 2 PHASE 1 Interview Yourself It may seem silly, but a lot of organizations skip this crucial pre-phase. Before you start interviewing people, you need to interview yourself about the role and its needs. You need to ask yourself: • What skills do you want? • Why do you want those skills? Are they truly required? • What will it take to get that skillset? • Can you do without some of these skills? Many companies ask the first question, but don’t sufficiently explore the second, third, or fourth.
    [Show full text]
  • Ms Career Opportunities and Job Search
    MS CAREER OPPORTUNITIES AND JOB SEARCH Patty Lopez, Intel Corporation Jennifer Hill, Capital One Agenda • Possible career paths with an MS degree • Finding an industry position • How to present yourself • Writing a resume that attracts attention, the importance of your LinkedIn profile • The interview (before, during & after) • Career growth & advancement What can you do now to prepare for your job search? What • Attend technical conferences and workshops • Use your professional network • Maintain an updated LinkedIn profile • Participate in hackathons, bug bounties and company campus visits • Attend career fairs • Use recruiters • Search broadly Where can I get a job? Where Many tech jobs are There are many positions for not in tech companies Master graduates: • Government • Developer • Program/Product Manager • Telecommunications • User Experience Designer / • Education Researcher • Healthcare... • Data Scientist • Product Evangelist • Consultant • Research Scientist (go on to Grad School – yay!) Organization Marketing Finance Sales Legal/HR Product Operations / Fabs (SW/HW) Organization → Product Software Datacenter Chrome & Android Client Internet of Things Organization → Product → Function Product Managers UX Engineering Marketing Organization → Product → Function → Role Prototyper Visual Designer Design Interaction Producer Motion Designer Designer UX UX Writer Researcher Apps Legal Big to mid-size Start-up Freelance company Scale Higher risk Range of work Resources Role breadth Career freedom Specific role Less bureaucracy Less
    [Show full text]
  • A Theory on Individual Characteristics of Successful Coding Challenge Solvers
    A theory on individual characteristics of successful coding challenge solvers Marvin Wyrich, Daniel Graziotin and Stefan Wagner Institute of Software Technology, University of Stuttgart, Stuttgart, Germany ABSTRACT Background. Assessing a software engineer's ability to solve algorithmic programming tasks has been an essential part of technical interviews at some of the most successful technology companies for several years now. We do not know to what extent individual characteristics, such as personality or programming experience, predict the perfor- mance in such tasks. Decision makers' unawareness of possible predictor variables has the potential to bias hiring decisions which can result in expensive false negatives as well as in the unintended exclusion of software engineers with actually desirable characteristics. Methods. We conducted an exploratory quantitative study with 32 software engineering students to develop an empirical theory on which individual characteristics predict the performance in solving coding challenges. We developed our theory based on an established taxonomy framework by Gregor (2006). Results. Our findings show that the better coding challenge solvers also have better exam grades and more programming experience. Furthermore, conscientious as well as sad software engineers performed worse in our study. We make the theory available in this paper for empirical testing. Discussion. The theory raises awareness to the influence of individual characteristics on the outcome of technical interviews. Should the theory
    [Show full text]