Manuscript submitted to International Journal of Human–Computer Interaction
Autonomous Taxi Service Design and User Experience
Sangwon Kim1, Jennifer Jah Eun Chang1, Hyun Ho Park1, Seon Uk Song1, Chang Bae Cha1, Ji Won Kim1, Namwoo Kang2,*
1Korea Advanced Institute of Science and Technology (KAIST), Korea 2Sookmyung Women’s University, Korea
* Corresponding author: [email protected]
ABSTRACT
As autonomous-vehicle technologies advance, conventional taxi and car sharing services are being combined into a shared autonomous vehicle service, and through this, it is expected that the transition to a new paradigm of shared mobility will begin. However, before the full development of technology, it is necessary to accurately identify the needs of the service’s users and prepare customer-oriented design guidelines accordingly. This study is concerned with the following problems: (1) How should an autonomous taxi service be designed and field- tested if the self-driving technology is imperfect? (2) How can imperfect self-driving technology be supplemented by using service flexibility? This study implements an autonomous taxi service prototype through a Wizard of Oz method. Moreover, by conducting field tests with scenarios involving an actual taxi, this study examines customer pain points, and provides a user-experience-based design solutions for resolving them.
Keywords: Autonomous Taxi, Car Sharing, Service Design, User Experience, Wizard of OZ
1. INTRODUCTION
Similar to conventional automobile companies and IT companies, car-sharing service providers such as Uber and Lyft are accelerating the commercialization of autonomous vehicles (TechWorld, 2018). Although there are some differences, most companies are aiming to create autonomous vehicles of level 4 or higher based on the standard of the Society of Automotive Engineers (SAE) by 2020 (Business Insider, 2016). Such an advancement of autonomous driving technology is expected to transform the current transportation system into a shared mobility system and give rise to various types of business models and services (Stocker & Shaheen, 2017). Conventional taxis and car-sharing services will be combined into a shared autonomous vehicle (SAV) service. In the case of public transportation, where buses and minivans travel on certain routes and load a large number of passengers, it is expected that a service operating ride-sharing SAV on certain routes or detours will be provided. However, the actual time at which autonomous vehicles will be deployed in the sharing service market is expected to be after 2040, and this differs from the technology development target dates set by companies (Litman, 2017).
Presently, sharing services using autonomous vehicles have only been provided in certain areas. In 2016, nuTonomy announced the launch of the first autonomous taxi service in Singapore (Forbes, 2016). Recently, Waymo, an autonomous vehicle company owned by Google, revealed that Early Riding Program, an autonomous taxi service, would be operated for the employees of Valley Metro, a public transportation company based in Phoenix, USA, for two years starting from August 2018 (The Verge, 2018). Nissan of Japan conducted an autonomous taxi service test using an app in Yokohama in March 2018 (Allianz Partners, 2018).
The adoption of SAV services is expected to have large economic, social, and environmental impact. According to the report of McKinsey & Company on 2016 (Gao, Kaas, Mohr, & Wee, 2016), new business models driven by shared mobility, connectivity services, and feature upgrades, could expand automotive revenue pools by 30%, adding up to USD 1.5 trillion by 2030. Simulation studies demonstrated that if the existing taxis in New York City were replaced by autonomous vehicles, the waiting time of passengers would be reduced by 29.82% (Shen & Lopes, 2015), and if an autonomous taxi service including ride-sharing was implemented in Stockholm, only 5% of currently existing automobiles would be needed (Burghout, Rigole, & Andreasson, 2015) Furthermore, it was shown that in Ann Arbor, an autonomous electric vehicle sharing service would greatly reduce greenhouse gas (GHG) emissions as well as the social cost of carbon, and would be highly profitable for the service providers (Kang, Feinberg, & Papalambros, 2017).
SAV services have their disadvantages, however. First, there is always the issue of safety. Even if technology advances, the successful market establishment of SAV services depends on customers’ perception of their safety (Tussyadiah et al., 2017). Moreover, a legal system supported by an ethical agreement that clarifies the responsible parties in case of an accident and a clear claims procedure is needed, but the social costs to reach such a consensus may be high. Second, there are environmental issues. If the vehicles use conventional engines, there are debates on whether autonomous vehicles are indeed environmentally friendly. A development of autonomous vehicles may increase the number of vehicles without drivers, which can in effect emit more gas compared to other means of mass transit. Third, transportation infrastructure such as ones in smart cities ought to be established beforehand for the services to be provided stably. Fourth, as in the simulation results, a decrease in the number of individually-owned cars can lead to longer wait time for SAV services, decreasing actual convenience. The price should be lower to compensate for such inconvenience, but this may make it difficult for firms to recoup their investment costs. Lastly, since autonomous vehicles go without drivers, there is a difficulty of engaging in maintenance in case of unexpected breakdowns or maintaining cleanliness of vehicles.
Amid these conflicting views of SAV services, the aim of this study is to design an autonomous taxi service, i.e., one of SAV service, from the customer perspective, and analyze user experience (UX) through prototyping and a field study. Regarding SAV users, studies on preference and acceptability have been conducted through surveys (Krueger, Rashidi, & Rose, 2016; Abraham et al., 2017). Regarding product/service design, several studies have been conducted on human–computer interaction (HCI) and human–machine interface (HMI), i.e., interaction in the autonomous vehicle system, in addition to the studies on design elements regarding user awareness and perception of autonomous vehicle systems (Kyriakidis et al., 2017; Politis et al., 2017; Du, Qin, Zhang, Cao, & Dou, 2018; Lee, Kim, Lee, & Shin, 2015; Strömberg et al., 2018; Rothenbücher, Li, Sirkin, Mok, & Ju, 2016; Kim et al., 2017).
In terms of goal and methodology, the studies most similar to the present are as follows. Strömberg et al. (2018) conducted a study on the interaction between autonomous vehicles and humans, and using this, several design methods such as Wizard of Oz methods, small-scale scenarios, design metaphors, enactment, and peer-to-peer interviews, have been devised. Rothenbücher, Li, Sirkin, Mok, and Ju (2016) investigated the reactions of pedestrians and bicycle riders to autonomous vehicles, and examined the reactions of users to real autonomous vehicles by using the Wizard of Oz method in which a person was actually driving, but they could not be seen from the outside owing to a special sheet. Kim et al. (2017) tested on-campus autonomous taxi service by using a real self-driving car and demonstrated the performance feasibility of autonomous vehicles. However, these conventional studies have not tested the interaction between autonomous vehicles and users from the “sharing service” viewpoint, and could not adequately identify elements necessary for designing the service itself.
1.1. Autonomous Taxi Service Design Issues
To provide a big picture of driverless taxi services in general, this section will first explain the differences between autonomous taxis and manned taxis through a customer journey map and touch points. We will also introduce some potential issues to be discussed when designing services for driverless taxis.
(Figure 1 about here)
Figures 1 (a) and (b) show the customer journey map and touch points of a manned taxi and an autonomous taxi, respectively. The touch points for the manned taxi involve the passenger and taxi driver, whereas those for the autonomous taxi were obtained by considering that the taxi driver is replaced by the automated process. Through the touch point design, the issue points that should be considered in the changes to autonomous taxis were obtained. Brainstorming was performed for the possible scenarios that can occur at each issue point, and Table 1 summarizes the major scenarios. These factors should be taken into consideration in a detailed service design.
(Table 1 about here)
1.2. Research Questions
Two major research questions are addressed in this study. The first is “How is an autonomous taxi service designed and field-tested if the autonomous driving technology is imperfect?” This is not merely a technical question. In South Korea, there is no legal provision for operating autonomous vehicles. Therefore, this study analyzes the needs of potential customers through Wizard of Oz methods, in which participants believe that they are actually using an automated service, but in reality, an experimenter is operating the automated system in the background (Dahlbäck, Jönsson, & Ahrenberg, 1993; Maulsby, Greenberg, & Mander, 1993). Thereby, the customer experiences are tested for an autonomous driving taxi service, and the requirements for customer-oriented service design are obtained.
The second question is “How can imperfect autonomous driving technology be supplemented by using service flexibility?” In this study, through the concept of virtual taxi stand, a method of providing service using limited autonomous driving technology is proposed and verified. A virtual taxi stand refers to a position where a passenger may board or disembark from an autonomous taxi. This can be understood as an intermediate concept between a bus and a taxi. Instead of a taxi stand at a fixed location, the optimal location for boarding/disembarkation is provided as a virtual taxi stand, which is selected among the places where the autonomous taxi can travel safely.
As shown in Figure 2, two scenarios can be considered according to the autonomous driving technology level. In Scenario A, as in the case of a manned taxi, an autonomous taxi can drive to the exact location where the customer wants to board/disembark. In Scenario B, a customer comes to a virtual taxi stand, boards an autonomous taxi, and finally disembarks at the virtual taxi stand closest to the desired destination.
(Figure 2 about here)
The aims of this study are to design a virtual taxi-stand-based autonomous taxi service (Scenario B) and to obtain customer experiences by testing a prototype. Thereby, customer pain points can be examined prior to launching the service.
Autonomous taxi services that utilize virtual taxi stands inevitably reduces flexibility compared to that offered by taxis with drivers. In spite of this, the reason why the current study adopted the virtual taxi stand scenario is to accumulate service experience data and preoccupy the market for future launching of autonomous taxis when an ideal level of technology is adopted. Once the autonomous driving technology reaches a certain level, virtual taxi stands will naturally be no longer needed. Until then, autonomous taxis will have to compete with manned taxis by emphasizing their advantages, and among those, the current study proposes various ways to increase UX satisfaction. For clarification, note that virtual taxi stands are different from designated spots occupied by short-term rental companies like ZipCar. Designated spots are fixed spaces usually in parking lots, but virtual taxi stands are placed next to the roads and are customized spots that are changeable.
1.3. Research Framework
(Table 2 about here) The entire research process and the methodologies used for each process are shown in Table 2. In the service design stage, to analyze the conventional manned taxi service and derive expected scenarios when the autonomous taxi service is adopted, customer journey mapping and touchpoint analysis are performed. Subsequently, a service blueprint is accordingly designed, and the issues that require attention in each service stage are examined. Customer journey mapping is a method for defining the process through which customers experience the service, and for visualizing customer experience during the process (Lemon & Verhoef, 2016). Touchpoint analysis is a method for defining and visualizing the contact points where a customer interacts with the provider during the service (Clatworthy, 2011). Service blueprint is an advanced design tool including the customer journey map and touchpoint analysis; it describes the relationships between the service elements in detail. It is the most widely used method in service design and product-service system design (Shostack, 1982, Fließ & Kleinaltenkamp, 2004, Geum & Park, 2011).
In the prototyping stage, the autonomous taxi service is implemented by using the Wizard of Oz methodology. The driver’s seat is blocked with partition panels so that the passenger is unable to see or communicate with the driver; thus, service information is received and requests are executed only through a display installed in the vehicle. In the control center, the customer behaviors are recorded in real time using cameras installed inside the vehicle. The vehicle is controlled by communicating with the driver, whereas voice and touch recognition are controlled through a display in the vehicle.
In the test design stage, a survey/interview method is developed to obtain the UX effectively by using Design Thinking, and specific autonomous taxi service experiment scenarios are designed. Design Thinking is a methodology that defines the design problem and derives solutions based on the empathy for the customers, and then tests users using a prototype (Kelley & Kelley, 2013).
In the test stage, an online survey is conducted before and after the service test, and one- to-one in-depth interviews are carried out. Moreover, through the recorded video, the customer behaviors are further analyzed. The service test was conducted in two steps: a pilot test in the first step and a main test in the second step. The pilot test results were used to supplement the main test design.
The remainder of this paper is organized as follows. In Section 2, the service design process and the prototyping of the autonomous taxi system are introduced. In Section 3, the process of designing the service test method is described, and in Section 4, the results of the service test are discussed. In Section 5, the analysis of the results and the insights for service developers are summarized, and in Section 6, the study is summarized and future research is presented.
2. SERVICE DESIGN AND PROTOTYPING
2.1. Service Design
To design an autonomous taxi service, it is necessary to analyze the current manned taxi service. In this study, to analyze the Kakao Taxi service, which is a mobile app taxi service in South Korea, videos were recorded (with the consent of the driver) for the entire process, i.e., from waiting for the service call to performing and terminating the service. Furthermore, an in- depth interview was conducted with the taxi driver. By subdividing this information into passenger routes, taxi routes, supporting process, and additionally required support factors, the customer journey map and touch points were analyzed, and the service blueprint for the manned taxi service was produced. Subsequently, by predicting the processes that will be added or deleted when an autonomous taxi service is adopted (in comparison with the manned taxi service), the corresponding customer journey map, touch points, and service blueprint were obtained. Results of customer journey map and touch points are presented in Section 1.1.
Through the service blueprint for manned taxis, the taxi service was divided into five stages: (1) calling the taxi, (2) pick-up, (3), travelling, (4) drop-off, and (5) preparing for the next service. Based on this, the blueprint for autonomous taxi service was designed as shown in Figure 3. Specifically, the blueprint comprises classifying (1) the passenger, (2) the autonomous taxi, (3) the supporting processes, and (4) additionally required support factors. Among the touch points (T) (i.e., instances of service–customer interaction), those involving problems occurring at high frequency are defined as issue points (I), whereas those that should be considered for safe service operation (although they involve problems occurring at lower frequency) are defined as exception points (E). The test (Section 3) was primarily designed to obtain the UX regarding the issue points.
(Figure 3 about here)
2.2. Service Prototyping
To perform an effective service test, the test subject should be provided with an experience almost identical to that of a real autonomous taxi service. However, because there is no regulation in South Korea that facilitates testing of autonomous vehicles on regular roads targeting ordinary persons, the Wizard of Oz methodology was used as an alternative solution. The setting was configured so that the test participants would believe that they were using a self-driving taxi, and would undergo all the processes of calling a taxi, boarding, travelling, and disembarking. The passengers were notified that a driver was in the car for safety, and the driver was in a partition that blocked the passengers from seeing the driver. A passenger sat on the right side of the back seat, and as the behavior of the driver could not be observed, interactions were performed only through voice recognition and a display installed inside the taxi. In the control center, the behavior of the passenger was recorded and observed through cameras (GoPro and cellphone) installed inside the taxi, and instructions were given to the driver for necessary actions.
Figure 4 shows the actual internal and external environment of the test vehicle. The most important factor in the internal environment is that the passenger is not aware of the driver driving the vehicle, so that he/she perceives the taxi as a self-driving vehicle. To this end, the partition panels of driver seat was fabricated by considering the following factors: (1) the driver cannot be seen from inside the vehicle; (2) as a passenger boards the car, the driver cannot be seen; (3) the driver can see the outside of the vehicle through the front glass, side mirrors, and rearview mirror so that he can drive safely; and (4) installation and dismantlement are possible, and the exterior appearance provides the feeling of an autonomous taxi.
(Figure 4 about here)
(Figure 5 about here)
(Figure 6 about here) In the control center, where all the interactions and test processes of the autonomous taxi are controlled, the internal setting of autonomous taxi is monitored (see Figure 5). The interaction between the control center and a test subject is performed through the user’s smartphone and the display (a tablet display) installed inside the vehicle (see Figure 6). Furthermore, to monitor the test at the control center, a smartphone is installed inside the vehicle to observe the passenger, and another smartphone for observing the situation on the road. In the control center, three notebook computers are linked in real time with (1) the display in the vehicle, (2) the smartphone for observing the passenger, and (3) the smartphone for observing the situation on the road, so that the control center can check the test status and change the display screen. For these communications, a portable Wi-Fi device is used. For real- time screen sharing, the commercial application TeamViewer is used. The screen on the tablet and the voice announcement system are all controlled at the control center, and the actions and utterances of the customer are responded to in real time so that he/she may have the impression that the service is in fact an artificial intelligence service.
3. TEST DESIGN
This section introduces the design of the test for “what will be evaluated by prototyping?” (Section 3.1) and “which scenario will be used for evaluation?” (Section 3.2). In addition difference between the pilot and main tests are summarized in Section 3.3.
3.1. Interview and Survey Design
The test is designed in four steps as shown in Table 3. For quantitative testing, a survey was conducted through a laptop, and for qualitative testing, an in-depth interview and video analysis were performed. The questions of the qualitative interview and the survey were designed by applying the Design Thinking methodology, and the content of the main test was determined based on the feedback obtained from the pilot test.
(Table 3 about here)
Interview Design
The pre-test and post-test interviews were designed through brainstorming and idea grouping for one-to-one in-depth interview questions, as shown in Table 4. A test subject calls the autonomous taxi after having taken the pre-test interview, and the post-test interview is conducted after arrival at a destination. The pre-test and post-test interview guidelines were used, and different questions were asked to every test participant according to the interview atmosphere and answers, so that in-depth experiences and thoughts can be learned from the users.
(Table 4 about here)
An emotion curve is a method of relatively expressing the degree of neutral, negative, and positive emotions for each service stage. This was used as reference data for the post-test interview and an aid for examining the user emotions regarding the service. Figure 7 shows an example of an emotion curve. (Figure 7 about here)
Survey Design
For user satisfaction in each service stage (call, pick-up, travelling, and drop-off), a 7-point Likert scale quantitative test was performed using the evaluation factors shown in Table 5. This was the result of selecting evaluation factors suitable for the autonomous taxi service by referencing the factors used in the evaluation of a traffic system in conventional studies (Stuart, Mednick, & Bockman, 2000; Eboli & Mazzulla, 2011, 2012; Redman, Friman, Gärling, & Hartig, 2013; Shaaban & Kim, 2016). Some evaluation factors were commonly used in the call, pick-up, travelling, and drop-off stages, whereas others were used selectively according to the characteristics of each stage. In the pilot test, evaluations were performed for “aesthetics” and “silence”, but they were excluded from the main test because it was determined that the test environment was limited and the evaluation results of the test subjects were not significant.
(Table 5 about here)
A 7-point Likert scale quantitative survey was conducted for the emotions overall experienced after the completion of all services, in addition to the satisfaction for each service stage. The emotions for the autonomous taxi service were divided into positive and negative. After their derivation through brainstorming and the semantic differential method, similar emotions were grouped together, and the emotions that should be evaluated mandatorily were selected for the autonomous taxi service design by considering the survey time and the customer concentration level. Finally, 12 positive emotions and 12 negative emotions were derived, as shown in Table 6. Finally, the survey was completed by an inquiry about the overall opinion, as shown in Table 7.
(Table 6 about here)
(Table 7 about here)
3.2. Test Scenario Design
Test Background
The pilot test was followed by the main test. The goal of the former is to examine the interaction method at touch points and test scenarios, which is a preliminary stage for increasing the effectiveness of the main test. In the pilot test, 15 people were tested, and in the main test, 43 persons were tested. In the pilot test, the regular scenario and the virtual taxi stand scenario of Figure 1 were all tested, whereas in the main test, only the virtual taxi stand scenarios were used.
Furthermore, a scenario was composed so that a passenger would change destination in the travelling stage after boarding the taxi. Note that the destination changing scenario does not occur often, but is one of situations that requires accurate communication between the customer and the driver. Therefore, this scenario was selected to be test how the communication in an autonomous taxi ought to be implemented. The system should be designed so as to respond to various user requests when an autonomous taxi service is adopted, in addition to the requests that can be made to the taxi driver in the current manned taxi service, such as adjusting the heating or air-conditioning, changing the destination, and adding a stopover. Other than the destination changing scenario, this study was composed with the simplest tasks, but future work will be designed to incorporate various tests with more complex and detailed tasks.
Interaction Methods
In the test, the passenger and the autonomous taxi primarily interact before and after boarding. Before boarding, the passenger interacts with the autonomous taxi by using a smartphone, and after boarding, by using the display installed in the vehicle.
Using the smartphone, the test participants interact with a “chatbot” through a messenger app. The autonomous taxi service starts as a passenger enters the chat room of the autonomous taxi, and using a map application, shares the routes from the departure point to the destination. Through the corresponding chat room, all information is provided and interaction takes place.
As a method for identifying the taxi, a QR code was used as a tag for the taxi in the pilot test, but several participants considered this inconvenient. Accordingly, the following scenario was selected in the main test: the user’s smartphone and the autonomous taxi recognize each other using near-field communication (NFC) so that they can identify each other automatically. Therefore, as the passenger approaches the front of the autonomous taxi he/she called, the doors are automatically unlocked.
After boarding, communication with the passenger is carried out through the tablet display, as shown in Figure 6. In the pilot test, the user used his/her hand to input a destination change on the display in the vehicle. However, several participants pointed out that using a new device was awkward, and predominantly, as input was entered while the taxi was travelling, the destination was difficult to change owing to shaking. In the main test, information is provided through both the screen and voice, and if the destination should be changed during travelling, once the passenger says “change the destination”, voice recognition is performed, and after the new destination is confirmed, the new destination is displayed on the screen.
The driver of the autonomous taxi and the control center share the passenger status through the chat room, and while they are connected through voice communication, the test is carried out. The control center monitors the situation inside the vehicle through audio as well as video, and provides voice instructions to the driver. Nevertheless, as only the driver can listen to the instructions of control center, the test subjects are not aware of the communication between the driver and the control center.
Specific Scenarios
One of the most important goals of this study is to evaluate the UX for the virtual taxi stand. Virtual taxi stand is a concept in which a passenger boards or disembarks from the taxi by setting a position among the places where the autonomous taxi can wait and where it is closest to the location from where the passenger called a taxi. After calling a taxi, the passenger should proceed to the instructed autonomous taxi stand and board the taxi. Therefore, path information has to be provided from the location of the passenger to the virtual taxi stand for pick-up, and from the virtual taxi stand for drop-off to the destination. The place where the passengers called the taxi was the Education Support Building (W8) in the Korea Advanced Institute of Science and Technology (KAIST), and the corresponding virtual taxi stand was the shuttle bus stop in front of W8 in the KAIST. The virtual taxi stand of the initial destination “Daejeon Observatory” was the entrance of Daejeon Observatory. Moreover, the virtual taxi stand of the changed destination “KAIST Twosome Place” café was in front of the “KAIST Professors Hall” restaurant nearby. The travelling distance was approximately 4 km and the travel time was approximately 10 min.
Furthermore, to provide exact location information of the autonomous taxi stand, a picture of it was provided in the chat room, as shown in Figure 8. The yellow part indicates the expected waiting position of the autonomous taxi, where a passenger boards or disembarks. Table 8 shows the specific scenarios for the autonomous taxi, control center, and passenger for all the stages from the autonomous taxi service preparation to the termination of the main test.
(Figure 8 about here)
(Table 8 about here)
Lastly, the following message was sent to the customer’s smart phone for him/her to check the taxi fee before getting off: “Thank you for using KAIST driverless taxi / total taxi fee: 3,000 won / driving distance: 2.3km / The taxi fee was automatically charged to the card you registered.”
3.3. Difference between the Pilot and Main Tests
We summarize the difference between the pilot and main tests as shown in Table 9.
(Table 9 about here)
4. TEST RESULTS
For the main test, 43 test participants were recruited through an online bulletin board. The males and females accounted for 58% and 42%, respectively, and regarding the age groups, 5% were 19 years old or younger, 59% were 20–24 years old, 24% were 25–29 years old, 7% were 30–34 years old, and 5% were 35 years old or older. Furthermore, the investigation on the taxi usage of the participants revealed that 35% of the participants used taxis less than once per week, 23% used them once a week, 33% used them 2–3 times per week, 7% used them 4– 6 times per week, and 2% used them daily. A gift certificate of 20,000 Korean won was paid to each participant.
4.1. Call Stage
Figure 9 shows the averages of the 7-point Likert scale evaluation in the call stage, and the standard deviation bars are shown together. In the call stage, “predictability” exhibited the lowest score (5.4).
(Figure 9 about here) To the qualitative question about inconvenience in the “call” stage, 50% of the responses pointed to the lack of information on the service status. It was difficult to estimate the progress of the service because the test participants could not check the location of the autonomous taxi and the estimated arrival time in real time, and were not familiar with the concept of a virtual taxi stand.
“The taxi call was simple… It was nice that it showed me where to take the taxi… But it would have been better to track exactly where the taxi is.” (P14)
“It’s good that it showed me where the pickup location would be, but I wish it could recommend a few locations. I would have liked to choose from different options.” (P38)
Nevertheless, the majority of participants responded that the service did not involve many inconvenient factors in contrast to traditional taxi services.
“I didn’t notice a significant difference from the conventional method of using a mobile app to call a taxi.” (P27)
4.2. Pick-up Stage
Figure 10 shows the evaluation results for the pick-up stage, where “confirmation” received the lowest score (5.0), and there were responses pointing out that as the exterior of the autonomous taxi was identical to that of a regular taxi, it was difficult to identify it. The “communication” factor exhibited a relatively low score (5.3), and “predictability” exhibited a low score, as was the case in the call stage. This seems to be because the test participants were accustomed to directly communicating with the driver in a manned taxi, as opposed to identifying an autonomous taxi.
(Figure 10 about here)
In the pilot test, a method of using QR code for identifying the passenger and the autonomous taxi was used, and to the qualitative question about inconvenience in the pick-up stage, the response “the confirmation procedure when getting in the taxi was inconvenient” accounted for 40%. Based on this, automatic opening of the door was provided when the passenger and the taxi were confirmed through NFC in the main test. However, this was still inconvenient for the customers who were accustomed to taxi drivers.
Furthermore, although there was no problem in identifying the taxi on a quiet road in the test environment, the test participants were concerned about identifying it in a more complex environment.
“I am not sure if I can communicate well with the taxi that is driving on the actual road, where the taxi cannot stop and wait.” (P4)
“It would be nice to have a way of identifying the taxi I called if there are many other taxis around.” (P37)
Another response stated that as the taxi must start after confirming that the passenger has boarded, clear communication with the passenger is necessary. “…what if the luggage is put in the car trunk without the door being shut and the taxi leaves immediately?” (P20)
There were also positive responses that the automated system is much more convenient and reliable.
“I like the fact that once I set my destination, everything else is done automatically afterwards. I also liked the accuracy of the taxi pickup location. If I call a regular taxi, I would need to talk to the driver multiple times to confirm where I am, or to change the pickup location.” (P3)
“The function that checks if the seatbelt is fastened increases the sense of security for using the service.” (P2)
4.3. Travelling Stage
Figure 11 shows the evaluation results for the travelling stage. The “comfortable” factor received the lowest score, (4.8). This was because the view as well as the interior space were limited owing to the video capturing devices and partitions in the autonomous taxi.
(Figure 11 about here)
The “communication” factor received the second lowest score. The preference for the method of changing the destination exhibited large differences. In the responses, the voice recognition used in the main test was considered more convenient compared to the display input method used in the pilot test; however, several participants questioned its accuracy.
“I don’t usually use voice-controlled functions. I’m a little skeptical about how accurate they are. It may be convenient, but I wouldn’t feel comfortable if this was the only way to communicate.” (P6)
The test subjects responded that even though using the relatively unfamiliar voice recognition method is convenient, the familiar display control should be provided at the same time so that the stability of service can be improved. Furthermore, in some cases, the passengers preferred a method of changing the destination on their own smartphones instead of the display in the taxi.
There was a concern about the communication method when a passenger does not know the exact name of the destination, when the destination name is in the form of slang or abbreviation, or the destination is unclearly expressed.
“Daejeon Observatory is not a uniquely identified location. What if there is a front gate and a rear gate?” (P5)
Regarding speed, 47% of the answers were “the taxi was driving slowly and I felt irritated.” This was because the autonomous vehicle drives at constant speed while obeying the signals, speed limits, and traffic regulations; thus, the speed was perceived as considerably lower compared to that of manned taxis. However, only few responded that the speed itself caused discomfort, and in fact, there was a customer who answered that the low speed offered a feeling of stability.
“I don’t mind if the taxi drives under the speed limit. But I would feel annoyed when the taxi does not drive faster in case of an emergency.” (P15)
“I liked that it didn’t involve rough or fast driving that some taxi drivers tend to do. It felt safe and smooth” (P1)
Prior to experiencing the service, 14 participants (the largest number) thought of safety first when considering the autonomous taxi, and in their answers, the expressions “cannot trust” and “nervous” were often used. However, after actually experiencing the service, the dissatisfaction was higher in “comfort”, “communication”, “speed”, and “convenience” than in “safety”. This implies that it is necessary to experience the service directly.
In a manned taxi, a passenger can ask any question and make a request to the taxi driver. However, in an autonomous taxi, a user interface (UI) is required whereby the passenger can control the taxi directly and obtain the desired information (e.g., using a Bluetooth connection or USB charging, music playing, controlling the air-conditioner or the speed, and inquiring about the location of the taxi, the weather, and the local area). Moreover, there were several requests that the design of the layout in a no-driver environment should be passenger-seat- oriented (although all the controls of the taxi are located near the driver seat).
“I think the passenger must be able to control and check everything inside the taxi. The things that can be asked or requested from conventional taxi drivers should also be possible in autonomous taxis.” (P25)
In the traveling stage, the video analysis is especially helpful. As shown in Figure 15, in the playback of the recorded videos of test participants, various behaviors were observed, such as eating food, drinking beverage, or playing a game while shouting. Furthermore, when the movement of the autonomous taxi was unstable, uncomfortable feelings were experienced; for instance, some participants were grabbing the seatbelt with their hands or were looking around. Such positive and negative behaviors are caused by the absence of a taxi driver and by the confinement in a closed space.
(Figure 15 about here)
The main feature of the customer experience in the video analysis was that passengers were either of the two groups: bored or anxious. Most passengers adjusted to the environment soon after boarding, and showed behaviors of boredom afterwards. For instance, passengers looked out the window or looked at their mobile phones throughout the ride. Facial expressions showed signs of tediousness or tiredness. The interview results revealed that even if the taxi was driverless, driving within speed and by the rules reduced any kind of tension but increased boredom. The related interview results are supportive evidence provided in the subsection of Section 5.2, titled “the need for providing a private and free space”.
In contrast, passengers who exerted fears or anxiousness showed behaviors like sitting up straight as soon as the taxi entered the road or looked alert at the window whenever a vehicle nearby approached. One passenger even tried to rip off the driver box to check if a driver was driving. This passenger pressed the mobile emergency button, and chose to get off the taxi in the middle of the experiment. These behaviors suggest that the degree of perceived risks is different across persons. Future research will focus on the uncovering the factors for such anxiousness involved in autonomous vehicles and solutions to resolve these issues.
4.4. Drop-off Stage
Figure 12 shows the evaluation results in the drop-off stage, where “accessibility” exhibited the lowest score. There were several vehicles parked near the virtual taxi stand for drop-off, and this was inconvenient considering the pick-up location. Moreover, there was some distance from the destination. The score was also low in the case of “communication”, and the answers regarding the reason, “I didn’t know when to get off” accounted for 13%. In particular, in the case of the virtual taxi stand, as the taxi stops not at the exact location that the passenger wants but nearby, the passengers were uncertain as to whether the taxi arrived at the accurate location.
“Although I was informed that the taxi arrived at the destination, I wasn’t sure when to get out of the car or whether it was safe to open the door.” (P1)
“During the process, I knew exactly where I will be picked up since I had to look for the virtual taxi stand in order to take the taxi. But when getting off, the taxi chooses a virtual stand near my destination, so I didn’t know exactly where I will be dropped off.” (P17)
Nevertheless, there were some positive responses about skipping the payment process and adding the checking process for any items left behind:
“The checking process for any left over items increases the overall stability of the service.” (P12)
“I always felt rushed to get off taxis so that the taxi driver did not have to wait for me, but there was no need to rush in autonomous taxis. I was actually able to check whether I left any of my belongings behind.” (P32)
“From calling a taxi to paying for the trip, I liked that everything was done automatically… I didn’t need to worry about getting overcharged for rides.” (P4)
(Figure 12 about here)
At the end of the interview, pictures of three virtual taxi stands with different distances from the final destination were shown to the test participants, and questions were asked regarding the optimal distance of the virtual taxi stand and related factors to be considered. The virtual taxi stand for drop-off in the test was set up 93m away or 1 min walk from the destination. To the question about the tolerable distance in the unavoidable situation in which the virtual taxi stand should be located further, only two persons (5%) out of 43 answered 300 m. 29 persons (67%) answered 200 m, and 12 persons (27%) answered 100 m. The answers assumed clear weather and no luggage. In the case of bad weather, luggage, or a tight schedule, the dominant opinion was that the service would not be used if the distance of the virtual taxi stand was even slightly increased. In the case of buses and subways, the boarding locations are fixed; however, as the consumer perception is that taxis can be used anywhere, the test participants responded that a denser network of virtual taxi stands should be established to provide better accessibility than buses and subways, or that inexpensive fares should be offered. Furthermore, they responded that even though it may not be feasible to set up taxi stands everywhere, well-known locations must be included. As there are destinations frequently used symbolically in a taxi service.
“There are popular destinations in Daejeon. For instance, many students will ask to be dropped off at school dorms of KAIST. In Dunsan-dong, Galleria department store is a popular place…Passenger convenience will greatly decline if virtual taxi stands do not include such landmarks.” (P25)
4.5. Overall Opinion
Table 10 shows the results for the questions regarding the overall autonomous taxi service.
(Table 10 about here)
The most important reason for preferring the autonomous taxi was “because there is no unnecessary communication with a taxi driver” (10 persons). In fact, all participants responded that they had experienced some kind of discomfort because of taxi drivers. In the case of manned taxis, the ambience in the car varies dramatically depending on the taxi driver; however, as passengers cannot choose the taxi driver, they feel some inconvenience. The test participants expected that if the concept of autonomous taxi is commercialized, unnecessary conversation will disappear and there will be no more refused passengers; moreover, the problem of taking detours can be eliminated, the taxi can be used even for a short distance without any feeling of pressure, and the destination can be freely changed as many times as desired. Furthermore, there was great expectation that crimes in conventional taxis will disappear.
“I felt comfortable that I didn’t need to deal with taxi drivers who talk about themselves or speak rudely to younger people.” (P12)
“I liked that I didn’t need to feel pressured to continue conversations with drivers when taking autonomous taxis” (P7)
“I liked that I didn’t receive any questions that I didn’t want to answer – like political ones.” (P32)
“I could do whatever I like since no one was there to watch me.” (P15)
By contrast, the reason for preferring a manned taxi was “safety” (8 persons) followed by “flexibility” (4 persons). Taxi drivers can easily provide feedback for requests regarding an accurate drop-off location and speed control. Therefore, it is necessary to design a service that will replace the conventional roles of taxi drivers. This is discussed in more detail in Section 5.1.
In addition, respondents who were indifferent between taking a manned taxi or an autonomous taxi said that they would take one that would take them to their destinations faster. Since the main reason for taking a taxi was to save time, they would choose a taxi that takes a shorter time. 74% of respondents in the main test expected lower fares compared to the conventional taxi fares. The majority answered that the reason was that labor cost is eliminated. Only 20% responded that they expect the same or higher fares compared to manned taxis. However, there was an opinion that if more value-added services are provided in addition to the simple transportation service, the fare increase can be tolerated.
In the evaluation results for the call, pick-up, travelling, and drop-off stages in Sections 4.1–4.4, “predictability” and “communication” were common problems. Therefore, a UI/UX design is necessary whereby customers who are not familiar with the autonomous taxi service can intuitively understand the process. Figure 13 shows the comparison of the results regarding the overall satisfaction in each stage, where it can be seen that satisfaction increased gradually from the call stage, and in the last drop-off stage, it declined slightly. Satisfaction for call and drop-off was relatively low because inconvenience is inherent in the concept of the virtual taxi stand compared to conventional manned taxis. As satisfaction increased overall except for the drop-off stage, it seems that the passengers who were experiencing the autonomous taxi service for first time had less anxiety after boarding.
(Figure 13 about here)
(Figure 14 about here)
Figure 14 shows the 7-point Likert scale evaluation results for the 12 positive emotions and 12 negative emotions introduced in Table 6. For positive emotions, the score is better if it is higher, and for negative emotions, it is better if it is lower. By examining the emotional evaluation of the overall autonomous taxi service, it can be confirmed that apprehension regarding the safety of autonomous vehicle technology and service is still a dominant emotion, as demonstrated by “nervous”, “uncomfortable”, and “afraid”. Moreover, owing to the experimental video capturing devices in the vehicle, the interior was unusual, and as the service information was delivered through text messages and the display only, the emotion for “familiarity” was low. Furthermore, owing to the partitions covering the driver seat, the view in the internal environment of the vehicle was narrow, and as the driving speed was low, the attribute “stuffy” was reported. The appearance of the attribute “dull” is consistent with the feedback provided by the “lonesomeness” emotion in the autonomous vehicle.
Regarding positive aspects, the autonomous taxi was experienced as a private space, and there was an opportunity to provide entertainment (14 persons). The highest expectation for real autonomous taxis was a change of the interior design.
“Since there won’t be a driver seat, I imagined various ways how the space can be utilized. Wider space and view would be more pleasant.” (P12)
“I think it’s a big advantage that the taxi can provide a personal space. I think it should also provide similar amenities as KTX (Korea Train Express) – with a free Wi-Fi and a desk.” (P17)
“It would be nice if I can enjoy music or a movie or edit documents easily… It would be like being in a first-class airplane seat.” (P8)
“We can now consider taxis not as a simple transportation method but as taking a ‘drive’.” (P30) Furthermore, some participants responded that there should be a separate safety device in the vehicle in case of an emergency (9 persons). There were ethical concerns before experiencing the service regarding possible accidents (9 persons), although no accident occurred during the service.
“Whose safety comes first between a pedestrian and a passenger? Who is responsible when an accident occurs?” (P1)
“Who decides what to prioritize in terms of safety in an algorithm?... I do not want a machine to determine who to save first.” (P11)
In addition, there were opinions where the respondents were concerned about the possibility of the system being hacked. They were worried that hackers may intentionally drive taxis into accidents or alter destinations when on road. There were also respondents who were concerned about jobs of taxi drivers disappearing. Some respondents called for a diversity of service options.
“It would be nice if I can select the types of taxis depending on the transportation purpose – like taxis with a large trunk space or that can fit more people.” (P43)
“I wish I could select the routes – like choosing a faster route even if I have to pay a toll fee.” (P2)
5. DISCUSSION
This section proposes major insights that service designers can obtain based on the test results.
5.1. Autonomous Taxi Service
Need for alternative solution for the role of manned taxi driver
Most passengers who had used manned taxis had negative experiences with taxi drivers and expected that such experiences would disappear in an autonomous taxi service. Therefore, it is necessary to develop a method for overcoming the “inconvenient factors caused by the absence of taxi driver” and a service element that can help users have positive experiences in “a private space and free time”. First, to overcome the “inconvenient factors caused by the absence of taxi driver”, it is necessary to analyze the roles of taxi drivers. Table 11 lists the roles of taxi drivers that currently use a taxi app.
(Table 11 about here)
The majority of roles can be replaced using currently existing technology; however, in terms of technical and service aspects, the following two cases should be examined.
Revising the specific route through conversation near the departure location or destination
Currently, even if a smartphone taxi app is used, if the departure location and destination are not accurately set up initially, the passenger communicates with the taxi driver by phone or explains the exact drop-off location verbally. Therefore, it is necessary to develop a method that can handle such a situation in the case of autonomous taxis, where there is no taxi driver. In conventional map applications, there is a method for locating a “pin” accurately, but the technology and a suitable UI should be developed for recognizing and handling situations such as disembarking on a narrow road, in front of a crosswalk, or suddenly. To this end, the concept of “virtual taxi stand” can be used.
Handling emergencies or abnormal passenger- or vehicle-related situations.
Abnormal situations can be detected through sensors, but to manage them, either physical devices are required or it is necessary to visit a maintenance center for inspection. Research is required to develop a technology that can recognize such situations, as well as devices and facilities for managing it.
Need of service design that can reduce safety concerns
The most serious obstacle to the adoption of autonomous taxi services is “distrust of user in the technology”. It is a major problem that should be overcome at the initial stage of service adoption. Thus, it will be necessary to publicize the fares and the numerical values that can prove the safety of the service so that customers may be convinced to use it. Moreover, as can be seen in the test results, it will be necessary to let the users understand in advance issues such as responsible parties, resolving methods, and compensation when a problem occurs. These will be the factors that can reduce apprehension.
Therefore, it must be specified in the service agreement that when a problem occurs during service operation, the responsibility lies in the operating company, and the information on compensation and insurance subscription for users must be provided. Moreover, as there is no driver, it will be mandatory to provide in advance information on how the problem will be handled through a simulation (e.g., reference should be made to the corresponding systems in the case of airplanes or buses). However, in the design of an autonomous taxi service, it is essential to consider the factors that help users trust the safety of the system while using the service. To this end, the following two factors must be taken into consideration.
Consideration of UX whereby a user may recognize and have control over a given situation
The test participants required information related to travelling status, such as the current route and the estimated arrival time at the destination. As there was no driver, the participants would have preferred being assured they were on the right track by identifying the current status. Therefore, it is necessary to provide this information. Through the display as well as the voice announcement system, information on the travelling or vehicle safety status should be provided to reduce user concern or anxiety.
Consideration of devices that can be used by the user in an emergency situation
The test participants would have liked to be able to disembark form the autonomous taxi when an emergency occurs, and it was confirmed that a device would be required for this. Therefore, it is necessary to consider a device that can be used in an emergency situation, whereby connection is made immediately with the call center that operates the service, and after human intervention and stopping of the vehicle, the passenger can disembark. In particular, human intervention plays an important role in reducing user anxiety. The test results revealed that even in a non-emergency, if the system reacts abnormally or slowly, the user will become nervous. Therefore, a device that can respond to passenger requests is crucial. Need for providing a private and free space
As there is no driver, passengers could act freely, as demonstrated by the interview results and the monitored behaviors in the recorded videos. Such a characteristic is similar to that of the UX in the first class seats of airplanes or high-speed trains. The test participants would have liked to control certain devices (for instance, music, lighting, temperature) inside the vehicle by themselves, and had strong desire to utilize their time while travelling. Accordingly, it will be possible to link with various services related to relaxation, business work, and meal. However, this would require additional operational management. Therefore, it will be necessary to provide a positive experience to the users and simultaneously consider operation efficiency. The latest services on premium express buses that provide conveniences (for instance, Wi-Fi, foldables, battery charging, music playing, video watching, seat backrest and footrest of seat) without attendant crew can be used as a reference.
5.2. Virtual Taxi Stand
Determination of appropriate fare and distance to taxi stand
When the concept of “virtual taxi stand” is used to alleviate technical imperfection, the most important aspects are “fare” and “distance”. For these, the following two factors should be considered.
The fare should be slightly higher than the bus fare but significantly lower than the taxi fare
The test participants expected that, overall, the autonomous taxi service would be considerably less expensive than the manned taxi service. Therefore, in the case of the “virtual taxi stand”, as the distance that a user should walk is longer than that in the case of a regular manned taxi, the inconvenience increases. Therefore, it seems that the fare should be low to motivate the customers to use the service. If the fare is considered relatively high compared to the distance that a user should walk, the motivation to use the autonomous taxi service will decrease. Hence, a fare policy between those of bus and manned taxi services is required.
The appropriate distance from a passenger location to a virtual taxi stand should be approximately 100 m at most
In the interview results of the test participants, 95% of the respondents answered that the tolerable distance for walking a virtual taxi stand was up to 200 m. However, this response was under the assumption that there was no baggage and the weather was clear. Therefore, considering service operation safety, it will be appropriate to apply 100 m as the maximum distance to a virtual taxi stand, and it is determined that it will be possible to allow up to 200 m in exceptional cases depending on the road situation. However, the appropriate distance as perceived by users will vary depending on the distance to nearby bus stops or subway stations and the probability of finding a manned taxi. Therefore, the distance to a virtual taxi stand must be shorter than the distance to a public transportation station such as bus stop or subway station.
Explaining location and directions of virtual taxi stand
It should be easy for a user to understand the location of a virtual taxi stand and the directions to pick-up location/final destination. In the test, as there was low traffic at the pick- up location, which was the same as at a “shuttle bus” stop, the stand was easy to recognize. However, if a real autonomous taxi stand is assigned, it may not be easy for a user to recognize its location. Therefore, it will be necessary to actively use symbolic locations that can increase the recognition rate, such as a store near a road, station, and sculpture, and a photograph of the surrounding area should be provided to the passenger in addition to the directions so that he/she can easily recognize the taxi stand. In particular, it is difficult for a passenger to quickly recognize the location of a virtual taxi stand for drop-off because his/her vision is affected by the speed and viewpoint of the vehicle. Therefore, before disembarkation, information on the virtual taxi stand and directions to the final destination should be provided in advance through the voice announcement system and the display, so that the passenger can recognize the location after disembarking.
5.3. UX Test
The main reason why this study employed the Wizard of Oz test is because of legal regulations in Korea that prohibits autonomous taxis to be tested on actual roads. Regardless, the Wizard of Oz technique is useful in conducting autonomous taxi research even in the absence of the legal constraints due to the following reasons.
First, it allows researchers to run prompt tests through fast prototyping of service concepts without much development costs or time. In the current experiment, the passengers were not aware that the vehicle was driverless, thus they perceived the environment as if they were receiving an actual autonomous taxi service. The method is especially appropriate for tests when interactions between persons and autonomous vehicles are necessary. Various means of interactions, like display or voice recognitions, could be compared. In addition, we are able to test interaction technologies that are yet to be fully developed and analyze and assess potential customer needs. In the upcoming studies, we plan to design additional interaction methods that are more refined and detailed to further examine the issues that are to be addressed when providing autonomous taxi services.
Secondly, the Wizard of Oz test is essential in an environment where safety is a priority. In providing autonomous taxi services, even a small malfunction of features cannot be tolerated. As recent autonomous vehicle experiments conducted by large companies suggest, accidents no matter how small are found to be fatal to the whole development process. In a Wizard of Oz test, since there is no risk of accidents that arises because of malfunctions of machinery, one can plan various scenarios without such restrictions. If the experiment was done with actual autonomous taxis, we would not have been able to test scenarios that produced any possibility of accidents. This is particularly true for experiments on actual roads where researchers cannot control the uncertainty involving unexpected situations of other vehicles.
We provide some insights earned from our experiment for UX designers who will employ the Wizard of Oz method for similar tests in the future.
First, a substantial number of experimenters are needed, and they must be skillfully trained through multiple prior testings before the actual experiment. For drivers, in particular, numerous rehearsals were conducted so they can drive well within restricted views in the driver box. In order to replicate the automated process accurately and without delays, the tasks and functions were divided into multiple steps for the experimenters to familiarize themselves with the whole process. In the current study, a total of 10 experimenter including the authors were involved in the experiment and a substantial amount of time was put into conducting multiple trials before the actual implementation. Second, a third party that is not involved in the experiment may be affected. In a Wizard of Oz test, people other than the participants themselves believe that the experiment is real as well. Therefore, a researcher ought to consider whether there are any parties that may be affected by the experiment prior to the implementation. There was one incident when we had to abort our experiment because a local taxi company protested that the vehicle under study were operating without official authorization. It was resolved once they understood that the vehicle was part of research and not for commercial reasons, but if we notified them earlier, we would not have had to terminate an experiment in progress.
Third, we found a pilot test to be absolutely necessary. This is not only for the experimenters, but for the actual experiment itself. A pilot test allows the researchers to address potential problems beforehand, and make necessary improvements before implementation. The Wizard of Oz test is indeed easier to implement and can be conducted with lower costs compared to when using autonomous vehicles. The preparatory stage may take less time, but the same amount of time is needed when implementing an experiment. Therefore, through a pilot test, it is important to extract and select only the necessary elements to implement in the actual experiment. In the current study, we were able to upgrade multiple facets of the main tests by conducting a pilot test.
Fourth, there must be strict verification of safety. In our experiment, we provided insurance for both participants and experimenters, inspected safety issues to the smallest detail, and created a protocol to follow in case of any accidents.
6. CONCLUSION
There exist opposing opinions on when autonomous taxis developed by firms will be launched in the market and how complete the level of technology would be when launched. Regardless, there appears to be a consensus that future taxi services ought to head that direction (CNBC, 2019). There are already unprecedented amounts of investments made and research conducted on autonomous vehicles that it seems the trend is now irreversible. The current study has provided several insights on how to enhance passenger experience when the time comes for firms to officially launch autonomous taxi services. We outline the summary of the results of our experiment, and introduce some research agenda for future work as follows.
6.1. Summary
By designing an autonomous taxi service and implementing a service prototype, this study tested the UX. Moreover, the factors that should be considered in a future autonomous taxi service were proposed, and guidelines were provided to service designers. In particular, by adopting the concept of virtual taxi stand, it was demonstrated that the imperfection of autonomous driving technology could be overcome by service flexibility.
The service design was carried out using the customer journey map, touchpoint analysis, and the service blueprint, and the service prototype was implemented using Wizard of Oz methods. Furthermore, the test was designed using Design Thinking. Quantitative and qualitative analysis was performed through the test of the prototype, a survey, in-depth interviews, and video analysis. Based on the evaluation results, the customer pain points were determined. First, to improve the predictability of the autonomous taxi service, it is necessary to implement a UI/UX design that enables users to understand the service process intuitively, namely, in a situation where there is no taxi driver. In particular, in the communication aspect, it is necessary to improve the method of identifying the taxi, and the communication method with the passenger should be reliable during disembarkation in a complicated situation. In addition, the layout inside the vehicle should be designed to accommodate passenger-seat-oriented controls.
The main directions to service designers were proposed as well. For smooth communication between the passenger and the autonomous taxi, the roles of the taxi driver were analyzed, and functions that may replace them were proposed. Moreover, a private and free space design was proposed, as well as a design method that can reduce safety concerns. Finally, by determining the appropriate “fare” and “distance”, a method for setting up virtual taxi stands was proposed.
The contribution of our work can be summarized as follows. First, the study proposed a test method to assess customer needs by evaluating their UX of autonomous taxi services conducted on actual roads. Second, we obtained some insights and design requirements to consider when designing the service, and issues to be further addressed in the future. Third, we proposed virtual taxi stands as a flexible function that would complement autonomous taxi services when the technology is not fully developed. Fourth, we have provided some useful insights for future researchers who will use the Wizard of Oz method under similar scenarios.
6.2. Future Research
As a follow-up research to the current study, the authors are conducting joint research with a car manufacturer that have plans to launch autonomous taxi services in the near future. The follow-up research will incorporate the design requirements obtained from the current study, and is working toward resolving physical limitations of previous experiments. First, we are testing and designing the UI located in the passenger seat. The UI will allow the passengers to control the main functions of services provided, monitor outside environment in real time, and enable various communication with the autonomous taxis. Second, we are conducting experiments on uncovering the factors behind anxiety issues as shown in the video analyses. Then, we will provide how to reduce the factors through HMI. Third, in the current study, a non-busy road near the university campus was selected for safety reasons. In a future experiment, complex real-road situations (e.g., in the middle of Seoul, the busiest city and capital of Korea) and long travel time (e.g., one hour) will be taken into consideration. Fourth, as the test participants in this study were mostly undergraduate or graduate students in their 20s and 30s, there was a limitation in the diversity of the test subjects. In a future study, UX will be evaluated by considering more diverse age groups and backgrounds. Finally, by using qualitative evaluation results, statistical analysis will be performed for the factors that affect service satisfaction.
Figures
Figure 1. Customer journey map and touch points of manned/autonomous taxi
(a) Manned taxi
(b) Autonomous taxi
Figure 2. Regular scenario vs. virtual taxi stand scenario
Figure 3. Service blueprint of autonomous taxi
Figure 4. Internal and external environment of autonomous taxi (partition panels, video capturing devices, display)
Figure 5. Autonomous taxi control center environment (left) and real-time passenger monitoring (right)
Figure 6. User’s smartphone and the tablet inside the vehicle
Figure 7. Example of emotion curve measurement result
Figure 8. Pictures used to provide location information of virtual taxi stand
Figure 9. Service evaluation for the call stage
Figure 10. Service evaluation for the pick-up stage
Figure 11. Service evaluation for the travelling stage
Figure 12. Service evaluation results for the drop-off stage
Figure 13. Satisfaction in each stage of autonomous taxi service
Figure 14. Emotional evaluation of autonomous taxi service
Figure 15. Screenshots of passenger monitoring
Tables
Table 1. Examples of potential problem scenarios for each issue point
Issue Point Potential Problem Scenarios 1. Confirming the user The user/vehicle locations do not match. No place to stop the car. The user has not location arrived. 2. Confirming the user who The user who called and the dispatched taxi do not match called the taxi 3. Confirming the user(s) The limit on number of persons is exceeded. A safety belt is not fastened. The user got in the taxi has baggage. 4. Confirming the The destination is changed. The route is changed. A stopover is added. destination and route 5. Confirming the exact Not arrived at the exact drop-off location. Impossible to stop the car. drop-off location 6. Payment information Insufficient payment amount. Impossible to process the payment transaction. and transaction 7. Confirming the drop-off The user has not gotten out of the taxi. There is a lost item. of user 8. Preparing for next Choosing the next waiting place or the next passenger pick-up location. Needs to service dispose garbage. Needs to deal with the lost item. Needs to refuel.
Table 2. Research processes and methods
Service Design Prototyping Test Design Test Process (Sections 1.1 and 2.1) (Section 2.2) (Section 3) (Section 4) Customer journey map Survey Methodology Touchpoint analysis Wizard of OZ Design thinking In-depth interview Service blueprint User video analysis
Table 3. Test process
Step Description Evaluation Introduction of the test and test 1. Orientation method to the test participants Interview regarding the manned taxi experiences of each test 1:1 interview (about 20 min) 2. Before the test participant and awareness for - Pre-test interview questions Table 4(a)) autonomous taxi Using the autonomous taxi 3. Test service 1:1 interview (about 20 min) - Post-test interview questions (Table 4(b)) - Emotion curve graph (Figure 7) Survey Interviews with the participants - Evaluation for each stage: call, pick-up, travelling, 4. After the test and survey on satisfaction after and drop-off (Table 5) using the autonomous taxi - Overall emotional evaluation (Table 6) - Summary questions (Table 7)
Video analysis
Table 4. Main questions of in-depth interview
(a) Pre-test interview
No Question 1 What do you think about the autonomous taxi? 2 Is there anything you worry about the autonomous taxi? 3 If the fares of autonomous taxi and manned taxi are identical, which service do you prefer? 4 What do you expect or imagine regarding the autonomous taxi?
(b) Post-test interview
No Question
1 Now that you have used the autonomous taxi, what are your overall feelings?
2 Please draw an emotion curve for each stage (call, pick-up, travelling, paying, drop-off) In depth interview regarding the emotion curve for each stage (comparison with manned taxi 3 experiences) If you were to design an autonomous taxi, what kind of autonomous taxi would you like to design? (e.g., 4 service function, internal environment) 5 Was there any inconvenience when you used the virtual taxi stand? The virtual taxi stand was provided in a picture (road view). Was it helpful in recognizing it? Do you 6 have any other suggestion? The distance between the virtual taxi stand and the actual destination was 100 m (2 min. walking). If the 7 virtual taxi stand is located far owing to unavoidable reason, what is the maximum distance that you can tolerate?
Table 5. The evaluation factors for each stage
Evaluated/not evaluated in the stage Evaluation of Description Factor Pick- Drop- Call Travelling up off Satisfaction Felt satisfaction overall. O O O O Reliability Felt that the service was reliable. O O O O Predictability It was possible to predict what is to be done. O O O O Information Necessary information was received properly. O O O O Kindness Felt kindness in the service. O O O O Safety Felt it was safe. O O O Communication with the taxi was performed Communication O O O properly. Accessibility The taxi came to the desired place. O O Punctuality The taxi arrived at the predicted time. O O Convenience It was convenient and easy to use. O O Promptness The service was carried out promptly. O Confirmation It was easy to identify my taxi. O Speed The speed was appropriate O Ride comfort The ride was smooth and comfortable. O Pleasantness Felt pleasant in the taxi. O Comfort Felt comfortable psychologically. O
Table 6. Emotion evaluation index
Type Emotions Convenient, comfortable, familiar, safe, reliable, excellent, simple, sophisticated, ingenious, trendy, Positive efficient, new Nervous, uncomfortable, afraid, unpleasant, annoying, disappointing, stuffy, tiresome, complicated, Negative dull, strange, frustrating
Table 7. Summary questions
No Question 1 Do you intend pay a fare to use the autonomous taxi? Compared to a conventional manned taxi, what do you think is the appropriate fare for the autonomous 2 taxi? 3 Compared to a manned taxi, do you feel that the autonomous taxi is convenient? 4 Do you intend to recommend the autonomous taxi service to friends or family? 5 What was the best part that you liked in the autonomous taxi service?
Table 8. Specific scenarios for each stage of main test
Stage Virtual Taxi Stand (Autonomous taxi/control center) The connection status between the control center and the Stage 0 smartphone is inspected for real-time confirmation of passenger and vehicle travelling. Service preparation (Autonomous taxi/control center) Voice communication is established. (Passenger) After setting the departure location and destination, share them in the chat room. Stage 1 (Control center) The information of the virtual taxi stand closest to the passenger is shared Using the service – Call between the passenger and the taxi. (Autonomous taxi) The video capturing devices start to record. Stage 2 (Autonomous taxi) After confirming the route, the taxi goes to the departure point. Dispatching a taxi (Control center) The departure of taxi is confirmed. Taxi information is provided. The estimated arrival time is provided. (Control center) After the taxi arrives at the pick-up location, the passenger is notified that the Stage 3 autonomous taxi has arrived. Taxi approaching – (Autonomous taxi) After confirming the passenger via NFC, the vehicle doors are unlocked. Confirmation of (Passenger) The passenger opens the door and gets in the taxi. passenger pick-up (Control center) After confirming that the passenger is picked up, an information message is sent. (Control center) The control center informs the autonomous taxi to start. (Passenger) If the passenger wants to change the destination, he/she says “change the destination”, and the voice recognition function is activated. When the passenger says the desired destination, Stage 4 the destination is changed. Depart – Changing the (Control center) After destination change, a virtual taxi stand for drop-off is assigned. destination (Control center) The assigned taxi stand information is passed to the passenger and shared with the taxi. (Control center) When the autonomous taxi arrives at the exact location for drop-off, the arrival Stage 5 information message is sent to the display. Vicinity of drop-off (Autonomous taxi) The door of vehicle is unlocked. location – Getting off (Control center) After confirming that the passenger is dropped off safely, the details of taxi service are sent to the passenger’s smartphone. Stage 6 (Autonomous taxi) After confirming that the passenger is dropped off, the taxi goes to the garage. Preparing for the next (Autonomous taxi) The video recording is terminated and the taxi prepares for the next service service
Table 9. Difference between pilot and main tests
Pilot test Main test Reasons Evaluate aesthetics Exclude aesthetics Pilot test results of those factors Survey and silence and silence were not significant. Test both regular scenario and Test only virtual taxi Regular scenario was for reference Scenario virtual taxi stand stand scenario only. scenario Identifying The pilot result showed QR code Use QR code Use NFC Interaction taxi was inconvenient methods Changing Use voice The pilot result showed display Use display input destination recognition input was inconvenient
Table 10. The results of overall questions
No Question Results Do you intend to pay a fare to use the autonomous taxi Yes: 88% 1 service? No: 12% Less than 50%: 7% 50-74%: 23% Compared to a conventional manned taxi, what do you 2 75-99%: 44% think is the appropriate fare for the autonomous taxi? 100%: 21% 101-124%: 5% Compared to a manned taxi, do you feel that the Average: 5.0 3 autonomous taxi is convenient? (7 point Likert scale) Standard deviation: 1.4 Do intend to recommend the autonomous taxi service Average: 5.2 4 to friends or family? (7 point Likert scale) Standard deviation: 1.3 What was the best part that you liked in the No. 1: Absence of taxi driver 5 autonomous taxi service? No. 2: Convenience of service
Table 11. The role analysis of taxi driver
Taxi Status Specific Circumstances Exception Circumstances Before - Go to a place where a call from passenger can be received receiving a - Wait at a place where a passenger call can be received
call of - Refuel passenger - Choose a passenger call Call Taxi is called through the Taxi app. After - Go to the passenger location receiving the - Call to inform the passenger of the departure, estimated arrival passenger time, and confirm/coordinate the exact passenger location call - The user and vehicle locations do not - Confirm the passenger near the pick-up place Arrive at the match - Usually the passenger checks the license plate number of taxi, passenger - No place to stop the car (stopping is not and the taxi driver identifies the passenger as the passenger waves location allowed) a hand or approaches - The user has not arrived User - The user who called and the taxi do not - The taxi driver checks again if the passenger is the right person confirmation match - Check if there is baggage to be put in the trunk - Check if the seatbelt is fastened and door is closed Confirmation - Check if the number of passengers - Depart of getting in exceeds the limit - Turn the meter on - Press the picked-up button on the Taxi app - Travelling - (Reconfirm the destination / confirm the route) Travelling - (Change of destination) - Control the internal devices of vehicle (air conditioner, heater, lights, etc.) Exact drop- - Did not arrive at the exact drop-off - Confirm the exact drop-off location near the destination off location location - Stop the car confirmation - Impossible to stop the car - Inform of the arrival - Payment (automatic payment using the registered credit card) or - Insufficient payment amount Passenger provide the payment information and receive the payment - The payment transaction is impossible drop-off - Unlock the door (check surroundings for safety) - The passenger does not get out (drunk - If there is a baggage in the trunk, inform (open the trunk) passenger/ sleeping /wants to stay longer) - The passenger gets out of taxi - Prepare for the next service After - Check if there is an item left (lost item) dropping off - Check the vehicle status (damage) the passenger - Passenger evaluation (using the smartphone app) - Refueling
ACKNOWLEDGEMENT
This work was supported by a National Research Foundation of Korea (NRF) grant funded by the Korean government (NRF-2017R1C1B2005266).
Declarations of interest: none
REFERENCES
Abraham, H., Lee, C., Brady, S., Fitzgerald, C., Mehler, B., Reimer, B., & Coughlin, J. F. (2017). Autonomous vehicles, trust, and driving alternatives: A survey of consumer preferences. Proceedings of annual meeting of the Transportation Research Board, Washington DC.
Allianz Partners Business Insights (2018). Autonomous vehicles: Nissan to test driverless taxis in Japan. https://allianzpartners-bi.com/news/autonomous-vehicles-nissan-to-test- driverless-taxis-in-japan-5498-333d4.html. Last accessed September 1, 2018.
Bagloee, S. A., Tavana, M., Asadi, M., & Oliver, T. (2016). Autonomous vehicles: challenges, opportunities, and future implications for transportation policies. Journal of Modern Transportation, 24(4), 284-303.
Burghout, W., Rigole, P. J., & Andreasson, I. (2015). Impacts of shared autonomous taxis in a metropolitan area. Proceedings of annual meeting of the Transportation Research Board, Washington DC.
Business Insider (2016). Here are all the companies racing to put driverless cars on the road by 2020. https://www.businessinsider.com/google-apple-tesla-race-to-develop-self- driving-cars-by-2020-2016-4. Last accessed September 1, 2018.
Clatworthy, S. (2011). Service innovation through touch-points: Development of an innovation toolkit for the first stages of new service development. International Journal of Design, 5(2), 15-28.
CNBC (2019). Elon Musk is wrong on robotaxi timing, Uber CEO Dara Khosrowshahi says. https://www.cnbc.com/2019/05/10/uber-ceo-dara-khosrowshahi-says-elon-musk-is- wrong-on-robotaxi-timing.html. Last accessed May 20, 2019.
Dahlbäck, N., Jönsson, A., & Ahrenberg, L. (1993). Wizard of Oz studies—why and how. Knowledge-based systems, 6(4), 258-266.
Du, Y., Qin, J., Zhang, S., Cao, S., & Dou, J. (2018). Voice User Interface Interaction Design Research Based on User Mental Model in Autonomous Vehicle. Proceedings of the International Conference on Human-Computer Interaction. Springer, Cham. Eboli, L., & Mazzulla, G. (2011). A methodology for evaluating transit service quality based on subjective and objective measures from the passenger’s point of view. Transport Policy, 18(1), 172-181.
Eboli, L., & Mazzulla, G. (2012). Structural equation modelling for analysing passengers’ perceptions about railway services. Procedia-Social and Behavioral Sciences, 54, 96- 106.
Fließ, S., & Kleinaltenkamp, M. (2004). Blueprinting the service company: Managing service processes efficiently. Journal of Business research, 57(4), 392-404.
Forbes (2016). World's First Self-Driving Taxis Hit Singapore. https://www.forbes.com/sites/rahilbhagat/2016/08/25/worlds-first-self-driving-taxis-hit- singapore-roads/#6b44afa67031. Last accessed September 1, 2018.
Gao, P., Kaas, H. W., Mohr, D., & Wee, D. (2016). Automotive revolution–perspective towards 2030 How the convergence of disruptive technology-driven trends could transform the auto industry. Advanced Industries, McKinsey & Company.
Geum, Y., & Park, Y. (2011). Designing the sustainable product-service integration: a product-service blueprint approach. Journal of Cleaner Production, 19(14), 1601-1614.
Kang, N., Feinberg, F. M., & Papalambros, P. Y. (2017). Autonomous electric vehicle sharing system design. Journal of Mechanical Design, 139(1), 011402.
Kelley, D., & Kelley, T. (2013). Creative confidence: Unleashing the creative potential within us all. New York: Crown Pub.
Kim, S.W., Gwon, G.P., Hur, W.S., Hyeon, D., Kim, D.Y., Kim, S.H., Kye, D.K., Lee, S.H., Lee, S., Shin, M.O., & Seo, S.W. (2017). Autonomous Campus Mobility Services Using Driverless Taxi. IEEE transactions on intelligent transportation systems, 18(12), 3513- 3526.
Krueger, R., Rashidi, T. H., & Rose, J. M. (2016). Preferences for shared autonomous vehicles. Transportation research part C: emerging technologies, 69, 343-355.
Kyriakidis, M., de Winter, J.C., Stanton, N., Bellet, T., van Arem, B., Brookhuis, K., Martens, M.H., Bengler, K., Andersson, J., Merat, N., & Reed, N. (2017). A human factors perspective on automated driving. Theoretical Issues in Ergonomics Science, 1- 27.
Lee, J. G., Kim, K. J., Lee, S., & Shin, D. H. (2015). Can autonomous vehicles be safe and trustworthy? Effects of appearance and autonomy of unmanned driving systems. International Journal of Human-Computer Interaction, 31(10), 682-691.
Lemon, K. N., & Verhoef, P. C. (2016). Understanding customer experience throughout the customer journey. Journal of Marketing, 80(6), 69-96.
Litman, T. (2017). Autonomous vehicle implementation predictions. Victoria, Canada: Victoria Transport Policy Institute. Maulsby, D., Greenberg, S., & Mander, R. (1993). Prototyping an intelligent agent through Wizard of Oz. Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, The Netherlands, ACM.
Politis, I., Langdon, P., Bradley, M., Skrypchuk, L., Mouzakitis, A., & Clarkson, P. J. (2017). Designing autonomy in cars: A survey and two focus groups on driving habits of an inclusive user group, and group attitudes towards autonomous cars. Proceedings of the International Conference on Applied Human Factors and Ergonomics. Springer, Cham.
Redman, L., Friman, M., Gärling, T., & Hartig, T. (2013). Quality attributes of public transport that attract car users: A research review. Transport Policy, 25, 119-127.
Rothenbücher, D., Li, J., Sirkin, D., Mok, B., & Ju, W. (2016). Ghost driver: A field study investigating the interaction between pedestrians and driverless vehicles. Proceedings of the IEEE International Symposium on Robot and Human Interactive Communication. New York.
Shaaban, K., & Kim, I. (2016). Assessment of the taxi service in Doha. Transportation Research Part A: Policy and Practice, 88, 223-235.
Shen, W., & Lopes, C. (2015). Managing autonomous mobility on demand systems for better passenger experience. Proceedings of the International Conference on Principles and Practice of Multi-Agent Systems. Springer, Cham.
Shostack, G.L. (1982). How to design a service. European Journal of Marketing, 16(1), 49- 63.
Stocker, A., & Shaheen, S. (2017). Shared automated vehicles: Review of business models. International Transport Forum.
Strömberg, H., Pettersson, I., Andersson, J., Rydström, A., Dey, D., Klingegård, M., & Forlizzi, J. (2018). Designing for social experiences with and within autonomous vehicles–exploring methodological directions. Design Science, 4.
Stuart, K., Mednick, M., & Bockman, J. (2000). Structural equation model of customer satisfaction for the New York City subway system. Transportation Research Record: Journal of the Transportation Research Board, 1735(1), 133-137.
TechWorld (2018). Which companies are making driverless cars? https://www.techworld.com/picture-gallery/data/-companies-working-on-driverless-cars- 3641537/. Last accessed September 1, 2018.
The Verge (2018). Waymo begins experimenting with self-driving taxi prices. https://www.theverge.com/2018/7/31/17635472/waymo-self-driving-cars-pricing-ride- hail-arizona. Last accessed September 1, 2018.
Tussyadiah, I. P., Zach, F. J., & Wang, J. (2017). Attitudes toward autonomous on demand mobility system: The case of self-driving taxi. In Information and Communication Technologies in Tourism 2017 (pp. 755-766). Springer, Cham.
Notes on contributors
Sangwon Kim is currently a product owner in Bungaejangter Inc., Korea. He received a MS in K-School from Korea Advanced Institute of Science and Technology (KAIST). His research interests include UX, HCI, and data analytics.
Jennifer Jah Eun Chang is currently a product designer at Kali Care in California, USA. She received a BS in Computer Science and a MS in K-School from Korea Advanced Institute of Science and Technology (KAIST). Her research interests include UI, UX and HCI.
Hyun Ho Park is currently a PhD student in the School of Business and Technology Management of Korea Advanced Institute of Science and Technology (KAIST). His research interests include innovation strategy and policy. Seon Uk Song is currently a CEO of Bee&Flower, Korea. He received a MS in K-School from Korea Advanced Institute of Science and Technology (KAIST). His research interests include service design.
Chang Bae Cha is currently a COO of Jr Coding Lab, Korea. He received a MS in K-School from Korea Advanced Institute of Science and Technology (KAIST). His research interests include service design.
Ji Won Kim received a MS in K-School from Korea Advanced Institute of Science and Technology (KAIST). Her research interests include service design and blockchain.
Namwoo Kang is currently an Assistant Professor of Department of Mechanical Systems Engineering at Sookmyung Women’s University. He received a BS in Mechanical and Aerospace Engineering from Seoul National University, a MS in Technology and Management from Seoul National University, and a PhD in Design Science from University of Michigan. His research interests include machine learning, design optimization, and HCI.