Interaction Design for Remote Control of Military Unmanned Ground Vehicles

Home , Military robot, Teleoperation, Unmanned ground vehicle

Linköping University | Department of Computer and Information Science

Master’s thesis, 30 ECTS | Design and Product Development

2021 | LIU-IDA/LITH-EX-A--2021/005--SE

Diana Saleh

Supervisor : Mattias Arvola Examiner : Stefan Holmlid

Linköpings universitet SE–581 83 Linköping +46 13 28 10 00 , www.liu.se Copyright

The publishers will keep this document online on the Internet - or its possible replacement - for a period of 25 years starting from the date of publication barring exceptional circumstances.

The online availability of the document implies permanent permission for anyone to read, to down- load, or to print out single copies for his/hers own use and to use it unchanged for non-commercial research and educational purpose. Subsequent transfers of copyright cannot revoke this permission. All other uses of the document are conditional upon the consent of the copyright owner. The publisher has taken technical and administrative measures to assure authenticity, security and accessibility.

According to intellectual property law the author has the right to be mentioned when his/her work is accessed as described above and to be protected against infringement.

For additional information about the Linköping University Electronic Press and its procedures for publication and for assurance of document integrity, please refer to its www home page: http://www.ep.liu.se/.

The fast technology development for military unmanned ground vehicles (UGVs) has led to a considerable demand to explore the soldier’s role in an interactive UGV system. This thesis explores how to design interactive systems for UGVs for infantry soldiers in the Swedish Armed Force. This was done through a user-centered design approach in three steps; (1) identifying the design drivers of the targeted military context through qualitative observations and user interviews, (2) using the design drivers to investigate concepts for controlling the UGV, and (3) create and evaluate a prototype of an interactive UGV system design.

Results from interviews indicated that design drivers depend on the physical and psychological context of the intended soldiers. In addition, exploring the different concepts showed that early conceptual designs helped the user express their needs of a non-existing system. Furthermore, the results indicate that an interactive UGV system does not necessarily need to be at the highest level of autonomy in order to be useful for the soldiers on the ﬁeld. The ﬁnal prototype of an interactive UGV system was evaluated using a demonstration video, a Technology Acceptance Model (TAM), and semi-structured user interviews. Results from this evaluation suggested that the soldiers see the potential usefulness of an interactive UGV system but are not entirely convinced.

In conclusion, this thesis argues that in order to design an interactive UGV system, the most critical aspect is the soldiers’ acceptance of the new system. Moreover, for soldiers to accept the concept of military UGVs, it is necessary to understand the context of use and the needs of the soldiers. This is done by involving the soldiers already in the conceptual design process and then throughout the development phases. Acknowledgments

Firstly, I would like to thank all of my supervisors at FOI, Anna Häägg, Kristofer Bengtsson, and Björn Johnsson. It has been a pleasure to write my thesis at FOI, and I am grateful for your genuine support. This includes everything from being actors in my demo videos and contributing to my user testings to providing me with knowledge about the military context and reading my reports.

Furthermore, I wish to express my sincere appreciation to my supervisor, Mattias Arvola, who has offered me invaluable guidance on the topic of Interaction Design. His persistent help and his words of encouragement have been of the highest value, especially when the roads got rough.

Finally, I would like to thank Stefan Holmlid, my examiner at Linköping University. His support and feedback during the examiner seminars have been helpful in my work.

iv Contents

Abstract iii

Acknowledgments iv

Contents v

List of Figures vii

List of Tables viii

Terminology and Abbreviations 1

1 Introduction 2 1.1 Background and motivation ...... 2 1.2 Aim...... 2 1.3 Research questions ...... 3 1.4 Delimitations ...... 3

2 Theory 4 2.1 Human-Robot Interaction ...... 4 2.2 Teleoperation ...... 5 2.3 Unmanned Ground Vehicle Teleoperation in Complex Environments ...... 6 2.4 Interactive System Design in Telerobotics ...... 7

3 Methods 9 3.1 Research Through Design Case Study ...... 9 3.2 User-Centered Design Approach ...... 10 3.3 Exploration Phase ...... 10 3.4 Concept Phase ...... 12 3.5 Prototyping Phase ...... 12 3.6 User Evaluation Phase ...... 13 3.7 Research Ethics ...... 14

4 Results 15 4.1 Exploration Phase ...... 15 4.2 Concept Phase ...... 16 4.3 Prototyping Phase ...... 22 4.4 User Evaluation Phase ...... 24

5 Discussion 28 5.1 Implication of the results ...... 28 5.2 Limitations with the method ...... 30 5.3 The work in a wider context ...... 30

v 6 Conclusion 32

Bibliography 33

vi List of Figures

2.1 The THeMIS UGV 5th generation. By Milrem Robotics(This ﬁle is licensed under the Creative Commons Attribution-Share Alike 4.0 International license...... 7

3.1 The modiﬁed Research through Design used in this thesis ...... 9 3.2 A process picture that gives an overview of the user-centered research approach. . 10 3.3 The participants watching the monitor of the remote control of the UAV on big screen ...... 11 3.4 The Design Drivers. Context, Function and Qualities ...... 12 3.5 Example of a concept generated using the design drivers ...... 12

4.1 Concept 1 with main function to guard ...... 17 4.2 Concept 2 with main function to transport ...... 18 4.3 Concept 3 with main function to carry ...... 19 4.4 Concept 4 with main function to carry ...... 20 4.5 Concept 5 with main function to spy ...... 21 4.6 Dashboard top bar design proposal 1 ...... 22 4.7 Dashboard top bar design proposal 2 ...... 22 4.8 Dashboard bottom bar design proposal 1 ...... 22 4.9 Dashboard bottom bar design proposal 2 ...... 22 4.10 Scenario 1: Identify subjects on map ...... 23 4.11 Scenario 2: Plan a route ...... 23 4.12 Dashboard design outcome ...... 23 4.13 Latest dashboard design outcome from the Design Detailing phase ...... 23 4.14 Final prototype of UI Design for remote control ...... 24 4.15 Results from the Scenario Questions...... 26

vii List of Tables

2.1 HRI deﬁnitions based on Goodrich and Schultz deﬁnition...... 5 2.2 Levels of Automation of decision and action selection ...... 6 2.3 Five design elements when developing a military UGV ...... 7

4.1 Context Identiﬁcation of Military UGVs ...... 15 4.2 Design Drivers of Military UGVs ...... 16 4.3 PMI on Concept 1 ...... 17 4.4 PMI for Concept 2 ...... 18 4.5 PMI for Concept 3 ...... 19 4.6 PMI for Concept 4 ...... 20 4.7 PMI for Concept 5 ...... 21 4.8 The main functions of the UGV and the speciﬁed functions of the user interface. . 22 4.9 Desired, Interesting and Necessary aspects of the prototype...... 23 4.10 The average answers to the six questions that revolved around usefulness...... 25 4.11 Perceived Usefulness of the Average Participant ...... 25 4.12 Opportunities and limitations with military UGVs ...... 27 4.13 Potential use cases with military UGVs ...... 27

viii Terminology and Abbreviations

FOI The Swedish Defence Research Agency

HRI Human-Robot Interaction

UAV Unmanned Aerial Vehicle

UGV Unmanned Ground Vehicle

1 1 Introduction

The following chapter presents the basis of the work carried out in collaboration with FOI, the Swedish Defence Research Agency. It contains the background and motivation for the research study followed by the aim, the research questions, and the delimitations.

1.1 Background and motivation

Today, a soldier in the ﬁeld carries about 30-40 kg of packing, which in the long run affects the soldiers’ endurance and performance. To make it easier for the soldiers, and preserve their strength and energy, research is conducted on how to use Unmanned Ground Vehicles (UGVs) as a carrying aid. These UGVs for carrying, are not used in the Swedish Armed Forces today but are becoming more of interest as the technology of autonomous vehicles is developing at a rapid pace.

Currently, the intended UGVs are not entirely autonomous and need some form of remote control from an operator. Therefore the remote control must be designed in a way that allows the operator to use and maneuver the UGV under the complex conditions prevailing. This applies to both the graphical interface, the selection of information that appears on the screen, and the physical design of the remote control.

The research is performed in collaboration with FOI, the Swedish Defence Research Agency, which is a government authority under the Ministry of Defence and one of Europe’s leading research institutes in defense and security. In this thesis, the end-users are infantry soldiers at the Swedish Armed Forces.

1.2 Aim

The aim of this thesis is to explore the usefulness of interactive UGV systems design for infantry soldiers on the ﬁeld. The goal is to investigate, prototype and test interfaces for interaction between humans and unmanned systems.

2 1.3. Research questions

1.3 Research questions

Based on the aim of this thesis, the main research question is: "How do we design interactive unmanned ground vehicle systems for infantry soldiers on the ﬁeld?"

In order to answer the main research question, the question is broken down into three research questions that further represent the phases of the project.

1. What characterizes the context of use for infantry soldiers on the ﬁeld and what is important to consider in the design of an interactive UGV system for this context?

2. What are the infantry soldiers’ needs in relation to an interactive UGV system on the ﬁeld?

3. How should the remote control for an interactive UGV system be designed for infantry soldiers on the ﬁeld?

1.4 Delimitations

• The end-user involvement with the Swedish Armed Forces has been limited due to the prevailing pandemic situation with Covid-19.

• The aim is not to develop a detailed user interface but rather to explore design and concepts related to the interaction design of military UGVs.

• Cognitive workload is identiﬁed as an important design aspect to consider when designing for robotic control. However, this aspect is not measured in the thesis since it is considered to be outside the scope of the design project.

3 2 Theory

To be able to understand the ﬁeld of interactive systems for unmanned ground vehicles in military settings, it is essential to get a broader perspective of human-robot interaction. This chapter provides a theoretical framework starting from the broad perspective of Human-Robot Interaction to further narrow the scope to Teleoperation in Complex Environments and later dive into Interactive System Design in Telerobotics.

2.1 Human-Robot Interaction

Human-Robot Interaction (HRI) is described as the study of communication between humans and robotic systems [7]. The field of robotic systems is currently a wide-ranging research and design activity that, during the last decade, has expanded at a fast pace and in various fields. This has lead to that there are several approaches to defining the interaction between robot and human. Because the field of robotic systems is wide, these approaches tend to be explaining specific cases instead of providing a general established HRI framework [44] [43] [26].

Scholtz [35], for example, defined HRI from the perspective of the human role. He has identified five roles when interacting with a robot that is: supervisor, operator, mechanic, peer, and bystander.

In contrast, Granda et al. [12] defined HRI focusing around the "robot-interaction" part. They defined HRI as five overlapping stages of capability: Bounded Autonomy, Teleoperation, Su- pervised Autonomy, Adaptive Autonomy and Virtual Symbiosis. These stages are not mutually exclusive. In the stage of Bounded Autonomy, robotic devices operate with little human in- terference. These robotic devices are usually used in repetitive and pre-programmed duties, such as in factory systems. The fundamental part of the Teleoperation stage is that the operator is responsible for all active information processing. This stage is commonly performed in dangerous situations where the operator can control the robot from a distance. Supervised Autonomy emphasizes a variable span of autonomous control where the robot can handle particular tasks without an operator. However, when needed, the human operator can intervene with the robot in real-time. Adaptive Autonomy is characterized by the robot’s capability to

4 2.2. Teleoperation learn from experience. In this stage, the role of the human is safety monitoring and making sure what is learned is not harmful. In the ﬁnal stage, there is Virtual Symbiosis. In this stage, the human and the robot have reached a team relationship that allows them to communicate on a high cognitive level.

Furthermore, Goodrich and Schultz have a deﬁnition that focuses on the interaction part between the human and the robot. Goodrich and Schultz deﬁne interaction as a form of communication that varies depending on the distance between the human and the robot [11]. Thus interaction is divided into the interaction categories "remote" and "proximate". Remote interaction is referring to the fact that the human and the robot are separated spatially and sometimes even temporally (e.g., Mars Rovers that are controlled on earth and hence separated both in space and in time). Whereby proximate interaction refers to that the humans and the robots are co-located ( e.g., service/assistant robots could be in the same room as humans) [11].

Goodrich and Schultz further mention that within the two subcategories, remote and proximate interaction, it is useful to distinguish between interactions that require mobility, physical manipulation, or social interaction [11]. Following terms illustrated in Table 2.1 are generated using this explanation: (1) Teleoperation: Remote interaction with mobile robots, (2) Tele- manipulation: Remote interaction with a physical manipulator (3) Robot assistant: Proximate interaction with mobile robots. This type of interaction may include a physical interaction (4) Peers/companions: Social interaction generally appears to be proximate rather than remote and includes the cognitive aspects of interaction. In this type of interaction, humans and robots interact as peers or companions. These terms are illustrated in Table 2.1 below.

Table 2.1: HRI deﬁnitions based on Goodrich and Schultz deﬁnition. HRI Remote Proximate Mobility Teleoperation Robot assistant: Physical manipulation Telemanipulation Physical robot assistant Social interaction Telecommunication Peers/companions

2.2 Teleoperation

As previously mentioned teleoperation is considered to be the remote interaction with mobile robots. However, it is argued that teleportation itself also could be further divided into subgroups depending on the degree of interaction. In fact, Milgram et al. introduce teleoperation from a director/agent control framework where the director is the human operator and the agent is the robot [20]. According to this deﬁnition, automation means the entire or the partial replacement of a function previously carried out by a human operator.

Similarly, Parasuraman et al. entail that automation is not all or nothing but a substance that can vary across a continuum of levels, from the lowest level to the highest level of full automation [29]. This deﬁnition is illustrated in Table 2.2 below.

5 2.3. Unmanned Ground Vehicle Teleoperation in Complex Environments

Table 2.2: Levels of Automation of decision and action selection Levels of Autonomy HIGH 10. The computer decides everything, acts autonomously 9. informs the human only if, it, the computer decides to 8. informs the human only if asked, or 7. executes automatically, then necessarily informs the human, and 6. allows the human a restricted time to veto before automatic execution, or 5. executes that suggestion if the human approves, or 4. suggests one alternative 3. narrows the selection down to a few, or 2. The computer offers a complete set of decision/action alternatives, or LOW 1. The computer offers no assistance: human must take all decisions and actions.

The lowest level of autonomy depicted in Table 2.2 shows a state of manual teleoperation while the highest level of autonomy shows a form of complete autonomous robotics. Con- sidering these levels of autonomy, Tseng and Mettler further discuss that the human operator can switch between modes of different levels of autonomy depending on the task the robot is performing [37]. In the task of searching, the operator could, for example, switch between manually controlling the robot or order the robot to do a pre-planned systematic scan of a subarea [21].

2.3 Unmanned Ground Vehicle Teleoperation in Complex Environments

A UGV is referring to a vehicle that can operate without the presence of an on-board human driver [42]. In popular speech, UGVs are usually referred to as autonomous vehicles. As of today, the development of vehicles with a certain degree of autonomy has become a matter of course for major vehicle manufacturers and research centres [4]. The research outcome shows that the industry of automotive has opted for the self-driving technology from a civil point of view and has become a reality in simple environments [45]. However, complex scenarios such as the military sector still have a long way to go [41] [19]. Simple environments refer to a structured environment with features such as streets, signs, trafﬁc lights, road marks and a network of communication [16]. Whereby, a complex environment refers to an unstructured environment that is regularly off-road with elements of assorted obstacles [22].

When developing an autonomous vehicle, following design elements have been identiﬁed: perception, decision-making and action [19]. Out of these elements, research has shown a greater focus towards exploring perception and decision-making since they are considered to be more complex in comparison to action [19]. Perception refers to the perception of the vehicle such as its sensor technology that allows for localisation and notiﬁcation. Decision-making refers to the tasks related to making decisions when the vehicle is navigating, driving and maneuvering [19].

In addition to the design elements for autonomous vehicles, developing a military UGV requires a broader scope of elements. In this case, five main elements have been identified. These elements are considered to be the platform model, the control/monitoring station, the sensors, the environment and the missions [22]. These five elements are depicted in Table 2.3. As seen in the table, The platform model refers to how the UGV is designed to be transported. The aspect could take advantage of a taxonomy for military UGV based on size. The sensors refer to the sight and perception of the military UGV such as computer vision systems and global positioning systems. This is used to identify for example obstacles. The task of the control/monitoring station is to manage the needed information by remotely sending/receiving commands to the UGV. Furthermore, environment and missions refer to the physical conditions and the tasks the UGV is performing.

6 2.4. Interactive System Design in Telerobotics

Table 2.3: Five design elements when developing a military UGV How the UGV is designed to be transported. Light-sized - transported by personnel or in a backpack. Mid-sized - transported to the desired location by another larger Platform model vehicle. Heavy-sized - classiﬁed as ordinary vehicles that can be teleoperated. Control/monitoring station Information managing by remotely sending/receiving commands to the UGV. The sensors Sight/perception of the UGV. E.g. computer vision systems and global positioning systems. The environment The physical conditions The missions The task of the UGV. E.g. navigating

Looking at the usage of UGVs it is clear that the unmanned ground platforms, of various sizes and functionalities, are growing rapidly where mainly countries such as the US, Great Britain, Russia, and China are investing considerable resources [30]. However, the UGV that is relevant in this thesis is the mid-sized UGV model THeMIS illustrated in Figure 2.1. Cur- rently, several countries including the United States (Marine Corps), France, Norway and the Netherlands, are conducting pilot activities with THeMIS, mainly for logistics and intelligence, surveillance, and reconnaissance [30].

Figure 2.1: The THeMIS UGV 5th generation. By Milrem Robotics(This ﬁle is licensed under the Creative Commons Attribution-Share Alike 4.0 International license.

2.4 Interactive System Design in Telerobotics

An interactive system is deﬁned as a combination of systems that users interact with in order to achieve a speciﬁc goal and the components of an interactive system is called the user interface [36]. The user interface serve as a form of interaction between two systems that involves inputs and outputs where at least one of the systems is a human user [5]. When designing a user interface, it is essential to remember that the user interface is equally important in all levels of autonomy. Even if the robot is capable of operating without any assistance from an

7 2.4. Interactive System Design in Telerobotics operator (the user), it still needs to communicate to the operator how and what it is performing during a task. This is of particular interest when the robot fails to complete a mission or encounters problems during the task. Hence, as robots become independent, interfaces are used less for control, and more for monitoring and diagnosis [10].

Whether the user interface is used to monitor or control the task, it still has a fundamental task of providing the human operator with good tactical system awareness in order to allow for mission effectiveness. This is especially important in emergency situations that include uncer- tainty factors that can inﬂuence the level of a disaster [31]. The role of the human operator is, therefore, (1) to make sense of the information provided, (2) share a collective understanding of the reality, and (3) in the same time evaluate the consequences of the event using all avail- able resources [31]. Therefore, the design of the interactive system automatically becomes a crucial aspect due to its strength to inﬂuence situation awareness and performance [9].

For many years, the history of interactive systems for control of ground vehicles used in the military has been permeated by laptop-based-systems. Up to this date, there are only a few examples of handheld interactive systems and even fewer interactive systems that have addressed to be portable [40]. The area of interactive UGV systems that are addressed to be portable is thereby in its precursor. These first-of-a-kind systems are argued to benefit from conceptual design processes by enabling the users to envision the system [17]. Moreover, it is mentioned that more empirical research is needed to design user interfaces for robotic systems to investigate and explore design features’ effects [27]. It is emphasized that achieving successful HRI requires a focus on cooperation acceptance and cognitive workload as well as cov- ering performance and satisfaction. Designers of robots should carefully choose design features to balance specific effects and implementation costs concerning tasks, work design aims, and employee needs in the specific work context [27].

In summary, there is a need for a more user-centered approach when designing robots in order to find a balance of design features in regards to user needs, work design aims, and the specific context of the environment. When designing for robotic control following aspects are important to take into consideration: (1) the level of control sharing between human-agents (2) human factors and cognitive workload (3) the specific work context.

8 3 Methods

In order to understand and explore the area of interactive systems design for remote control of military UGVs, a user-centered approach was carried through with the methodology of a Research Through Design Case Study. The design process included four phases: Exploration, Concept Generation, Prototyping, and User Evaluation. This chapter provides a methodological background of the activities performed in this thesis. The results of those activities are presented in the succeeding chapter.

3.1 Research Through Design Case Study

The research in this study was inspired by a Research through Design approach which means that the research is developed through design activities [13]. The design activities in this research was performed as a user-centered case study [34] that is illustrated in Figure 3.1. The figure shows a simplified research through design process in relation to the phases in this specific research case study.

Research through Design

Object, process is New information, Background research Object is created analysed new/modiﬁed theory

Phases in this research study

Exploration Concept Generation

Prototyping

User Evaluation

Figure 3.1: The modiﬁed Research through Design used in this thesis

9 3.2. User-Centered Design Approach

3.2 User-Centered Design Approach

The case study followed a user-centered design, which is a design approach that aims to improve the understanding of user needs and the iteration of the design with the aid of active user involvement [18]. The user-centered design approach was valuable as the ﬁeld of remote-controlling unmanned ground vehicles in military environments is relatively new. This means that a purely theoretical research study would have led to limited results. An overall view of the design process can be shown in Figure 3.2 below.

Concept Exploration Prototype User Evaluation Generation

"What characterizes the context of use "How should the for infantry soldiers "What are the infan- remote Research on the field and try soldiers’ control for an inter- Question what is important to needs in relation to active UGV system consider in the an interactive UGV be designed for design of an interac- system on the field?" infantry soldiers on tive UGV system for the field?" this context?".

- Qualitative Context - Concept Genera- - Low-ﬁdelity Proto- - Demonstration observation tion with Design typing Video How? Drivers - Explorative Inter- - Prototype Evalua- - Technology Accept- views - Itemised Response tion ance Model and PMI - Task-based Testing

To understand the To create a tangible To test the concept user needs and To understand the prototype for user and the acceptance Why? hence the Design user context. testings. of an interactive UGV Drivers in an interac- system tive UGV system.

Figure 3.2: A process picture that gives an overview of the user-centered research approach.

3.3 Exploration Phase

The aim of the Exploration Phase was to answer the research question: "What characterizes the context of use for infantry soldiers on the ﬁeld and what is important to consider in the design of an interactive UGV system for this context?". The following subchapter explains the methods used in the Exploration Phase to answer the question. The methods used in the phase include Qualitative Context Observation and Explorative Interviews.

3.3.1 Qualitative Context Observation: The activity of taking ﬁeld notes on the behavior and actions of individuals is referred to as a qualitative observation [6]. Early in the design process, the report author participated as a passive observant in testing for Unmanned Aerial Vehicles (UAV). UAVs are more developed and frequently used than UGVs, and are thereby suitable objects for observation to learn about participant response towards unmanned systems.

In the testing, a user scenario was conducted with UAVs to demonstrate the concept of UAVs in the military context as well as identify the desired qualities of the user interface and func-

10 3.3. Exploration Phase tions of the UAVs. Therefore, the testing aimed to understand the context environment, interact with the users, and get an insight into functions required and wanted when designing an interactive system of a UGV. All of the gained insights were noted down and later tran- scribed. The testing set involved a user scenario of ﬁnding enemy soldiers using a UAV. Land mines were set up on the ﬁeld, and two people in the test group role-played as enemy soldiers hiding in the ambush. The UAV monitor was connected to a large display (shown in Figure 3.3) to demonstrate UAVs’ usability when searching for land mines and hidden enemy soldiers. The test participants consisted of two focus groups with soldiers from the home guard, 10-15 soldiers participated in each focus group.

Figure 3.3: The participants watching the monitor of the remote control of the UAV on big screen

3.3.2 Explorative Interviews: The main goal of the explorative interview [32] was to learn about the end-users. Semi-formal interviews were carried through with two soldiers from the home guard, one paramedic and one group leader.

The questions asked focused on:

• Introduction and military context (profession, education, experiences)

• User story (Experiences, problems, highlights)

• Futuristic (If you had a magical helper what would it do?)

• Autonomous vehicles

The data obtained by the Qualitative Context Observation and the Explorative Interviews with the soldiers from the home guard gave valuable insights that were analysed and con- secutively clustered around the themes Context, Functions and Qualities.

11 3.4. Concept Phase

3.4 Concept Phase

The Concept Phase aimed to answer the research question: "What are the infantry soldiers’ needs in relation to an interactive UGV system on the ﬁeld?". The following subchapter explains the methods used to answer the question. The methods used was Concept Generation with Design Drivers and Itemised Response and PMI.

3.4.1 Concept Generation with Design Drivers As this is an iterative process, the outcome from the Exploration Phase, the design drivers illustrated in Figure 3.4 below, were used as a basis in the concept generation. This could be referred to as the ﬁrst step in reversed engineering [28].

In this type of reversed engineering, the product was seen as a “black box” and the design drivers were seen as the input while the output was the generated holistic concepts. An example of the holistic concepts generated from chosen design drivers is illustrated in Figure 3.5 below. The intention, of the speciﬁc method usage, was to understand the overall product function in the comprehensive view and identify different types of concepts in an ‘unbiased’ perception of possible product evolution. The design drivers were chosen randomly to opti- mize creativity.

Main function Simple Perceive as Carry if it is in the background

Adaptable

Interpret instructions

Figure 3.4: The Design Drivers. Con- Figure 3.5: Example of a concept generated us- text, Function and Qualities ing the design drivers

3.4.2 Itemised Response and PMI The Itemised Response and PMI [38] was used to systematically examine positive, negative and interesting aspects of each holistic concept. During a workshop session, the method was carried through with two of the client representatives from the Swedish Defence Research Agency. The client representatives got the task to list each concept’s positive and negative features. This method was used to overview the most promising directions to develop further. The importance of client involvement was to identify interesting design ﬁelds and better understand the solution space. After the workshop with the client representatives, the char- acteristics of the most promising concepts were broken down and the PMI concept screening was analyzed.

3.5 Prototyping Phase

The Prototyping Phase aimed to answer the research question: "How should the remote control for an interactive UGV system be designed for infantry soldiers on the ﬁeld?". It

12 3.6. User Evaluation Phase further explains the methods for Low-ﬁdelity Prototyping and the methods used in Prototype Evaluation.

3.5.1 Low-fidelity Prototyping Low-fidelity prototypes were created iteratively, by first using paper prototypes and finally using Figma as an interface design tool. A low-fidelity prototype is a prototype version that is complete enough to demonstrate the value it brings to the users. It contains the "bare bones" set of features to test with users allowing for people-driven innovation [39].

The prototype was iterated continuously by informal guerilla testings. The guerilla testing method aims to gather user input by taking a low-ﬁdelity prototype into the public domain and ask for feedback. The method’s simplicity allows ideas to be tested quickly and at a low- cost [3]. In this case, the guerilla testing was performed with the Swedish Defence Research Agency employees.

3.5.2 Prototype Evaluation After the prototype iterations, a semi-formal user testing was planned to evaluate the wire- frame prototype’s functionality. In the user test, five participants were individually asked to perform specific tasks using the system. The participants were researchers at the Swedish Defense Research Agency with a previous background in The Swedish Armed Forces. The prototype testing involved the following three tasks: (1) Plan a route to a given point, (2) Plan a reroute passing the given point (3) Evaluate the user interface. While performing the tasks and integrating with the prototype, the participants were continuously asked to think aloud. This method is called Thinking out Aloud and it means to ask the participants to verbalize their thoughts as they move through the user interface [14] [25]. The numbers of participants needed in Thinking out Aloud tests have been discussed throughout time and Nielsen suggests that five test subjects are enough to find the majority of problems [23].

Following the task-based test, a post-session interview based on Nielsen’s ten heuristics [24] was conducted with each participant. The information gained from the method was used in order to guide the future design and a ﬁnal design was formed.

Effectiveness, efﬁciency, and satisfaction are recommended parameters in usability metrics according to the ISO/IEC 9126-4 [1] and the post-session interview was used as a tool to evaluate users’ subjective satisfaction with the aspects of the human/computer interface. In comparison, the task-based activity was used to evaluate effectiveness and efﬁciency.

3.6 User Evaluation Phase

The User Evaluation Phase aimed to test the ﬁnal design concept of the remote control with the end-users. This was done through two studies. Study 1 focused on the concept placed in a scenario, while Study 2 focused on the design of the user interface.

3.6.1 Study 1: Concept To test the usefulness and the area of use, a demonstration video of the ﬁnal design was created. This demonstration video showed three driving control modes of the remote control. The user testing consisted of 20 participants, aged 20-29, that work at The Swedish Armed Forces. The test was done remotely and started by showing the participants the video of the concept that demonstrated the control of the UGV. The video showed three control modes. One mode when the UGV is at its highest level of autonomy. Another mode when the control of the UGV is semi-autonomous meaning that the UGV is following the pilot. Finally, the

13 3.7. Research Ethics third mode of control, that is manually manoeuvring the UGV. After this concept video was shown, participants were asked to ﬁll out a questionnaire.

Six questions, see Table 4.10, were asked to measure the perceived usefulness of the system. These questions were taken from the Technology Acceptance Model (TAM) [8].The specific TAM used was a modified TAM by Lewis. In this modified TAM (mTAM)[15], the participants were asked to provide their level of agreement on a 7 point scale (1=Completely dis- agree and 7 = Completely agree). According to Davis [8], usefulness is 1.5 times more important than ease of use since it has a bigger impact on the users’ attitude towards the system. The TAM questions were followed by three questions regarding three scenarios and how the participant would use the UGV in each case. The three scenarios were: Combat, Reconnoitre and Logistics and the control modes to choose between were the same as the ones showed in the demonstration video. Possibilities for free text answers were added to each question in the survey to provide the participant with the opportunity to explain their answers.

3.6.2 Study 2: User Interface Five participants out of the 20 participants from Study 1 took part in Study 2. During Study 2 the participants were asked to perform a simple route-planning task using the remote control. This was done by sharing the screen of the wire-frame prototype made in Figma with the participant. The participant was asked to verbalise the thinking process during the task. This task was followed by a semi-structured interview. The semi-structured interview revolved around the application of the system and the areas of use of a UGV in a military context. The interviews were done remotely. The main purpose with the individual testing was to evaluate the user interface of the current design as well as to get in-depth answers of the concept of UGVs.

3.7 Research Ethics

The participants that have been of relevance in this thesis participated voluntarily. The par- ticipation was confirmed either verbally or written. The participants were explained that the study is in connection with a master thesis. They were also explained that their answers will be saved locally and treated confidentially. Further, it was explained that all processing and performance reporting will be performed with de-identified data. Analysis and presentation of collected data take place collectively for the entire group’s results and not at the individual level.

14 4 Results

The material in this chapter is based on the methodological approach that was presented in the preceding chapter. Subchapter 4.1. shows the results of the Exploration Phase. The concepts are presented in subchapter 4.2, and the prototypes are shown in subchapter 4.3. Finally, subchapter 4.4 describes the outcomes of the user tests with the end-users.

4.1 Exploration Phase

The Exploration Phase was aimed to answer the research question: "What characterizes the complex environments in military settings and what is important to design for in the situation?". The data obtained by the methods used in the phase, Qualitative Context Observation and the Explorative Interviews, with the soldiers from the home guard gave insights regarding context identiﬁcation, shown in Table 4.1 below. Context is identiﬁed as both physical context, which refers to the physical environment that the UGV is driving in, and the psychological context, which refers to the cognitive mindset of the soldier likewise UGV pilot.

Table 4.1: Context Identiﬁcation of Military UGVs Context Physical Psychological ¨ Rough terrain ¨ Exhausted soldiers ¨ Mobile environment ¨ Stressful ¨ External noise ¨ Time pressure ¨ Urban envrionment ¨ Tired soldiers ¨ Dirty environment ¨ Dangerous environment ¨ Changing light condition

Furthermore, functions and qualities were identiﬁed and described as Design Drivers. See Table 4.2. The design drivers focus mainly on the design drivers of the UGV rather than the design drivers of the user interface of the control of the UGV.

15 4.2. Concept Phase

Table 4.2: Design Drivers of Military UGVs Functions Qualities ¨ To spy ¨ Smart ¨ To deliver ¨ Loyal ¨ To carry ¨ Controllable ¨ To transport ¨ Adaptable ¨ To localize ¨ Secure ¨ To offer protection ¨ Discreet ¨ To lead ¨ Perceive as if it is in the background ¨ To mislead ¨ Portable ¨ To guard ¨ Independent ¨ To watch ¨ Durable ¨ To lead different platforms simultaneusly ¨ To chase ¨ To pursue ¨ To plan ¨ To give feedback ¨ To interpret instructions

In addition to the Design Drivers and the Context Identiﬁcation, the results from the Exploration Phase also identiﬁed the end-users. The outcome showed a need to design for home guard soldiers since they are usually less trained than the rest of the Swedish Armed Force.

4.2 Concept Phase

After the Exploration Phase, the aim was to conceptualise the outcome and understand the user needs. Therefore, the Concept Phase aimed to answer the research question: "What are the user needs of the intended soldiers?". Based on the result from 4.1 five concepts were generated. Following five figures with connecting tables illustrate the concepts and their positive, negative and interesting aspects.

Figure 4.1, describes a concept with the main function to guard and the qualities to be controllable and to be perceived as if the UGV is in the background. This concept works as an integrated user interface system consisting of a watch and a remote controller. The function of the watch is to notify the pilot through vibrations when needed. This could for example be when the UGV notices enemies approaching. The multi-modal function would thereby allow the pilot to rest when active control is not needed. Table 4.3 describes the positive, negative and interesting aspects with Concept 1.

16 4.2. Concept Phase

Main function Perceive as if it is in the background Guard

Controllable

Figure 4.1: Concept 1 with main function to guard

Table 4.3: PMI on Concept 1 Positive Negative Interesting ¨ Multi-modal ¨ Could be easy to manipulate. What ¨ Geofencing, assign does it see and what does it think that zones it is seeing. ¨ Allows to be pas- ¨ Objects are easier to identify than ¨ Prototype for guard- sively alert living things ing

17 4.2. Concept Phase

The second ﬁgure, Figure 4.2, describes a concept with the main function to transport and the quality to be independent. This concept focuses on the complete autonomy of the UGV and pictures the UGV as a provider of necessities. Table 4.4 describes the positive, negative and interesting aspects with this concept.

Main function

Transport Independent

Figure 4.2: Concept 2 with main function to transport

Table 4.4: PMI for Concept 2 Positive Negative Interesting ¨ Automated logistics for food is not ¨ Follow route A to B that interesting ¨ "Find me" - function

18 4.2. Concept Phase

Concept 3, see Figure 4.3, describes a concept with the main function to carry, the subfunction to offer protection and the qualities to be controllable and perceive as if it is in the background. The idea of this UGV concept is to act as carrying aid for wounded soldiers. It further allows the human operator to attach the remote control to the UGV and use it as a steering wheel. Table 4.5 describes the positive, negative and interesting aspects with the concept.

Main function Perceive as if it is in the background Carry

Controllable

Oﬀer protection

Figure 4.3: Concept 3 with main function to carry

Table 4.5: PMI for Concept 3 Positive Negative Interesting ¨ Controlling with gesture ¨ What happens to the UGV- ¨ A function to destroy itself control if the pilot gets shot ¨ Click function, allowing ¨ It needs to be able to weigh to maneuver the vehicle on the load the UGV ¨ The remote control and ¨ Minimal amount of log in what is shown on the screen is needed ¨ Following something ¨ Finding things

19 4.2. Concept Phase

Concept 4, in Figure 4.4, distinguish a concept with the main function to carry, the subfunction to interpret instructions and the qualities to be simple, adaptable and perceive as it is in the background. The main function of this concept is similar to Concept 3 (Figure 4.3) however the concept tend to be different due to the desired functions. These functions put the human operator in a passive control where the UGV is following the operator with the connected boots. The control of the UGV is through tactical responses. Table 4.6 describes the positive, negative and interesting aspects with the concept.

Main function Simple Perceive as Carry if it is in the background

Adaptable

Interpret instructions

Figure 4.4: Concept 4 with main function to carry

Table 4.6: PMI for Concept 4 Positive Negative Interesting ¨ Gestures ¨ You are always moving. ¨ Learn the ground surface Will it recognise what it is going to follow ¨ Adapts to the pilot ¨ Connected system ¨ Gives feedback

20 4.2. Concept Phase

The last concept, presented in Figure 4.5, describes a concept with the main function to spy and the qualities to be loyal and smart. In this concept, the emphasis is on the communication between the UGV and the human operator. The idea is that the interaction system is some kind of helmet of the human operator that provides the operator with a mental connection with the UGV. This enables the human to "feel" what the UGV is sensing. If the UGV identiﬁes danger for instance then the human operator might "feel fear". Table 4.7 that describes the positive, negative and interesting aspects with the concept.

Main function Loyal

To spy Smart

Figure 4.5: Concept 5 with main function to spy

Table 4.7: PMI for Concept 5 Positive Negative Interesting ¨ Image interpreter ¨ Consumables. You do not want ¨ Learn the ground sur- emotional connection face ¨ The robot learns how the soldiers in the group think and can map out solutions

All in all, the result from the Concept Phase indicated that the clients are mainly interested in the remote control device and the graphical user interface of the screen, see Concept 3 (Figure 4.3, Table 4.5). However, there is an open-mind towards the multi-modal aspects seen in Concept 1 (Figure 4.1, Table 4.3), Concept 3 (Figure 4.3, Table 4.5) and Concept 4 (Figure 4.4, Table 4.6). The table below, Table 4.8, concludes the main functions of the UGV and the speciﬁed functions the user interface of the control system that should take into consideration when prototyping.

21 4.3. Prototyping Phase

Table 4.8: The main functions of the UGV and the speciﬁed functions of the user interface. Main functions Speciﬁed functions of the user interface ¨ To spy ¨ Show position of vehicle related to the pilot ¨ To guard ¨ Allow manual steering mode to maneuver UGV ¨ To carry ¨ Show battery level of vehicle ¨ To plan ¨ Show speed of vehicle ¨ To give feedback ¨ Allow to change sight mode: IR, map, camera view ¨ Include a follow-me button ¨ Create a navigation route ¨ On/off - button ¨ Allow steering mode to maneuver camera view ¨ Be able to navigate on map

4.3 Prototyping Phase

The outcome of the Concept Phase served as an input for the Prototyping Phase. This phase, aimed to answer the research question: "How should the remote control for an interactive UGV system be designed for infantry soldiers on the ﬁeld?".

The ﬁrst designs of the graphical user interface of the remote control was based on the functions identiﬁed in Table 4.8. The inhouse guerilla testing of the wireframed prototype resulted in a fast prototype iteration of the graphical user interface. The evolution of the prototype is presented in Figure 4.6 to Figure 4.13.

n n w e KM w e 5 KM 5 s s 20 KM/H 20 KM/H ERROR 22 H 33 MIN MESSAGE! 22 H 33 MIN

Figure 4.6: Dashboard top bar design Figure 4.7: Dashboard top bar design proposal 1 proposal 2

w e KM 22 H 33 MIN descriptive text 5 20 KM/H 5 km 20 km/H 14 H 33 min s ERROR BUTTON BUTTON MESSAGE!

Figure 4.8: Dashboard bottom bar Figure 4.9: Dashboard bottom bar design proposal 1 design proposal 2

22 4.3. Prototyping Phase

Choose path Path 1 Path 2 Path 3 1 2

n w e 5 KM s

20 KM/H n ERROR w e 5 KM MESSAGE! 22 H 33 MIN 3 s 20 KM/H ERROR MESSAGE! 22 H 33 MIN Figure 4.10: Scenario 1: Identify subjects on map Figure 4.11: Scenario 2: Plan a route

13:37 fastest route on track CURRENT STATUS 5 km 20 km/H 14 H 33 min 02/12/2020 5 min 1500 m

BUTTON BUTTON BUTTON BUTTON

BUTTON ir 5864.329,1533.688 5865.253,1537.886

BUTTON ir 5864.329,1533.688 5865.253,1537.886 Figure 4.13: Latest dashboard design Figure 4.12: Dashboard design out- outcome from the Design Detailing come phase

The Prototype Evaluation resulted in the design changes themed into the three categories:

1. Desired features - Not a priority, but a wish

2. Interesting aspects - Aspects to further investigate

3. Necessary aspects - Aspects that need to be handled with highest priority.

Table 4.9: Desired, Interesting and Necessary aspects of the prototype. Desired Interesting Necessary ¨ 50/50 camera-map screen ¨ Compass on map, relative ¨ Cancel/Stop button option to me as a pilot. Compass on camera, relative to the UGV ¨ In Swedish instead of En- ¨ Make the “Feedback- glish if it is designed for the of-UGV-message-box” Swedish Armed Forces clickable for more information of the process ¨ Scalemeter on map

These design aspects were taken into consideration and the ﬁnal prototype design shown in Figure 4.14 was created:

23 4.4. User Evaluation Phase

levels 1. Follow Pilot. 3. Semi-autonomous. of control UGV is following pilot.

2. Plan Route. Highest level of autonomy. UGV is monitored by pilot. Battery level of the remote control 3. Manual Mode. UGV is maneuvered actively by pilot.

Maneuver UGV Camera View

Maneuver UGV

Selection button

Find Pilot. Find UGV. Locate Pilot Locate UGV on map on map

Figure 4.14: Final prototype of UI Design for remote control

4.4 User Evaluation Phase

The User Evaluation Phase aimed to test the ﬁnal prototype design with its end-users. The process was divided into two studies. The goal of the ﬁrst study was to evaluate the concept and its future use cases. While the second study focused on testing the end-user interaction with the user interface.

4.4.1 Study 1: Concept The result of Study 1 is divided into 2 parts; Technology Acceptance Questions and Scenario Questions.

Technology Acceptance Questions Table 4.10 show the average values of the six TAM questions. The mean numbers show a slight shift towards the negative spectrum, as the Likert scale is from 1-7 and the mean (µ) re-

24 4.4. User Evaluation Phase sults vary from 3.7-3.95. The standard deviation (σ) measures the variation in the participant answers. With the lowest value of 1.34 in question 1 and the highest value of 1.53 in Question 6. The average perceived usefulness (M) is 46.25 from a scale of 0 to 100, with a standard deviation (SD) of 20.36 and a 95% conﬁdence interval (CI) shown in Table 4.11. Table 4.11 is followed by the free text answers that have been summarised.

Table 4.10: The average answers to the six questions that revolved around usefulness. Questions µ σ 1. Using [this product] in my job would enable me to accomplish tasks more quickly. 3.8 1.44 2. Using [this product] would improve my job performance. 3.65 1.46 3. Using [this product] in my job would increase my productivity. 3.95 1.43 4. Using [this product] would enhance my effectiveness on the job. 3.7 1.45 5. Using [this product] would make it easier to do my job. 3.7 1.34 6. I would ﬁnd [this product] useful in my job. 3.85 1.53

Table 4.11: Perceived Usefulness of the Average Participant M 46.25 SD 20.36 95% CI [37.33, 55.17]

Question 1: Accomplish tasks more quickly. The free text answers indicated that the participants could see the beneﬁt with a UGV but not in their ﬁeld (airbase and security). The answers also reported on trust issues towards the involvement of robotic systems. Further- more, participants expressed that it is hard to know if the system would enable to accomplish tasks quicker without physically testing the system.

Question 2: Improve work performance. Participants mentioned that it depends on the task performed. The system would improve work performance if it is used in the case of simple patrol or surveillance missions that allow the soldiers to recover. Participants also mentioned that sending a UGV to control unknown objects/ﬁndings or critical terrain would save time, lives and would allow for taking better decision making. On the other hand, one participant also mentioned the convenience of letting a human perform the work since the human know how to do it and how it has been done previously.

Question 3: Increase productivity. In the case of productivity, the participants mentioned that the system would increase the productivity if it is used for simple tasks (patrol or surveillance missions) enabling soldiers to solve more important missions in the meantime. Since the soldiers work with different types of systems they ﬁnd it of high importance to maintain knowledge about managing the UGV in order to strive for productivity using the system.

Question 4: Increase effectiveness. In the case of effectiveness, the participants pointed out that they believe the human is able to detect important details of a higher precision compared to a robot.

Question 5: Make it easier to do the job. The free text answers indicated that the soldiers see a potential mainly in terms of safety of personnel.

Question 6: Useful in the job. In the case of usefulness, the participants expressed that they could see useful aspects of the system that could be used as complement in their work. For instance the system could be used to cover large areas or indicate/locate the opponent at an early stage. Participants did however mention that they see more needs of the system in other military units.

25 4.4. User Evaluation Phase

Scenario Questions Figure 4.15 illustrate the outcome of the Scenario Questions in Study 2. The diagram com- pares the desired control modes in the tested scenarios: Combat, Reconnoitre and Logistics. The diagram is followed by a summary of the free text questions related to the Scenario Questions.

Figure 4.15: Results from the Scenario Questions.

Scenario 1: Combat In the case of Combat, the majority of the participants (14 participants), decided to send the UGV in front of the group and manually control it. 11 out of these 14 participants chose this control mode in order to keep the soldiers safe and 3 participants chose this control mode due to that they did not trust the UGV to take own decisions in uncertain conditions as Combat. 3 participants chose the semi-autonomous mode of letting the UGV follow the operator. Two out of these three chose this option in order to shift their focus towards the terrain/environment instead of looking into the screen. The third one chose this option due to the value of having the UGV as protection in an uncertain environment. Fur- thermore, one person chose the option to let the UGV drive in its highest level of autonomy out of the reason that the participant wanted to be able to plan for possible ﬁre attack and in that case, it would be easier if the UGV could handle itself. Another answer given by a participant was to send the UGV in another part of the combat ﬁeld to disturb and create confusion.

Scenario 2: Reconnoitre The majority of the participants (17 participants) decided to control the UGV manually. Eight participants mentioned that they would use the UGV manually in order to place the UGV at a stationary point and be able to control the sight of the UGV manually. Seven participants chose to control the UGV manually due to safety reasons and lack of trust to the UGV. Moreover, two participants chose the option to let the UGV follow pilot. The reasoning mentioned was to keep the UGV close as protection. Finally, one participant decided to send the UGV in its highest level of autonomy with the reasoning that if the

26 4.4. User Evaluation Phase

UGV gets hit then the area has been localised and enemy troops have been identiﬁed and if it makes it then the area has been secured.

Scenario 3: Logistics In the situation of sending the UGV to get necessities, the majority (14 participants) decided to use the UGV in full-autonomous degree. The participants mentioned that this would enable the soldiers to focus on more important tasks as well as keeping the soldiers away from unnecessary danger. One participant chose to manually control the UGV due to lack of trust to a UGV. Five participants mentioned the importance of involving at least one soldier in the process since soldiers know what kind of necessities that are needed.

Questions about the UGV Table 4.12 summarizes the opportunities and limitations that the participants of the study expressed with military UGVs. Table 4.13 summarizes the potential use cases of military UGVs.

Table 4.12: Opportunities and limitations with military UGVs Opportunities Limitations ¨ Extra protection ¨ Trust issues ¨ Another sensor ¨ Exposure of soldiers. ¨ Carry heavy packing ¨ Technical failure ¨ Use UGV in dangerous situations to save lifes ¨ How do I make sure it does what it says it is doing

Table 4.13: Potential use cases with military UGVs Potential use cases ¨ Distraction ¨ Logistics ¨ Demining ¨ Reconnaissance tool ¨ Stationary monitoring

4.4.2 Study 2: User Interface All the participants managed to perform the task of planning a route successfully. However, two of the participants mentioned the desire of pointing out navigation points manually instead of trusting the route made by the system. One participant mentioned that the buttons could be too small concerning the use of gloves but that it is beneﬁcial that the system has buttons instead of touchscreen. Another participant pointed out the button of the compass as unnecessary since soldiers usually have a compass on themselves, while another participant mentioned that it would be good to have the compass on the monitor.

The result from the post-session interview with the ﬁve participants showed a pervading openness towards UGVs in military contexts. The interviews indicated that the participants have a concern towards involving robots in decision-making processes and trusting the robot to take the best possible road towards a destination. The participants however mentioned that they would use UGVs in safe situations. For example to get necessities. One of the participants mentioned that the UGV could serve as the soldiers extended eyes and ears which puts a huge weight on the optics of the system. Another aspect mentioned was that the size of the UGV would have huge effect on how the soldiers would use the concept.

27 5 Discussion

The main research question of this thesis was: How do we design interactive unmanned ground vehicle systems for infantry soldiers on the ﬁeld? The following chapter will discuss the implications of the result thoroughly by discussing the result per subquestion. It will also discuss the limitations of its methods used, and the work in a wider context.

5.1 Implication of the results

The implication of results will be discussed per sub-research question, as they cover the research done in this thesis.

1. What characterizes the context of use for infantry soldiers on the ﬁeld, and what is important to consider in the design of an interactive UGV system for this context?

The ﬁrst step to designing an interactive UGV system is understanding its context. The outcome of this thesis identiﬁed that the context for a military UGV is not necessarily only the environment and the physical conditions prevailing [22] but also the psychological context of the human operator (Table 4.1). The human operator’s psychological state is described as an aspect that could affect the mental workload, and the decision-making ability [27].

When talking about UGVs, the ultimate state is described as the level of autonomy where the UGV is teaming with the human. Granda et al. refers to this state as the Virtual Symbiosis [12]. Nevertheless, one could argue whether the UGV need to be at its highest level of autonomy in order to do its job. In fact, the outcome of the research study, indicated that different work contexts and tasks such as Combat, Reconnoitre and Logistics require different levels of autonomy (see Figure 4.15). In Combat and Reconnoitre, the majority of the participants wished to have a manual operator role, and in the case of Logistics the majority of the participants would be more likely to give the UGV a higher level of autonomy. One explanation for this could be a lack of trust towards letting a military robot take decisions in uncertain situations. However, this could also be because the system does not exist as of now, which gives the participant a hard time truly knowing if they would beneﬁt from full autonomy. This lack of knowledge of their desires in relation to the doubt towards the UGV’s capabilities might

28 5.1. Implication of the results put the participant in a position thinking that having full control of the UGV is the most convenient way to maneuver it.

2. What are the infantry soldiers’ needs in relation to an interactive UGV system on the ﬁeld?

The needs of the infantry soldiers were explored in iterations. Firstly, functions and qualities of the military UGV were identified as important design drivers that affect the concept of the interactive system (see Table 4.2). These design drivers were then conceptualized into holistic concepts of interactive design systems (see Figure 4.1-4.5) that resulted in a list of main functions for the remote control as a part of an interactive UGV system (Table 4.8). The remote control acted as a way to conceptualize the relationship between the human and the UGV into something tangible that the end-users could relate to in the process to further explore the needs of an interactive system. In fact, one could argue that the research study orbited around identifying the usage of an interactive UGV system, and thus explore the soldier’s needs, with the help of the early holistic concepts and the low-fidelity prototype of the remote control. This was especially highlighted in the User Evaluation Phase, where the final users got to test the interactive UGV system. Moreover, the user evaluation showed that the users have different needs compared to specific tasks (Combat, Reconnoitre and Logistics) that the UGV performs (see Figure 4.15). In addition, the results obtained from this research indicate that the participants would like to test the control system in proximate relation to the UGV to further understand the functionalities and capabilities of the UGV and its response level when giving it instructions and successively build up trust towards the new system. The trust issues that the participants were experiencing with the system highlighted that an iterative user-centered approach is required to further introduce the UGV as a member of a human team. Moreover, the iterative process also served as a starting point in finding the real end-users within the military troops, which could be infantry soldiers in the Home Guard (see Chapter 4.1).

To answer the question, when designing a ﬁrst-of-a-kind system such as an interactive UGV system in this thesis, it is not easy for the user to know what their needs are nor what the new system’s contribution is. However, by introducing the user in a conceptual design process, it becomes easier for the user to envision the new system [17]. The research in this thesis could therefore serve as the ﬁrst iteration of designing an interactive UGV system. Furthermore, the user involvement needs to be maintained during the next phases of development in order to achieve an end-product that cooperates with the user needs [27].

3. How should the remote control for an interactive UGV system be designed for infantry soldiers on the ﬁeld?

The remote control should allow the operating soldier to feel that the UGV serves as extended ears and eyes (see Chapter 4.4.2). This indicates that the soldiers require to be able to see how the UGV is performing on a high level, and therefore a graphical user interface is suggested. Moreover, it should also be designed so that the soldiers’ uniforms and gears are not limiting the use of the remote control (see Chapter 4.4.2). For instance, this could be by making the buttons big enough to wear gloves and having physical buttons instead of touch. In other words, the remote control should be designed in a way that is suitable for the military context that involves environments such as rough terrain, dirt and urban environments (4.1). Furthermore, the remote control design should also allow the operating soldier to change the level of control based on different contexts of use (see Table 4.13). The results show that as of today, the Swedish Armed Forces are not ready for a fully autonomous UGV. Nonetheless, several potential future use cases where the UGV serves as a complement to the operator were identiﬁed (see Table 4.13). This shows an openness to involve UGVs in some parts of the military work, however, the context of when and how needs to be further investigated. It

29 5.2. Limitations with the method also show that the highest level of autonomy (e.g. Level 10 in [29] or Virtual Symbiosis in [12]) might not be the ultimate goal for a UGV to facilitate in military conditions. In the mentioned use cases, the common denominator is that the UGV is not intended to be used in situations that might put humans in danger, allowing the human operator to serve in other situations that the individual ﬁnds as "more important".

Finally, in order to design the graphical user interface of the remote control, an essential aspect identified is whether the participants find the interactive system useful in the first place [8]. The average soldier’s perceived usefulness resulted in a value of 46.25 on a scale of 0 to 100. With a confidence level of 95% the perceived usefulness is between 37.33 and 55.17. This indicates that the soldiers see the potential of the usefulness of the UGV but are not entirely convinced. This could be because the end-users were within the airbase security and not the intended infantry users. It could also be because the UGV does not exist within the Swedish Armed Forces today, which makes it hard to understand the use of the system.

All things considered, to design an interactive system, the most critical aspect is user acceptance. If the user is not accepting the new system in the ﬁrst place, then the user interface’s detail design is useless. To accept the concept of military UGVs it is necessary to understand the context of use and the needs of the users. This is suggested by involving the end-users in the conceptual design process. Lastly, an interactive UGV system does not necessarily need to be fully autonomous in order to achieve autonomy for the users.

5.2 Limitations with the method

The research study has had its limitations, especially in user testing. This is mainly due to the prevailing situation of Covid-19 that limited the user interaction part. However, the fact that the user testing had to be held remotely also helped to ﬁnd new ways to test the concept with the end-users. This ended up creating a demonstration video of the interactive UGV system that now can be used to quickly investigate the concept of UGV with more soldiers in the Swedish Armed Forces and thereby generate a higher validity. Testing a prototype of a physical remote control without the presence of the object that is supposed to be controlled has been challenging. However, the demonstration video helped put the users in the mindset and generate ideas and thoughts of the UGV.

To strive for validity, the selection of test participants was aimed at soldiers in the military ﬁeld. The intended end-users were infantry soldiers in the Home Guard. However, due to the pandemic situation, it was harder to get the intended users, and the soldiers in the testing were instead within the Airbase security service. The positive part of the participants’ background was their familiarity with unmanned systems in the form of UAVs.

Moreover, the method of TAM does not have many published benchmarks even though it is a method with a high theoretical foundation. The use of TAM is even more limited within the HRI studies [27]. This makes it difﬁcult to compare the results of "perceived usefulness" with other related work. However, it could be used as a benchmark for future HRI studies or for future user testing with other military troops.

5.3 The work in a wider context

As automation is seen more frequently in various human activities such as health care, transportation and wars, more questions concerning the social issues with automation are arising [33]. Throughout this thesis work, several questions arose concerning the ethical and social aspects of UGVs in military settings. Even though the UGV in this thesis is not aimed to kill people, the capabilities of UGVs is expanding to a level where it has the potential to become

30 5.3. The work in a wider context fully autonomous in combat situations. This puts humans in the role of reflection. Who has the responsibility if the UGV kills another human? The person who designed the UGV or the person that initiated the war? Do emotionless robots fit on the battlefield? Does the human fit on the battlefield in the first place? Humans have not evolved to function in war conditions, but could UGVs be engineered to function well in them? The questions are not easy to answer and there is certainly no doubt that the subject of the robot’s role in society is a controversial topic. The ethical consideration of robots in the military has both its benefits and limitation [2].

31 6 Conclusion

The thesis’s purpose has been to explore how to design interactive systems for Unmanned Ground Vehicles (UGVs) for users in military contexts. The aim was to create a prototype of a controller interface that is accepted by the users and meets their user needs through a user-centered design approach.

The sum of the results indicates that the key to designing interactive systems for UGVs is understanding the context the UGV and the soldiers will work within. Further, the context deﬁnes the level of decision-making shared between the human operator respectively the UGV. The interaction with the soldiers in the Swedish Armed Forces shows that the ﬁeld of automation is relatively new; hence more research is needed in regards to trust autonomous vehicles. Early user involvement and keeping the user in the design loop are therefore fundamental aspects in order to build up the level of trust and consequently the user acceptance.

32 Bibliography

[1] Alain Abran, Adel Khelifi, Witold Suryn, and Ahmed Seffah. “Usability meanings and interpretations in ISO standards”. In: Software quality journal 11.4 (2003), pp. 325–338. [2] Ronald C Arkin. “Ethical robots in warfare”. In: IEEE Technology and Society Magazine 28.1 (2009), pp. 30–33. [3] Lina Linnea Aspen. “Guerilla testing of mobile medical applications in clinical settings Experiences from field testing in four Ugandan health institutions”. In: 2016. [4] Keshav Bimbraw. “Autonomous cars: Past, present and future a review of the develop- ments in the last century, the present scenario and the expected future of autonomous vehicle technology”. In: 2015 12th international conference on informatics in control, automation and robotics (ICINCO). Vol. 1. IEEE. 2015, pp. 191–198. [5] Adream Blair-Early and Mike Zender. “User interface design principles for interaction design”. In: Design Issues 24.3 (2008), pp. 85–107. [6] John W Creswell and Cheryl N Poth. Qualitative inquiry and research design: Choosing among five approaches. Sage publications, 2016. [7] Kerstin Dautenhahn. “Methodology & themes of human-robot interaction: A growing research field”. In: International Journal of Advanced Robotic Systems 4.1 (2007), p. 15. [8] Fred D Davis. “A technology acceptance model for empirically testing new end-user information systems: Theory and results”. PhD thesis. Massachusetts Institute of Tech- nology, 1985. [9] Sarah-Louise Donovan and Tom Triggs. Investigating the effects of display design on unmanned underwater vehicle pilot performance. Tech. rep. DEFENSE SCIENCE and TECH- NOLOGY ORGANIZATION VICTORIA (AUSTRALIA) MARITIME . . ., 2006. [10] Terrence Fong and Charles Thorpe. “Vehicle teleoperation interfaces”. In: Autonomous robots 11.1 (2001), pp. 9–18. [11] Michael A Goodrich and Alan C Schultz. “Human–Robot Interaction: A Survey”. In: Human–Computer Interaction 1.3 (2007), pp. 203–275. [12] Thomas M Granda, Mark Kirkpatrick, Tracye D Julien, and Larry A Peterson. “The evolutionary role of humans in the human-robot system”. In: Proceedings of the Human Factors Society Annual Meeting. Vol. 34. 9. SAGE Publications Sage CA: Los Angeles, CA. 1990, pp. 664–668.

33 Bibliography

[13] Richard Herriott. “What kind of research is research through design”. In: Design Revolu- tions; Manchester School of Art, Manchester Metropolitan University: Manchester, UK (2019). [14] Monique WM Jaspers, Thiemo Steen, Cor Van Den Bos, and Maud Geenen. “The think aloud method: a guide to user interface design”. In: International journal of medical informatics 73.11-12 (2004), pp. 781–795. [15] Urška Lah, James R Lewis, and Boštjan Šumak. “Perceived usability and the modified technology acceptance model”. In: International Journal of Human–Computer Interaction 36.13 (2020), pp. 1216–1230. [16] Peiqun Lin, Jiahui Liu, Peter J Jin, and Bin Ran. “Autonomous vehicle-intersection coor- dination method in a connected vehicle environment”. In: IEEE Intelligent Transportation Systems Magazine 9.4 (2017), pp. 37–47. [17] Jonas Lundberg, Mattias Arvola, Carl Westin, Stefan Holmlid, Mathias Nordvall, and Billy Josefsson. “Cognitive work analysis in the conceptual design of first-of-a-kind systems–designing urban air traffic management”. In: Behaviour & Information Technol- ogy 37.9 (2018), pp. 904–925. [18] Ji-Ye Mao, Karel Vredenburg, Paul W. Smith, and Tom Carey. “The State of User- Centered Design Practice”. In: Commun. ACM 48.3 (2005), pp. 105–109. ISSN: 0001-0782. URL: https://doi.org/10.1145/1047671.1047677. [19] Carlos Martínez and Felipe Jiménez. “Implementation of a Potential Field-Based Decision-Making Algorithm on Autonomous Vehicles for Driving in Complex Envi- ronments”. In: Sensors 19.15 (2019), p. 3318. [20] Paul Milgram, Shi Yin, and Julius J Grodski. “An augmented reality based teleoperation interface for unstructured environments”. In: ANS 7th Meeting on Robotics and Remote Systems, Augusta. Citeseer. 1997, pp. 101–123. [21] Robin Murphy and J Sprouse. “Strategies for searching an area with semi-autonomous mobile robots”. In: Proceedings of Robotics for Challenging Environments. Citeseer. 1996, pp. 15–21. [22] Jose E Naranjo, Felipe Jimenez, Manuel Anguita, and Jose Luis Rivera. “Automation kit for dual-mode military unmanned ground vehicle for surveillance missions”. In: IEEE Intelligent Transportation Systems Magazine 12.4 (2020). [23] Jakob Nielsen. “Estimating the number of subjects needed for a thinking aloud test”. In: International journal of human-computer studies 41.3 (1994), pp. 385–397. [24] Jakob Nielsen. Ten usability heuristics. https://www.nngroup.com/articles/ ten-usability-heuristics/. Online; accessed 11 February 2021. 2005. [25] Janni Nielsen, Torkil Clemmensen, and Carsten Yssing. “Getting access to what goes on in people’s heads? Reflections on the think-aloud technique”. In: Proceedings of the second Nordic conference on Human-computer interaction. 2002, pp. 101–110. [26] Linda Onnasch and Eileen Roesler. “A Taxonomy to Structure and Analyze Human– Robot Interaction”. In: International Journal of Social Robotics (2020), pp. 1–17. [27] Sonja K Ötting, Lisa Masjutin, Jochen J Steil, and Günter W Maier. “Let’s Work To- gether: A Meta-Analysis on Robot Design Features That Enable Successful Human– Robot Interaction at Work”. In: Human Factors (2020), pp. 1–24. [28] Kevin N Otto and Kristin L Wood. “Product evolution: a reverse engineering and re- design methodology”. In: Research in engineering design 10.4 (1998), pp. 226–243. [29] Raja Parasuraman, Thomas B Sheridan, and Christopher D Wickens. “A model for types and levels of human interaction with automation”. In: IEEE Transactions on systems, man, and cybernetics-Part A: Systems and Humans 30.3 (2000), pp. 286–297.

34 Bibliography

[30] Jouni Rantakokko, Fredrik Näsström, Jonas Nygårds, Kristoffer Bengtsson, and Rogier Woltjer. Tekniköversikt autonoma och obemannade system Del 2: Markstriden. Tech. rep. FOI, 2019. [31] Dale Richards and Alex Stedmon. “Designing for human–agent collectives: display considerations”. In: Cognition, Technology & Work 19.2-3 (2017), pp. 251–261. [32] Jennifer Rowley. “Conducting research interviews”. In: Management Research Review 35.3/4 (2012), pp. 260–271. [33] Lambèr Royakkers and Rinie van Est. “A literature review on new robotics: automation from love to war”. In: International journal of social robotics 7.5 (2015), pp. 549–570. [34] Per Runeson and Martin Höst. “Guidelines for conducting and reporting case study research in software engineering”. In: Empirical software engineering 14.2 (2009), pp. 131– 164. [35] Jean Scholtz. “Theory and evaluation of human robot interactions”. In: 36th Annual Hawaii International Conference on System Sciences, 2003. Proceedings of the. IEEE. 2003, 10–pp. [36] International Organization for Standardization. “ISO 9241–210: 2019 (en) Ergonomics of human-system interaction—Part 210: Human-centred design for interactive systems”. In: (2019). [37] Kuo-Shih Tseng and Bérénice Mettler. “Analysis and Augmentation of Human Perfor- mance on Telerobotic Search Problems”. In: IEEE Access 8 (2020), pp. 56590–56606. [38] Annemiek Van Boeijen, Jaap Daalhuizen, Roos van der Schoor, and Jelle Zijlstra. Delft Design Guide: Design Strategies and Methods. BIS Publishers, 2014. [39] Robert A Virzi. “What can you learn from a low-ﬁdelity prototype?” In: Proceedings of the Human Factors Society Annual Meeting. Vol. 33. 4. SAGE Publications Sage CA: Los Angeles, CA. 1989, pp. 224–228. [40] Amber M Walker, David P Miller, and Chen Ling. “User-centered design of an attitude- aware controller for ground reconnaissance robots”. In: Journal of Human-Robot Interac- tion 4.1 (2015), pp. 30–59. [41] Thomas Weiskircher, Qian Wang, and Beshah Ayalew. “Predictive guidance and control framework for (semi-) autonomous vehicles in public trafﬁc”. In: IEEE Transactions on control systems technology 25.6 (2017), pp. 2034–2046. [42] Liu Xin and Dai Bin. “The latest status and development trends of military unmanned ground vehicles”. In: 2013 Chinese automation congress. IEEE. 2013, pp. 533–537. [43] Holly A Yanco and Jill Drury. “Classifying human-robot interaction: an updated taxonomy”. In: 2004 IEEE International Conference on Systems, Man and Cybernetics (IEEE Cat. No. 04CH37583). Vol. 3. IEEE. 2004, pp. 2841–2846. [44] Holly A Yanco and Jill L Drury. “A taxonomy for human-robot interaction”. In: Pro- ceedings of the AAAI Fall Symposium on Human-Robot Interaction. sn. 2002, pp. 111–119. [45] Julius Ziegler, Philipp Bender, Markus Schreiber, Henning Lategahn, Tobias Strauss, Christoph Stiller, Thao Dang, Uwe Franke, Nils Appenrodt, Christoph G Keller, et al. “Making bertha drive—an autonomous journey on a historic route”. In: IEEE Intelligent transportation systems magazine 6.2 (2014), pp. 8–20.