Validating Safety in Human–Robot Collaboration: Standards and New Perspectives

robotics

Article Validating Safety in Human–Robot Collaboration: Standards and New Perspectives

Marcello Valori 1,* , Adriano Scibilia 1, Irene Fassi 1 , José Saenz 2 , Roland Behrens 2, Sebastian Herbster 2, Catherine Bidard 3 , Eric Lucet 3 , Alice Magisson 4 , Leendert Schaake 5, Jule Bessler 5,6 , Gerdienke B. Prange-Lasonder 5,7 , Morten Kühnrich 8, Aske B. Lassen 8 and Kurt Nielsen 8

1 Institute of Intelligent Industrial Technologies and Systems for Advanced Manufacturing, National Research Council of Italy, 20133 Milan, Italy; [email protected] (A.S.); [email protected] (I.F.) 2 Fraunhofer IFF, 39106 Magdeburg, Germany; [email protected] (J.S.); [email protected] (R.B.); [email protected] (S.H.) 3 Université Paris-Saclay, CEA List, F-91120 Palaiseau, France; [email protected] (C.B.); [email protected] (E.L.) 4 CEA, CEA Tech Grand Est, F-57075 Metz, France; [email protected] 5 Roessingh Research and Development, 7522 AH Enschede, The Netherlands; [email protected] (L.S.); [email protected] (J.B.); [email protected] (G.B.P.-L.) 6 Department of Biomedical Signals and Systems, University of Twente, 7522 NB Enschede, The Netherlands 7 Department of Biomechanical Engineering, University of Twente, 7522 NB Enschede, The Netherlands 8 Department of Robot Technology, Danish Technological Institute, 5230 Odense, Denmark; [email protected] (M.K.); [email protected] (A.B.L.); [email protected] (K.N.) * Correspondence: [email protected]; Tel.: +39-329-312-0837 Abstract: Human–robot collaboration is currently one of the frontiers of industrial robot implemen- Citation: Valori, M.; Scibilia, A.; tation. In parallel, the use of robots and robotic devices is increasing in several fields, substituting Fassi, I.; Saenz, J.; Behrens, R.; humans in “4D”—dull, dirty, dangerous, and delicate—tasks, and such a trend is boosted by the Herbster, S.; Bidard, C.; Lucet, E.; recent need for social distancing. New challenges in safety assessment and verification arise, due to Magisson, A.; Schaake, L.; et al. both the closer and closer human–robot interaction, common for the different application domains, Validating Safety in Human–Robot and the broadening of user audience, which is now very diverse. The present paper discusses a Collaboration: Standards and New Perspectives. Robotics 2021, 10, 65. cross-domain approach towards the definition of step-by-step validation procedures for collaborative https://doi.org/10.3390/ robotic applications. To outline the context, the standardization framework is analyzed, especially robotics10020065 from the perspective of safety testing and assessment. Afterwards, some testing procedures based on safety skills, developed within the framework of the European project COVR, are discussed and Academic Editor: Aman Behal exemplary presented.

Received: 15 February 2021 Keywords: robotic safety; collaborative robots; robot standards Accepted: 28 April 2021 Published: 29 April 2021

Publisher’s Note: MDPI stays neutral 1. Introduction with regard to jurisdictional claims in The concept of human–robot collaboration (HRC), based on the synergic work of published maps and institutional affil- robots and humans, questions the traditional paradigm of physical barriers separating iations. machines and workers, lowering fences and closing distances. This paradigm shift was enabled by two main technological factors: the integration of safety-related features in the robot architectures and control systems and the use of multi-modal interfaces for more intuitive, aware and safer human–robot interaction (HRI) [1,2]. Moreover, thanks to the Copyright: © 2021 by the authors. advances in artificial intelligence, robots can adapt their tasks and behaviors, becoming suit- Licensee MDPI, Basel, Switzerland. able to act in unstructured scenarios and interacting with unskilled and undertrained users. This article is an open access article The statistics show that, in the last years, several robot fields are increasing their distributed under the terms and markets, such as: robots for domestic tasks, entertainment robots, logistic robots, robots for conditions of the Creative Commons public environments, defense applications, inspection and maintenance robots, professional Attribution (CC BY) license (https:// creativecommons.org/licenses/by/ cleaning, field robots, powered human exoskeletons, medical robots, construction and 4.0/). demolition [3]. Focusing just on the medical and healthcare domain, a non-comprehensive

Robotics 2021, 10, 65. https://doi.org/10.3390/robotics10020065 https://www.mdpi.com/journal/robotics Robotics 2021, 10, 65 2 of 20

list includes: surgical robotics, boosting the transition from open surgery to laparoscopic and other minimally invasive surgical procedures [4]; bionic prostheses, enabling wearers to autonomously carry out their daily activities; the category of “RACA” robots, including a variety of devices, either wearable or static, to perform rehabilitation, assessment, compensation and alleviation (RACA); and caregiver robots, which are expected to play a fundamental role in aging societies [5]. In the agri-food sector, autonomous mobile robots provided by navigation and perception technologies represent a breakthrough, with the potential of performing mass phenotyping, crop management, and selective harvesting [6]. The importance of robots was underlined by the recent pandemic, as it is recognized that robot adoption can be extremely useful to support social distancing. Accordingly, robotic-assisted surgery has become a safe alternative, protecting the patient, the operating team, and saving resources for people with COVID-19 [7]. Due to their safe nature and possibility of “learning from demonstration”, collaborative robots can be implemented as sterilizable physical mediators between patients and clinicians, for example, for mass swabbing and physiotherapy treatments; at the same time, autonomous robots can perform UV sterilization of the environments and optimize internal logistics of health fa- cilities [8]. Furthermore, from the perspective of social distancing, robotics can enable pandemic-sustainable travel, tourism and hospitality, performing environment sterilization, measuring body temperatures, welcoming and entertaining [9]. The aforementioned examples are characterized by different levels of HRI, spanning from teleoperation to workspace sharing and synergistic co-control. Taking a step back, one can observe that boundaries between industrial and service robotics are becoming blurred [10] and HRC is assuming a broader role, becoming applicable to a wide variety of robot applications in which the close interaction between human and robots is envisaged. However, the implementation of robots sharing space with humans, either to increase productivity rates, relieve us from heavy and repetitive tasks or mitigate certain hazard sources, always has to consider the mechanical hazards associated with HRC. Furthermore, the implementation and observance of safety is closely related to overall task performance, as safer interaction enables lower barriers and can help robots to reach their full potential. In such a rapidly evolving landscape, in which industrial and service robotics become increasingly closer, identifying, interpreting and fulﬁlling the applicable standards for robot safety is critical. Based on the experience of the authors with robot end-users, manufacturers and integrators, the promotion of cobot technologies in different domains is slowed down by the path towards ensuring safety. This is due to both the limited knowledge about relevant standards and the absence of clear procedures to prove the compliance in these standards [11,12]. To address this need, a novel cross-domain validation approach is proposed in this paper, based on testing procedures made available through an online toolkit.

1.1. Safety in HRC Increasing safety in collaborative robot operations is a goal pursued at different levels. Collaborative robots are intrinsically safer, but the implementation of external sensing strategies further improves safety, and safety-related systems can be used to enable HRC with traditional robots [13]. The research community is addressing several issues related to sensing, robot hardware and safeguarding measures; the main drivers concern increasing the perception of both robots and humans, lowering the potential damage due to mechanical features, control strategies or external protections, providing user- friendly programming strategies [1]. Aiming at increasing collaboration synergy and safety, human–robot interfacing can be enhanced by implementing different technologies, such as robot integrated vision, voice recognition, head-mounted displays for augmented reality, haptic-based devices and even the detection of bio-signals [2]. Safety-related aspects have a relevant impact in the design of HRC application, as hazard identiﬁcation, risk assessment and the identiﬁcation of risk mitigation strategies become even more crucial. In [14], a typical process is described, demonstrating that Robotics 2021, 10, 65 3 of 20

an appropriate modeling of safety aspects can reduce both the design time and the final workspace required by the layout. In a recent review [15], it is shown how the research outputs concerning safety and ergonomics are increasing, in the last years, with the rise of collaborative robotics, identifying four main clusters of interest: contact avoidance, contact detection and mitigation, physical ergonomics, cognitive and organizational ergonomics. Even if the two former categories are more widely addressed, the authors claim that ergonomic issues will play a major role when the advances in contact avoidance strategies will push the limits of HRI, leading to a complementarity of these aspects towards safe and efficient HRC. Concerning the evaluation of safety in HRC, some approaches use objective measures to plan and evaluate the performance of applications featuring “speed and separation monitoring” collaborative scenarios [16,17]. Unfortunately, they also demonstrate the difficulties in identifying the point in time during a robot’s trajectory where a specific algorithm is the least safe, requiring either a simulation or a test with the completed system. Even if these approaches can provide a certain measure of the HRC safety, they are not aimed at the validation of robotic applications with reference to directives and standards. Furthermore, they are targeting robot experts dealing with HRC scenarios in the industrial domain, while the ambition of the present work is to meet the needs of a diverse user audience, operating in different domains with the potential for implementing cobots in a wide range of tasks.

1.2. Paper Contribution The risks related to an autonomous robot acting in a public environment are similar to the ones typical of a driverless mobile robot in industry. In the same way, similar risks characterize a manipulator for collaborative assembly and one implemented for delicate health diagnostic procedures. This leads the consideration that the specific safety-related testing procedures could rely on a common base of knowledge and experience. It is also worth observing that, even in the fields characterized by higher levels of expertise, close HRC brings new challenges for safety verification and validation. Accordingly, the analogue hazards make the guidance for clear, step-by-step testing procedures a common need of both experienced and new users. The aim of this paper is to outline emerging necessities and possible trends concerning safety testing procedures for cobot applications, based on an overview of the landscape of relevant standards for the validation of safety, and to describe a new cross-domain approach based on safety skills and testing protocols. In this context, we refer to “safety skills” as the capability of a robotic system to reduce a specific risk and to “protocols” as step-by-step instructions for executing validation measurements. It is a belief of the authors, indeed, that clearer safety assessment processes allow cobots to be used with more confidence in more situations; this can be a valuable boost to increase the variety of cobots on the market and the variety of services cobots can offer to the general population. As a remark, the proposed protocols are not intended to substitute standards; they represent instead a tool to guide and facilitate the users in the identification of relevant standards and their fulfillment. The paper is organized as follows. In Section2, the standard framework is addressed; building on the overview of all the relevant standards, some recent publications dealing with detailed test procedures, are described and gaps with user needs are highlighted. Section3 deals with the proposed cross-domain approach, which is based on the definition of safety skills and the development of testing protocols, while in Section4 some exemplary cases of protocol application are reported.

2. Robot Safety: The Regulatory Framework 2.1. Overview The main regulation in the European community dealing with robot safety is the Machinery Directive 2006/42/EC [18]. It is translated in all the national languages and Robotics 2021, 10, 65 4 of 20

transposed into laws by each member country. A robot falls in the scope of the Machinery Directive, as it is composed of “linked parts or components, at least one of which moves” and being actuated by a drive system. A programmable robot supplied by a robot manufacturer is regarded by the Machinery Directive to be a “partly completed machinery”. This means that the robot itself is not CE-marked for Machinery Directive, but that all information needed by integrators to ensure safety is provided by the robot manufacturer. To be considered as a “completed” machinery, it must be designed or integrated for a specific application. In industrial robotics, the concept of machinery applies to robotics applications or robotic cells and whereby the integrator is considered to be the manufacturer. In other fields, such as robots for consumers, assistive robotics or medical robotics, the intended use must be clearly defined. The Medical Device Regulation EU 2017/745 [19] applies to rehabilitation and, more generally, to healthcare-related activities. It replaced the Medical Device Directive, which focused mainly on the design of the system, providing more safety considerations over the entire device lifetime [20]. The Medical Device certification is strongly based on an “intended use” and valid only for this “intended use” for which clinical evidence must be provided. So, in accordance with the Machinery Directive, a generic robot without his specific application cannot be certified. It is worth noting that, for a robotic medical device, the Machinery Directive must also be observed. Other directives may apply depending on the field of use, such as the General Product Safety Directive [21], when a robotic device is made available on the consumer market, or a type-approval regulation when a robot becomes a vehicle (this category is outside the scope of this paper). Finally, there are more general relevant directives, such as the Low Voltage Directive [22], the Electromagnetic Compatibility Directive [23], and the Radio Equipment Directive [24]. Standards are technical reference documents, representing the consensus on the state of the art and compliance to them is not mandatory. Technical committees (TCs) are in charge of developing standards and the main committee within ISO for industrial and service robotics (excluding toys and military applications) is the ISO TC 299 “Robotics”. Robotics ISO standards from this group mainly address vocabulary, performance and safety. Concerning safety, they describe solutions gathering a certain level of consensus within the specific community. Indeed, robot manufacturers typically take part in standardization committees, along with integrators, end-users and stakeholders representing public health and health insurance. “Harmonized” standards are recognized by a European standard organization as fully compliant with the relevant directive or regulation; thus, their application provides the so-called “presumption of conformity”. The most relevant standards for robot safety from the perspective of HRC are listed in Table1. Safety standards are categorized into: type A, providing the basic design principles and valid for all machines; type B, which are generic standards concerning either specific aspects of safety or devices for safeguarding; and type C, covering particular classes of machines (i.e., robots) and having precedence over type A and B. Accordingly, for robots belonging to the categories of industrial robots or personal care robots, type C standards are the main reference. The list reported in Table1 is not comprehensive of all the robotics- relevant type B standards, as it is limited to standards dealing with HRC, either explicitly or not. An up-to-date list of all the robotic standards, which can be filtered by application domain and device type, can be found in COVR Toolkit [25]. The Machinery Directive sets out the “Essential health and safety requirements relating to the design and construction of machinery”. Fulfilling these requirements is based on an “iterative process of risk assessment and risk reduction” for which one can rely on the standard ISO 12100 [26] (type A). It should be noted that the medical device safety is addressed by a dedicated standard for risk management, the ISO 14971 [27]. The Machinery Directive promotes the integration of safety in all design stages, giving priority to inherent safety, followed by protective measures, and, lastly, by organizational measures, such as user training and personal protective equipment. Robotics 2021, 10, 65 5 of 20

The ISO 10218 is a type C standard for industrial robotics, divided into two parts: the first [28] dealing with robots considered as “partly completed machinery” and the second [29] addressing integrated applications, which are the machineries to be considered in compliance with the Machinery Directive. Even though harmful HRI had already been addressed in the first versions, intended physical interaction has been introduced in 2009 in both 10218-1 and -2, by describing “hand-guiding” (HG) and “power and force limiting” (PFL) operating modes. The possibility to avoid safeguards by the use of distance sensing was introduced with “safety-rated monitored stop” (SRMS) and “speed and separation monitoring” (SSM). The safety requirements for these collaborative modalities are currently under development for the upcoming revision of ISO 10218 (to be published in 2022), as will be addressed in the following paragraph. Technical specifications (TS) and technical reports (TR) are different ISO deliverables. TS address work still under development but with a chance to be included in an international standard, while TR contain other kinds of information, for example, the perceived state of the art of a specific topic. ISO/TS 15066 [30] represented a milestone among HRC standardization. Besides including information regarding collaborative robot system design, hazard identification, risk assessment and the requirements for the applications, it provides a more detailed description of the collaborative modes SRSM, SRSS, PFL, HG and limit values for quasi-static and transient contact forces are illustrated.

Table 1. List of the most relevant standards dealing with safety requirements in HRC.

ID Standard HRC Relevance Type Harmonized Safety of machinery—general principles for ISO 12100:2010 [26] Risk assessment A Yes design—risk assessment and risk reduction Safety of machinery—positioning of safeguards with ISO 13855:2010 [31] respect to the approach speeds of parts of the Suitable for SSM B Yes human body Agricultural machinery and tractors—safety of highly Safety requirements, ISO 18497:2018 [32] B Yes automated machinery veriﬁcation Safety of machinery—two-hand control ISO 13851:2019 [33] Suitable for HG B Yes devices—principles for design and selection Robots and robotic devices—safety requirements for HRC operation ISO 10218-1:2011 [28] C Yes industrial robots—part 1: robots requirements Robots and robotic devices—safety requirements for HRC operation ISO 10218-2:2011 [29] C Yes industrial robots—part 2: robot systems and integration requirements Robots and robotic devices—safety requirements for Guidance for safety ISO 13482:2014 [34] C Yes personal care robots (lifecycle) Industrial trucks—safety requirements and ISO 3691-4: 2020 [35] veriﬁcation—part 4: driverless industrial trucks and Human detection C Yes their systems Medical electrical equipment—part 2-78: particular requirements for basic safety and essential performance Safe HRI with RACA IEC 80601-2-78: 2020 1 [36] n.a. No of medical robots for rehabilitation, assessment, robots compensation or alleviation ISO/TS 15066:2016 [30] Robots and robotic devices—collaborative robots Focused on HRC TS n.a. Application of ISO 13482—part 1: safety-related Testing procedures for ISO ISO/TR 23482-1:2020 [37] TR n.a. test methods 13482 Robotics—safety design for industrial robot Requirements for ISO/TR 20218-1:2018 [38] TR n.a systems—part 1: end-effectors end-effectors Robotics—safety requirements for industrial Req. for load/unload ISO/TR 20218-2:2017 [39] TR n.a robots—part 2: manual load/unload stations stations 1 IEC 80601-2-78 belongs to a series of particular standards that can modify, replace or delete requirements of the general standard IEC 60601-1:2020 (Medical electrical equipment—part 1: general requirements for basic safety and essential performance). Robotics 2021, 10, 65 6 of 20

2.2. A New Trend in Standardization? If in domains characterized by well-established validation practices and reference standards, such as the industrial field, the presence of collaborative robots brings new challenges in the assessment of human–robot interaction, the increasing spread of robots in environments shared with humans generates new application scenarios, involving new types of users. An inexpert end-user can harbor the false perception that the mere implementation of a collaborative robot is sufficient to ensure safety, due to its low weight, ease of use and even friendly appearance. As a result, these two parallel trends—the spread of robots among humans and the increase of end-users new to the robot world— increasingly pose the need of clear procedures for testing robots sharing their workspaces with humans. There are recent signals that the standardization world is moving to meet this need and provide the required procedures. In Table2, the tests currently available in these standards are listed. The new version of the ISO 10218-2, currently under development, is expected to give greater attention to collaborative applications. As a “draft international standard” (DIS), the ISO/DIS 10218-2 [40] was published in late 2020 and is currently under evaluation. Col- laborative applications are identified as characterized by one or more of three technologies: “hand-guided controls” (HGC), “speed and separation monitoring” (SSM), and “power and force limiting” (PFL). Specific risk assessment is envisaged for potential human–robot contact conditions, as well as related passive safety design measures and active risk reduction measures. To the latter category belongs the safety function “monitored-standstill”, corresponding to the capability of performing the SRMS collaborative mode, no longer addressed. Furthermore, new annexes are specifically dedicated to HRC. Among these, Annex L describes how to calculate the protective separation distance in SSM and Annex M reports the limits for quasi-static and transient contact. These annexes incorporate the information previously provided by the ISO/TS 15066 [30]. The most relevant from the perspective of safety verification and validation testing is Annex N, dedicated to pressure and force measurements in PFL robotic applications.

Table 2. HRI safety-related test procedures reported in the cited standards.

Standard Test Section Description The required measuring device for the measurements is described, which has to incorporate a spring and a soft pad, Power- and force-limited whose hardness and stiffness, respectively, must comply ISO/DIS 10218-2: robot applications—pressure Annex N with specific values depending on the body part involved in 2020 [40] and force measurements the potential contact. The setup, measurement and data analysis procedures are then described to test both quasi-static (clamping) and transient contact force events. The truck travels towards a cylindrical test piece with defined dimensions, placed in different positions and poses along the path. Test success corresponds to a full truck stop Tests for detection of persons § 5.2 before contact (in case of contactless detection means) or to ISO 3691-4: 2020 a contact with a limited interaction force (in case of [35] bumper-based detection), observed over three repetitions. The worst-case conditions must be replicated (loaded, unloaded, lift height, slope, turn, forward direction, Stability tests § 5.3 backward direction, floor/ground slope) and be stable, or otherwise, stability can be proved by calculations. Robotics 2021, 10, 65 7 of 20

Table 2. Cont.

Standard Test Section Description Physical hazard It includes voltage at user-accessible parts, acoustic noise § 6 characteristics and surface temperature. This category includes the tests of injury parameters in collision, performed using a dummy with features as per the US code of Federal Regulations and the tests of force Physical hazard control for intended and unintended contact with a robot, characteristics § 7 performed with a contact piece simulating the human body (for mobile robots) part and a force transducer and a pressure sensor as sensing equipment. In both cases, the acquired impact force values are compared with the ones listed in the ISO/TS 15066. The physical stress or strain to the user is tested by means of a setup composed by a force sensor, a dummy, a cuff with Physical hazard force/pressure sensing capability and a manipulator characteristics (for § 8 moving the cuff. The combination of tangential traction restraint-type physical ass. forces and continuous repeated rubbing contact acquired is robots) compared with reference curves referred to the generation of blisters in human skin. The robot is placed on a test plane with a slope, if necessary Static stability § 11 for the type of vehicle transporting a dummy, and the characteristics stability is observed. The robot is placed on a test plane with a slope, if necessary Dynamic stability with a dummy simulating shifting loads. The worst-case characteristics with respect to § 12 directions are considered, and the robot moves, maximizing moving parts (mobile robot) the generated dynamic forces and, if appropriate, moving loads or dynamic passengers are also simulated. ISO/TR 23482-1: Replicating the same working conditions, with the use of 2020 [37] dummies if necessary, stability tests are performed on a flat Dynamic stability surface (breaking and acceleration tests), on an inclined characteristics with respect to § 13 surface (maximum speed, acceleration and braking on travel (for mobile robot) downward slope, upward slope acceleration, downward slope full turn, crossing, pivot turn) and in relation to potential steps and gaps in the environment. Different tests consider the electro-sensitive protective Safety-related control equipment (ESPE), performed with human-like objects to be § 14 functions (universal) detected, the operation in slippery environments, and electro-magnetic immunity. These tests include the assessment of: - the distance of protective stop, performed towards a Response to safety-related wall, a cylindrical post or a dummy, with different obstacles on the ground § 15 travel directions, facing angle and position of test (mobile robot) piece, and measuring the distance after stop; - the stopping distance before concave terrain moving towards it (similar procedure). With a pre-defined environment map and programmed Safety-related localization and path, an obstacle is positioned in different positions along § 16 navigation the robot path. Jerky robot movements, unexpected stops or other potentially hazardous movements are detected. These tests are related to the autonomous action of Reliability of autonomous identifying an object, interpreting user commands, choosing decisions and actions § 17 a strategy to minimize the collision risk, etc. The document (universal) describes only the test for object identification. Robotics 2021, 10, 65 8 of 20

Table 2. Cont.

Standard Test Section Description The aim of the test is to determine the wheeled robot rated speed for travel-related tasks. In a cycle consisting in Rated speed § 5 acceleration, constant speed, and deceleration, two sensors detect the passage of the robot at constant speed and the ISO 18646-1:2016 rated speed is obtained by calculation. [41] The robot moves in a straight line up to the rated speed; afterwards a stop command is initiated. Once the robot is Stopping characteristics § 6 fully stopped, stopping distance and times are obtained by the acquisitions. This is a static test in which six different obstacles are positioned at speciﬁc distances from the robot, Obstacle detection § 6 corresponding to the maximum and minimum acquisition ranges declared by the manufacturer, with different orientations with respect to the line of sight.

ISO 18646-2:2019 This test determines the ability of the robot of reaching a [42] goal position avoiding moving obstacles along its path. The test is performed by commanding moving obstacles to move along a path, normally causing collision with the Obstacle avoidance § 7 robot, which is, in turn, commanded to autonomously move along a path. The test is repeated with different obstacle trajectories. The test is successful if the robot reaches the goal position in all the trails, and a time “delay factor” is calculated.

The recently published ISO 3691-4 [35] provides some testing procedures. It applies to driverless industrial trucks, which are powered trucks, designed to work automatically. Examples of trucks falling in this category are all-automated guided vehicles, autonomous mobile robots, bots, automated guided carts, tunnel tuggers, under carts, etc., whereas trucks guided by rails or other mechanical means or controlled remotely are excluded. Therefore, industrial mobile platforms and several types of mobile service robots are included. From the perspective of the safety assessment, the relevant section is the one related to the “veriﬁcation of safety requirements and/or protective measures”, in which testing procedures are speciﬁed, including tests for the detection of persons and stability tests. As a technical report, the ISO/TR 23482-1 [37] “describes test methods which are guidelines to verify compliance to the requirements of 13482”, which, in turn, describes “safety requirements for personal care robots”. In the TR, test conditions are provided as general rules and practices for the performance of all the tests. Afterwards, a list of several tests is provided, organized depending on the related hazard, along with a detailed description of the test principle, apparatus, procedure, and pass/fail criteria. Even if the ISO 18646 standard series deals principally with robot performance criteria and assessment, this aspect is closely related to safety, especially in the case of performance related to safety-related functionalities. Accordingly, some performance testing procedures are reported in Table2, extracted from the ISO 18646-1 [41] and the ISO 18646-2 [42].

2.3. Analysis of the Gaps Whereas, 20 years ago, industrial robots were almost solely used for high volume manufacturing, today the same robot manipulator can be used for manufacturing, logistics, rehabilitation, or even agricultural applications. This can lead to uncertainty with respect to safety and applicable standards, especially due to the domain-specific organization of international safety standards, through two specific ways. On the one hand, in situations where innovations occur faster than standardization, there are not always relevant domain- specific standards for the application in question. On the other hand, combinations of device types can lead to conflicting issues from different safety standards. An example is a Robotics 2021, 10, 65 9 of 20

mobile platform equipped with a robot manipulator. The forces specified during contact are different when considering an industrial manipulator (in the ISO/TS 15066 [30]) or an autonomous truck (ISO 3691-4 [35]). Further challenges arise when considering what separation distances to apply, as the ISO/TS15066 [30] also specifies that the approach speed of humans needs to be taken into account, whereas the ISO 3691-4 [35] does not. While adherence to standards is not legally binding, they do represent the state of the art and can be extremely helpful for considering the safety of collaborative robotics applications. A streamlined approach that offers robotics stakeholders the means to con- ceptually talk about the safety of their system, regardless of the specific domain, would be helpful here. Furthermore, it would be extremely helpful to the robotics community if the same approach were to extend to methods for validating the implemented risk reduction measures. This approach should, however, respect the fact that there are a wide variety of technical means available for implementing the safety.

3. Safety Skills and Testing Protocols in a Cross-Domain Perspective In the consolidated practice, boundaries between different robotic domains are well- defined and recognized. One typical example is that HRI with robotic medical devices can also generate physical benefits, balancing the risks associated to their operation, while this is not applicable for other types of robots. However, this approach can represent a limit when similar hazards have to be addressed in different domains; in these cases, indeed, cross-domain fertilization can provide an extra gear to define the best practices for the verification of safety requirements. This section proposes a safety verification approach based on the definition of cross-domain safety skills and the development of testing protocols, drafted on the basis of practical needs and updated by expert consultations and by the examination and addressment of real application cases. In Section 3.1, the safety skill concept and the skill-based approach is described, whereas the testing protocol structure and the list of the protocols are addressed in Section 3.2.

3.1. The Skill-Based Approach When considering the safety of collaborative robotic systems, it is essential to start with a risk assessment, whereby hazards specific to a concrete application are identified and risk reduction measures (RRM) are chosen. The operation methods for collaborative robotics applications, such as the SRMS, SSM, PFL, HG, as currently defined by the ISO 10218, can be implemented through a variety of technical means and offer the planner a level of abstraction to discuss and consider the safety of the application. These definitions may become challenging when other device types and domains rather than manufacturing are considered, such as industrial robotics for rehabilitation or mobile robots for agriculture. To overcome this challenge, the concept of safety skills has been proposed in [43]. Safety skills are defined as an abstract representation of the ability of the robot system to reduce some risk, i.e., to deploy suitable RRM. The implementation of the protective measure can be executed in a number of ways. A safety skill therefore indicates what kind of protection is desired and is independent of the execution of how this protection is delivered. A further characteristic of safety skills is that they can be validated for specific applications at a system level. A two-pronged approach towards the definition of cross-domain safety skills was applied. In addition to an identification of existing operating modes from available standards, an analysis of possible hazards and risk reduction methods for a large variety of applications featuring a combination of different robotic device types for six domains (manufacturing, rehabilitation, agriculture, civil, logistics, and consumer/home) was executed. While the latter analysis was quite large, it cannot be considered comprehensive, as robotics are increasingly being used for new applications in an even wider variety of domains than those analyzed. Nevertheless, the concept of cross-domain safety skills is explicitly conceived to deal with the novelty of future robotics applications and provide guidance Robotics 2021,, 10,, xx FORFOR PEERPEER REVIEWREVIEW 9 of 19

Robotics 2021, 10, x FOR PEER REVIEW 9 of 19

Robotics 2021, 10, x FOR PEER REVIEW 9 of 19 Robotics 2021, 10, x FOR PEER REVIEW 9 of 19 A two-pronged approach towards the definition of cross-domain safety skills was applied.A two-pronged In addition to approach an identification towards of the existing definiti operatingon of cross-domain modes from safety available skills stand- was ards,applied. Aan two-pronged Inanalysis addition of topossible approach an identification hazards towards and of the existingrisk definiti reduction operatingon of methodscross-domain modes forfrom a safetylargeavailable varietyskills stand- was of A two-pronged approach towards the definition of cross-domain safety skills was applicationsards,applied. an analysisIn addition featuring of possibleto aan combination identification hazards andof of different existingrisk reduction operatingrobotic methodsdevice modes types for from a for largeavailable six varietydomains stand- of applied. In addition to an identification of existing operating modes from available stand- (manufacturing,(manufacturing,applicationsards, an analysis featuring rehabilitation,rehabilitation, of possible a combination hazards agriculture,agriculture, ofand different civil,riskcivil, reduction logistics,logistics, robotic and andmethodsdevice consumer/home)consumer/home) types for a for large six variety waswasdomains exe-exe- of ards, an analysis of possible hazards and risk reduction methods for a large variety of Robotics 2021, 10, 65 cuted.(manufacturing,applications While thefeaturing rehabilitation,latter analysisa combination agriculture,was quite of differentlarge, civil, itlogistics, cannot robotic beand device considered consumer/home) types comprehensive,for six wasdomains10 of exe- 20 applications featuring a combination of different robotic device types for six domains ascuted.(manufacturing, robotics While are the increasingly rehabilitation,latter analysis being agriculture,was used quite for large,new civil, applicationsit logistics, cannot beand inconsidered consumer/home)an even wider comprehensive, variety was exe- of (manufacturing, rehabilitation, agriculture, civil, logistics, and consumer/home) was exe- domainsascuted. robotics While than are thethose increasingly latter analyzed. analysis being Nevertheless, was used quite for large,new the applications conceptit cannot of be cross-domain inconsidered an even wider comprehensive, safety variety skills ofis cuted. While the latter analysis was quite large, it cannot be considered comprehensive, explicitlydomainsas robotics thanconceived are those increasingly toanalyzed. deal withbeing Nevertheless, the used novelty for new ofthe future applicationsconcept robotics of cross-domain in applications an even wider safety and variety provideskills isof foras how robotics to consider are increasingly their safety. being The used safety for skills new that applications were identified in an even and associatedwider variety and of guidanceexplicitlydomains forthanconceived how those to considertoanalyzed. deal with their Nevertheless, the safety. novelty The ofthe safety future concept skills robotics ofthat cross-domain wereapplications identified safety and and provide skills asso- is knowndomains operating than those modes analyzed. from existing Nevertheless, standards the are concept listed inof Table cross-domain3. safety skills is ciatedguidanceexplicitly and for conceivedknown how operatingto considerto deal modeswith their the safety.from novelty exis Theting tingof safety future standardsstandards skills robotics that areare werelistedlistedapplications identified inin TableTable and 3.3. and provide asso- explicitly conceived to deal with the novelty of future robotics applications and provide ciatedguidance and for known how operatingto consider modes their safety.from exis Theting safety standards skills that are werelisted identifiedin Table 3. and asso- Table 3. Safety skills identifiedguidance and corresponding for how to operating consider modes their safety. and/or The testing safety procedures skills that from were existing identified standards. and asso- Table 3. Safety skills identifiedciated and and corresponding known operating operating modes mode ssfrom and/orand/or exis testingtestingting standardsproceduresprocedures are fromfrom listed existingexisting in Table standards.standards. 3. ciated and known operating modes from existing standards are listed in Table 3. Table 3. Safety skills identified and correspondingCorresponding operatingCorresponding mode Operatings and/or Operating testingModes Modes proceduresand/or and/or Testing from Testing Procedures existing Procedures standards. Icon Safety Skill Safety Skill Corresponding Operating Modes and/or Testing Procedures IconTable 3. Safety skillsSafety identified Skill and corresponding operating modewiths and/or Standardwith testing Standard Reference procedures Reference from existing standards. Table 3. Safety skills identified and correspondingCorresponding operating mode Operatingwiths and/or Standard testingModes Reference proceduresand/or Testing from Procedures existing standards. Icon Safety Skill • •CorrespondingISO/TS 15066 Operatingwith [30]: Standard SRMS Modes (§5.5.2), Reference and/or SSM Testing (§5.5.4). Procedures Maintain safe dis- • ISO/TSISO/TS 1506615066 [30]:[30]: SRMSSRMS (§5.5.2),(§5.5.2), SSMSSM (§5.5.4).(§5.5.4). Icon SafetyMaintain Skill safe distance •CorrespondingISO/TR 23482-1 Operating [37]: response Modes and/or to safety-related Testing Procedures obstacles on the Icon Safetytance Skill •• ISO/TR 23482-1 [37]: responsewith to Standardsafety-related Reference obstacles on the ground (§15). Maintaintance safe (MSD) dis- ISO/TRISO/TS 1506623482-1ground [30]: [37]: SRMS response (§15). (§5.5.2),with to Standard safety-related SSM (§5.5.4). Reference obstacles on the ground (§15). (MSD) • ISO 3691-4 [35]: tests for detection of persons (§5.2). Maintain(MSD)tance safe dis- •• ISOISO/TRISO/TS 3691-4 23482-1•15066 [35]:ISO [30]: tests[37]: 3691-4 SRMS forresponse detection [35 (§5.5.2),]: tests to safety-related SSMof for persons detection (§5.5.4). (§5.2). obstacles of persons on (§5.2). the ground (§15). Maintain safe dis- • ISO/TS 15066 [30]: SRMS (§5.5.2), SSM (§5.5.4). (MSD)tance •• ISOISO/TR 3691-4 23482-1 [35]: tests[37]: forresponse detection to safety-related of persons (§5.2). obstacles on the ground (§15). tance •• ISO/TR 23482-1 [37]: response to safety-related obstacles on the ground (§15). (MSD) •• ISOISOISO 3691-43691-4 3691-4• [35]:[35]: [35]:ISO stabilitystability tests 3691-4 for teststests [ 35detection]: (§5.3).(§5.3). stability of persons tests (§5.3). (§5.2). Maintain(MSD) dynamic •• ISO 3691-4 [35]: tests for detection of persons (§5.2). (MSD) •• ISO/TRISO/TRISO 3691-4 23482-123482-1• [35]:ISO/TR [37]:[37]: tests staticstatic for 23482-1 detection stabilitystability [37]: characteristics characteristicsof static persons stability (§5.2). (§11),(§11), characteristics dynamicdynamic (§11),stabilitystability dynamic characteris-characteris- Maintainstabilitystability dynamic stabilityISO 3691-4 [35]: stability tests (§5.3). Maintain dynamic •• ticstics withwith respectrespectstability toto movingmoving characteristics partsparts (§12),(§12), with dydynamic respect stability to moving characteristics parts (§12), with dynamic respect (DYS) (DYS) • ISO/TRISO 3691-4 23482-1 [35]: [37]: stability static tests stability (§5.3). characteristics (§11), dynamic stability characteris- Maintainstability(DYS) dynamic • toISO travel 3691-4 (§13), [35]:stability response stability characteristics to tests safety-r (§5.3).elated with obstacles respect toon travel the ground (§13), response(§15). to Maintain dynamic • toticsISO/TR travel with 23482-1 (§13),respect response [37]:to moving static to safety-r stabilityparts (§12),elated characteristics dy obstaclesnamic stability on(§11), the dynamic groundcharacteristics (§15). stability with characteris- respect stability(DYS) • ISO/TR 23482-1safety-related [37]: static stability obstacles characteristics on the ground (§11), (§15). dynamic stability characteris- stability • ISO/TStotics travel with 15066 (§13), respect [30]: response to PFL moving (§5.5.5), to safety-r parts (HG, (§12),elated §5.5.3). dy obstaclesnamic stability on the groundcharacteristics (§15). with respect (DYS) ISO/TStics with 15066 respect [30]: to PFL moving (§5.5.5), parts (HG, (§12), §5.5.3). dynamic stability characteristics with respect (DYS) • to travel •(§13),ISO/TS response 15066 to safety-r [30]: PFLelated (§5.5.5), obstacles (HG, on §5.5.3). the ground (§15). •• IEC-80601-2-78IEC-80601-2-78ISO/TSto travel 15066 (§13), [30]: response[36]:[36]: PFL overtravelovertravel (§5.5.5), to safety-r endend(HG,elated stopsstops §5.5.3). (§obstacles(§201.9.2.3.2), on the movement ground (§15). beyond pre-set Limit physical inter- to travel •(§13),IEC-80601-2-78 response to safety-r [36]:elated overtravel obstacles end stops on the (§201.9.2.3.2), ground (§15). movement •• limitslimitsIEC-80601-2-78ISO/TS forfor 15066 individual.individual. [30]: [36]: PFL overtravel PatientPatient (§5.5.5), movementmovement end (HG, stops §5.5.3). (§201.(§201. (§201.9.2.3.2), 9.2.3.101), movement movement beyond or force/torque pre-set of action energy • beyond pre-set limits for individual. Patient movement LimitLimit physical physical inter- interaction• RACAISO/TS robot 15066 (§201.12.101), [30]: PFL (§5.5.5), mechanical (HG, §5.5.3). hazar ds associated with misalignment (LIE) • limitsIEC-80601-2-78 for individual.(§201.9.2.3.101), [36]: overtravelPatient movement movement end stops (§201. or(§201.9.2.3.2), force/torque9.2.3.101), movementmovement of RACA robotbeyondor force/torque pre-set of Limitaction physical(LIE) energy energy inter- • (§201.9.101).IEC-80601-2-78 [36]: overtravel end stops (§201.9.2.3.2), movement beyond pre-set Limit physical inter- (§201.9.101).RACAlimits forrobot individual. (§201.12.101),(§201.12.101), Patient mechanical mechanical movement hazar hazards (§201.ds9.2.3.101), associated associated movement with with misalignment misalignment or force/torque of action(LIE) energy (LIE) • ISO/TRlimits for 23482-1 individual. [37]: physical Patient movement hazard char (§201.acteristics9.2.3.101), (for mobilemovement robots), or force/torque (§7) of action energy ISO/TR(§201.9.101).RACA 23482-1robot (§201.12.101),(§201.9.101). [37]: physical mechanical hazard char hazaracteristicsds associated (for mobile with robots),misalignment (§7) (LIE) RACA robot (§201.12.101), mechanical hazards associated with misalignment • ISO/TR 23482-1• ISO/TR [37]: physical 23482-1 hazard [37]: physical characteristics hazard characteristics(for mobile robots), (for mobile (§7) (LIE) • ISOISO(§201.9.101). 10218-110218-1 [28]:[28]: speedspeed controlcontrol (§5.6),(§5.6), axisaxis andand spacespace limitinglimiting (§5.12).(§5.12). Limit range of move- • (§201.9.101). robots), (§7) Limit range of move- • IEC-80601-2-78ISO/TR 23482-1 [36]: [37]: movement physical hazard beyond char pre-acteristicsset limits (for for mobileindividual robots), patient (§7) move- • IEC-80601-2-78ISOISO/TR 10218-1 23482-1 [28]: [36]: [37]: speed movement physical control hazard (§5.6),beyond charaxis pre-acteristics andset spacelimits (for limitingfor mobileindividual (§5.12). robots), patient (§7) move- Limit rangement of move- •• mentIEC-80601-2-78ISO 10218-1 (§201.9.2.3.101), [28]: [36]: speed movement overtrav controlel beyond(§5.6), end stops axis pre- (§201.9.2.3.2)andset limitsspace limitingfor individual (§5.12). patient move- (LRM)(LRM) •• • ISO 10218-1 [28]: speed control (§5.6), axis and space limiting (§5.12). Limit rangement of move-•• (ISO/TS(ISO/TSISO 10218-1 1506615066 [28]: [30]:[30]: speed HG,HG, control§5.5.3).§5.5.3). (§5.6), axis and space limiting (§5.12). LimitLimit range range of move- of movement mentIEC-80601-2-78 (§201.9.2.3.101),• IEC-80601-2-78 [36]: movement overtrav [el36 beyond]:end movement stops pre- (§201.9.2.3.2)set beyond limits pre-setfor individual limits for patient individual move- (LRM)ment • IEC-80601-2-78 [36]: movement beyond pre-set limits for individual patient movement (LRM) • (ISO/TSment (§201.9.2.3.101), 15066 patient[30]: HG, movementovertrav §5.5.3).el (§201.9.2.3.101),end stops (§201.9.2.3.2) overtravel end stops (§201.9.2.3.2) (LRM) ment (§201.9.2.3.101), overtravel end stops (§201.9.2.3.2) (LRM) • (ISO/TS 15066• (ISO/TS [30]: HG, 15066 §5.5.3). [30 ]: HG, §5.5.3). Maintain proper • (ISO/TS 15066 [30]: HG, §5.5.3). Maintain proper • (ISO/TSIEC-80601-2-78 15066 [30]: [36]: HG, mechanical §5.5.3). hazards associated with misalignment (§201.9.101) IEC-80601-2-78 [36]: mechanical hazards associated with misalignment (§201.9.101) alignmentMaintain proper(MPA) • IEC-80601-2-78 [36]: mechanical hazards associated with misalignment (§201.9.101) alignmentMaintain proper(MPA) • MaintainMaintain proper proper alignment• IEC-80601-2-78• IEC-80601-2-78 [36]: mechanical [36]: hazards mechanical associated hazards with associated misalignment with (§201.9.101) alignment (MPA) •• ISO/TRISO/TRIEC-80601-2-78 23482-123482-1 [37]:[37]: [36]: testtest mechanical ofof physicalphysical hazards hazardhazard as chchsociatedaracteristics with misalignment(for restraint-type (§201.9.101) physical alignment (MPA)(MPA) misalignment (§201.9.101) • assistance robots), (§8). Limit restraining en- ISO/TR 23482-1 [37]: test of physical hazard characteristics (for restraint-type physical •• IEC 80601-2-78 [36]: accuracy of controls and instruments and protection against haz- ergy IECassistanceISO/TR 80601-2-78 23482-1 robots), [36]: [37]: (§8). accuracy test of physical of controls hazard and chinstrumentsaracteristics and (for protection restraint-type against physical haz- Limit restrainingergy en- • ISO/TR 23482-1 [37]: test of physical hazard characteristics (for restraint-type physical • ardousISO/TR outputs 23482-1• ISO/TR (§201.12), [37]: test 23482-1 ofmechanical physical [37]: test hazard hazards of physical ch associatedaracteristics hazard with characteristics(for misalignment restraint-type (for physical (LRE)(LRE) IECassistance 80601-2-78 robots), [36]: (§8). accuracy of controls and instruments and protection against haz- Limit restrainingergy en- • (§201.9.101),(§201.9.101),assistance robots), movementmovementrestraint-type (§8). beyondbeyond physical pre-setpre-set assistance limitslimits forfor robots), individualindividual (§8). patientpatient movementmovement Limit restraining en- • ardousIEC 80601-2-78 outputs (§201.12),[36]: accuracy mechanical of controls hazards and instrumentsassociated with and misalignmentprotection against haz- Limit(LRE)ergy restraining • energy(§201.9.2.3.101).IEC 80601-2-78• IEC [36]: 80601-2-78 accuracy [of36 controls]: accuracy and of instruments controls and and instruments protection and against haz- ergy (§201.9.2.3.101).(§201.9.101),ardous outputs movement (§201.12), beyond mechanical pre-set h azardslimits for associated individual with patient misalignment movement (LRE) (LRE) ardous outputsprotection (§201.12), against mechanical hazardous hazards outputs associated (§201.12), with mechanicalmisalignment hazards (LRE) (§201.9.2.3.101).(§201.9.101), movement beyond pre-set limits for individual patient movement (§201.9.101), movementassociated beyond with misalignment pre-set limits (§201.9.101), for individual movement patient beyondmovement pre-set 3.2. Protocols(§201.9.2.3.101). for Skill-Based Validation of Applications (§201.9.2.3.101).limits for individual patient movement (§201.9.2.3.101). 3.2. ProtocolsAs per their for Skill-Based definition Validation and scope, of Applicationsskills are abstract concepts applying to different collaborative3.2. Protocols foroperation Skill-Based scenarios. Validation Their of Applicationsproper and effective application depends on sev- collaborative3.2. ProtocolsAs per their foroperation Skill-Based definition scenarios. Validation and scope, Their of Applicationspropskillser ar ande abstract effective concepts application applying depends to different on sev- 3.2.eralcollaborative Protocols operationAs per for their operation Skill-Basedfeatures, definition scenarios.such Validation and as the scope, Their of domain Applications propskills erin ar andwhiche abstract effective they concepts are application tested, applying the depends design to different on of sev-the As per their definition and scope, skills are abstract concepts applying to different robotizederalcollaborativeAs operation per operation, their operation features, definition and scenarios. suchthe and type scope,as the of Their robotic skillsdomain prop are system iner abstract andwhich involved. effective concepts they areVerifying application applying tested, and the todepends validatingdifferentdesign onof col- sev-thethe collaborative operation scenarios. Their proper and effective application depends on sev- laborativeapplicationrobotizederal operation operation operation, of a features,safety scenarios. and skill suchthe in typea Their asspecific theof properrobotic domain scen andario system in effectivecorresponds which involved. they application to areVerifying assessing tested, depends and thethe validating application’sdesign on several of thethe eral operation features, such as the domain in which they are tested, the design of the operationsafetyapplicationrobotized features features, operation, of a from safety such the and skill as perspective thethe in domain typea specific of roboticof in scen whichthe spario systemecific they corresponds are skill,involved. tested, providing to theVerifying assessing design evidence and of the the validating application’sof robotized the effec- the robotized operation, and the type of robotic system involved. Verifying and validating the operation,tivenesstivenesssafetyapplication features ofof and thethe of the a fromsafetysafety safety type the measures ofmeasuresskill perspective robotic in a specific systemimplementedimplemented of scenthe involved. sparioecific with corresponds Verifying skill, reference providing andto to assessing validatingthe evidence relevant the the application’sof standards. applica-the effec- application of a safety skill in a specific scenario corresponds to assessing the application’s tiontivenesssafety of a features safety of the skill fromsafety in athe specificmeasures perspective scenario implemented of correspondsthe specific with skill, toreference assessing providing to the the evidence application’s relevant of standards. the safety effec- safety features from the perspective of the specific skill, providing evidence of the effec- featurestiveness from of the the perspectivesafety measures of the implemented specific skill, with providing reference evidence to the of relevant the effectiveness standards. tiveness of the safety measures implemented with reference to the relevant standards. of the safety measures implemented with reference to the relevant standards. Such an

assessment is not a trivial task, as it requires, in order: (i) a comprehensive knowledge of the applicable regulation landscape, (ii) critical awareness of the relevant physical metrics

and performance data to be measured, (iii) technical knowledge of the most appropriate testing equipment and methodologies, (iv) the production of clear, complete and self- explanatory reports. Robotics 2021, 10, 65 11 of 20

A suitable methodology to validate safety skills relies on the application of testing protocols, developed with the specific scope of providing a step-by-step guide on how to execute a validation measurement to check the safety of a robotics application. To exploit the potential of the cross-domain approach, protocols have to be general procedures, applicable in several domains and individuated only by two fundamental variables, that are the safety skill to be validated and the robotic device involved in the specific task to be used for the validation. To maximize their impact, protocols have to pursue the following scopes: - increasing the familiarity in the robotics community with possible measuring tech- niques; - informing protocol users of what aspects of their risk analysis and system behavior are relevant for the validation. In fact, the risk assessment, conducted as per the relevant standards, is a fundamental, preliminary step for the validation of a safety skill. It is necessary, indeed, to define the conditions in which protocol procedures must be applied; these conditions depend on the specific installation, environment, task, user awareness, safeguarding measures provided by the manufacturer, prescribed protective measures, and so on. A set of safety skill validation protocols is available in COVR Toolkit [25]; they are designed for the system-level evaluation of the safety performance. To cope with the aforementioned requirements, the structure of a protocol is organized to guide the user in all the steps of the safety skill validation (see Table4). Aiming at enhancing clarity of contents and usability, relevant stakeholders and protocol users can provide feedback during the whole protocol lifecycle. As a result, the available protocols are proven testing procedures, intended to be dynamically and periodically updated according to the increasing usage feedbacks. They are indeed published once a “Protocol Readiness Level” 7 (PRL, defining the level of protocol advancement on a nine-level scale) is achieved and they are open for further user feedbacks, collected by means of specific questionnaires.

Table 4. General structure of safety skill validation protocols.

Section Contents • Relevant regulations and standards Introduction • Scope and limitations

• Characterization of hazards to consider for skill validation • Target metrics: physical, measurable quantities on which depends Concept and objectives the validation (or not) of a skill in relation to a speciﬁc risk reduction level

Conditions • System, environment, possible subsystems, other relevant aspects

• Based on the measurement device indicated for the speciﬁc Setup safety skill

• Step-by-step testing procedure Procedure • Practices for data analysis • Formats for reporting and documenting the validation of the skill

Annexes • Record sheets and eventual further relevant information

The development of a protocol depends on the analysis of the meaningful combinations of safety skills and robot device type, as reported in Table5. The list of robotic device types reported in the table is based on the systems typical of the industrial practice, whose implementation and use are extended to several application domains (i.e., robotic arms in assistive robotics or mobile robots in the agricultural domain) and on the robotic systems typically used in the rehabilitation ﬁeld. This set represents a ﬁrst nucleus of devices, but further systems can be included to develop more validation protocols. In some cases, for Robotics 2021, 10, 65 12 of 20

the same device–skill combination, several protocols were developed, considering the use of different measuring equipment. Likewise, there can be speciﬁc operation conditions, which can variate depending on task design or the context of use, leading to different testing approaches. These aspects result in the development of several protocols for the same device–skill combination, individuated by the increasing numbers in Table5.

Table 5. Identiﬁcation of protocols as a combination of device types and safety skills.

Device Type MSD DYS LIE LRM MPA LRE Robotic arm ROB 1, 2 * - 1, 2 1, 2, 3, 4, 5 * - 1 Mobile platform MOB 1 1 1 1 - - Exoskeleton EXO - - 1 1 1 1 * Gripper ** GRI - - 1 1 - - Weight support WSU - - 1 * 1 * 1 * 1 * Balance Trainer BAT - - 1 * 1 * 1 * 1 * Mobile robot MRO - 1 1 - - - NOTE: Safety skills acronyms (MSD, DYS, LIE, LRM, MPA, LRE) are explained in Table3. The numbers represent the different protocols developed per each combination of device type and skill. * Still not published (PRL ≤ 6). ** Considered separately as characterized by a speciﬁc motion (closing jaws).

The list of protocols currently available in COVR Toolkit is reported in Table6. In the table, some details are specified, in order to enable the observation of the main similarities and differences. With this perspective, the following considerations arise: • Protocol applicability conditions are general and valid in different scenarios. The main advantage of basing protocol development on a cross-domain perspective is the possibility of reducing their quantity, still meeting the specific needs of a wide variety of application cases. • The assessed metrics can be based on measured values or on Boolean variables. This is mainly related to the required testing equipment, as in the case of video cameras, the observation of the test enables analysis only on a threshold-based assessment. • Regardless of the test metrics, safety skills are validated based on the verification of the compliance with specific thresholds, which are provided by specific standards, or, if not available, determined by the manufacturer. None of the thresholds are proposed by protocol designers, in compliance with the protocol concept and aims. • To maximize the usability of protocols and, consequently, further shorten the distances between the users and the safety validation, where possible, several kinds of testing equipment and methodologies are suggested. • All the LIE validation protocols are based on the use of sensing devices with the same basic principle, which is acquiring normal force and pressure. Furthermore, they all can be characterized as “biofidel”, referring to the capability of reproducing the biomechanical behavior of the human body part potentially involved in the contact. This approach represents the state-of-the-art for the assessment of human–robot physical interaction, which is expected to be adopted by the relevant ISO standards. As shown in Section 2.2, this is indeed one of the new aspects introduced in the ISO/DIS 10218-2 [40] and a similar device is also envisaged by the ISO/TR 23482-1 [37]. Robotics 2021, 10, 65 13 of 20

Table 6. Summary table of the safety skill validation protocols available on COVR Toolkit (PRL ≥ 7).

ID 1 Conditions Object Main Reference Metrics Sensing Device(s) Data Analysis Repetitions Human entering ISO 13855 [31] String potentiometer, Ss ≤ S ROB-MSD-1 Indoor/factory Safety Distance Ss P 5 robot workspace ISO/TS 15066 [30] triggering actuator or Xs ≤ XH ISO/TS 15066 [30] Force and pressure Filtering, correction 2, Moving obstacle Transient contact ROB-LIE-1 Indoor/factory DGUV FBHM 080 (biofidelity to the comparison with 3 (human body part) force and pressure [44] specific body part) limits in [30] ISO/TS 15066 [30] Transient and Force and pressure Stationary obstacle Filtering, comparison ROB-LIE-2 Indoor/factory DGUV FBHM 080 quasi-static contact (biofidelity to the 3 (human body part) with limits in [30] [44] force and pressure specific body part) Robotic device, Pre-defined torque, Obtain torque and ROB-LRE-1 IEC 80601-2-78 [36] Effective torque Force sensor 2 per 3 sets single human joint human body part compare with limits Robot structure point Light barrier or Vertical plane ISO 10218-2 [29] Check P does not ROB-LRM-1 Indoor/ factory P crosses plane target plane + camera cross 3 (change angle) (workspace limit) ISO 10218-1 [28] cross cross the plane [T/F] or similar Optoelectronic ROM exit [T/F], RACA, 3D mov., Pre-defined ROM, measurement or Acquire robot motion ROB-LRM-2 IEC 80601-2-78 [36] potential collision robot control human body part motion tracking and compare to ROM [T/F] system Optoelectronic ROM exit [T/F], Acquire robot motion RACA, 3D mov., Pre-defined ROM, measurement or ROB-LRM-3 IEC 80601-2-78 [36] potential collision (external co-guiding) co-control human body part motion tracking [T/F] and compare to ROM system Instrumented limb ROM exit [T/F], Obtain robot motion RACA, 3D mov., Pre-defined ROM, (angular encoders) ROB-LRM-4 IEC 80601-2-78 [36] potential collision (external co-guiding) co-control human body part attached to the [T/F] and compare to ROM end-effector Footprint Position tracker or Acquire robot motion observation: video-camera or light MOB-LRM-1 Indoor Forbidden area from top or process 3 entrance in forbidden curtain or laser the acquisition area [T/F] scanner or similar Accelerations, stops, EN 1525 [45] ROLL, PITCH angles Video-camera, Apply safety factor, MOB-DYS-1 Indoor 5 payload, floor tilt ISO 3691-4 [35] below limits [T/F] inclinometer compare with limits Robotics 2021, 10, 65 14 of 20

Table 6. Cont.

ID 1 Conditions Object Main Reference Metrics Sensing Device(s) Data Analysis Repetitions ISO/TS 15066 [30] Force and pressure Stationary obstacle Quasi-static contact Filtering, comparison MOB-LIE-1 Indoor/ factory DGUV FBHM 080 (biofidelity to the 3 (human body part) force and pressure with limits in [30] [44] specific body part) Distance from Distance measuring EN 1525 [45] Compare with limit MOB-MSD-1 Indoor/outdoor Stationary obstacle detected obstacle system, ground 3 ISO 3691-4 [35] in risk assessment after stop markers. Stationary/moving ISO/TS 15066 [30] Transient and Force and pressure Filtering, correction EXO-LIE-1 Exoskeleton obstacle (therapist DGUV FB HM080 quasi-static contact (biofidelity to the (trans.), comparison 3 body part) [44] force and pressure specific body part) with limits in [30] Joint Filtered values must Segments of an Swinging limb EXO-LRM-1 IEC 80601-2-78 [36] flexion/extension Electro-goniometer be >0◦ and comply exoskeleton segments angles with subject limits Optoeletronic Force and torque in Calculate joint forces Exoskeleton/ measurement or EXO-MPA-1 Limb segments IEC 80601-2-78 [36] joint(s), and torqes and Restrain-type RACA similar, instrumented misalignment compare with limits limb Filtering, correction Stationary/moving ISO/TS 15066 [30] Transient and Force and pressure (transient), GRI-LIE-1 Any gripper obstacle (human DGUV FBHM 080 quasi-static contact (biofidelity to the 3 comparison with body part) [44] force and pressure specific body part) thresholds in [30] Observe object Obstacle (human Minimum gap after Gauge blocks or GRI-LRM-1 Any gripper position after (failed) 5 body part) closure (pre-defined) similar pick EN 1525 [45] Accelerations, stops, ROLL, PITCH angles Apply safety factor, MRO-DYS-1 Indoor ISO 3691-4 [35] Inclinometer 5 payload, floor tilt below limits [T/F] compare with limits ISO 18646-1 [41] ISO/TS 15066 [30] Force and pressure Filtering, comparison Stationary obstacle Quasi-static contact MRO-LIE-1 Indoor/ factory DGUV FBHM 080 (biofidelity to the with thresholds in 3 (human body part) force and pressure [44] specific body part) [30] 1 The code identifying each protocol has the structure XXX-YYY-Z, in which XXX refers to the device type, YYY to the safety skill and Z to protocol variant (see Table5). 2 Adaptation for the assessment of transient contact according to the methodology in [46]. Robotics 2021, 10, 65 15 of 20

4. Examples of Protocol Use and Application In this Section, three application cases of the validation protocols are shown. It is worth mentioning that the process of developing validation protocols involved a clear speciﬁcation of the use-case and protocol user needs, input from the experience of the protocol developers from previous measurements, and several feedback loops from users within the COVR consortium and from the projects sponsored through the cascade funding mechanism within the overall COVR project framework. The application cases hereafter reported refer to: • an experimental campaign, belonging to the in-house trials performed by the COVR partners, aimed at obtaining the maximum permissible velocities of a collaborative robot in some areas of the workspace; • the validation of a mobile robot for a retail environment; • the validation of a rehabilitation robotic device.

4.1. Admissible Velocities of a Robot Arm in a Shared Workspace The test procedure described in the protocol ROB-LIE-1 “Test robot arm for collision with movable object (Impact)” was used to perform a series of impact tests in a pre-defined set of positions, in order to obtain an indication of the maximum permissible operational velocities of a Universal Robots UR10 in different areas of its working volume, with reference to specific human–robot contact scenarios. The protocol is aimed at testing the contact force and pressure affecting the operator in an unintended contact scenario with a robot with PFL functions activated, to be then compared with the force and pressure limits provided by ISO TS/15066 [30]. The contact scenario identified in the risk assessment is reproduced to obtain the force and pressure values related to the transient contact phase. The robot program from the real application is executed and the measurement takes place along the position in space where the risk analysis specifies that contact is likely to occur. The robot collides with the sensing equipment, which consists in a biofidelic load-cell sensor and a foil pressure sensor. Data is then post-processed and the peak value is obtained for the transient contact phase (tc ≤ t ≤ tc + 0.5 s, with tc representing the time in which contact occurs). The tests carried out considered the contact of a test piece with the sternum, the abdominal region and the back of the hand at three different points (see Figure1a) in the robot workspace. The measurements were executed at different velocities for each position (with a 50 mm/s increment for linear trajectories and 5 deg/s for the point-to-point motions). The following table reports the setup of each of the tests, performed using a GTE CBSF-75 Basic with adjustable spring and impact pad as force sensor. The obtained force values were then compared to the limits in the ISO/TS 15066 [30] to identify the permissible robot velocities for each position with the tested configurations. The results related to the tests performed in CNR-STIIMA laboratories are also reported in Table7.

Table 7. Setup details and permissible velocities validated by the application of ROB-LIE-1 protocol.

Spring Damping Permissible Values TR 1 Validated Permissible Velocities Contact Body Stiffness Material Point Region Force Pressure x (PtP 1) y (Linear) z (Linear) [N/mm] [44] [SH] [N] [N/cm2] [deg/s] [mm/s] [mm/s] P1 Sternum 25 70 280 240 30 200 unlikely P2 Abdomen 10 10 220 220 30 200 unlikely P2 Back Hand 75 70 280 400 20 150 300 P3 Abdomen 10 10 220 220 45 400 unlikely P3 Back Hand 75 70 280 400 30 300 100 1 TR: Transient contact; PtP: Point-to-Point. Robotics 2021, 10, x FOR PEER REVIEW 14 of 19

4. Examples of Protocol Use and Application In this Section, three application cases of the validation protocols are shown. It is worth mentioning that the process of developing validation protocols involved a clear specification of the use-case and protocol user needs, input from the experience of the protocol developers from previous measurements, and several feedback loops from users within the COVR consortium and from the projects sponsored through the cascade funding mechanism within the overall COVR project framework. The application cases hereafter reported refer to: • an experimental campaign, belonging to the in-house trials performed by the COVR partners, aimed at obtaining the maximum permissible velocities of a collaborative robot in some areas of the workspace; • the validation of a mobile robot for a retail environment; • the validation of a rehabilitation robotic device.

4.1. Admissible Velocities of a Robot Arm in a Shared Workspace The test procedure described in the protocol ROB-LIE-1 “Test robot arm for collision with movable object (Impact)” was used to perform a series of impact tests in a pre-defined set of positions, in order to obtain an indication of the maximum permissible operational velocities of a Universal Robots UR10 in different areas of its working volume, with reference to specific human–robot contact scenarios. The protocol is aimed at testing the contact force and pressure affecting the operator in an unintended contact scenario with a robot with PFL functions activated, to be then compared with the force and pressure limits provided by ISO TS/15066 [30]. The contact scenario identified in the risk assessment is reproduced to obtain the force and pressure values related to the transient contact phase. The robot program from the real application is executed and the measurement takes place along the position in space where the risk analysis specifies that contact is likely to occur. The robot collides with the sensing equipment, which consists in a biofidelic load-cell sensor and a foil pressure sensor. Data is then post-processed and the peak value is obtained for the transient contact phase (tc ≤ t ≤ tc + 0.5s, with tc representing the time in which contact occurs). The tests carried out considered the contact of a test piece with the sternum, the abdominal region and the back of the hand at three different points (see Figure 1a) in the robot workspace. The measurements were executed at different velocities for each position (with a 50 mm/s increment for linear trajectories and 5 deg/s for the point-to-point motions). The following table reports the setup of each of the tests, performed using a GTE CBSF-75 Basic with adjustable spring and impact pad as force sensor. The obtained force Robotics 2021, 10, 65 values were then compared to the limits in the ISO/TS 15066 [30] to identify the permissi-16 of 20 Robotics 2021, 10, x FOR PEER REVIEW 15 of 19 ble robot velocities for each position with the tested configurations. The results related to the tests performed in CNR-STIIMA laboratories are also reported in Table 7.

Table 7. Setup details and permissible velocities validated by the application of ROB-LIE-1 protocol.

Spring Damping Permissible Values TR 1 Validated Permissible Velocities Contact Body Stiffness Material [44] Force Pressure x (PtP 1) y (Linear) z (Linear) Point Region [N/mm] [SH] [N] [N/cm2] [deg/s] [mm/s] [mm/s] P1 Sternum 25 70 280 240 30 200 unlikely P2 Abdomen 10 10 220 220 30 200 unlikely P2 Back Hand 75 70 280 400 20 150 300 P3 Abdomen 10 10 220 220 45 400 unlikely P3 Back Hand 75 70 280 400 30 300 100 1 Figure 1. Impact test TR: conﬁgurationconfiguration Transient contact; ((a)) andand setupsetupPtP: Point-to-Point. ((b),), detaildetail ofof aa verticalvertical impactimpact ((cc).).

4.2. Stockbot: A Mobile Robot for a Retail Environment 4.2. Stockbot: A Mobile Robot for a Retail Environment The Stockbot (pal-robotics.com/robots/stockbot, accessed on 15 April 2021) mobile The Stockbot (pal-robotics.com/robots/stockbot, accessed on 15 April 2021) mobile robot from Pal-Robotics (Figure 2a) is a mobile robot intended for the implementation in robot from Pal-Robotics (Figure2a) is a mobile robot intended for the implementation in retail environmentsenvironments for for stocking stocking purposes purposes during during normal normal activity activity in shops. in shops. The MOB-MSD-The MOB- 1MSD-1 “Test Mobile“Test Mobile Platform Platform to Maintain to Maintain a Separation a Separation Distance” Distance” protocol protocol was used was to validateused to thevalidate ability the of ability the robot of the of maintainingrobot of maintaining a safety distancea safety distance from humans from humans and obstacles. and obsta- The protocolcles. The protocol MOB-MSD-1 MOB-MSD-1 describes describes the validation the validation procedure procedure for a mobile for a mobile robot sharingrobot shar- an environmenting an environment with humans with humans and objects and objects encountered encountered along along its path, its path, aiming aiming at avoidingat avoid- contacting contact by keepingby keeping a safety a safety distance distance in in any any condition. condition. This This is is obtained obtained by by settingsetting upup aa test using a cylindrical object as a dummy, placedplaced verticallyvertically onon thethe floorfloor alongalong thethe robotrobot path. During the test, the robot, with enable enabledd safety functions and after a warm-up phase, moves towards the obstacle and is expected to stopstop automaticallyautomatically beforebefore impactingimpacting withwith the obstacle. The distance between the robot an andd the obstacle is then acquired (Figure (Figure2 2b)b ) and compared to a safetysafety distance,distance, which isis pre-definedpre-defined dependingdepending onon thethe applicationapplication requirements and specifiedspecified in thethe riskrisk assessment.assessment. The test must then bebe repeatedrepeated withwith different-sized obstacles.

Figure 2.2. The Stockbot mobile platform (a), an image from MOB-MSD-1MOB-MSD-1 protocolprotocol ((b),), anan acquisitionacquisition ofof StockbotStockbot testtest setupsetup ((cc).). CourtesyCourtesy ofof PalPal Robotics.Robotics.

In the Stockbot case study, aa widewide varietyvariety of cylindricalcylindrical dummies were tested, considering the the following following colors colors and and materials: materials: ma matttt black, black, bright bright red,red, mirror, mirror, cardboard, cardboard, alu- aluminum,minum, transparent transparent plastic, plastic, and andtwo twodifferent different dimensions. dimensions. They They were wereplaced placed at different at dif- ferentheights heights for testing for testingby using by a usingpost with a post sliding with guides. sliding guides.Tests were Tests performed were performed in an oper- in anating operating environment environment exactly exactlylike in likereal inscenar realios. scenarios. The protocol The protocol was applied was applied with a with main a mainmodification: modiﬁcation: in order in orderto assess to assess the Stockbot the Stockbot advanced advanced functionality functionality of creating of creating an alter- an alternativenative path pathwhen when an obstacle an obstacle is encounter is encountered,ed, the therobot robot was was not notexpected expected to stop to stop during dur- ingthe test, the test, but butthe theminimum minimum distance distance from from the theobstacle obstacle was was calculated, calculated, post-processing post-processing the theacquisitions acquisitions from from an HTC an HTC Vive Vive tracking tracking system system (Figure (Figure 2c), 2whichc), which is recognized is recognized as a assuit- a able 3D tracking system [47]. Besides the application of the MOB-MSD-1, the robot was

Robotics 2021, 10, 65 17 of 20 Robotics 2021, 10, x FOR PEER REVIEW 16 of 19

suitable 3D tracking system [47]. Besides the application of the MOB-MSD-1, the robot validated also considering other hazards (impacts, stability vs. steps or slopes) and the was validated also considering other hazards (impacts, stability vs. steps or slopes) and protocol-based test campaign enabled to successfully validate the use of the robot in the protocol-based test campaign enabled to successfully validate the use of the robot in shared public environments.

4.3. Achilles: A A Robotic Robotic Device Device for for Ankle Rehabilitation The protocol ROB-LRE-1ROB-LRE-1 was was co-developed co-developed and an executedd executed in anin an earlier earlier version version with with the partnersthe partners Amsterdam Amsterdam UMC UMC (location (location VUmc), VUmc), TU Delft TUand Delft LUMC and LUMC (all from (all NL) from to validate NL) to validatethe safety the of safety two hapticof two rehabilitationhaptic rehabilitation robots, robots, Achilles Achilles and Wristalyzer and Wristalyzer (Figure (Figure3) from 3) fromMoog Moog B.V. (B.V.www.moognetherlands.nl (www.moognetherlands.nl,, accessed accessed on 15 on April 15 Ap 2021),ril 2021), used used for for diagnostics diagnos- ticsof joint of joint hyper-resistance hyper-resistance and and quantification quantification of of passive passive stiffness stiffness versus versusreflex reflex torquetorque in neurological patients. They They are are used used for for the ankle and wrist joint, respectively, and cancan apply a torque to the human joint in two different scenarios:scenarios: • to counteract a maximum torque applied by the humanhuman subject,subject, maintainingmaintaining a setset position to determine the subject’s capabilities; • to generategenerate aa motionmotion of of the the attached attached body body segments segments within within pre-defined pre-defined physiologically physiologi- callysafe torque safe torque limits. limits.

Figure 3. WristalyzerWristalyzer robot robot ( (aa),), Achilles Achilles robot robot (b (b) and) and Achilles Achilles during during use use (c). ( cCourtesy). Courtesy of Amsterdam of Amsterdam UMC, UMC, TU Delft TU Delft and andLUMC. LUMC.

The applied protocolprotocol includes includes two two tests tests to to validate validate (i) that(i) that the the torques torques actually actually applied ap- pliedby the by robot the robot match match those those set in set the in robot the robo controlt control (“torque (“torque mode” mode” test) test) and and (ii) that(ii) that the pre-setthe pre-set maximum maximum allowed allowed torques torques are not are exceeded not exceeded (“position (“position mode” mode” test). In test). the torqueIn the torquemode test, mode arod test, is aconnected rod is connected to the robotto the and robot then and connected then connected to the ﬂoorto the via floor a non-elastic via a non- elasticrope with rope a with force a sensor force sensor and a springand a spring (Figure (Figure4). The 4). robotic The robotic device device must be must set tobe apply set to applyan increasing an increasing torque torque according according to a linear to a linear incremental incremental ramp ramp proﬁle; profile; torque torque values values from fromthe robot the robot software software are then are then compared compared to those to those obtained obtained by computingby computing the the force force sensor sen- soracquisitions. acquisitions. In the In positionthe position mode mode test, atest, non-elastic a non-elastic rope, torope, which to which weights weights can be can added be addedthough though a pulley a pulley system, system, is attached is attached to the to robotic the robotic device device (Figure (Figure4). By 4). computing By computing the thedata data obtained obtained by aby force a force sensor, sensor, the torquesthe torques generated generated by the by weightsthe weights and and imposed imposed onthe on therobot robot are obtainedare obtained and comparedand compared to the to values the values acquired acquired by the by robotic the robotic device. device. A spring A springdampens dampens the force the peak force that peak may that result may from result placement from placement of the test of weights. the test weights. The safety The of Achillessafety of wasAchilles validated was validated by applying by theapplying protocol: the inprotocol: the torque in the mode torque test, mode the reliability test, the reliabilityof the torques of the computed torques computed by the robotic by the devices robotic was devices proven, was while proven, in the while position in the mode posi- tiontest itmode was observedtest it was that observed robot torquethat robot application torque application was stopped was before stopped reaching before the reaching pre-set themaximum pre-set torque.maximum torque.

Robotics 2021, 10, 65 18 of 20 Robotics 2021, 10, x FOR PEER REVIEW 17 of 19

Figure 4. Setup for the torque mode (a) and position mode (b) tests of ROB-LRE-1.

5. Conclusions Conclusions This paper dealsdeals withwith safety-related safety-related testing testing procedures procedures in in human–robot human–robot collaboration. collaboration.With With HRC, HRC, the authors the authors refer torefer all thoseto all roboticthose robotic application application scenarios, scenarios, whether whether industrial in- dustrialor not, in or which not, in close which human–robot close human–robot interaction interaction is envisaged is envisaged during robot during operation. robot opera- The tion.authors The claim authors that claim the potential that the ofpotential the latest of frontiersthe latest infrontiers HRC, along in HRC, with along the spread with the of spreadrobots inof arobots variety in a of variety domains of domains and applications, and applications, is currently is currently limited limited by the difficultiesby the dif- ficultiesfaced when faced dealing when withdealing safety with validation safety validation of the specific of the robotic specific application. robotic application. This is due This to isthe due very to articulated the very articulated regulation regulation and standard and framework, standard framework, combined with combined the (still) with scarce the (still)availability scarce ofavailability step-by-step of step-by-step testing procedures. testing procedures. To address thisthis topic,topic, this this paper paper first first addresses addresses the the standards standards and and European European regulation regula- tionwithin within thecontext the context of safety of safety for HRC. for HRC. Likewise, Likewise, an overview an overview of all of the all testing the testing procedures proce- duresprovided provided in the in latest the standards,latest standards, useful useful for safety for safety verification verification and validation, and validation, is provided. is provided.However, However, based onbased the on experience the experience of the authorsof the authors with real with applications, real applications, safety safety validation val- idationissues arising issues due arising to the due cross-fertilization to the cross-fertili of differentzation of domains different pose do newmains challenges, pose new whose challenges,solution whose often requiressolution theoften consideration requires the ofco non-domain-specificnsideration of non-do roboticmain-specific standards robotic and standardsbest practices and asbest a practices whole. By as addressinga whole. By asaddressing a whole theas a industrialwhole the andindustrial non-industrial and non- industrialspecific issues, specific analogies issues, areanalogies highlighted, are highli providingghted, theproviding foundation the foundation for an innovative for an cross-inno- vativedomain cross-doma perspective.in perspective. Accordingly, a new approach is proposed, based on the definitiondefinition of a selected set of safety skills that are applicable to a wide variety of robotic applications, regardless of the implementationimplementation domain. domain. The The cross-domain cross-domain safe safetyty skill skill concept has been translated into testingtesting “protocols”, “protocols”, each each defined defined by by the robo robotictic device implemented in the task and the validated safety skill. Protocols Protocols build build on on standards and best practices and are aimed at providing roboticrobotic users,users, either either non-expert non-expert or or expert expert but but dealing dealing with with new HRCnew HRC challenges, challenges,with guidelines with guidelines for the identificationsfor the identifications of relevant of relevant standards standards and the and testing the procedurestesting proce- to duresfulfill theirto fulfill requirements. their requirements. The first The set of first protocols set of protocols was published was published and is freely and available is freely availablewithin the within COVR the Toolkit COVR [ Toolkit25], and [25], it is and constantly it is constantly updated updated based onbased the on feedback the feedback from fromusers users and experts. and experts. As examples As examples of protocol of protoc use,ol some use, realsome cases real arecases reported are reported in the paper,in the paper,belonging belonging to different to different application application domains. domains. The proposed methodology aims at guiding ne neww users in their approach to safety in HRI, the verification verification procedures and the rele relevantvant standards and at providing a different perspective to experts dealing with HRC in different fields.fields.

Author Contributions: Conceptualization and methodology, M.V., A.S., I.F., J.S., R.B., S.H., C.B., E.L., Author Contributions: Conceptualization and methodology, M.V., A.S., I.F., J.S., R.B., S.H., C.B., A.M., L.S., J.B., G.B.P.-L., M.K., and A.B.L.; writing—original draft preparation, M.V., J.S., C.B., L.S., E.L., A.M., L.S., J.B., G.B.P.-L., M.K., and A.B.L.; writing—original draft preparation, M.V., J.S., C.B., and J.B.; writing—review and editing, M.V., J.S., C.B., L.S., J.B. and I.F.; project administration, A.B.L., L.S., and J.B.; writing—review and editing, M.V., J.S., C.B., L.S., J.B. and I.F.; project administration, A.B.L.,and K.N. and All K.N. authors All authors have read have and read agreed and toagreed the published to the publis versionhed version of the manuscript. of the manuscript. Funding: ThisThis research research was funded funded by by the the European European Union’s Horizon 2020 research and innovation programme, under grantgrant agreement No.No 779966. 779966.

Robotics 2021, 10, 65 19 of 20

Acknowledgments: The authors would like to thank the STOCKBOT team from PAL Robotics and specifically Sarah Terreri, Alessandro di Fava, Sergio Ramos and Francesco Ferro, for the feedback and surveys on the application of MOB-MSD-1 protocol, as well as the SAFEharbor team and specifically Marjolein van der Krogt, Ad de Snaaier (both Amsterdam UMC), Winfred Mugge, Alfred Schouten (both TU Delft) and Jurriaan de Groot (LUMC) for their work on the ROB-LRE-1 protocol. Conflicts of Interest: The authors declare no conflict of interest.

References 1. Villani, V.; Pini, F.; Leali, F.; Secchi, C. Survey on human–robot collaboration in industrial settings: Safety, intuitive interfaces and applications. Mechatronics 2018, 55, 248–266. [CrossRef] 2. Ajoudani, A.; Zanchettin, A.M.; Ivaldi, S.; Albu-Schäffer, A.; Kosuge, K.; Khatib, O. Progress and prospects of the human–robot collaboration. Auton. Robot. 2018, 42, 957–975. [CrossRef] 3. IFR. World Robotic Report 2020. Available online: https://ifr.org/ifr-press-releases/news/record-2.7-million-robots-work-in- factories-around-the-globe (accessed on 30 January 2021). 4. Peters, B.S.; Armijo, P.R.; Krause, C.; Choudhury, S.A.; Oleynikov, D. Review of emerging surgical robotic technology. Surg. Endosc. 2018, 32, 1636–1655. [CrossRef][PubMed] 5. Riek, L.D. Healthcare robotics. Commun. ACM 2017, 60, 68–78. [CrossRef] 6. Duckett, T.; Pearson, S.; Blackmore, S.; Grieve, B.; Wilson, P.; Gill, H.; Hunter, A.J.; Georgilas, I. Agricultural Robotics: The Future of Robotic Agriculture; UK-RAS Network: London, UK, 2018. 7. Kimmig, R.; Verheijen, R.H.M.; Rudnicki, M. Robot assisted surgery during the COVID-19 pandemic, especially for gyne-cological cancer: A statement of the Society of European Robotic Gynaecological Surgery (SERGS). J. Gynecol. Oncol. 2020, 31, e59. [CrossRef][PubMed] 8. Tavakoli, M.; Carriere, J.; Torabi, A. Robotics, smart wearable technologies, and autonomous intelligent systems for healthcare during the COVID-19 pandemic: An analysis of the state of the art and future vision. Adv. Intell. Syst. 2020, 2.[CrossRef] 9. Zeng, Z.; Chen, P.-J.; Lew, A.A. From high-touch to high-tech: COVID-19 drives robotics adoption. Tour. Geogr. 2020, 22, 724–734. [CrossRef] 10. Good Work Charter of the European Robotics Industry. EUnited Robotics. Available online: https://www.eu-nited.net/eunited+ aisbl/robotics/news/european-charter-for-robots-and-humans-released-by-engineering-association-eunited.html (accessed on 30 January 2021). 11. Vasic, M.; Billard, A. Safety issues in human-robot interactions. In Proceedings of the 2013 IEEE International Conference on Robotics and Automation, Karlsruhe, Germany, 6–10 May 2013; pp. 197–204. 12. Guiochet, J.; Do Hoang, Q.A.; Kaâniche, M.; Powell, D. Applying Existing Standards to a Medical Rehabilitation Robot: Limits and Challenges. In Proceedings of the 2012 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), Vilamoura, Portugal, 7–12 October 2012. 13. Gopinath, V.; Johansen, K.; Derelöv, M.; Gustafsson, Å.; Axelsson, S. Safe collaborative assembly on a continuously moving line with large industrial robots. Robot. Comput. Manuf. 2021, 67, 102048. [CrossRef] 14. Saenz, J.; Behrens, R.; Schulenburg, E.; Petersen, H.; Gibaru, O.; Neto, P.; Elkmann, N. Methods for considering safety in design of robotics applications featuring human-robot collaboration. Int. J. Adv. Manuf. Technol. 2020, 107, 2313–2331. [CrossRef] 15. Gualtieri, L.; Rauch, E.; Vidoni, R. Emerging research ﬁelds in safety and ergonomics in industrial collaborative robotics: A systematic literature review. Robot. Comput. Manuf. 2021, 67, 101998. [CrossRef] 16. Kumar, S.; Savur, C.; Sahin, F. Survey of human–robot collaboration in industrial settings: Awareness, intelligence, and compliance. IEEE Trans. Syst. Man Cybern. Syst. 2021, 51, 280–297. [CrossRef] 17. Marvel, J.A. Performance metrics of speed and separation monitoring in shared workspaces. IEEE Trans. Autom. Sci. Eng. 2013, 10, 405–414. [CrossRef] 18. European Parliament. Directive 2006/42/EC on Machinery; European Parliament: Brussels, Belgium, 2006. 19. European Parliament. Regulation (EU) 2017/745 on Medical Devices; European Parliament: Brussels, Belgium, 2017. 20. Bessler, J.; Prange-Lasonder, G.B.; Schaake, L.; Saenz, J.F.; Bidard, C.; Fassi, I.; Valori, M.; Lassen, A.B.; Buurke, J.H. Safety assessment of rehabilitation robots: A review identifying safety skills and current knowledge gaps. Front. Robot. AI 2021, 8. [CrossRef] 21. European Parliament. Directive 2001/95/EC on General Product Safety; European Parliament: Brussels, Belgium, 2001. 22. European Parliament. Low Voltage Directive 2014/13/EU; European Parliament: Brussels, Belgium, 2013. 23. European Parliament. Directive 2014/30/EU on the Harmonization of the Laws of the Member States Relating to Electromagnetic Compatibility; European Parliament: Brussels, Belgium, 2014. 24. European Parliament. Radio Equipment Directive 2014/53/EU; European Parliament: Brussels, Belgium, 2014. 25. COVR. Toolkit. Available online: https://www.safearoundrobots.com/home (accessed on 30 January 2021). 26. ISO 12100:2010. Safety of Machinery—General Principles for Design—Risk Assessment and Risk Reduction; International Or-ganization for Standardization: Geneva, Switzerland, 2010. 27. ISO 14971:2019. Medical Devices—Application of Risk Management to Medical Devices; International Organization for Standardization: Geneva, Switzerland, 2019. Robotics 2021, 10, 65 20 of 20

28. ISO 10218-1:2011. Robots and Robotic Devices—Safety Requirements for Industrial Robots—Part 1: Robots; International Organization for Standardization: Geneva, Switzerland, 2011. 29. ISO 10218-2:2011. Robots and Robotic Devices—Safety Requirements for Industrial Robots—Part 2: Robot Systems and Inte-gration; International Organization for Standardization: Geneva, Switzerland, 2011. 30. ISO/TS 15066:2016. Robots and Robotic Devices—Collaborative Robots; International Organization for Standardization: Geneva, Switzerland, 2016. 31. ISO 13855:2010. Safety of Machinery—Positioning of Safeguards with Respect to the Approach Speeds of Parts of the Human Body; International Organization for Standardization: Geneva, Switzerland, 2010. 32. ISO 18497:2018. Agricultural Machinery and Tractors—Safety of Highly Automated Machinery; International Organization for Standardization: Geneva, Switzerland, 2018. 33. ISO 13851:2019. Safety of Machinery—Two-Hand Control Devices—Principles for Design and Selection; International Organi-zation for Standardization: Geneva, Switzerland, 2019. 34. ISO 13482:2014. Robots and Robotic Devices—Safety Requirements for Personal Care Robots; International Organization for Standard- ization: Geneva, Switzerland, 2014. 35. ISO 3691-4:2020. Industrial Trucks—Safety Requirements and Verification—Part 4: Driverless Industrial Trucks and Their Systems; International Organization for Standardization: Geneva, Switzerland, 2020. 36. IEC 80601-2-78:2020. Medical Electrical Equipment—Part 2-78: Particular Requirements for Basic Safety and Essential Performance of Medical Robots for Rehabilitation, Assessment, Compensation or Alleviation; International Electrotechnical Commission: Geneva, Switzerland, 2020. 37. ISO/TR 23482-1:2020. Application of ISO 13482—Part 1: Safety-Related Test Methods; International Organization for Stand-ardization: Geneva, Switzerland, 2020. 38. ISO/TR 20218-1:2018. Robotics—Safety Design for Industrial Robot Systems—Part 1: End-Effectors; International Organization for Standardization: Geneva, Switzerland, 2018. 39. ISO/TR 20218-2:2017. Robotics—Safety Requirements for Industrial Robots—Part 2: Manual Load/Unload Stations; International Organization for Standardization: Geneva, Switzerland, 2017. 40. ISO/DIS 10218-2:2020. Robotics—Safety Requirements for Robot Systems in an Industrial Environment—Part 2: Robot Systems, Robot Applications and Robot Cells Integration; International Organization for Standardization: Geneva, Switzerland, 2020. 41. ISO 18646-1:2016. Robotics—Performance Criteria and Related Test Methods for Service Robots—Part 1: Locomotion for Wheeled Robots; International Organization for Standardization: Geneva, Switzerland, 2016. 42. ISO 18646-2:2019. Robotics—Performance Criteria and Related Test Methods for Service Robots—Part 2: Navigation; International Organization for Standardization: Geneva, Switzerland, 2019. 43. Saenz, J.; Lassen, A.B.; Bidard, C.; Burke, J.H.; Nielsen, K.; Schaake, L.; Vicentini, F. COVR—Towards simplified evaluation and validation of collaborative robotics applications across a wide range of domains using robot safety skills. In Proceedings of the 9th International Conference on Safety of Industrial Automated Systems (SIAS), Nancy, France, 10–12 October 2018. 44. Collaborative robot systems. Design of Systems with "Power and Force Limiting" Function; Fachbereich Holz und Metall, German Social Accident Insurance (DGUV): Heidelberg, Germany, 2017. 45. CEN. EN 1525-Safety of Industrial Trucks—Driverless Trucks and Their Systems; European Committee for Standardization: Brussels, Belgium, 1997. 46. Herbster, S.; Behrens, R.; Elkmann, N. A New Conversion Method to Evaluate the Hazard Potential of Collaborative Robots in Free Collisions. Experimental Robotics; Springer International Publishing: Berlin/Heidelberg, Germany, 2021; pp. 222–232. 47. Diederick, C.; Niehorster, L.; Lappe, M. The accuracy and precision of position and orientation tracking in the HTC vive virtual reality system for scientific research. I-Perception 2017, 8, 2041669517708205.