PhysFrame: Type Checking Physical Frames of Reference for Robotic Systems
Sayali Kate Michael Chinn Hongjun Choi Purdue University University of Virginia Purdue University USA USA USA [email protected] [email protected] [email protected] Xiangyu Zhang Sebastian Elbaum Purdue University University of Virginia USA USA [email protected] [email protected]
ABSTRACT Engineering (ESEC/FSE ’21), August 23–27, 2021, Athens, Greece. ACM, New A robotic system continuously measures its own motions and the ex- York, NY, USA, 16 pages. https://doi.org/10.1145/3468264.3468608 ternal world during operation. Such measurements are with respect to some frame of reference, i.e., a coordinate system. A nontrivial 1 INTRODUCTION robotic system has a large number of different frames and data Robotic systems have rapidly growing applications in our daily life, have to be translated back-and-forth from a frame to another. The enabled by the advances in many areas such as AI. Engineering onus is on the developers to get such translation right. However, such systems becomes increasingly important. Due to the unique this is very challenging and error-prone, evidenced by the large characteristics of such systems, e.g., the need of modeling the phys- number of questions and issues related to frame uses on developers’ ical world and satisfying the real time and resource constraints, forum. Since any state variable can be associated with some frame, robotic system engineering poses new challenges to developers. reference frames can be naturally modeled as variable types. We One of the prominent challenges is to properly use physical frames hence develop a novel type system that can automatically infer of reference. Specifically, the operation of a robotic system involves variables’ frame types and in turn detect any type inconsistencies moving individual body parts and interacting with the external and violations of frame conventions. The evaluation on a set of 180 world. It entails precisely measuring positions and orientations of publicly available ROS projects shows that our system can detect body parts and external objects. All these measurements are repre- 190 inconsistencies with 154 true positives. We reported 52 to devel- sented with respect to a set of coordinate systems (also called frames opers and received 18 responses so far, with 15 fixed/acknowledged. of reference or frames in short). For example, in a three-dimensional Our technique also finds 45 violations of common practices. coordinate system, the location (1,2,3) is meaningless unless we know the frame to which this location refers to. The origin of a CCS CONCEPTS frame provides the reference position (0,0,0), whereas the orienta- • Software and its engineering → Abstract data types; Soft- tion of the three axes tells us the directions in which they point to. ware defect analysis. Further, the origin and orientation of a frame itself may be defined relative to another frame and so on. KEYWORDS Robotic systems usually follow a modular design, in which body parts and control software components are developed indepen- Physical Frame of Reference, Frame Consistency, Type Checking, dently by various parties. As such, different components often Static Analysis, z-score Mining, Robotic Systems use different frames. For example, a camera hasthe camera frame ACM Reference Format: through which the physical world is measured from the camera’s arXiv:2106.11266v1 [cs.RO] 21 Jun 2021 Sayali Kate, Michael Chinn, Hongjun Choi, Xiangyu Zhang, and Sebastian perspective. In this frame, the center of camera is the origin and Elbaum. 2021. PhysFrame: Type Checking Physical Frames of Reference the axes follow the orientations of the camera. The body of a robot for Robotic Systems. In Proceedings of the 29th ACM Joint European Soft- has the body frame whose origin is the center of the body and axes ware Engineering Conference and Symposium on the Foundations of Software are pre-defined based on the shape of robot. Measurements inthe camera frame have to be translated to the body frame before they This is an extended version of an article published in Proc. ESEC/FSE 2021 [12 pages, https://doi.org/10.1145/3468264.3468608]. can be used in body related computation. Since the camera can be mounted at different places and moves during operation, such Permission to make digital or hard copies of part or all of this work for personal or translation has to be done by the developers in software. More classroom use is granted without fee provided that copies are not made or distributed discussion about frames and an example can be found in Section2. for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for third-party components of this work must be honored. Most robotic systems are developed in general purpose languages For all other uses, contact the owner/author(s). such as C/C++, facilitated by domain specific libraries. Such lan- ESEC/FSE ’21, August 23–27, 2021, Athens, Greece guages do not have intrinsic support for the additional complexity © 2021 Copyright held by the owner/author(s). ACM ISBN 978-1-4503-8562-6/21/08. induced by the use of frames. For example, although popular li- https://doi.org/10.1145/3468264.3468608 braries such as ROS [32] provide functions to facilitate translation ESEC/FSE ’21, August 23–27, 2021, Athens, Greece S. Kate, M. Chinn, H. Choi, X. Zhang, S. Elbaum
ROS-Enhancement-Proposals (REPs) and the semantics of frame related operations such as displacements and rotations. As part of our contribution we present a type language as a vehicle to de- scribe these type rules , which constitute the foundation for our domain-specific static analyzer. Our contributions are summarized as follows: • We propose a fully automated type inference and checking tech- Figure 1: Forum posts that show frames in robotic program- nique for physical frames in ROS-based programs to detect frame ming are difficult to handle. (Note: right-side snapshot is taken from inconsistencies and convention violations. http://wiki.ros.org/ROS/Troubleshooting page licensed under the CC BY 3.0) • We define frame and transformation abstract types, and our sys- tem automatically infers such types for variables. This is achieved between frames, the onus is on the developers to determine the by novel abstraction of frame related program semantics. reference frames of program variables, the places where translation • We present a data-driven approach to identifying commonly is needed, and correctly implement the concrete translations. Since followed practices that may not be documented anywhere. Our translation is often done by vector/matrix operations, many de- system also checks for violations of such practices. velopers even realize their own translation functions from scratch • We implement a tool, PhysFrame, and evaluate it on 180 ROS without using ROS APIs. As a result, use of frames is error-prone, projects from Github. It reports 190 type inconsistencies with even for experienced developers. This is evidenced by the fact that 154 true positives (81.05%). We report 52 from projects that are a large number of questions and issues for a robotic system project recently active to developers and have received 18 responses are usually regarding reference frames. Figure1 shows that ques- so far, with 15 fixed/confirmed. It also detects 45 violations of tions tagged with ‘tf’ (a keyword for ROS frame support [15, 36]) common practices. are among the top ten types of questions by ROS developers, such as“confused about coordinate frames", “tf transforms are confusing", 2 REFERENCE FRAMES AND ROS and “is there an easier way to find the coordinates of a point in an- other frame". Also, ROS wiki identifies transform issues as one of 2.1 Frames the top 5 common problems. Besides the difficulties of getting them A coordinate system that is used as a reference for measurements of right during development, misuse of frames may cause robot run- quantities such as position and velocity is called a frame of reference, time malfunction [5], code reuse problems [1, 4], and maintenance in short, a frame. It is defined by two components: an origin point difficulties after deployment. and the axes system. A robotic system often consists of many parts, ROS provides a number of tools to help developers debug frame each having at least a body frame with the origin at the center of related problems [2]. These are mostly runtime tools that can facili- the rigid shape of the part, and axes pointing to some orthogonal tate visualization of frames, relation between frames, details of a directions. It allows measurements in the perspective of the part. transformation between two frames (e.g. when it is created and by The body frame for the base of a robot is often called base_link. It which component). However, it is always more desirable to detect is one of the most important frames in a robot as measurements in problems as early as possible in the development life-cycle. A static this frame describe how the robot as a whole sees the world. The tool that can scan and detect frame misuses before running the body frame for a camera is called the camera frame. If the part is system would be highly desirable. To the best of our knowledge, a sensor, it often has other frames to denote sensor readings. For there are unfortunately no such tools. example, a camera has multiple optical frames, including the color In this paper, we introduce and implement a novel fully auto- frame measuring RGB format image, the depth frame measuring mated static tool to identify frame related problems in ROS-based depth image, and the ir frame measuring infra red image. These projects. Developers direct the tool to operate on a project direc- frames are centered at the corresponding sensors (e.g., lens) instead tory, and the tool automatically models the project, and checks for of the camera body. They are different from the camera frame. potential frame faults. Since each physical state related variable There are also the odometry (odom) frame, the map frame, and (e.g., sensor reading, acceleration, velocity, and position) is associ- the earth frame to denote the physical world, i.e., how the world ated with some frame, our tool builds on such such associations sees the robot and its surroundings. The odom frame represents which are similar to variable types by nature. We hence propose robot’s initial pose and does not move with the robot. It is like a a type system to model reference frames. A traditional type sys- third person standing at the robot’s starting position and observing tem often starts with type annotations that require developers to the motion of robot in the environment. The map frame projects know variable types in the first place, which is difficult in ourcon- everything in a localized map such as inside a room. It is like a third text as knowing the right frames is usually difficult. Instead, our person standing somewhere in the room (not the starting point of technique automatically infers frames and performs type consis- the robot) and observing the robot and its environment. The earth tency checks, leveraging ROS conventions that can be extracted frame is an earth centered frame. from its specifications and mined from code repositories. The tech- During operation, sensor readings are acquired regarding the nique uses frame information sources such as static data values corresponding sensor frames. They are then transformed to the exchanged between components and the ROS frame APIs for as- body frames of the parts where the sensors are installed, and even- signing frame types, and defines type propagation and checking tually to the base_link frame and some world frame(s) in which rules based on the standard coordinate conventions documented in control algorithms use the transformed values to compute actuation PhysFrame: Type Checking Physical Frames of Reference for Robotic Systems ESEC/FSE ’21, August 23–27, 2021, Athens, Greece
ROS Transforms. To specify frames and transformations, devel- opers leverage the tf library to explicitly publish transformations between frames (called transforms in ROS), which also implicitly introduce the frames. A transform consists of a parent frame and a child frame, uniquely identified by their ids. ROS developers of- ten say a transform is from the parent to the child. For example “map→odom” means a transform from the map frame to the odom frame. It is defined by providing the displacements of the child frame’s origin and the rotation of its axes regarding the parent frame. However, the transform is used to translate states in the child frame to the parent frame (not the other way suggested by its name). We unfortunately have to inherit such term ambiguity from ROS. In the rest of the paper, we explicitly distinguish the name of a transform and its use in state translation to avoid confusion. After transforms are published, other ROS components, such as Figure 2: Frames in a Simple Robot standard control algorithms, subscribe to these transforms based signals. These signals may undergo the inverse transformations on their names and use them in computation without knowing until they reach the actuators. A simple commodity robot may have how they are implemented. Hence, these core algorithms can be up to 81 different frames35 [ ] as shown in Figure 7 in Appendix and agnostic to concrete system composition. values have to be correctly transformed between any pair of frames. Launch Files. Many parts have fixed and static relative positions, It is a heavy burden for developers to get all these transformations e.g., camera mounted at a fixed position of an aerial robot. The right. transforms between their frames can be statically defined in launch Figure2 shows a simple robot. Observe that each of its parts has files, which are a kind of static initialization files that define how its own body frame (e.g., camera_link, laser, and base_link). There to start executing a robotic system. The transforms for frames are also the world frames such as odom and map. Suppose if the with dynamic relative positions, e.g., when a camera is mounted vehicle locates an obstacle, then it knows the obstacle’s position on a gimbal, have to be published and updated on-the-fly during in the camera_depth_optical frame. This position is different from operation. They are hence implemented by code. An example launch the position of the obstacle in the map frame. In order to answer file snippet can be found in Appendix A. the question “what is the position of an obstacle in the room?", TF Tree. Transforms are potentially needed in between each pair the vehicle needs to know how the camera_depth_optical frame is of frames. However, specifying/implementing transforms for each positioned and oriented relative to the base_link frame and how pair entails substantial and error-prone efforts. ROS has a clean the base_link frame is positioned and oriented relative to the map hierarchical design to reduce the complexity. If we consider a frame frame, then has to convert the obstacle position from the camera_- as a node and a transform between two frames as an edge, ROS depth_optical frame all the way to the map frame. Moreover, the requires all published frames and transforms to form a tree, that position of the obstacle in the frame of front right wheel, front_- is, no cycles and each node having only one parent. We call it a right_wheel, is different from that in the base_link frame. Hence, TF tree. As such, the developer only needs to focus on realizing to know the position with respect to the front right wheel, the the transforms denoting individual edges. ROS then automatically vehicle needs to know the position and orientation of front_right_- constructs the transform from an arbitrary frame 푓1 to another wheel frame relative to the base_link frame. Different body frames arbitrary frame 푓2, by first performing the transforms from 푓1 to the may have different axis orientation conventions. Figure2 presents lowest common ancestor frame of the two, and then the transform two axis orientations labeled as F1 and F2. Each part has a default from the ancestor to 푓2. It can be easily inferred that such a design forward-facing direction (e.g., the nose of an aerial robot and the substantially reduces the developers’ burden. The below figure front of a camera), the values forward, backward, left, right, etc., are shows the TF tree for the robot in Figure2. Observe that while defined regarding the default direction. F1 and F2 are widely used the developers only need to develop and publish 12 transforms in ROS. Moreover, various libraries use different conventions. For (denoted by the edges), ROS allows 156 transforms from a frame example, (x: right, y: up, z: backward) axis orientation is widely to any other frame. The red edges form a transform path between used in OpenGL. front_right_wheel to camera_depth_optical, when the right wheel on the front of the robot uses data from the camera depth sensor. 2.2 ROS and Frames The Robotic Operating System (ROS) [32] is an open source soft- ware framework for building robotic applications. Its modular, dis- tributed nature has made it widely used. Its users range from a novice developer working on a hobby project to industries. ROS provides various components such as drivers, localization packages, 2.3 Frame Conventions in ROS odometer packages developed by the community in addition to the All ROS projects follow certain conventions when manipulating core components. It provides a package called tf or tf2 [15, 36] to frames. Violation to these conventions would cause integration, support transformation between frames. maintenance, and reuse problems. Our technique leverages these ESEC/FSE ’21, August 23–27, 2021, Athens, Greece S. Kate, M. Chinn, H. Choi, X. Zhang, S. Elbaum conventions to automatically infer and check frame types. There are two sources for extracting conventions: ROS specification and launch files. The former is explicit and the latter is implicit. Since ROS conventions are often related to static coefficients values (e.g., orthogonal axes orientations), they are difficult to extract directly from code where coefficients are largely variables. Explicit Conventions From ROS Specifications. These conven- tions can be classified into the following three categories. Naming Conventions. ROS applications use messages to exchange data between components. The frame_id field in a message speci- fies the frame for the data in that message. These ids follow certain conventions. For example, “map” is used for the map frame [27], and “base_link” for the body frame of the robot’s base [27, 28]; optical frames should have an “_optical” suffix [16]. Axis Orientation Conventions. According to [16], body frames should have the (x: forward, y: left, z: up) or FLU axis orientation (i.e., F1 orientation in Figure2); optical frames should have the (x: right, y: down, z: forward) orientation (i.e., F2 in Figure2). Tree Order Conventions. Recall that in a robotic system, the pub- lished frames and transforms are organized in a TF tree. ROS re- quires TF trees follow a partial order: earth → map → odom → base_link [27]. Note that while a concrete TF tree for a specific robotic system may omit some of these frames and have additional frames, such an order must be respected. The reason is that the order Figure 3: Example of incorrect transform between frames optimizes the performance for mostly commonly seen transforms. Implicit Conventions from ROS Launch Files. Launch files source: https://git.io/JtSHb, fixed source: https://git.io/JtSRt provide a rich resource for mining implicit conventions. We fo- in the child camera frame with the conventional FLU orientation, it cus on extracting two kinds of implicit conventions. should be translated into ORB’s RDF orientation, (x:right, y:down, Co-occurrence Conventions. Robotic systems of a same kind share z:forward). Second, the position of the camera frame in the map similarity in their physical compositions. As such, they often use frame needs to be determined. However, since ORB-SLAM2 mod- similar frames and transforms. This is reflected by co-occurrences els the environment in the camera frame, one can only query the of transforms in launch files. For example, a ground vehicle robot position of map frame in the camera frame, so it needs to be trans- with two wheels has one wheel on its left and the other wheel on posed to acquire the position of camera frame in the map frame. its right. If a transform from base_link to the frame of left wheel Third, the transform should translate the results back to ROS’s FLU is defined, then a transform from base_link to the frame of right orientation. The code shown fails to accomplish the last step. wheel is also defined. Function PublishPositionAsTransform() starting at line 27 Value Conventions. Certain transforms are associated with fixed takes the ORB-SLAM2’s position matrix (denoting the map frame’s coefficient values such as 0 rotation angles or 0 displacement values. position in the camera frame), i.e., variable position, and con- For example, a transform from camera_depth to camera_depth_- structs a ROS transform (line 28) by invoking TransformFrom- optical must have 0 displacement values. Mat(), which starts at line 1. In line 4 it acquires the rotation matrix from the ORB-SLAM2’s position matrix, which includes both dis- 3 MOTIVATION placements and rotation, and performs transposition by function To motivate our technique we use code from the simultaneous lo- t(). Line 6 clones it to tf_camera_rotation. Line 10 creates a calization and mapping (SLAM) component ORB2-SLAM2 (links matrix denoting rotation of an FLU axis system anti clockwise in the caption of Figure3)[ 3, 29]. Localization and mapping algo- for 90 degree with respect to the 푥 axis (shown to the right of rithms like ORB are key to the operation of modern mobile robots, the code). Lines 13 and 17 create two additional rotation matrices. incrementally building a map of the robot surroundings utilizing Lines 20-23 compose the three matrices and the earlier rotation its camera as the robot navigates. This component, however, uses matrix. Line 25 creates a ROS transform from displacements de- different conventions from ROS and hence the most important task noted by tf_camera_translation and rotation denoted by tf_- for interfacing then is to perform appropriate frame conversions. camera_rotation. When the transform is used in state translation, The code snippet in Figure3 aims to implement a transform to a state is multiplied with tf_camera_rotation, which equals to facilitate translation of states from the ROS camera frame to the invYZ*Rz*Rx*rotation. The first three matrixes are equivalent to ROS map frame using the environment model of ORB-SLAM2. The transforming the FLU orientation of the state to the RDF orienta- conversion implementation, however, is incorrect, and PhysFrame tion as shown on the right of line 25. The fourth matrix (rotation) catches this error by type checking. then translates the state from the camera frame to the map frame. Constructing the aforementioned transform is non-trivial, and However, the constructed transform is problematic as it forgets consists of three groups of operations. First, given some ROS state to further translate back to FLU. This bug will cause downstream PhysFrame: Type Checking Physical Frames of Reference for Robotic Systems ESEC/FSE ’21, August 23–27, 2021, Athens, Greece
process is executed once on the collected static transform data- base, but can be updated as new static transforms are incorporated into the database. The produced rule conventions are then used in the analysis for each subject ROS project to identify anti-patterns. Details can be found in Appendix B . A ROS project often contains multiple executable subsystems, each having its own TF-tree. PhysFrame has to analyze these sub- systems one at a time. The file-processor separates a project into subsystems, each consisting of a group of C/C++ source files and launch files that are executed together. A subsystem usually has Figure 4: Overview of PhysFrame a separate initial launch file. The file-processor hence starts from these top level launch files, identifies the child launch files and system mis-behaviors, which could be devastating if the camera the corresponding CMakeLists files, which indicate the source files readings are the dominant source for control decisions. It is fixed involved in each subsystem, and in turn finds file groups that rep- by appending the inverse rotation matrices of invYZ, Rz, and Rx to resent the project’s subsystems. Details are elided. tf_camera_rotation after line 23, to change RDF to FLU. The type checker then takes patterns and file groups information, PhysFrame associates each state variable with a frame type that together with project files, and types variables and statements in abstracts both the displacements and orientation of the frame in the C/C++ source files. Type errors are reported when the program its parent frame. It also associates each transform variable with a cannot be properly typed. PhysFrame reports 7 kinds of type errors transform type that denotes the displacements and rotation needed and 2 kinds of type warnings as inconsistencies, and reports 3 in frame translation. It propagates such types following operation kinds of implicit convention violation warnings (see Section 4.2.3). semantics and checks consistency. In the example code, it associates Warnings may not lead to broken functionalities but problems in matrix Rx at line 10 with a transform type from the FLU orientation code maintenance and reuse. In the following, we will focus on 퐴 to the orientation 퐵. The type is derived from the constant matrix explaining the type system. values. It further associates matrix Rz at line 13 with a transform 4.2 Type System invYZ type from FLU to the orientation 퐶; and at line 17 from FLU Before introducing the formal language and type rules, we intu- to 퐷. By modeling the semantics of matrix multiplications at lines itively explain how frames and transforms are used. Typically, one- 20, 21, and 23, it determines that the transform at line 25 has the step transform objects are explicitly created to facilitate translation type from FLU to RDF. This yields a type error at lines 30-31 when of states in a child frame (e.g., position) to a parent frame. A trans- sendTransform() the transform is published by function because form specifies the relative position and orientations of the child ROS convention demands both camera frame and map frame to frame in the parent frame, denoted by displacements and rotations. have FLU, whereas the transform converts FLU to RDF. It also specifies the (string) ids for the child and parent frames. Anid Observe that it is difficult to get such transformation right dueto is the unique representation of a frame. The creations of transforms its intrinsic subtlety and complexity. While this is only a one-step also implicitly define frames (through the ids). They are implicit as transform, as mentioned earlier, a non-trivial robotic system has there are no explicit ROS data structures for frames. A state variable more than 80 different frames and transforms between any pair can be explicitly associated with some frame id, yielding a stamped may be necessary. The example bug is just one of the many kinds of (a ROS term) state variable. A transform can be explicitly applied to bugs PhysFrame detects. Others include missing frame and broken some stamped variable (in a child frame) to acquire its correspon- TF tree. More discussion can be found in Section 4.2.3. dence in the parent frame. One-step transforms can be published and hence become part of the global TF tree. As such, transforms 4 OUR APPROACH between arbitrary frames (e.g., those multiple steps apart) can be 4.1 Overview automatically constructed by traversing the tree (Section 2.2). To define a one-step transform, developers often explicitly specify the Figure4 presents an overview of PhysFrame. It takes as input a entailed rotation. A rotation can be composed from others. For C++ ROS project, and outputs frame inconsistencies and implicit example, a 60 degree rotation can be composed from two 30 degree convention violations. PhysFrame consists of three components: rotations. Plain state variables (not stamped) can be operated on implicit convention miner, file-processor, and type checker. just like regular C/C++ variables/data-structures. As such, even The miner takes a database of static transforms extracted from they (implicitly) belong to some frames, frame inconsistencies can- the launch files of a large repository of over 2200 mature ROS not be detected. Furthermore, ROS does not provide type checking projects from Github and performs data mining to infer implicit con- mechanism even for stamped variables. The C/C++ type system ventions. The miner identifies frequent static patterns that include cannot offer help either as it does not have any domain knowledge. pair of transforms that commonly co-exist in a project, transforms For example, it can type a variable to a ROS transform data struc- commonly having zero displacements, and transforms commonly ture but cannot identify the specific transform type, i.e., the parent having zero rotation. Since these patterns may happen coinciden- and child frames and the displacements and rotation involved. tally, the miner computes a frequency for each pattern to quantify its certainty (we use z-score [14] as a frequency measure) and re- 4.2.1 A Simplified Language and Types. ROS is based on C/C++ ports it if it is above a specified z-score threshold. The mining with a set of library functions. While our type system supports the ESEC/FSE ’21, August 23–27, 2021, Athens, Greece S. Kate, M. Chinn, H. Choi, X. Zhang, S. Elbaum complex syntax of C/C++/ROS and models the frame related APIs, ⟨푃푟표푔푟푎푚⟩ 푃 F 푆 we use a simplified language for discussion brevity. The language ⟨푆푡푎푡푒푚푒푛푡 ⟩ 푆 F 푆1;푆2 | 푥 := 푣 | 푥 := 푦 | 푥 := 푦 op 푧 | fx := stamped(푠, 푥) | fx.id := 푠 | focuses on modeling frame related semantics and ignores the stan- 푥 := get_data (fx) | fx := fy | fx :=external() | dard C/C++ types, operations and statements. It directly models fx := transform_to(푠, fy) | a number of ROS tf APIs as language primitives. Moreover, since tx := new_transform(푠푐ℎ푑 , 푠푝푛푡 , 푥, 푦, 푧, rx) | sendTransform (tx) | our analysis is flow insensitive, control structure like conditional tx := lookupTransform (푠푝푛푡 , 푠푐ℎ푑 ) | statements and loops are less relevant and hence not modeled. fx := apply_transform (tx, fy) | rx := 푣3×3 | Types. We define three types: (1) frame typeFrame ( ) that repre- rx := 푥 | rx := ry ⊗ rz | publish (푠푡표푝푖푐 , fx) <푆푡푎푡푒푉 푎푟> 푥, 푦, 푧 sents the frame information of a state variable, (2) transform type <푆푡푎푚푝푒푑푆푡푎푡푒푉 푎푟> fx, fy, fz (Transform) that represents frame transforms, and (3) rotation type <푇푟푎푛푠푓 표푟푚푉 푎푟> tx, ty, tz (푅표푡푎푡푖표푛) that represents changes in the axes’ directions. Note that <푅표푡푎푡푖표푛푉 푎푟> rx, ry, rz <퐶표푛푠푡푉푒푐푡표푟> 푣 these are not ROS types, but rather types in PhysFrame. <퐶표푛푠푡푆푡푟푖푛푔> 푠 Frame Type. This is an abstract type and always used with some concrete state type such as Point and Pose, indicating points and Figure 5: Language poses in a specific frame, respectively. For example, ROS uses tf::Stamped
Table 1: Type Rules Table 2: Type checking of example in Figure3
푦 : 휏 푦 : 휏 푧 : 휏 푅 푅 St# Statement Type Rule 1 푥 := 푦 : 휏 2 푥 := 푦 or 푧 : 휏 6 tf _cam_rotation := tmp ⊤ 푅13 = : = 휏.푖푑 푠 푥 휏 휏.푖푑 푠 10 Rx := [[1, 0, 0], [0, 0, −1], [0, 1, 0]] ⟨푥, −푧, 푦⟩ 푅12 푅3 푅4 fx := stamped(푠, 푥) : 휏 fx.id := 푠 : 휏 13 Rz := [[0, −1, 0], [1, 0, 0], [0, 0, 1]] ⟨−푦, 푥, 푧⟩ 푅12 fx : 휏 fy : 휏 20 tf _cam_rotation := Rx ⊗ tf _cam_rotation ⟨푥, −푧, 푦⟩ 푅15 푅 푅 21 tf _cam_rotation := Rz ⊗ tf _cam_rotation ⟨푧, 푥, 푦⟩ 푅 5 푥 := get_data(fx) : 휏 6 fx := fy : 휏 14 23 tf _cam_rotation := invYZ ⊗ ... ⟨푧, −푥, −푦⟩ 푅14 휏.푖푑 = 푠 reachable(휏.푖푑, 휏 .푖푑) fy : 휏 25 rtmp := tf _cam_rotation ⟨푧, −푥, −푦⟩ 푅 1 1 7 fx := transform_to(푠, fy) : 휏 error because rotate(⟨푓 , 푙,푢⟩, = ⟨ ( ) ( ) ( ) ⟩ 30 ttmp := new_transform(푠 , 푠 , ..., rtmp) 푅 rx : 휏푟 휏푐 푠푐ℎ푑 , 푠푝푛푡 , disp 푥 , disp 푦 , disp 푧 ,푑푥 ,푑푦,푑푧 cam map ⟨푧, −푥, −푦⟩) = 8 푠푐ℎ푑 ! = 푠푝푛푡 휏푝 .푖푑 = 푠푝푛푡 ⟨푟,푑, 푓 ⟩ ⟨푑푥 ,푑푦,푑푧 ⟩ = rotate(⟨휏푝 .푑푥 , 휏푝 .푑푦, 휏푝 .푑푧 ⟩, 휏푟 ) 푅8 the body frame. Note that in this case, the angular transformation tx := new_transform(푠푐ℎ푑 , 푠푝푛푡 , 푥, 푦, 푧, rx) : < 푠푐ℎ푑 , 푠푝푛푡 , disp(푥), disp(푦), disp(푧), 휏푟 > is independent of frame orientations. Both the camera and the body frames have orthogonal orientations. Rule 푅 types a rotation com- tx : 휏푡 tree_form_with_edge(휏푡 .푝푖푑, 휏푡 .푐푖푑) 14 푅9 () sendTransform(tx) : 휏푡 position statement. It leverages function compose to derive a new ′ ′ ′ reachable(푠푝푛푡 , 푠푐ℎ푑 ) 휏푡 .푝푖푑 = 푠푝푛푡 휏푡 .푐푖푑 = 푠푐ℎ푑 rotation type. In the function definition in Figure6, ⟨푑푠푥,푑푠푦,푑푠푧⟩ 푅10 ′′ ′′ ′′ tx := lookupTransform(푠푝푛푡 , 푠푐ℎ푑 ) : 휏푡 and ⟨푑푠푥 ,푑푠푦 ,푑푠푧 ⟩, both denote change in the orientation, where tx : 휏 fy : 휏 휏 .푝푖푑 = 휏′.푖푑 휏 .푐푖푑 = 휏.푖푑 the former change is followed by the latter. For example, Rz and Rx 푅 푡 푡 푡 11 fx := apply_transform(tx, fy) : 휏′ vectors in our motivating example (Figure3) represent ⟨−푦, 푥, 푧⟩ 휏푟 = orthogonalize(푣3×3) and ⟨푥, −푧,푦⟩ changes respectively as per rule 푅12. Part A in the be- 푅12 rx := 푣3×3 : 휏푟 low figure shows these orientation changes in a sequence; whereas 푅 13 rx := 푥 : ⊤ part B shows the orientation change that is computed by aggregat- ′ ′ ′ ′ ′′ ′′ ′′ ′′ 휏푟 = ⟨푑푠푥 ,푑푠푦,푑푠푧 ⟩ 휏푟 = ⟨푑푠푥 ,푑푠푦,푑푠푧 ⟩ ing these two changes. Note that both parts result into the same ′ ′′ ′ ′′ ry : 휏푟 rz : 휏푟 휏r = compose(휏푟 , 휏푟 ) orientation. 푅14 rx := ry ⊗ rz : 휏푟 ry : ⟨푑푠푥 ,푑푠푦,푑푠푧 ⟩ rz : ⊤ 푅15 rx := ry ⊗ rz : ⟨푑푠푥 ,푑푠푦,푑푠푧 ⟩ fx : 휏 푅16 publish(푠푡표푝푖푐, 푓 푥) : 휏 푅15 types a rotation composition in which one of the rotations is not static or not orthogonal. Note that an arbitrary angular rotation frames. Rule 푅 specifies that when typing a statement applying a 11 can be integrated with an orthogonal axes orientation rotation. transform tx to a stamped variable fy, the transform’s child frame Since we only model and check axes orientation, the resulted type must equal to fy’s frame, and its parent frame is the resulting frame. is the one that represents orthogonal orientation changes. 푅 types a statement that publishes a stamped variable. It re- 16 Example. Table2 presents part of the example code in Figure3 quires that the variable is properly typed. written in our language and the types computed by the type rules. Rotation-related Rules. Rule 푅 types the creation of a rotation from 12 The first column shows the original line numbers of the statements. a 3 × 3 constant vector. Function orthogonalize() is used to abstract St# 6 is an assignment of a rotation variable from a state variable. the vector to the corresponding DirSwitch abstract values. Its defini- The state variable tmp gets a value from the external value state tion can be found in Figure6. Specifically, an orthogonal 3×3 vector variable rotation. Therefore, as per rule 푅 , tf_cam_rotation is typed is composed of three orthogonal unit vectors, i.e., each row and 13 to ⊤. St# 10 and 13 are the assignments of rotation variables from each column has only one non-zero element of value 1 or -1. Now, constant vectors. Rule 푅 allows typing Rx and Rz to ⟨푥, −푧,푦⟩ and if we consider that rows 1,2,3 represent initial 푥,푦, 푧 orientation 12 ⟨−푦, 푥, 푧⟩, respectively. Further, St# 20 and 21 denote the composi- and columns 1,2,3 represent then changed 푥,푦, 푧 orientation, then tion of rotations: the former is typed with the type of Rx by 푅 as the column position of the non-zero element in a row indicates 15 the type of tf_cam_rotation is ⊤, whereas the latter is typed with the axis of the changed orientation that is the same as the axis of ⟨푧, 푥,푦⟩, which is computed by composing types of Rz and tf_cam_- the initial orientation represented by the row. For example, value rotation using rule 푅 2. St# 23 and 25 are similarly typed. Finally, -1 in row 2 and column 1 denotes that the direction of 푦 axis of 14 St# 30 constructs a new transform object. However, it cannot be initial orientation becomes the direction of −푥 axis of the changed properly typed by rule 푅 as applying rotation ⟨푧, −푥, −푦⟩ to the par- orientation. When the matrix does not denote orthogonal rotation, 8 ent map frame with the conventional orientation ⟨forward, left, up⟩, the resulted type is ⊤. i.e, FLU, yields the orientation ⟨right, down, forward⟩, contradicting Rule 푅 types a rotation creation from a variable (i.e., non- 13 with the ROS orientation convention FLU for the camera frame. On constant). It usually corresponds to angular transformation that is the other hand, PhysFrame correctly type-checks the fixed version. needed for state translation across frames, but not for orientation Soundness and Completeness. The essence of PhysFrame is to changes. For example, if a camera can spin regarding the body, its leverage flow-insensitive static analysis (like type checking) to readings (regarding the camera frame) have to go through a dy- namic angular transformation in order to be interpreted regarding 2PhysFrame uses SSA to handle variable re-definitions. PhysFrame: Type Checking Physical Frames of Reference for Robotic Systems ESEC/FSE ’21, August 23–27, 2021, Athens, Greece find frame related bugs. It is automated and does not require any Implicit Convention Violations. As mentioned in Section 2.3, user type annotations for subject ROS projects. As such, the type implicit conventions are mined from launch files. As such, they may system is neither sound nor complete. This is because some type not be required but rather commonly seen. PhysFrame reports three rules are uncertain. For example in rule 푅12, a matrix representing kinds of warnings. The first is null displacement expected warning, orthogonal angular transformation may not be intended for frame meaning that certain transforms or frames are supposed to have axes orientation changes, but rather because the parent and child null displacements. The second is null rotation expected warning, body parts are installed in a way that they form some orthogonal meaning that certain transforms are supposed to have null rotations. angle(s), while their frame axes orientations are identical (both The third is that co-occurrence violation warnings when frames that FLU). However, since there is no way for PhysFrame to know the are commonly seen together (in other projects) are not present real meanings of such rotation without user input, it considers this in a project. Such checkings are performed as an additional step an axes orientation rotation, which is the most common case. And after frame and transform types are inferred. The reason is that it is stamped variables from third party libraries are widely used and hard to implement implicit conventions as type rules as the mined PhysFrame may not be able to type them. Therefore, PhysFrame conventions may change depending on the repositories used and has both false positives and false negatives. On the other hand, the z-score threshold setting. we foresee future robotic system development frameworks may support full-fledged frame types which the developer can useto 5 EVALUATION declare their variables. With such explicit type declarations, a sound We address the following questions: RQ1. How effective and ef- and complete type system can be developed. ficient is PhysFrame in disclosing frame related inconsistencies? RQ2. What is the response from developers on the inconsistencies 4.2.3 Frame Related Problems. PhysFrame reports 7 kinds of reported by PhysFrame? RQ3. What kinds of implicit conventions type errors, 2 kinds of type warnings and 3 kinds of violations of are violated and what is the impact of z-score on results? implicit conventions. The first two are also called inconsistencies. Type Errors. This first kind is TF tree related errors. These errors include a frame having multiple parent frames, frames forming a 5.1 Experiment Setup cycle, and frames not following conventional orders such as a map Implementation. We have implemented a prototype of Phys- frame must be the ancestor of a body frame (Section 2.3). Frame in around 4800 lines of Python code. It utilizes a third-party The second kind of type errors is incorrect frames or transforms. component, cppcheck [26], to obtain an XML dump for a C/C++ file ROS allows developers to stamp a state variable 푥 with an empty providing an intermediate representation. It contains program to- id. If these variables are published, meaning that other compo- kens, symbol tables for scopes, functions and variables, and an AST nents from different parties may receive such data, they should for each statement. We have also implemented constant propagation be stamped with a proper frame (rule 푅16). Moreover, if they are to resolve variables to static values when possible. The miner is im- used in transforms, runtime errors will be triggered as well. In plemented using MySQL database and SQL script. Implementation these cases, PhysFrame reports a missing frame error. Although details can be found in Appendix C . The database for the implicit not explicitly modeled by our rules, a few ROS data types require convention miner is built from a list of ROS projects that contain at explicitly specifying child frame ids. If child frames are not set, least one launch file with a static_transform_publisher. We use only PhysFrame reports missing child frame errors. the mature projects from this list. We consider a project mature A transform is required to have different parent and child frames when it has more than 30 commits. The database contains 12.1K (rule 푅8). Otherwise, it is meaningless. PhysFrame reports a re- static transforms in 5.3K launch files from 2.2K mature projects. The dundant transform. Rule 푅8 requires that the displacements and tool and data is available at https://doi.org/10.5281/zenodo.4959920 rotation in a newly created transform should yield (relative) posi- Artifacts. We run the type checker on 180 ROS projects from tions and axes orientations consistent with the parent frame and Github. They include (a) 7 projects from the ROS Kinetic distri- ROS conventions (e.g., body frames should have forward-left-up bution that contain at least one issue or a commit related to frame; (FLU) orientations). Otherwise, it reports incorrect transforms. (b) 23 randomly selected projects from the ROS indigo distribution; Type Warnings. A sensor transform that defines a sensor frame and (c) 150 randomly selected ROS projects that contain at least with respect to some other robotic body part like base_link, wrist, one static transform. Among these projects, 69 are mature and 99 etc. or vice versa should have at least some non-zero displace- not (with less than 30 commits). Note that we consider evaluation ment(s)/rotation(s) as a sensor’s center unlikely aligns with a body on unmature projects important as well because they are likely by part’s center. PhysFrame reports a warning when a sensor trans- non-expert ROS developers that can benefit more from PhysFrame. form has zero displacements and rotations. For a transform from Table 7 in Appendix shows the statistics of these projects. On aver- parent to child, ROS naming conventions follow templates ‘
Table 3: Inconsistencies by Category (with the first seven type errors, the last two type warnings)
Category Number of Inconsistencies Number of Repositories Total (#) TP (#) FP (#) Total (#) TP (#) FP (#) C/C++ Launch C/C++ Launch c_MULTIPLE_PARENTS_IN_FRAME_TREE 4 3 1 0 0 4 4 0 c_CYCLE_IN_FRAME_TREE 1 1 0 0 0 1 1 0 td_INCORRECT_FRAME_ORDER_IN_TREE 7 3 4 0 0 7 7 0 c_INCORRECT_TRANSFORM 17 4 4 0 9 9 3 6 td_REDUNDANT_TRANSFORM 2 2 - 0 - 2 2 0 td_MISSING_FRAME 108 92 - 16 - 56 49 13 td_MISSING_CHILD_FRAME 12 8 - 4 - 7 4 3 td_REVERSED_NAME (type warning) 33 1 28 1 3 19 17 4 c_SENSOR_NULL (type warning) 6 0 3 0 3 5 2 3 Total 190 114 40 21 15 110 89 29 [81.05%] [18.95%]
Table 4: Summary of True Positive Inconsistencies Table 5: Implicit Convention Warnings by Category Category z=1 z=2 z=5 z=10 Report & Response Count Inconsistency Categories W# R# W# R# W# R# W# R# found by searching 3 td_MISSING_FRAME commit-history or issue 1 c_INCORRECT_TRANSFORM w_sig_null_rot_exp 18 9 16 7 14 5 9 2 reports of repositories. w_name_null_rot_exp 20 10 20 10 2 2 0 0 fixed by developers 1 td_MISSING_FRAME w_name_co-occurrence 7 3 0 0 0 0 0 0 before we reported 1 td_REDUNDANT_TRANSFORM them. 1 c_CYCLE_IN_FRAME_TREE developers. The implicit convention violation warnings are by their 1 c_MULTIPLE_PARENTS_IN_FRAME_TREE nature more difficult to determine if they denote real bugs. We reported, and either 2 td_MISSING_FRAME hence run the tool on the same set of projects with different z-score fixed by developers or 1 td_MISSING_CHILD_FRAME ready to accept a 1 td_REDUNDANT_TRANSFORM threshold values and study the reported warnings. All type check- pull-request. 1 td_INCORRECT_FRAME_ORDER_IN_TREE ing runs are on a Dell PowerEdge R420 Linux server with two Intel reported, and 1 td_MISSING_FRAME Xeon E5-2690 2.90GHz 8-core processors and 192 GB of RAM. acknowledged by 5 td_REVERSED_NAME developers (in some 2 c_SENSOR_NULL cases, confirmed as a 2 td_INCORRECT_FRAME_ORDER_IN_TREE 5.2 Results and Observations useful fix to others for RQ1. Effectiveness and Efficiency. PhysFrame reports 190 in- reusing the package node). consistencies with 154 true positives and 81.05% true positive rate. reported, but refused 3 td_MISSING_FRAME These inconsistencies belong to 100 projects, over 55% of the total by developers on the we examined. This number is specially significant considering that grounds that reuse of many more frame related bugs reported in earlier versions of the the package node is not easy, or the package is systems may have been detected by PhysFrame. PhysFrame also no longer maintained. reports 45, 36, 16, 9 implicit convention violation warnings when reported, but no 12 td_MISSING_FRAME the z-score threshold is set to 1, 2, 5, 10, respectively, which cor- response from 1 td_MISSING_CHILD_FRAME responds to p-values smaller than 0.15 for z=1 to p-values smaller developers yet. 4 c_INCORRECT_TRANSFORM 14 td_REVERSED_NAME than 0.00001 for z=10. The average analysis time is 52 seconds with 1 td_INCORRECT_FRAME_ORDER_IN_TREE the maximum 1190 seconds. Further details can be found in Table 6 2 c_MULTIPLE_PARENTS_IN_FRAME_TREE in Appendix . found in a package 2 td_MISSING_FRAME Inconsistencies by Category. Table3 summarizes inconsistencies by node that is reused by 4 td_MISSING_CHILD_FRAME other developer. The their category, with the first seven type errors and the last two type issue is either fixed or warnings. The first column lists the categories. Each has either푐 a‘ _’ acknowledged by other users in the original or ‘푡푑_’ prefix, denoting whether it is a “correctness” or a “technical repository of the node. debt” issue, respectively. The “correctness” issues represent the found in backup-copy 5 td_MISSING_FRAME types of inconsistencies that violate ROS conventions such as a tree files in repositories. representation of all the frames in an application, sensor frames should have non-null transforms with respect to robot body frames, potential issues. 8 td_MISSING_FRAME and incorrect transforms. The incorrect transform issues include projects not active in 55 td_MISSING_FRAME those that have invalid displacement/rotation (like our motivating past two years. 2 td_MISSING_CHILD_FRAME 3 c_INCORRECT_TRANSFORM example in Section3) which can lead to problems during runtime, 10 td_REVERSED_NAME and those that do not follow ROS naming conventions (discussed 1 c_SENSOR_NULL 3 td_INCORRECT_FRAME_ORDER_IN_TREE in Section 2.2) while defining a child frame in the transform which 1 c_MULTIPLE_PARENTS_IN_FRAME_TREE affect code readability and may cause issues during code reuse. The “technical debt” issues represent the types of inconsistencies that PhysFrame: Type Checking Physical Frames of Reference for Robotic Systems ESEC/FSE ’21, August 23–27, 2021, Athens, Greece may make code maintenance and understanding difficult and may support. As such, respecting conventions is critical for these infras- lead to potential runtime problems during code reuse/collaboration. tructures. Unfortunately, they provide no compile time support to The next five columns show the total number of inconsistencies for ensure frame conventions, one of the most error-prone aspects of each category, TP cases in C/C++ code and in launch files, FP cases robot programming. PhysFrame is a type checking technique that in C/C++ code and in launch files, respectively. Further, the last systematically detects problems in using frames. three columns show the number of projects with inconsistencies, Type Checking in Robotic Systems. Type checking [11, 33] is a TPs, and FPs, respectively. well-known technique to improve software reliability. In robotic Many of the reported inconsistencies are about missing frames, software, due to its domain-specific characteristics, variables of- i.e., developers forgot to set frame ids for published state values. ten have physical-world semantics. Additional type checking can FP cases of this type are observed due to usage of stamped data hence be performed considering the physical world. Unit type types for storing non-frame data (e.g., binary mask of image), usage checking [7, 13] is the most common robotic software type system, of stamped transform for tf operations that do not need frame_id which type variables with physical unit information. The seminal field value, and passing of stamped data type variables to functions work [21, 22, 34] in the type checking of units and dimensions ex- whose signature is not available during analysis. Similar sources of tends a programming language. Ayudante [19] builds and compares false positives affect other categories. PhysFrame also found many clusters of variables based on dataflow and the meaning of vari- reversed name warnings. For example, ROS expects transform vari- ables names to detect type (e.g., unit) inconsistencies. The recent ables to follow the naming convention of “⟨parent⟩_to_⟨child⟩”. work on Phriky [30, 31] supports dimensionality type checks in We find that many developers use the names reversed. This may ROS without developer annotations, and Phys [20] extends it with cause maintenance and reuse issues as the convention is expected probabilistic type inference to detect more inconsistencies. In this by other parties. PhysFrame also identified 17 frame type incon- work we extend the type checking to reference frames in robotic sistencies similar to our motivating example in Section3 and 12 software, which requires a distinct type system. TF tree errors. Given that ROS is built to encourage collaborative Frame Safety. Lowry et al. [23] proposed a general abstract in- development, even the issues affecting code readability or reuse terpretation system for certifying domain-specific properties and can be important. We have three case studies in Appendix E . provide a case study to certify NAIF library programs’ frame-safety. RQ2. Developer Responses. Table4 summarizes the distribution The case study checks limited properties (e.g., frame consistencies) of the true positives across different reporting actions, developer and the system needs annotations for program inputs. In contrast, responses and inconsistency categories. Row 1 (below the table ours is a fully automated tool checking many properties for ROS title) shows that 4 of the 154 true positives PhysFrame uncovered projects (e.g., TF tree shape and frame orientations), without re- were known issues by the developers. Row 2 shows that another quiring annotations. Our abstract domains are hence different from four were fixed by the time we reached the developers. Among the theirs and our method is type system based. The work by Basir et remaining true positives, we reported the 52 that belong to projects al. [6] on automated theorem proving for automatically generated that showed activity in the past 2 years (rows 3-6). We received code also presented a case study to verify two frame-safety require- 18 responses with 15 either acknowledging this issue or fixing it, ments. META-AMPHION system [24] helps domain experts build and 3 declining our reports because either it is not seen as relevant domain-specific program synthesizers. In the context of frame- (project is not intended for external use hence conventions do not safety, we believe that this work would enable the synthesis of need to be respected), or project no longer maintained. Row 8 shows programs that are frame-safe given the correct domain theory. In that PhysFrame found 6 bugs in reused versions of projects that our work, we focus on finding frame issues in the already devel- have already been fixed in the original repositories. oped code. Overall we do not need annotations, have a push-button RQ3. Implicit Convention Violations. Table5 shows the three approach, and provide a broad assessment of real code. kinds of implicit convention violations PhysFrame has found and the impact of z-score threshold. W# means the number of warnings 7 CONCLUSION and R# the number of projects with warnings. The most popu- We develop a fully automated type checking technique for reference lar kind is w_name_null_rot_exp, meaning that a transform with frames in robotic system, which is one of the most subtle and a specific name is expected to have 0 rotation; w_sig_null_rot_- error-prone aspects in robotic software engineering. We explain the exp is similar except that a transform is identified by parent and semantics of frames and frame transformations, and propose a novel child frame ids instead of name. Some of the w_sig_null_rot_exp way to abstract them. Type checking rules are developed based on warnings are persistent (with different z-threshold), meaning that the domain specific semantics. Applying our technique to 180 ROS conventions adopted by almost all projects are being violated. projects from GitHub discloses over 190 errors and warnings, with 154 true positives. We reported 52 of them and received 18 responses 6 RELATED WORK by the time of submission and 15 fixed/acknowledged. Component-based Robot Software. Component-based program- ming [8, 9] has been extensively used in robot development, aim- ACKNOWLEDGMENTS ing to facilitate reuse. Following the principle, several robot soft- We thank our insightful reviewers. This research was supported, in ware frameworks [25], such as Open Robot Control Software (Oro- part by NSF 1901242, 1910300, 1909414 and 1853374, ONR N0001417- cos) [10], ROS [32]. Player [18], Miro [37], have been developed and 12045, N000141410468 and N000141712947. Any opinions, findings, widely used. These frameworks commonly provide communica- and conclusions in this paper are those of the authors only and do tion infrastructure across components [17] and substantial library not necessarily reflect the views of our sponsors. ESEC/FSE ’21, August 23–27, 2021, Athens, Greece S. Kate, M. Chinn, H. Choi, X. Zhang, S. Elbaum
REFERENCES [21] Andrew Kennedy. 1994. Dimension Types. In Proceedings of the 5th European [1] 2015. diff_drive_controller is hard-coded into the ’odom’ reference frame. #204 Symposium on Programming: Programming Languages and Systems (ESOP ’94). https://github.com/ros-controls/ros_controllers/issues/204 Springer-Verlag, Berlin, Heidelberg, 348–362. [2] 2017. TF: Debugging Tools. http://wiki.ros.org/tf/Debuggingtools [22] Andrew Kennedy. 2009. Types for Units-of-Measure: Theory and Practice. In [3] 2019. ORB-SLAM2 ROS node. http://wiki.ros.org/orb_slam2_ros Proceedings of the Third Summer School Conference on Central European Functional [4] 2019. T265 frame questions #772. https://github.com/IntelRealSense/realsense- Programming School (Budapest, Hungary) (CEFP’09). Springer-Verlag, Berlin, ros/issues/772 Heidelberg, 268–305. [5] 2020. Incorrect frame tree cartographer ros and t265 camera #1599. https://github. [23] Michael Lowry, Thomas Pressburger, and Grigore Rosu. 2001. Certifying Domain- com/IntelRealSense/realsense-ros/issues/1599 Specific Policies. In Proceedings of the 16th IEEE International Conference on [6] Nurlida Basir, Ewen Denney, and Bernd Fischer. 2010. Deriving Safety Cases for Automated Software Engineering (ASE ’01). IEEE Computer Society, USA, 81. Hierarchical Structure in Model-Based Development. In Proceedings of the 29th [24] Michael R. Lowry and Jeffrey Van Baalen. 1997. META-AMPHION: Synthesis of International Conference on Computer Safety, Reliability, and Security (Vienna, Efficient Domain-Specific Program Synthesis Systems. Automated Software Engg. Austria) (SAFECOMP’10). Springer-Verlag, Berlin, Heidelberg, 68–81. 4, 2 (April 1997), 199–241. https://doi.org/10.1023/A:1008637201658 [7] Geoffrey Biggs. 2007. Designing an application-specific programming language [25] Gergely Magyar, Peter Sinčák, and Zoltán Krizsán. 2015. Comparison study of for mobile robots. Ph.D. Dissertation. ResearchSpace@ Auckland. robotic middleware for robotic applications. In Emergent Trends in Robotics and [8] Alex Brooks, Tobias Kaupp, Alexei Makarenko, Stefan Williams, and Anders Intelligent Systems. Springer, 121–128. Oreback. 2005. Towards component-based robotics. In 2005 IEEE/RSJ International [26] Daniel Marjamaeki. 2017. Cppcheck - A tool for static C/C++ code analysis. http: Conference on Intelligent Robots and Systems. IEEE, 163–168. //cppcheck.sourceforge.net/ [9] Davide Brugali and Patrizia Scandurra. 2009. Component-based robotic engi- [27] Wim Meeussen. 2010. Coordinate Frames for Mobile Platforms. https://www.ros. neering (part i)[tutorial]. IEEE Robotics & Automation Magazine 16, 4 (2009), org/reps/rep-0105.html 84–96. [28] Thomas Moulard. 2011. Coordinate Frames for Humanoid Robots. https://www. [10] Herman Bruyninckx. 2001. Open robot control software: the OROCOS project. In ros.org/reps/rep-0120.html Proceedings 2001 ICRA. IEEE international conference on robotics and automation [29] Raúl Mur-Artal and Juan D. Tardós. 2017. ORB-SLAM2: an Open-Source SLAM (Cat. No. 01CH37164), Vol. 3. IEEE, 2523–2528. System for Monocular, Stereo and RGB-D Cameras. IEEE Transactions on Robotics [11] Luca Cardelli. 1996. Type systems. ACM Computing Surveys (CSUR) 28, 1 (1996), 33, 5 (2017), 1255–1262. https://doi.org/10.1109/TRO.2017.2705103 [30] John-Paul Ore, Carrick Detweiler, and Sebastian Elbaum. 2017. Lightweight 263–264. Detection of Physical Unit Inconsistencies Without Program Annotations. In [12] crigroup. 2017. cubes_task.launch. https://git.io/JtDhp Proceedings of the 26th ACM SIGSOFT International Symposium on Software Testing [13] Kostadin Damevski. 2009. Expressing measurement units in interfaces for sci- and Analysis (Santa Barbara, CA, USA) (ISSTA 2017). ACM, New York, NY, USA, entific component software. In Proceedings of the 2009 Workshop on Component- 341–351. https://doi.org/10.1145/3092703.3092722 Based High Performance Computing. 1–8. [31] John-Paul Ore, Carrick Detweiler, and Sebastian Elbaum. 2017. Phriky-Units: [14] Dawson Engler, David Yu Chen, Seth Hallem, Andy Chou, and Benjamin Chelf. A Lightweight, Annotation-Free Physical Unit Inconsistency Detection Tool. 2001. Bugs as Deviant Behavior: A General Approach to Inferring Errors in In Proceedings of the 26th ACM SIGSOFT International Symposium on Software Systems Code. In Proceedings of the Eighteenth ACM Symposium on Operating Testing and Analysis (Santa Barbara, CA, USA) (ISSTA 2017). Association for Systems Principles (Banff, Alberta, Canada) (SOSP ’01). Association for Computing Computing Machinery, New York, NY, USA, 352–355. https://doi.org/10.1145/ Machinery, New York, NY, USA, 57–72. https://doi.org/10.1145/502034.502041 3092703.3098219 [15] Tully Foote. 2013. tf: The transform library. In IEEE International Conference on [32] Morgan Quigley, Ken Conley, Brian Gerkey, Josh Faust, Tully Foote, Jeremy Leibs, Technologies for Practical Robot Applications (TePRA), 2013 (Open-Source Software Rob Wheeler, and Andrew Y Ng. 2009. ROS: an open-source Robot Operating workshop). 1–6. https://doi.org/10.1109/TePRA.2013.6556373 System. In ICRA workshop on open source software, Vol. 3.2. Kobe, Japan, 5. [16] Tully Foote and Mike Purvis. 2010. Standard Units of Measure and Coordinate [33] John C Reynolds. 1974. Towards a theory of type structure. In Programming Conventions. https://www.ros.org/reps/rep-0103.html Symposium. Springer, 408–425. [17] Open Source Robotics Foundation. [n.d.]. ROS Topics homepage. http://wiki.ros. [34] Mikael Rittri. 1995. Dimension Inference under Polymorphic Recursion. In org/Topics Proceedings of the Seventh International Conference on Functional Programming [18] Brian Gerkey, Richard T Vaughan, and Andrew Howard. 2003. The player/stage Languages and Computer Architecture (La Jolla, California, USA) (FPCA ’95). project: Tools for multi-robot and distributed sensor systems. In Proceedings of Association for Computing Machinery, New York, NY, USA, 147–159. https: the 11th international conference on advanced robotics, Vol. 1. 317–323. //doi.org/10.1145/224164.224197 [19] Irfan Ul Haq, Juan Caballero, and Michael D Ernst. 2015. Ayudante: Identifying [35] Clearpath Robotics. 2017. Robots - PR2. http://wiki.ros.org/Robots/PR2 undesired variable interactions. In Proceedings of the 13th International Workshop [36] Wim Meeussen Tully Foote, Eitan Marder-Eppstein. [n.d.]. Ros tf package home- on Dynamic Analysis. 8–13. page. http://wiki.ros.org/tf,http://wiki.ros.org/tf2 [20] Sayali Kate, John-Paul Ore, Xiangyu Zhang, Sebastian Elbaum, and Zhaogui Xu. [37] Hans Utz, Stefan Sablatnog, Stefan Enderle, and Gerhard Kraetzschmar. 2002. 2018. Phys: probabilistic physical unit assignment and inconsistency detection. In Miro-middleware for mobile robot applications. IEEE Transactions on Robotics Proceedings of the 2018 26th ACM Joint Meeting on European Software Engineering and Automation 18, 4 (2002), 493–497. Conference and Symposium on the Foundations of Software Engineering. 563–573. PhysFrame: Type Checking Physical Frames of Reference for Robotic Systems ESEC/FSE ’21, August 23–27, 2021, Athens, Greece
APPENDIX likelihood that a conclusion is satisfied. We use 푝0 = 0.9 following the literature. Note that 푧 grows as 푛 increases and 푛 − 푒 decreases.
base_footprint
xyz: 0 0 0.051 rpy: 0 -0 0
base_footprint_joint
base_link
xyz: -0.29 0 0.8 xyz: 0.275 0 0.252 xyz: -0.2246 0.2246 0.0282 xyz: -0.2246 -0.2246 0.0282 xyz: 0.2246 0.2246 0.0282 xyz: 0.2246 -0.2246 0.0282 xyz: -0.05 0 0.739675 xyz: -0.15 0 0.7 rpy: 0 -0 0 rpy: 0 -0 0 rpy: 0 -0 0 rpy: 0 -0 0 rpy: 0 -0 0 rpy: 0 -0 0 rpy: 0 -0 0 rpy: 0 -0 0
base_bellow_joint base_laser_joint bl_caster_rotation_joint br_caster_rotation_joint fl_caster_rotation_joint fr_caster_rotation_joint torso_lift_joint torso_lift_motor_screw_joint
base_bellow_link base_laser_link bl_caster_rotation_link br_caster_rotation_link fl_caster_rotation_link fr_caster_rotation_link torso_lift_link torso_lift_motor_screw_link
xyz: 0 0.049 0 xyz: 0 -0.049 0 xyz: 0 0.049 0 xyz: 0 -0.049 0 xyz: 0 0.049 0 xyz: 0 -0.049 0 xyz: 0 0.049 0 xyz: 0 -0.049 0 xyz: -0.01707 0 0.38145 xyz: -0.02977 -0.1497 0.164 xyz: 0 0.188 0 xyz: 0.0535 0.209285 0.176625 xyz: 0.09893 0 0.227 xyz: 0 -0.188 0 xyz: 0.0535 -0.209285 0.176625 rpy: 0 -0 0 rpy: 0 -0 0 rpy: 0 -0 0 rpy: 0 -0 0 rpy: 0 -0 0 rpy: 0 -0 0 rpy: 0 -0 0 rpy: 0 -0 0 rpy: 0 -0 0 rpy: 3.14159 -2.06823e-13 3.14159 rpy: 0 -0 0 rpy: 0 -0 0 rpy: 0 -0 0 rpy: 0 -0 0 rpy: 0 -0 0 C PROTOTYPE IMPLEMENTATION
bl_caster_l_wheel_joint bl_caster_r_wheel_joint br_caster_l_wheel_joint br_caster_r_wheel_joint fl_caster_l_wheel_joint fl_caster_r_wheel_joint fr_caster_l_wheel_joint fr_caster_r_wheel_joint head_pan_joint imu_joint l_shoulder_pan_joint l_torso_lift_side_plate_joint laser_tilt_mount_joint r_shoulder_pan_joint r_torso_lift_side_plate_joint
bl_caster_l_wheel_link bl_caster_r_wheel_link br_caster_l_wheel_link br_caster_r_wheel_link fl_caster_l_wheel_link fl_caster_r_wheel_link fr_caster_l_wheel_link fr_caster_r_wheel_link head_pan_link imu_link l_shoulder_pan_link l_torso_lift_side_plate_link laser_tilt_mount_link r_shoulder_pan_link r_torso_lift_side_plate_link
xyz: 0.068 0 0 xyz: 0.1 0 0 xyz: 0 0 0.03 xyz: 0.1 0 0 rpy: 0 -0 0 rpy: 0 -0 0 rpy: 0 -0 0 rpy: 0 -0 0 head_tilt_joint l_shoulder_lift_joint laser_tilt_joint r_shoulder_lift_joint Prototype implementation of PhysFrame contains around 4800
head_tilt_link l_shoulder_lift_link laser_tilt_link r_shoulder_lift_link
xyz: 0.0232 0 0.0645 xyz: 0 0 0 xyz: 0 0 0 rpy: 0 -0 0 rpy: 0 -0 0 rpy: 0 -0 0 head_plate_frame_joint l_upper_arm_roll_joint r_upper_arm_roll_joint lines of python code and 160 lines of SQL script. It can be invoked head_plate_frame l_upper_arm_roll_link r_upper_arm_roll_link
xyz: 0 0.11 0.0546 xyz: 0 0 0 xyz: 0 0 0 xyz: 0 0 0 rpy: 0 -0 0 rpy: 0 -0 0 rpy: 0 -0 0 rpy: 0 -0 0
projector_wg6802418_frame_joint sensor_mount_frame_joint l_upper_arm_joint r_upper_arm_joint
projector_wg6802418_frame sensor_mount_link l_upper_arm_link r_upper_arm_link on command line with the project path input.
xyz: 0 0 0 xyz: 0 0 0 xyz: 0.046457 -0.11 0.0546 xyz: 0.4 0 0 xyz: 0.4 0 0 rpy: 0 -1.5708 0 rpy: 0 -0 0 rpy: 0 -0 0 rpy: 0 -0 0 rpy: 0 -0 0
projector_wg6802418_child_frame_joint double_stereo_frame_joint high_def_frame_joint l_elbow_flex_joint r_elbow_flex_joint
projector_wg6802418_child_frame double_stereo_link high_def_frame l_elbow_flex_link r_elbow_flex_link The implicit convention miner uses MySQL to store the static xyz: 0.045 0.06 0.0501 xyz: 0.045 0.03 0.0501 xyz: 0 0 0 xyz: 0 0 0 xyz: 0 0 0 rpy: 0 -0 0 rpy: 0 -0 0 rpy: -1.5708 -5.55112e-17 -1.5708 rpy: 0 -0 0 rpy: 0 -0 0
narrow_stereo_frame_joint wide_stereo_frame_joint high_def_optical_frame_joint l_forearm_roll_joint r_forearm_roll_joint
narrow_stereo_link wide_stereo_link high_def_optical_frame l_forearm_roll_link r_forearm_roll_link
xyz: 0 0 0 xyz: 0 0 0 xyz: 0 0 0 xyz: 0 0 0 xyz: 0.135 0 0.044 xyz: 0 0 0 xyz: 0.135 0 0.044 xyz: 0 0 0 transforms (present as static_transform_publisher nodes in launch rpy: 0 -0 0 rpy: -1.5708 -5.55112e-17 -1.5708 rpy: 0 -0 0 rpy: -1.5708 -5.55112e-17 -1.5708 rpy: -1.5708 -0.562869 0 rpy: 0 -0 0 rpy: 1.5708 -0.562869 0 rpy: 0 -0 0
narrow_stereo_l_stereo_camera_frame_joint narrow_stereo_optical_frame_joint wide_stereo_l_stereo_camera_frame_joint wide_stereo_optical_frame_joint l_forearm_cam_frame_joint l_forearm_joint r_forearm_cam_frame_joint r_forearm_joint
narrow_stereo_l_stereo_camera_frame narrow_stereo_optical_frame wide_stereo_l_stereo_camera_frame wide_stereo_optical_frame l_forearm_cam_frame l_forearm_link r_forearm_cam_frame r_forearm_link
xyz: 0 0 0 xyz: 0 -0.09 0 xyz: 0 0 0 xyz: 0 -0.09 0 xyz: 0 0 0 xyz: 0.321 0 0 xyz: 0 0 0 xyz: 0.321 0 0 rpy: -1.5708 -5.55112e-17 -1.5708 rpy: 0 -0 0 rpy: -1.5708 -5.55112e-17 -1.5708 rpy: 0 -0 0 rpy: -1.5708 -5.55112e-17 -1.5708 rpy: 0 -0 0 rpy: -1.5708 -5.55112e-17 -1.5708 rpy: 0 -0 0 files) extracted from a list of 6K projects. The list was generated
narrow_stereo_l_stereo_camera_optical_frame_joint narrow_stereo_r_stereo_camera_frame_joint wide_stereo_l_stereo_camera_optical_frame_joint wide_stereo_r_stereo_camera_frame_joint l_forearm_cam_optical_frame_joint l_wrist_flex_joint r_forearm_cam_optical_frame_joint r_wrist_flex_joint
narrow_stereo_l_stereo_camera_optical_frame narrow_stereo_r_stereo_camera_frame wide_stereo_l_stereo_camera_optical_frame wide_stereo_r_stereo_camera_frame l_forearm_cam_optical_frame l_wrist_flex_link r_forearm_cam_optical_frame r_wrist_flex_link
xyz: 0 0 0 xyz: 0 0 0 xyz: 0 0 0 xyz: 0 0 0 rpy: -1.5708 -5.55112e-17 -1.5708 rpy: -1.5708 -5.55112e-17 -1.5708 rpy: 0 -0 0 rpy: 0 -0 0 using the GitHub code search API which followed: downloading of narrow_stereo_r_stereo_camera_optical_frame_joint wide_stereo_r_stereo_camera_optical_frame_joint l_wrist_roll_joint r_wrist_roll_joint
narrow_stereo_r_stereo_camera_optical_frame wide_stereo_r_stereo_camera_optical_frame l_wrist_roll_link r_wrist_roll_link
xyz: 0 0 0 xyz: 0 0 0 rpy: 0 -0 0 rpy: 0 -0 0
l_gripper_palm_joint r_gripper_palm_joint 55K launch files that contained a static_transform_publisher from
l_gripper_palm_link r_gripper_palm_link
xyz: 0.07691 0.01 0 xyz: 0.0513 0 0.0244 xyz: 0 0 0 xyz: 0.16828 0 0 xyz: 0.07691 -0.01 0 xyz: 0.18 0 0 xyz: 0.07691 0.01 0 xyz: 0.0513 0 0.0244 xyz: 0 0 0 xyz: 0.16828 0 0 xyz: 0.07691 -0.01 0 xyz: 0.18 0 0 rpy: 0 -0 0 rpy: 0 -0 0 rpy: 0 -0 0 rpy: 0 -0 0 rpy: 0 -0 0 rpy: 0 -0 0 rpy: 0 -0 0 rpy: 0 -0 0 rpy: 0 -0 0 rpy: 0 -0 0 rpy: 0 -0 0 rpy: 0 -0 0 l_gripper_l_finger_joint l_gripper_led_joint l_gripper_motor_accelerometer_joint l_gripper_motor_slider_joint l_gripper_r_finger_joint l_gripper_tool_joint r_gripper_l_finger_joint r_gripper_led_joint r_gripper_motor_accelerometer_joint r_gripper_motor_slider_joint r_gripper_r_finger_joint r_gripper_tool_joint public GitHub repositories; downloading of an original version of
l_gripper_l_finger_link l_gripper_led_frame l_gripper_motor_accelerometer_link l_gripper_motor_slider_link l_gripper_r_finger_link l_gripper_tool_frame r_gripper_l_finger_link r_gripper_led_frame r_gripper_motor_accelerometer_link r_gripper_motor_slider_link r_gripper_r_finger_link r_gripper_tool_frame
xyz: 0.09137 0.00495 0 xyz: 0 0 0 xyz: 0.09137 -0.00495 0 xyz: 0.09137 0.00495 0 xyz: 0 0 0 xyz: 0.09137 -0.00495 0 rpy: 0 -0 0 rpy: 0 -0 0 rpy: 0 -0 0 rpy: 0 -0 0 rpy: 0 -0 0 rpy: 0 -0 0 l_gripper_l_finger_tip_joint l_gripper_motor_screw_joint l_gripper_r_finger_tip_joint r_gripper_l_finger_tip_joint r_gripper_motor_screw_joint r_gripper_r_finger_tip_joint each file that was found to be modified in the commit historyof l_gripper_l_finger_tip_link l_gripper_motor_screw_link l_gripper_r_finger_tip_link r_gripper_l_finger_tip_link r_gripper_motor_screw_link r_gripper_r_finger_tip_link
xyz: 0 0 0 xyz: 0 0 0 rpy: 0 -0 0 rpy: 0 -0 0
l_gripper_joint r_gripper_joint
l_gripper_l_finger_tip_frame r_gripper_l_finger_tip_frame its GitHub repository, removal of files which could not be parsed correctly, removal of duplicate files as the launch files are often Figure 7: PR2 Robot Coordinate Frames (a square node rep- shared between projects. Each row in the database represents a resents a frame) static transform with the following fields: a launch file id, a project id, each of the attributes and arguments of the static_transform_pub- lisher node, a flag indicating the version of the launch file (original or later commit), a flag indicating if the project is mature (more A AN EXAMPLE LAUNCH FILE SNIPPET than 30 commits). Note that we skip static transforms that employ The following shows a snippet from a launch file defining a trans- macro expansion in the node arguments (which we leave it as a form from a base_link frame to an optical depth frame for a Robotiq future task). In total our database contains 26K static transforms gripper simulation [12]. from 13K files in 6K projects, out of which 12.1K static transforms
Figure 8: Example for redundant transform (source: https://git.io/Jt5Vs ) during the computation of a net transform between two frames. transform detected by PhysFrame. The code invokes the lookup- Transform function to obtain the transform between identical parent and child frames (‘base_link’) from the tf tree, which is essentially an identity transform with zero displacements and zero rotation. Manual analysis found that the transform is used in a a number of places computing distance between the robot (‘base_link’) and the target goal. They simply correspond to useless computation. After we reported it to the developer, it was acknowledged that though it does not affect the correctness, it is completely unnecessary and can cause confusion. Case II: Missing frame. Figure9 shows the code snippet related to PhysFrame: Type Checking Physical Frames of Reference for Robotic Systems ESEC/FSE ’21, August 23–27, 2021, Athens, Greece
Table 6: Inconsistencies and Implicit Convention Violation Warnings Implicit warnings #: single value 0 = no warnings were reported for all the thresholds; and value - = a project does not qualify for the implicit convention checks as it does not contain launch files with static transforms.
Github Repository Inconsistencies Implicit Run- Github Repository Inconsistencies Implicit Run- Warnings for time Warnings for time TP FP TP FP z=1,2,5,10 (#) (s) z=1,2,5,10 (#) (s) (#) (#) (#) (#) pheeno_ros 1 0 - 0 victim_localization 1 1 0 4 mavros 3 0 0 52 LAM_SLAM 1 0 0 62 rosflight 3 0 - 2 ipc_slam_cnn 1 1 1, 1, 0, 0 1 ur_modern_driver 1 0 - 13 denso_tomato 0 4 0 5 orb_slam_2_ros 1 0 - 291 mbzirc_task2 2 0 0 4 gps_umd 1 0 - 1 SpCoSLAM_Lets 0 1 0 135 se306p1 4 0 - 1 catkin_ws 3 2 0 8 roman-technologies 4 0 - 6 robotx 2 0 0 9 CRF 1 0 0 8 IntelligentRobotics 1 0 0 8 roscorobot 2 0 - 20 riptide2017 1 0 0 0 rotors_simulator_with_drcsim_env 2 0 - 6 air_manipulator 12 0 0 23 px4_offboard_velocity_control 1 0 - 0 summit-xl-ros-stack 2 0 0 10 velocity_marker_tracker 2 0 - 0 pick_n_place 0 0 2, 2, 0, 0 200 MultiRobotExplorationPackages 4 1 0 424 Mobile-Robotics-Development-Platform 1 0 0 25 body_axis_controller 1 0 - 0 jaguar-bot 2 2 0 53 body_axis_velocity_controller 1 0 - 0 semantic_labels_sys 0 0 1, 0, 0, 0 45 marker_tracker 1 0 - 1 Racecar-Platform 1 1 0 4 Loam_Matlab_SMP 1 0 - 5 fyp_uav_radar 1 0 0 7 arduino_brushless 1 0 - 1 PX4_Drone 3 0 0 0 frobit_lego_transporter 1 0 - 1 depth_tools 1 0 0 0 wiic_twist 1 0 - 1 IntelligentCar 0 3 0 39 Create_PointCloud-ROS 1 0 - 1 rapid_pbd 0 1 0 9 autorally 2 0 - 9 Lidar_Utility 1 0 0 18 ua-ros-pkg 2 0 0 162 ros_webcam_lidar 1 0 0 0 model_car 3 0 0 41 lidar_slam_3d 0 1 0 1 door_detection_smr 3 0 - 4 Autonomous_wheelchair_navigation 3 0 0 0 table_cleaner_ur10 6 0 - 18 AutoNavQuad 1 0 0 3 ece6460_examples 1 0 0 1 Mini-Lab 2 1 0 14 lsd_slam 1 0 0 133 gamesh_bridge 0 1 0 0 los_slam 1 0 0 2 rockin_logger 1 0 0 1 Robotic 2 0 0 3 hpp_source_code 0 0 1, 1, 0, 0 9 USV-ROS-Stack 1 2 0 7 chairbot_selfdrive 0 0 1, 1, 1, 0 2 Inverse-Kinematics 2 0 0 2 mycomputer 1 0 0 2 ifacWC2020 0 0 3, 2, 0, 0 17 NWPU_robot 4 0 0 35 gki_pr2_symbolic_planning 0 1 0 7 match_catch 5 0 2, 2, 2, 0 1190 util 2 0 0 1 HislandAutonomousRobot 1 0 0 1 derail-fetchit-public 2 1 2, 1, 1, 0 13 epuck_world 1 0 0 0 WaterGun2016 1 0 0 139 human_dialogue 0 2 0 22 PiObs 2 0 0 0 monoodometer 3 0 0 49 16-662_Robot_Autonomy_Project 1 0 0 0 pushing_benchmark 0 0 5, 5, 2, 0 217 dyros_jet_dg 1 0 0 69 hrp_mpc_lidar 1 0 4, 4, 2, 2 868 robo-games 2 0 0 2 scan_tools 0 0 1, 1, 0, 0 3 ROS-Challenges 2 0 0 2 kill_test 0 1 0 18 RAS-2016 3 1 0 10 thesis-repo 3 0 0 1 cam_mocap_calib 0 3 0 1 dashgo 0 0 6, 6, 0, 0 1 bigfoot 0 0 1, 1, 0, 0 2 iarc-2017 4 0 1, 1, 0, 0 21 ros_project 4 0 0 5 iarc-mission8 1 0 1, 0, 0, 0 26 ae-scoot 0 0 7, 7, 7, 7 40 grasping_oru 0 2 0 13 MR2_ROS 0 2 5, 0, 0, 0 1 navigation_oru-release 4 0 0 457 Fake-LaserScan 0 0 1, 1, 1, 0 0 Distributed_Collaborative_Task_- 0 1 0 14 Total 154 36 45, 36, 16, 9 5162 Tree_ubuntu-version-16.04 [81.05%] ESEC/FSE ’21, August 23–27, 2021, Athens, Greece S. Kate, M. Chinn, H. Choi, X. Zhang, S. Elbaum
Table 7: Projects used in the evaluation of PhysFrame C=C/C++ source files, L=Launch files, LOC=non-blank and non-commented lines of C/C++ source and header files as computed byCLOC
Github Repository C(#) L(#) LOC Github Repository C(#) L(#) LOC Github Repository C(#) L(#) LOC cam_mocap_calib 2 3 397 bigfoot 10 14 1282 ros_project 16 4 3224 ae-scoot 44 15 12405 jaguar-bot 96 52 20089 tbot_arm_ws 1 2 65 nbvplanner 11 10 3291 MR2_ROS 5 6 1459 depth_tools 4 2 473 IntelligentCar 86 44 46421 semantic_labels_sys 102 151 21031 LAM_SLAM 151 30 25638 ipc_slam_cnn 3 3 2769 rapid_pbd 33 15 7361 Baxter-Uncertainty 3 3 3783 DaVinci-Haptics 1 3 314 Racecar-Platform 10 22 3885 PX4_Drone 3 3 236 A2DR_CHULA 21 37 9221 csir-manipulation 3 9 465 robotx 25 7 5698 robond_project_6 4 5 71 IntelligentRobotics 17 37 7435 ME495Final 1 6 229 velodyne_driver 17 7 1799 Robotics 28 30 4814 crazy_lidar 5 3 1698 crazypi 1 2 471 RAS-2016 43 34 7835 denso_tomato 20 12 4458 mbzirc_task2 17 29 3422 ros_assignments 15 2 926 catkin_ws 44 66 5872 Robotics-Localization 2 6 28 team2 7 17 18263 team3 8 10 18499 AGV_ws 16 31 6799 16-662_Robot_Autonomy_- 3 5 103 dyros_jet_dg 34 24 16401 Project kobuki_navigation 1 10 163 quadrotor_moveit_nav 5 13 336 robo-games 6 17 1124 ROS-Challenges 8 16 1728 mbot_arm_DRL 1 3 103 ArDroneControl 6 7 1585 kuka_lwr_simulation 10 59 2177 contamination_stack 5 6 47856 VehicleControl 147 38 31301 riptide2017 2 6 507 air_manipulator 78 27 13507 udacity_bot 1 3 28 summit-xl-ros-stack 28 187 173054 fyp_uav_radar 33 64 445617 ece6460_examples 29 14 1486 lsd_slam 48 12 16489 at_home_rsbb 14 6 3873 los_slam 8 5 1434 Robotic 18 17 2164 udacity_bot-d 1 3 29 USV-ROS-Stack 27 16 5323 Inverse-Kinematics 18 20 2380 ifacWC2020 30 8 9102 Udacity-RoboND-project-2- 3 18 1199 Kinematics-Project-master gki_pr2_symbolic_planning 49 3 7461 robot_2d_nav 3 1 362 NASA-RMC-2020- 21 37 2436 NorthstarRobotics util 4 4 970 carl_bot 14 17 2745 carl_moveit 2 9 1118 derail-fetchit-public 39 29 10420 rail_ceiling 3 11 846 WaterGun2016 93 14 15151 PiObs 1 5 17 Where-Am-I-Udacity 2 6 29 victim_localization 16 3 2667 Fake-LaserScan 1 1 44 pick_n_place 37 6 21281 Mobile-Robotics-Development- 74 21 14200 Platform RosBat 6 7 1261 ultrasonic_to_laserscan 5 1 532 gps_umd 3 0 452 mavros 69 13 13065 orb_slam_2_ros 59 5 22460 pheeno_ros 5 4 549 realsense-ros 3 12 1771 rosflight 9 0 1850 ur_modern_driver 22 12 5697 door_detection_smr 17 3 8127 table_cleaner_ur10 62 50 17235 ua-ros-pkg 125 123 38418 model_car 22 16 13928 autorally 35 42 9220 arduino_brushless 4 0 447 frobit_lego_transporter 2 5 195 RoboticBugAlgo 8 0 1170 wiic_twist 3 0 795 Create_PointCloud-ROS 6 0 384 CS491_Autonomous_Buggy 74 15 28874 MultiRobotExplorationPackages 206 78 7115053 body_axis_controller 1 0 61 body_axis_velocity_controller 1 0 59 marker_tracker 1 3 76 px4_offboard_velocity_control 1 0 214 rotors_simulator_with_drc- 27 47 4579 velocity_marker_tracker 1 0 53 sim_env Loam_Matlab_SMP 6 3 2209 roscorobot 47 19 14831 CRF 23 4 3386 roman-technologies 23 0 13102 catkin_roboway 46 38 25360 NWPU_robot 95 59 23834 baxter_project 413 20 79454 AprilTag_Localization 93 22 196152 autonomus_transport_indus- 2 5 561 trial_system gx2019_omni_simulations 5 12 921 uwb-localization 23 3 6324 match_catch 65 146 135516 Lidar_Utility 72 41 9300 ros_leishen_lidar 2 1 253 ros_webcam_lidar 3 2 253 pose_estimation 3 3 1083 lidar_slam_3d 4 1 885 surgical_robot 1 12 17 Autonomous_wheelchair_navi- 1 21 94 slam_autonomous_navigation 38 14 8450 F1tenth 5 11 171 gation AutoNavQuad 17 26 3492 SpCoSLAM_Lets 63 11 11488 Mini-Lab 49 66 6941 gamesh_bridge 1 2 74 rockin_logger 1 2 160 head_meka 0 1 0 meka_pmb2_ros 4 59 398 hpp_source_code 0 2 0 RoboND-Localization-Project 2 6 29 amcl_3d 11 2 1518 autoware_rc_car 5 6 57767 rc_car 3 9 200 vehicle_sim 2 7 196 ros_lecture19_pkg 5 1 179 keti_ws 76 13 42784 ros-detection 1 3 12822 chairbot_selfdrive 10 20 1497 tms_ss_pot 12 10 2565 mycomputer 14 50 5079 epuck_world 2 3 531 Distributed_Collaborative_- 46 73 8867 Task_Tree_ubuntu-version- 16.04 human_dialogue 60 68 13760 ENPM-661-Planning-Projects 1 1 67 monoodometer 73 16 17056 3D-Mapping-Navigation-of- 2 17 2015 pushing_benchmark 58 25 62266 tangentbug 2 1 332 Quadrotors hrp_mpc_lidar 154 95 108073 pioneer_tools 5 10 230 turtlesim_pioneer 5 1 266 rcah18_pepper_navigation 1 11 280 se306p1 16 0 1851 sequential_scene_parsing 22 4 8755 3d_mapping 6 21 629 scan_tools 12 8 2346 ROSBeginner 4 10 125 px4VIO 9 1 1491 ua-catvehicle-dev 5 24 567 octagon 24 14 2425 kill_test 50 26 15288 contactdb_utils 11 12 1620 thesis-repo 33 5 860 oko_slam 29 23 7381 dashgo 4 19 3346 HislandAutonomousRobot 6 6 632 iarc-2017 45 56 10578 iarc-mission8 27 39 14238 theLearner 11 1 1290 onigiri_war 1 8 106 Mini-Lab-8 8 39 626 grasping_oru 15 16 6911 navigation_oru-release 134 95 84833 yumi_demos 3 12 917 orsens_ros 2 4 1137