robotics Article Leveraging Graphical User Interface Automation for Generic Robot Programming Tudor B. Ionescu Human-Machine Interaction Group, Vienna University of Technology, 1040 Vienna, Austria;
[email protected]; Tel.: +43-(1)-58801-33001 Abstract: A novel approach to generic (or generalized) robot programming and a novel simplified, block-based programming environment, called “Assembly”, are introduced. The approach lever- ages the newest graphical user interface automation tools and techniques to generate programs in various proprietary robot programming environments by emulating user interactions in those environments. The “Assembly” tool is used to generate robot-independent intermediary program models, which are translated into robot-specific programs using a graphical user interface automation toolchain. The generalizability of the approach to list, tree, and block-based programming is assessed using three different robot programming environments, two of which are proprietary. The results of this evaluation suggest that the proposed approach is feasible for an entire range of programming models and thus enables the generation of programs in various proprietary robot programming envi- ronments. In educational settings, the automated generation of programs fosters learning different robot programming models by example. For experts, the proposed approach provides a means for generating program (or task) templates, which can be adjusted to the needs of the application at hand on the shop floor. Keywords: generic robot programming; GUI automation; robotic process automation; graphical pro- gramming; block-based programming Citation: Ionescu, T.B. Leveraging 1. Introduction Graphical User Interface Automation Recent advances in safe, collaborative robotics technology and the increasing avail- for Generic Robot Programming. ability of open source robot hardware facilitated the emergence of numerous new vendors Robotics 2021, 10, 3.