BEng Individual Project Imperial College London Department of Computing Unsupervised Path Regression Networks Supervisor: Dr. Ronald Clark Author: Michal Pándy Second Marker: Dr. Amir Alansary June 15, 2020 Abstract Path planning problems are conventionally solved in an iterative manner. Recent iterative learning methods have greatly improved inference time, but require ground truth paths for training, which are costly to compute. In this work, we demonstrate that complex shortest path problems can be solved via direct regression learned in an unsupervised manner, making the training pipeline simple and scalable. Key to our approach is a novel cost function which only requires a scene and a planning problem specification, and whose minima guarantee collision-free solutions. We show that this cost function is capable of attaining high-quality paths, and experimental results further demonstrate that our method outperforms the objectives of optimisation-based planners, beats state-of-the-art supervised learning baselines for shortest path planning, and is capable of planning in domains with partial observability. Acknowledgements I would like to thank Dr. Ronald Clark and Daniel Lenton for our fruitful meetings and discussions, which resulted in countless ideas for tackling the problems presented in this work. I would also like to thank my parents Ladislav and Alicja for their restless support through- out all of my eduction. Lastly, I want to thank Maximilian, Alex, and Emmie for their unconditional love. Contents 1 Introduction 7 1.1 Objectives.........................................8 1.2 Challenges.........................................8 1.3 Contributions.......................................9 1.4 Publication........................................9 1.5 Report Layout......................................9 2 Preliminaries 10 2.1 Deep learning....................................... 10 2.1.1 Neural networks................................. 10 2.1.2 Convolutional neural networks......................... 11 2.1.3 Recurrent neural networks............................ 12 2.2 Non-uniform rational basis splines (NURBS)..................... 12 2.2.1 B-splines..................................... 12 2.2.2 NURBS interpolation.............................. 14 2.3 Signed Distance Function................................ 15 3 Background 16 3.1 Classical planning.................................... 16 3.1.1 Node-based algorithms.............................. 16 3.1.2 Sampling-based algorithms........................... 17 3.1.3 Mathematical model-based algorithms..................... 19 3.1.4 Bio-inspired algorithms............................. 20 3.1.5 Summary..................................... 20 3.2 Stochastic optimisation-based planning......................... 20 3.2.1 Two-dimensional path planning......................... 20 3.2.2 Higher-dimensional path planning....................... 22 3.2.3 Summary..................................... 25 4 Approach 26 4.1 Cost function derivation................................. 26 4.2 Parameterisation..................................... 29 4.3 General optimisation process.............................. 32 4.3.1 General network architecture.......................... 32 4.3.2 Training considerations............................. 32 4.4 Summary......................................... 35 5 Evaluation 36 5.1 Objective comparisons.................................. 36 5.2 Continuous full-state planning.............................. 38 5.2.1 Data generation................................. 38 5.2.2 Approach setup.................................. 39 5.2.3 Path correction.................................. 39 5.2.4 Results...................................... 39 5.3 Planning from Images.................................. 41 5.3.1 Data generation................................. 41 5.3.2 Approach setup.................................. 41 2 5.3.3 Results...................................... 42 5.4 Current limitation: maze-like environments...................... 44 5.4.1 Data generation................................. 44 5.4.2 Approach setup.................................. 44 5.4.3 Enforcing physical constraints.......................... 45 5.4.4 Results...................................... 45 5.5 Common failure modes.................................. 47 5.6 Summary......................................... 47 6 Conclusion 48 6.1 Future work........................................ 48 3 List of Figures 1.1 (left) Gradient-based planning methods use cost functions consisting of two heuristic terms, a path-length l term and a collision term ccoll combined using an arbitrary weighting, c = l+αccoll. In this work we design l and ccoll to guarantee collision-free paths at the optimum without requiring a calibrated weighting factor. We further propose to solve this optimisation problem using an efficient regression network (right)...........................................7 2.1 Example of a single artificial neuron........................... 10 2.2 Example of an artificial neural network with one input layer, one output layer, and three hidden layers.[1] Each node in the network represents a single artificial neuron, as seen in Figure 2.1.................................... 11 2.3 A standard CNN architecture for image classification. [2].............. 12 2.4 The spline above illustrates an example of fitting piecewise cubic polynomials be- tween pairs of anchor points............................... 13 2.5 B-spline with an open-uniform knot vector, 6 control points, and highlighted com- ponents........................................... 14 2.6 At the top, we have an example two dimensional shape, and the corresponding negative signed distance function values illustrated at the bottom. Notice that the gradients of the signed distance function generally point in directions away from the shape............................................ 15 3.1 On the left, we have a pseudocode for Dijkstra’s shortest path algorithm. Note that the EXTRACT-MIN function simply returns the lowest weight vertex from Q. Q in practice is often implemented as a priority queue. On the right, we have an example of running Dijkstra’s shortest path algorithm on a simple two-dimensional planning problem[3]. The algorithm is terminated when it reached the target location. The grid cells represent the vertices explored by the algorithm before finding the target location........................................... 17 3.2 A* due to its heuristic allows us to solve the planning problem presented in Figure 3.1 with less unnecessary exploration.[4]........................ 17 3.3 RRT algorithm at different iteration steps. From left to right, we can observe how the RRT fills the space from the centre starting configuration as iterations increase.[5] 18 3.4 Demonstration of the potential field gradients created by a sphere obstacle in a scene.[6].......................................... 19 3.5 Example of an output from a linear programming-based path planning algorithm of Plessen et al. (2017)[7]. The figure demonstrates the optimal path’s awareness of car dimensions....................................... 19 3.6 Examples of successfully planned LSTM-based paths from the work of Inoue et al.(2019)[8]......................................... 21 3.7 Banzhaf et al. (2019) [9] proposed CNN architecture for vehicle motion prediction. The network receives past path, and obstacles as an input, while outputting a feasible path projection together with a heading angle prediction............... 21 3.8 Sample path planning problems solved by VIN[10] in maze-like grid environments. 22 3.9 Red paths are generated by Motion Planning Networks based on the work of Qureshi et al. (2019)[11], while the blue paths represent ground truth RRT* paths..... 23 4 3.10 This figure illustrates the different planning components of Motion Panning Networks[11] described above. On the left, we have the training of the encoder network, together with the demonstration of PNet. We can notice that the output of a single PNet forward pass is the next desired robot state x^t+1. On the right, Neural Planner is capable of unrolling the full path online......................... 23 3.11 From left to right, we can see 3, 4, and 6 link robotic arms moving to a (green) target configuration based on the OracleNet approach................. 24 3.12 In this image, Jurgenson et al. (2019) illustrate the distribution of end-effector poses when trained in a supervised setting via RRT* or A* supervision. The edge distributions are much less dense than the ones passing in the middle........ 24 3.13 Illustration of the gradients in a single optimisation step of the GPMP2[12] method. 25 4.1 The T transformation transforms a colliding path segment into a non-colliding seg- ment............................................ 28 4.2 This figure illustrates the approximation of ccoll via cOP T −coll. The green points along the path are non-colliding, hence their cp = 0 and they do not have an effect on the collision cost. On the other hand, the red points lie within objects, hence we need to include the corresponding bounding sphere circumferences in the collision cost. The object specific ∆ simply ensures that in the final cOP T −coll sum, an object contributes exactly only by its bounding sphere circumference............ 30 4.3 Visualisation of the gradient of our cost function’s collision component based on Figure 4.2 and definition 4.28............................... 31 4.4 Visualisation of a possible path obtained from