DOCSLIB.ORG

Action Model Learning: Challenges and Techniques

Total Page：16

File Type：pdf, Size：1020Kb

Download full-text PDF Read full-text
Action Model Learning: Challenges and Techniques Comenius University, Bratislava Faculty of Mathematics, Physics and Informatics Department of Applied Informatics Action Model Learning: Challenges and echniques Dissertation Thesis Michal Certick" y$ Bratislava, 2013 Comenius University Faculty of Mathematics, Physics and Informatics Department of Applied Informatics Action Model Learning: Challenges and Techniques Dissertation Thesis Author: RNDr. Michal Certic!"y #upervisor: doc. PhDr. $"an #efr"anek CSc. #tudy Programme: &.2.(. Computer #cience Bratislava 2013 15391687 �������� ���������� �� ���������� ������� �� ������������ ������� ��� ����������� ������ ���������� ���� ��� �������� ����� ������ �������� ����� ���������� �������� ������� ������� ������ ������ ����� ���� ����� ���� ���� ����� ����� �� ������ ������ �������� �������� ����������� ���� �� ������� ������������ ������ �������� �� ������� ������� ��������� ��������� ������ ������ ������ ����� ��������� ���������� ��� ���������� ���� � �� ������� ��� �������� ������� ��� ��������� ������ ����� ��������� ����� �� ��� ��� �� ���������� �������� �� ����� ������������� �� ���������� ����������� � ������ ��� ���������� ��� ������� ����� � ������� ����������� �� ������ ��� ���������� �� ���������� ����������� ������ ���� ���� ��� ��������� ���� ����������� �������� � ���������� �� ������� ����������� ������ �������� ���� ����� ��� ������ ���� ��������� ���������� ��������� ���������� ����� ����� ��������� ������ ���� ��������� �� ����� ��������� ������� ����� 15391687 ���������� ���������� � ���������� ������� ����������� ������ � ����������� ������� ���������� ����� ���� � ���������� ��������� ����� ������ �������� �������� �������� ����������� �������������� �������� ������������ ���� ���� ����� ������ �������� ������ ������ ����������� ��� ���������� ������ ���������� ����� ���������� ������ �������� ���������� ������ ��������� ������ ������ ������ ������ ����� � �������� ����� � ���������� ���������� ������ ������������� ������ ������ ���������� ���������� ���� ����������� �������� � �������� ���������������� ������ � �������� ����������� � �������� ������� ��������� � ������� ��� ���������� � �������������� ������ ��������� ������� ��������� ���� ���� ��� ��������� ���� �������� �������� � ������� ����������� ����������� ������ �������� ���� ����� ��� ������ ���� ������ ������������� ������������� ������ ������ ��� ���������� ����� �������� ���������� ����� ����������� ���������� ����� ����� ��������� ������ ���� ������ ���������� �������� ������� �������� Abstract Knowledge about domain dynamics, describing how certain actions affect the world, is called an action model and constitutes the essential requirement for planning and goal-oriented intelligent behaviour. Action learning, as the au- tomatic construction of action models based on the observations, has become a hot research topic in recent years, and various methods employing a wide variety of AI tools have been developed. Diversity of these methods renders each of them usable under different conditions and in various domains. We have identi ed a common collection of important properties and used them as a basis for the comprehensive comparison and as a set of challenges that we were trying to overcome when designing our two new methods. First of them, an online algorithm called "SG, can be used to learn the pro- babilistic action models with conditional effects in the presence of incomplete observations, sensoric noise, and action failures, while #eeping the computa- tional complexity low $polynomial w.r.t. the size of the input&, thus managing to cope with all the appointed challenges. 'ur experiments, conducted in two fundamentally different domains (action computer game Unreal Tour- nament 2004 and a real-world robotic platform SyRoTek&, demonstrate its usability in computationally intensive scenarios, as well as in the presence of action failures and signi cant sensoric noise. 'ur second method, based on a certain paradigm of logic programming called Reactive ASP, is not as versatile, being able to deal with only half of the chal- lenges, but may still prove practical in some situations because of its purely declarative nature. Keywords: action model, learning, partially observable, probabilistic, noise, action failures, conditional effects, answer set programming Abstrakt Znalosti o dynami#e dom*eny, popisu+*uce a#o jednotliv*eakcie ovplyv,nu+*u svet, sa na%*yva+*u modelom akci!" a predstavu+*unutn*upodmien#u pre pl*anova- nie a ciel-avedom*einteligentn*espr*avanie. U#ceniesa akci!", teda automatic- #*a#on,stru#cia a spres,novanie modelov akci*. na %*a#lade pozorovan*., sa v posledn*ych rokoch stalo atra#t*.vnou v*yskumnou t*emou a ob+avilo sa niekol-#o zau+*.mav*ych met´od, vyu,%*.va+*ucich ,siro#*upaletu met*odUI. Ich diverzita m*a za n*asledok fa#t, ,%e #a,%d*a% nich je pou,%itel-n*a%a r0oznych podmieno#, resp. v rozdielnych typoch dom*en. Identi #ovali sme spolo,cn*umno,zinu d0ole,%it*ych vlastnost*.t*ychto met*od a pou,zili ich ako %*aklad pre ich d0o#ladn*uanal*y%u a a#o ciele pri tvorbe na,sich dvoch nov*ych met´od. Prv*a% nich, online algoritmus s n*azvom "SG, produ#uje pravdepodobnostn!e modely akci*.s podmienen!ymiefektami, a vyrovn*asa so $lyhan!ımakci!", sen- zorick!ym#sumom, a ne!upln!ymipozorovaniami, pri,com v,sak udr,ziava svoju ,casov*uzlo,%itost- dostato,cne n*.%#u $polynomi*alnu vzhl-adom na vel-#ost- vstu- pu&. Na,seexperimenty vy#onan*ev dvoch fundament*alne odli,sn*ych dom*e- nach $ak,cnej po,c*.ta,cove+ hre Unreal Tournament 2004 a re*alnej robotic#ej platforme SyRoTek), demon,stru+*upou,%itel-nost- tohoto algoritmu vo v*ypo,ctovo n*aro,cn*ych prostrediach, a#o a+ za pr*.tomnosti nezanedbatel-n*eho senzoric#*eho ,sumu a zlyhan*.a#ci*.. Na,sa druh*amet*oda, %alo,zen*ana ,speci c#ej paradigme logic#*eho programova- nia naz*yvanej Reakt!ıvneASP, nie je natol-#o univerz*alna, #ed-,zesp*l,na iba polovicu nami stanoven*ych ciel-ov. 2apriek tomu v,sa# m0o,zebyt- vhodn*ym #andid*atom v ist*ych situ*aci*ach, vd-a#a svojej deklarat*.vnej povahe. Kl’´uˇcov´eslov´a: model akci*., u,cenie, pl*anovanie, ,ciasto,cn*apozorovatel-nost-, pravdepodobnost-, ,sum, zlyhania a#ci*., podmienen*eefekty, AS1 Contents 1 Introduction 9 4.4 5ackground . ............. ............ 6 4.7 1roblems . ............. ............ 48 4." 1roposed 3olutions............ ............ 44 4.9 'utline. ............. ............ 47 2 Preliminaries 14 7.4 1lanning: Why do we need a domain description; . ............. ......... 49 7.7 <epresentation =anguages: >ow to express our action model; ........... ......... 4? 7." Action =earning: Why do we need to obtain action models automatically; . ......... 4@ 7.9 Ahallenges and 1roperties . ............ 4B Evaluation based on Properties 27 ".4 3=AF - 3imultaneous =earning and Filtering . ............. ......... 76 ".7 ARC3 - Action-Relation Codelling 3ystem . ............. ......... "" "." =earning through =ogic 1rogramming and A-Prolog . ............. ......... "6 ".9 Kernelised Doted 1erceptrons with Deictic Coding . ............. ......... 9B ".? Ereedy 3earch for =earning Noisy Deictic <ule 3ets ............. ......... ?9 ".@ New Methods . ............. ............@7 4 Evaluation based on Domain Compatibility 64 & Basic Notions %) % Representation Structures #, @.4 Fransition Relation ............ ......... B8 TR @.7 Gffect <elation ............. ........... B4 ER @." Gffect !ormula ............ ............B7 EF @.9 A-Prolog =iterals ............. .......... B9 AL @.? 1robability !unction ........... ............B@ # !-amples. Action /earning and Performance 2easures #) B.4 1ositive and 2egative G(amples . ............BH B.7 Action =earning . ............. ............H7 B." Ciscellaneous Notions .......... ............H" )"SG Algorithm )& H.4 Fhe Algorithm . ............. ............H? H.7 Aorrectness and Aomplexity . ............H6 H." 1erformance Cetrics . ............ 6" H.9 Additional Ideas . ............. ............ 6@ 9 !-"eriments with"SG Algorithm 97 6.4 Unreal Fournament7889......... ............ 6B 6.7 <obotic 1latform 3yRoTe# . ............489 6." 3ummary . ............. ............48B 1, Reactive ASP Method 109 40.4 <eactive AS1 . ............. ............448 40.7 =earning with <eactive AS1 . ............444 40." Noise and Non-determinism . ............44@ 40.9 3imilarities and Differences . ............44B 40.? 1erformance Cetrics . ............474 11 !-"eriments wit3 Reactive ASP 2ethod 123 41.4 !ully 'bservable 5loc#s World . 479 41.7 1artially 'bservable 5locks World . ............47@ 41." 3ummary . ............. ............47H 12 Conclusion and 4uture 5ork 129 A""endices 1 A Additional Properties of !6ects (learned by"SG8 1 ' 5orst Case 1or"SG Algorithm 1 # C Dynamic Error 9olerance 1or *eactive ASP Method 143 Chapter 1 Introduction 1.1 Back0round Knowledge about domain dynamics, describing how certain actions affect the world, is essential for planning and intelligent goal-oriented behaviour of both living and arti cial agents. 3uch #nowledge is in arti cial systems referred to as action model, and is usually manually constructed by domain experts. Complete speci cation of action models in case of complex domains is, how- ever, a diIcult and time consuming task. In addition to that, it is often needed to modify this action model when confronted with new information. <esearch on various methods of automatic construction of action models has therefore become a hot topic in recent years. Fhis inductive process of con- structing and subsequent modi cation of action models is referred to as action
Load more

Recommended publications
  • Classical Planning in Deep Latent Space
    Journal of Artificial Intelligence Research 1 (2020) 1-15 Submitted 1/1; published 1/1 (Table of Contents is provided for convenience and will be removed in the camera-ready.) Contents 1 Introduction 1 2 Preliminaries and Important Background Concepts 4 2.1 Notations . .4 2.2 Propositional Classical Planning . .4 2.3 Processes Required for Obtaining a Classical Planning Model . .4 2.4 Symbol Grounding . .5 2.5 Autoencoders and Variational Autoencoders . .7 2.6 Discrete Variational Autoencoders . .9 3 Latplan: Overview of the System Architecture 10 4 SAE as a Gumbel-Softmax/Binary-Concrete VAE 13 4.1 The Symbol Stability Problem: Issues Caused by Unstable Propositional Symbols . 15 4.2 Addressing the Systemic Uncertainty: Removing the Run-Time Stochasticity . 17 4.3 Addressing the Statistical Uncertainty: Selecting the Appropriate Prior . 17 5 AMA1: An SAE-based Translation from Image Transitions to Ground Actions 19 6 AMA2: Action Symbol Grounding with an Action Auto-Encoder (AAE) 20 + 7 AMA3/AMA3 : Descriptive Action Model Acquisition with Cube-Space AutoEncoder 21 7.1 Cube-Like Graphs and its Equivalence to STRIPS . 22 7.2 Edge Chromatic Number of Cube-Like Graphs and the Number of Action Schema in STRIPS . 24 7.3 Vanilla Space AutoEncoder . 27 7.3.1 Implementation . 32 7.4 Cube-Space AE (CSAE) . 33 7.5 Back-to-Logit and its Equivalence to STRIPS . 34 7.6 Precondition and Effect Rule Extraction . 35 + arXiv:2107.00110v1 [cs.AI] 30 Jun 2021 8 AMA4 : Learning Preconditions as Effects Backward in Time 36 8.1 Complete State Regression Semantics .
    [Show full text]
  • Discovering Underlying Plans Based on Shallow Models
    Discovering Underlying Plans Based on Shallow Models HANKZ HANKUI ZHUO, Sun Yat-Sen University YANTIAN ZHA, Arizona State University SUBBARAO KAMBHAMPATI, Arizona State University XIN TIAN, Sun Yat-Sen University Plan recognition aims to discover target plans (i.e., sequences of actions) behind observed actions, with history plan libraries or action models in hand. Previous approaches either discover plans by maximally “matching” observed actions to plan libraries, assuming target plans are from plan libraries, or infer plans by executing action models to best explain the observed actions, assuming that complete action models are available. In real world applications, however, target plans are often not from plan libraries, and complete action models are often not available, since building complete sets of plans and complete action models are often difficult or expensive. In this paper we view plan libraries as corpora and learn vector representations of actions using the corpora; we then discover target plans based on the vector representations. Specifically, we propose two approaches, DUP and RNNPlanner, to discover target plans based on vector representations of actions. DUP explores the EM-style (Expectation Maximization) framework to capture local contexts of actions and discover target plans by optimizing the probability of target plans, while RNNPlanner aims to leverage long-short term contexts of actions based on RNNs (recurrent neural networks) framework to help recognize target plans. In the experiments, we empirically show that our approaches are capable of discovering underlying plans that are not from plan libraries, without requiring action models provided. We demonstrate the effectiveness of our approaches by comparing its performance to traditional plan recognition approaches in three planning domains.
    [Show full text]
  • Learning to Plan from Raw Data in Grid-Based Games
    EPiC Series in Computing Volume 55, 2018, Pages 54{67 GCAI-2018. 4th Global Con- ference on Artificial Intelligence Learning to Plan from Raw Data in Grid-based Games Andrea Dittadi, Thomas Bolander, and Ole Winther Technical University of Denmark, Lyngby, Denmark fadit, tobo, [email protected] Abstract An agent that autonomously learns to act in its environment must acquire a model of the domain dynamics. This can be a challenging task, especially in real-world domains, where observations are high-dimensional and noisy. Although in automated planning the dynamics are typically given, there are action schema learning approaches that learn sym- bolic rules (e.g. STRIPS or PDDL) to be used by traditional planners. However, these algorithms rely on logical descriptions of environment observations. In contrast, recent methods in deep reinforcement learning for games learn from pixel observations. However, they typically do not acquire an environment model, but a policy for one-step action selec- tion. Even when a model is learned, it cannot generalize to unseen instances of the training domain. Here we propose a neural network-based method that learns from visual obser- vations an approximate, compact, implicit representation of the domain dynamics, which can be used for planning with standard search algorithms, and generalizes to novel domain instances. The learned model is composed of submodules, each implicitly representing an action schema in the traditional sense. We evaluate our approach on visual versions of the standard domain Sokoban, and show that, by training on one single instance, it learns a transition model that can be successfully used to solve new levels of the game.
    [Show full text]
  • Learning Complex Action Models with Quantifiers and Logical Implications
    Artificial Intelligence 174 (2010) 1540–1569 Contents lists available at ScienceDirect Artificial Intelligence www.elsevier.com/locate/artint Learning complex action models with quantifiers and logical implications ∗ Hankz Hankui Zhuo a,b, Qiang Yang b, , Derek Hao Hu b,LeiLia a Department of Computer Science, Sun Yat-sen University, Guangzhou, China, 510275 b Department of Computer Science and Engineering, Hong Kong University of Science and Technology, Clearwater Bay, Kowloon, Hong Kong article info abstract Article history: Automated planning requires action models described using languages such as the Planning Received 2 January 2010 Domain Definition Language (PDDL) as input, but building action models from scratch is a Received in revised form 4 September 2010 very difficult and time-consuming task, even for experts. This is because it is difficult to Accepted 5 September 2010 formally describe all conditions and changes, reflected in the preconditions and effects of Available online 29 September 2010 action models. In the past, there have been algorithms that can automatically learn simple Keywords: action models from plan traces. However, there are many cases in the real world where Action model learning we need more complicated expressions based on universal and existential quantifiers, Machine learning as well as logical implications in action models to precisely describe the underlying Knowledge engineering mechanisms of the actions. Such complex action models cannot be learned using many Automated planning previous algorithms. In this article, we present a novel algorithm called LAMP (Learning Action Models from Plan traces), to learn action models with quantifiers and logical implications from a set of observed plan traces with only partially observed intermediate state information.
    [Show full text]
  • Arxiv:1803.02208V1 [Cs.AI] 4 Mar 2018 Subbarao Kambhampati Arizona State University E-Mail: [email protected] 2 Hankz Hankui Zhuo Et Al
    Noname manuscript No. (will be inserted by the editor) Discovering Underlying Plans Based on Shallow Models Hankz Hankui Zhuo · Yantian Zha · Subbarao Kambhampati the date of receipt and acceptance should be inserted later Abstract Plan recognition aims to discover target plans (i.e., sequences of actions) behind observed actions, with history plan libraries or domain models in hand. Pre- vious approaches either discover plans by maximally “matching” observed actions to plan libraries, assuming target plans are from plan libraries, or infer plans by execut- ing domain models to best explain the observed actions, assuming that complete do- main models are available. In real world applications, however, target plans are often not from plan libraries, and complete domain models are often not available, since building complete sets of plans and complete domain models are often difficult or expensive. In this paper we view plan libraries as corpora and learn vector represen- tations of actions using the corpora; we then discover target plans based on the vector representations. Specifically, we propose two approaches, DUP and RNNPlanner, to discover target plans based on vector representations of actions. DUP explores the EM-style framework to capture local contexts of actions and discover target plans by optimizing the probability of target plans, while RNNPlanner aims to lever- age long-short term contexts of actions based on RNNs (recurrent neural networks) framework to help recognize target plans. In the experiments, we empirically show that our approaches are capable of discovering underlying plans that are not from plan libraries, without requiring domain models provided. We demonstrate the effective- ness of our approaches by comparing its performance to traditional plan recognition approaches in three planning domains.
    [Show full text]
  • Towards Learning in Probabilistic Action Selection: Markov Systems and Markov Decision Processes
    Towards Learning in Probabilistic Action Selection: Markov Systems and Markov Decision Processes Manuela Veloso Carnegie Mellon University Computer Science Department Planning - Fall 2001 Remember the examples on the board. 15-889 – Fall 2001 Veloso, Carnegie Mellon Different Aspects of “Machine Learning” Supervised learning • – Classification - concept learning – Learning from labeled data – Function approximation Unsupervised learning • – Data is not labeled – Data needs to be grouped, clustered – We need distance metric Control and action model learning • – Learning to select actions efficiently – Feedback: goal achievement, failure, reward – Search control learning, reinforcement learning 15-889 – Fall 2001 Veloso, Carnegie Mellon Search Control Learning Improve search efficiency, plan quality • Learn heuristics • (if (and (goal (enjoy weather)) (goal (save energy)) (state (weather fair))) (then (select action RIDE-BIKE))) 15-889 – Fall 2001 Veloso, Carnegie Mellon Learning Opportunities in Planning Learning to improve planning efficiency • Learning the domain model • Learning to improve plan quality • Learning a universal plan • Which action model, which planning algorithm, which heuristic control is the most efficient for a given task? 15-889 – Fall 2001 Veloso, Carnegie Mellon Reinforcement Learning A variety of algorithms to address: • – learning the model – converging to the optimal plan. 15-889 – Fall 2001 Veloso, Carnegie Mellon Discounted Rewards “Reward” today versus future (promised) reward • $100K + $100K + $100K + . • INFINITE!
    [Show full text]
  • Outline of Machine Learning
    Outline of machine learning The following outline is provided as an overview of and topical guide to machine learning: Machine learning – subfield of computer science[1] (more particularly soft computing) that evolved from the study of pattern recognition and computational learning theory in artificial intelligence.[1] In 1959, Arthur Samuel defined machine learning as a "Field of study that gives computers the ability to learn without being explicitly programmed".[2] Machine learning explores the study and construction of algorithms that can learn from and make predictions on data.[3] Such algorithms operate by building a model from an example training set of input observations in order to make data-driven predictions or decisions expressed as outputs, rather than following strictly static program instructions. Contents What type of thing is machine learning? Branches of machine learning Subfields of machine learning Cross-disciplinary fields involving machine learning Applications of machine learning Machine learning hardware Machine learning tools Machine learning frameworks Machine learning libraries Machine learning algorithms Machine learning methods Dimensionality reduction Ensemble learning Meta learning Reinforcement learning Supervised learning Unsupervised learning Semi-supervised learning Deep learning Other machine learning methods and problems Machine learning research History of machine learning Machine learning projects Machine learning organizations Machine learning conferences and workshops Machine learning publications
    [Show full text]
  • From Unlabeled Images to PDDL (And Back)
    Classical Planning in Latent Space: From Unlabeled Images to PDDL (and back) Masataro Asai, Alex Fukunaga Graduate School of Arts and Sciences University of Tokyo Abstract Current domain-independent, classical planners re- ? quire symbolic models of the problem domain and Initial state Goal state image Original instance as input, resulting in a knowledge acquisi- image (black/white) Mandrill image tion bottleneck. Meanwhile, although recent work in deep learning has achieved impressive results in Figure 1: An image-based 8-puzzle. many fields, the knowledge is encoded in a subsym- bolic representation which cannot be directly used [ ] by symbolic systems such as planners. We propose Mnih et al., 2015; Graves et al., 2016 . However, the current LatPlan, an integrated architecture combining deep state-of-the-art in pure NN-based systems do not yet provide learning and a classical planner. Given a set of un- guarantees provided by symbolic planning systems, such as labeled training image pairs showing allowed ac- deterministic completeness and solution optimality. tions in the problem domain, and a pair of images Using a NN-based perceptual system to automatically pro- representing the start and goal states, LatPlan uses vide input models for domain-independent planners could a Variational Autoencoder to generate a discrete greatly expand the applicability of planning technology and latent vector from the images, based on which a offer the benefits of both paradigms. We consider the problem PDDL model can be constructed and then solved of robustly, automatically bridging the gap between such sub- by an off-the-shelf planner. We evaluate LatPlan us- symbolic representations and the symbolic representations ing image-based versions of 3 planning domains: required by domain-independent planners.
    [Show full text]
  • Integrating Planning, Execution and Learning to Improve Plan Execution
    Computational Intelligence, Volume 00, Number 000, 2009 Integrating Planning, Execution and Learning to Improve Plan Execution Sergio Jimenez´ , Fernando Fernandez´ and Daniel Borrajo Departamento de Inform´aatica, Universidad Carlos III de Madrid. Avda. de la Universidad, 30. Legan´es(Madrid). Spain Algorithms for planning under uncertainty require accurate action models that explicitly cap- ture the uncertainty of the environment. Unfortunately, obtaining these models is usually complex. In environments with uncertainty, actions may produce countless outcomes and hence, specifying them and their probability is a hard task. As a consequence, when implementing agents with planning capabilities, practitioners frequently opt for architectures that interleave classical planning and execution monitoring following a replanning when failure paradigm. Though this approach is more practical, it may produce fragile plans that need continuous replanning episodes or even worse, that result in execution dead-ends. In this paper, we propose a new architecture to relieve these shortcomings. The architecture is based on the integration of a relational learning component and the traditional planning and execution monitoring components. The new component allows the architecture to learn probabilistic rules of the success of actions from the execution of plans and to automatically upgrade the planning model with these rules. The upgraded models can be used by any classical planner that handles metric functions or, alternatively, by any probabilistic planner. This architecture proposal is designed to integrate off-the-shelf interchangeable planning and learning components so it can profit from the last advances in both fields without modifying the architecture. Key words: Cognitive architectures, Relational reinforcement learning, Symbolic planning. 1.
    [Show full text]
  • Discrete Word Embedding for Logical Natural Language Understanding
    Under review as a conference paper at ICLR 2021 DISCRETE WORD EMBEDDING FOR LOGICAL NATU- RAL LANGUAGE UNDERSTANDING Anonymous authors Paper under double-blind review ABSTRACT We propose an unsupervised neural model for learning a discrete embedding of words. Unlike existing discrete embeddings, our binary embedding supports vec- tor arithmetic operations similar to continuous embeddings. Our embedding rep- resents each word as a set of propositional statements describing a transition rule in classical/STRIPS planning formalism. This makes the embedding directly com- patible with symbolic, state of the art classical planning solvers. 1 INTRODUCTION When we researchers write a manuscript for a conference submission, we do not merely follow the probability distribution crystalized in our brain cells. Instead, we modify, erase, rewrite sentences over and over, while only occasionally let the fingers produce a long stream of thought. This writing process tends to be a zero-shot attempt to materialize and optimize a novel work that has never been written. Except for casual writing (e.g. online messages), intelligent writing inevitably contains an aspect of backtracking and heuristic search behavior, and thus is often like planning for information delivery while optimizing various metrics, such as the impact, ease of reading, or conciseness. After the initial success of the distributed word representation in Word2Vec (Mikolov et al., 2013b), natural language processing techniques have achieved tremendous progress in the last decade, pro- pelled primarily by the advancement in data-driven machine learning approaches based on neural networks. However, these purely data-driven approaches that blindly follow the highest probability interpolated from data at each time step could suffer from biased decision making (Caliskan et al., 2017; Bolukbasi et al., 2016) and is heavily criticized recently.
    [Show full text]
  • Extending Symbolic Planning Operators Through Targeted Reinforcement Learning
    Main Track AAMAS 2021, May 3-7, 2021, Online SPOTTER: Extending Symbolic Planning Operators through Targeted Reinforcement Learning Vasanth Sarathy∗ Daniel Kasenberg∗ Shivam Goel Smart Information Flow Technologies Tufts University Tufts University Lexintgon, MA Medford, MA Medford, MA [email protected] [email protected] [email protected] Jivko Sinapov Matthias Scheutz Tufts University Tufts University Medford, MA Medford, MA [email protected] [email protected] ABSTRACT used to complete a variety of tasks), available human knowledge, Symbolic planning models allow decision-making agents to se- and interpretability. However, the models are often handcrafted, quence actions in arbitrary ways to achieve a variety of goals in difficult to design and implement, and require precise formulations dynamic domains. However, they are typically handcrafted and that can be sensitive to human error. Reinforcement learning (RL) tend to require precise formulations that are not robust to human approaches do not assume the existence of such a domain model, error. Reinforcement learning (RL) approaches do not require such and instead attempt to learn suitable models or control policies models, and instead learn domain dynamics by exploring the en- by trial-and-error interactions in the environment [28]. However, vironment and collecting rewards. However, RL approaches tend RL approaches tend to require a substantial amount of training in to require millions of episodes of experience and often learn poli- moderately complex environments. Moreover, it has been difficult cies that are not easily transferable to other tasks. In this paper, to learn abstractions to the level of those used in symbolic planning we address one aspect of the open problem of integrating these approaches through low-level reinforcement-based exploration.
    [Show full text]
  • Pre-Proceedings of the 12Th International Workshop on Neural-Symbolic Learning and Reasoning Nesy’17
    Pre-Proceedings of the 12th International Workshop on Neural-Symbolic Learning and Reasoning NeSy’17 Tarek R. Besold, Artur d’Avila Garcez, Isaac Noble (eds.) City, University of London – London, UK – 17th & 18th of July 2017 We, as workshop organizers, want to thank the following members of the NeSy’17 program committee for their time and efforts in reviewing the submissions to the workshop and providing valuable feedback to accepted and rejected papers and abstracts alike: - Raquel Alhama, University of Amsterdam. - James Davidson, Google Brain. - Richard Evans, Google DeepMind. - Barbara Hammer, University of Bielefeld. - Thomas Icard, Stanford University. - Kai-Uwe Kühnberger, University of Osnabrück. - Luis Lamb, Universidade Federal do Rio Grande do Sul. - Thomas Lukasiewicz, University of Oxford. - Edjard Mota, Federal University of Amazonas. - Terrence C. Stewart, University of Waterloo. - Serge Thill, Plymouth University. - Son Tran, CSIRO Australia. - Stefan Wermter, University of Hamburg. These workshop pre-proceedings are available online from the workshop website under http://www.neural-symbolic.org/NeSy17/. Bremen, 12th of July 2017 Tarek R. Besold, Artur S. d’Avila Garcez, and Isaac Noble.1 1 Tarek R. Besold is a postdoctoral researcher at the Center for Computing and Communication Technologies (TZI) of the University of Bremen, Germany; Artur S. d’Avila Garcez is Professor of Computer Science at the Department of Computer Science, City, University of London, UK; Isaac Noble is a Software Architect at Playground Global, USA. Contents: Contributed Papers Towards Grounding Conceptual Spaces in Neural Representations (Lucas Bechberger and Kai-Uwe Kühnberger) Propositional Rule Extraction from Neural Networks under Background Knowledge (Maryam Labaf, Pascal Hitzler, and Anthony B.
    [Show full text]