O Rationale for the Project

Structure  4 research proposals (Proposal #) o Title o Summary o Rationale for the project o Evaluation using “porposal evaluation criteria” . Working hypothesis  Proposal evaluation criteria (whether project would meet identified criterias)  Identified literature

P.S. Approach & Existing work will be discussed later 1. Proposals (ordered by preference)

1.1. Proposal #1 Simple Draft Title “Blind” Robot Navigation Simple Draft Summary Develop a controller that would allow a robot to navigate using haptic/tactile sensors (i.e. in NAO use tactile sensors) to “feel” the surrounding and find its way out. It is unknown how often robot will use its haptic/tactile sensors to progress further. Robot will learn by reinforcement learning algorithm that would allow the robot to progress further. It is interesting to see whether a robot will be able to realise whether it is progressing or not. In addition, it is interesting to see how the robot will be able to cope with the dynamic terrain (i.e. different wall texture). Rationale Be able to utilize alternative sensing abilities in cases when visual/audio/sonar/etc. are either inoperational or cannot be used in a particular environment Evaluation # Aspect

1 Be able to learn/adapt to changing environment utilizing basic (inate) rules

2.1 Hopefully

2.2 Can impose additional constraint – if robot moves slow/does not move – it dies

2.3 Try to investigate Deja Visite (opposite of Defa Visite) phenomena

2.4 Investigate how visual impairnment affects behaviour

2.5 Unknown

3 Do not program perception or behavior in the environment

4.1 Yes

4.2 Yes

4.3 No

5 One possible application: Be able to guide people in hostile environments

6.1 Haptic/tactile 6.2 No

6.3 Inner critic to learn about the environment

6.4 No

6.5 No

Hypothesis Robot will be able to learn about the surroundings using tactile information Approach To be discussed with the supervisor Existing work Some presented in the list of revised literature

1.2. Proposal #2 Simple Draft Title Dancing Robot Simple Draft Summary Develop a controller that allows a robot to learn how to perform sequenced movements, while external critic will give scores to the performance (based on idea of what a good dance is). Idea can be extended to synchronous movements of a robot, immitating a leader (i.e. dancer) Rationale Should be discussed with supervisor Evaluation # Aspect

1 Be able to learn new acceptable (good) dance moves

2.1 Hopefully

2.2 The ones that dance bad – must stop dancing Based on critic response, the robot can be modelled to exhibit shame, which can 2.3 lead to embarrasement. How such result would impact learning? 2.4 Investigate how judgement affects performance

2.5 How a robot would be able to synthesize moves

3 No moves will be hardcoded 4.1 Yes

4.2 Yes

4.3 Yes

5 Be able to synthesize moves based on previously learned patterns

6.1 Unknown

6.2 Unknown (will need additional model for that)

6.3 Yes

6.4 Yes

6.5 Yes

Hypothesis Robot will be able to synthesize complex coherent moves while learning from a critic Approach To be discussed with the supervisor Existing work Some presented in the list of revised literature

1.3. Proposal #3 Implement and test research project done by Christopher Alan “Implementing a Data Mining Approach to Emtional Memory Modelling for RoboCup Players”

In addition: Optimize memory mining techniques to speed up choice of the next behavior. Would not result in the most optimal behavior, but reduce the search space. How will that affect overall reinforcement learning and emotion of a robot? Investigate the affect on the emotion of a robot with a specific personality trait (i.e. pleasure from missing penalties)

1.4. Proposal #4 Simple Draft Title Prediction of behaviour of objects present in an environment based on experience Simple Draft Summary Be able to predict the near future of objects/subjects present in an environment. Such prediction will be generated based on the previous experiences of the agent. The agent will be able prevent or support execution of predicted activities within the environment. Rationale Predictive autonomous operation of an agent in an environment. Evaluation # Aspect

2.1 Hopefully Prediction is important (i.e. be able to distinguish a potentially dangerous 2.2 activity/behaviour) 2.3 Safety feeling in the environment/Fear of unknown environment

2.4 Which aspects of environment a robot will become familiar with faster

2.5 No Unknown which objects/subjects will be studied faster and which ones robot 3 will be most confident about 4.1 Robot will have to interact with object/subject to study it better Robot will have to adopt a model, which would combine object with 4.2 object’s/subject’s possible behaviour and robots attitude towards that object/subject 4.3 Partly (if considering a subject in an environment) Robot will be able to keep environment safe and predictable (prevent or alert of 5 unsafe condition) 6.1 Vision/haptic/tactile/etc.

6.2 Partly

6.3 Machine learning, supervised learning and unsupervised learning could be used

6.4 Yes (navigation through environment)

6.5 Yes (interaction with objects)

Hypothesis Given past, robot can learn to predict objects behaviour Approach To be discussed with the supervisor Existing work Some presented in the list of revised literature

2. Proposal evaluation criteria # Aspect

1 Learn new skills and obtain new knowledge

2 High-level objectives:

2.1 Create useful controller Explore Baldwin effect (evolution by survival [not transfer of genetical 2.2 code]) 2.3 Reproduce psychological phenomena Find out about biological neural network by studying artificial neural 2.4 network 2.5 Understand human body (i.e. body structure)

3 Robot's output is not known a priori (Evolutionary Robotics)

4 Emulate

4.1 Physical

4.2 Cognitive

4.3 Social

aspects of human High-level rationale: act safely along side humans, extending human capacities in 5 task/environment 6 Humanoid robot incorporates:

6.1 Perception (vision/haptic/tactile/sonar/etc.) HRI (human control/tasking of robots in efficient, accurate, convenient 6.2 way) [be able to pick up gestures and/or facial expressions to be able to communicate] Learning&Adaptive behaviour (use existing capabilities [few simple rules] to adapt to the changing environment [complex behaviour]. Be able to learn new tasks [complex behaviour] on the fly using existing capabilities 6.3 [few simple rules]) [Machine learning (-) / Supervised Learning (human trainer) / Unsupervised Learning (built-in critic)] -> should be told WHAT to do, not HOW to do it Legged locomotion (must be able to do complex locomotion [i.e. walk up 6.4 the stairs / walk inclined/declined path / walk uneven terrain]) 6.5 Arm control & skilful manipulations (i.e. catch ball / juggle / etc.)

3. Preliminary Literature Review 3.1.Have Read # Author "Title" Description Connection between cognition & emotion, which can be useful in designing Cynthia Breazeal autonomous robots to operate in complex “Function Meets Style: environments, where human can be working 1 Insights From Emotion along an agent and both will have to Theory Applied to HRI” synchronize their progress partially by emotive expressions, which convey a lot of information in human-human interaction Albert Mehrabian “Pleasure, Arousal, Dominance: A General How to use PAD and its rationale. Can use Framework for PAD to measure and predict such 2 Describing and psychological conditions as anxiety, Measuring Individual depression, panic, empathy, etc. Differences in Temperament” Cognitive stages of development of an infant, which are driven by emotions. Later on emotions become deliberately suppressed so that behavior won’t be lead by the emotions Dolores Canamero any longer – progression in cognitive stages. “Modeling Motivations 3 Another level of perception – existance not and Emotions as a Basis only to survive, but also to maintain a for Intelligent Behavior” ‘happy’ state. Aprirori innate structures that support the initial survival in the environment. Different approaches to learning are mentioned. Important factors: 1. People must feel in Donald Norman “How control; 2. Researchers must be realistic; 3. 4 might people interact Safety&Privacy come first; 4. What is the with agents” approapriate form of Human-Agent interaction. Christopher Allan Model of memory + emotion. Theory: “Implementing a Data emotion affects memory. Test-field: football Mining Approach to 5 field (penalty situation) - DID HE ACTUALLY Emtional Memory PROVED/DISPROVED THAT EMOTION Modelling for RoboCup AFFECTS MEMORY Players” Scott Carson “An Investigation of NAO RoboCup edition robot, Investigation into various tests on it. Information provided 6 developing Search & seem to be outdated (maybe due to the Approach behavior using newer versions of SDK provided with the NAO Robots” robot) Use of Kintect to map the layout of the room Guillaume Carre and find differences between images “Visually Impaired and 7 (prior/posterior). Some good methods Guide Dog Automated mentioned and possibilities for future Helper” improvements. Yutaka Kanayama Robot example with multiple processing "Concurrent 8 units that are parallel. Testing - robot Programming of walking in a maze Intelligent Robots" Christopher Alan, Ruby Implementing a Data Mining Approach to Valverde Iban ̃ez, 9 Emotional Memory Modelling for RoboCup Matthias Keysermann, Players Patricia Vargas

3.2.To Read # Author "Title" 1 Anita Flynn "Redundant Sensors for Mobile Robot Navigation" Georges Giralt et al. "An Integrated Navigation and Motion Control 2 Systems for Autonomous Multisensory Mobile Robots" Micael Couceiro et al. "Marsupial teams of robots: deployment of 3 miniature robots for swarm exploration under communication constraints" Renan Moioli "Neuronal Assembly Dynamics in Supervised and 4 Unsupervised Learning Scenarios" 5 William Bechtel "A companion to cognitive science" 6 Tenreiro Machado, Manuel Silva "An Overview of Legged Robots" 7 Kazuo Tanie "Humanoid and its Potential Applications" Lim, Gi Hyun et al. "Interactive Teaching and Experience Extraction for 7 Learning about Objects and Robot Activities" Ayan Ghosh et al. "Experience of Using a Haptic Interface to Follow a 8 robot without Visual Feedback" Valverede Ibanez, Ruby et al. "Emotional Memories in Autonomous 9 Robots" Jitviriya, Wisanu "Development of Behaviour and Emotion Models for 10 Conbe-I Using a self-organising Map Learning" Angel Fernandez et al. "Studying People's Emotional Responses to 11 Robot's Movements in a Small Scene" Addo, Ivor "Applying Affective Feedback to Reinforcement Learning in 12 ZOEI, a Comic Humanoid Robot" Liu, Lingzhi "A Modified Motion Mapping Method for haptic Device 13 Based Space Teleoperation" Kirandziska, Vesna "A Robot that Perceives Human Emotions and 14 Implications in Human-Robot Interaction" Feldmaier, Johannes "Path-Finding Using Reinforcement Learning and 15 Affective States" 16 Karami, Abir-Beatrice "Learn to adapt based on user's feedback" 17 Lawrence Erickson "Probabilistic localisation with a blind robot"