One way in which real animals are different from most computational agents is that they are motivated by basic drives like hunger and thirst. These drives mostly determine where animals go and what they do. Furthermore, the senses and behaviors of a biological organism are designed for and oriented towards the satisfaction of these drives, at least once basic survival is accounted for.
Yet most AI or robotics research begins with the assumption that an agent has a task rather than a set of drives. In a way, an agent trained for a task could be considered driven to complete that task, but there is a stark difference between task-driven behavior and motivation-driven behavior: the sensors and actuators of biological organisms are designed to satisfy drives, whereas we often train agents to use tools that are not easily adapted to tasks. What is really happening is that we want general-purpose agents that can solve many tasks using a complex set of tools. However, from a design perspective, it might be better to start from the ground up, building agents that can add complexity layer by layer. And the bottom layer, then, might be some set of basic motivations to drive the agent that are tightly connected to its physical form.
This repository is about building a model of that bottom layer. For that purpose, we place a simple agent in a simulated world containing virtual trees. Each tree has fruit that may be edible or poisonous. Our agent has to stay alive. It has sensors that help it locate the trees, but its sense of space must be inferred from the sensor data, mediated through its own actions.
The goal is to develop a simple controller for the agent that can keep the agent alive. Rather than focusing on complex action plans, the agent merely has to approach edible fruit trees and avoid poisonous fruit trees. If the agent is not hungry, then it should explore the space to build a map that it can use for finding fruit when it becomes hungry.
It is my belief that this basic system can be gradually levered up to generate hierarchical action plans for both concrete and abstract space. The core ideas driving this belief are discussing in my blog, Embodied Language and Cognition. The essence, though, is that once we have a map, we can build multiple maps for different spaces and connect them at various juncture points. These juncture points induce a higher-level map, and abstract movement within the higher-level map can be concretized into a plan in the lower-level map. This process can be continued, producing ever more abstract maps, and the spatial management techiques can then be mapped onto other problems, such as tool use or social cognition. The idea of mapping to other problems is naturally vague, as I have yet to do the work that would make demonstrate their feasibility. That is the work I am starting with here in a basic setting.
The notebooks below go through this work step by step, setting up the simulation environment, defining the map, showing how to train the map with basic agents, then eventually producing a controller that uses the map for movement and homeostatic drive management.
The next steps after this is to look at maps with a one-layer hierarchy. But that is a problem for later.