Working as a Surdna Undergraduate Research Fellow, Oliver spent the summer of 2005 at Bowdoin working with Professor Stephen Majercik on autonomous, event-based state creation and event-driven learning in an artificial agent.
An artificial agent is usually created to operate in a very particular environment, with a human-designed set of appropriate states and actions, and is helpless if put down in a very different environment. In the last few years, Rich Sutton, a leading researcher in artificial intelligence, has been arguing that we need to turn over more of the responsibility *for* the agent, e.g. creating environment-appropriate states and actions, *to* the agent.
Oliver is exploring this idea using the Computer Science Department's Sony Aibo robots. He is developing a framework within which an Aibo can construct its own knowledge, grounded in its experience. Ideally, the robot will be able to extract significant events from the raw sensor data of its experience and construct a useful notion of state---what is relevant to its action decisions---based on these events. The robot should then be able to gather data about how its actions affect these event-based states and learn how to act in its environment. If, after learning how to operate in a particular environment, the robot finds itself in a completely new environment, it can follow the same process to learn about that new environment, rather than doing nothing because the states and actions provided by the human designer don't make sense in the new environment.
During the summer, Oliver implemented ARAVQ (Adaptive Resource Allocating Vector Quantizer), an event extraction and data classification algorithm recently developed by Fredrik Linaker, and developed a framework to explore autonomous state creation and event-driven learning. ARAVQ can be applied to a variety of tasks; in this case, event extraction on a data stream from a robot. As ARAVQ is fed a stream of input vectors, it develops a codebook of model vectors that modify themselves slightly over time to more closely match the dataset being read. After the algorithm buffers a certain number of input vectors, it begins to create and/or modify model vectors, eventually creating a smaller representation of what may be a relatively "noisy" data set. ARAVQ takes a set of input parameters that can adjust model vectors based on stability (how much weight we want to give to past events), novelty (how much weight we want to give to new events), and the learning rate (the magnitude of the changes we make during the learning process). This makes ARAVQ an extremely general event extractor which can be applied to a large variety of tasks.
Oliver is continuing his work in this area during the 2005-06 academic year. In addition to exploring the general potential of this approach, he hopes to apply this system to the data that Bowdoin's Aibo soccer team, the Northern Bites, gathers during the course of a game. The tremendous amount of data generated could potentially be simplified by ARAVQ into a smaller set of model vectors to describe general events on the soccer field and these events could be the building blocks of a set of strategies that would be useful at the yearly Robocup tournament. Oliver will also be working on developing a GUI (Graphical User Interface) for the system that would make it easier to use and would allow introductory computer science students to explore important computer science concepts in a fun and engaging context.