Theoretical and Empirical Investigations of Active Learning and Multi-agent Learning in Autonomous Vehicles

ABOUT THIS PROJECT

At a glance

Safety in AI systems has been a central problem, especially in the environment of autonomous driving, where the potential for accidents or injuries is high. The current autonomous driving systems are built upon either pre-programmed rules or neural networks that are trained from pre-collected datasets. Such driving system can be fragile when faced with environments not covered in previous learned rules or collected dataset. The deep neural networks responsible for self-driving functions require exhaustive training, both in situations they’re likely to encounter during daily trips, as well as unusual ones they’ll hopefully never come across.
To robustify the driving system, one of the most straightforward ways is \textit{active learning}, which searches for rare cases that the current system is uncertain about, and further trains the model specifically on the spotted rare cases with human labels. This calls for the design of active query algorithms that can efficiently find the rare cases.

On the other hand, the mainstream autonomous driving systems are based on single-agent algorithms, which first predicts the other driver's behavior, and plans according to the prediction. Such model is limited when the interactions between the autonomous vehicle and the other vehicles become more complicated. In reality, the interactions may happen in multiple rounds. At the beginning of each round, the ego vehicle first takes actions based on the past observations and future predictions. Then the other drivers respond to the ego vehicle's behavior by taking actions that maximizes their own (possibly long-term) reward functions. Different from the single agent view, such multi-agent multi-round model accounts for the possibility of the other drivers adjusting their strategy according to the ego driver's behavior. Such multi-round interactions can be characterized as a multi-agent sequential Stackelberg game, and calls for new framework of training pipeline.

In this proposal, we aim to both empirically and theoretically develop principled and efficient algorithms for active learning and multi-agent learning in Stackelberg game. For active learning, we hope to combine the insights from the new theoretically-motivated online learning algorithms with the applications in autonomous driving. For Stackelberg game, we hope to propose new policy training framework that incorporates the multi-round interactions in the real-world driving scenario.