Tractable Large-Scale Deep Reinforcement Learning, CVIU 2023

Our paper Tractable Large-Scale Deep Reinforcement Learning has been accepted for publication as a journal paper by Computer Vision and Image Understanding. The work is co-authored by Nima Sarang and Charalambos Poullis. A preprint is available at https://papers.ssrn.com/sol3/papers.cfm?abstract_id=4121080

Abstract: Reinforcement learning (RL) has emerged as one of the most promising and powerful techniques in deep learning. The training of intelligent agents requires a myriad of training examples which imposes a substantial computational cost. Consequently, RL is seldom applied to real-world problems and historically has been limited to computer vision tasks, similar to supervised learning. This work proposes an RL framework for complex, partially observable, large-scale environments. We introduce novel techniques for tractable training on commodity GPUs, and significantly reduce computational costs. Furthermore, we present a self-supervised loss that improves the learning stability in applications with a long-time horizon, shortening the training time. We demonstrate the effectiveness of the proposed solution on the application of road extraction from high-resolution satellite images. We present experiments on satellite images of fifteen cities that demonstrate comparable performance to state-of-the-art methods. To the best of our knowledge, this is the first time RL has been applied for extracting road networks. The code is publicly available at https://github.com/nsarang/road-extraction-rl