Integration Framework of Monocular Vision-Based Drivable Region Detection and Contour-Based Vehicle Localization for Autonomous Driving Systems

Keywords: Autonomous vehicle systems, monocular camera, road detection, localization, map matching, region growing, inverse perspective mapping (IPM), iterative closest point (ICP)


Perception and localization are the keys in autonomous vehicle systems and driver assistance systems. The perception provides the information of environments around the vehicle, like other vehicles, pedestrians, and road signs. The localization provides the position and heading of vehicle, which can be used for path planning, navigation. With perception and localization process, the safety of vehicle driving could be increased. In this paper, an image segmentation method called region growing, using threshold estimated from previous indicated road region, is proposed to determine that the pixels in the image belong to road region or not. With a defined initial partial road region, the whole road region can be obtained. On the other hand, with a prior birdeye view map of the area where the vehicle drives, the contours of road region extracted from captured images are matching with the contour on the map by iterative closest point to obtain the vehicle position. In addition, in order to increase the precision of matching, the movements of camera are also estimated by matching the contour in consecutive frames. Furthermore, the position estimated from visual information integrated with the information from GPS to obtain more accurate position. Comparing with vision-based localization only, the integration with GPS reduces the weight and influence of bad matching results, which make the estimated position more accurate. The experimental results show that in structured road, with the localization by road signs, stop lines, and lane lines, the global positions of vehicle can be estimated while the relative movements are very close to GPS data.


Hsu et al. 2009 Chih-Ming Hsu, Fei-Hong Chao, Feng-Li Lian, and Jong-Hann Jean, "Monocular vision-based drivable region labeling using adaptive region growing." in Proceedings of the Society of Instrument and Control Engineers Annual Conference, pp. 2108-2112, Sapporo, Japan, Sep. 9 - 12, 2009.

Álvarez et al. 2013 José M. Alvarez, Theo Gevers, Ferran Diego, and Antonio M. Lopez, “Road Geometry Classification by Adaptive Shape Models,” IEEE Transactions on Intelligent Transportation Systems, vol. 14, no. 1, pp. 459-468, Mar., 2013.

Fritsch et al. 2014 Jannik Fritsch, Tobias Kühnl, and Franz Kummert, “Monocular Road Terrain Detection by Combining Visual and Spatial Information,” IEEE Transactions on Intelligent Transportation Systems, vol. 15, no. 4, pp. 1586-1596, Aug., 2014.

Siogkas and Dermatas 2013 George K. Siogkas, and Evangelos S. Dermatas, “RandomWalker Monocular Road Detection in Adverse Conditions Using Automated Spatiotemporal Seed Selection,” IEEE Transactions on Intelligent Transportation Systems, vol. 14, no. 2, pp. 527-538, Jun., 2013.

Alonso et al. 2012 Ignacio Parra Alonso, David Fernández Llorca, Miguel Gavilan, Sergio Álvarez Pardo, Miguel Ángel Garcia-Garrido, Ljubo Vlacic, and Miguel Ángel Sotelo, “Accurate Global Localization Using Visual Odometry and Digital Maps on Urban Environments,” IEEE Transactions on Intelligent Transportation Systems, vol. 13, no. 4, pp. 1535-1545, Dec., 2012.

Durrant-Whyte and Madhavan 2005 Hugh Durrant-Whyte, and Raj Madhavan, “2D mapbuilding and localization in outdoor environments,” Journal of Robotic Systems, vol. 22, no. 1, pp. 45-63, Jan., 2005.

Hata and Wolf 2016 Alberto Y. Hata, and Denis F. Wolf, “Feature Detection for Vehicle Localization in Urban Environments Using a Multilayer LIDAR,” IEEE Transactions on Intelligent Transportation Systems, vol. 17, no. 2, pp. 420-429, Feb., 2016.

Sivaraman and Trivedi 2013 Sayanan Sivaraman, and Mohan Manubhai Trivedi, “Integrated Lane and Vehicle Detection, Localization, and Tracking: A Synergistic Approach,” IEEE Transactions on Intelligent Transportation Systems, vol. 14, no. 2, pp. 906-917, Jun., 2013.

Cui et al. 2016 Dixiao Cui, Jianru Xue, and Nanning Zheng, “Real-Time Global Localization of Robotic Cars in Lane Level via Lane Marking Detection and Shape Registration,” IEEE Transactions on Intelligent Transportation Systems, vol. 17, no. 4, pp. 1039-1050, Apr., 2016.

Cheng 2011 Hong Cheng, “Autonomous Intelligent Vehicles Theory, Algorithms, and Implementation, “Springer, London, 2011.