|Table of Contents|

Vehicle spatial morphology estimation in road scene under monocular camera(PDF)

长安大学学报（自然科学版）[ISSN:1006-6977/CN:61-1281/TN]

Issue:
2023年2期
Page:
100-110
Research Field:
交通工程
Publishing date:

Info

Title:
Vehicle spatial morphology estimation in road scene under monocular camera
Author(s):
WANG Wei;  TANG Xin-yao;  ZHAO Chun-hui;  LI Ying;  CUI Hua
(School of Information Engineering, Chang’an University, Xi’an 710064, Shaanxi, China)
Keywords:
traffic engineering;  vehicle spatial morphology estimation;  monocular 3D;  vehicle 3D information in road scene;  automatic calibration;  3D-2D projection constraint
PACS:
U495
DOI:
10.19721/j.cnki.1671-8879.2023.02.010
Abstract:
Due to the limitation of projective geometry, it was difficult for monocular camera to obtain accurate 3D point cloud and scale for the three-dimensional structure of the object. To solve this problem, a method of vehicle spatial morphology estimation based on monocular traffic camera was proposed. Firstly, an automatic calibration model of the road scene was established to obtain the 3D-2D projection mapping and scale information. Based on the “diamond space” method, the horizon line of the scene could be accurately obtained by vehicle trajectories and edges. Then, the geometric model of the vehicle spatial morphology could be jointly constructed with calibration information and vanishing point constraints. Secondly, the projection constraints of the vehicle were extracted from the image, including sequences of vehicle contour constraints and vehicle edge constraints. Based on these constraints, the error constraint function could be derived to estimate the projection errors of the vehicle spatial morphology. Finally, according to the initial vehicle recognition results and prior information, the parameter constraint space could be iteratively optimized according to the error constraint function, and the accurate vehicle spatial morphology information could be obtained. The experiments were validated on the public dataset BrnoCompSpeed and videos were collected from actual roads. The proposed method was also compared with similar methods. The results show that the proposed method is strongly adaptive to various road scenes with an accuracy of more than 94% for 3D vehicle size estimation, which requires few prior conditions. In the meanwhile, real-time vehicle spatial position and deflection angle relative to the road plane can be estimated with a comprehensive accuracy of more than 92% for vehicle spatial morphology estimation and a process speed of less than 0.5 seconds for a single frame with several vehicles. Moreover, compared with existing methods, the proposed method is more suitable for vehicle spatial morphology estimation by using surveillance cameras in road scenes.8 tabs, 10 figs, 35 refs.

References:

[1] NOVAK L.Vehicle detection and pose estimation for autonomous driving[D].Czech:Czech Technical University in Prague,2017.
[2]GUPTA I,RANGESH A,TRIVEDI M.3D bounding boxes for road vehicles:A one-stage,localization prioritized approach using single monocular images[C]//ECCV.Proceedings of the European Conference on Computer Vision(ECCV).Cham:Springer,2018:1-16.
[3]CHEN X Z,KUNDU K,ZHANG Z Y,et al.Monocular 3D object detection for autonomous driving[C]//IEEE.Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition(CVPR).New York:IEEE,2016:2147-2156..
[4]PENG T,LIU X,FANG R,et al.Lane-change path planning and control method for self-driving articulated trucks[J].Journal of Intelligent and Connected Vehicles,2020,3(2):49-66.
[5]HE H S,SHAO Z Z,TAN J D.Recognition of car makes and models from a single traffic-camera image[J].IEEE Transactions on Intelligent Transportation Systems,2015,16(6):3182-3192.
[6]LAN J H,LI J,HU G D,et al.Vehicle speed measurement based on gray constraint optical flow algorithm[J].Optik,2014,125(1):289-295.
[7]QI C R,LIU W,WU C X,et al.Frustum PointNets for 3D object detection from RGB-D data[C]//IEEE.Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition(CVPR).New York:IEEE,2018:918-927.
[8]AHMED S M,CHEW C M.Density-based clustering for 3D object detection in point clouds[C]//IEEE.Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition(CVPR).New York:IEEE,2020:10605-10614.
[9]ZHOU Y,TUZEL O.VoxelNet:End-to-end learning for point cloud based 3D object detection[C]//IEEE.Proceedings of the Conference on Computer Vision and Pattern Recognition.New York:IEEE,2018:4490-4499.
[10]KU J,MOZIFIAN M,LEE J,et al.Joint 3D proposal generation and object detection from view aggregation[C]//IEEE.Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems(IROS).New York:IEEE,2018:1-8.
[11]SHI S S,GUO C X,JIANG L,et al.PV-RCNN:Point-voxel feature set abstraction for 3D object detection[C]//IEEE.Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition(CVPR).New York:IEEE,2020:10526-10535.
[12]NAJIBI M,LAI G D,KUNDU A,et al.DOPS:Learning to detect 3D objects and predict their 3D shapes[C]//IEEE.Proceedings of the Conference on Computer Vision and Pattern Recognition(CVPR).New York:IEEE,2020:11910-11919.
[13]SHI S S,WANG X G,LI H S.PointRCNN:3D object proposal generation and detection from point cloud[C]//IEEE.Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition(CVPR).New York:IEEE,2019:770-779.
[14]CHEN X Z,KUNDU K,ZHU Y K,et al.3D object proposals using stereo imagery for accurate object class detection[J].IEEE Transactions on Pattern Analysis and Machine Intelligence,2018,40(5):1259-1272.
[15]LI P L,CHEN X Z,SHEN S J.Stereo R-CNN based 3D object detection for autonomous driving[C]//IEEE.Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition(CVPR).New York:IEEE,2019:7636-7644.
[16]CAETANO T S,CAELLI T,SCHUURMANS D,et al.Graphical models and point pattern matching[J].IEEE Transactions on Pattern Analysis and Machine Intelligence,2006,28(10):1646-1663.
[17]ROMANONI A,SORRENTI D G,MATTEUCCI M.Backward-simulation particle smoother with a hybrid state for 3D vehicle trajectory,class and dimension simultaneous estimation[J].Machine Vision and Applications,2015,26(2):369-385.
[18]ZHANG Z X,TAN T N,HUANG K Q,et al.Three-dimensional deformable-model-based localization and recognition of road vehicles[J].IEEE Transactions on Image Processing,2012,21(1):1-13.
[19]CORRAL-SOTO E R,ELDER J H.Slot cars:3D modelling for improved visual traffic analytics[C]//IEEE.Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition(CVPR).New York:IEEE,2017:889-897.
[20]SOCHOR J,SPANHEL J,HEROUT A.BoxCars:Improving fine-grained recognition of vehicles using 3D bounding boxes in traffic surveillance[J].IEEE Transactions on Intelligent Transportation Systems,2019,20(1):97-108.
[21]YANG S C,SCHERER S.CubeSLAM:Monocular 3D object SLAM[J].IEEE Transactions on Robotics,2019,35(4):925-938.
[22]HE T,SOATTO S.Mono3d++:Monocular 3d vehicle detection with two-scale 3d hypotheses and task priors[C]//AAAI.Proceedings of the AAAI Conference on Artificial Intelligence.Menlo Park:AAAI Press,2019:8409-8416.
[23]QIN Z Y,WANG J L,LU Y.MonoGRNet:A geometric reasoning network for monocular 3D object localization[C]//AAAI.Proceedings of the AAAI Conference on Artificial Intelligence.Menlo Park:AAAI Press,2019:8851-8858.
[24]DING M Y,HUO Y Q,YI H W,et al.Learning depth-guided convolutions for monocular 3D object detection[C]// IEEE.Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition(CVPR).New York:IEEE,2020:11669-11678.
[25]XU B,CHEN Z Z.Multi-level fusion based 3D object detection from monocular images[C]//IEEE.Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition(CVPR).New York:IEEE,2018:2345-2353.
[26]MOUSAVIAN A,ANGUELOV D,FLYNN J,et al.3D bounding box estimation using deep learning and geometry[C]//IEEE.Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition(CVPR).New York:IEEE,2017:5632-5640.
[27]LI B Y,OUYANG W L,SHENG L,et al.GS3D:An efficient 3D object detection framework for autonomous driving[C]//IEEE.Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition(CVPR).New York:IEEE,2019:1019-1028.
[28]王 伟,张朝阳,唐心瑶,等.道路场景下相机自动标定及优化算法[J].计算机辅助设计与图形学学报,2019,31(11):1955-1962.
WANG Wei,ZHANG Chao-yang,TANG Xin-yao,et al.Automatic self-calibration and optimization algorithm of traffic camera in road scene[J].Journal of Computer-Aided Design & Computer Graphics,2019,31(11):1955-1962.
[29]KANHERE N K,BIRCHFIELD S T.A taxonomy and analysis of camera calibration methods for traffic monitoring applications[J].IEEE Transactions on Intelligent Transportation Systems,2010,11(2):441-452.
[30]张润初,杜倩云,俞祝良,等.一种利用参考图象与路面信息的道路监控摄像机标定方法[J].公路交通科技,2014,31(11):137-141.
ZHANG Run-chu,DU Qian-yun,YU Zhu-liang,et al.A calibration method for road monitoring cameras exploiting reference images and roadway information[J].Journal of Highway and Transportation Research and Development,2014,31(11):137-141.
[31]DUBSKA M,HEROUT A.Real projective plane mapping for detection of orthogonal vanishing points[C]//BMVC.Proceedings of the British Machine Vision Conference.Cambridge:British Machine Vision Association,2013:1-10.
[32]BOLYA D,ZHOU C,XIAO F Y,et al.YOLACT:Real-time instance segmentation[C]//IEEE.Proceedings of the IEEE International Conference on Computer Vision(ICCV).New York:IEEE,2019:9156-9165.
[33]HE K M,GKIOXARI G,DOLLAR P,et al.Mask R-CNN[C]//IEEE.Proceedings of the IEEE International Conference on Computer Vision.New York:IEEE,2017:2980-2988.
[34]GROMPONE VON G R,JAKUBOWICZ J,MOREL J M,et al.LSD:A fast line segment detector with a false detection control[J].IEEE Transactions on Pattern Analysis and Machine Intelligence,2010,32(4):722-732.
[35]SOCHOR J,JURANEK R,SPANHEL J,et al.Comprehensive data set for automatic single camera visual speed measurement[J].IEEE Transactions on Intelligent Transportation Systems,2019,20(5):1633-1643.

Memo

Memo:
-
Last Update: 2023-03-30