|Table of Contents|

Spatio-temporal axle localization method based on YOLOv5 DeepSORT and virtual detection area(PDF)

长安大学学报（自然科学版）[ISSN:1006-6977/CN:61-1281/TN]

Issue:
2023年3期
Page:
34-44
Research Field:
桥梁与隧道工程
Publishing date:

Info

Title:
Spatio-temporal axle localization method based on YOLOv5 DeepSORT and virtual detection area
Author(s):
QIAO Peng1;  YUAN Biao1;  SHEN Ying-gang1;  DUAN Chang-jiang1;  DI Jin2
(1. School of Civil Engineering, Chang'an University, Xi'an 710061, Shaanxi, China; 2. School of Civil Engineering, Chongqing University, Chongqing 400045, China)
Keywords:
bridge engineering;  vehicle axle localization;  multi-objective tracking;  load indentification;  virtual detection region;  semi-automatic labeling
PACS:
U446
DOI:
10.19721/j.cnki.1671-8879.2023.03.004
Abstract:
In order to obtain the distribution of car axles on road and bridge, the method of spatio-temporal localization of car axles in surveillance video was studied based on YOLOv5 DeepSORT machine vision technology. First, based on the characteristics of multi-scale and small targets of axles in surveillance video, a semi-automatic image annotation method based on Faster R-CNN algorithm was proposed to quickly construct the axle target detection dataset. The YOLOv5 algorithm was used to detect the axle targets in the video, and the performance of YOLOv5 series algorithms was evaluated. Then, a virtual detection area was proposed to be set up in the video monitoring area, first the position and state of the axle target using Kalman filtering algorithm, and then matching the axle targets in the front and rear frames were predicted, using the re-identification algorithm, Hungarian algorithm and cascade matching method, respectively, to complete the axle multi-target tracking based on DeepSORT algorithm and generate the axle trajectory. The multi-target tracking results, combined with direct linear transformation and position presumption based on uniform speed assumption were used, and the spatio-temporal localization of all axles on the bridge was achieved. Finally, the spatio-temporal localization of all axles on the bridge was achieved by using the multi-objective tracking results combined with direct linear transformation and position presumption based on uniform speed.The results show that the YOLOv5s6 model has the best performance in target detection, with 96.42% accuracy and 19.2 ms per frame detection time, which has high accuracy and faster detection speed for axles. In multi-objective tracking, the multi-objective tracking method based on virtual detection area and YOLOv5 DeepSORT has better detection and tracking effect, compared with no virtual, the MOTA and IDF1 are improved by 14.7% and 10.1%, respectively, and the IDS is reduced by 108 times. The axle localization method based on YOLOv5 DeepSORT and virtual detection zone can accurately detect and locate the axle in the cases of axle occlusion, small target and driving out of the detection area, which can provide accurate position information for moving load identification and also provide a new idea for axle information detection in dynamic weighing method of bridges.4 tabs, 11 figs, 29 refs.

References:

[1] 安家禾,赵 华,马鹏飞,等.应用于OSD的两种轴重识别算法对比分析[J].中外公路,2022,42(4):132-138.
AN Jia-he,ZHAO Hua,MA Peng-fei,et al.Comparative analysis of two axle load identification algorithms apply on OSD[J].Journal of China & Foreign Highway,2022,42(4):132-138.
[2]YU Y,CAI C,DENG L.State-of-the-art review on bridge weigh-in-motion technology[J].Advances in Structural Engineering,2016,19(9):1514-1530.
[3]任伟新,左小晗,王宁波,等.非路面式桥梁动态称重研究综述[J].中国公路学报,2014,27(7):45-53.
REN Wei-xin,ZUO Xiao-han,WANG Ning-bo,et al.Review of non-pavement bridge weigh-in-motion[J].China Journal of Highway and Transport,2014,27(7):45-53.
[4]REDMON J,FARHADI A.YOLOv3:An incremental improvement[J].ArXiv E-prints,2018,1804:1-6.
[5]REDMON J,FARHADI A.YOLO9000:Better,faster,stronger[C]//IEEE.Proceedings of 2017 IEEE Conference on Computer Vision and Pattern Recognition(CVPR).New York:IEEE,2017:6517-6525.
[6]REDMON J,DIVVALA S,GIRSHICK R,et al.You only look once:Unified,real-time object detection[C]//IEEE.Proceedings of 2016 IEEE Conference on Computer Vision and Pattern Recognition(CVPR).New York:IEEE,2016:779-788.
[7]LIU W,ANGUELOV D,ERHAN D,et al.SSD:Single shot MultiBox detector[M]//LEIBE B,MATAS J,SEBE N,et al.Computer Vision-ECCV 2016.Cham:Springer,2016:21-37.
[8]LI Z,ZHOU F.FSSD:Feature fusion single shot MultiBox detector[EB/OL].(2017-12-04)[2023-04-06].https://arxiv.org/abs/1712.00960.
[9]FU C,LIU W,RANGA A,et al.DSSD:Deconvolutional single shot detector[EB/OL].(2017-01-23)[2023-04-06].https://arxiv.org/abs/1701.06659.
[10]GIRSHICK R,DONAHUE J,DARRELL T,et al.Rich feature hierarchies for accurate object detection and semantic segmentation[C]//IEEE.Proceedings of 2014 IEEE Conference on Computer Vision and Pattern Recognition.New York:IEEE,2014:580-587.
[11]GIRSHICK R.Fast R-CNN[C]//IEEE.Proceedings of 2015 IEEE International Conference on Computer Vision(ICCV).New York:IEEE,2016:1440-1448.
[12]REN S Q,HE K M,GIRSHICK R,et al.Faster R-CNN:Towards real-time object detection with region proposal networks[J].IEEE Transactions on Pattern Analysis and Machine Intelligence,2017,39(6):1137-1149.
[13]邓 露,何 维,俞 扬,等.公路车-桥耦合振动的理论和应用研究进展[J].中国公路学报,2018,31(7):38-54.
DENG Lu,HE Wei,YU Yang,et al.Research progress in theory and applications of highway vehicle-bridge coupling vibration[J].China Journal of Highway and Transport,2018,31(7):38-54.
[14]CATBASF N,ZAURIN R,GUL M,et al.Sensor networks,computer imaging,and unit influence lines for structural health monitoring:Case study for bridge load rating[J].Journal of Bridge Engineering,2012,17(4):662-670.
[15]XIA Y,JIAN X D,YAN B,et al.Infrastructure safety oriented traffic load monitoring using multi-sensor and single camera for short and medium span bridges[J].Remote Sensing,2019,11(22):2651.
[16]JIAN X D,XIA Y,LOZANO-GALANT J A,et al.Traffic sensing methodology combining influence line theory and computer vision techniques for girder bridges[J].Journal of Sensors,2019,2019:1-15.
[17]CHEN Z,LI H,BAO Y,et al.Identification of spatio-temporal distribution of vehicle loads on long-span bridges using computer vision technology[J].Computers,Networks & Communications,2016,23(3):517-534.
[18]ZHOU Y,PEI Y L,LI Z W,et al.Vehicle weight identification system for spatiotemporal load distribution on bridges based on non-contact machine vision technology and deep learning algorithms[J].Measurement,2020,159:107801.
[19]DAN A H,GE X F,YAN X F,et al.Identification of moving loads based on the information fusion of weigh-in-motion system and multiple camera machine vision[J].Measurement,2019,144:155-166.
[20]易雨时,乔可鑫,张 路.基于机器视觉技术和深度学习算法的桥梁荷载时空分布识别系统[C]//世界交通运输大会.世界交通运输工程技术论坛(WTC2021)论文集(上).北京:人民交通出版社股份有限公司,2021:726-738.
YI Yu-shi,QIAO Ke-xin,ZHANG Lu.Bridge load spatio-temporal distribution recognition system based on machine vision technology and deep learning algorithm[C]// World Transport Congress.Proceedings of the World Transportation Engineering Forum(WTC2021)(Previous).Beijing:China Communication Press Co., Ltd.,2021:726-738.
[21]ZHANG B,ZHOU L M,ZHANG J.A methodology for obtaining spatiotemporal information of the vehicles on bridges based on computer vision[J].Computer-Aided Civil and Infrastructure Engineering,2019,34(6):471-487.
[22]夏 烨,简旭东,邓 露,等.交通视频辅助的桥梁动态称重方法研究[J].中国公路学报,2021,34(12):104-114.
XIA Ye,JIAN Xu-dong,DENG Lu,et al.Research on traffic-video-aided bridge weigh-in-motion approach[J].China Journal of Highway and Transport,2021,34(12):104-114.
[23]ZHAO D D,HE W,DENG L,et al.Trajectory tracking and load monitoring for moving vehicles on bridge based on axle position and dual camera vision[J].Remote Sensing,2021,13(23):4868.
[24]柏 强,邵宇麒,蒙思源,等.基于视频的机场出发层违规接客车辆识别方法[J].长安大学学报(自然科学版),2022,42(4):73-86.
BAI Qiang,SHAO Yu-qi,MENG Si-yuan,et al.Algorithm for identifying violation behavior of vehicles of picking up passengers at airport departure floor based on surveillance video[J].Journal of Chang'an University(Natural Science Edition),2022,42(4):73-86.
[25]石岩青,常彩霞,刘小红,等.面阵相机内外参数标定方法及进展[J].激光与光电子学进展,2021,58(24):9-29.
SHI Yan-qing,CHANG Cai-xia,LIU Xiao-hong,et al.Calibration methods and progress for internal and external parameters of area-array camera[J].Laser & Optoelectronics Progress,2021,58(24):9-29.
[26]韩学源,金先龙,张晓云,等.基于视频图像与直接线性变换理论的车辆运动信息重构[J].汽车工程,2012,34(12):1145-1149.
HAN Xue-yuan,JIN Xian-long,ZHANG Xiao-yun,et al.Vehicle movement information reconstruction based on video images and DLT theory[J].Automotive Engineering,2012,34(12):1145-1149.
[27]ABDEL-AZIZ Y I,KARARA H M,HAUCK M.Direct linear transformation from comparator coordinates into object space coordinates in close-range photogrammetry[J].Photogrammetric Engineering & Remote Sensing,2015,81(2):103-107.
[28]YEH C H,LIN M H,CHANG P C,et al.Enhanced visual attention-guided deep neural networks for image classification[J].IEEE Access,2020,8:163447-163457.
[29]ZHENG L,SHEN L Y,TIAN L,et al.Scalable person re-identification:A benchmark[C]//IEEE.Proceedings of 2015 IEEE International Conference on Computer Vision(ICCV).New York:IEEE,2016:1116-1124.

Memo

Memo:
-
Last Update: 2023-06-30