A Sensor-Fused Deep Reinforcement Learning Framework for Multi-Agent Decision-Making in Urban Driving Environments
DOI:
https://doi.org/10.71222/g3f6jw09Keywords:
autonomous driving, deep reinforcement learning, sensor fusion, multi-agent system, urban traffic simulationAbstract
Achieving robust and efficient autonomous driving in complex and dynamically changing urban traffic environments faces numerous significant challenges, especially the need to properly handle complex and time-varying interaction behaviors among multiple agents. This study innovatively proposes a sensor-integrated deep reinforcement learning framework (SIDRL), which organically combines multimodal sensor data fusion technology with multi-agent decision-making methods based on policy optimization. The system inputs include data from lidar, cameras and vehicle-to-everything (V2X), which are initially processed through a fusion perception module and subsequently fed into a decision-making network based on proximal policy optimization (PPO) for training and inference. Comprehensive evaluation experiments were conducted on the high-fidelity CARLA 0.9.15 simulation platform, and comparisons were performed with classical deep Q-network (DQN), asynchronous advantage actor-critic (A3C), as well as advanced methods such as soft actor-critic (SAC) and multi-agent proximal policy optimization (MAPPO). The experimental results clearly demonstrate that the proposed method enhances collision avoidance capability by 23.5% and decision-making efficiency by 17.2% under complex urban traffic scenarios. The research outcomes effectively confirm the critical role of multi-sensor fusion within deep reinforcement learning frameworks in improving environmental adaptability and safety for autonomous driving vehicles, providing a valuable new direction for the development of urban autonomous driving technology.
References
1. E. B. Lieberthal, N. Serok, J. Duan, G. Zeng, and S. Havlin, “Addressing the urban congestion challenge based on traffic bottlenecks,” Philos. Trans. A, vol. 382, no. 2285, p. 20240095, 2024, doi: 10.1098/rsta.2024.0095.
2. K. Mo, et al., “Dral: Deep reinforcement adaptive learning for multi-UAVs navigation in unknown indoor environment,” 2024, arXiv preprint arXiv:2409.03930, doi: 10.48550/arXiv.2409.03930.
3. S. Wang, R. Jiang, Z. Wang, and Y. Zhou, “Deep learning-based anomaly detection and log analysis for computer networks,” 2024, arXiv preprint arXiv:2407.05639, doi: 10.48550/arXiv.2407.05639.
4. L. Engelfriet and E. Koomen, “The impact of urban form on commuting in large Chinese cities,” Transportation, vol. 45, no. 5, pp. 1269–1295, 2018, doi: 10.1007/s11116-017-9762-6.
5. C. Gong, X. Zhang, Y. Lin, H. Lu, P. C. Su, and J. Zhang, “Federated learning for heterogeneous data integration and privacy protection,” Comput. Sci. Math., 2025, doi :10.20944/preprints202503.2211.v1.
6. K. Shih, Y. Han, and L. Tan, “Recommendation system in advertising and streaming media: Unsupervised data enhance-ment sequence suggestions,” 2025, arXiv preprint arXiv:2504.08740, doi: 10.48550/arXiv.2504.08740.
7. N. Lopac, I. Jurdana, A. Brnelić, and T. Krljan, “Application of laser systems for detection and ranging in the modern road transportation and maritime sector,” Sensors, vol. 22, no. 16, p. 5946, 2022, doi: 10.3390/s22165946.
8. Q. Bao, Y. Chen, and X. Ji, “Research on evolution and early warning model of network public opinion based on online Latent Dirichlet distribution model and BP neural network,” 2025, arXiv preprint arXiv:2503.03755, doi: 10.48550/arXiv.2503.03755.
9. A. M. Vepa, Z. Yang, A. Choi, J. Joo, F. Scalzo, and Y. Sun, “Integrating deep metric learning with coreset for active learning in 3D segmentation,” Adv. Neural Inf. Process. Syst., vol. 37, pp. 71643–71671, 2024, doi: 10.48550/arXiv.2411.15763.
10. Z. Yang and Z. Zhu, “CuriousLLM: Elevating multi-document QA with reasoning-infused knowledge graph prompting,” 2024, arXiv preprint arXiv:2404.09077, doi:10.48550/arXiv.2404.09077.
11. Z. Li, Q. Ji, X. Ling, and Q. Liu, “A comprehensive review of multi-agent reinforcement learning in video games,” 2025, Authorea Preprints, doi: 10.36227/techrxiv.173603149.94954703/v1.
12. B. R. Kiran, et al., “Deep reinforcement learning for autonomous driving: A survey,” IEEE Trans. Intell. Transp. Syst., vol. 23, no. 6, pp. 4909–4926, 2021, doi: 10.1109/TITS.2021.3054625.
13. W. Zhang, Z. Li, and Y. Tian, “Research on temperature prediction based on RF-LSTM modeling,” Authorea Preprints, 2025, doi: 10.36227/techrxiv.173603336.69370585/v1.
14. Z. Li, “Advances in deep reinforcement learning for computer vision applications,” J. Ind. Eng. Appl. Sci., vol. 2, no. 6, pp. 16–26, 2024, doi: 10.70393/6a69656173.323234.
15. J. Dinneweth, A. Boubezoul, R. Mandiau, and S. Espié, “Multi-agent reinforcement learning for autonomous vehicles: A survey,” Auton. Intell. Syst., vol. 2, no. 1, p. 27, 2022, doi: 10.1007/s43684-022-00045-z.
16. J. Liu, et al., “Application of deep learning-based natural language processing in multilingual sentiment analysis,” Mediterr. J. Basic Appl. Sci. (MJBAS), vol. 8, no. 2, pp. 243–260, 2024, doi: 10.46382/MJBAS.2024.8219.
17. X. Tang, Z. Wang, X. Cai, H. Su, and C. Wei, “Research on heterogeneous computation resource allocation based on da-ta-driven method,” in Proc. 2024 6th Int. Conf. Data-driven Optim. Complex Syst. (DOCS), Aug. 2024, pp. 916–919, IEEE, doi: 10.1109/DOCS63458.2024.10704406.
18. K. Moghaddasi, S. Rajabi, F. S. Gharehchopogh, and A. Ghaffari, “An advanced deep reinforcement learning algorithm for three-layer D2D-edge-cloud computing architecture for efficient task offloading in the Internet of Things,” Sustain. Comput. Inform. Syst., vol. 43, p. 100992, 2024, doi: 10.1016/j.suscom.2024.100992.
19. H. Feng, “The research on machine-vision-based EMI source localization technology for DCDC converter circuit boards,” in Proc. 6th Int. Conf. Inf. Sci., Electr. Autom. Eng. (ISEAE), vol. 13275, pp. 250–255, Sep. 2024, doi: 10.1117/12.3037693.
20. A. Abdulmaksoud and R. Ahmed, “Transformer-based sensor fusion for autonomous vehicles: A comprehensive review,” IEEE Access, 2025, doi; 10.1109/ACCESS.2025.3545032.
21. J. Zhu, J. Ortiz, and Y. Sun, “Decoupled deep reinforcement learning with sensor fusion and imitation learning for auton-omous driving optimization,” in Proc. 2024 6th Int. Conf. Artif. Intell. Comput. Appl. (ICAICA), Nov. 2024, pp. 306–310, doi: 10.1109/ICAICA63239.2024.10823066.
22. J. Zhu, Y. Sun, Y. Zhang, J. Ortiz, and Z. Fan, “High fidelity simulation framework for autonomous driving with augmented reality based sensory behavioral modeling,” in IET Conf. Proc. CP989, vol. 2024, no. 21, pp. 670–674, Oct. 2024, doi: 10.1049/icp.2024.4298.
23. A. Guo, J. Huang, C. Lv, L. Chen, and F. Y. Wang, “Advances in autonomous vehicle testing: The state of the art and future outlook on driving datasets, simulators and proving grounds,” 2024, Authorea Preprints, doi: 10.22541/au.172446843.37817639/v1.
24. Y. Sun, N. S. Pargoo, P. J. Jin, and J. Ortiz, “Optimizing autonomous driving for safety: A human-centric approach with LLM-enhanced RLHF,” 2024, arXiv preprint arXiv:2406.04481, doi: 10.1145/3675094.3677588.
25. J. Yang, T. Chen, F. Qin, M. S. Lam, and J. A. Landay, “HybridTrak: Adding full-body tracking to VR using an off-the-shelf webcam,” in Proc. 2022 CHI Conf. Human Factors Comput. Syst., Apr. 2022, pp. 1–13, doi: 10.1145/3491102.3502045.
26. D. Luo, J. Gu, F. Qin, G. Wang, and L. Yao, “E-seed: Shape-changing interfaces that self drill,” in Proc. 33rd Annu. ACM Symp. User Interface Softw. Technol., Oct. 2020, pp. 45–57, doi: 10.1145/3379337.3415855.
27. A. I. Sabbir, “Comparative analysis of double deep Q-network and proximal policy optimization for lane-keeping in au-tonomous driving,” Probl. Inf. Soc., pp. 12–25, 2025, doi: 10.30574/ijsra.2024.13.2.2237.
28. Y. Yao, “Design of neural network-based smart city security monitoring system,” in Proc. 2024 Int. Conf. Comput. Multimedia Technol., May 2024, pp. 275–279, doi: 10.1145/3675249.3675297.
