Safe-D: Safety through Disruption

Cooperative Perception of Connected Vehicles for Safety


In cooperative perception, reliably detecting surrounding objects and communicating the information between vehicles is necessary for safety. However, vehicle-to-vehicle transmission of huge datasets or images can be computationally expensive and often not feasible in real time. A robust approach to ensure cooperation involves relative pose estimation between two vehicles sharing a common field of view. Detecting the object and transferring its location information in real time is necessary when the object is not in the ego vehicle’s field of view. In such scenarios, reliable and robust pose recovery of the object at each instant ensures the ego vehicle accurately estimates its trajectory. Once pose recovery is established, the object’s location information can be obtained for future trajectory prediction. Deterministic predictions provide only point estimates of future states which is not trustworthy under dynamic traffic scenarios. Estimating the uncertainty associated with the predicted states with a certain level of confidence can lead to robust path planning. This study proposed quantifying this uncertainty during forecasting using stochastic approximation, which deterministic approaches fail to capture. The current method is simple and applies Bayesian approximation during inference to standard neural network architectures for estimating uncertainty. The predictions between the probabilistic neural network models were compared with the standard deterministic models. The results indicate that the mean predicted path of probabilistic models was closer to the ground truth when compared with the deterministic prediction. The study has been extended to multiple datasets, providing a comprehensive comparison for each model.

Project Highlights

  • This project developed an end-to-end prediction algorithm that can take raw noisy sensor data and forecast the trajectory of a dynamic object.
  • This project showed that probabilistic prediction of trajectory is more accurate than deterministic future state prediction with mean predicted path closer to ground truth during probabilistic forecasting.
  • This project compared numerous feature matching algorithms and showed SIFT and ORB are the two best algorithms for cooperative perception and relative pose estimation.

Final Report

05-115 Final Report

EWD & T2 Products

This project funded a PhD student in Mechanical Engineering at Virginia Tech. Their Student Impact Statement is Student Impact statement-05-115.

“Cooperative Prediction of Vulnerable Road Users” Presentation available here. This can be used for relevant Mechanical Engineering courses.

Eskandarian, A., “Advanced Methods and Algorithms for Selected Connected Autonomous Vehicles Benefits,” IEEE ITS Society and Virginia Tech Sponsored Workshop on Connected and Autonomous Vehicles Impact on Safety, Mobility, Traffic, and Environment: Advances and Challenges,” Tysons Corner, VA, Oct. 27-28, 2022


Pedestrian Future Trajectory forecasting using Bayesian Approximation. (2022). Annual Walter O’Brien Symposium at Virginia Tech

Pedestrian Trajectory Forecasting Using Deep Ensembles Under Sensing Uncertainty. (2023). Annual Walter O’Brien Symposium at Virginia Tech

Wu, X., and Eskandarian, A., “Motion Planning of Autonomous Vehicles Under Dynamic Traffic Environment in Intersections Using Probabilistic RRT,” in SAE International Journal of Connected and Automated Vehicles, 2021.

Sun, C., and Eskndarian, A., (2021). “Predictive Frontal and Oblique Collision Mitigation System for Autonomous Vehicles,” ASME Letters in Dynamic Systems and Control, 1(4), p.041012.

Nayak, A., A. Eskandarian, and Z. Doerzaph, Uncertainty Estimation of Pedestrian Future Trajectory Using Bayesian Approximation. IEEE Open Journal of Intelligent Transportation Systems, Vol. 3, 2022, pp. 617-630.

Ghorai, P., A. Eskandarian, Y.-K. Kim, and G. Mehr. State Estimation and Motion Prediction of Vehicles and Vulnerable Road Users for Cooperative Autonomous Driving: A Survey. IEEE Transactions on Intelligent Transportation Systems, 23, No. 10, 2022. pp. 16983-17002.

Nayak, A., A. Eskandarian, P. Ghorai, and Z. Doerzaph. A Comparative Study on Feature Descriptors for Relative Pose Estimation in Connected Vehicles. ASME International Mechanical Engineering Congress and Exposition, 85628, American Society of Mechanical Engineers, 2021, p. V07BT07A022.

Ghorai, P., A. Eskandarian, A. Nayak, and Z. Doerzaph. Relative Pose Estimation for Cooperative Vehicles and Tracking of Multiple Dynamic Objects. In ASME International Mechanical Engineering Congress and Exposition, Vol. 85628, American Society of Mechanical Engineers, 2021, p. V07BT07A017.

Final Dataset

The final datasets for this project are located in the Safe-D Collection on the VTTI Dataverse; DOI: 10.15787/VTT1/BTEWCS

Research Investigators (PI*)

Azim Eskandarian(VTTI)*
Prasenjit Ghorai (VTTI/VT)
Anshul Nayak (VT/Student)

Project Information

Start Date: 2020-10-01
End Date: 2023-05-10
Status: Completed
Grant Number: 69A3551747115
Total Funding: $290,895
Source Organization: Safe-D National UTC
Project Number: 05-115

Safe-D Theme Areas

Automated Vehicles
Connected Vehicles

Safe-D Application Areas

Vulnerable Users
Driver Factors and Interfaces
Vehicle Technology

More Information

UTC Project Information Form

Sponsor Organization

Office of the Assistant Secretary for Research and Technology
University Transportation Centers Program
Department of Transportation
Washington, DC 20590 United States

Performing Organization

Virginia Polytechnic Institute and State University
Virginia Tech Transportation Institute
3500 Transportation Research Plaza
Blacksburg, Virginia 24061