Safe-D: Safety through Disruption

Formalizing Human-Machine Communication in the Context of Autonomous Vehicles



While driving behavior is generally governed by the nature and driving objectives of the driver, there are many situations (typically in crowded traffic conditions) where tacit communication between the drivers and pedestrians govern the overall driving behavior, significantly enhancing driving safety. We intend to study and formalize the communication pattern between human drivers and pedestrians, as also investigate effective communication mechanisms between an autonomous vehicle and humans. Current autonomous vehicles engage in decision making that is primarily driven by on-board or external sensory information, and do not explicitly consider communication with pedestrians. We will incorporate the formalized communications from this study into decision making algorithms of an autonomous vehicle. Use of the results of this study would lead to improved safety of both autonomous vehicles as well as conventional vehicles.

Project Highlights

  • Provides a dataset, called the CommAutoExport, which contains 1,808 annotated pedestrian interactions with detailed descriptions of each variable. The CommAuto Driver Dataset includes information on the driving context, driving route, narration, and participant demographics
  • Separated a specific language (for recording observations and posing scenarios) from its realization via probabilistic models.
  • Examined a human-vehicle interaction scenario that involves ambiguity, formulating a decision-theoretic model that captures elements related to a pedestrian’s intent as a form of uncertainty, which is ultimately expressed as non-determinism and partial observability.

Final Report

02-014 Final Report

EWD & T2 Products

Course CSCE-691-646: Co-PI Shell used the topic of uncertainty-aware planning as a means to realize implicit communication as a basis for a topical discussion on 3-Sep-2019 with the seven students, all registered for Fall 2019’s offering of CSCE-691, section 646 (each had familiarity with Markov Decision Processes; background on the partially-observable case was mixed).

Course MEEN 401/402: Co-PI Saripalli led a group of senior undergraduate students as part of the capstone project. Five students teamed up as part of the MEEN 401/402 course, over a period of two semesters, and built a communication system consisting of LED, Speakers, GPS that can communicate intentions of the Autonomous Vehicle (in this case the Polaris GEM e4) with pedestrians. Design Comparison Photo. The project ended in December 2019 (Jan 2019 – Dec 2019).

Educational aids: Co-PI Shell examined how information gathering is treated in standard formulations, and finally, discussion with application to social settings where the planner realizes communication for interaction was described by the student (Hsu). This constituted a topic for 1.5 hours of time and culminated in a student present (not affiliated with the original topic) expressing that both the application and the non-standard formulation were interesting and novel to him.

The software associated with the DSL was created using the “Meta Programming System” from Jet Brains. The model and generator files are located at this following GitHub repository:


Hsu, Y.C., Gopalswamy, S., Saripalli, S., Shell, D. A. (2018, August). An MDP Model of Vehicle-Pedestrian Interaction at an Unsignalized Intersection. Workshop paper appearing in IEEE Vehicular Technology Conference (VTC), Chicago, Illinois, August 2018. (Published)

Hsu, Y., Gopalswamy, S., Saripalli, S., Shell, D. A., (August 2018) An MDP Model of Vehicle-Pedestrian Interaction at an Unsignalized Intersection. Presented at the 88th IEEE Vehicular Technology Conference, Chicago, IL. (Published)

Eick, E., Faunce, C., Roediger, M., Hickman, J.S., & Geller, E.S. (2018). Safety-related pedestrian behaviors among University students: Naturalistic observations from a moving vehicle. Paper presented at the annual Virginia Association of Behavior Analysis in Roanoke, VA. (Published)

Roediger, M., Hickman, J.S., & Geller, E.S. (2018). Exploring human-vehicle communication to balance transportation safety and efficiency. Paper presented at the 7th International Symposium on Naturalistic Driving Research, Blacksburg, VA. (Published)

Roediger, M., Eick, E., Faunce, C., Hickman, J.S., & Geller, E.S. (2018). Behavioral Impact of an Autonomous Vehicle: A naturalistic study of pedestrian-vehicle communication and pedestrian behavior. Paper presented at the annual Virginia Association of Behavior Analysis in Roanoke, VA. (Published)


Research Investigators (PI*)

Swaminathan Gopalswamy (TTI/TAMU)*
Srikanth Saripalli (TTI/TAMU)
Jeffrey Hickman (VTTI/VT)
Scott Geller (VT)
Micah Roediger (VT)

Project Information

Start Date: 2017-09-01
End Date: 2019-05-31
Status: Complete
Grant Number: 69A3551747115
Total Funding: $326,218
Source Organization: Safe-D National UTC
Project Number: 02-014

Safe-D Theme Areas

Automated Vehicles
Connected Vehicles

Safe-D Application Areas

Driver Factors and Interfaces
Operations and Design
Vehicle Technology

More Information

UTC Project Information Form

Sponsor Organization

Office of the Assistant Secretary for Research and Technology
University Transportation Centers Program
Department of Transportation
Washington, DC 20590 United States

Performing Organization

Texas A&M University
Texas A&M Transportation Institute
3135 TAMU
College Station, Texas 77843-3135

Virginia Polytechnic Institute and State University
Virginia Tech Transportation Institute
3500 Transportation Research Plaza
Blacksburg, Virginia 24061