Workshop on the Detection, Recognition, and Prediction of Pedestrian Behaviors

In ITSC 2021

Registration through the ITSC conference to attend


With the progress of automated driving technologies, self-driving cars are already running safely on highways and freeways. However, besides other limitations, the lack of safe and smooth interactions with pedestrians becomes significant obstacles preventing fully autonomous vehicles in city streets. Protections of vulnerable road users like pedestrians are always of the highest priority in traffic safety, and crashes with pedestrians will significantly impact the trust and public attitudes towards the new mobility technology. Disruptive interactions with pedestrians may also lower both the riding experiences and moving efficiency in the pedestrian-rich road environments. With these issues, it is vital to equip autonomous vehicles with the capabilities to understand pedestrians and make motion planning based on the predictions of their behaviors.

Learning-based pedestrian detection and tracking algorithms have achieved significant improvements in the past several years, allowing vehicle perception systems to accurately calculate pedestrians' motions through sensor-fusion, predict potential crashes, and make motion-planning decisions accordingly. However, these safety-oriented solutions are not enough for longer-duration predictions of pedestrians' movements and thus are not ideal for motion planning and cannot support vehicle-pedestrian communications. Many research efforts have been put into recognizing pedestrians' behaviors and predicting their trajectories in the past few years. Compared with detection and tracking, the prediction of pedestrians' behaviors is way further from mature and needs more progress to support fully autonomous driving systems.

This Workshop will be the first one focusing on the detection, recognition, and prediction of pedestrian behaviors. The goal is to build a platform for sharing state-of-the-art models, algorithms, and datasets in the field and identifying research needs and directions.



Josh Domeyer

Toyota Motor North America
Talk title: Perceptual control guides driver-pedestrian crosswalk negotiation

Abstract: Predicting pedestrian crossing intent can be difficult due to complex behaviors that are often interpreted differently depending on the context. Additionally, driver and pedestrian behaviors are dynamic and evolve over the encounter to produce safety, efficiency, and fairness outcomes. While machine learning techniques can sometimes predict pedestrian behaviors well, they fall short in helping to understand the mechanisms that guide pedestrian behavior. A deeper understanding of pedestrian behaviors can inform the features that are used to predict pedestrian intent. This talk will show how mechanistic models can reveal new information about pedestrian behavior. Perceptual models are implemented as differential equations and simulated to reproduce observed pedestrian and driver behaviors from naturalistic driving videos. Future directions and potential applications are discussed.

Short bio: Josh Domeyer is a senior researcher in the Toyota Collaborative Safety Research Center. He received a Ph.D. in Industrial and Systems Engineering at the University of Wisconsin-Madison in 2021 and B.S. and M.S. degrees in Experimental Psychology from Central Michigan University in 2009 and 2011. He has been a human factors researcher at Toyota since 2011 where his research has focused on driver-vehicle interface, distraction, and human-automation interaction. He joined the Collaborative Safety Research Center in 2017 to explore the communication between drivers and pedestrians with the goal of informing the development of vehicle automation. He is the Chair of SAE’s Safety and Human Factors Steering Committee and a U.S. expert for the ISO TC22/SC39/WG8 standards group.

Ali Karimoddini

North Carolina A&T State University
Talk title: Robust and Computationally Effective Pedestrian Detection for Autonomous Cars

Abstract: Real-time accurate pedestrian detection and tracking are crucial to ensure safety and reliability in autonomous driving. This often requires the estimation of multiple pedestrian trajectories from multi-modal sensor data. There are, however, several challenges that make pedestrian detection and tracking a notoriously hard task. For instance, pedestrians may appear in a scene with different articulations of body parts, partial occlusion, and appearance in various poses. Furthermore, the detection and tracking may fail due to the sensitivity of some sensors under low illumination or bright sunlight conditions.Despite all these challenges, it is vitally important for autonomous driving to achieve accurate and robust pedestrian detection and tracking under various conditions in real-time to be able to transition from the innovation space to real-world operations. In this talk, we will review our recent results on robust and efficient techniques for pedestrian detection. In particular, we present our developed techniques for fusing the information from different sensors and networks to enhance the accuracy of pedestrian detection, while managing the computation costs. We also present the results of the extensive evaluation of the developed methods in terms of runtime- efficiency and robustness of the proposed methods both on public datasets and our autonomous driving dataset. Results show our fusion method significantly improves detection rate while achieving competitive runtime efficiency.

Short bio: Ali Karimoddini is the Director of NCCAV Center of Excellence on Advanced Transportation Technology, the Deputy Director of the TECHLAV Center of Excellence in Autonomy, and the Director of the ACCESS Laboratory. His research interests include development of solutions for “Testing, evaluation, analysis, control, and decision-making for autonomous vehicles,”“Cyber-physical systems,” “Cooperative and distributed control of multi-agent systems,” “Reliable and fault tolerant control systems,” and “Human-machine interactions” with focus on their application in smart transportation systems, battlefield management, and smart agriculture domains.

Amir Rasouli

Huawei Technologies Canada
Talk title: Predicting Pedestrian Behavior: Premise, Progress, and Prospects

Abstract: Humans commonly rely on predicting each other’s behaviors to accomplish their goals. The driving task is no exception, which also applies to autonomous vehicles (AVs). To be safer, AVs require robust prediction systems that can accurately forecast behaviors of other road users, in particular, pedestrians as the most vulnerable ones.
In this presentation, I will provide an overview of pedestrian behavior prediction in the context of autonomous driving. I will begin by discussing the motivation behind developing predictive models, followed by a summary of psychological studies of pedestrians and factors that impact their behavior. I will continue by describing the current progress in the field and will talk about existing driving datasets for research and development and state-of-the-art pedestrian behavior prediction models. I will conclude the presentation by enumerating some of the open problems in the field.

Short bio: Amir Rasouli is a senior research scientist leading pedestrian prediction team at Noah’s Ark Laboratory, Huawei, Canada. He received B.Eng. degree in computer systems engineering and B.A. in Business Management from the Royal Melbourne Institute of Technology. He obtained M.A.Sc. degree in computer engineering under the supervision of Prof. John Tsotsos at York University studying the role of attention in active visual object search for mobile robots. He completed his Ph.D. degree in the same lab on the topic of pedestrian behavior understanding and prediction in the context of autonomous driving. His current work focuses on various aspects of pedestrian behavior prediction including analysis of naturalistic driving data, understanding theoretical foundations of pedestrian-driver interaction, modeling pedestrian behavior in simulation, and development of predictive models for intelligent driving systems.

Tina Chen

Indiana University Purdue University Indianapolis
Talk title: Pedestrian Intention Prediction using Environmental Context

Abstract: For autonomous vehicles and pedestrians to co-exist on the road as seamlessly as possible, autonomous vehicles should have the capability to navigate with pedestrian intention and motivation in mind rather than just react to pedestrian actions that are already in progress. Predicting human behavior can be a highly difficult task due to the many variables that can affect our behavior, but through computer vision techniques and the usage of neural networks, we can include more and more context in our predictive models. In this presentation, I will focus on research in predicting pedestrian crossing intention from the ego-view which mainly uses computer vision and deep learning. I will start with an overview of the state-of-the-art prediction models in the autonomous vehicle field. Then I will introduce a predictive model that I am working on that factors in environment and pedestrian visual features for prediction. Many of the important context cues that factor into a pedestrian’s intention are missing from predictive models, and my model will show context improves intention prediction. The presentation will be concluded with future research directions.

Short bio: Tina Chen recently graduated with an M.S. in Computer Engineering from Indiana University
Purdue University - Indianapolis. She received her B.S. in Electrical Engineering from the University of Wisconsin - Madison in 2015. She has more than 5 years of experience working on, researching, and evaluating sensing systems and algorithms for advanced driver assistance and automated driving systems. Her current work mainly focuses on using machine learning and computer vision to improve pedestrian behavior prediction for autonomous vehicle applications. She is working on a deep learning algorithm that uses both visual features and object spatial relationships to predict pedestrian crossing behavior. Additionally, she is also developing a dataset focused on understanding pedestrian behavior through driving decisions.

Renran Tian

Indiana University Purdue University Indianapolis
Talk title: Situated Pedestrian Intent from the Driver’s Perspective: Theory and Challenge

Abstract: With the progress of automated driving technology going into the city streets, pedestrian behavior has drawn a lot of attention. Most of the continued efforts in developing benchmark datasets and prediction algorithms focus on pedestrian trajectories and activities. However, one question is whether pedestrian trajectory and activity are entirely predictable or not, or what shall be the expected performance level. In this talk, I will describe the view of situated pedestrian intent from the driver’s perspective to depict the dynamics of pedestrian behavior during driver-pedestrian encounters. Some empirical results show the differences among human drivers in understanding pedestrian intents, highlighting the difficulties in accurately predicting their behaviors and advocating the importance of designing vehicle control strategies considering the nondeterministic characteristics of the encountering process.

Short bio: Dr. Renran Tian is an assistant professor in the Department of Computer Information & Graphics Technology and the Transportation & Autonomous Systems Institute (TASI) at Indiana University-Purdue University Indianapolis (IUPUI). He received his Ph.D. degree from the School of Industrial Engineering at Purdue University –West Lafayette in 2013, and B.S. and M.S. in Mechanical
Engineering from Tsinghua University – Beijing, China in 2002 and 2005. His research interests include human-centered computing, cognitive ergonomic, computational behavior analysis, human-AI interaction, and autonomous driving. He serves as the co-chair for Technical Activities Committee on Human Factors in Intelligent Transportation Systems (HFITS), IEEE Intelligent Transportation Systems Society.