Understanding the building blocks of human movement is crucial for advancements in fields ranging from athlete training to neurological diagnosis. While motion capture (MoCap) and computer vision technologies have facilitated movement tracking and help generate vast datasets, manual labelling and action segmentation remains a bottleneck. This study applies unsupervised machine learning to address this challenge, revealing the underlying structure of actions over time. Specifically, we analyse behavioural responses to threat within a virtual reality (VR) experiment. In the study, participants (n=27) avoided threats while collecting fruit in VR environments (5x10m, designed using VRThreat toolkit for Unity, and additionally performed instructed actions. Motion was captured at 100 Hz using a Coda Motion marker-based motion capture system. A wireless HTC vive pro VR headset was used to stream the experiments. Spatio-temporal kinematic features were extracted from MoCap. A total of 242 movement groups were initially identified, which reduced to 58 after fusing similar locomotion actions. Labels were hierarchically organised. The top 5 most frequent actions across all trials were forwards walk/run (16.2%), right upper reach (9.6%), left upper reach front (8.9%), slow hesitant motion in spot (8.6%), and standing dynamic (7.7%). This study successfully demonstrates the potential of unsupervised machine learning to identify movement motifs in VR using marker-based motion capture data, addressing the challenge of manual labelling and segmentation. The results lay the groundwork for further analysis of action sequences and their structure.

Animals including humans must cope with immediate threat and make rapid decisions to survive. Without much leeway for cognitive or motor errors, this poses a formidable computational problem. Utilizing fully immersive virtual reality with 13 natural threats, we examined escape decisions in N = 59 humans. We show that escape goals are dynamically updated according to environmental changes. The decision whether and when to escape depends on time-to-impact, threat identity and predicted trajectory, and stable personal characteristics. Its implementation appears to integrate secondary goals such as behavioral affordances. Perturbance experiments show that the underlying decision algorithm exhibits planning properties and can integrate novel actions. In contrast, rapid information-seeking and foraging-suppression are only partly devaluation-sensitive. Instead of being instinctive or hardwired stimulus-response patterns, human escape decisions integrate multiple variables in a flexible computational architecture. Taken together, we provide steps toward a computational model of how the human brain rapidly solves survival challenges.