OpenCap Monocular: 3D Human Kinematics and Musculoskeletal Dynamics from a Single Smartphone Video

Selim Gilon, Emily Y. Miller, Scott D. Uhlrich

[paper]

Absract

Scalable measurement of human movement (kinematics) and musculoskeletal forces (kinetics), for example, estimating quadriceps force during sit-to-stand—could transform the prediction, treatment, and monitoring of mobility-related conditions. Yet these analyses still rely on costly, time-intensive laboratory workflows, limiting clinical translation. We present OpenCap Monocular, a pipeline that estimates 3D skeletal kinematics and musculoskeletal kinetics from a single static smartphone video. The method refines 3D global pose outputs from a monocular pose estimation model (WHAM) through a physics-inspired pose optimization that enforces reprojection consistency and foot-floor contact constraints. Keypoints are extracted from the mesh using the refined global pose, and skeletal kinematics of a 33-degree-of-freedom musculoskeletal model are obtained using inverse kinematics. Kinetics (i.e., ground forces and joint moments) are estimated via physics-based simulation and machine learning, without force plates. Validated against marker-based motion capture and force plates for walking, squatting, and sit-to-stand, OpenCap Monocular achieves 4.8◦ mean absolute error (MAE) for rotational kinematics and 3.4 cm for translations, corresponding to 48% and 69% lower error than a direct computer vision baseline. It estimates walking ground reaction forces with 9.7% bodyweight MAE, comparable to a two-camera system. We demonstrate that the algorithm estimates important kinetic outcomes with clinically meaningful accuracy in applications related to frailty (knee extension, hip exten- sion, and ankle plantarflexion moments during sit-to-stand transitions) and knee osteoarthritis (knee adduction moment during walking). OpenCap Monocular is deployed via a smartphone app, a web app, and secure cloud comput- ing (https://opencap.ai), enabling free, accessible single- smartphone biomechanical assessments. Our code is available at github.com/utahmobl/opencap-monocular