Pose Assisted Dancing
Project
There are many modern solutions for detecting the position of the human body in an image. My partner Grayson Snyder and I wanted to explore some of these while also designing a fun solution for comparing a user to an instructional dance video.
Videos
The demo videos for this project are intentially muted for copyright reasons, feel free to check out the originals here:
charlixcx
sydwingold
Method
We implemented two frameworks for doing pose detection, both Ultralytics YOLO11 and Googles Mediapipe. The rest of this project was built upon these, with the ability to switch between them seamlessly using Python's duck typing to abstract the framework specific logic away from the pose comparison and the user interaction. Once a user's pose is detected, it is normalized and then flattened into a vector for comparison to a reference using cosine similarity.
There are many different options that one can use for comparing human poses such as euclidean distance or joint-wise angle comparison, however cosine similarity works well because it is very sensitive to the relative orientation of points and not their position. While this has some drawbacks, it works very well for comparing dances.
This produces a score from -1.00 to 1.00, with 1.00 representing a perfect alignment. This score is then averaged over time to provide a running total. Our system supports any number of users; if multiple poses are detected they will each be assigned a score.
Many detections can affect performance, with YOLO11 Nano generally being faster than Mediapipe in our testing even though it is a little less accurate.
My Amazing Partner