3D pose estimation from multiple uncalibrated cameras
Project title/description
3D pose estimation from multiple uncalibrated cameras
More detailed description of the project
LivePose is an open-source pose and action detection software, capable of using multiple cameras to help with creating interactive experiences.
With using multiple cameras comes the need to calibrate them together, to extract 3D coordinates from the 2D pose estimations associated with each camera. This project aims for facilitating the calibration process by using the human body as a calibration tool, hence removing the need to manually place calibration points or any such cumbersome method.
Some preliminary literature describing a human-based calibration:
- Human Pose as Calibration Pattern; 3D Human Pose Estimation with Multiple Unsynchronized and Uncalibrated Cameras
- MetaPose: Fast 3D Pose from Multiple Views without 3D Supervision (this one makes use of deep learning to achieve better results at the cost of computation)
Expected outcomes
- Implementation of a sub-module from one of the papers above along with a working prototype (or of similar work found by the beginning of this project)
- Partial representation of results by training/testing on a subset of a publicly available dataset
Skills required/preferred
- required: experience with Python
- required: experience with computer vision
- required: strong foundation in math
- if going the deep learning road: experience with machine learning frameworks
Possible mentors
Emmanuel Durand, Christian Frisson
Expected size of project
350 hours
Rating of difficulty
hard