Read our blog post for the full write-up.
These instructions will show how to acquire pose estimates from a participant, train a pix2pix model, transfer source dance video, and generate a dance gif.
- Tensorflow 2.0 (not required on EdgeTPU)
- EdgeTPU Python API (for the EdgeTPU only)
- OpenCV
- Pillow
- Google Cloud Storage Python client library
We recommend using a virtual environment to avoid any conflicts with your system's global configuration. You can install the required dependencies via pip. The EdgeTPU will require the Python API as described above.
This demo also requires that you have a Google Cloud account and have configured the python client with credentials. See this for more resources.
In the pose/ directory, you will find all of the EdgeTPU resources required to capture training images from your booth participant.
On the EdgeTPU, run:
python3 run.pyThis script will generate the pose estimation overlays and raw image assets and send them to your declared Google Cloud Storage bucket.
After setting up a VM with a GPU instance in your Google Cloud account, install the requirements listed above. Then you can use the train.py script to train the Pix2Pix model.
python3 train.pyPeriodic checkpoints will be stored in the checkpoints/ directory. This script trains for 50 epochs to reduce training time.
We've included default source video frames from the Bruno Mars - That's What I Like music video. If you are using another source video, keep the framing, perspective, and background of your source in mind for better results.
To generate a dance gif of your participant, run the following on your Google Cloud training VM instance:
python3 generate_gif.pyThe asset will be stored in the results/ directory and in your declared Google Cloud Storage bucket.
Please read CONTRIBUTING.md for details on our code of conduct, and the process for submitting pull requests to us.
This project is licensed under the GNU General Public License v3.0 - see the LICENSE.md file for details