MOLDABLE FACE

CSCE-450 Final Project, Yida Zou

This project implements a manipulatable 3D face through the use of blendshapes. The user can cycle through a selection of blendshapes that each correspond to a specific facial feature of the 3D head and move each feature around using the keyboard.

Inspiration

I wanted to make a project inspired by the video game “Who’s Lila?” where one of the main features is you drag around a face to match specific emotions.

The trailer of the game can be seen here: 

Implementation

I started with the base code from Assignment 3a, Blendshape Facial Animation, and modified it.  My implementation can be outlined with the following:

1. Create and store a selection of blendshapes as deltas of the neutral face.
   1. I selected blendshapes that I thought were the most basic in that they moved around the smallest areas of the face in order to have finer control of facial features and less overlap between the blendshapes.
2. Be able to cycle through the selection of blendshapes using the keyboard.
3. Get indices of vertex points on the neutral face that match the vertices in the blendshapes
4. Move the vertex points around using the keyboard
5. Using a blendshape-solve equation, the rest of the blendshape moves along with the vertex.
   1. We have to solve for a weight such that when the subset of deltas is multiplied by the weight and added to the original vertex, it moves the vertex to the new position specified by the user.
   2. Using this weight, we can multiply it with the rest of the vertices in the blendshape and add it to the original vertex positions and move the entire blendshape.

Future Improvements

Initially, there were more features I wanted to implement but they turned out more complex than I thought. These are some of the features that I may try in the future:

• Use the mouse to drag around the facial features instead of the keyboard.
  • Would most likely involve raycasting.
• Be able to detect what emotion is being made with the Facial Action Coding System or FACS for short.
  • The game I was inspired by did this with a neural network.

Acknowledgments

• Inspired by the game "Who's Lila?"
• Starter code from Assignment 3 (Blendshape Facial Animation) by Shinjiro Sueda
• Blendshape Solve equation from Shinjiro Sueda
• Developed using OpenGL and C++
• Head model from facewaretech.com
• Blendshapes created using Autodesk Maya