Master Thesis Proposals

DeepPrivacy is a system for automatic anonymization of faces in image data, and the master thesis topics presented here are all related to this system. The topics will explore methods in deep learning for the anonymization of image/video data, especially detection, segmentation, and generative models. The topics require experience with neural network programming, strong coding skills, and preferably have taken a course about deep learning or computer vision (TDT4265, IT3030, or equivalent).

Below are four different topics for a master thesis. If you’d like to work with similar topics and the presented topics don’t fit you, you could propose a subject by contacting Håkon (hakon.hukkelas@ntnu.no). Each project description is very flexible and you are free to design the project after your interest. Each proposed topic is very little explored in the academic community, and has a high potential for a contribution to the current literature.

If you find any of the topics interesting, please don’t hesitate to contact me! I’m happy to answer any questions and would love to have a chat with you :) You can contact me through mail (hakon.hukkelas@ntnu.no) or on Microsoft Teams.

Introduction to DeepPrivacy

DeepPrivacy is a two-stage system where the first stage detects faces and the second stage anonymizes each face with a generative model. The second stage anonymizes faces with image inpainting: first, the part of the image containing the face is completely removed, then a generative model completes the missing region (see below).

For those interested, I recommend the following resources to read up on DeepPrivacy:

Flaws of the current System

The current system works well for image data, but there are several areas that can be improved upon:

  • The current model process each image frame-by-frame without considering temporal consistency. This results in “flickering” in the anonymized region for video data.
  • The privacy-preserving aspect of the current system can be questioned, as other parts of the human body are identifying. A possible solution to this is full-body anonymization.
  • The quality of the anonymized faces is low resolution (128x128), and the runtime of the model is quite slow.

Topic 1. Human Body Anonymization

The human body has several identifying properties other than the face, such as the gait of a person or their ears. Full-body anonymization, where we replace the entire body instead of only the face, is a possible solution to this issue. However, generating an entire human body that fits the environment is difficult, and its application for anonymization is not well explored in current literature. Possible solutions to generate realistic anonymizations could be to combine a dense 3d prediction of the pose of the human and synthesizing a new body to replace the human.

Topic 2. Controllable Image Inpainting and Face Synthesis

Prior work has shown that it’s possible to represent a full-HD image with a low-dimensional representation (for example 512 numbers) to represent an image. In essence, this is done the following way: First, you train a model that maps a latent vector (a vector with N numbers, e.g. N=512) to an arbitrary image. With this trained model, we can search for the latent vector that reproduces our image. An example of this is shown below: the leftmost image is the input image (the shaded area is the missing region), the second image is the initial generated image with a random latent vector, and the rightmost image is the generated image after searching for the optimal variable. This video shows the search process.

A possible thesis topic is to explore this technique for image inpainting and face anonymization. By finding a latent variable for a given person, we can select a latent variable that is different from the original person’s variable, ensuring that we do not re-generate the original face.

Topic 3. Incorporating Semantic Knowledge for Image Inpainting

Semantic-guided image inpainting is the task of mapping an image with a missing region and a semantic segmentation to a generated image. One use case of this is to process the output of DensePose to generate a semantic map and generate a human body using this information. This topic will explore how to incorporate semantic knowledge into an image inpainting model and/or other generative models.

Topic 4: Anonymization of video data

Handling video data can remove the flickering effects of generated faces. This thesis will explore video inpainting and anonymization of faces in video data.