İsmail Özyurt/ Work

11 / 12 · Research · 2024

Face Tracking with HMDs

Feasibility study on tracking faces in real time from a headset user’s view.

  • Erasmus+ research internship
  • Feasibility study
Role
Researcher
Where
University of Salerno
Year
2024
A head-mounted display looking at three people; tracking boxes mark their faces. Three approaches were tested: dlib, OpenCV and VGG Face

Illustration

How it works

Presenting? ← → walk through this page and on to the next project.

Built with

Every layer of the stack, and what it does in this project.

  1. L1Hardware
    • Varjo XR3

    Passthrough view of the people around the user.

  2. L2Engines
    • Unity
    • Unreal Engine

    Where the tracking had to sync.

  3. L3Vision
    • OpenCV
    • dlib
    • VGG Face

    Three approaches compared.

  4. L4Language
    • Python

    OpenCV detection on the Python side.

What I did

  1. Targeted a real gap: almost all existing work tracks the HMD user’s own face; tracking the people around the user was largely unexplored.

  2. Tested three approaches: dlib failed on hardware incompatibility; OpenCV detected faces on the Python side but hit performance and compatibility problems when synced with Unity/Unreal; VGG Face was strongest on paper but hit the same integration issues.

  3. Concluded it wasn’t feasible then: the Varjo SDK didn’t support face tracking directly, expert resources were limited, and EU research and data constraints applied.

  4. Authored the project’s technical report.

Team & thanks · Thanks to Prof. Alessandro Naddeo, Dr. Carlotta Fontana, PhD researcher Alfonso Magliano and the whole team.

The Erasmus+ research internship, on ozyurt.dev

Where to next?

Next project · 12 / 12OzearA personal issue tracker with its own MCP server: AI assistants log the work and record how the code behind it works.Open it