Translating Gesture

ROLE

Interaction Designer

CLIENT

MFA Thesis — Mason Gross School of the Arts

TIMELINE

2025–2026

OVERVIEW

Gesture is often assumed to be universal, yet the same movement can carry different meanings across cultures, communities, and individual experience.

Translating Gesture is an interactive installation that uses real-time hand tracking to detect participants’ gestures and project a rotating series of possible interpretations onto a fifteen-foot gallery wall.

The system does not decide what a gesture means. It makes ambiguity visible; its Afterimage Archive then preserves each 12-second movement as a layered hand-motion trace.

RESULTS

21

Hand Landmarks

12 sec

Motion Capture

PROBLEM

How might an interactive system reveal that gestures are interpreted rather than universal?

People often read gestures as fixed signs, but meaning changes with culture, social context, age, and personal experience. The challenge was to reveal that multiplicity without replacing it with another authoritative answer.

RESEARCH

I surveyed participants from varied cultural and generational backgrounds, asking one open-ended question: “What does this hand gesture mean to you?” The conflicting answers became the installation’s language library.

BRAND & STYLE

A spare gallery, large projected type, a thermographic display, and an open white floor ring create a legible environment that invites participation without instructions.

LOW FIDELITY

Early spatial sketches mapped the relationship between body, camera, floor marker, projection, and gesture archive before technical development began.

HIGH FIDELITY

The final installation pairs immediate bodily feedback with shifting text, allowing participants to understand cause and effect while leaving interpretation deliberately open.

AFTERIMAGE

Afterimage records each interaction as a 12-second hand-motion trace. Successive MediaPipe landmark positions accumulate into a luminous drawing, preserving rhythm, hesitation, repetition, and reach instead of reducing the gesture to one label.

GESTURE ARCHIVE

What does a gesture leave behind?

Each capture becomes one entry in a growing archive. Together, the traces turn gesture recognition into a record of movement and memory—showing not only what a gesture could mean, but the distinct form it leaves behind.

EXHIBITION

Exhibited at Mason Gross School of the Arts in Spring 2026, the installation invited visitors to enter the floor ring, test gestures, and turn recognition into physical play. Afterimage Archive extends those encounters into a growing collection of 12-second hand-motion traces.

WHAT I LEARNED

Clear interaction rules created confidence; interpretive openness created discovery. The system shows what a gesture could mean, while Afterimage Archive reveals what that gesture leaves behind.

Next project