Zitong Wang
Brand Identity, Generative Design, Publication, Installation Design, Web Design
2. EaseUp
VUI Design, Generative Design, Visual Identity
3. Reunion: ài + AI
Installation Design, Web Design
4. HerVoice
UI Design, Visual Identity
©2024 Zitong Wang From Abyss to Air
Game Design, UI/UX Design, Installation Design
Not all stories begin with words. Some begin with a movement, a current, a breath.
About
Embark on a journey through the depths of the ocean, experiencing life as a jellyfish, dolphin, and whale. This project combines wearable technology with immersive gameplay to explore the intersection of marine life and human industrial impact.
Technical Insights
Design Diagram
This project connects physical gestures to digital marine motion.
A custom-built glove captures nuanced finger movements and translates them into real-time controls for marine creatures inside Unity.
The entire system translates physical motion into environmental navigation—letting the body drive the story.
Why a Glove, Not Hand Tracking?
Hand tracking often prioritizes precision and recognition—detecting what gesture you're doing. But this project isn't about identifying gestures. It's about feeling them.
A glove allows for continuous, analog sensing. It doesn’t interpret your hand—it listens to it.
Rather than relying on optical accuracy, the glove creates a more intimate, embodied interface: one that’s grounded in pressure, resistance, and motion. It becomes part of the body, not just a camera input.
Glove Prototyping
The glove was built using five flex sensors—initially prototyped on a breadboard, then soldered for stability.
Development progressed in stages: starting with Arduino and Processing to observe finger response, then moving to serial communication with Unity.
The focus was on making the system both accurate and intuitive to move with.
Serial Communication
The glove streams five flex sensor values through an Arduino using basic serial communication.
Inside Unity, the data is read line-by-line and parsed into usable inputs for movement and interaction.
This setup allows for low-latency, plugin-free communication, and full control over parsing logic inside Unity.
Gesture State Machine
To ensure stable gesture recognition, I implemented a simple state machine that monitors flex sensor thresholds. Each gesture is mapped to a distinct combination of finger states, allowing consistent control even with slight sensor noise.
Gesture Mapping
Each gesture controls a different creature—across three stages, each representing a different ocean depth.
Jellyfish: Curl and stretch all fingers to float.
Dolphin: Pinch and release thumb and index finger to swim.
Whale: Thumb up. Curl and release other fingers to move forward.
Game Mechanics
Players collect symbolic fragments—of memory, motion, or resilience—to continue their journey.
Each stage introduces new challenges: from the stillness of the deep, to drifting debris, to human-made structures.
As players move upward, the camera gradually shifts from distant observation to full immersion, reflecting a growing intimacy with the ocean and the creatures they embody.
Completing one level unlocks the next, rising from depth to surface, from drifting to direction.
How does a body move through something it cannot see, yet is shaped by every day?
What if gestures weren’t about control, but about sensing? remembering? becoming?
I started this project by wondering if a small gesture could carry more than input—maybe even reflect the pressures marine life endures, both natural and human-made.
Where It Takes Place
This story takes place near the Port of Long Beach—
one of the first U.S. ports to allow offshore drilling, connected to the vast Wilmington Oil Field, with underwater pipes and platforms stretching across the seafloor.
It also overlaps with a gray whale migration corridor, now increasingly disrupted by ships, metal, and light.
And not everything stays hidden. Floating trash and oil traces now drift across the surface—details that quietly surface throughout the game.
The tension between industrial systems and natural rhythms is where this project lives.
Gesture as Performance, not Control
Wearing the glove is not just about playing.
The goal isn’t to “simulate” the creature’s movement, but to ask:
What does your body have to give up to move like them? What does it have to remember?
Each gesture is a small choreography.
It’s not about doing it right—it’s about noticing what feels unfamiliar, awkward, or strangely natural.
A quiet reminder: we were never only human.