Kumamotalk — Interactive AI Conversation Bot
An interactive AI conversation bot built for Kumamoto EXPO 2025 in a 3-person team. I designed and implemented the frontend — face-detection-driven reactions and microphone-aware interactions — and the bot was used by 80+ visitors at the event.
What shipped
Face-aware interactions
Browser-based face detection lets the bot respond to the visitor standing in front of it.
Microphone-aware flow
Recording and permission states are explicit, so visitors always know what the bot is doing.
Expo-ready interface
Designed for walk-up use: legible at distance, forgiving of first-time users.
3-person delivery
Designed and implemented the frontend within a 3-person team.
What changed
- Used by 80+ visitors at Kumamoto EXPO 2025.
- Face detection and microphone handling run entirely in the browser.
- The frontend — UI/UX design and implementation — was my contribution within the 3-person team.
Process details
A bot that has to work in a noisy hall
Expo visitors walk up cold: no manual, no patience, one chance to understand what this thing does. The interface had to invite a conversation and show its state at a glance — listening, thinking, speaking.
Everything had to run in the browser on event hardware. Face detection had to feel responsive rather than eerie, and the microphone flow had to handle permission and recording states without ever stranding a visitor.
Design the conversation, then engineer the senses
Interface
A conversation UI built for a booth
I designed the interaction flow for walk-up visitors: large states, obvious affordances, and a dialogue layout that reads from a distance.
Perception
Face detection as feedback, not surveillance
face-api.js and TensorFlow.js run in the browser so the bot can react to the visitor in front of it — attention drives the conversation's rhythm.
Voice
A microphone flow with no dead ends
react-mic drives the recording states with explicit status feedback: requesting permission, listening, processing — every state visible.
Design file
Open in FigmaGallery
