top of page
.png)
Gesture Backgrond
Background
Loomo is a mobile robot sidekick, both a self-balance transporter and a personal assistant. Loomo went on sale in April 2018 worldwide and has been integrated into the lives of 30,000+ satisfied customers.
My Role:
Solo Product Designer
Build gesture from0 to 1
Duration:
3 Months
Cross-Platform:
Robot, iOS, Android
Gesture Quick View
The Problem
Loomo has using scenarios both indoors and outdoors. Although it already got voice interaction, it's hard for voice commands to work in a noisy outdoor environment.
As users, they need an interaction method to quickly access outdoor high-frequency features like follow, take photos.
Why Gesture
There were 2 reasons I thought gesture is an excellent fit:
1. Loomo is designed as a little buddy of users. Just like when people call their dog with gestures, users could call Loomo in the same way.
2. Gesture as a Long-distance interaction fits the high-frequency features like follow, take photos very well.
Goals of Gesture
-
Natural, quick, safe gestures for 3 feature: Follow, Stop Follow, Take Photo
-
High accuracy of gesture recognition
-
Easy to learn
Pre-design: Define Gestures
3 principles to choose gesture positions
1.
Fit to life scenario, match the meaning of commands
2.
Take future gesture plan into consideration
3.
Evaluated by algorithm team
Final chosen gestures:
Start Follow
Stop Follow
Take a Photo
CHALLENGE & SOLUTIONS
Challenge 1 - False Triggering
After gesture poses define, I put the gestures in robot idle states and started simplest testing.
Challenges & Solutions
What is False Triggering?
Like Alexa or Siri have false voice triggering when users did not call them, but they react. Gestures were also falsely triggered by mistakenly recognizing the random body language as the gesture commands and potentially starting to move suddenly!
False triggering is a very high-risk problem because it can cause security problems:
-
Poor experience
-
Smashes objects, potentially hurt people
-
Lost because followed wrong person…
KET DESIGN DECISION #1
Wake-up Gesture
I get inspiration from smart speakers, which always have a wake-up voice command like "Alexa", "Hey Siri ".
Alexa
How's the weather?
After testing, the wake-up gesture is an effective solution for false triggering; it’s like a barrier to prevent command gestures from being recognized accidentally.
Wake-up gestures can also save CPU usage by minimizing counts, reducing delays and crashes.
Take Photo
Follow
Follow
Take Photo
Waken
Wake-up gesture choosing
A say-hi gesture.
* Accessibility considerations: for all one hand gesture, both right-hand and left-hand gestures are okay for recognization.
KET DESIGN DECISION #2
Visual Feedback for Holding Gestures
Another technical reason for the false triggering is the short recognition time required. Lead the users to hold the gesture for 2 seconds could avoiding false triggering.
Turn design ideas into visual interfaces, and I designed 2 visual feedback to guide users holding gestures:
1.
Loading animation
2.
Circle process bar
After testing, I chose the circle process bar feedback because testers know better about how long it will take. It's always good to set the right expectations for users.
CHALLENGE & SOLUTIONS
Challenge 2 -
Bridging the Gap between Design and Development
Why the developed result different from the designer's imagination? 😂
Delayed Feedback Causing Low Recognition Rate
After hand-off the design docs to the engineer, I found out that the developed version was not ideal enough due to the delayed feedback. Testers got so confused because when they raised the hand, the bar didn’t run; when they put the hand down, it didn’t stop.
SOLUTION
Design in the Engineer's Shoes
After an in-depth talk with the engineer, he let me know that the technical definition he was making was recognizing each frame, not the seconds that I considered in the design. I found that the gap between us resulted in insufficient detail in the design hand-off.
Designer: Seconds?
Engineer: Frames?
Thinking in the engineer’s shoe, I redefined the start and end of recognition based on frame. And solved the delay feedback problem.
Take start recognition for example:
Evaluation
After continuously iterations and testings:
92.6%
The general gesture recognition rate:
Never
False Triggering
happen again
Evaluation
Final Design
Final Design
Start Follow
Stop Follow
Auto-shot
Take a Photo
Start Follow
Stop Follow
Onboarding
Ready for Launch - Onboarding
For the goal of easy to learn, I designed gesture notification and tutorials as the new feature onboarding experience.
Mobile Notification & Tutorials
When the version including Gesture was shipped and installed by the user, the notification will be sent and lead the user to go through a feature introduction.
Interactive Robot Tutorial
Considered the robot’s moving and speaking capabilities, I designed an interactive tutorial to let Loomo teach users how to use gestures with its own tones.
The best way to learn is to practice! 🎓
Impact of Gesture
The Gesture feature of Loomo was launched in October 2018.
4.1 - 4.3
Loomo Application score:
6.2h - 7.35 h
Product weekly usage time:
5.8 - 8.2
Times of follow and take photo feature weekly used:
Impact
Gesture feature also received positive evaluations and sincere suggestions from the Loomo support website.
Takeaways
-
This is a precious opportunity to let me realize that in-depth communication between different team roles can profoundly change the design.
-
A poor experience may be caused by multiple reasons. It is equally essential for designers to troubleshooting and finding solutions.
Check More I Designed for Loomo
More about Loomo
Voice Auto-shot
Voice Auto-shot
Interactive Tutorial
Over-the-Air (OTA) Update
Over-the-Air (OTA) Update
User testing &
Product testing
User Testing & Product Testing
bottom of page