play - AnswerLab

How Research
Plays:
The UX of Mobile Gaming
CONFIDENTIAL
UPA 2011
1
Introductions
Sal Becerra
Director of Research
Laura Smith
UX Designer
CONFIDENTIAL
UPA 2011
2
Introductions
Sal Becerra
Director of Research @ AnswerLab
•
Over 10 years experience leading research programs (Market & User Research)
•
Expertise in quantitative and qualitative methods, across web and mobile devices
•
Founded & implemented User Research @ LinkedIn
CONFIDENTIAL
UPA 2011
3
Introductions
Laura Smith
User Experience Designer @ Electronic Arts
•
Manage Usability & Playability Research for Pogo
•
Over 10 years experience in UX
•
Games in the marketplace I have worked on while at Electronic Arts:
CONFIDENTIAL
UPA 2011
4
Audience?
CONFIDENTIAL
UPA 2011
5
Play!
CONFIDENTIAL
UPA 2011
6
Agenda
• Game Changing The Digital Space
– Background, Proliferation, & Influence
• Research Beyond Usability
• UX Approach Within Agile Mobile Gaming
• Gaming Benchmarks
• Return on Investment (ROI) of UX in Mobile
Gaming
CONFIDENTIAL
UPA 2011
7
We spend 3 billion hours a
week as a planet playing
videogames.
CONFIDENTIAL
UPA 2011
8
Digital Games Past Domains…
CONFIDENTIAL
UPA 2011
9
Digital Games Past Domains…
CONFIDENTIAL
UPA 2011
10
Digital Games Past Domains…
CONFIDENTIAL
UPA 2011
11
World Rapidly Became More Digital
CONFIDENTIAL
UPA 2011
12
World Rapidly Became More Digital
CONFIDENTIAL
UPA 2011
13
Traditional Gaming Consoles Grew
CONFIDENTIAL
UPA 2011
14
As Did Digital Gaming Opportunities
CONFIDENTIAL
UPA 2011
15
Digital Games Are Everywhere
CONFIDENTIAL
UPA 2011
16
Digital Games Are Everywhere
CONFIDENTIAL
UPA 2011
17
Digital Games Are Everywhere
CONFIDENTIAL
UPA 2011
18
Digital Games Are Everywhere
CONFIDENTIAL
UPA 2011
19
Digital Games: Powerful & Influential
CONFIDENTIAL
UPA 2011
20
Digital Games: Powerful & Influential
CONFIDENTIAL
UPA 2011
21
Digital Games: Powerful & Influential
CONFIDENTIAL
UPA 2011
22
Digital Games: Powerful & Influential
CONFIDENTIAL
UPA 2011
23
Digital Games: Powerful & Influential
CONFIDENTIAL
UPA 2011
24
Why You Should Care
• Digital Games Everywhere
• Driving Adoption & Engagement of Products &
Services
• Impacting Brands & Industries Beyond Gaming
CONFIDENTIAL
UPA 2011
25
We don’t use games…
CONFIDENTIAL
We play them
UPA 2011
26
most usable game
=
most boring game
CONFIDENTIAL
UPA 2011
27
Congratulations… You won!
Game Over.
CONFIDENTIAL
UPA 2011
28
What Makes A Fun Game?
“Flow” concept by Mihaly Csikszentmihalyi
CONFIDENTIAL
UPA 2011
29
Game world
Structured
Hard fun
Easy fun
“fiero”
“curiosity”
People fun
Serious fun
“amuse”
“relax”
Open
ended
Real world
Four keys of fun – Nicole Lazzaro
CONFIDENTIAL
30
www.xeodesign.com
UPA 2011
A
Beyond Usability:
The Value of User
Research
CONFIDENTIAL
UPA 2011
31
Without The Challenge, Where’s The
Fun?
CONFIDENTIAL
UPA 2011
32
Motivations: Fun Vs. Task
CONFIDENTIAL
UPA 2011
33
Hidden Or Unclear Elements Can
Surprise & Delight
CONFIDENTIAL
UPA 2011
34
Business Drivers: Fun Vs. Ease Of Use
CONFIDENTIAL
UPA 2011
35
Playability Is Not The Same As Usability
Playability
Usability
Challenges are
good
Challenges are bad
Surprising or
unclear key
elements can be
delightful
Discovery &
comprehension of
all key elements
should be
immediate
Motivations are funfocused
Motivations are
task-focused
Fun is a large
business driver
Ease of use can be
a large business
driver
CONFIDENTIAL
UPA 2011
36
Playability Is Not The Same As Usability
…Except When It Is
• Objectives & rules to complete the
objectives must be understood
• Key elements to complete the
objectives must be discoverable and
usable
• Contextual help is key
• Concise messaging is successful
• Showcasing progress & success
furthers engagement
CONFIDENTIAL
UPA 2011
37
A
UX in the mobile
games industry
CONFIDENTIAL
UPA 2011
38
What makes a
great game?
 People want it
 People play it
 People love it
CONFIDENTIAL
UPA 2011
39
Player experience team
new
Game
designer
Games
User
Research
UX
Designer
Visual
designer
Prototyper
CONFIDENTIAL
UPA 2011
40
Pogo Games Case Study
Business objective:
Reach the #1
position for free
apps in iTunes.
CONFIDENTIAL
UPA 2011
41
Pogo Games Case Study
Business objective:
Reach the #1
position for free
apps in iTunes.
CONFIDENTIAL
UPA 2011
42
UX best practice that worked for us
1. Put the player experience first
2. Don’t wait for an invitation
3. Understand the development lifecycle
4. Relate research to business metrics
5. Know when to bring in specialists
CONFIDENTIAL
UPA 2011
43
1. Put the player experience first
CONFIDENTIAL
UPA 2011
44
2. Don’t want for an invitation
“Having people hate your work is the most uncomfortable part
of being a game designer.” Jesse Schell
CONFIDENTIAL
UPA 2011
45
3. Run UX Sprints in parallel
Engineering
Design
CONFIDENTIAL
UPA 2011
46
4. Relate research to business metrics
• How many people download?
• How many people make it
through the first 5 minutes/
past the tutorial?
• What do players enjoy?
CONFIDENTIAL
UPA 2011
47
5. Know when to bring in specialists
CONFIDENTIAL
UPA 2011
48
A
How Research
Plays:
Establishing A Framework
For Assessing Gaming
Development
CONFIDENTIAL
UPA 2011
49
We don’t use games…
CONFIDENTIAL
We play them
UPA 2011
50
What We Set Out To Learn
How do players engage with the
application?
How well do they discover key items?
How well do players comprehend
the application’s elements?
How enjoyable and
engaging are the various
areas of the application?
CONFIDENTIAL
UPA 2011
51
Methodology (1)
Introductory
Interview
•
One-hour lab sessions with qualified participants who
meet the following criteria:
•
Non-Pogo players
•
Pogo.com players
•
Regularly play casual games on their iPhone or iPod touch
Open Explore
Follow-up Tasks
Wrap-up
Interview
Non-Pogo Players
Pogo.com Players
Please see Appendix summary of screener survey document and profile grids for further details.
*Pogo.com players will be denoted throughout this presentation with an asterix (*).
CONFIDENTIAL
UPA 2011
52
Methodology (2)
•
To ensure each session began with the new user experience of the application, we installed the
application onto 2 iPhones. As 1 device was being tested, the other had the application reinstalled.
Remote Viewers’ Screen
Application Entry
CONFIDENTIAL
UPA 2011
53
CONFIDENTIAL
UPA 2011
54
Ask Them
#
Recommend
Rule
Comprehension
Instructions
Ease of
Expertise
Replay
P1
8
8
8
9
9
PX
1
1
4
7
2
How likely are you to recommend Turbo 21 to a friend or colleague? Use a scale of 0 to 10, where 0
is not at all likely and 10 is extremely likely.
How easy or difficult was to understand the rules of Turbo 21? Use a scale of 0 to 10, where 0 is
extremely difficult and 10 is extremely easy.
How easy or difficult was it to play Turbo 21? Use a scale of 0 to 10, where 0 is extremely difficult and
10 is extremely easy.
How long do you think it would take you to become an expert at Turbo 21, compared to other games
like this? Use a scale of 0 to 10, where 0 is much more time and 10 is much less time.
If you had the Pogo game application on your device, how likely would you be to replay Turbo 21?
Use a scale of 0 to 10, where 0 is not at all likely and 10 is extremely likely.
CONFIDENTIAL
UPA 2011
55
Watch Them
• Key Metric 1
• Key Metric 2
• Key Metric 3…
CONFIDENTIAL
UPA 2011
56
Ask & Watch Them
• Perception Vs. Reality
– Time on Task
– Natural Replay
– Reported metrics vs.
Observed metrics
– Partner attitudes and
perceptions of the
experience with the
behavior observed
CONFIDENTIAL
UPA 2011
57
Analysis of Self-Reported & Observed
Key Performance Metrics
Key Metric 1
Key Metric 2
Key Metric 3
Key Metric 4
Key Metric 5
Key Metric 6
Poppit
Word Whomp
Turbo 21
Mahjong Safari
Sweet Tooth 2
Poor
Good
Fair
(Success among many – all players)
(Success among some players)
CONFIDENTIAL
(Success among few – none players)
UPA 2011
58
A
Demonstrating ROI:
Effective
communication
techniques
CONFIDENTIAL
UPA 2011
59
Benchmark performance over time
Agile
sprint
Agile
sprint
CONFIDENTIAL
UPA 2011
60
Reduced development costs
Café test wireframes
CONFIDENTIAL
UPA 2011
61
Onboarding funnel
People in
Download
People leak out
Tutorial
Play
Players
CONFIDENTIAL
UPA 2011
62
Onboarding funnel
Download
Tutorial
In this funnel, small
improvements in
usability make a big
business impact
Play
CONFIDENTIAL
UPA 2011
63
What do players do?
CONFIDENTIAL
UPA 2011
64
What do players do?
Intended design
CONFIDENTIAL
UPA 2011
65
We don’t use games…
CONFIDENTIAL
We play them
UPA 2011
66
Questions & Follow Up
Sal Becerra
Laura Smith
Director of Research
Phone: 415.814.9627
Email: sal@answerlab.com
UX Designer
Phone:
Email: lasmith@ea.com
salbecerra
laurasmith99
sbecerra
LauraSmith99
AnswerLab
160 Spear Street, Suite 700
San Francisco, CA 94105
Electronic Arts
CONFIDENTIAL
UPA 2011
67