Identifying MMORPG Bots: A Traffic Analysis Approach

Identifying MMORPG Bots:
A Traffic Analysis Approach
(MMORPG: Massively Multiplayer Online Role Playing Game)
Kuan-Ta Chen
National Taiwan University
Collaborators:
Jhih-Wei Jiang
Polly Huang
Hao-Hua Chu
Chin-Laung Lei
Wen-Chin Chen
Talk Outline
Motivation
Trace collection
Traffic analysis and bot identification schemes
Performance evaluation
Scheme Robustness
Conclusion
Identifying MMORPG Bots: A Traffic Analysis Approach
2
Game Bots
AI programs that can perform many tasks in place of
gamers
Can reap rewards efficiently in 24 hours a day Î
break the balance of power and economies in the
game world
Therefore bots are forbidden in most games
Identifying MMORPG Bots: A Traffic Analysis Approach
3
Bot Detection
Detecting whether a character is controlled by a bot
is difficult since a bot obeys the game rules perfectly
No general detection methods are available today
The state of practice is identifying via human
intelligence (as bots cannot talk like humans)
Labor-intensive and may annoy innocent players
This work is dedicated to automatic
detection of game bots
(without intrusion in players’ gaming experience)
Identifying MMORPG Bots: A Traffic Analysis Approach
4
Key Contributions
We proposed to detect bots with a traffic analysis
approach
We proposed four strategies to distinguish bots from
human players based on their traffic characteristics
Identifying MMORPG Bots: A Traffic Analysis Approach
5
Bot Detection: A Decision Problem
Q: Whether a bot is controlling a game client
given the traffic stream it generates?
A: Yes or No
Game client
Game server
Traffic stream
Identifying MMORPG Bots: A Traffic Analysis Approach
6
Ragnarok Online -- a screen shot
Ragnarok Online
One of the most popular MMORPGs
(they claimed 17 million subscribers
worldwide recently)
Notorious for the prevalence of the use of
game bots
Figure courtesy
www.Ragnarok.co.kr
Identifying
MMORPG of
Bots:
A Traffic Analysis Approach
7
Game Bots in Ragnarok Online
Two mainstream bot series:
Kore -- KoreC, X-Kore, modKore, Solos, Kore, wasu, Erok,
iKore, and VisualKore
DreamRO (popular in China and Taiwan)
Both bots are standalone (game clients not needed),
fully-automated, script-based, and interactive
Identifying MMORPG Bots: A Traffic Analysis Approach
8
DreamRO -- A Screen Shot
View Scope
World Map
Character
Status
ter
c
a
r
Ch a
Identifying MMORPG Bots: A Traffic Analysis Approach
er
is h
e
9
Trace Collection
Category
Trace #
Participants
Average Length
Network
Human
players
8 traces
2 rookies
2 experts
2.6 hours
Bots
11 traces
2 bots
17 hours
ADSL,
Cable Modem,
Campus Network
Heterogeneity was preserved
Player skills
Character levels / equipments
Network connections
Network conditions (RTT, loss rate, etc)
206 hours and 3.8 million packets were traced in total
Identifying MMORPG Bots: A Traffic Analysis Approach
10
Traffic Analysis of Collected Game Traces
Traffic is analyzed in terms of
Command timing
Traffic burstiness
Reaction to network conditions
Four bot identification strategies are proposed
Identifying MMORPG Bots: A Traffic Analysis Approach
11
Command Timing
Observation
Bots often issue their commands based on arrivals of server packets,
which carry the latest status of the character and environment
State update
t1
Response time
game client
T = t2 – t1
Client command
t2
game server
time
Client response time (response time)
Time difference between the release of a client packet and the arrival
of the most recent server packet
Identifying MMORPG Bots: A Traffic Analysis Approach
12
CDF of Response Times
DreamRO
> 50% response times are
extremely small
Kore
Zigzag pattern (multiples
of a certain value)
Identifying MMORPG Bots: A Traffic Analysis Approach
13
Histograms of Response Times (DreamRO traces)
Many client packets are sent in response to server packets
1 ms
1 ms
multiple
peaks
Identifying MMORPG Bots: A Traffic Analysis Approach
multiple
peaks
14
Histograms of Response Times
Scheme #1: Command Timing
Regularity
in the distribution
A traffic
stream is considered
from a bot ifof
it bots’
has …
response times
Quick response times (< 10 ms) clustered
Regularity in the distribution of response times, i.e., if
any frequency component exists
Identifying MMORPG Bots: A Traffic Analysis Approach
15
Traffic Burstiness
Traffic burstiness
An indicator of how traffic fluctuates over time
The variability of packet/byte counts observed in successive
periods
Index of Dispersion for Counts (IDC)
The IDC at time scale t is defined as
Var(Nt )
,
It =
E(Nt )
where Nt indicates the number of arrivals in intervals
of time t.
Identifying MMORPG Bots: A Traffic Analysis Approach
16
Example: Wine Sales and IDC
The period is approximately 12 months
The IDC at 12 months is the lowest
Identifying MMORPG Bots: A Traffic Analysis Approach
17
The Trend of Traffic Burstiness
Conjecture for Bot Traffic
1. Each iteration of the bot program’s main loop takes roughly
the same amount of time
2. Each iteration of the main loop sends out roughly the same
number of packets
3. Bot traffic burstiness will be the lowest in the time scale
around the time needed to complete each iteration
Traffic generated by human players, of course, has no reason to
exhibit such property
Identifying MMORPG Bots: A Traffic Analysis Approach
18
Examining the Trend of Traffic Burstiness
Scheme #2: Trend of Traffic Burstiness
Regularity
in the distribution
A traffic
stream is considered
from a bot ifof
… bots’
response times
the IDC curve has a falling trend at first and after that
a rising trend, and
both trends are detected at time scales < 10 sec
Identifying MMORPG Bots: A Traffic Analysis Approach
19
The Magnitude of Traffic Burstiness
Conjecture
Bot traffic is relatively smooth than human player traffic
Difficulty
no “typical” burstiness of human player traffic
Solution
compare the burstiness of client traffic with that of the
corresponding server traffic (as servers treat all game clients
equally)
Scheme #3: Burstiness Magnitude
A traffic stream is considered to be generated by a bot if the
client traffic burstiness is much lower than the corresponding
server traffic burstiness
Identifying MMORPG Bots: A Traffic Analysis Approach
20
Human Reaction to Network Conditions
Conjecture for Human Player Traces
1. The network delay of packets will influence the pace of game
playing (the rate of screen updates, character movement)
2. Human players will unconsciously adapt to the game pace
(the faster the game pace is, the faster the player acts)
Traffic jam!!
server
Is there any relationship between network delay and
the pace of user actions?
Identifying MMORPG Bots: A Traffic Analysis Approach
21
Packet Rate vs. Network Delay
Human player traces: downward trend
Scheme #4: Pacing
A traffic stream is considered from a bot if …
correlation between pkt rate vs. network delay is nonnegative
Identifying MMORPG Bots: A Traffic Analysis Approach
22
Performance Evaluation
Metrics
Correct rate
the ratio the client type of a trace is correctly
determined
False positive rate
the ratio a player is misjudged as a bot
False negative rate
the ratio a bot is misjudged as a human player
Evaluate the sensitivity of input size by dividing
traces into segments, and computing the above
metrics on a segment basis
Identifying MMORPG Bots: A Traffic Analysis Approach
23
Performance Evaluation Results
[Burstiness magnitude]
always achieves low false positive rates (< 5%) and
yields a moderate correct rate (≈ 75%)
[Command timing and Burstiness trend]
Correct rates higher than 95% and false negative rates
lower than 5% given an input size > 2,000 packets
Identifying MMORPG Bots: A Traffic Analysis Approach
24
An Integrated Approach
In practice, we can carry out multiple schemes
simultaneously and combine their results according
to preference
Conservative approach:
command timing AND burstiness trend
Aggressive approach:
command timing OR burstiness trend
Identifying MMORPG Bots: A Traffic Analysis Approach
25
An Integrated Approach -- Results
Aggressive
Aggressive
Conservative
approach
approach
(2,000
(10,000
packets):
packets):
false
≈ 0%negative
false positive
rate <rate
1% and
and95%
> 90%
correct
correct
rate
rate
Identifying MMORPG Bots: A Traffic Analysis Approach
26
Robustness against Counter-Attacks
Just like anti-virus software vs. virus writers
Our schemes only rely on packet timings
An obvious attack is adding random delays to the
release time of client packets
Command timing scheme will be ineffective
Schemes based on traffic burstiness are robust
y
Adding random delays will not eliminate the bot signature
unless the added delay is longer than the iteration time by
orders of magnitude or heavy-tailed
y
However, adding such long delays will make the bots
incompetent as this will slowdown the character’s actions by
orders of magnitude
Identifying MMORPG Bots: A Traffic Analysis Approach
27
Simulating the Effect of Random Delays on IDC
Identifying MMORPG Bots: A Traffic Analysis Approach
28
Summary
Traffic analysis is effective to identify game bots
Proposed four bot decision strategies and two
integrated schemes for practical use
The proposed schemes (except the one based on
command timing) are robust under counter-attacks
Identifying MMORPG Bots: A Traffic Analysis Approach
29
Thank You!
Kuan-Ta Chen