08. Personalized Information Access (Guest lecture)

Key Ideas...
• Information Access Modalities
• Navigation, Search, Recommendation
Personalized Information Access!
• Interactions & Feedback
• Mining users interactions rather than item content.
Barry Smyth
University College Dublin, Ireland
• Profiling & Privacy
• The Economics of Personalization
Information Access Modalities
Lessons learned along the way ...
Screen Size
Searching
tion
om
Rec
• Exploit domain constraints. Keep it simple. Build for scale.
da
men
• Learn from live-user trials to understand real benefits.
Browsing
Input Capabilities
“Joystick”
+/- Keypad
• Understand user problems to identify application sweetspots.
Touch-Screen
Keyboard
Mouse +
Keyboard
The Case for Personalization
The Road to Personalization
• Information Overload
• Personalization | Browsing
• Device Variability
• PC, Mobile, TV, ...
• Interactions
Adaptation
• Profiling & Privacy
• Individuals / sessions / communities
s
ofile
r navigation
• Menu adaptation in mobile portals; Large-scale, profiling
r pand
use
prediction
files
pro
n
• Targeted feedback in conversational recommendation;
sioIn-session adaptation &
ses
mining compound critiques
• Personalization | Recommendation
les
rofi
p
• Exploiting query repetition and selection regularity in Web
nitysearch; Communitybased profiling for anonymous personalization mmu
co
• Personalization | Search
• The Economics of Personalization
• Benefits & Costs
Browsing on the Mobile Internet
1. Personalization | Browsing
Adapting the Structure of Mobile Portals to the Navigation Patterns of Users
Mobile Home
Entertainment
Cinema
News
Email
Events
Music.
Music
Movie Times
Booking.
Banking
Travel
TV
Cinema
Reviews
Gossip
Ents.
Lifestyle
Dining
Comedy
Meetings
Ringtones
Browsing on the Mobile Internet
The Click-Distance Problem
30
Movie Times
Pick a cinema …
Derry
UCI Tallaght
Donegal
UCI Coolock
Down
UCI Blanchardstown
Dundalk
Ster Century
Dublin
Ormonde
Welcome
Now Showing
New Releases
Booking Office
Coming Soon
CD < 12 CD < 6
Pick a location …
Mean Click-Distance
25
Movie Times
20
15
10
5
KEY:
0
Portals
High-Volume Usage
Regular Usage
Home
Content
Mobile portals are compromised by large click-distances that render up
to 75% of content all but invisible to the most determined of users.
Mining Navigation Trails
Link Reordering & Promotion
Navigation Profile
Reordering
Promotion .........
A (40)
C (30)
B (10)
Mobile
Home
Entertainment
D
(5)
E
(5)
F
(20)
(10)
G
Menu Construction
Recording user navigation trails provides
a basis for profiling ...
... which in turn can be used as the
basis for a probabilistic navigation
model.
P(F|A)=P(C|A)•P(F|C)
=(30/40)•(20/30)
=0.5
Each personalized version of a requested
menu, m, is constructed by adding the k most
probable options, which are descendants of m.
By default all of m ‘s options are then ordered
according to their access probabilities.
Ents.
Events
News
Music.
Ents.
Email
News
Ents.
Email
Cinema
Banking
Ents.
Ster
Century
Banking
Travel
TV
Travel
Ents.
Comedy
Lifestyle
Link Reordering & Promotion
Deployment Architecture
• Scalability Issues [Smyth & Cotter, AH 2002]
• Depth-limited, breadth-first search
for real-time promotion and reordering.
P(o|m) ! P(o’|m’)
if o’! m’ and Descendant(m’,m)
Expand o’ iff P(o’|m) > kth best
probability so far.
k=2
A
C 0.786
F
0.214 B
0.142
D
E
F
G
0.642
• Mobile OpCo requirements: approx. 1000 personalized pages per
second with a latency of <500msec.
Trial opt-in page for all 66/67
users at portal home page.
BASELINE
8 weeks
Analysis Period
TRIAL
8 weeks
10
0%
9
8.7
8
7
6
-6%
8.1
-7%
8.1
Approx.
4-5% reduction
per week
-11%
0%
-14%
-23%
7.7
-29%
7.4
6.7
6.2
-32%
-40%
5.9
-60%
5
4
-80%
TW1 TW2 TW3 TW4 TW5 TW6 TW7 TW8
~800 unique GPRS trialists
(~80,000 full subscriber base)
-20%
Relative Benefit (%)
Click-Distance
Average Click Distance
per Session
Evaluation
Total Requests
150%
Active Users
250
102.63%
110%
90%
200
70%
150
100
141
50%
30%
0.00%
10%
50
20000
-10%
PreTrial
Trial
Total Requests
130%
Relative Benefit (%)
286
300
17852
74.97%
70%
10203
10000
20%
0
0
-30%
PreTrial
Trial
Lessons Learned
• Selling AI ...Return on Investment is key!
• Usability " Click-Distance " Usage
• The “AI Blackbox” vs the need for operator controls.
• Integrated portal management and personalization administration.
• Personalization changes the way that operators manage their portals!
• User perceptions & the personalization-privacy trade-off
• Satisfaction, usage, bandwidth, privacy, 90%+ opt-in rates.
• See also: [Smyth et al. IAAI 2007, AI Magazine (forthcoming)]
120%
2. Personalization | Recommendation
Mining Feedback in Conversational Recommender Systems
Relative Benefit (%)
Active Users
Recommender Systems (Single-Shot)
Conversational Recommender Systems
{c1,...,cn}
Recommend
Review
User feedback
Q
Profile
R
Items
f
Q’
Revise
“Show me more like this but cheaper”
(Unit critique over the price feature)
The Problem with (Unit) Critiques ...
• Session Analysis
• Protracted sessions in simple product-spaces.
• False-leads.
Recommend(c,f):1. Filter remaining cases by critique, f.
• Changes of mind.
• Dynamic Critiquing
• Unit
Compound Critiques
Target Similarity
Critiquing-Based Recommenders
1
0.8
0.6
0.4
0.2
0
2. Rank filtered cases by similarity to, c.
3. Return top k (k=1) most similar cases.
1
• Incremental Critiquing
6
11
16
Cycle
21
• How to adapt recommendations in response to new and changing
preferences?
26
Compound Critiques
[Reilly, Zhang et al. EC 2007]
Recommendation
Compound
Critiques
UNIVERSITY COLLEGE DUBLIN
C4
Each remaining case is converted in to a critique pattern to reflect the
differences between it and the current recommendation....
Cn
sk
ice
di
y
rd
-
or
Pr
C3
Ha
C2
em
r
C1
ito
Critique Pattern
!=
<
<
<
=
<
!=
>
M
Alternative
Sony
12
256
30
Pentium 3
900
Laptop
3000
on
Recommendation
Compaq
14
512
40
Pentium 3
1400
Desktop
1500
From Critique Patterns to Compound Critiques
M
Critique Patterns
er
Smart Media Institute
Manufacturer
Monitor
Memory(MB)
Hard-Disk(GB)
Processor
Speed(Mhz)
Type
Price(!)
Generate compound critiques on-the-fly by using data mining
techniques to extract common patterns of critiques from the remaining
cases (relative to the current recommendation)
ak
Unit
Critiques
Dynamic Critiquing: Creating Compound Critiques
M
14 of 26
Standard Unit Critiques
Recommendation
=
=
"
"
>
>
<
<
<
<
<
>
>
<
>
>
>
<
<
<
"
>
>
>
>
Maker
=
Monitor
>
Memory
<
Maker
"
Hard-disk
>
Price
<
Ranking & Selecting Compound Critiques
Incremental Critiquing
• Use Apriori to produce compound critiques (for the current cycle)
containing 2-3 unit critiques
• Large lists of candidates (50+/cycle in typical scenarios)
[Price < 995]
[Memory < 512]
[Resolution < 6.2]
[Price < 405]
[Manufacturer " Sony]
[Resolution < 5]
[Price < 995]
[Memory < 512]
[Resolution < 6.2]
[Price < 405]
[Manufacturer ! Sony]
[Memory < 512]
[Resolution < 5]
• Ranking based on support and confidence thresholds
• Low-support critiques shown to provide best balance of filtering power
and likely applicability Best-k (k=3) compound critiques to present
• See also:
1. Comparison of ranking strategies (Reilly et al, ECCBR 2004)
2. MAUT critique mining (Zhang & Pu, AH 2006; Reilly, Zhang et al, IUI’07)
U
Incremental Critiquing
Incremental Critiquing
• Incremental critiquing constructs an in-session user model, U,
from the critiques provided by the user during each cycle.
• Recommending a new case ...
• Sometimes critiques are inconsistent with previous critiques
stored in U
• Contraditory critiques - [Price > !750] vs. [Price < !500]
• Complementary critiques - [Resolution > 5M] vs. [Resolution > 6.2M]
• The user model is maintained by a “newer is better” strategy in
which more recent critiques over-ride past critiques.
• Incremental critiquing considers the user’s critique history (U) during
recommendation.
• Each candidate case c’ is ranked according to its similarity to c (the
current case) and its compatibility with U.
• Candidates are preferred if they are similar to c and if they satisfy many
of the users past critiques.
Compatibility(c’,U) =
!!i
satisfies(Ui,c’)
U
Quality(c,c’,U) = Compatibility(c’,U) * Similarity(c,c’)
Evaluation
Standard Critiquing
• E-Commerce Scenario
• Online digital camera store
• 200+ cameras
• Trial Setup
• 76 participants were asked to use the
online store to ‘shop’ for a digital camera of their choice.
• Low vs high freq. use of compound critiques (0"100%, median " 20%)
• Systems Compared
• Standard vs Incremental Critiquing x Unit vs Compound Critiques
Incremental Critiquing
User Satisfaction
Lessons Learned
• Critiquing is an effective form of feedback, but unit critiques can lead to
protracted sessions (false-leads, dead-ends etc.)
• Dynamic, compound critiques offer a richer form of feedback.
3. Personalization | Search
• Incremental critiquing facilitates a more natural form of product exploration
that adapts to the in-session behaviour of the current user.
Harnessing Community Expertise in Collaborative Web Search
• Significant benefits in session length without compromising user satisfaction.
• No persistent profiles
Limited privacy implications.
Collaborative Web Search
The Challenges of Web Search
• Observations on Web Search ...
• Vague Queries
• Vague queries and the vocabulary gap.
• Jaguar, Jaguar, Jaguar, ....
• Repetition & regularity in communities of like-minded searchers.
• Query Vocabulary # Indexing Vocabulary (The Vocabulary Gap)
• Query profiling is a privacy nightmare!
• Generic Search vs Community-Based Search
• Community-based personalization
• Communities are commonplace on the modern Web (ad hoc, formal, ...)
• Leverage existing search engine resources.
• A Case-Based Reasoning Perspective.
- Reuse of search cases
search expertise
• No need for individual search profiles.
community promotions
• Communities often search for similar things in similar ways
• Query repetition and selection regularity is commonplace within
communities or like-minded searchers.
Vague Queries
The Vocabulary Gap
Generic vs Community-Based Search
Repetition & Regularity in Web Search
• Generic search engines (Google, Yahoo, Ask,...) appear to
dominate the world of Web search ....
• ... but many search sessions reflect the niche interests of
communities of like-minded individuals
• ... and there are increasingly many examples of searchers
searching in a community context.
• Students searching for information about a particularor term-paper;
• Related employees searching for work related materials;
• Visitors to a themed web site availing of syndicated search services.
• ...
A Case-Based Reasoning Perspective
15
• Promotion " Case Reuse
• Most relevant selections are
promoted within the result-list.
• Result relevance based on query
relevance and query similarity.
jaguar pics 10
33
jaguar
21
“jaguar pics”
“used jaguar cars”
= (21/89)*1+(33/43)*1/2+(5/44)*1/3
(1 + 1/2 + 1/3)
= 0.64
0.36
-p
“jaguar”
ca
r
Relevance(“jaguar”,jaguarcars.com)
5
56
25
14
ho
to
s.c
jag
om
ua
rca
rs.
co
m
Mo
to
rw
or
ld
us
ed .com
ca
rs .
co
Ju
m
stJ
ag
s.c
om
used jaguar
cars
12
rca
to
A Case-Based Reasoning Perspective
ua
ho
[Smyth et al., JIR 2006, UMUAI 2004]
jag
Sn
r-p
S2
ca
S1
56
25
14
m
s.c
Ju
o
stJ
ag m
s.c
om
HTj
Adapter
m
Adapter
5
s.c
o
Adapter
used
jaguar
cars
12
.co
• Select search cases for similar
queries.
21
ar
pj
RM
META-SEARCH
jaguar
dc
• Query Reuse " Case
Retrieval
rld
qT
33
us
e
qT
om
qT
• Cases capture community
search experiences as a
ci jaguar pics 10
query and a set of selections.
rw
o
RT
Sol(ci)
Spec(ci)
rs.
c
RELEVANCE ENGINE
• Hit-Matrix " Case-Base
to
HIT MATRICES
Mo
Collaborative Web Search Architecture
Evaluation
Repetition & Regularity in Web Search
• Enterprise Search Scenario [Coyle & Smyth, ACM TOIT, IEEE Computer 2007]
• Community composed of the employees of a local software company;
search redirected through CWS-enhanced Google.
• Approx. 70 employees over a 10 week period > 12,600 individual search
sessions
• Methodology
• Promoted vs. Standard Sessions
• Successful vs. Failed Sessions
• Promotion Sources (Self vs. Peer)
Average Success Rates
Sources of Promotions
85% of participants became actively involved in the
creation and consumption of search knowledge.
Lessons Learned
• Many Web search scenarios have a community context.
20% of participants acted as search leaders, being
frequently first to contribute search knowledge on a
particular topic.
20% of participants acted as search followers,
primarily consuming search knowledge created by
others.
Further Work...
• Beyond click-thru mining [Boydell & Smyth, IUI 2007, SIGIR 2006]
• Snippet-based Collaborative Web Search
• Generating community-focused result summaries
• Click-spam and trust-based filtering [Briggs & Smyth, AIR, IUI, ECIR 2007]
• Malicious users
forced promotion
• Towards Inter-Community Web Search [Freyne & Smyth, AH,ECCBR 2006]
• Sharing result promotions between related search communities.
• Community experrise " folksonomy learning
• Communities of like-minded searchers search for similarity
information in similar ways.
• Community search histories can be harnessed as a source of
search expertise and used to enhance standard search results.
• Aggregating community search experiences facilitates a form of
anonymous, personalized Web search.
Conclusions
With special thanks to...
• Personalized Information Access
• University College Dublin
• Navigation, Recommendation, Search
• Profiling strategies (sessions, individuals, communities)
• Privacy and the Economics of Personalization
• Trading privacy for features ...
• Mobile subscriber take-up; Recent FaceBook/MySpace studies
• Next Steps ...
• Social information access - integrating navigation, search and
recommendation [Farzan, Coyle, et al, IUI, Hypertext 2007]
• ...
• Evelyn Balfe, Oisin Boydell, Peter Briggs, Maurice Coyle, Jill Freyne,
Michael O’Mahony, Karen Church, James Reilly, Kevin McCarthy,
Lorraine McGinty
• Collaborators
• ChangingWorlds Ltd.
• Peter Brusilovsky, Rosta Farzan (University of Pittsburgh)
• Pearl Pu, Jiyong Zhang, Ecole Polytechnique Fédérale de Lausanne
(EPFL)
• Maria Salamo, Universitat de Barcelona