Knowledge-based agents

Games & Agents 2012-2013
Games & Agents
Lecture 2
Agents and Intelligence
Jan Broersen
J.M.Broersen@uu.nl
Tuesday, May 28, 13
Games & Agents 2012-2013
The subject of today
I will give a very broad overview of different ideas from AI about
endowing agents with intelligence.
First: a movie about how the game-industry looks at intelligence
J.M.Broersen@uu.nl
Tuesday, May 28, 13
Games & Agents 2012-2013
The AI in Killzone 2
J.M.Broersen@uu.nl
Tuesday, May 28, 13
Intelligence
Games & Agents 2012-2013
•
In the trailer:
– (Agent) intelligence: (1) enemies pin you down and rush up to
you, (2) you do not see the same thing twice, (3) the
environment is destructible (4) the AI responds to changes
in the environment, (5) lean and peak system
– Multi-agent intelligence: buddies in the game that tell you to
take cover, saving your skin
•
In AI:
– Intelligence = intelligent behavior (we endorse Turing’s test)
– Intelligent behavior = selecting the ‘intelligent choices’ from
all the ones that are feasible and performing them
– Agent theory: intelligence is a function from percepts to
actions; the function can either be programmed or learned
J.M.Broersen@uu.nl
Tuesday, May 28, 13
Games & Agents 2012-2013
Simple reflex agents
J.M.Broersen@uu.nl
Tuesday, May 28, 13
Games & Agents 2012-2013
Simple Reflex Agent
J.M.Broersen@uu.nl
Tuesday, May 28, 13
Decision tree
Games & Agents 2012-2013
Conditionalization of actions is
of the simplest kind
In the tree: decisions at the
squares depend only on
observations at these squares.
Cycle: sense, act
J.M.Broersen@uu.nl
Tuesday, May 28, 13
Are these used in computer games?
Games & Agents 2012-2013
Yes: these are the scripted agents
Where do agent actions in scripts conditionalize on?
Very simple information like the player being nearby
However, intelligent choices should
1.
2.
3.
4.
be conditional on the state of the environment
be conditional on the state of other agents
be conditional on internal states
adapt after earlier experiences
J.M.Broersen@uu.nl
Tuesday, May 28, 13
Shortcomings simple reflex agents
Games & Agents 2012-2013
(1) No conditionalization on appropriate circumstances
(2) No situation awareness
(3) No looking ahead for decision making
=> agents need a model of the world, that is, we have to give them
knowledge
=> agents need a goal in life, that is, we have to give them
motivations (desires, intentions, obligations, etc.)
J.M.Broersen@uu.nl
Tuesday, May 28, 13
Games & Agents 2012-2013
Knowledge-based agents
J.M.Broersen@uu.nl
Tuesday, May 28, 13
Games & Agents 2012-2013
Reflex + State Agent
J.M.Broersen@uu.nl
Tuesday, May 28, 13
Games & Agents 2012-2013
Decision tree
Decisions conditionalize not
directly on the percepts, but
on the agent’s model of the
world (recalls what happened
before and what it has done)
How the world is, may now be
derived instead of observed
Allows for ‘reasoning’ before
acting
Cycle: sense, reason, act
J.M.Broersen@uu.nl
Tuesday, May 28, 13
Games & Agents 2012-2013
Knowledge-based agents
Perceived as a major break-through in the late 50ies, just after the
birth of artificial intelligence as a discipline (maybe the same
brake-through still has to be realized in game-AI..)
Model not only knowledge, but also lack thereof (uncertainty)
Logic based approach 1: epistemic logic
Logic based approach 2: non-monotonic reasoning
Bayesian approach: beliefs as probabilities
J.M.Broersen@uu.nl
Tuesday, May 28, 13
Are these used in computer games?
Games & Agents 2012-2013
Hardly (do you know of examples?)
However, if we agree the cognitive SOAR architecture gives us
knowledge based agents, then we do have an interesting
example.
Paper: [It knows what you’re going to do: adding anticipation to a
Quakebot, John Laird]
J.M.Broersen@uu.nl
Tuesday, May 28, 13
Games & Agents 2012-2013
A cognitive architecture: SOAR
Close to ACT-R
SOAR is claimed to be:
a candidate unified theory of cognition (Newell, 1990),
and
an architecture for intelligent behaviour
Decisions are never precompiled into uninterruptible
sequences => continuous (and concurrent) interaction
with the environment and with multiple internal
processes.
J.M.Broersen@uu.nl
Tuesday, May 28, 13
The SOAR Architecture
Games & Agents 2012-2013
SOAR is an ‘AI engine’ for making and executing decisions
Basic objects of decision: operators (we would call them actions)
Operators are either environmental actions, internal actions,
complex actions, or abstract goals.
SOAR is completely rule-based
So, far, very much like 3APL/2APL
J.M.Broersen@uu.nl
Tuesday, May 28, 13
The SOAR Architecture
Long-Term Memories
Games & Agents 2012-2013
Procedural/Skill
Semantic
Learning
Rule
Learning
Short-Term Memory
Control
Procedure
Action
Perception
J.M.Broersen@uu.nl
Tuesday, May 28, 13
Semantic/Concept
Body
Episodic
Episodic
Learning
Games & Agents 2012-2013
But, unlike in 3APL/2APL:
There is no deliberation or control language! Also there is no
predefined order in which to apply rules: the system is
completely data-driven.
The current state (‘working memory’) consists of perceptions,
elaborations on it, and everything else derived by earlier rule
applications.
SOAR continually and concurrently proposes, selects and applies
operator rules to the current state.
Abstract (goal) operators that cannot be executed directly generate
sub-states for which additional operators are proposed, selected
and applied (remember HTN-planning)
J.M.Broersen@uu.nl
Tuesday, May 28, 13
Adding anticipation to a Quakebot
Games & Agents 2012-2013
•
•
•
Central idea: enable a Quakebot to project itself in its
enemies to predict its behavior and to take advantage of
that.
Similar to a 1 ply deep minimax or alfa-beta algorithm
(Chess, game theory) where the idea is also to use the
same evaluation function for opponents.
Add three new sets of rules to SOAR bots:
1. For deciding when to start predicting
2. For generating expected opponent behavior
3. For proposing when to stop predicting and for how to
take advantage of the prediction (e.g. ambush)
J.M.Broersen@uu.nl
Tuesday, May 28, 13
When to start predicting? (task 1)
Games & Agents 2012-2013
Not always..
Decided by a special set of rules
Which are up to the Quakebot designer..
J.M.Broersen@uu.nl
Tuesday, May 28, 13
Games & Agents 2012-2013
Generating expected behavior (task 2)
First the quake bot generates an expected internal state of its
opponent, by applying special rules on his perceptions of the
other (his position, health, armor level, current weapon).
Then he applies his own behavioral rules on this state, thereby
projecting himself in the other!
The behavioral projection works on the generated internal state and
projected environment, which is less computational intensive
than the ‘real thing’.
J.M.Broersen@uu.nl
Tuesday, May 28, 13
Games & Agents 2012-2013
Predictions, and how they are used (task 3)
Predictions should not go on forever: special rules decide when to
stop. For instance if an ambush is not possible due to
environmental conditions or the timing.
Predictions also stop when they encounter too much uncertainty
(impasses in the prediction!).
In the paper’s set-up the technique is only used to hunt, ambush and
deny power-ups.
J.M.Broersen@uu.nl
Tuesday, May 28, 13
Games & Agents 2012-2013
Limitations
Anticipations is restricted to situations deemed worthwhile by the
programmer.
So, tactics are not ‘discovered’ but preprogrammed by the Quake
bot designer.
Alternative: always do predictions, and add general evaluation
functions and planning ⇒ too expensive, computationally.
J.M.Broersen@uu.nl
Tuesday, May 28, 13
Conclusions SOAR application to Quake
Games & Agents 2012-2013
SOAR quake bots use about 750 rules.
Adding anticipation requires only 4 rules for deciding when to start
predicting, 16 to generate an approximate opponent internal
state, 5 to use the prediction.
So adding anticipation turns out to be quite easy, from a
programming point of view.
But, computational much more intensive.
J.M.Broersen@uu.nl
Tuesday, May 28, 13
Shortcomings of knowledge-based agents
Games & Agents 2012-2013
Goals are implicit in the set of rules
Agents decide without explicitly looking ahead in the tree
J.M.Broersen@uu.nl
Tuesday, May 28, 13
Games & Agents 2012-2013
Goal-based Agent
J.M.Broersen@uu.nl
Tuesday, May 28, 13
Deliberation cycles for the different agent types
Games & Agents 2012-2013
Simple reflex agents: sense, act
Reflex agents with state: sense, reason, act
Goal-based agents: sense, update beliefs, select goals, select
intentions, etc..., act
J.M.Broersen@uu.nl
Tuesday, May 28, 13
Games & Agents 2012-2013
The tree view: planning
J.M.Broersen@uu.nl
Tuesday, May 28, 13
Games & Agents 2012-2013
Are these agents used in computer games?
Yes: path-planning using A*
Knowledge: the network of waypoints
Goal: finding a shortest route
However, this is a very specific type of planning, because actions
(movements) and goals (wanting to be somewhere else)
themselves are very specific
Knowledge about waypoints is simply assumed (and not learned
through observation)
Because the problem is so specific, we might even consider to
calculate most of it offline for a given fixed environment..
J.M.Broersen@uu.nl
Tuesday, May 28, 13
Planning
Games & Agents 2012-2013
See course “intelligent systems”
State space planning versus plan space planning (HTN planning)
Planning under uncertainty (conformant planning, contingency
planning, etc.)
Main focus of research: controlling complexity, heuristics
J.M.Broersen@uu.nl
Tuesday, May 28, 13
BDI-theory
Games & Agents 2012-2013
Prominent theory: Belief-Desire-Intention
Issue: how do all three depend on each other?
Example: commitment strategies [Cohen and Levesque]: you keep
an intention as long as you belief the associated desire to be
attainable
Example: realism: you do not desire things you believe to be true
already; you do not believe things only because they match your
desire (wishful thinking)
J.M.Broersen@uu.nl
Tuesday, May 28, 13
3APL/2APL
Games & Agents 2012-2013
Inspired by BDI
Goal-base, Belief-base, and Plan-base (intentions)
A language to specify the deliberation cycle
Intuitions from Prolog
J.M.Broersen@uu.nl
Tuesday, May 28, 13
Shortcomings
Games & Agents 2012-2013
Knowledge is not ‘fine grained’
Goals are not ‘fine grained’
J.M.Broersen@uu.nl
Tuesday, May 28, 13
Games & Agents 2012-2013
Utility-based agents
J.M.Broersen@uu.nl
Tuesday, May 28, 13
Games & Agents 2012-2013
Utility-based agent
J.M.Broersen@uu.nl
Tuesday, May 28, 13
Games & Agents 2012-2013
Maximal expected utility
• Cycle: sense, find optimal
tactic given beliefs,
utilities, act
• Reasoning is now in terms
of utilities, chances and
look aheads: MEU, MDPs,
PoMDPs
J.M.Broersen@uu.nl
Tuesday, May 28, 13
Games & Agents 2012-2013
MDPs and PoMDPs
Again see the theory explained in the course “intelligent
systems” (Policies, Bellman equations, value iteration, policy
iteration, etc.)
I do not know of any example from computer games where this
agent theory is applied (do you?)
J.M.Broersen@uu.nl
Tuesday, May 28, 13
Bayesian update
Games & Agents 2012-2013
Bayes' theorem adjusts probabilities given new evidence in the following way:
Where H represents a specific hypothesis, which may or may not be some null
hypothesis.
P(H) is called the prior probability of H that was inferred before new evidence,
E, became available.
P(E | H) is called the conditional probability of seeing the evidence E if the
hypothesis H happens to be true. It is also called a likelihood function when it
is considered as a function of H for fixed E.
P(E) is called the marginal probability of E: the a priori probability of
witnessing the new evidence E under all possible hypotheses.
P(H | E) is called the posterior probability of H given E.
J.M.Broersen@uu.nl
Tuesday, May 28, 13
Games & Agents 2012-2013
The three prisoners
Three prisoners A, B and C are told they have a 2/3 chance of being
executed.
At a certain point the decision is made by the director of the prison.
Then the director visits prisoner A. A asks: will B or C be
executed? The director answers: B
What is A’s chance of survival?
Answer: unclear!
J.M.Broersen@uu.nl
Tuesday, May 28, 13
Games & Agents 2012-2013
The three doors
The Monty Hall dilemma
Conditionalizing on extra information
The chances for getting the car when not changing are 1/3. When
changing, the chances are 2/3! (1/3 for initially correct and
changing to the wrong door and 2/3 for initially wrong after
which changing guarantees the good outcome).
J.M.Broersen@uu.nl
Tuesday, May 28, 13
Shortcomings
Games & Agents 2012-2013
• However, No learning (like all of the previous)
• Or, is there?
• Any update of the model of the world can be seen as a
learning action
J.M.Broersen@uu.nl
Tuesday, May 28, 13
Games & Agents 2012-2013
Learning Agent
Can be combined with any of the previous
J.M.Broersen@uu.nl
Tuesday, May 28, 13
Games & Agents 2012-2013
Trends in Neural Networks
There has always been the “black box” criticism towards neural
networks (I call it “the tweaking problem”)
Current trend: add structure to neural networks
How should such structure look like?
Inspiration from biology?
NEAT: the structure of the neural network evolves determined by a
genetic algorithm
Liquid state machines
Neuro-symbolic computing
J.M.Broersen@uu.nl
Tuesday, May 28, 13
Games & Agents 2012-2013
Rationality versus emotion
J.M.Broersen@uu.nl
Tuesday, May 28, 13
Games & Agents 2012-2013
Some functions of emotions
Frijda: emotions are (search) heuristics.
Emotions may thus be said to guide planning.
E.g. happiness may keep an agent on the right track, because it
makes him selecting similar choices to reinforce his happiness.
E.g. anger or distress may guide an agent into selecting choices he
would otherwise not consider. Those choices maybe the solution
to his problem.
A thought: A* is an optimistic algorithm. Because of that it is
complete (always finds a solution if it exists) and optimal (it will
always find the ‘best’ solution). Optimism seems to be related to
positive emotions...
J.M.Broersen@uu.nl
Tuesday, May 28, 13
Games & Agents 2012-2013
Emotions and rationality
What is the relation between rationality and emotion? Both are
involved in selecting (intelligent) decisions.
For instance, rationally you estimate it is better to kill your enemy,
or he will probably kill you. But, emotionally you feel extremely
bad about killing.
Or, rationally you estimate it is good to punish your child,
emotionally you detest it.
There is an example case of a man without emotion due to brain
damage. He kept an average IQ, but could no longer decide on
simple things like what to wear.
J.M.Broersen@uu.nl
Tuesday, May 28, 13
Games & Agents 2012-2013
The Multi-Agent paradigm
J.M.Broersen@uu.nl
Tuesday, May 28, 13
Computing perspective
Games & Agents 2012-2013
Multi-agent systems constitute a new computational paradigm
based on:
Distribution of computational power
Interconnection and communication
Autonomy and Intelligence
Delegation and distribution of control
Anthropomorphic concepts
J.M.Broersen@uu.nl
Tuesday, May 28, 13
Software engineering perspective
Games & Agents 2012-2013
Multi-agent systems are a new software engineering paradigm.
Analysis ⇒ Design ⇒ Implementation ⇒ Test
The aim is to provide high-level abstraction, to model and develop complex
systems
70’s: Structural analysis methodology
80’s / 90’s: Object-oriented methodology
90’s / 00’s: Agent-oriented methodology
J.M.Broersen@uu.nl
Tuesday, May 28, 13
Artificial intelligence perspective
Games & Agents 2012-2013
Multi-agent systems and Artificial Intelligence do not coincide.
Artificial Intelligence
Planning
Learning
Vision
Language understanding
Multi-agent Systems
Understand and model social intelligence and emergent behavior
Interaction and Communication
Social concepts: obligation, norms, responsibilities, etc.
Optimal solutions can be obtained by co-ordination and co-operation
Simulation can verify social and economic theory
J.M.Broersen@uu.nl
Tuesday, May 28, 13
Games & Agents 2012-2013
Characteristics of multi-agent systems
Multi-agent systems consist of a number of interacting autonomous agents.
Multi-agent systems are designed to achieve some global goal.
Agents need special abilities to cooperate, coordinate, and negotiate to achieve
their objectives.
Multi-agent systems are specified in terms of high-level abstract concepts such as
role, permission, responsibility, and interaction.
Applications for multi-agent systems are, for example, power management
systems, transportation systems, and auctions.
J.M.Broersen@uu.nl
Tuesday, May 28, 13
Autonomous agents
Games & Agents 2012-2013
Autonomous agents are intelligent agents from a multi-agent perspective.
Autonomous agents are active, social, and adaptable computer systems situated in
some dynamic environment that autonomously perform actions to achieve
their objectives.
1.
2.
3.
4.
Reactivity: responding to changes in the environment
Pro-activity (deliberativeness): goal-directed behavior
Social awareness: interaction and communication
Rationality: maximization of utilities
•
•
Agents decide which action to perform based on their internal state.
The internal state of agents can be specified in terms of high-level abstract
concepts such as belief, desire, goal, intention, plan, and action.
J.M.Broersen@uu.nl
Tuesday, May 28, 13
Games & Agents 2012-2013
The End
J.M.Broersen@uu.nl
Tuesday, May 28, 13