COSC 3P71: Week 2 Notes

Note

Missed the first 3 lectures, starting from Sept. 11, 2025

Reminder

Ombuki wants a 85% major average to be part of the main group for the industry project. I am just below that, but remember to email her anyways since I am interested in working on the project.

Agents

An agent is anything that can be viewed as perceiving its environment through sensors and acting upon that environment through actuators.

Definitions

Percept: content an agent’s sensors are perceiving

• perceptual input (e.g. text, images, sounds)

Percept sequence: a complete history of everything the agent has ever percieved

Rational agent: for each possible percept sequence, a rational agent selects the action that maximizes performance measure, given evidence provided

• does the right thing

Performance measure: criteria for success

• good vs bad
• better vs worse
• clear criterion vs less well defined

Rationality: reasonably correct

• not perfection!

Agent function: maps from percept histories to actions

• [f: P* -> A]

Agent program: runs on the physical architecture to produce f

Agent: architecture + program

• Human agent: sensors (eyes, ears) + actuators (hands, legs, voice).
• Robotic agent: sensors (cameras, infrared) + actuators (motors).
• Software agent: input (files, network, user actions) + output (files, network, display, sounds).
• Environment: only the relevant part of the world that affects/gets affected by the agent.

Robotic vacuum cleaner agent example

• Environment: Two squares (A and B)
• Percepts: Current location + status (e.g., [A, Dirty])
• Actions: Move left, move right, suck (clean), no-op

Percept sequence	Action
[A, Clean]	Right
[A, Dirty]	Suck
[B, Clean]	Left
[B, Dirty]	Suck
[A, Clean], [A, Clean]	Right
[A, Clean], [A, Dirty]	Suck
…	…

function REFLEX-VACUUM-AGENT ([location, status]) returns an action
	if status == Dirty then return Suck
	else if location == A then return Right
	else if location == B then return Left

Note: the vacuum agent program is very small compared to the table

Rationality

What is rational at a given time depends on four things:

1. Performance measure, that defines criteria for success
2. Agent’s prior knowledge of the environment
3. Actions that the agent can perform
4. Percept sequence to date (history)

Rational agent: something that does the right thing (every entry in the table is filled out correctly)

• What is the “right thing”?
  • The most successful agent based on the performance measure
• Performance measure should be objective
  • E.g. amount of dirt cleaned within a certain time
  • E.g. how clean the floor is
  • Performance measure according to what is wanted in the environment instead of how the agent should behave

Nature of task environments

• To design a rational agent, we must specify its task environment (the “problems” to which rational agents are the “solutions”).
• The task environment directly affects the appropriate agent design.
• Designing an agent requires fully specifying the task environment first.
• Use PEAS to describe the task environment:
  • P: Performance measure
  • E: Environment
  • A: Actuators
  • S: Sensors

Automated taxi driver agent example

PEAS Description

• Performance:

  • Safety
  • Fast arrival to destination
  • Maximize profits
  • Legal compliance
  • Comfortable trip
  • Minimize impact on other road users

• Environment:

  • Streets/freeways
  • Other traffic, police, pedestrians
  • Customers, weather

• Actuators:

  • Steering, accelerator, brake
  • Signal, horn, speaker/display

• Sensors:

  • Cameras/video, radar, GPS
  • Accelerometer/speedometer
  • Engine sensors
  • Touchscreen/keyboard

What makes a good performance measure?

• Balances goals: Not just speed or profit, but also safety, legality, and comfort.
• Avoids perverse incentives: e.g., only rewarding speed might encourage reckless driving.
• Considers all stakeholders: passengers (comfort), company (profit), and society (safety, minimal traffic impact).
• Adaptable: Works across different environments (weather, traffic, legal rules).

Medical diagnosis system example

PEAS Description

• Performance measure:

  • Healthy patient outcomes
  • Minimize costs
  • Avoid lawsuits

• Environment:

  • Patient
  • Hospital
  • Staff

• Actuators:

  • Screen display (questions, test requests, diagnoses, treatments, referrals)

• Sensors:

  • Touchscreen/voice input
  • Patient’s symptoms, findings, answers

What makes a good performance measure?

• Patient-focused: Ensures accurate diagnosis and effective treatment.
• Cost-aware: Balances healthcare quality with affordability.
• Risk-reducing: Minimizes harm, misdiagnosis, and legal exposure.
• Holistic: Considers patient health, hospital efficiency, and staff usability.

Environment types

The range of task environments arising in AI is wide.

• environments can be categorized with a fairly small number of dimensions
• to a large extent, these dimensions determine the appropriate agent design and the applicability of each of the principal families of techniques for agent implementation

Properties of environment types

Fully Observable

• Agent’s sensors access the complete state of the environment at all times.
• All relevant aspects for decision-making are visible.
• Convenient: no need to maintain internal state.

Partially Observable

• Sensors are noisy, inaccurate, or provide incomplete information.
• Hidden/missing data about the environment.
• Examples:
  • Automated taxi cannot see what other drivers are thinking.
  • Vacuum with only a local dirt sensor cannot detect dirt in other squares.

Deterministic vs. Stochastic

• Deterministic:

  • Next state fully depends on current state + agent’s action.
  • No uncertainty if environment is fully observable.
  • Example: vacuum world (deterministic variant), crossword puzzle.

• Stochastic:

  • Next state is probabilistic; outside factors influence outcomes.
  • Example: taxi driving (traffic behavior, random tire blowouts).

Episodic vs. Sequential

• Episodic:

  • Tasks are independent, self-contained.
  • Each decision is based only on the current situation.
  • Example: spotting defective parts on an assembly line (each part judged separately).

• Sequential:

  • Current decision affects future outcomes.
  • Example: driving a taxi, cleaning strategy for vacuum agent.

Static vs. Dynamic

• Static:
  • Environment doesn’t change while the agent is deliberating.
  • Example: Crossword puzzle.
• Dynamic:
  • Environment can change during decision-making.
  • Example: Taxi driving.

Discrete vs. Continuous

• Discrete:
  • Finite number of states, percepts, and actions.
  • Example: Chess (discrete moves and states).
• Continuous:
  • Smooth, infinite-scale states and time.
  • Example: Taxi driving (continuous position, speed, time).

Single-Agent vs. Multi-Agent

• Single-Agent:
  • Only one agent in the environment.
  • Example: Crossword puzzle.
• Multi-Agent:
  • More than one agent → adversarial or cooperative.
  • Example:
    • Chess = competitive multi-agent.
    • Taxi driving = cooperative (traffic flow) and competitive (right of way).