Markov Decision Process

A mathematical framework for modelling decision-making in situations where outcomes are partly random and partly under the control of a decision-maker. In accessibility and assistive technology, Markov decision processes and their extension, partially observable Markov decision processes (POMDPs), are used to build adaptive systems that can make intelligent decisions about when and how to intervene based on observations of a user's behaviour. For example, an assistive system might use a POMDP to decide whether to prompt a person with dementia during an activity based on inferred engagement levels, balancing the benefit of maintaining engagement against the cost of unnecessary interruptions that could cause confusion.