Randomised decision rule

Uses your system voice; voice and rate in settings.

Snapshot

≈ 7 min read

Names in other languages

No names in other languages are held for this entry yet.

Article bodies are English and are not machine-translated. A name in another language is not a translation of the article.

In statistical decision theory, a randomised decision rule or mixed decision rule is a decision rule that associates probabilities with deterministic decision rules. In finite decision problems, randomised decision rules define a risk set which is the convex hull of the risk points of the nonrandomised decision rules.

As nonrandomised alternatives always exist to randomised Bayes rules, randomisation is not needed in Bayesian statistics, although frequentist statistical theory sometimes requires the use of randomised rules to satisfy optimality conditions such as minimax, most notably when deriving confidence intervals and hypothesis tests about discrete probability distributions. A statistical test making use of a randomized decision rule is called a randomized test.

In statistical decision theory, a randomised decision rule or mixed decision rule is a decision rule that associates probabilities with deterministic decision rules. In finite decision problems, randomised decision rules define a risk set which is the convex hull of the risk points of the nonrandomised decision rules.

As nonrandomised alternatives always exist to randomised Bayes rules, randomisation is not needed in Bayesian statistics, although frequentist statistical theory sometimes requires the use of randomised rules to satisfy optimality conditions such as minimax, most notably when deriving confidence intervals and hypothesis tests about discrete probability distributions. A statistical test making use of a randomized decision rule is called a randomized test.

Let \mathcal D =\{d_1, d_2 ..., d_h\} be a set of non-randomised decision rules with associated probabilities p_1, p_2, ..., p_h . Then the randomised decision rule d^* is defined as \sum_{i = 1}^h p_i d_i and its associated risk function R(\theta, d^*) is \sum_{i = 1}^h p_i R(\theta, d_i) . This rule can be treated as a random experiment in which the decision rules d_1, ... ,d_h \in \mathcal D are selected with probabilities p_1, ... p_h respectively. Alternatively, a randomised decision rule may assign probabilities directly on elements of the actions space \mathcal A for each member of the sample space.

More formally, d^*(x, A) denotes the probability that an action a \in \mathcal A is chosen. Under this approach, its loss function is also defined directly as: \int_{A \in \mathcal A}d^*(x, A) L(\theta, A) dA . The introduction of randomised decision rules thus creates a larger decision space from which the statistician may choose his decision.

As non-randomised decision rules are a special case of randomised decision rules where one decision or action has probability 1, the original decision space \mathcal D is a proper subset of the new decision space \mathcal D^* .

Admissibility

An admissible decision rule is one that is not dominated by any other decision rule, i.e. there is no decision rule that has equal risk as or lower risk than it for all parameters and strictly lower risk than it for some parameter. In a finite decision problem, the risk point of an admissible decision rule has either lower x-coordinates or y-coordinates than all other risk points or, more formally, it is the set of rules with risk points of the form (a, b) such that \{(R_1, R_2): R_1 \leq a, R_2 \leq b \} \cap \mathcal S = (a, b) . Thus the left side of the lower boundary of the risk set is the set of admissible decision rules.

In a finite decision problem with two possible parameters, the minimax rule can be found by considering the family of squares Q(c) = \{(R_1, R_2): 0 \leq R_1 \leq c, 0 \leq R_2 \leq c \} . The value of c for the smallest of such squares that touches \mathcal S is the minimax risk and the corresponding point or points on the risk set is the minimax rule.

If the risk set intersects the line R_1 = R_2 , then the admissible decision rule lying on the line is minimax. If R_2 > R_1 or R_1 > R_2 holds for every point in the risk set, then the minimax rule can either be an extreme point (i.e. a nonrandomised decision rule) or a line connecting two extreme points (nonrandomised decision rules).

A randomised Bayes rule is one that has infimum Bayes risk r(\pi, d^*) among all decision rules. In the special case where the parameter space has two elements, the line \pi_1 R_1 +(1 - \pi_1) R_2 = c , where \pi_1 and \pi_2 denote the prior probabilities of \theta_1 and \theta_2 respectively, is a family of points with Bayes risk c . The minimum Bayes risk for the decision problem is therefore the smallest c such that the line touches the risk set.

This line may either touch only one extreme point of the risk set, i.e. correspond to a nonrandomised decision rule, or overlap with an entire side of the risk set, i.e. correspond to two nonrandomised decision rules and randomised decision rules combining the two. This is illustrated by the three situations below: As different priors result in different slopes, the set of all rules that are Bayes with respect to some prior are the same as the set of admissible rules.

Note that no situation is possible where a nonrandomised Bayes rule does not exist but a randomised Bayes rule does. The existence of a randomised Bayes rule implies the existence of a nonrandomised Bayes rule. This is also true in the general case, even with infinite parameter space, infinite Bayes risk, and regardless of whether the infimum Bayes risk can be attained.

This supports the intuitive notion that the statistician need not utilise randomisation to arrive at statistical decisions. In the usual formulation of the likelihood ratio test, the null hypothesis is rejected whenever the likelihood ratio \Lambda is smaller than some constant K , and accepted otherwise.

However, this is sometimes problematic when \Lambda is discrete under the null hypothesis, when \Lambda = K is possible. A solution is to define a test function \phi(x) , whose value is the probability at which the null hypothesis is accepted: \phi(x) = \left\{ \begin{array}{l} 1 & \text{ if } \Lambda > K \\ p(x) & \text{ if } \Lambda = K \\ 0 & \text{ if } \Lambda This can be interpreted as flipping a biased coin with a probability p(x) of returning heads whenever \Lambda = k and rejecting the null hypothesis if a heads turns up.

A generalised form of the Neyman–Pearson lemma states that this test has maximum power among all tests at the same significance level \alpha , that such a test must exist for any significance level \alpha , and that the test is unique under normal situations. As an example, consider the case where the underlying distribution is Bernoulli with probability p , and we would like to test the null hypothesis p \leq \lambda against the alternative hypothesis p > \lambda .

It is natural to choose some k such that P(\hat{p} > k|H_0) = \alpha , and reject the null whenever \hat{p} > k , where \hat{p} is the test statistic. However, to take into account cases where \hat p = k , we define the test function: \phi(x) = \left\{ \begin{array}{l} 1 & \text{ if } \hat{p} > k\\ \gamma & \text{ if } \hat{p} = k \\ 0 & \text{ if } \hat{p} where \gamma is chosen such that P(\hat{p} > k|H_0) + \gamma P(\hat{p} = k|H_0) = \alpha . \left\{ \begin{array}{l} Pr (\hat{p} k|p = L) + \gamma P(\hat{p} = k|p = L) &= \alpha/2 \end{array} \right. where \gamma is a uniform random variable on (0, 1).

Text derived from https://en.wikipedia.org/w/index.php?oldid=1368420120 under CC BY-SA 4.0; modifications have been made.

Provenance

snapshot enwiki-001368420120 · captured 1786224527.0 · version history

Cite this page as: encyclopegia.org, “Randomised decision rule”, snapshot enwiki-001368420120. https://encyclopegia.org/a/randomised-decision-rule

Sources cited in this revision

1 named source(s) extracted from the cited revision. A source with no author is shown by its publication; nothing is invented.