discriminative model
mathematical model used for classification or regression
≈ 4 min read
Names in other languages
- Русскийдискриминативная модель
- Italianomodello discriminativo
- Españolmodelo discriminadore
- Українськарозрізнювальна модель
- Englishdiscriminative model
- Françaismodèle discriminatif
- 日本語識別モデル
- 中文判别模型
Article bodies are English and are not machine-translated. A name in another language is not a translation of the article.
Discriminative models, also referred to as conditional models, are a class of models frequently used for classification. In machine learning, it typically models the conditional distribution P(Y∣X), or it learns a direct decision rule that maps inputs X to outputs Y. Discriminative models are commonly used for classification and regression, where the main goal is accurate prediction on new data.
They are typically used to solve binary classification problems, i.e. assign labels, such as pass/fail, win/lose, alive/dead or healthy/sick, to existing datapoints. Discriminative models are usually trained to separate classes or to minimize prediction error under a chosen loss function.
They are often contrasted with generative models, which aim to model how the data are generated and can be used to sample new data. Types of discriminative models include logistic regression (LR), conditional random fields (CRFs), decision trees among many others.
A discriminative model is a model of the conditional probability P(Y\mid X = x) of the target Y, given an observation x. It can be used to "discriminate" the value of the target variable Y, given an observation x. Classifiers computed without using a probability model are also referred to loosely as "discriminative". The distinction between these last two classes is not consistently made.
An alternative division defines these symmetrically as: a generative model is a model of the conditional probability of the observable X, given a target y, symbolically, P(X\mid Y = y) a discriminative model is a model of the conditional probability of the target Y, given an observation x, symbolically, P(Y\mid X = x) Regardless of precise definition, the terminology is constitutional because a generative model can be used to "generate" random instances (outcomes), either of an observation and target (x, y) , or of an observation x given a target value y, while a discriminative model or discriminative classifier (without a model) can be used to "discriminate" the value of the target variable Y, given an observation x. P(y|x) = \frac{p(x|y)p(y)}{\textstyle \sum_{i}p(x|i)p(i) \displaystyle}=\frac{p(x|y)p(y)}{p(x)} . The following approach is based on the assumption that it is given the training data-set D=\{(x_i;y_i)|i\leq N\in \mathbb{Z}\} , where y_i is the corresponding output for the input x_i . f(x;w)=\arg \max_y w^T \phi(x,y) According to Memisevic's interpretation, w^T \phi(x,y) , which is also c(x,y;w) , computes a score which measures the compatibility of the input x with the potential output y . Then the \arg \max determines the class with the highest score.
P(y|x;w)= \frac{1}{Z(x;w)} \exp(w^T\phi(x,y)) , with Z(x;w)= \textstyle \sum_{y} \displaystyle\exp(w^T\phi(x,y)) The equation above represents logistic regression. Notice that a major distinction between models is their way of introducing posterior probability. Posterior probability is inferred from the parametric model.
We then can maximize the parameter by following equation: L(w)=\textstyle \sum_{i} \displaystyle \log p(y^i|x^i;w) It could also be replaced by the log-loss equation below: l^{\log} (x^i, y^i,c(x^i;w)) = -\log p(y^i|x^i;w) = \log Z(x^i;w)-w^T\phi(x^i,y^i) Since the log-loss is differentiable, a gradient-based method can be used to optimize the model. A global optimum is guaranteed because the objective function is convex.
The gradient of log likelihood is represented by: \frac{\partial L(w)}{\partial w} = \textstyle \sum_{i} \displaystyle \phi(x^i,y^i) - E_{p(y|x^i;w)} \phi(x^i,y) where E_{p(y|x^i;w)} is the expectation of p(y|x^i;w) . The above method will provide efficient computation for the relative small number of classification.
Text derived from https://en.wikipedia.org/w/index.php?oldid=1371193918 under CC BY-SA 4.0; modifications have been made.