---
title: Submodular set function
slug: submodular-set-function
qid: 
snapshot: enwiki-001366703148
content_hash: 
license: CC BY-SA 4.0
source: https://en.wikipedia.org/w/index.php?oldid=1366703148
tags: [22904282, 22951058, 23118434, 23603337, 24745122, 25888436, 29176491, 29264798, 68328988]
---

# Submodular set function

> [!note] Provenance
> Snapshot `enwiki-001366703148` · content root `…`
> Derived from <https://en.wikipedia.org/w/index.php?oldid=1366703148> · Licensed CC BY-SA 4.0

In mathematics, a submodular set function (also known as a submodular function) is a set function that, informally, describes the relationship between a set of inputs and an output, where adding more of one input has a decreasing additional benefit (diminishing returns). The natural diminishing returns property which makes them suitable for many applications, including approximation algorithms, game theory (as functions modeling user preferences) and electrical networks. Recently, submodular functions have also found utility in several real world problems in machine learning and artificial intelligence, including automatic summarization, multi-document summarization, feature selection, active learning, sensor placement, image collection summarization and many other domains.

In mathematics, a submodular set function (also known as a submodular function) is a set function that, informally, describes the relationship between a set of inputs and an output, where adding more of one input has a decreasing additional benefit (diminishing returns). The natural diminishing returns property which makes them suitable for many applications, including approximation algorithms, game theory (as functions modeling user preferences) and electrical networks. Recently, submodular functions have also found utility in several real world problems in machine learning and artificial intelligence, including automatic summarization, multi-document summarization, feature selection, active learning, sensor placement, image collection summarization and many other domains. f defined by f(S) = 1 if S is finite and f(S) = 0 if S is infinite satisfies the first condition above, but the second condition fails when S and T are infinite sets with finite intersection. Linear (Modular) functions : Any function of the form f(S)=\sum_{i\in S}w_i is called a linear function. Additionally if \forall i,w_i\geq 0 then f is monotone. Budget-additive functions : Any function of the form f(S)=\min\left\{B,~\sum_{i\in S}w_i\right\} for each w_i\geq 0 and B\geq 0 is called budget additive. Coverage functions : Let \Omega=\{E_1,E_2,\ldots,E_n\} be a collection of subsets of some ground set \Omega' . The function f(S)=\left|\bigcup_{E_i\in S}E_i\right| for S\subseteq \Omega is called a coverage function. This can be generalized by adding non-negative weights to the elements. Entropy : Let \Omega=\{X_1,X_2,\ldots,X_n\} be a set of random variables. Then for any S\subseteq \Omega we have that H(S) is a submodular function, where H(S) is the entropy of the set of random variables S , a fact known as Shannon's inequality. Further inequalities for the entropy function are known to hold, see entropic vector. Matroid rank functions : Let \Omega=\{e_1,e_2,\dots,e_n\} be the ground set on which a matroid is defined. Then the rank function of the matroid is a submodular function. A non-monotone submodular function f is called symmetric if for every S\subseteq \Omega we have that f(S)=f(\Omega-S) . Examples of symmetric non-monotone submodular functions include: Graph cuts : Let \Omega=\{v_1,v_2,\dots,v_n\} be the vertices of a graph. For any set of vertices S\subseteq \Omega let f(S) denote the number of edges e=(u,v) such that u\in S and v\in \Omega-S . This can be generalized by adding non-negative weights to the edges. Mutual information : Let \Omega=\{X_1,X_2,\ldots,X_n\} be a set of random variables. Then for any S\subseteq \Omega we have that f(S)=I(S;\Omega-S) is a submodular function, where I(S;\Omega-S) is the mutual information. Directed cuts : Let \Omega=\{v_1,v_2,\dots,v_n\} be the vertices of a directed graph. For any set of vertices S\subseteq \Omega let f(S) denote the number of edges e=(u,v) such that u\in S and v\in \Omega-S . This can be generalized by adding non-negative weights to the directed edges. Formally, a set function f:2^{\Omega}\rightarrow \mathbb{R} with |\Omega|=n can be represented as a function on \{0, 1\}^{n} , by associating each S\subseteq \Omega with a binary vector x^{S}\in \{0, 1\}^{n} such that x_{i}^{S}=1 when i\in S , and x_{i}^{S}=0 otherwise. A continuous extension of f is a continuous function F:[0, 1]^{n}\rightarrow \mathbb{R} , that matches the value of f on x\in \{0, 1\}^{n} , i.e. F(x^{S})=f(S) . Several kinds of continuous extensions of submodular functions are commonly used, which are described below. This extension is named after mathematician László Lovász. Consider any vector \mathbf{x}=\{x_1,x_2,\dots,x_n\} such that each 0\leq x_i\leq 1 . Then the Lovász extension is defined as f^L(\mathbf{x})=\mathbb{E}(f(\{i|x_i\geq \lambda\})) where the expectation is over \lambda chosen from the uniform distribution on the interval [0,1] . The Lovász extension is a convex function if and only if f is a submodular function. Consider any vector \mathbf{x}=\{x_1,x_2,\ldots,x_n\} such that each 0\leq x_i\leq 1 . Then the multilinear extension is defined as F(\mathbf{x})=\sum_{S\subseteq \Omega} f(S) \prod_{i\in S} x_i \prod_{i\notin S} (1-x_i) . Intuitively, x i represents the probability that item i is chosen for the set. For every set S, the two inner products represent the probability that the chosen set is exactly S. Therefore, the sum represents the expected value of f for the set formed by choosing each item i at random with probability xi, independently of the other items. Consider any vector \mathbf{x}=\{x_1,x_2,\dots,x_n\} such that each 0\leq x_i\leq 1 . Then the convex closure is defined as f^-(\mathbf{x})=\min\left(\sum_S \alpha_S f(S):\sum_S \alpha_S 1_S=\mathbf{x},\sum_S \alpha_S=1,\alpha_S\geq 0\right) . The convex closure of any set function is convex over [0,1]^n . Consider any vector \mathbf{x}=\{x_1,x_2,\dots,x_n\} such that each 0\leq x_i\leq 1 . Then the concave closure is defined as f^+(\mathbf{x})=\max\left(\sum_S \alpha_S f(S):\sum_S \alpha_S 1_S=\mathbf{x},\sum_S \alpha_S=1,\alpha_S\geq 0\right) . For the extensions discussed above, it can be shown that f^{+}(\mathbf{x}) \geq F(\mathbf{x}) \geq f^{-}(\mathbf{x})=f^L(\mathbf{x}) when f is submodular. The class of submodular functions is closed under non-negative linear combinations. Consider any submodular function f_1,f_2,\ldots,f_k and non-negative numbers \alpha_1,\alpha_2,\ldots,\alpha_k . Then the function g defined by g(S)=\sum_{i=1}^k \alpha_i f_i(S) is submodular. For any submodular function f , the function defined by g(S)=f(\Omega \setminus S) is submodular. The function g(S)=\min(f(S),c) , where c is a real number, is submodular whenever f is monotone submodular. More generally, g(S)=h(f(S)) is submodular, for any non decreasing concave function h . Consider a random process where a set T is chosen with each element in \Omega being included in T independently with probability p . Then the following inequality is true \mathbb{E}[f(T)]\geq p f(\Omega)+(1-p) f(\varnothing) where \varnothing is the empty set. More generally consider the following random process where a set S is constructed as follows. For each of 1\leq i\leq l, A_i\subseteq \Omega construct S_i by including each element in A_i independently into S_i with probability p_i . Furthermore let S=\cup_{i=1}^l S_i . Then the following inequality is true \mathbb{E}[f(S)]\geq \sum_{R\subseteq [l]} \Pi_{i\in R}p_i \Pi_{i\notin R}(1-p_i)f(\cup_{i\in R}A_i) . === Submodular set function minimization=== The hardness of minimizing a submodular set function depends on constraints imposed on the problem. The unconstrained problem of minimizing a submodular function is computable in polynomial time, and even in strongly-polynomial time. Computing the minimum cut in a graph is a special case of this minimization problem. The problem of minimizing a submodular function with a cardinality lower bound is NP-hard, with polynomial factor lower bounds on the approximation factor. === Submodular set function maximization=== Unlike the case of minimization, maximizing a generic submodular function is NP-hard even in the unconstrained setting. Thus, most of the works in this field are concerned with polynomial-time approximation algorithms, including greedy algorithms or local search algorithms. The problem of maximizing a non-negative submodular function admits a 1/2 approximation algorithm. Computing the maximum cut of a graph is a special case of this problem. The problem of maximizing a monotone submodular function subject to a cardinality constraint admits a 1 - 1/e approximation algorithm. The maximum coverage problem is a special case of this problem. The problem of maximizing a monotone submodular function subject to a matroid constraint (which subsumes the case above) also admits a 1 - 1/e approximation algorithm. Many of these algorithms can be unified within a semi-differential based framework of algorithms. ===Related optimization problems=== Apart from submodular minimization and maximization, there are several other natural optimization problems related to submodular functions. Minimizing the difference between two submodular functions is not only NP hard, but also inapproximable. Minimization/maximization of a submodular function subject to a submodular level set constraint (also known as submodular optimization subject to submodular cover or submodular knapsack constraint) admits bounded approximation guarantees. Partitioning data based on a submodular function to maximize the average welfare is known as the submodular welfare problem, which also admits bounded approximation guarantees (see welfare maximization). ==External links== http://www.cs.berkeley.edu/~stefje/references.html has a longer bibliography http://submodularity.org/ includes further material on the subject
