Hilbert projection theorem

In mathematics, the Hilbert projection theorem is a famous result of convex analysis that says that for every vector x in a Hilbert space H and every nonempty closed convex C \subseteq H, there exists a unique vector m \in C for which \|c…

Uses your system voice; voice and rate in settings.

Snapshot

≈ 6 min read

Names in other languages

No names in other languages are held for this entry yet.

Article bodies are English and are not machine-translated. A name in another language is not a translation of the article.

In mathematics, the Hilbert projection theorem is a famous result of convex analysis that says that for every vector x in a Hilbert space H and every nonempty closed convex C \subseteq H, there exists a unique vector m \in C for which \|c - x\| is minimized over the vectors c \in C ; that is, such that \|m - x\| \leq \|c - x\| for every c \in C. Consider a finite dimensional real Hilbert space H with a subspace C and a point x. If m \in C is a or of the function N : C \to \R defined by N(c) := \|c - x\| (which is the same as the minimum point of c \mapsto \|c - x\|^2 ), then derivative must be zero at m. In matrix derivative notation: \begin{aligned} \partial \lVert x - c \rVert^2 &= \partial \langle c - x, c - x \rangle \\ &= 2 \langle c - x, \partial c\rangle \end{aligned} Since \partial c is a vector in C that represents an arbitrary tangent direction, it follows that m - x must be orthogonal to every vector in C. It suffices to prove the theorem in the case of x = 0 because the general case follows from the statement below by replacing C with C - x. 0 = \langle \,c, \,c\, \rangle = \|c\|^2, which implies c = 0. \blacksquare : Let P := \prod_{c \in C} \mathbb{F} where \mathbb{F} is the underlying scalar field of H and define \begin{alignat}{4} L : \,& H && \to \,&& P \\ & h && \mapsto\,&& \left(\langle \,h, \,c\, \rangle\right)_{c \in C} \\ \end{alignat} which is continuous and linear because this is true of each of its coordinates h \mapsto \langle h, c \rangle. The set C^{\bot} = L^{-1}(0) = L^{-1}\left(\{ 0 \}\right) is closed in H because \{ 0 \} is closed in P and L : H \to P is continuous.

The kernel of any linear map is a vector subspace of its domain, which is why C^{\bot} = \ker L is a vector subspace of H. \blacksquare : Let x \in H. The Hilbert projection theorem guarantees the existence of a unique m \in C such that \|x - m\| \leq \|x - c\| \text{ for all } c \in C (or equivalently, for all x - c \in x - C ). Let p := x - m so that x = m + p \in C + p and it remains to show that p \in C^{\bot}. The inequality above can be rewritten as: \|p\| \leq \|z\| \quad \text{ for all } z \in x - C. Because m \in C and C is a vector space, m + C = C and C = - C, which implies that x - C = x + C = p + m + C = p + C. The previous inequality thus becomes \|p\| \leq \|z\| \quad \text{ for all } z \in p + C. or equivalently, \|p\| \leq \|p + c\| \quad \text{ for all } c \in C. But this last statement is true if and only if \langle \,p, c\, \rangle = 0 every c \in C. Thus p \in C^{\bot}. \blacksquare d_{C,x} : C \to [0, \infty) \quad \text{ by } c \mapsto \|x - c\|.

A of d_{C,x}, if one exists, is any point m in \,\operatorname{domain} d_{C,x} = C\, such that d_{C,x}(m) \,\leq\, d_{C,x}(c) \quad \text{ for all } c \in C, in which case d_{C,x}(m) = \|m - x\| is equal to the of the function d_{C, x}, which is: \inf_{c \in C} d_{C,x}(c) = \inf_{c \in C} \|x - c\|. Effects of translations and scalings When this global minimum point m exists and is unique then denote it by \min(C, x); explicitly, the defining properties of \min(C, x) (if it exists) are: \min(C, x) \in C \quad \text { and } \quad \left\|x - \min(C, x)\right\| \leq \|x - c\| \quad \text{ for all } c \in C. The Hilbert projection theorem guarantees that this unique minimum point exists whenever C is a non-empty closed and convex subset of a Hilbert space.

However, such a minimum point can also exist in non-convex or non-closed subsets as well; for instance, just as long is C is non-empty, if x \in C then \min(C, x) = x. If C \subseteq H is a non-empty subset, s is any scalar, and x, x_0 \in H are any vectors then \,\min\left(s C + x_0, s x + x_0\right) = s \min(C, x) + x_0 which implies: \begin{alignat}{6} \min&(s C, s x) &&= s &&\min(C, x) \\ \min&(- C, - x) &&= - &&\min(C, x) \\ \end{alignat} \begin{alignat}{6} \min\left(C + x_0, x + x_0\right) &= \min(C, x) + x_0 \\ \min\left(C - x_0, x - x_0\right) &= \min(C, x) - x_0 \\ \end{alignat} \begin{alignat}{6} \min&(C, - x) {} &&= \min(C + x, 0) - x \\ \min&(C, 0) \;+\; x\;\;\;\; &&= \min(C + x, x) \\ \min&(C - x, 0) {} &&= \min(C, x) - x \\ \end{alignat} Examples The following counter-example demonstrates a continuous linear isomorphism A : H \to H for which \,\min(A(C), A(x)) \neq A(\min(C, x)). Endow H := \R^2 with the dot product, let x_0 := (0, 1), and for every real s \in \R, let L_s := \{ (x, s x) : x \in \R \} be the line of slope s through the origin, where it is readily verified that \min\left(L_s, x_0\right) = \frac{s}{1+s^2}(1, s). Pick a real number r \neq 0 and define A : \R^2 \to \R^2 by A(x, y) := (r x, y) (so this map scales the x- coordinate by r while leaving the y- coordinate unchanged). Then A : \R^2 \to \R^2 is an invertible continuous linear operator that satisfies A\left(L_s\right) = L_{s/r} and A\left(x_0\right) = x_0, so that \,\min\left(A\left(L_s\right), A\left(x_0\right)\right) = \frac{s}{r^2 + s^2} (1, s) and A\left(\min\left(L_s, x_0\right)\right) = \frac{s}{1 + s^2} \left(r, s\right).

Consequently, if C := L_s with s \neq 0 and if (r, s) \neq (\pm 1, 1) then \,\min(A(C), A\left(x_0\right)) \neq A\left(\min\left(C, x_0\right)\right). If C is a closed subspace, then it is convex. In this case, the projection function P: H \to C is an orthogonal projection (a continuous linear operator that is self-adjoint).

A classic theorem states that, if C_1, \dots, C_n are closed subspaces, then \lim_{k \to \infty}\|(P_{C_1} \cdots P_{C_n})^kx - P_{C_1 \cap \dots \cap C_n} x\| = 0, \quad \forall x \in H

Text derived from https://en.wikipedia.org/w/index.php?oldid=1363412230 under CC BY-SA 4.0; modifications have been made.

Sources cited in this revision

3 named source(s) extracted from the cited revision. A source with no author is shown by its publication; nothing is invented.