{
  "id": "universal-approximation-theorem-888026-4be1b6add989b284",
  "slug": "universal-approximation-theorem",
  "title": "universal approximation theorem",
  "qid": "Q7894110",
  "lead": "In the field of machine learning, the universal approximation theorems (UATs) state that neural networks with a certain structure can, in principle, approximate any continuous function to any desired degree of accuracy. These theorems provide a mathematical justification for using neural networks, assuring researchers that a sufficiently large or deep network can model the complex, non-linear relationships often found in real-world data. The best-known version of the theorem applies to feedforward networks with a single hidden layer. It states that if the layer's activation function is non-polynomial (which is true for common choices like the sigmoid function or ReLU), then the network can act as a \"universal approximator.\" Universality is achieved by increasing the number of neurons in the hidden layer, making the network \"wider.\" Other versions of the theorem show that universality can also be achieved by keeping the network's width fixed but increasing its number of layers, making it \"deeper.\" These are existence theorems. They guarantee that a network with the right structure exists, but they do not provide a method for finding the network's parameters (training it), nor do they specify exactly how large the network must be for a given function.…",
  "body": "In the field of machine learning, the universal approximation theorems (UATs) state that neural networks with a certain structure can, in principle, approximate any continuous function to any desired degree of accuracy. These theorems provide a mathematical justification for using neural networks, assuring researchers that a sufficiently large or deep network can model the complex, non-linear relationships often found in real-world data. The best-known version of the theorem applies to feedforward networks with a single hidden layer. It states that if the layer's activation function is non-polynomial (which is true for common choices like the sigmoid function or ReLU), then the network can act as a \"universal approximator.\" Universality is achieved by increasing the number of neurons in the hidden layer, making the network \"wider.\" Other versions of the theorem show that universality can also be achieved by keeping the network's width fixed but increasing its number of layers, making it \"deeper.\" These are existence theorems. They guarantee that a network with the right structure exists, but they do not provide a method for finding the network's parameters (training it), nor do they specify exactly how large the network must be for a given function. Finding a suitable network remains a practical challenge that is typically addressed with optimization algorithms like backpropagation. === Reservoir computing and quantum reservoir computing=== In reservoir computing a sparse recurrent neural network with fixed weights equipped of fading memory and echo state property is followed by a trainable output layer. Its universality has been demonstrated separately for what concerns networks of rate neurons and spiking neurons, respectively. In 2024, the framework has been generalized and extended to quantum reservoirs where the reservoir is based on qubits defined over Hilbert spaces. | name = Universal approximation theorem (Uniform non-affine activation, arbitrary depth, constrained width). | math_statement = Let \\mathcal{X} be a compact subset of \\mathbb{R}^d . Let \\sigma:\\mathbb{R} \\to \\mathbb{R} be any non-affine continuous function which is continuously differentiable at at least one point, with nonzero derivative at that point. Let \\mathcal{N}_{d,D:d+D+2}^\\sigma denote the space of feed-forward neural networks with d input neurons, D output neurons, and an arbitrary number of hidden layers each with d + D + 2 neurons, such that every hidden neuron has activation function \\sigma and every output neuron has the identity as its activation function, with input layer \\phi and output layer \\rho . Then given any \\varepsilon > 0 and any f \\in C(\\mathcal{X}, \\mathbb{R}^D) , there exists \\hat{f} \\in \\mathcal{N}_{d,D:d+D+2}^\\sigma such that \\sup_{x \\in \\mathcal{X}} \\left\\|\\hat{f}(x) - f(x)\\right\\| In other words, \\mathcal{N} is dense in C(\\mathcal{X}; \\mathbb{R}^D) with respect to the topology of uniform convergence. Quantitative refinement: The number of layers and the width of each layer required to approximate f to \\varepsilon precision known; moreover, the result hold true when \\mathcal{X} and \\mathbb{R}^D are replaced with any non-positively curved Riemannian manifold. }} Certain necessary conditions for the bounded width, arbitrary depth case have been established, but there is still a gap between the known sufficient and necessary conditions.",
  "categories": [
    "10639794",
    "108502701",
    "22921969",
    "23118434",
    "23603337",
    "23693535",
    "26709280",
    "29176512",
    "31347309",
    "43604620",
    "7692542",
    "99044476"
  ],
  "license": "CC BY-SA 4.0",
  "source_uri": "https://en.wikipedia.org/w/index.php?oldid=1375059547",
  "content_hash": "4be1b6add989b284b17ae1a7e17d30d8b1d1fcc350bea78c04b53275b5bbde76",
  "created_at": 1792311752.0812707,
  "updated_at": 1789482604.0,
  "claims": [],
  "contradictions": [],
  "confidence": 0.6,
  "csl": {
    "id": "universal-approximation-theorem-888026-4be1b6add989b284",
    "type": "webpage",
    "title": "universal approximation theorem",
    "author": [
      {
        "family": "Anticloud FZ LLE",
        "literal": "Anticloud FZ LLE"
      }
    ],
    "container-title": "encyclopegia.org",
    "publisher": "Anticloud FZ LLE",
    "publisher-place": "Dubai, United Arab Emirates",
    "URL": "https://encyclopegia.org/a/universal-approximation-theorem",
    "issued": {
      "date-parts": [
        [
          2026,
          10,
          18
        ]
      ]
    },
    "accessed": {
      "date-parts": [
        [
          2026,
          11,
          20
        ]
      ]
    },
    "license": "CC BY-SA 4.0",
    "DOI": "eo1:universal-approximation-theorem-888026-4be1b6add989b284",
    "isBasedOn": "https://en.wikipedia.org/w/index.php?oldid=1375059547",
    "note": "Immutable snapshot universal-approximation-theorem-888026-4be1b6add989b284, content root 4be1b6add989b284…"
  }
}