Radial basis function kernel

In machine learning, the radial basis function kernel, or RBF kernel, is a popular kernel function used in various kernelized learning algorithms.

Uses your system voice; voice and rate in settings.

Snapshot

≈ 5 min read

Names in other languages

No names in other languages are held for this entry yet.

Article bodies are English and are not machine-translated. A name in another language is not a translation of the article.

In machine learning, the radial basis function kernel, or RBF kernel, is a popular kernel function used in various kernelized learning algorithms. In particular, it is commonly used in support vector machine classification.[1]

The RBF kernel on two samples 𝐱,𝐱′∈ℝk, represented as feature vectors in some input space, is defined as[2]

K(𝐱,𝐱′)=exp⁡(−‖𝐱−𝐱′‖22σ2)

‖𝐱−𝐱′‖2 may be recognized as the squared Euclidean distance between the two feature vectors. σ is a free parameter. An equivalent definition involves a parameter γ=12σ2:

K(𝐱,𝐱′)=exp⁡(−γ‖𝐱−𝐱′‖2)

Since the value of the RBF kernel decreases with distance and ranges between zero (in the infinite-distance limit) and one (when x = x'), it has a ready interpretation as a similarity measure.[2] The feature space of the kernel has an infinite number of dimensions; for σ=1, its expansion using the multinomial theorem is:[3]

exp⁡(−12‖𝐱−𝐱′‖2)=exp⁡(22𝐱⊤𝐱′−12‖𝐱‖2−12‖𝐱′‖2)=exp⁡(𝐱⊤𝐱′)exp⁡(−12‖𝐱‖2)exp⁡(−12‖𝐱′‖2)=∑j=0∞(𝐱⊤𝐱′)jj!exp⁡(−12‖𝐱‖2)exp⁡(−12‖𝐱′‖2)=∑j=0∞∑n1+n2+⋯+nk=jexp⁡(−12‖𝐱‖2)x1n1⋯xknkn1!⋯nk!exp⁡(−12‖𝐱′‖2)x′1n1⋯x′knkn1!⋯nk!=⟨φ(𝐱),φ(𝐱′)⟩

φ(𝐱)=exp⁡(−12‖𝐱‖2)(aℓ0(0),a1(1),…,aℓ1(1),…,a1(j),…,aℓj(j),…) where ℓj=(k+j−1j), aℓ(j)=x1n1⋯xknkn1!⋯nk!|n1+n2+⋯+nk=j∧1≤ℓ≤ℓj

Contents

Approximations

edit edit source

Because support vector machines and other models employing the kernel trick do not scale well to large numbers of training samples or large numbers of features in the input space, several approximations to the RBF kernel (and similar kernels) have been introduced.[4] Typically, these take the form of a function z that maps a single vector to a vector of higher dimensionality, approximating the kernel:

⟨z(𝐱),z(𝐱′)⟩≈⟨φ(𝐱),φ(𝐱′)⟩=K(𝐱,𝐱′)

where φ is the implicit mapping embedded in the RBF kernel.

Fourier random features

edit edit source

One way to construct such a z is to randomly sample from the Fourier transformation of the kernel[5]φ(x)=1D[cos⁡⟨w1,x⟩,sin⁡⟨w1,x⟩,…,cos⁡⟨wD,x⟩,sin⁡⟨wD,x⟩]Twhere w1,...,wD are independent samples from the normal distribution N(0,σ−2I).

Theorem: E⁡[⟨φ(x),φ(y)⟩]=e‖x−y‖2/(2σ2).

Proof: It suffices to prove the case of D=1. Use the trigonometric identity cos⁡(a−b)=cos⁡(a)cos⁡(b)+sin⁡(a)sin⁡(b), the spherical symmetry of Gaussian distribution, then evaluate the integral

∫−∞∞cos⁡(kx)e−x2/22πdx=e−k2/2.

Theorem: Var⁡[⟨φ(x),φ(y)⟩]=O(D−1). (Appendix A.2[6]).

Nyström method

edit edit source

Another approach uses the Nyström method to approximate the eigendecomposition of the Gram matrix K, using only a random sample of the training set.[7]

See also

edit edit source

References

edit edit source
  1. ^ Chang, Yin-Wen; Hsieh, Cho-Jui; Chang, Kai-Wei; Ringgaard, Michael; Lin, Chih-Jen (2010). "Training and testing low-degree polynomial data mappings via linear SVM". Journal of Machine Learning Research. 11: 1471–1490. Wikipedia revision 1347040306, retrieved 2026-11-19
  2. ^ a b Jean-Philippe Vert, Koji Tsuda, and Bernhard Schölkopf (2004). "A primer on kernel methods". Kernel Methods in Computational Biology. Wikipedia revision 1347040306, retrieved 2026-11-19
  3. ^ Shashua, Amnon (2009). "Introduction to Machine Learning: Class Notes 67577". arXiv:0904.3664v1 [cs.LG]. Wikipedia revision 1347040306, retrieved 2026-11-19
  4. ^ Andreas Müller (2012). Kernel Approximations for Efficient SVMs (and other feature extraction methods). Wikipedia revision 1347040306, retrieved 2026-11-19
  5. ^ Rahimi, Ali; Recht, Benjamin (2007). "Random Features for Large-Scale Kernel Machines". Advances in Neural Information Processing Systems. 20. Curran Associates, Inc. Wikipedia revision 1347040306, retrieved 2026-11-19
  6. ^ Peng, Hao; Pappas, Nikolaos; Yogatama, Dani; Schwartz, Roy; Smith, Noah A.; Kong, Lingpeng (2021-03-19). "Random Feature Attention". arXiv:2103.02143 [cs.CL]. Wikipedia revision 1347040306, retrieved 2026-11-19
  7. ^ C.K.I. Williams; M. Seeger (2001). "Using the Nyström method to speed up kernel machines". Advances in Neural Information Processing Systems. 13. Wikipedia revision 1347040306, retrieved 2026-11-19

Text derived from https://en.wikipedia.org/w/index.php?oldid=1347040306 under CC BY-SA 4.0; modifications have been made.

Provenance

snapshot enwiki-001347040306 · captured 2026-04-04 · not yet sealed in the ledger · verify the chain · version history

Cite this page as: encyclopegia.org, “Radial basis function kernel”, snapshot enwiki-001347040306. https://encyclopegia.org/a/radial-basis-function-kernel