Differentiable neural computer
and a 1 bit interrupt signal.
≈ 9 min read
Names in other languages
No names in other languages are held for this entry yet.
Article bodies are English and are not machine-translated. A name in another language is not a translation of the article.
In artificial intelligence, a differentiable neural computer (DNC) is a memory augmented neural network architecture (MANN), which is typically (but not by definition) recurrent in its implementation. The model was published in 2016 by Alex Graves et al. of DeepMind.[1]
Contents
Applications
edit edit sourceDNC indirectly takes inspiration from Von-Neumann architecture, making it likely to outperform conventional architectures in tasks that are fundamentally algorithmic that cannot be learned by finding a decision boundary.
So far, DNCs have been demonstrated to handle only relatively simple tasks, which can be solved using conventional programming. But DNCs don't need to be programmed for each problem, but can instead be trained. This attention span allows the user to feed complex data structures such as graphs sequentially, and recall them for later use. Furthermore, they can learn aspects of symbolic reasoning and apply it to working memory. The researchers who published the method see promise that DNCs can be trained to perform complex, structured tasks[1][2] and address big-data applications that require some sort of reasoning, such as generating video commentaries or semantic text analysis.[3][4]
DNC can be trained to navigate rapid transit systems, and apply that network to a different system. A neural network without memory would typically have to learn about each transit system from scratch. On graph traversal and sequence-processing tasks with supervised learning, DNCs performed better than alternatives such as long short-term memory or a neural turing machine.[5] With a reinforcement learning approach to a block puzzle problem inspired by SHRDLU, DNC was trained via curriculum learning, and learned to make a plan. It performed better than a traditional recurrent neural network.[5]
Architecture
edit edit sourceDNC networks were introduced as an extension of the Neural Turing Machine (NTM), with the addition of memory attention mechanisms that control where the memory is stored, and temporal attention that records the order of events. This structure allows DNCs to be more robust and abstract than a NTM, and still perform tasks that have longer-term dependencies than some predecessors such as Long Short Term Memory (LSTM). The memory, which is simply a matrix, can be allocated dynamically and accessed indefinitely. The DNC is differentiable end-to-end (each subcomponent of the model is differentiable, therefore so is the whole model). This makes it possible to optimize them efficiently using gradient descent.[3][6][7]
The DNC model is similar to the Von Neumann architecture, and because of the resizability of memory, it is Turing complete.[8]
Traditional DNC
edit edit sourceDNC, as originally published[1]
| Independent variables | |
| 𝐱t | Input vector |
| 𝐳t | Target vector |
| Controller | |
| 𝝌t=[𝐱t;𝐫t−11;⋯;𝐫t−1R] | Controller input matrix |
| Deep (layered) LSTM | ∀0≤l≤L |
| 𝐢tl=σ(Wil[𝝌t;𝐡t−1l;𝐡tl−1]+𝐛il) | Input gate vector |
| 𝐨tl=σ(Wol[𝝌t;𝐡t−1l;𝐡tl−1]+𝐛ol) | Output gate vector |
| 𝐟tl=σ(Wfl[𝝌t;𝐡t−1l;𝐡tl−1]+𝐛fl) | Forget gate vector |
| 𝐬tl=𝐟tl𝐬t−1l+𝐢tltanh(Wsl[𝝌t;𝐡t−1l;𝐡tl−1]+𝐛sl) | State gate vector, s0=0 |
| 𝐡tl=𝐨tltanh(𝐬tl) | Hidden gate vector, h0=0;ht0=0∀t |
| 𝐲t=Wy[𝐡t1;⋯;𝐡tL]+Wr[𝐫t1;⋯;𝐫tR] | DNC output vector |
| Read & Write heads | |
| ξt=Wξ[ht1;⋯;htL] | Interface parameters |
| =[𝐤tr,1;⋯;𝐤tr,R;β̂tr,1;⋯;β̂tr,R;𝐤tw;βtŵ;𝐞̂t;𝐯t;ft1̂;⋯;ftR̂;ĝta;ĝtw;𝝅̂t1;⋯;𝝅̂tR] | |
| Read heads | ∀1≤i≤R |
| 𝐤tr,i | Read keys |
| βtr,i=oneplus(β̂tr,i) | Read strengths |
| fti=σ(f̂ti) | Free gates |
| 𝝅ti=softmax(𝝅̂ti) | Read modes, 𝝅ti∈ℝ3 |
| Write head | |
| 𝐤tw | Write key |
| βtw=β̂tw | Write strength |
| 𝐞t=σ(𝐞̂t) | Erase vector |
| 𝐯t | Write vector |
| gta=σ(ĝta) | Allocation gate |
| gtw=σ(ĝtw) | Write gate |
| Memory | |
| Mt=Mt−1∘(E−𝐰tw𝐞t⊺)+𝐰tw𝐯t⊺ | Memory matrix, Matrix of ones E∈ℝN×W |
| 𝐮t=(𝐮t−1+𝐰t−1w−𝐮t−1∘𝐰t−1w)∘𝝍t | Usage vector |
| 𝐩t=(1−∑i𝐰tw[i])𝐩t−1+𝐰tw | Precedence weighting, 𝐩0=𝟎 |
| Lt=(𝟏−𝐈)[(1−𝐰tw[i]−𝐰tw[j])Lt−1[i,j]+𝐰tw[i]𝐩t−1j] | Temporal link matrix, L0=𝟎 |
| 𝐰tw=gtw[gta𝐚t+(1−gta)𝐜tw] | Write weighting |
| 𝐰tr,i=𝝅ti[1]𝐛ti+𝝅ti[2]ctr,i+𝝅ti[3]fti | Read weighting |
| 𝐫ti=Mt⊺𝐰tr,i | Read vectors |
| 𝒞(M,𝐤,β)[i]=exp{𝒟(𝐤,M[i,⋅])β}∑jexp{𝒟(𝐤,M[j,⋅])β} | Content-based addressing, Lookup key 𝐤, key strength β |
| ϕt | Indices of 𝐮t, sorted in ascending order of usage |
| 𝐚t[ϕt[j]]=(1−𝐮t[ϕt[j]])∏i=1j−1𝐮t[ϕt[i]] | Allocation weighting |
| 𝐜tw=𝒞(Mt−1,𝐤tw,βtw) | Write content weighting |
| 𝐜tr,i=𝒞(Mt−1,𝐤tr,i,βtr,i) | Read content weighting |
| 𝐟ti=Lt𝐰t−1r,i | Forward weighting |
| 𝐛ti=Lt⊺𝐰t−1r,i | Backward weighting |
| 𝝍t=∏i=1R(𝟏−fti𝐰t−1r,i) | Memory retention vector |
| Definitions | |
| 𝐖,𝐛 | Weight matrix, bias vector |
| 𝟎,𝟏,𝐈 | Zeros matrix, ones matrix, identity matrix |
| ∘ | Element-wise multiplication |
| 𝒟(𝐮,𝐯)=𝐮⋅𝐯‖𝐮‖‖𝐯‖ | Cosine similarity |
| σ(x)=1/(1+e−x) | Sigmoid function |
| oneplus(x)=1+log(1+ex) | Oneplus function |
| softmax(𝐱)j=exj∑k=1Kexk for j = 1, ..., K. | Softmax function |
Extensions
edit edit sourceRefinements include sparse memory addressing, which reduces time and space complexity by thousands of times. This can be achieved by using an approximate nearest neighbor algorithm, such as Locality-sensitive hashing, or a random k-d tree like Fast Library for Approximate Nearest Neighbors from UBC.[9] Adding Adaptive Computation Time (ACT) separates computation time from data time, which uses the fact that problem length and problem difficulty are not always the same.[10] Training using synthetic gradients performs considerably better than Backpropagation through time (BPTT).[11] Robustness can be improved with use of layer normalization and Bypass Dropout as regularization.[12]
See also
edit edit sourceReferences
edit edit source- ^ a b c Graves, Alex; Wayne, Greg; Reynolds, Malcolm; Harley, Tim; Danihelka, Ivo; Grabska-Barwińska, Agnieszka; Colmenarejo, Sergio Gómez; Grefenstette, Edward; Ramalho, Tiago (2016-10-12). "Hybrid computing using a neural network with dynamic external memory". Nature. 538 (7626): 471–476. Bibcode:2016Natur.538..471G. doi:10.1038/nature20101. ISSN 1476-4687. PMID 27732574. S2CID 205251479.
- ^ "Differentiable neural computers | DeepMind". DeepMind. 12 October 2016. Retrieved 2016-10-19.
- ^ a b Burgess, Matt. "DeepMind's AI learned to ride the London Underground using human-like reason and memory". WIRED UK. Retrieved 2016-10-19.
- ^ Jaeger, Herbert (2016-10-12). "Artificial intelligence: Deep neural reasoning". Nature. 538 (7626): 467–468. Bibcode:2016Natur.538..467J. doi:10.1038/nature19477. ISSN 1476-4687. PMID 27732576.
- ^ a b James, Mike. "DeepMind's Differentiable Neural Network Thinks Deeply". www.i-programmer.info. Retrieved 2016-10-20.
- ^ "DeepMind AI 'Learns' to Navigate London Tube". PCMAG. Retrieved 2016-10-19.
- ^ Mannes, John (13 October 2016). "DeepMind's differentiable neural computer helps you navigate the subway with its memory". TechCrunch. Retrieved 2016-10-19.
- ^ "RNN Symposium 2016: Alex Graves - Differentiable Neural Computer". YouTube. 22 March 2017.
- ^ Jack W Rae; Jonathan J Hunt; Harley, Tim; Danihelka, Ivo; Senior, Andrew; Wayne, Greg; Graves, Alex; Timothy P Lillicrap (2016). "Scaling Memory-Augmented Neural Networks with Sparse Reads and Writes". arXiv:1610.09027 [cs.LG].
- ^ Graves, Alex (2016). "Adaptive Computation Time for Recurrent Neural Networks". arXiv:1603.08983 [cs.NE].
- ^ Jaderberg, Max; Wojciech Marian Czarnecki; Osindero, Simon; Vinyals, Oriol; Graves, Alex; Silver, David; Kavukcuoglu, Koray (2016). "Decoupled Neural Interfaces using Synthetic Gradients". arXiv:1608.05343 [cs.LG].
- ^ Franke, Jörg; Niehues, Jan; Waibel, Alex (2018). "Robust and Scalable Differentiable Neural Computer for Question Answering". arXiv:1807.02658 [cs.CL].
External links
edit edit source- A bit-by-bit guide to the equations governing differentiable neural computers
- DeepMind's Differentiable Neural Network Thinks Deeply
- Automated reasoning
- Automated planning
- Constraint satisfaction
- Knowledge representation
- Parameter
- Loss functions
- Regression
- Clustering
- Gradient descent
- Backpropagation
- Attention
- Convolution
- Normalization
- Activation
- Gating
- Weight initialization
- Regularization
- Datasets
- Prompt engineering
- Reinforcement learning
- Diffusion
- Latent diffusion model
- Autoregression
- Adversary
- RAG
- Uncanny valley
- LLM post-training
- RLHF
- Self-supervised learning
- Reflection
- Recursive self-improvement
- Hallucination
- Word embedding
- Vibe coding
- Blended AI
- Open-source AI
- Sovereign AI
- Symbolic AI
- Neuro-symbolic AI
- Situated approach
- Actor-critic algorithm
- Alan Turing
- Warren Sturgis McCulloch
- Walter Pitts
- John von Neumann
- Christopher D. Manning
- Claude Shannon
- Shun'ichi Amari
- Kunihiko Fukushima
- Takeo Kanade
- Marvin Minsky
- John McCarthy
- Nathaniel Rochester
- Allen Newell
- Cliff Shaw
- Herbert A. Simon
- Oliver Selfridge
- Frank Rosenblatt
- Bernard Widrow
- Joseph Weizenbaum
- Seymour Papert
- Seppo Linnainmaa
- Paul Werbos
- Geoffrey Hinton
- John Hopfield
- Jürgen Schmidhuber
- Yann LeCun
- Yoshua Bengio
- Lotfi A. Zadeh
- Stephen Grossberg
- Alex Graves
- James Goodnight
- Andrew Ng
- Fei-Fei Li
- Alex Krizhevsky
- Ilya Sutskever
- Oriol Vinyals
- Quoc V. Le
- Ian Goodfellow
- Demis Hassabis
- David Silver
- Andrej Karpathy
- Ashish Vaswani
- Noam Shazeer
- Aidan Gomez
- John Schulman
- Mustafa Suleyman
- Jan Leike
- Daniel Kokotajlo
- François Chollet
architectures
- Neural Turing machine
- Differentiable neural computer
- Transformer
- Recurrent neural network (RNN)
- Long short-term memory (LSTM)
- Gated recurrent unit (GRU)
- Echo state network
- Multilayer perceptron (MLP)
- Convolutional neural network (CNN)
- Residual neural network (RNN)
- Highway network
- Mamba
- Autoencoder
- Variational autoencoder (VAE)
- Generative adversarial network (GAN)
- Graph neural network (GNN)
and economic
- AI boom
- AI bubble
- AI data center
- AI effect
- AI infrastructure
- AI literacy
- AI slop
- AI winter
- Anthropomorphism
- Arms race
- Competition
- Environmental impact
- Explainable AI
- Generative engine optimization
- In architecture
- In education
- In fiction
- In healthcare
- In marketing
- In video games
- In visual art
- Military applications
- Workplace impact
Text derived from https://en.wikipedia.org/w/index.php?oldid=1303916533 under CC BY-SA 4.0; modifications have been made.
References
- ↩Wikipedia “Differentiable neural computer” publication date (×1) · confidence 0.95 revision URL