Lesson plan
Small language model
Small language models (SLM) or compact language models are artificial intelligence language models designed for human natural language processing including language and text generation. Small language models typically have less than forty billion parameters. This make them feasible to train or host entirely on consumer electronics such as personal computers, laptops, or smart devices. In contrast, a large language model typically contains hundreds of billions of training parameters, with some models exceeding a trillion parameters. This enables the model to encode vast amounts of information, improving the generalizability and accuracy of its outputs. However, training and hosting such models demands enormous computational resources, rendering it infeasible for an individual to do so using a single computer and graphics processing unit. Most contemporary small language models use the same architecture as a large language model, but with a smaller parameter count and sometimes lower arithmetic precision. Similar to LLMs, SLMs are optimized via knowledge distillation, pruning, and quantization.
Article summary
Small language models (SLM) or compact language models are artificial intelligence language models designed for human natural language processing including language and text generation. Small language models typically have less than forty billion parameters. This make them feasible to train or host entirely on consumer electronics such as personal computers, laptops, or smart devices. In contrast, a large language model typically contains hundreds of billions of training parameters, with some models exceeding a trillion parameters. This enables the model to encode vast amounts of information, improving the generalizability and accuracy of its outputs. However, training and hosting such models demands enormous computational resources, rendering it infeasible for an individual to do so using a single computer and graphics processing unit. Most contemporary small language models use the same architecture as a large language model, but with a smaller parameter count and sometimes lower arithmetic precision. Similar to LLMs, SLMs are optimized via knowledge distillation, pruning, and quantization.
Key claims
No claims recorded.
Discussion questions
- Summarise the article in your own words. Which single claim would you verify first, and why?
- Which claim in this article is most likely to be revised as the sources update? How would you check its current status?
- If two sources disagreed about one of these facts, what would you need in order to trust one over the other?
Bibliography
No sources recorded.
Key claims and bibliography are generated from the same tracked claims and sources as the article page.