DBRX

Uses your system voice; voice and rate in settings.

Snapshot

≈ 2 min read

Names in other languages

No names in other languages are held for this entry yet.

Article bodies are English and are not machine-translated. A name in another language is not a translation of the article.

DBRX is a large language model (LLM) developed by Mosaic under its parent company Databricks, released on March 27, 2024 under the Databricks Open Model License. It is a mixture-of-experts transformer model, with 132 billion parameters in total. 36 billion parameters (4 out of 16 experts) are active for each token. The released model comes in either a base foundation model version or an instruction-tuned variant. At the time of its release, DBRX outperformed prominent models such as Meta's Llama 2, Mistral AI's Mixtral, and xAI's Grok-1, in several benchmarks ranging from language understanding, programming ability and mathematics. It was trained for 2.5 months and reported using on 3,072 Nvidia H100s connected by 3.2 terabytes per second bandwidth (InfiniBand), for a training cost of US$10M.

DBRX is a large language model (LLM) developed by Mosaic under its parent company Databricks, released on March 27, 2024 under the Databricks Open Model License. It is a mixture-of-experts transformer model, with 132 billion parameters in total. 36 billion parameters (4 out of 16 experts) are active for each token. The released model comes in either a base foundation model version or an instruction-tuned variant. At the time of its release, DBRX outperformed prominent models such as Meta's Llama 2, Mistral AI's Mixtral, and xAI's Grok-1, in several benchmarks ranging from language understanding, programming ability and mathematics. It was trained for 2.5 months and reported using on 3,072 Nvidia H100s connected by 3.2 terabytes per second bandwidth (InfiniBand), for a training cost of US$10M.

Text derived from https://en.wikipedia.org/w/index.php?oldid=1359042008 under CC BY-SA 4.0; modifications have been made.

References

1 claim could not be tied to a specific sentence and is listed in the table below instead.