GGUF

Uses your system voice; voice and rate in settings.

Snapshot

≈ 3 min read

Names in other languages

No names in other languages are held for this entry yet.

Article bodies are English and are not machine-translated. A name in another language is not a translation of the article.

The GGUF ( Universal File) file format is a binary file format that stores both tensors and metadata in a single file and is designed for fast saving and loading of model data. It was introduced in August 2023 by the llama.cpp project to better maintain backward compatibility as support was added for other model architectures.

It superseded previous formats used by the project such as GGML and is typically produced by converting models developed with a different machine-learning library such as PyTorch. GGUF has become the standard format for distributing quantized large language models for local inference and is natively supported by tools such as llama.cpp, Ollama, LM Studio, GPT4All, Jan, and koboldcpp.

As of 2026, tens of thousands of GGUF checkpoints are hosted on Hugging Face, which provides first-class integration, including a metadata viewer, an inference endpoint service, and a JavaScript parser library.

The GGUF ( Universal File) file format is a binary file format that stores both tensors and metadata in a single file and is designed for fast saving and loading of model data. It was introduced in August 2023 by the llama.cpp project to better maintain backward compatibility as support was added for other model architectures.

It superseded previous formats used by the project such as GGML and is typically produced by converting models developed with a different machine-learning library such as PyTorch. GGUF has become the standard format for distributing quantized large language models for local inference and is natively supported by tools such as llama.cpp, Ollama, LM Studio, GPT4All, Jan, and koboldcpp.

As of 2026, tens of thousands of GGUF checkpoints are hosted on Hugging Face, which provides first-class integration, including a metadata viewer, an inference endpoint service, and a JavaScript parser library. // example metadata general.architecture: 'llama', general.name: 'LLaMA v2', llama.context_length: 4096, ... , general.file_type: 10, // (typically indicates quantization level, here "MOSTLY_Q2_K") tokenizer.ggml.model: 'llama', tokenizer.ggml.tokens: [ ' ', ' ', ' ', ' ', ' ', ' ', ' ', ' ', ' ', ' ', ' ', ' ', ... ], ... // n-th tensor name: GGUF string, // ex: "blk.0.ffn_gate.weight" n_dimensions: UINT32, // ex: 2 dimensions: UINT64[], // ex: [ 4096, 32000 ] type: UINT32, // ex: 10 (typically indicates quantization level, here "GGML_TYPE_Q2_K") offset: UINT64 // starting position within the tensor_data block, relative to the start of the block // (n+1)-th tensor ...

Text derived from https://en.wikipedia.org/w/index.php?oldid=1370527197 under CC BY-SA 4.0; modifications have been made.

Sources cited in this revision

11 named source(s) extracted from the cited revision. A source with no author is shown by its publication; nothing is invented.