---
title: Llama.cpp
slug: llama-cpp
qid: 
snapshot: enwiki-001378655897
content_hash: 
license: CC BY-SA 4.0
source: https://en.wikipedia.org/w/index.php?oldid=1378655897
tags: [127661130, 22985424, 23118434, 23230146, 23603337, 24745122, 26849358, 27077925, 30752582, 49989138, 89061594]
---

# Llama.cpp

> [!note] Provenance
> Snapshot `enwiki-001378655897` · content root `…`
> Derived from <https://en.wikipedia.org/w/index.php?oldid=1378655897> · Licensed CC BY-SA 4.0

llama.cpp is an open-source software library that performs inference on various large language models such as Meta's Llama model. It is co-developed alongside the project, a general-purpose tensor library. Command-line tools are included with the library, alongside a server with a simple web interface. llama.cpp has been considered as the de facto standard as the core of almost all local inference tools, including Ollama and LM Studio.

llama.cpp is an open-source software library that performs inference on various large language models such as Meta's Llama model. It is co-developed alongside the project, a general-purpose tensor library. Command-line tools are included with the library, alongside a server with a simple web interface. llama.cpp has been considered as the de facto standard as the core of almost all local inference tools, including Ollama and LM&nbsp;Studio. // example metadata general.architecture: 'llama', general.name: 'LLaMA v2', llama.context_length: 4096, ... , general.file_type: 10, // (typically indicates quantization level, here "MOSTLY_Q2_K") tokenizer.ggml.model: 'llama', tokenizer.ggml.tokens: [ ' ', ' ', ' ', ' ', ' ', ' ', ' ', ' ', ' ', ' ', ' ', ' ', ... ], ... // n-th tensor name: GGUF string, // ex: "blk.0.ffn_gate.weight" n_dimensions: UINT32, // ex: 2 dimensions: UINT64[], // ex: [ 4096, 32000 ] type: UINT32, // ex: 10 (typically indicates quantization level, here "GGML_TYPE_Q2_K") offset: UINT64 // starting position within the tensor_data block, relative to the start of the block // (n+1)-th tensor ...
