Llama.cpp ========= llama.cpp is an open-source software library that performs inference on various large language models such as Meta's Llama model. It is co-developed alongside the project, a general-purpose tensor library. Command-line tools are included with the library, alongside a server with a simple web interface. llama.cpp has been considered as the de facto standard as the core of almost all local inference tools, including Ollama and LM Studio. llama.cpp is an open-source software library that performs inference on various large language models such as Meta's Llama model. It is co-developed alongside the project, a general-purpose tensor library. Command-line tools are included with the library, alongside a server with a simple web interface. llama.cpp has been considered as the de facto standard as the core of almost all local inference tools, including Ollama and LM Studio. // example metadata general.architecture: 'llama', general.name: 'LLaMA v2', llama.context_length: 4096, ... , general.file_type: 10, // (typically indicates quantization level, here "MOSTLY_Q2_K") tokenizer.ggml.model: 'llama', tokenizer.ggml.tokens: [ ' ', ' ', ' ', ' ', ' ', ' ', ' ', ' ', ' ', ' ', ' ', ' ', ... ], ... // n-th tensor name: GGUF string, // ex: "blk.0.ffn_gate.weight" n_dimensions: UINT32, // ex: 2 dimensions: UINT64[], // ex: [ 4096, 32000 ] type: UINT32, // ex: 10 (typically indicates quantization level, here "GGML_TYPE_Q2_K") offset: UINT64 // starting position within the tensor_data block, relative to the start of the block // (n+1)-th tensor ... REFERENCES ---------- BIBLIOGRAPHY (BibTeX) -------------------- @article{ref0, title = {Llama.cpp}, url = {https://encyclopegia.org/a/llama-cpp} } BIBLIOGRAPHY (RIS) ------------------- TY - JOUR TI - Llama.cpp UR - https://encyclopegia.org/a/llama-cpp ER - CITE THIS ARTICLE ---------------- Cite this article: Llama.cpp Derived from: https://en.wikipedia.org/w/index.php?oldid=1378655897 Licence: CC BY-SA 4.0 Snapshot: enwiki-001378655897 Permalink: /a/llama-cpp