{
  "id": "enwiki-001375305667",
  "slug": "awk",
  "title": "AWK",
  "qid": "",
  "lead": "AWK ( ) is a scripting language designed for text processing and typically used as a data extraction and reporting tool. Like sed and grep, it is a filter, and it is a standard feature of most Unix-like operating systems. The shell command that runs the AWK processor is named . The AWK language is a data-driven scripting language consisting of a set of actions to be taken against streams of textual data – either run directly on files or used as part of a pipeline – for purposes of extracting or transforming text, such as producing formatted reports. The language extensively uses the string datatype, associative arrays (that is, arrays indexed by key strings), and regular expressions. While AWK has a limited intended application domain and was especially designed to support one-liner programs, the language is Turing-complete, and even the early Bell Labs users of AWK often wrote well-structured large AWK programs. AWK was created at Bell Labs in the 1970s, and its name is derived from the surnames of its authors: Alfred Aho (author of egrep), Peter Weinberger (who worked on tiny relational databases), and Brian Kernighan. The acronym is pronounced the same as the name of the bird species auk, which is illustrated on the cover of The AWK Programming Language.",
  "body": "AWK ( ) is a scripting language designed for text processing and typically used as a data extraction and reporting tool. Like sed and grep, it is a filter, and it is a standard feature of most Unix-like operating systems. The shell command that runs the AWK processor is named . The AWK language is a data-driven scripting language consisting of a set of actions to be taken against streams of textual data – either run directly on files or used as part of a pipeline – for purposes of extracting or transforming text, such as producing formatted reports. The language extensively uses the string datatype, associative arrays (that is, arrays indexed by key strings), and regular expressions. While AWK has a limited intended application domain and was especially designed to support one-liner programs, the language is Turing-complete, and even the early Bell Labs users of AWK often wrote well-structured large AWK programs. AWK was created at Bell Labs in the 1970s, and its name is derived from the surnames of its authors: Alfred Aho (author of egrep), Peter Weinberger (who worked on tiny relational databases), and Brian Kernighan. The acronym is pronounced the same as the name of the bird species auk, which is illustrated on the cover of The AWK Programming Language. ==History== According to Brian Kernighan, one of the goals of AWK was to have a tool that would easily manipulate both numbers and strings. AWK was also inspired by Marc Rochkind's programming language that was used to search for patterns in input data, and was implemented using yacc. As one of the early tools to appear in Version 7 Unix, AWK added computational features to a Unix pipeline besides the Bourne shell, the only scripting language available in a standard Unix environment. It is one of the mandatory utilities of the Single UNIX Specification, and is required by the Linux Standard Base specification. In 1983, AWK was one of several UNIX tools available for Charles River Data Systems' UNOS operating system under Bell Laboratories license. AWK was significantly revised and expanded in 1985–88, resulting in the GNU AWK implementation written by Paul Rubin, Jay Fenlason, and Richard Stallman, released in 1988. GNU AWK may be the most widely deployed version because it is included with GNU-based Linux packages. GNU AWK has been maintained solely by Arnold Robbins since 1994. Brian Kernighan's nawk (New AWK) source was first released in 1993 unpublicized, and publicly since the late 1990s; many BSD systems use it to avoid the GPL license. AWK was preceded by sed (1974). Both were designed for text processing. They share the line-oriented, data-driven paradigm, and are particularly suited to writing one-liner programs, due to the implicit main loop and current line variables. The power and terseness of early AWK programs – notably the powerful regular expression handling and conciseness due to implicit variables, which facilitate one-liners – together with the limitations of AWK at the time, were important inspirations for the Perl language (1987). In the 1990s, Perl became very popular, competing with AWK in the niche of Unix text-processing languages. condition { action } condition { action } ... where condition is typically an expression and action is a series of commands. The input is split into records, where by default records are separated by newline characters so that the input is split into lines. The program tests each record against each of the conditions in turn, and executes the action for each expression that is true. Either the condition or the action may be omitted. The condition defaults to matching every record. The default action is to print the record. This is the same pattern-action structure as sed. In addition to a simple AWK expression, such as foo == 1 or /^foo/ , the condition can be BEGIN or END causing the action to be executed before or after all records have been read, or pattern1, pattern2 which matches the range of records starting with a record that matches pattern1 up to and including the record that matches pattern2 before again trying to match against pattern1 on subsequent lines. In addition to normal arithmetic and logical operators, AWK expressions include the tilde operator, ~ , which matches a regular expression against a string. As handy syntactic sugar, /regexp/ without using the tilde operator matches against the current record; this syntax derives from sed, which in turn inherited it from the ed editor, where / is used for searching. This syntax of using slashes as delimiters for regular expressions was subsequently adopted by Perl and ECMAScript, and is now common. The tilde operator was also adopted by Perl. /regex_pattern/ { # Actions to perform in the event the record (line) matches the above regex_pattern print 3+2 print foobar(3) print foobar(variable) print sin(3-2) } Output may be sent to a file: /regex_pattern/ { # Actions to perform in the event the record (line) matches the above regex_pattern print \"expression\" > \"file name\" } or through a pipe: /regex_pattern/ { # Actions to perform in the event the record (line) matches the above regex_pattern print \"expression\" | \"command\" } $ awk ' > BEGIN { > > OFMT = \"%.14g\" > > print 331 % -79 > print -3.31e-11 % 7.9e-23 > > print 331 / 79 > print (331 - 331 % 79) / 79 > }' 15 -1.1999020635259e-23 4.1898734177215 4 The int( dividend / divisor ) approach may lead to edge cases that round incorrectly. For string concatenation, simply place two variables (or string constants) next to each other. It is optional to use a space in between if string constants are involved, but two variable names placed adjacent to each other require a space in between. Unlike Unix shells or other languages like Perl, the sigil-less single underscore variable name ( _ ) is neither reserved nor special, and can be freely utilized by the user. Since its default value is the special AWK NULL value that is simultaneously numeric zero and the empty string, it could even mimic Perl's $_ as an alias for $0 : jot 30 | perl -nle ' print $_ Double quotes ( \"...\" ), and only double quotes, delimit string constants. Statements and / or expressions need not end with semicolons. In fact, the only place where the semi-colon ; is absolutely unavoidable is the 3-argument variant of the for () loop that largely resembles its counterpart in C/C++ . All non-array arguments to functions are passed \"by-value\", making them freely mutable by the function while guaranteeing being side-effect free to the function's upstream caller. Unlike Python, all augmented assignments in awk such as += -= *= /= ^= %= are expressions. This allows for effective unlimited-length chaining. One of its biggest benefits is demonstrated by the algorithm for calculating Fibonacci numbers having complexity of O(n) because of the reason that it keeps adding back on itself: For the first 77 or so Fibonacci numbers, this approach is highly effective, since it bypasses having to examine bits, swap variables, or incur recursion overhead. Finally, comments can be added to programs by using # as the first character on a line, or behind a command or sequence of commands. function add_three(number) { return number + 3 } This statement can be invoked as follows: (pattern) { print add_three(36) # Outputs 39 } Functions can have variables that are in the local scope. The names of these are added to the end of the argument list, though values for these should be omitted when calling the function. It is convention to add some whitespace in the argument list before the local variables, to indicate where the parameters end and the local variables begin. BEGIN { print \"Hello, world!\" exit } length($0) > 80 { words += NF chars += length + 1 # add one to account for the newline character at the end of each record (line) } END { print NR, words, chars } As there is no pattern for the first line of the program, every line of input matches by default, so the increment actions are executed for every line. words += NF is shorthand for words = words + NF . { s += $NF } END { print s + 0 } s is incremented by the numeric value of $NF , which is the last word on the line as defined by AWK's field separator (by default, white-space). NF is the number of fields in the current line, e.g. 4. Since $4 is the value of the fourth field, $NF is the value of the last field in the line regardless of how many fields this line has, or whether it has more or fewer fields than surrounding lines. $ is actually a unary operator with the highest operator precedence. (If the line has no fields, then NF is 0, $0 is the whole line, which in this case is empty apart from possible white-space, and so has the numeric value 0.) At the end of the input, the END pattern matches, so s is printed. However, since there may have been no lines of input at all, in which case no value has ever been assigned to s , s will be an empty string by default. Adding zero to a variable is an AWK idiom for coercing it from a string to a numeric value. This results from AWK's arithmetic operators, like addition, implicitly casting their operands to numbers before computation as required. (Similarly, concatenating a variable with an empty string coerces from a number to a string, e.g., s \"\" . Note, there is no operator to concatenate strings, they are just placed adjacently.) On an empty input, the coercion in { print s + 0 } causes the program to print 0 , whereas with just the action { print s } , an empty line would be printed. NR % 4 == 1, NR % 4 == 3 { printf \"%6d %s\\n\", NR, $0 } The action statement prints each line numbered. The printf function emulates the standard C printf and works similarly to the print command described above. The pattern to match, however, works as follows: is the number of records, typically lines of input, AWK has so far read, i.e. the current line number, starting at 1 for the first line of input. is the modulo operator. is true for the 1st, 5th, 9th, etc., lines of input. Likewise, is true for the 3rd, 7th, 11th, etc., lines of input. The range pattern is false until the first part matches, on line 1, and then remains true up to and including when the second part matches, on line 3. It then stays false until the first part matches again on line 5. Thus, the program prints lines 1,2,3, skips line 4, and then 5,6,7, and so on. For each line, it prints the line number (on a 6 character-wide field) and then the line contents. For example, when executed on this input: Rome Florence Milan Naples Turin Venice The previous program prints: 1 Rome 2 Florence 3 Milan 5 Turin 6 Venice /^--cut here--$/, 0 prints lines of input from the first line matching the regular expression , that is, a line containing only the phrase \" \", to the end. BEGIN { FS=\"[^a-zA-Z]+\" } { for (i=1; i The BEGIN block sets the field separator to any sequence of non-alphabetic characters. Separators can be regular expressions. After that, we get to a bare action, which performs the action on every input line. In this case, for every field on the line, we add one to the number of times that word, first converted to lowercase, appears. Finally, in the END block, we print the words with their frequencies. The line 9 creates a loop that goes through the array , setting to each subscript of the array. This is different from most languages, where such a loop goes through each value in the array. The loop thus prints out each word followed by its frequency count. tolower was an addition to the One True awk (see below) made after the book was published. !/bin/sh pattern=\"$1\" shift awk '/'\"$pattern\"'/ { print FILENAME \":\" $0 }' \"$@\" The $pattern in the awk command is not protected by single quotes so that the shell does expand the variable but it needs to be put in double quotes to properly handle patterns containing spaces. A pattern by itself in the usual way checks to see if the whole line ( $0 ) matches. FILENAME contains the current filename. awk has no explicit concatenation operator; two adjacent strings concatenate them. $0 expands to the original unchanged input line. There are alternate ways of writing this. This shell script accesses the environment directly from within awk: !/bin/sh export pattern=\"$1\" shift awk '$0 ~ ENVIRON[\"pattern\"] { print FILENAME \":\" $0 }' \"$@\" This is a shell script that uses ENVIRON , an array introduced in a newer version of the One True awk after the book was published. The subscript of ENVIRON is the name of an environment variable; its result is the variable's value. This is like the getenv function in various standard libraries and POSIX. The shell script makes an environment variable pattern containing the first argument, then drops that argument and has awk look for the pattern in each file. ~ checks to see if its left operand matches its right operand; !~ is its inverse. A regular expression is just a string and can be stored in variables. The next way uses command-line variable assignment, in which an argument to awk can be seen as an assignment to a variable: !/bin/sh pattern=\"$1\" shift awk '$0 ~ pattern { print FILENAME \":\" $0 }' pattern=\"$pattern\" \"$@\" Or You can use the command line option (e.g. ). Finally, this is written in pure awk, without help from a shell or without the need to know too much about the implementation of the awk script (as the variable assignment on command line one does), but is a bit lengthy: BEGIN { pattern = ARGV[1] for (i = 1; i The BEGIN is necessary not only to extract the first argument, but also to prevent it from being interpreted as a filename after the BEGIN block ends. ARGC , the number of arguments, is always guaranteed to be ≥1, as ARGV[0] is the name of the command that executed the script, most often the string \"awk\" . ARGV[ARGC] is the empty string, \"\" . # initiates a comment that expands to the end of the line. Note the if block. awk only checks to see if it should read from standard input before it runs the command. This means that awk 'prog' only works because the fact that there are no filenames is only checked before prog is run! If you explicitly set ARGC to 1 so that there are no arguments, awk will simply quit because it feels there are no more input files. Therefore, you need to explicitly say to read from standard input with the special filename - . !/usr/bin/awk -f { print $0 } It can be invoked with: ./print.awk The -f tells awk that the argument that follows is the file to read the AWK program from, which is the same flag that is used in sed. Since they are often used for one-liners, both these programs default to executing a program given as a command-line argument, rather than a separate file.",
  "categories": [
    "101754115",
    "130042203",
    "1876694",
    "22906073",
    "22911899",
    "22932021",
    "22932022",
    "22932023",
    "23118434",
    "23603337",
    "24623845",
    "24745122",
    "27068476",
    "27515961",
    "28081657",
    "3471581",
    "42143503",
    "43907796",
    "63481706"
  ],
  "license": "CC BY-SA 4.0",
  "source_uri": "https://en.wikipedia.org/w/index.php?oldid=1375305667",
  "content_hash": "",
  "created_at": null,
  "updated_at": 1789600160.0,
  "claims": [
    {
      "id": 29378316,
      "subject_qid": "",
      "subject_slug": "awk",
      "predicate": "release date",
      "object_text": "1988",
      "object_num": 19880000.0,
      "object_unit": null,
      "polarity": 1,
      "qualifier": "",
      "source_uri": "https://en.wikipedia.org/w/index.php?oldid=1375305667",
      "source_name": "Wikipedia",
      "source_tier": 3,
      "retrieved_at": 1792482332.7595143,
      "confidence": 0.95,
      "article_slug": "awk",
      "span_start": 154,
      "span_end": 171
    },
    {
      "id": 29378317,
      "subject_qid": "",
      "subject_slug": "awk",
      "predicate": "release date",
      "object_text": "1993",
      "object_num": 19930000.0,
      "object_unit": null,
      "polarity": 1,
      "qualifier": "",
      "source_uri": "https://en.wikipedia.org/w/index.php?oldid=1375305667",
      "source_name": "Wikipedia",
      "source_tier": 3,
      "retrieved_at": 1792482332.759515,
      "confidence": 0.95,
      "article_slug": "awk",
      "span_start": 50,
      "span_end": 98
    }
  ],
  "contradictions": [],
  "confidence": 0.95,
  "csl": {
    "id": "enwiki-001375305667",
    "type": "webpage",
    "title": "AWK",
    "author": [
      {
        "family": "Anticloud FZ LLE",
        "literal": "Anticloud FZ LLE"
      }
    ],
    "container-title": "encyclopegia.org",
    "publisher": "Anticloud FZ LLE",
    "publisher-place": "Dubai, United Arab Emirates",
    "URL": "https://encyclopegia.org/a/awk",
    "issued": {
      "date-parts": [
        [
          2026,
          11,
          19
        ]
      ]
    },
    "accessed": {
      "date-parts": [
        [
          2026,
          11,
          19
        ]
      ]
    },
    "license": "CC BY-SA 4.0",
    "DOI": "eo1:enwiki-001375305667",
    "isBasedOn": "https://en.wikipedia.org/w/index.php?oldid=1375305667",
    "note": "Immutable snapshot enwiki-001375305667"
  }
}