bash-awk

Last updated Sep 20, 2023 Edit Source

• Bash

• GeeksforGeeks - awk

awk is a scripting language that allows text manipulation, but also arithmetic and conditional logic. It’s commonly used to:

• Validate/search through text
• Transforming text input, and thus producing formatted text files.
• Performing calculations
• Looping through data and applying conditional logic.

TL;DR:

• awk runs line by line. Each line will be split into fields, represented by each string (if delimited by space).
• The delimiter of what represents a “line” or “field” can be changed.
• awk is capable of arithmetic, often by using variable assignment also.
• awk is also capable of if-else, for and while logic.

1

awk options 'selection _criteria {action }' input-file > output-file

• selection_criteria is used to pattern match against fields (strings).
• The pattern to be matched is denoted by single quotes ('').
• Examples of {actions} include:
  • print: print the field.
  • printf: formatted printing similar to python.
  • delete: can be used to delete both fields and lines.

Common commands/arguments: Example syntax incorporation:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11

#print the first and last fields
awk '{print $1,$NF}' employee.txt

#print all lines, with line numbers
awk '{print NR,$0}' employee.txt 

#print lines from 3 to 6
awk 'NR3, NR6 {print NR,$0}' employee.txt

#count the number of lines in a file
awk 'END { print NR }' test.txt

END: - END is a special block used to specify an action once all records are processed. It’s useful when looking to validate an entire file, such as by counting number of lines.

Positional variables: - When Awk processes a line/record, it will split each delimited set of characters (string) into variables. - “$1”, “$2”, “$3” etc. refer to each field within the record – it is NOT zero-indexed. - “$0” refers to the entire input record (or line) in “awk”.

Variable assignment:

1
2
3
4
5
6
7
8

# syntax:
variable_name=value

# e.g:
awk '{ name = $1; age = $2; print "Name:", name, "Age:", age }' data.txt

# you can also declare variables with defined values, prior to processing: 
awk -v age=25 '{ print "Age:", age }' data.txt

Common special variables:

• NR: NR command keeps a current count of the number of input records.
• NF: NF command keeps a count of the number of fields within the current input record.
• FS: FS command contains the field separator character which is used to divide fields on the input line. The default is “white space”, meaning space and tab characters.
• RS: RS command stores the current record separator character. Since, by default, an input line is the input record, the default record separator character is a newline.
• OFS: OFS command stores the output field separator, which separates the fields when Awk prints them. The default is a blank space.

awk can be used to delete both fields or lines.

Deleting per pattern:

1
2

#The following command will delete all lines that contain the word "foo":
awk '/foo/ delete' input.txt

Deleting per positional variable:

1
2

#The following deletes the second field from each line, but only after printing it first:
awk '/foo/ {print $2} delete $1' input.txt

Notes:

• The delete action is applied after all other awk actions have been executed.
• It only be used to delete fields or records that have already been created.
• It cannot delete fields or records that are created by other awk actions, such as split.

Example if-else logic:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11

awk '{  
if(condition_here)  
{  
print "condition matched!"  
}  
else  
{  
print "condition not matched!"  
}  
   
}' sample_data.txt