Awk
AWK Programming
Variables
Operators
Formatting
Print the Next Line
Common Examples
AWK Programming
awk is filter. awk performs the specified action on lines that match the pattern. Like 'sed', a regular expression pattern must be enclosed in a forward slashes. If no pattern is listed, the action is performed on every input line.
awk 'pattern {action}' [files]
$ awk '/^S/' phone.list
$ awk '/^S/ {print $1}' phone.list
Variables
$0 represents the whole current input record $1 represents the first field of the current input record $NF is number of fields for the current input record NR is number of Lines read
Operators
Relational Operators
< and > Less than and Greater than <= and >= Less than or equl to AND Greater than or equl to == and != Equl to AND Not qual to ~ and !~ Matches AND Doesn't match regular expressions
Boolean Operator:
|| Logical OR
&& Logical AND
! Logical NOT
Assignment Operators
++ Add 1 to variable. -- Subtract 1 from variable. += Assign result of addition. -= Assign result of subtraction. *= Assign result of multiplication. /= Assign result of division. %= Assign result of modulo. ^= Assign result of exponentiation. **= Assign result of exponentiation.[6]
Arithmetic Operator
+ Addition - Subtraction * Multiplication / Division % Modulo Division (Remainder) = assignment
Formatting
printf fuction: printf statement can be used to specify the width and alignment of output fields. Unlike print function, it does not give any new line charater at the end of the line. The contents of the field will ne right justif ied by default. You musr specify "-" to get left justification. Thus "%-20s" out puts a string left justif ied in a fiels 20 character wide.
Format Specifiers used in printf
c ASCII Character d Decimal Integer e Floating point format s String % Literal %
Example of formatting:
Input file is comma separated.
$ cat file.txt hiddenhausen,99.60,y herbstein,99.021,n bangalore,98.82,y
Lets try to format the above file:
$ awk '{ printf "%-15s%-8s%s\n",$1,$2,$3}' FS=\, file.txt
hiddenhausen 99.60 y
herbstein 99.021 n
bangalore 98.82 y
So its printing first field ($1) as a string of 15 characters that are left-justified, 2nd field ($2) as 8 characters left-justified and then third field ($3)
Another Example
# cat abc.txt
id:name:IO_group_id:IO_group_name:status:mdisk_grp_id:mdisk_grp_name:capacity:type:FC_id:FC_name:RC_id:RC_name:vdisk_UID:fc_map_count:copy_count:fast_write_state
0:D53R-R1A-0915:3:prod1:online:2:D53R-R1A:256.00GB:striped:::0:HERCOPY125:60050768018F028260000000000006D9:0:1:not_empty
1:D53I-R1A-0551:2:prod0:online:1:D53I-R1A:256.00GB:striped:::1:HERCOPY126:60050768018F028260000000000006DA:0:1:not_empty
2:D83B-R1A-0717:3:prod1:online:5:D83B-R1A:256.00GB:striped:76:EDBACK077:::60050768018F028260000000000006DB:1:1:not_empty
# cat abc.txt | awk -F: '{printf "%-5s%-16s%-8s%-10s%-10s %s\n", $1,$2,$4,$7,$8,$14}'| head
id name IO_group_namemdisk_grp_namecapacity vdisk_UID
0 D53R-R1A-0915 prod1 D53R-R1A 256.00GB 60050768018F028260000000000006D9
1 D53I-R1A-0551 prod0 D53I-R1A 256.00GB 60050768018F028260000000000006DA
2 D83B-R1A-0717 prod1 D83B-R1A 256.00GB 60050768018F028260000000000006DB
3 D83B-R1A-0718 prod0 D83B-R1A 256.00GB 60050768018F028260000000000006DC
Printing the next line of a search Pattern
To search and print the next line use the following awk commands
awk '/pattern/{getline;print}'
# ioscan –funC disks | awk '/ROM/{getline;print}'
disk 4 0/0/2/1.2.0 sdisk CLAIMED DEVICE HP CD-ROM 305
Print the current line and nextline
# ioscan -funC disk | awk '$1=="disk" { getline L2 ; print $0,L2 }'
Same as above, but another way
# ioscan -funNC disk | awk '{if ($1=="disk") {L1=$0;getline; sub (" *","");print L1,$0}}'
Common Examples
To find a patern "Dec" in 6th field
ls -l | awk '$6 == "Dec"'
To display the first and second fields of each line
awk '{print $1, $2}' <filename>
To display all the lines containing string1 in file called file1
awk '/string1/' file1
To display the first and 3rd fields of /etc/passwd file
awk -F: '{print $1, $3}'
To display the first and last field of /etc/passwd file
awk -F: '{print $1, $NF}'
To display the previous field to the last filed of any file
awk '{print $(NF-1)}' file_name
To sum all the fields using awk command:
ls -l | awk '{sum += $5} {print sum}'
ls -l | awk '{sum += $5} END {print sum}'
To add few numbers and find out the average
echo "test 10 20 30" | awk '{total = $2 + $3 + $4} {ave = total / 3} {print ave}'
To add the value of Unix variable $var1 to the beginning of each line in file1
cat file1 | awk -v name=$var1 '{print name" " $0}'
Running Unix commands directly using awk
# ps -ef | grep syncd | grep -v grep | awk '{system("kill -9 " $2)}'
The above command find the process ID of sysncd and kills that process
To replace DS with DSsmtp.comcast.net if the line has only single word 'DS'
awk '{if ($0~/^DS$/) {print "DSsmtp.comcast.net"} \
else {print}}' /etc/mail/sendmail.cf
Another AWK Complex example
awk '{if ($0!~/^\*/ && $0~/:$/ && $1~/^default:$/) {print $0"\n\tregistry = NIS"} \
else if ($0!~/^\*/ && $0~/:$/ && $1!~/^default:$/) {print $0"\n\tregistry = files"} \
else {print}}' /etc/security/user > /etc/security/user.new
To add # mark to the beginning of each line except for the lines starting with #, ftp, shell, and telnet:
awk '{if ($1~/^#/ || $1~/^ftp/ \
|| $1~/^shell/ || $1~/^telnet/) \
{print} else {print "#"$0}}' /etc/inetd.conf
Some examples with sample output
$ awk -F":" '{ print "username: " $1 "\t\tuid:" $3" }' /etc/passwd
username: halt uid:7
username: operator uid:11
username: root uid:0
http://www.ibm.com/developerworks/linux/library/l-awk1.html
http://www.ibm.com/developerworks/linux/library/l-awk2.html