The `uniq` command

report or omit repeated lines 11 examples from the lessons of the IVski Linux Playground — try them in the terminal next to them.

Options in the examples

sort file.txt | uniq

Remove duplicates

sort file.txt | uniq -c

Count occurrences

sort file.txt | uniq -d

Show only duplicates

sort file.txt | uniq -u

Show only unique lines

history | awk '{print $2}' | sort | uniq -c | sort -rn | head -10

cat access.log | awk '{print $1}' | sort | uniq -c | sort -rn

cut -d: -f7 /etc/passwd | sort | uniq -c

awk '{print $1}' /var/log/nginx/access.log | sort | uniq -c | sort -rn | head -3

awk '{print $9}' /var/log/nginx/access.log | sort | uniq -c

awk '$9 == 404 {print $7}' /var/log/nginx/access.log | sort | uniq -c | sort -rn

awk '{print $1}' ~/logs/access.log | sort | uniq -c | sort -rn | head -3

Requests per address, most first