uniq

report or omit repeated lines

Syntax

uniq [OPTION]... [INPUT [OUTPUT]]

Description

Filter adjacent matching lines from INPUT (or standard input), writing to OUTPUT (or standard output). With no options, matching lines are merged to the first occurrence.

Options

-c, --count
       prefix lines by the number of occurrences

-d, --repeated
       only print duplicate lines

-D, --all-repeated[=delimit-method]
       print all duplicate lines delimit-method={none(default),prepend,separate} Delimiting is done with blank lines.

-f, --skip-fields=N
       avoid comparing the first N fields

-i, --ignore-case
       ignore differences in case when comparing

-s, --skip-chars=N
       avoid comparing the first N characters

-u, --unique
       only print unique lines

-z, --zero-terminated
       end lines with 0 byte, not newline

-w, --check-chars=N
       compare no more than N characters in lines

Example 1

Demonstrate uniq command.

[eleiss@eleiss ~]$ paste group-a group-b group-c
Noor	Shoiab	Afzal
Afzal	Azhar	Ali
Shoaib	Asif	Zafar
Shakib	Zafar	Shakib
Husain	Imraan	Noor
Salmaan	Kasif	Husain
[eleiss@eleiss ~]$ cat group-a group-b group-c | sort
Afzal
Afzal
Ali
Asif
Azhar
Husain
Husain
Imraan
Kasif
Noor
Noor
Salmaan
Shakib
Shakib
Shoaib
Shoiab
Zafar
Zafar
[eleiss@eleiss ~]$ cat group-a group-b group-c | sort | uniq
Afzal
Ali
Asif
Azhar
Husain
Imraan
Kasif
Noor
Salmaan
Shakib
Shoaib
Shoiab
Zafar
[eleiss@eleiss ~]$ 

There are three files "group-a", "group-b" and "group-c". The content of all the three files is displayed in three different columns using paste command. There are many names which are common to any of the two files.

The second command displays the content of all the three files at a time sorted in alphabetical order using the combination of cat and sort commands.

Finally, the uniq command prints unique lines in the previous commands output.

uniq command eliminates the repetitive lines only from a sorted output.

Example 2

Display only repeated lines and count them using uniq command.

[eleiss@eleiss ~]$ cat group-a group-b group-c | sort | uniq -dc
      2 Afzal
      2 Husain
      2 Noor
      2 Shakib
      2 Zafar
[eleiss@eleiss ~]$

Consider the same three files as explained in the above example. In this example the uniq displays only the repeated lines using the option -d. It also counts the occurrences of the repeated lines using -c option and prefix it to each line.


You may write to resources@eleiss.com for any Linux related queries.

advertisement