3.65 muniq - Unique Records

Remove duplicate values and create unique records.

Format

muniq [k=] [i=] [o=] [-nfn] [-nfno] [-x] [-q] [--help] [--version]

Parameter

`k=`	Specify the field name (s) as the unique identifier of the records.

Examples

Example 1: Basic Example

Remove duplicate records in the date field.

$ more dat1.csv
date,customer
20081201,A
20081202,A
20081202,B
20081202,B
20081203,C
$ muniq k=date i=dat1.csv o=rsl1.csv
#END# kguniq i=dat1.csv k=date o=rsl1.csv
$ more rsl1.csv
date%0,customer
20081201,A
20081202,B
20081203,C

Example 2: Delete duplicate rows in multiple columns

Remove duplicate records based on unique values in date and customer field.

$ muniq k=date,customer i=dat1.csv o=rsl2.csv
#END# kguniq i=dat1.csv k=date,customer o=rsl2.csv
$ more rsl2.csv
date%0,customer%1
20081201,A
20081202,A
20081202,B
20081203,C

Related Command

mbest : Use mbest command to select the line number for records with the same key.