tags:

views:

259

answers:

3

Looking for an awk (or sed) one-liner to remove lines from the output if the first field is a duplicate.

An example for removing duplicate lines I've seen is:

awk 'a !~ $0; {a=$0}'

Tried using it for a basis with no luck (I thought changing the $0's to $1's would do the trick, but didn't seem to work).

A: 
$1 !~ /pattern/
Darius Bacon
-1 You're not answering the question.
glenn jackman
"remove lines from the output if the first field matches." I'll delete this answer after the question gets worded clearly, since it is clear now that this isn't what he meant.
Darius Bacon
+2  A: 
awk '{ if (a[$1]++ == 0) print $0; }' "$@"

This is a standard (very simple) use for associative arrays.

Jonathan Leffler
That worked! I had another bug that I didn't realize as well that may have been giving me problems aswell. Thanks!
Kyle
+1  A: 

this is how to remove duplicates

awk '!_[$1]++' file
ghostdog74
Using '_' as the array name invites misunderstanding - but it works.
Jonathan Leffler