Multiple rows share a value in a column, how do I put all of these rows into one single row?

Question

Hi,

I'm working with a text file that looks something like this:

rs001  EEE /n
rs008  EEE /n
rs345  EEE /n
rs542  CHG /n
re432  CHG /n

I want to be able to collapse all of the rows that share the same value in column 2 into one single row (for example, rs001 rs008 rs345 EEE). Is there an easy way to do this using unix text processing or python?

Thanks

Answer 1

#!/usr/bin/env python
from __future__ import with_statement
from itertools import groupby
with open('file','r') as f:
    # We define "it" to be an iterator, for each line
    # it yields pairs like ('rs001','EEE') 
    it=(line.strip().split() for line in f)
    # groupby does the heave work.
    # lambda p: p[1] is the keyfunction. It groups pairs according to the
    # second element, e.g. 'EEE'
    for key,group in groupby(it,lambda p: p[1]):
        # group might be something like [('rs001','EEE'),('rs008','EEE'),...]
        # key would be something like 'EEE', the value that we're grouping by.
        print('%s %s'%(' '.join([p[0] for p in group]),key))

Answer 2

One option is to build a dictionary keyed on the column 2 data:

from collections import defaultdict  #defaultdict will save a line or two of code

d = defaultdict(list)  # goal is for d to look like {'EEE':['rs001', 'rs008', ...
for line in file('data.txt', 'r'):
    v, k = line.strip().split()
    d[k].append(v)

for k, v in d.iteritems():  # print d as the strings you want
    print ' '.join(v+[k])

This approach has the advantage that it doesn't require the column 2 terms to be grouped together (though whether or not column 2 is pre-grouped is not directly specified in the question).

Answer 3

here's gawk for you

$ awk '{a[$2]=a[$2]FS$1}END{for(i in a)print i,a[i]}' file
EEE  rs001 rs008 rs345
CHG  rs542 re432