using DictWriter in Python to write a subset of a dictionary's keys

Question

I wrote a function that serializes a list of dictionaries as a CSV file using the CSV module. I sometimes want to write out to a file only a subset of each dictionary's keys however. I use the following code:

def dictlist2file(dictrows, filename, fieldnames, delimiter='\t',
          lineterminator='\n'):
out_f = open(filename, 'w')

# Write out header
header = delimiter.join(fieldnames) + lineterminator
out_f.write(header)

# Write out dictionary
data = csv.DictWriter(out_f, fieldnames,
          delimiter=delimiter,
          lineterminator=lineterminator)
data.writerows(dictrows)
out_f.close()

If I pass in "fieldnames" a subset of the keys that each dictionary has, I get the error:

"dict contains fields not in fieldnames"

How can I make it so that DictRows will write just a subset of the fields I specify to CSV, ignoring those fields that are in the dictionary but not in fieldnames?

thanks.

Answer 1

Simplest and most direct approach is to pass extrasaction='ignore' when you initialize your DictWriter instance, as documented here:

If the dictionary passed to the writerow() method contains a key not found in fieldnames, the optional extrasaction parameter indicates what action to take. If it is set to 'raise' a ValueError is raised. If it is set to 'ignore', extra values in the dictionary are ignored.

It also works on writerows, which, internally, just calls writerow repeatedly.

Answer 2

Changes to your code:

Forget Dictwriter, use ordinary writer.

Then loop over your list of dicts:

for d in dictrows:
    ordinary_writer.writerow([d[fieldname] for fieldname in fieldnames])

Use d.get(fieldname, "") instead of d[fieldname] if you don't want an exception if there is no entry in d for a fieldname.

Note to anonymous downvoters: This is doing what Alex's solution is doing under the hood (see Lib/csv.py) and doing it a bit better ... csv.py calls a function to get each row in a list, and the guts of that function is

return [rowdict.get(key, self.restval) for key in self.fieldnames]