Remove Duplicates from List of HashMap Entries

Question

I have a List<HashMap<String,Object>> which represents a database where each list record is a database row.

I have 10 columns in my database. There are several rows where the values of 2 particular columns are equals. I need to remove the duplicates from the list after the list is updated with all the rows from database.

What is the efficient way?

FYI - I am not able to do distinct while querying the database, because the GroupName is added at a later stage to the Map after the database is loaded. And since Id column is not primary key, once you add GroupName to the Map. You will have duplicates based on Id + GroupName combination!

Hope my question makes sense. Let me know if we need more clarification.

Answer 1

1. create a Comparator that compares HashMaps, and compares them by comparing the key/value pairs you are interested in.
2. use Collections.sort(yourlist, yourcomparator);
3. Now all maps that are similar to each other, based on your comparator, are adjacent in the list.
4. Create a new list.
5. Iterate through your first list, keeping track of what you saw last. If the current value is different than the last, add this to your new list.
6. You new list should contain no duplicates according to your comparator.

The cost of iterating through the list is O(n). Sorting is O(n log n). So this algorithm is O(n log n).

We could also sort on-the-fly by using a TreeSet with that comparator. Inserts are O(log n). And we have to do this n times. So we get O(n log n).

Answer 2

I am new to Comparators in Java. Can someone guide me on the following - create a Comparator that compares HashMaps, and compares them by comparing the key/value pairs that are of interest.