compare pairs stored in hashtable java

Question

at first i have a 2d array storing many numbers and then i used an array of hushtables to store pairs formed by the numbers of each row of the 2d array. for example in the first row of the 2d array, the numbers are 1 2 3 4 5 then the pairs should be 1,2 1,3 1,4 ,1,5 etc. the code for generating the pairs and store in hashtable is

     Hashtable [] table=new Hashtable [lineCounter];
    int [] pairCounter=new int[lineCounter];
    for(int i=0;i<lineCounter;i++)
  {
     table[i]=new Hashtable();
     for (int j=0;j<lineitem2[i]-1;j++)
     {
         for(int t=j+1;t<lineitem2[i];t++)
         {
             int firstnum=freItem[i][j];
             int secnum=freItem[i][t];
             String value=firstnum+":"+secnum;
             //System.out.println(firstnum+"``"+secnum);
             table[i].put(pairCounter[i],value);
             pairCounter[i]++;
             //System.out.println(i+":::"+table[i].get(firstnum));
         }
     }
 }

after stored each pair of everyline, I want to compare each pair in every to the other lines to find how many times this pair shows up, the code is

Hashtable resulttable=new Hashtable();
   int [] pairresult=new int [lineCounter];

   for(int i=0;i<lineCounter;i++)
   {
           //Set set1 = table[i].entrySet();
        //Iterator it1 = set1.iterator();
        Enumeration keys = table[i].keys();


       //for(int j=i+1;j<lineCounter;j++)
       //{


         while (keys.hasMoreElements())
             {
                int pairs=0;
               //Map.Entry entry1 = (Map.Entry) it1.next();
               int key = (Integer)keys.nextElement();
               String curvalue = (String)table[i].get( key );

               for(int j=i+1;j<lineCounter;j++)
               {

                 //Set set2 = table[j].entrySet();
                 //Iterator it2 = set2.iterator();
                 Enumeration keys2 = table[j].keys();

                 while (keys2.hasMoreElements())
                 {

                   //Map.Entry entry2 = (Map.Entry) it2.next();
                   //System.out.println(entry2.getKey() + " and " + entry2.getValue());
                   int key2 = (Integer)keys2.nextElement();
                   String curvalue2 = (String)table[j].get( key2 );
                   if(curvalue.equals(curvalue2))
                   {
                       pairs++;
                       table[j].remove(key2);
                   }
                 }

                 }
                if(pairs>=0.02*lineCounter)
                {
                  resulttable.put(curvalue,pairs);
                }







            }


   //    }


   }

but after ran it with the input file, I got the error:

Exception in thread "main" java.lang.OutOfMemoryError: Java heap space at java.lang.StringBuilder.toString(StringBuilder.java:430)

is there anything wrong with my method of comparing the pairs? and why did I get that error please help thanks.

Answer 1

If your code works on a smaller data set, it might just be that the default 64Mb that the JVM uses is not enough, does it work when you pass -Xmx512m as argument to the Java command line?

Answer 2

Okay, suppose you have a Pair class as follows:

public class Pair {

    private final int value1;
    private final int value2;

    public Pair(int value1, int value2) {
        this.value1 = value1;
        this.value2 = value2;
    }

    public int value1() {
        return value1;
    }

    public int value2() {
        return value2;
    }

    @Override
    public int hashCode() {
        final int prime = 31;
        int result = 1;
        result = prime * result + value1;
        result = prime * result + value2;
        return result;
    }

    @Override
    public boolean equals(Object obj) {
        if (this == obj)
            return true;
        if (obj == null)
            return false;
        if (getClass() != obj.getClass())
            return false;
        Pair other = (Pair) obj;
        if (value1 != other.value1)
            return false;
        if (value2 != other.value2)
            return false;
        return true;
    }

    @Override
    public String toString() {
        return "(" + value1 + ", " + value2 + ")";
    }

}

Note that it's important to properly implement the equals(Object) and hashCode() methods if you expect instances of the Pair class to behave properly when used in hash-based data structures (for example: Hashtable, HashMap, HashSet, HashMultimap, HashMultiset).

Now, this code will read in a file (requires the Guava libraries):

    File file = ...;

    final Map<Pair, Collection<Integer>> lineNumbersByPair = new HashMap<Pair, Collection<Integer>>();

    /*
     * Step 1: Read in the lines, one by one.
     */
    Reader reader = new FileReader(file);
    try {
        BufferedReader bufferedReader = new BufferedReader(reader);
        try {
            String line;

            int lineNumber = 0;
            while ((line = bufferedReader.readLine()) != null) {
                lineNumber++;

                String[] tokens = line.split("\\s+");
                int[] values = new int[tokens.length];

                for (int i = 0; i < tokens.length; i++) {
                    values[i] = Integer.parseInt(tokens[i]);
                }

                for (int i = 0; i < values.length; i++) {
                    for (int j = i + 1; j < values.length; j++) {
                        Pair pair = new Pair(values[i], values[j]);

                        Collection<Integer> lineNumbers;
                        if (lineNumbersByPair.containsKey(pair)) {
                            lineNumbers = lineNumbersByPair.get(pair);
                        } else {
                            lineNumbers = new HashSet<Integer>();
                            lineNumbersByPair.put(pair, lineNumbers);
                        }
                        lineNumbers.add(lineNumber);
                    }
                }
            }
        } finally {
            bufferedReader.close();
        }
    } finally {
        reader.close();
    }

    /*
     * Step 2: Identify the unique pairs. Sort them according to how many lines they appear on (most number of lines to least number of lines).
     */
    List<Pair> pairs = new ArrayList<Pair>(lineNumbersByPair.keySet());
    Collections.sort(
            pairs,
            new Comparator<Pair>() {
                @Override
                public int compare(Pair pair1, Pair pair2) {
                    Integer count1 = lineNumbersByPair.get(pair1).size();
                    Integer count2 = lineNumbersByPair.get(pair2).size();
                    return count1.compareTo(count2);
                }
            }
        );
    Collections.reverse(pairs);

    /*
     * Step 3: Print the pairs and their line numbers.
     */
    for (Pair pair : pairs) {
        Collection<Integer> lineNumbers = lineNumbersByPair.get(pair);
        if (lineNumbers.size() > 1) {
            System.out.println(pair + " appears on the following lines: " + lineNumbers);
        }
    }

In a test, the code read in a file with 20,000 lines, each line containing 10 numbers ranging anywhere between 0 and 1000.