Why is my program getting slower and slower ?

Question

I'm using the program to send data from database to the Excel file . It works fine at the beginning and then becomes more and more slowly,finally it run out of the memory and the following error ocurrs: "java.lang.OutOfMemoryError: Java heap space...".

The problem can be resolved by adding the jvm heap sapce.But the question is that it spends too much time to run out the program.

After several minutes,it finished a loop with 4 seconds which can be finished with 0.5 seconds at the beginning . I can't found a solution to make it always run in a certain speed.

Is it my code problem?

Any clues on this?

Here is the code:

public void addAnswerRow(List<FinalUsers> finalUsersList,WritableWorkbook book){

   if (finalUsersList.size() >0 ) {
    try {
       WritableSheet  sheet = book.createSheet("Answer", 0);
       int colCount = 0;
       sheet.addCell(new Label(colCount++,0,"Number"));
       sheet.addCell(new Label(colCount++,0,"SchoolNumber"));
       sheet.addCell(new Label(colCount++,0,"District"));
       sheet.addCell(new Label(colCount++,0,"SchoolName"));
       sheet.setColumnView(1, 15);
       sheet.setColumnView(3, 25);

       List<Elements> elementsList = this.elementsManager.getObjectElementsByEduTypeAndQuestionnaireType(finalUsersList.get(0).getEducationType().getId(),     this.getQuestionnaireByFinalUsersType(finalUsersList.get(0).getFinalUsersType().getId()));

       Collections.sort(elementsList, new Comparator<Elements>(){

           public int compare(Elements o1, Elements o2) {

             for(int i=0; i< ( o1.getItemNO().length()>o2.getItemNO().length()?  o2.getItemNO().length(): o1.getItemNO().length());i++){
                  if (CommonFun.isNumberic(o1.getItemNO().substring(0, o1.getItemNO().length()>3? 4: o1.getItemNO().length()-1)) && !CommonFun.isNumberic(o2.getItemNO().substring(0, o2.getItemNO().length()>3? 4: o2.getItemNO().length()-1))){
                 return 1;
                  }
                 if (!CommonFun.isNumberic(o1.getItemNO().substring(0, o1.getItemNO().length()>3? 4: o1.getItemNO().length()-1)) && CommonFun.isNumberic(o2.getItemNO().substring(0,o2.getItemNO().length()>3? 4:o2.getItemNO().length()-1))){ 
                 return -1;
                 }  
                if ( o1.getItemNO().charAt(i)!=o2.getItemNO().charAt(i) ){

                    return   o1.getItemNO().charAt(i)-o2.getItemNO().charAt(i);
                 }
           }
           return  o1.getItemNO().length()> o2.getItemNO().length()? 1:-1;
       }});

       for (Elements elements : elementsList){
           sheet.addCell(new Label(colCount++,0,this.getTitlePre(finalUsersList.get(0).getFinalUsersType().getId(), finalUsersList.get(0).getEducationType().getId())+elements.getItemNO()+elements.getItem().getStem())); 
       }

       int sheetRowCount =1;
       int sheetColCount =0;

       for(FinalUsers finalUsers : finalUsersList){

          sheetColCount =0;

          sheet.addCell(new Label(sheetColCount++,sheetRowCount,String.valueOf(sheetRowCount)));
          sheet.addCell(new Label(sheetColCount++,sheetRowCount,finalUsers.getSchool().getSchoolNumber()));
          sheet.addCell(new Label(sheetColCount++,sheetRowCount,finalUsers.getSchool().getDistrict().getDistrictNumber().toString().trim()));
          sheet.addCell(new Label(sheetColCount++,sheetRowCount,finalUsers.getSchool().getName()));

          List<AnswerLog> answerLogList = this.answerLogManager.getAnswerLogByFinalUsers(finalUsers.getId());


          Map<String,String> answerMap = new HashMap<String,String>();

          for(AnswerLog answerLog :answerLogList ){
             if (answerLog.getOptionsId() != null)
             {
                answerMap.put(answerLog.getElement().getItemNO(), this.getOptionsAnswer(answerLog.getOptionsId()));
             }else if (answerLog.getBlanks()!= null){

                answerMap.put(answerLog.getElement().getItemNO(), answerLog.getBlanks());
             }else{

                answerMap.put(answerLog.getElement().getItemNO(), answerLog.getSubjectiveItemContent());  
             }   
          }
          for (Elements elements : elementsList){

             sheet.addCell(new Label(sheetColCount++,sheetRowCount,null==answerMap.get(elements.getItemNO())?"0":answerMap.get(elements.getItemNO())));

          }

         sheetRowCount++; 
       }

       book.write();
       book.close();

     } catch (IOException e) {
        // TODO Auto-generated catch block
        e.printStackTrace();
     } catch (RowsExceededException e) {
      // TODO Auto-generated catch block
      e.printStackTrace();
     } catch (WriteException e) {

      // TODO Auto-generated catch block
      e.printStackTrace();
   }

} }

Answer 1

Somewhere you are creating objects and retaining reachable references to them.

You are probably adding objects to a collection and never removing them... so the collection just grows until you run out of memory. As you get close to the maximum heap size the garbage collector is over-taxed to find free memory to reorganize things and the program get exponentially slower the closer you get to the limit.

Also, it's possible you are forgetting to close some object(s) each time through, such as your workbook.

Answer 2

I highly suspect that there is a memory leak on your program. Use profiler like JProfiler or YourKit to detect memory leak. Maybe you forgot to close the WritableWorkbook?

http://jexcelapi.sourceforge.net/resources/javadocs/current/docs/jxl/write/WritableWorkbook.html

Answer 3

When your application is running out of heap space, it will take more and more time in the GC trying to reclaim space before eventually giving up and throwing an OutOfMemoryError. I recommend doing the following:

• add the -XX:+UseGCOverheadLimit JVM option to cause the JVM to fail earlier when it is running out of memory.

• use a memory profiler to look for possible memory leaks

• if you cannot find any leaks, just increase the heap size.

If you still get slowdown effects with a bigger heap, the problem may be something to do with the algorithms being used. In that case, you need to use an execution profiler to figure out where your application is spending most of its time.

[Theory: If your addAnswerRow is being called repeatedly, the problem could be related to repeatedly opening an Xcel spreadsheet file that is getting larger each with each addAnswerRow call. It is possible that each time you open the file, it is being loaded into memory in its entirety.]

Answer 4

Use the -verbose:gc JVM option to easily check if the slowdown is caused by GC thrashing.

Answer 5

Increasing the heap size might help. You can try setting the min and max heap size by including -Xms and -Xmx arguments. The following command will set min heap size as 512 MB and max heap as 1024 MB.

java -Xms512m -Xmx1024m MyProgram