JDBC (mysql) saves queries in heap memory

Question

Hi,

I am trying to run Hibernate Search on database on a table with 12,500,000 records.

On startup, Hibernate Search queries the database to obtain the schema information.

The problem is that I get OutOfMemory: heap size exception.

I read around, and found out that JDBC connector of MySQL puts queries on JAVA heap memory and it's a bug.

Is there a workarround for this bug?

I am using 5.1 connector.

Please see my post to Hibernate forum where they write that there is a bug in MySQL Hibernate Search Forum

The question is how can I work arround it?

Answer 1

It isn't necessarily a bug if you run out of memory, you might just not have enough memory allocated to Java. Heap space is where data in Objects is stored, and the JDBC connection is an Object. If the code is correct, and you just need more heap space, increase the size of the heap.

java -Xms<initial heap size> -Xmx<maximum heap size>

If this doesn't solve it, odds are the code has a deeper issue; something's eating memory that shouldn't be. Find the problem before continually adding more memory; otherwise you're just putting a temporary bandage on the issue, and the bandage probably won't last long.

Answer 2

Database metadata doesn't need to much memory. Start the application with a profiler and you will see right away how memory is used.

Answer 3

I found the solution.

In MySQL I had to add to connection string:

&useServerPrepStmts=true&useCursorFetch=true

This option enables streaming results by default.