should i activate c3p0 statement pooling?

Question

Hi,

we are running java6/hibernate/c3p0/postgresql stack. Our JDBC Driver is 8.4-701.jdbc3

I have a few questions about Prepared Statements. I have read excellent document about Prepared Statements

But i still have a question how to configure c3p0 with postgresql.

At the moment we have

 c3p0.maxStatements = 0
 c3p0.maxStatementsPerConnection  =   0

In my understanding the prepared statements and statement pooling are two different things:

Our hibernate stack uses prepared statements. Postgresql is caching the execution plan. Next time the same statement is used, postgresql reuses the execution plan. This saves time planning statements inside DB.

Additionally c3p0 can cache java instances of "java.sql.PreparedStatement" which means it is caching the java object. So when using
c3p0.maxStatementsPerConnection = 100 it caches at most 100 different
objects. It saves time on creating objects, but this has nothing to do with the postgresql database and its prepared statements.

Right?

As we use about 100 different statements I would set c3p0.maxStatementsPerConnection = 100

But the c3p0 docs say in c3p0 known shortcomings

The overhead of Statement pooling is too high. For drivers that do not perform significant preprocessing of PreparedStatements, the pooling overhead outweighs any savings. Statement pooling is thus turned off by default. If your driver does preprocess PreparedStatements, especially if it does so via IPC with the RDBMS, you will probably see a significant performance gain by turning Statement pooling on. (Do this by setting the configuration property maxStatements or maxStatementsPerConnection to a value greater than zero.).

So: Is it reasonable to activate maxStatementsPerConnection with c3p0 and Postgresql? Is there a real benefit activating it?

kind regards Janning

Answer 1

I don't remember offhand if Hibernate actually stores PreparedStatement instances itself, or relies on the connection provider to reuse them. (A quick scan of BatcherImpl suggests it reuses the last PreparedStatement if executing the same SQL multiple times in a row)

I think the point that the c3p0 documentation is trying to make is that for many JDBC drivers, a PreparedStatement isn't useful: some drivers will end up simply splicing the parameters in client-side and then passing the built SQL statement to the database anyway. For these drivers, PreparedStatements are no advantage at all, and any effort to reuse them is wasted. (The Postgresql JDBC FAQ says this was the case for Postgresql before sever protocol version 3 and there is more detailed information in the documentation).

For drivers that do handle PreparedStatements usefully, it's still likely necessary to actually reuse PreparedStatement instances to get any benefit. For example if the driver implements:

• Connection.prepareStatement(sql) - create a server-side statement
• PreparedStatement.execute(..) etc - execute that server-side statement
• PreparedStatement.close() - deallocate the server-side statement

Given this, if the application always opens a prepared statement, executes it once and then closes it again, there's still no benefit; in fact, it might be worse since there are now potentially more round-trips. So the application needs to hang on to PreparedStatement instances. Of course, this leads to another problem: if the application hangs on to too many, and each server-side statement consumes some resources, then this can lead to server-side issues. In the case where someone is using JDBC directly, this might be managed by hand- some statements are known to be reusable and hence are prepared; some aren't and just use transient Statement instances instead. (This is skipping over the other benefit of prepared statements: handling argument escaping)

So this is why c3p0 and other connection pools also have prepared statement caches- it allows application code to avoid dealing with all this. The statements are usually kept in some limited LRU pool, so common statements reuse a PreparedStatement instance.

The final pieces of the puzzle are that JDBC drivers may themselves decide to be clever and do this; and servers may themselves also decide to be clever and detect a client submitting a statement that is structurally similar to a previous one.

Given that Hibernate doesn't itself keep a cache of PreparedStatement instances, you need to have c3p0 do that in order to get the benefit of them. (Which should be reduced overhead for common statements due to reusing cached plans). If c3p0 doesn't cache prepared statements, then the driver will just see the application preparing a statement, executing it, and then closing it again. Looks like the JDBC driver has a "threshold" setting for avoiding the prepare/execute server overhead in the case where the application always does this. So, yes, you need to have c3p0 do statement caching.

Hope that helps, sorry it's a bit long winded. The answer is yes.

Answer 2

Remember that statements have to be cached per connection which will mean you're going to have to consume quite a chunk of memory and it will take a long time before you'll see any benefit. So if you set it to use 100 statements to be cached, that's actually 100*number of connections or else 100/no of connections but you will still need to take quite some time until your cache will have any meaningful effect.