Strategies for performance optimizations on an inherited EJB3 application

Question

I was asked to have a look at a legacy EJB3 application with significant performance problems. The original author is not available anymore so all I've got is the source code and some user comments regarding the unacceptable performance. My personal EJB3 skill are pretty basic, I can read and understand the annotated code but that's all until know.

The server has a database, several EJB3 beans (JPA) and a few stateless beans just to allow CRUD on 4..5 domain objects for remote clients. The client itself is a java application. Just a few are connected to the server in parallel. From the user comments I learned that

• the client/server app performed well in a LAN
• the app was practically unusable on a WAN (1MBit or more) because read and update operations took much too long (up to several minutes)

I've seen one potential problem - on all EJB, all relations have been defined with the fetching strategy FetchType.EAGER. Would that explain the performance issues for read operations, is it advisable to start tuning with the fetching strategies?

But that would not explain performance issues on update operations, or would it? Update is handled by an EntityManager, the client just passes the domain object to the manager bean and persisting is done with nothing but manager.persist(obj). Maybe the domain objects that are sent to the server are just too big (maybe a side effect of the EAGER strategy).

So my actual theory is that too many bytes are sent over a rather slow network and I should look at reducing the size of result sets.

From your experience, what are the typical and most common coding errors that lead to performance issues on CRUD operations, where should I start investigating/optimizing?

Answer 1

On all EJB, all relations have been defined with the fetching strategy FetchType.EAGER. Would that explain the performance issues for read operations?

Depending on the relations betweens classes, you might be fetching much more (the whole database?) than actually wanted when retrieving entities?

is it advisable to start tuning with the fetching strategies?

I can't say that making all relations EAGER is a very standard approach. To my experience, you usually keep them lazy and use "Fetch Joins" (a type of join allowing to fetch an association) when you want to eager load an association for a given use case.

But that would not explain performance issues on update operations, or would it?

It could. I mean, if the app is retrieving a big fat object graph when reading and then sending the same fat object graph back to update just the root entity, there might be a performance penalty. But it's kinda weird that the code is using em.persist(Object) to update entities.

From your experience, what are the typical and most common coding errors that lead to performance issues on CRUD operations, where should I start investigating/optimizing?

The obvious ones include:

1. Retrieving more data than required
2. N+1 requests problems (bad fetching strategy)
3. Poorly written JPQL queries
4. Non appropriate inheritance strategies
5. Unnecessary database hits (i.e. lack of caching)

I would start with writing some integration tests or functional tests before touching anything to guarantee you won't change the functional behavior. Then, I would activate SQL logging and start to look at the generated SQL for the major use cases and work on the above points.

Answer 2

From DBA position.

From your experience, what are the typical and most common coding errors that lead to performance issues on CRUD operations, where should I start investigating/optimizing?

1. Turn off caching
2. Enable sql logging Ejb3/Hibernate generates by default a lots of extremely stupid queries.
3. Now You see what I mean.
4. Change FetchType.EAGER to FetchType.LAZY
5. Say "no" for big business logic between em.find em.persist
6. Use ehcache http://ehcache.org/
7. Turn on entity cache
8. If You can, make primary keys immutable ( @Column(updatable = false, ...)
9. Turn on query cache

Never ever use Hibernate if You want big performance: http://www.google.com/search?q=hibernate+sucks

Answer 3

In the question, and in the other answers, I'm hearing a lot of "might"s and "maybe"s.

First find out what's going on. If you haven't done that, we're all just poking in the dark.

I'm no expert on this kind of system, but this method works on any language or OS.

When you find out what's making it take too long, why don't you summarize it here? I'm especially interested to know if it was something that might have been guessed.