ActiveMQ 5.2.0 + REST + HTTP POST = java.lang.OutOfMemoryError

Question

First off, I am a newbie when it comes to JMS & ActiveMQ.

I have been looking into a messaging solution to serve as middleware for a message producer that will insert XML messages into a queue via HTTP POST. The producer is an existing system written in C++ that cannot be modified (so Java and the C++ API are out).

Using the "demo" examples and some trial and error, I have cobbled together a working example of what I want to do (on a windows box).

The web.xml I configured in a test directory under "webapps" specifies that the HTTP POST messages received from the producer are to be handled by the MessageServlet.

I added a line for the text app in "activemq.xml" ('ow' is the test app dir):

I created a test script to "insert" messages into the queue which works well.

The problem I am running into is that it as I continue to insert messages via REST/HTTP POST, the memory consumption and thread count used by ActiveMQ continues to rise (It happens when I have timely consumers as well as slow or non-existent consumers).

When memory consumption gets around 250MB's and the thread count exceeds 5000 (as shown in windows task manager), ActiveMQ crashes and I see this in the log:

Exception in thread "ActiveMQ Transport Initiator: vm://localhost#3564" java.lang.OutOfMemoryError: unable to create new native thread

It is as if Jetty is spawning a new thread to handle each HTTP POST and the thread never dies.

I did look at this page:

http://activemq.apache.org/javalangoutofmemory.html

and tried but that didn't fix the problem (although I didn't fully understand the implications of the change either).

Does anyone have any ideas?

Thanks!

• Bruce Loth

PS - I included the "test message producer" python script below for what it is worth. I created batches of 100 messages and continued to run the script manually from the command line while watching the memory consumption and thread count of ActiveMQ in task manager.

def foo():
    import httplib, urllib

    body = "<?xml version='1.0' encoding='UTF-8'?>\n \
    <ROOT>\n \
        [snip: xml deleted to save space]
    </ROOT>"

    headers = {"content-type": "text/xml",
               "content-length": str(len(body))}

    conn = httplib.HTTPConnection("127.0.0.1:8161")
    conn.request("POST", "/ow/message/RDRCP_Inbox?type=queue", body, headers)
    response = conn.getresponse()
    print response.status, response.reason
    data = response.read()
    conn.close()
## end method definition


## Begin test code
count = 0;

while(count < 100):
    # Test with batches of 100 msgs
    count += 1
    foo()

Answer 1

The error is not directly caused by ActiveMQ but by the Java Runtime. Take a look here:

http://activemq.apache.org/javalangoutofmemory.html

how you can up your memory for the Java HEAP. There is also interessting stuff about WHY this happens and what you might do to prevent it. ActiveMQ is pretty good but needs some customizing here and there in the config files.

Answer 2

Bruce would you share this MessageServelet code? I am in the same boat trying to use a REST producer and after aprox. 500 messages BOOM error..