I would suggest before you go down the road of customizing BlazeDS to support NIO that you profile your application and verify were the hotspots are. Have you verified that it is the BlazeDS networking stack that is causing lost messages? Have you profiled your code to see if there are optimizations that can be done to better optimize message handling?
Some actually contend the Java NIO doesn't actually improve through-put - http://paultyma.blogspot.com/2008/03/writing-java-multithreaded-servers.html
I say this because BlazeDS does not support NIO only the commercial version of the server does - LCDS. What LCDS does actually set-up it's own NIO sockets and manages requests through these connections, bypassing the standard servlet stack. To get NIO support Yakov said "To support thousands concurrent users you also need to customize networking layer of BlazeDS" I would be willing to guess this customized networking layer is not production ready and is more of a prototype because it is extremely difficult to reliably customize the networking layer of any server.