We are having some problems with a JBoss 3.2.6 instance running out of memory (java.lang.OutOfMemoryError) with an error message about not being able to create native threads. First of I though it was memory for the instance so I added a bunch of more memory that that didn’t seem to fix the problem. I poked a little around on Google and found JBoss issue 1369 which is basically what we are experiencing. For the moment I have tried disabling the UIL2 (Universal Invocation Layer 2) and just leaving the JVM Invocation Layer (jvm-il) to see if that solves the problem. Only using the jvm-il should solve the need to using separate threads etc.
We are crossing our fingers.