OutOfMemoryError with new java?

simonpg · Post by **simonpg** » Mon Dec 19, 2005 8:10 pm

I have /opt/zimbra/java -> /usr/lib/j2sdk1.5-sun. I run zmprov normally and get: zimbra@xanadu:~/bin$ zmprov

Exception in thread "main" java.lang.OutOfMemoryError

at java.util.zip.ZipFile.open(Native Method)

at java.util.zip.ZipFile.(ZipFile.java:203)

at java.util.jar.JarFile.(JarFile.java:132)

at java.util.jar.JarFile.(JarFile.java:70)

at sun.misc.URLClassPath$JarLoader.getJarFile(URLClassPath.java:579)

at sun.misc.URLClassPath$JarLoader.(URLClassPath.java:546)

at sun.misc.URLClassPath$3.run(URLClassPath.java:324)

at java.security.AccessController.doPrivileged(Native Method)

at sun.misc.URLClassPath.getLoader(URLClassPath.java:313)

at sun.misc.URLClassPath.getLoader(URLClassPath.java:290)

at sun.misc.URLClassPath.getResource(URLClassPath.java:160)

at java.net.URLClassLoader$1.run(URLClassLoader.java:192)

at java.security.AccessController.doPrivileged(Native Method)

at java.net.URLClassLoader.findClass(URLClassLoader.java:188)

at java.lang.ClassLoader.loadClass(ClassLoader.java:306)

at java.lang.ClassLoader.loadClass(ClassLoader.java:299)

at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:268)

at java.lang.ClassLoader.loadClass(ClassLoader.java:251)

at java.lang.ClassLoader.loadClassInternal(ClassLoader.java:319)
If I run zmlocalconfig, all is fine:
zimbra@xanadu:~/bin$ zmlocalconfig

ldap_host = ldap.thesimonet.org

ldap_is_master = false

ldap_master_url =

ldap_port =

ldap_root_password = *

ldap_url =

logger_mysql_bind_address = localhost

logger_mysql_data_directory = ${zimbra_home}/logger/db/data

...
If I run com.zimbra.cs.localconfig.Main using the opened up defines that zmprov uses, I get the same error:
zimbra@xanadu:~/bin$ java -client -Dzimbra.home=/opt/zimbra -Djava.library.path=/opt/zimbra/lib -Djava.ext.dirs=/opt/zimbra/java/jre/lib/ext:/opt/zimbra/lib com.zimbra.cs.localconfig.Main

Exception in thread "main" java.lang.OutOfMemoryError

at java.util.zip.ZipFile.open(Native Method)

at java.util.zip.ZipFile.(ZipFile.java:203)

at java.util.jar.JarFile.(JarFile.java:132)

at java.util.jar.JarFile.(JarFile.java:70)

at sun.misc.URLClassPath$JarLoader.getJarFile(URLClassPath.java:579)

at sun.misc.URLClassPath$JarLoader.(URLClassPath.java:546)

at sun.misc.URLClassPath$3.run(URLClassPath.java:324)

at java.security.AccessController.doPrivileged(Native Method)

at sun.misc.URLClassPath.getLoader(URLClassPath.java:313)

at sun.misc.URLClassPath.getLoader(URLClassPath.java:290)

at sun.misc.URLClassPath.getResource(URLClassPath.java:160)

at java.net.URLClassLoader$1.run(URLClassLoader.java:192)

at java.security.AccessController.doPrivileged(Native Method)

at java.net.URLClassLoader.findClass(URLClassLoader.java:188)

at java.lang.ClassLoader.loadClass(ClassLoader.java:306)

at java.lang.ClassLoader.loadClass(ClassLoader.java:299)

at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:268)

at java.lang.ClassLoader.loadClass(ClassLoader.java:251)

at java.lang.ClassLoader.loadClassInternal(ClassLoader.java:319)
Any pointers would be appreciated.

17629anand · Post by **17629anand** » Mon Dec 19, 2005 8:14 pm

Add -verbose:load to the Java command line. Or run it under strace. It would be interesting to see which jar file is causing the VM to trip over itself.

17629anand · Post by **17629anand** » Mon Dec 19, 2005 8:15 pm

btw what is your OS, hardware, zimbra version etc?

simonpg · Post by **simonpg** » Tue Dec 20, 2005 12:35 pm

If you would like the full dumps, let me know where to send them. The following line worked: /opt/zimbra/java/bin/java -client -Dzimbra.home=/opt/zimbra -cp /opt/zimbra/lib/zimbrastore.jar:/opt/zimbra/lib/commons-logging.jar:/opt/zimbra/lib/commons-cli-2.0.jar:/opt/zimbra/lib/dom4j-1.5.jar:/opt/zimbra/lib/log4j-1.2.8.jar:/opt/zimbra/lib/commons-httpclient-2.0.1.jar -verbose com.zimbra.cs.localconfig.Main (Note: 'man java' showed no -verbose:load, so I choose just -verbose). It loaded a few more classes:
Good load:

[Loaded sun.misc.ExtensionDependency from /usr/lib/j2sdk1.5-sun/jre/lib/rt.jar]

[Loaded java.security.PrivilegedActionException from /usr/lib/j2sdk1.5-sun/jre/lib/rt.jar]

[Loaded java.util.zip.ZipEntry from /usr/lib/j2sdk1.5-sun/jre/lib/rt.jar]

Bad load:

[Loaded sun.misc.ExtensionDependency from /usr/lib/j2sdk1.5-sun/jre/lib/rt.jar]

[Loaded java.util.zip.ZipEntry from /usr/lib/j2sdk1.5-sun/jre/lib/rt.jar]

Good load:

[Loaded java.lang.Math from /usr/lib/j2sdk1.5-sun/jre/lib/rt.jar]

[Loaded sun.misc.Resource from /usr/lib/j2sdk1.5-sun/jre/lib/rt.jar]

[Loaded sun.misc.URLClassPath$JarLoader$1 from /usr/lib/j2sdk1.5-sun/jre/lib/rt.jar]

[Loaded java.lang.Package from /usr/lib/j2sdk1.5-sun/jre/lib/rt.jar]

[Loaded java.util.jar.Manifest from /usr/lib/j2sdk1.5-sun/jre/lib/rt.jar]

Bad load:

[Loaded java.lang.Math from /usr/lib/j2sdk1.5-sun/jre/lib/rt.jar]

[Loaded java.util.jar.Manifest from /usr/lib/j2sdk1.5-sun/jre/lib/rt.jar]

The following line did not work: /opt/zimbra/java/bin/java -client -Dzimbra.home=/opt/zimbra -Djava.library.path=/opt/zimbra/lib -Djava.ext.dirs=/opt/zimbra/java/jre/lib/ext:/opt/zimbra/lib -verbose com.zimbra.cs.localconfig.Main The class loading really started to differ here:
Good load:

[Loaded sun.misc.ASCIICaseInsensitiveComparator from /usr/lib/j2sdk1.5-sun/jre/lib/rt.jar]

[Loaded java.util.jar.JarVerifier from /usr/lib/j2sdk1.5-sun/jre/lib/rt.jar]

[Loaded java.io.ByteArrayOutputStream from /usr/lib/j2sdk1.5-sun/jre/lib/rt.jar]

[Loaded sun.security.util.ManifestEntryVerifier from /usr/lib/j2sdk1.5-sun/jre/lib/rt.jar]

[Loaded java.security.Provider from /usr/lib/j2sdk1.5-sun/jre/lib/rt.jar]

[Loaded sun.security.provider.Sun from /usr/lib/j2sdk1.5-sun/jre/lib/rt.jar]

[Loaded java.security.Provider$ServiceKey from /usr/lib/j2sdk1.5-sun/jre/lib/rt.jar]

[Loaded java.security.Provider$EngineDescription from /usr/lib/j2sdk1.5-sun/jre/lib/rt.jar]

Bad load:

[Loaded sun.misc.ASCIICaseInsensitiveComparator from /usr/lib/j2sdk1.5-sun/jre/lib/rt.jar]

[Loaded java.util.jar.JarVerifier from /usr/lib/j2sdk1.5-sun/jre/lib/rt.jar]

[Loaded java.io.ByteArrayOutputStream from /usr/lib/j2sdk1.5-sun/jre/lib/rt.jar]

[Loaded java.io.IOException from /usr/lib/j2sdk1.5-sun/jre/lib/rt.jar]

[Loaded java.io.FileNotFoundException from /usr/lib/j2sdk1.5-sun/jre/lib/rt.jar]

[Loaded java.security.PrivilegedActionException from /usr/lib/j2sdk1.5-sun/jre/lib/rt.jar]

[Loaded sun.reflect.ClassFileConstants from /usr/lib/j2sdk1.5-sun/jre/lib/rt.jar]

[Loaded sun.reflect.AccessorGenerator from /usr/lib/j2sdk1.5-sun/jre/lib/rt.jar]
As to particulars, Debian testing on AMD64 using ZCS 3.0 M2 source. Java provided using jdk-1_5_0_06-linux-amd64.bin. The following mod to get_plat_tag.sh allows ZimbraBuild to work very well.

grep "testing" /etc/debian_version > /dev/null 2>&1

if [ $? = 0 ]; then

echo "DEBIAN3.1"

exit 0

fi
Kudos to the Zimbra team for an already amazing build system!

17629anand · Post by **17629anand** » Tue Dec 20, 2005 2:04 pm

-verbose:load output looks ok - atleast the part you posted.
Don't have 64 bit JVM here to try.
How how strace to see what few files the JVM touches before the error?
Is it the case that 32-bit JVM from zimbra works ok but 64-bit JVM does not? In that case I would look into ZimbraNative - a little bit of our native code and see if that needs recompiling.

simonpg · Post by **simonpg** » Tue Dec 20, 2005 5:56 pm

Found that "attach files" button

so I have attached straces and a build output showing that ZimbraNative gets built along with everything else. Then did ./install.sh and it still blows up at the same spot. My environment is "pure64", so no 32-bit. I do have other machines; are you suggesting that I install jdk-1_5_0_06-linux-i586.bin and see if that works?
zimbra.zip

17629anand · Post by **17629anand** » Tue Dec 20, 2005 6:04 pm

yes, use the 32 bit JDK.

simonpg · Post by **simonpg** » Wed Dec 21, 2005 6:42 pm

Installed in a 32-bit Debian testing environment and it works great; I get a prov> prompt. I used the CVS version because it was easier to download. Should I try the CVS version on the 64-bit system? Or is it Something Else? Since I have seen another post with the same problem, it would appear to be a 64-bit issue.

17629anand · Post by **17629anand** » Wed Dec 21, 2005 6:56 pm

don't have any 64 bit installs handy. Please let us know if you narrow this down.

simonpg · Post by **simonpg** » Thu Dec 22, 2005 4:09 pm

So I added -Xrunhprof to zmjava to start debugging and it works great! So, other than slow startup and the creation of a large debugging file, 64-bit works just fine. This kind of majic typically points to an internal problem with the vendor supplied code. I haven't seen anything after googling about. Any ideas?