Help! my zimbra server is crashing with a rlimit error

Discuss your pilot or production implementation with other Zimbra admins or our engineers.
kechols
Advanced member
Advanced member
Posts: 72
Joined: Fri Sep 12, 2014 10:11 pm

Help! my zimbra server is crashing with a rlimit error

Postby kechols » Thu Mar 01, 2007 5:37 pm

I'm running Suse Enterprise 9 SP3 and Zimbra 3.0.1GA (yes I know it's old, they won't let me upgrade until it's stable again).

My message logs show this error



Mar 1 15:18:35 mailserv6 kernel: open files rlimit 1024 reached for uid 1001 pid 14156

Mar 1 15:18:35 mailserv6 kernel: open files rlimit 1024 reached for uid 1001 pid 14408

Mar 1 15:18:35 mailserv6 kernel: open files rlimit 1024 reached for uid 1001 pid 14408



Then my loads jump from no higher than 15 - 25 (peak) to 85+ immediately

and I can't run more than one command at a console without losing access to that shell. Thus forcing me to press the reset button to get control of the server back.
I've determined that uid 1001 is the zimbra user, once this happens I get repeated auth failures and a few minutes later
auth failed: curl_easy_perform: error(7): Connect failed


starts showing up in my logs.
Has anyone seen something like this ? I've checked and it looks like the limit should be 10240 files for zimbra instead of 1024. So I'm not sure why this keeps happening, it's happened twice now.
If it helps I'm running zimbra over an IBM ds300 SAN with 4 146 GB u320 drives (firmware on the drives and SAN are current per IBM). I'll be happy to provide any information I can, just ask and I'll get it posted asap.

this seems to be happening every couple of days now.
Thanks


marcmac
Elite member
Elite member
Posts: 2091
Joined: Fri Sep 12, 2014 9:53 pm

Help! my zimbra server is crashing with a rlimit error

Postby marcmac » Thu Mar 01, 2007 5:46 pm

on rh/fedora, we set the hard limit for open files to 10000 in /etc/security/limits.conf, then do ulimit -n 10000 in the zimbra environment. Do the analagous on suse, and you should be fine.
kechols
Advanced member
Advanced member
Posts: 72
Joined: Fri Sep 12, 2014 10:11 pm

Help! my zimbra server is crashing with a rlimit error

Postby kechols » Thu Mar 01, 2007 5:48 pm

Would this be a one time setting or do I need to add this to a boot/init script?
kechols
Advanced member
Advanced member
Posts: 72
Joined: Fri Sep 12, 2014 10:11 pm

Help! my zimbra server is crashing with a rlimit error

Postby kechols » Fri Mar 02, 2007 3:35 pm

So i ran ulimit -n 10000 this morning, and verified the following entries in my /etc/security/limits.conf


zimbra soft nofile 10000

zimbra hard nofile 10000


And the server just crashed again with another rlimit error. This time loads jumped up to 120. After rebooting, ulimit shows:


zimbra@mailserv6:~> ulimit -a

core file size (blocks, -c) 0

data seg size (kbytes, -d) unlimited

file size (blocks, -f) unlimited

max locked memory (kbytes, -l) unlimited

max memory size (kbytes, -m) unlimited

open files (-n) 1024

pipe size (512 bytes, -p) 8

stack size (kbytes, -s) unlimited

cpu time (seconds, -t) unlimited

max user processes (-u) 32251

virtual memory (kbytes, -v) unlimited


I then do the following:



mailserv6:~ # ulimit -n 10000

mailserv6:~ # ulimit -a

core file size (blocks, -c) 0

data seg size (kbytes, -d) unlimited

file size (blocks, -f) unlimited

max locked memory (kbytes, -l) unlimited

max memory size (kbytes, -m) unlimited

open files (-n) 10000

pipe size (512 bytes, -p) 8

stack size (kbytes, -s) unlimited

cpu time (seconds, -t) unlimited

max user processes (-u) 32251

virtual memory (kbytes, -v) unlimited


is there something I need to do to make these settings stick between reboots ? How about to make these take effect immediately? should it be run before zimbra is started everytime?

Thanks.
marcmac
Elite member
Elite member
Posts: 2091
Joined: Fri Sep 12, 2014 9:53 pm

Help! my zimbra server is crashing with a rlimit error

Postby marcmac » Fri Mar 02, 2007 4:01 pm

the ulimit -n is in zimbra's login shell, too, so the startup scripts should get the change.
Have you changed the .bashrc? Try adding something to the zmcontrol script to output the ulimit -a data to a file, then reboot the machine and check the file. IF the file has 1024, not 10000, then SuSE is doing something you don't want it to.
kechols
Advanced member
Advanced member
Posts: 72
Joined: Fri Sep 12, 2014 10:11 pm

Help! my zimbra server is crashing with a rlimit error

Postby kechols » Fri Mar 02, 2007 4:13 pm

at the bottom of zimbra's .bashrc is

 ulimit -n 10000 > /dev/null 2>&1 


think i'd do better to place it at the top of the zimbra startup script?

Return to “Administrators”

Who is online

Users browsing this forum: Google [Bot] and 21 guests