zimbra dies

Discuss your pilot or production implementation with other Zimbra admins or our engineers.
Post Reply
jakebriems
Posts: 13
Joined: Fri Sep 12, 2014 10:08 pm

zimbra dies

Post by jakebriems »

I get a call from a user today saying that email is down:
[zimbra@[HOSTNAME] root]$ zmcontrol status

Host [HOSTNAME].[DOMAIN].com

antispam Running

antivirus Running

ldap Running

logger Running

mailbox Stopped

mta Running

snmp Running

spell Running

[zimbra@[HOSTNAME] root]$ zmcontrol start mailbox

Host [HOSTNAME].[DOMAIN].com

Starting antispam...Done.

Starting antivirus...Done.

Starting logger...Done.

Starting mailbox...Done.

Starting mta...FAILED

getService: sasl

getService: webxml

getService: perdition

DO: /opt/zimbra/postfix/sbin/postconf -e content_filter='smtp-amavis:[127.0.0.1] :10024'

DO: /opt/zimbra/postfix/sbin/postconf -e myhostname='[HOSTNAME].[DOMAIN].com'

DO: /opt/zimbra/postfix/sbin/postconf -e recipient_delimiter=''

DO: /opt/zimbra/postfix/sbin/postconf -e smtpd_sasl_auth_enable='yes'

DO: /opt/zimbra/postfix/sbin/postconf -e smtpd_tls_auth_only='yes'

DO: /opt/zimbra/postfix/sbin/postconf -e disable_dns_lookups='no'

DO: /opt/zimbra/postfix/sbin/postconf -e message_size_limit='10240000'

DO: /opt/zimbra/postfix/sbin/postconf -e relayhost=''

DO: /opt/zimbra/postfix/sbin/postconf -e smtpd_recipient_restrictions='reject_no n_fqdn_recipient, permit_sasl_authenticated, permit_mynetworks, reject_invalid_h ostname, reject_non_fqdn_sender, reject_unauth_destination, permit'

postsuper: Requeued: 16 messages

saslauthd[6281] :detach_tty : Cannot start saslauthd

saslauthd[6281] :detach_tty : Another instance of saslauthd is currently ru nning


Starting snmp...Done.

Starting spell...Done.

[zimbra@[HOSTNAME] root]$ zmcontrol status

Host [HOSTNAME].[DOMAIN].com

antispam Running

antivirus Running

ldap Running

logger Running

mailbox Running

mta Running

snmp Running

spell Running


Any ideas what's going on? I will check the logs.
jb
bobby
Outstanding Member
Outstanding Member
Posts: 515
Joined: Fri Sep 12, 2014 10:01 pm

zimbra dies

Post by bobby »

if mailbox stops again, try running this to find out exactly what part isn't running (though you may need to post the output here for interpretation):
sh -x ~/bin/zmmailboxctl status
a quick guess is that tomcat died. are there any clues in stacktraces or catalina.outs from whenever mailbox became stopped? check in /opt/zimbra/tomcat/logs/
you can get see when mailbox stopped by checking in /var/log/zimbra.log. the zmcontrol status is logged there pretty frequently
scott_mcneilage
Posts: 2
Joined: Fri Sep 12, 2014 10:10 pm

zimbra dies

Post by scott_mcneilage »

I'm getting this same error and symptoms.

Everything was installed and running OK until I did zmcontrol shutdown and startup.

Rebooted the server with same result.

Reinstalled zimbra and it rebooted ok

Added a few domains and accounts and it rebooted ok

Changed some POP settings (to allow clear text) and restart / reboot failed.
Following Zimbra Log and Tomcat Catalina.out:

---------------------------

2006-05-04 10:53:42,285 INFO [main] [] soap - Servlet SoapServlet starting up

2006-05-04 10:53:42,336 INFO [main] [] soap - Adding service AccountService to SoapServlet

2006-05-04 10:53:42,483 INFO [main] [] soap - Adding service MailService to SoapServlet

2006-05-04 10:53:42,645 INFO [main] [] soap - Adding service IMService to SoapServlet

2006-05-04 10:53:42,655 INFO [main] [] misc - version=3.0.1_GA_160.RHEL4 release=20060216172310 builddate=20060216-1726 buildhost=build.liquidsys.com

2006-05-04 10:53:42,656 INFO [main] [] misc - LANG environment is set to: en_AU.UTF-8

2006-05-04 10:53:42,883 INFO [main] [] DbPool - Setting mysql connector property: maxActive=10

2006-05-04 10:53:42,884 INFO [main] [] misc - Setting mysql connection pool size to 10

2006-05-04 10:53:43,874 INFO [main] [] extensions - Loading extensions from /opt/zimbra/lib/ext

2006-05-04 10:53:43,893 INFO [main] [] extensions - extension com.zimbra.clam.ClamScanner found in /opt/zimbra/lib/ext/clamscanner/clamscanner.jar

2006-05-04 10:53:43,894 INFO [main] [] extensions - Initializing extensions

2006-05-04 10:53:43,904 INFO [main] [] ClamScanner - attachment scan is disabled

2006-05-04 10:53:44,162 INFO [main] [] extensions - Initialized extension clamscanner: com.zimbra.clam.ClamScanner@com.zimbra.cs.extension.ZimbraExtensionClassLoader@119e583

2006-05-04 10:53:44,228 INFO [IndexWritersSweeper] [] MailboxIndex - IndexWritersSweeper thread starting

2006-05-04 10:53:44,361 INFO [main] [] RedoLogManager - Starting pre-startup crash recovery

2006-05-04 10:53:44,419 INFO [main] [] RedoPlayer - No uncommitted transactions to redo

2006-05-04 10:53:44,431 INFO [main] [] RedoLogManager - Finished pre-startup crash recovery

2006-05-04 10:53:44,452 FATAL [main] [] soap - Unable to start servlet

com.zimbra.cs.service.ServiceException: system failure: Could not bind to port=7025 bindaddr=clickmail.click.com.au ssl=false

at com.zimbra.cs.service.ServiceException.FAILURE(ServiceException.java:131)

at com.zimbra.cs.util.NetUtil.getBoundServerSocket(NetUtil.java:54)

at com.zimbra.cs.lmtpserver.LmtpServer.startupLmtpServer(LmtpServer.java:195)

at com.zimbra.cs.util.Zimbra.startup(Zimbra.java:106)

at com.zimbra.soap.SoapServlet.init(SoapServlet.java:118)

at javax.servlet.GenericServlet.init(GenericServlet.java:211)

at org.apache.catalina.core.StandardWrapper.loadServlet(StandardWrapper.java:1068)

at org.apache.catalina.core.StandardWrapper.load(StandardWrapper.java:900)

at org.apache.catalina.core.StandardContext.loadOnStartup(StandardContext.java:3823)

at org.apache.catalina.core.StandardContext.start(StandardContext.java:4087)

at org.apache.catalina.core.ContainerBase.addChildInternal(ContainerBase.java:759)

at org.apache.catalina.core.ContainerBase.addChild(ContainerBase.java:739)

at org.apache.catalina.core.StandardHost.addChild(StandardHost.java:524)

at org.apache.catalina.startup.HostConfig.deployDirectory(HostConfig.java:909)

at org.apache.catalina.startup.HostConfig.deployDirectories(HostConfig.java:872)

at org.apache.catalina.startup.HostConfig.deployApps(HostConfig.java:474)

at org.apache.catalina.startup.HostConfig.start(HostConfig.java:1106)

at org.apache.catalina.startup.HostConfig.lifecycleEvent(HostConfig.java:310)

at org.apache.catalina.util.LifecycleSupport.fireLifecycleEvent(LifecycleSupport.java:119)

at org.apache.catalina.core.ContainerBase.start(ContainerBase.java:1019)

at org.apache.catalina.core.StandardHost.start(StandardHost.java:718)

at org.apache.catalina.core.ContainerBase.start(ContainerBase.java:1011)

at org.apache.catalina.core.StandardEngine.start(StandardEngine.java:440)

at org.apache.catalina.core.StandardService.start(StandardService.java:450)

at org.apache.catalina.core.StandardServer.start(StandardServer.java:683)

at org.apache.catalina.startup.Catalina.start(Catalina.java:537)

at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)

at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)

at java.lang.reflect.Method.invoke(Method.java:585)

at org.apache.catalina.startup.Bootstrap.start(Bootstrap.java:271)

at org.apache.catalina.startup.Bootstrap.main(Bootstrap.java:409)

Caused by: java.net.BindException: Cannot assign requested address

at sun.nio.ch.Net.bind(Native Method)

at sun.nio.ch.ServerSocketChannelImpl.bind(ServerSocketChannelImpl.java:119)

at sun.nio.ch.ServerSocketAdaptor.bind(ServerSocketAdaptor.java:59)

at sun.nio.ch.ServerSocketAdaptor.bind(ServerSocketAdaptor.java:52)

at com.zimbra.cs.util.NetUtil.newBoundServerSocket(NetUtil.java:78)

at com.zimbra.cs.util.NetUtil.getBoundServerSocket(NetUtil.java:52)

... 30 more

2006-05-04 10:53:44,457 INFO [main] [] [/service] - Marking servlet SoapServlet as unavailable

2006-05-04 10:53:44,457 ERROR [main] [] [/service] - Servlet /service threw load() exception

javax.servlet.UnavailableException: system failure: Could not bind to port=7025 bindaddr=clickmail.click.com.au ssl=false

at com.zimbra.soap.SoapServlet.init(SoapServlet.java:121)

at javax.servlet.GenericServlet.init(GenericServlet.java:211)

at org.apache.catalina.core.StandardWrapper.loadServlet(StandardWrapper.java:1068)

at org.apache.catalina.core.StandardWrapper.load(StandardWrapper.java:900)

at org.apache.catalina.core.StandardContext.loadOnStartup(StandardContext.java:3823)

at org.apache.catalina.core.StandardContext.start(StandardContext.java:4087)

at org.apache.catalina.core.ContainerBase.addChildInternal(ContainerBase.java:759)

at org.apache.catalina.core.ContainerBase.addChild(ContainerBase.java:739)

at org.apache.catalina.core.StandardHost.addChild(StandardHost.java:524)

at org.apache.catalina.startup.HostConfig.deployDirectory(HostConfig.java:909)

at org.apache.catalina.startup.HostConfig.deployDirectories(HostConfig.java:872)

at org.apache.catalina.startup.HostConfig.deployApps(HostConfig.java:474)

at org.apache.catalina.startup.HostConfig.start(HostConfig.java:1106)

at org.apache.catalina.startup.HostConfig.lifecycleEvent(HostConfig.java:310)

at org.apache.catalina.util.LifecycleSupport.fireLifecycleEvent(LifecycleSupport.java:119)

at org.apache.catalina.core.ContainerBase.start(ContainerBase.java:1019)

at org.apache.catalina.core.StandardHost.start(StandardHost.java:718)

at org.apache.catalina.core.ContainerBase.start(ContainerBase.java:1011)

at org.apache.catalina.core.StandardEngine.start(StandardEngine.java:440)

at org.apache.catalina.core.StandardService.start(StandardService.java:450)

at org.apache.catalina.core.StandardServer.start(StandardServer.java:683)

at org.apache.catalina.startup.Catalina.start(Catalina.java:537)

at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)

at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)

at java.lang.reflect.Method.invoke(Method.java:585)

at org.apache.catalina.startup.Bootstrap.start(Bootstrap.java:271)

at org.apache.catalina.startup.Bootstrap.main(Bootstrap.java:409)

2006-05-04 10:53:44,466 INFO [main] [] soap - Servlet AdminServlet starting up

2006-05-04 10:53:44,578 INFO [main] [] soap - Adding service AdminService to AdminServlet

2006-05-04 10:53:44,602 INFO [main] [] soap - Adding service AccountService to AdminServlet

2006-05-04 10:53:44,602 INFO [main] [] soap - Adding service MailService to AdminServlet

2006-05-04 10:53:44,603 INFO [main] [] soap - Adding service IMService to AdminServlet

2006-05-04 10:53:44,603 INFO [main] [] misc - version=3.0.1_GA_160.RHEL4 release=20060216172310 builddate=20060216-1726 buildhost=build.liquidsys.com

2006-05-04 10:53:44,603 INFO [main] [] misc - LANG environment is set to: en_AU.UTF-8

2006-05-04 10:53:44,603 INFO [main] [] extensions - Loading extensions from /opt/zimbra/lib/ext

2006-05-04 10:53:44,604 INFO [main] [] extensions - extension com.zimbra.clam.ClamScanner found in /opt/zimbra/lib/ext/clamscanner/clamscanner.jar

2006-05-04 10:53:44,605 INFO [main] [] extensions - Initializing extensions

2006-05-04 10:53:44,605 INFO [main] [] ClamScanner - attachment scan is disabled

2006-05-04 10:53:44,605 INFO [main] [] extensions - Initialized extension clamscanner: com.zimbra.clam.ClamScanner@com.zimbra.cs.extension.ZimbraExtensionClassLoader@119e583

2006-05-04 10:53:44,613 INFO [main] [] ClamScanner - attachment scan is disabled

2006-05-04 10:53:44,613 INFO [main] [] extensions - Initialized extension clamscanner: com.zimbra.clam.ClamScanner@com.zimbra.cs.extension.ZimbraExtensionClassLoader@1a41cc7

2006-05-04 10:53:44,614 INFO [IndexWritersSweeper] [] MailboxIndex - IndexWritersSweeper thread exiting

2006-05-04 10:53:44,614 INFO [main] [] MailboxIndex - Flushing all open index writers

2006-05-04 10:53:44,615 INFO [main] [] RedoLogManager - Starting pre-startup crash recovery

2006-05-04 10:53:44,615 INFO [IndexWritersSweeper] [] MailboxIndex - IndexWritersSweeper thread starting

2006-05-04 10:53:44,619 INFO [main] [] RedoPlayer - No uncommitted transactions to redo

2006-05-04 10:53:44,629 INFO [main] [] RedoLogManager - Finished pre-startup crash recovery

2006-05-04 10:53:44,631 FATAL [main] [] soap - Unable to start servlet

com.zimbra.cs.service.ServiceException: system failure: Could not bind to port=7025 bindaddr=clickmail.click.com.au ssl=false

at com.zimbra.cs.service.ServiceException.FAILURE(ServiceException.java:131)

at com.zimbra.cs.util.NetUtil.getBoundServerSocket(NetUtil.java:54)

at com.zimbra.cs.lmtpserver.LmtpServer.startupLmtpServer(LmtpServer.java:195)

at com.zimbra.cs.util.Zimbra.startup(Zimbra.java:106)

at com.zimbra.soap.SoapServlet.init(SoapServlet.java:118)

at javax.servlet.GenericServlet.init(GenericServlet.java:211)

at org.apache.catalina.core.StandardWrapper.loadServlet(StandardWrapper.java:1068)

at org.apache.catalina.core.StandardWrapper.load(StandardWrapper.java:900)

at org.apache.catalina.core.StandardContext.loadOnStartup(StandardContext.java:3823)

at org.apache.catalina.core.StandardContext.start(StandardContext.java:4087)

at org.apache.catalina.core.ContainerBase.addChildInternal(ContainerBase.java:759)

at org.apache.catalina.core.ContainerBase.addChild(ContainerBase.java:739)

at org.apache.catalina.core.StandardHost.addChild(StandardHost.java:524)

at org.apache.catalina.startup.HostConfig.deployDirectory(HostConfig.java:909)

at org.apache.catalina.startup.HostConfig.deployDirectories(HostConfig.java:872)

at org.apache.catalina.startup.HostConfig.deployApps(HostConfig.java:474)

at org.apache.catalina.startup.HostConfig.start(HostConfig.java:1106)

at org.apache.catalina.startup.HostConfig.lifecycleEvent(HostConfig.java:310)

at org.apache.catalina.util.LifecycleSupport.fireLifecycleEvent(LifecycleSupport.java:119)

at org.apache.catalina.core.ContainerBase.start(ContainerBase.java:1019)

at org.apache.catalina.core.StandardHost.start(StandardHost.java:718)

at org.apache.catalina.core.ContainerBase.start(ContainerBase.java:1011)

at org.apache.catalina.core.StandardEngine.start(StandardEngine.java:440)

at org.apache.catalina.core.StandardService.start(StandardService.java:450)

at org.apache.catalina.core.StandardServer.start(StandardServer.java:683)

at org.apache.catalina.startup.Catalina.start(Catalina.java:537)

at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)

at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)

at java.lang.reflect.Method.invoke(Method.java:585)

at org.apache.catalina.startup.Bootstrap.start(Bootstrap.java:271)

at org.apache.catalina.startup.Bootstrap.main(Bootstrap.java:409)

Caused by: java.net.BindException: Cannot assign requested address

at sun.nio.ch.Net.bind(Native Method)

at sun.nio.ch.ServerSocketChannelImpl.bind(ServerSocketChannelImpl.java:119)

at sun.nio.ch.ServerSocketAdaptor.bind(ServerSocketAdaptor.java:59)

at sun.nio.ch.ServerSocketAdaptor.bind(ServerSocketAdaptor.java:52)

at com.zimbra.cs.util.NetUtil.newBoundServerSocket(NetUtil.java:78)

at com.zimbra.cs.util.NetUtil.getBoundServerSocket(NetUtil.java:52)

... 30 more

2006-05-04 10:53:44,635 INFO [main] [] [/service] - Marking servlet AdminServlet as unavailable

2006-05-04 10:53:44,635 ERROR [main] [] [/service] - Servlet /service threw load() exception

javax.servlet.UnavailableException: system failure: Could not bind to port=7025 bindaddr=clickmail.click.com.au ssl=false

at com.zimbra.soap.SoapServlet.init(SoapServlet.java:121)

at javax.servlet.GenericServlet.init(GenericServlet.java:211)

at org.apache.catalina.core.StandardWrapper.loadServlet(StandardWrapper.java:1068)

at org.apache.catalina.core.StandardWrapper.load(StandardWrapper.java:900)

at org.apache.catalina.core.StandardContext.loadOnStartup(StandardContext.java:3823)

at org.apache.catalina.core.StandardContext.start(StandardContext.java:4087)

at org.apache.catalina.core.ContainerBase.addChildInternal(ContainerBase.java:759)

at org.apache.catalina.core.ContainerBase.addChild(ContainerBase.java:739)

at org.apache.catalina.core.StandardHost.addChild(StandardHost.java:524)

at org.apache.catalina.startup.HostConfig.deployDirectory(HostConfig.java:909)

at org.apache.catalina.startup.HostConfig.deployDirectories(HostConfig.java:872)

at org.apache.catalina.startup.HostConfig.deployApps(HostConfig.java:474)

at org.apache.catalina.startup.HostConfig.start(HostConfig.java:1106)

at org.apache.catalina.startup.HostConfig.lifecycleEvent(HostConfig.java:310)

at org.apache.catalina.util.LifecycleSupport.fireLifecycleEvent(LifecycleSupport.java:119)

at org.apache.catalina.core.ContainerBase.start(ContainerBase.java:1019)

at org.apache.catalina.core.StandardHost.start(StandardHost.java:718)

at org.apache.catalina.core.ContainerBase.start(ContainerBase.java:1011)

at org.apache.catalina.core.StandardEngine.start(StandardEngine.java:440)

at org.apache.catalina.core.StandardService.start(StandardService.java:450)

at org.apache.catalina.core.StandardServer.start(StandardServer.java:683)

at org.apache.catalina.startup.Catalina.start(Catalina.java:537)

at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)

at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)

at java.lang.reflect.Method.invoke(Method.java:585)

at org.apache.catalina.startup.Bootstrap.start(Bootstrap.java:271)

at org.apache.catalina.startup.Bootstrap.main(Bootstrap.java:409)

2006-05-04 10:53:44,644 INFO [main] [] ContentServlet - Servlet ContentServlet starting up

2006-05-04 10:53:44,647 INFO [main] [] PublicICalServlet - Servlet PublicICalServlet starting up

2006-05-04 10:53:44,653 INFO [main] [] account - Servlet PreAuthServlet starting up

2006-05-04 10:53:44,686 INFO [main] [] mailbox - Servlet UserServlet starting up

2006-05-04 10:53:44,697 INFO [main] [] FileUploadServlet - Servlet FileUploadServlet starting up

2006-05-04 10:53:44,702 INFO [main] [] StatsImageServlet - Servlet StatsImageServlet starting up
---------------------------
catalina.out:
4/05/2006 10:53:37 org.apache.coyote.http11.Http11Protocol init

INFO: Initializing Coyote HTTP/1.1 on http-80

4/05/2006 10:53:38 org.apache.coyote.http11.Http11Protocol init

INFO: Initializing Coyote HTTP/1.1 on http-7071

4/05/2006 10:53:38 org.apache.catalina.startup.Catalina load

INFO: Initialization processed in 1861 ms

4/05/2006 10:53:38 org.apache.catalina.core.StandardService start

INFO: Starting service Catalina

4/05/2006 10:53:38 org.apache.catalina.core.StandardEngine start

INFO: Starting Servlet Engine: Apache Tomcat/5.5.7

4/05/2006 10:53:38 org.apache.catalina.core.StandardHost start

INFO: XML validation disabled

log4j:WARN No appenders could be found for logger (org.apache.catalina.session.ManagerBase).

log4j:WARN Please initialize the log4j system properly.

Zimbra server reserving server socket port=110 bindaddr=null ssl=false

Zimbra server reserving server socket port=995 bindaddr=null ssl=true

Zimbra server reserving server socket port=143 bindaddr=null ssl=false

Zimbra server reserving server socket port=993 bindaddr=null ssl=true

Zimbra server process is running as root, changing to user=zimbra uid=501 gid=502

Zimbra server process, after change, is running with uid=501 euid=501 gid=502 egid=502

4/05/2006 10:53:45 org.apache.coyote.http11.Http11Protocol start

INFO: Starting Coyote HTTP/1.1 on http-80

4/05/2006 10:53:45 org.apache.coyote.http11.Http11Protocol start

INFO: Starting Coyote HTTP/1.1 on http-7071

4/05/2006 10:53:45 org.apache.catalina.startup.Catalina start

INFO: Server startup in 6661 ms
rsharpe
Outstanding Member
Outstanding Member
Posts: 254
Joined: Fri Sep 12, 2014 9:59 pm

zimbra dies

Post by rsharpe »

When zimbra is being restarted it looks as though it is not kill all the processes properly. There were some lines that indicated it couldn not bing to port 7025 which would mean there is still some mailbox process running.
fredo
Posts: 37
Joined: Fri Sep 12, 2014 10:09 pm

zimbra dies

Post by fredo »

I have some problems in the sense that we're having some power outage issues from time to time currently. I've noticed that every time after a power failure, the antivirus, mailbox and mta are stopped. I have to re-start all services for them to be up and running again.
Can someone clear that issue up ? Can Zimbra keep it's instances alive when such issues happen ?
phoenix
Ambassador
Ambassador
Posts: 27272
Joined: Fri Sep 12, 2014 9:56 pm
Location: Liverpool, England

zimbra dies

Post by phoenix »

[quote user="fredo"]I have some problems in the sense that we're having some power outage issues from time to time currently. I've noticed that every time after a power failure, the antivirus, mailbox and mta are stopped. I have to re-start all services for them to be up and running again.
Can someone clear that issue up ? Can Zimbra keep it's instances alive when such issues happen ?[/QUOTE]

Is there anything in the logs indicating why they haven't restarted? I've had power-failures and Zimbra has restarted OK. If you're able to document it then file it in bugzilla.
Regards

Bill

Rspamd: A high performance spamassassin replacement

Per ardua ad astra
Post Reply