ERROR: service.PROXY_ERROR 503 service unavailable

Discuss your pilot or production implementation with other Zimbra admins or our engineers.
kayasaman
Posts: 20
Joined: Thu Nov 22, 2018 9:05 pm

Re: ERROR: service.PROXY_ERROR 503 service unavailable

Post by kayasaman »

Nope!

Code: Select all

# sestatus
SELinux status:                 disabled
User avatar
king0770
Outstanding Member
Outstanding Member
Posts: 242
Joined: Fri Sep 12, 2014 10:44 pm
Contact:

Re: ERROR: service.PROXY_ERROR 503 service unavailable

Post by king0770 »

Do you have seLinux enabled by chance?

sestaus
User avatar
king0770
Outstanding Member
Outstanding Member
Posts: 242
Joined: Fri Sep 12, 2014 10:44 pm
Contact:

Re: ERROR: service.PROXY_ERROR 503 service unavailable

Post by king0770 »

how much space is left on /opt/zimbra/
--
Rick King
kayasaman
Posts: 20
Joined: Thu Nov 22, 2018 9:05 pm

Re: ERROR: service.PROXY_ERROR 503 service unavailable

Post by kayasaman »

df -h:

Code: Select all

7.9T   93G  7.8T   2% /opt/zimbra
User avatar
king0770
Outstanding Member
Outstanding Member
Posts: 242
Joined: Fri Sep 12, 2014 10:44 pm
Contact:

Re: ERROR: service.PROXY_ERROR 503 service unavailable

Post by king0770 »

Alright, can you lower zimbraImapActiveSessionEhcacheMaxDiskSize to 2G?

zmprov -l ms `zmhostname` zimbraImapActiveSessionEhcacheMaxDiskSize 2147483648

zmcontrol stop

zmcontrol start
--
Rick King
kayasaman
Posts: 20
Joined: Thu Nov 22, 2018 9:05 pm

Re: ERROR: service.PROXY_ERROR 503 service unavailable

Post by kayasaman »

Still same....

Code: Select all

zmthrdump: Requested thread dump [PID 27790] at Wed Dec 12 00:12:45 2018
2018-12-12 00:12:45.952:INFO:oejs.ServerConnector:Thread-12: Stopped ServerConnector@6e2c9341{HTTP/1.1,[http/1.1]}{0.0.0.0:8080}
2018-12-12 00:12:45.953:INFO:oejs.ServerConnector:Thread-12: Stopped ServerConnector@12028586{SSL,[ssl, http/1.1]}{0.0.0.0:8443}
2018-12-12 00:12:45.956:INFO:oejs.ServerConnector:Thread-12: Stopped ServerConnector@17776a8{SSL,[ssl, http/1.1]}{0.0.0.0:7071}
2018-12-12 00:12:45.957:INFO:oejs.ServerConnector:Thread-12: Stopped ServerConnector@69a10787{SSL,[ssl, http/1.1]}{0.0.0.0:7073}
2018-12-12 00:12:45.958:INFO:oejs.ServerConnector:Thread-12: Stopped ServerConnector@2d127a61{SSL,[ssl, http/1.1]}{0.0.0.0:7072}
2018-12-12 00:12:46.003:INFO:oejsh.ContextHandler:Thread-12: Stopped o.e.j.w.WebAppContext@3eb7fc54{/zimlet,[file:///opt/zimbra/jetty_base/webapps/zimlet/, file:///opt/zimbra/zimlets-deployed/],UNAVAILABLE}{/zimlet}
2018-12-12 00:12:46.012:INFO:oejsh.ContextHandler:Thread-12: Stopped o.e.j.w.WebAppContext@79b06cab{/zimbraAdmin,null,UNAVAILABLE}{/zimbraAdmin}
2018-12-12 00:12:46.016:INFO:oejsh.ContextHandler:Thread-12: Stopped o.e.j.w.WebAppContext@4690b489{/zimbra,null,UNAVAILABLE}{/zimbra}
2018-12-12 00:12:46.037:INFO:oejsh.ContextHandler:Thread-12: Stopped o.e.j.w.WebAppContext@4b14c583{/service,null,UNAVAILABLE}{/service}
OpenJDK 64-Bit Server VM warning: .hotspot_compiler file is present but has been ignored.  Run with -XX:CompileCommandFile=.hotspot_compiler to load the file.
2018-12-12 00:14:35.638:INFO::main: Logging initialized @2096ms
JettyMonitor monitoring thread pool qtp1286783232{STOPPED,10<=0<=250,i=0,q=0}
Zimbra server reserving server socket port=7110 bindaddr=null ssl=false
Zimbra server reserving server socket port=7995 bindaddr=null ssl=false
Zimbra server reserving server socket port=7143 bindaddr=null ssl=false
Zimbra server reserving server socket port=7993 bindaddr=null ssl=false
Zimbra server reserving server socket port=7025 bindaddr=null ssl=false
2018-12-12 00:14:38.332:INFO:oejs.SetUIDListener:main: Setting umask=027
2018-12-12 00:14:38.333:INFO:oejs.SetUIDListener:main: Current rlimit_nofiles (soft=524288, hard=524288)
2018-12-12 00:14:38.333:INFO:oejs.SetUIDListener:main: Set rlimit_nofiles (soft=65536, hard=65536)
2018-12-12 00:14:38.358:INFO:oejs.SetUIDListener:main: Opened ServerConnector@6e2c9341{HTTP/1.1,[http/1.1]}{0.0.0.0:8080}
2018-12-12 00:14:38.359:INFO:oejs.SetUIDListener:main: Opened ServerConnector@12028586{SSL,[ssl, http/1.1]}{0.0.0.0:8443}
2018-12-12 00:14:38.359:INFO:oejs.SetUIDListener:main: Opened ServerConnector@17776a8{SSL,[ssl, http/1.1]}{0.0.0.0:7071}
2018-12-12 00:14:38.360:INFO:oejs.SetUIDListener:main: Opened ServerConnector@69a10787{SSL,[ssl, http/1.1]}{0.0.0.0:7073}
2018-12-12 00:14:38.361:INFO:oejs.SetUIDListener:main: Opened ServerConnector@2d127a61{SSL,[ssl, http/1.1]}{0.0.0.0:7072}
2018-12-12 00:14:38.361:INFO:oejs.SetUIDListener:main: Setting GID=500
2018-12-12 00:14:38.366:INFO:oejs.SetUIDListener:main: Setting UID=500
2018-12-12 00:14:38.377:INFO:oejs.Server:main: jetty-9.3.5.v20151012
2018-12-12 00:14:51.686:WARN:oejs.SecurityHandler:main: ServletContext@o.e.j.w.WebAppContext@4b14c583{/service,file:///opt/zimbra/jetty_base/webapps/service/,STARTING}{/service} has uncovered http methods for path: /
Zimbra server process is running as uid=500 euid=500 gid=500 egid=500
2018-12-12 00:15:12.580:WARN:/service:main: unavailable
javax.servlet.UnavailableException: Tiering Inversion: 'Pool {107374182400 B offheap}' is not smaller than 'Pool {10737418240 B disk(persistent)}'
	at com.zimbra.soap.SoapServlet.init(SoapServlet.java:134)
	at javax.servlet.GenericServlet.init(GenericServlet.java:244)
	at org.eclipse.jetty.servlet.ServletHolder.initServlet(ServletHolder.java:626)
	at org.eclipse.jetty.servlet.ServletHolder.initialize(ServletHolder.java:405)
	at org.eclipse.jetty.servlet.ServletHandler.initialize(ServletHandler.java:875)
	at org.eclipse.jetty.servlet.ServletContextHandler.startContext(ServletContextHandler.java:346)
	at org.eclipse.jetty.webapp.WebAppContext.startWebapp(WebAppContext.java:1380)
	at org.eclipse.jetty.webapp.WebAppContext.startContext(WebAppContext.java:1342)
	at org.eclipse.jetty.server.handler.ContextHandler.doStart(ContextHandler.java:772)
	at org.eclipse.jetty.servlet.ServletContextHandler.doStart(ServletContextHandler.java:259)
	at org.eclipse.jetty.webapp.WebAppContext.doStart(WebAppContext.java:518)
	at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68)
	at org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:132)
	at org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:114)
	at org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:61)
	at org.eclipse.jetty.server.handler.ContextHandlerCollection.doStart(ContextHandlerCollection.java:161)
	at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68)
	at org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:132)
	at org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:114)
	at org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:61)
	at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68)
	at org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:132)
	at org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:106)
	at org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:61)
	at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68)
	at org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:132)
	at org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:106)
	at org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:61)
	at org.eclipse.jetty.server.handler.gzip.GzipHandler.doStart(GzipHandler.java:212)
	at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68)
	at org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:132)
	at org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:106)
	at org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:61)
	at org.eclipse.jetty.server.handler.DebugHandler.doStart(DebugHandler.java:143)
	at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68)
	at org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:132)
	at org.eclipse.jetty.server.Server.start(Server.java:405)
	at org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:106)
	at org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:61)
	at org.eclipse.jetty.server.Server.doStart(Server.java:372)
	at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68)
	at org.eclipse.jetty.xml.XmlConfiguration$1.run(XmlConfiguration.java:1510)
	at java.security.AccessController.doPrivileged(Native Method)
	at org.eclipse.jetty.xml.XmlConfiguration.main(XmlConfiguration.java:1435)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at org.eclipse.jetty.start.Main.invokeMain(Main.java:214)
	at org.eclipse.jetty.start.Main.start(Main.java:457)
	at org.eclipse.jetty.start.Main.main(Main.java:75)
2018-12-12 00:15:15.285:WARN:/service:main: unavailable
javax.servlet.UnavailableException: Couldn't lock rootDir: /opt/zimbra/data/mailboxd
	at com.zimbra.soap.SoapServlet.init(SoapServlet.java:134)
	at javax.servlet.GenericServlet.init(GenericServlet.java:244)
	at org.eclipse.jetty.servlet.ServletHolder.initServlet(ServletHolder.java:626)
	at org.eclipse.jetty.servlet.ServletHolder.initialize(ServletHolder.java:405)
	at org.eclipse.jetty.servlet.ServletHandler.initialize(ServletHandler.java:875)
	at org.eclipse.jetty.servlet.ServletContextHandler.startContext(ServletContextHandler.java:346)
	at org.eclipse.jetty.webapp.WebAppContext.startWebapp(WebAppContext.java:1380)
	at org.eclipse.jetty.webapp.WebAppContext.startContext(WebAppContext.java:1342)
	at org.eclipse.jetty.server.handler.ContextHandler.doStart(ContextHandler.java:772)
	at org.eclipse.jetty.servlet.ServletContextHandler.doStart(ServletContextHandler.java:259)
	at org.eclipse.jetty.webapp.WebAppContext.doStart(WebAppContext.java:518)
	at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68)
	at org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:132)
	at org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:114)
	at org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:61)
	at org.eclipse.jetty.server.handler.ContextHandlerCollection.doStart(ContextHandlerCollection.java:161)
	at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68)
	at org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:132)
	at org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:114)
	at org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:61)
	at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68)
	at org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:132)
	at org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:106)
	at org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:61)
	at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68)
	at org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:132)
	at org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:106)
	at org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:61)
	at org.eclipse.jetty.server.handler.gzip.GzipHandler.doStart(GzipHandler.java:212)
	at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68)
	at org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:132)
	at org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:106)
	at org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:61)
	at org.eclipse.jetty.server.handler.DebugHandler.doStart(DebugHandler.java:143)
	at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68)
	at org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:132)
	at org.eclipse.jetty.server.Server.start(Server.java:405)
	at org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:106)
	at org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:61)
	at org.eclipse.jetty.server.Server.doStart(Server.java:372)
	at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68)
	at org.eclipse.jetty.xml.XmlConfiguration$1.run(XmlConfiguration.java:1510)
	at java.security.AccessController.doPrivileged(Native Method)
	at org.eclipse.jetty.xml.XmlConfiguration.main(XmlConfiguration.java:1435)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at org.eclipse.jetty.start.Main.invokeMain(Main.java:214)
	at org.eclipse.jetty.start.Main.start(Main.java:457)
	at org.eclipse.jetty.start.Main.main(Main.java:75)
2018-12-12 00:15:15.974:WARN:oejw.WebAppContext:main: Failed startup of context o.e.j.w.WebAppContext@4b14c583{/service,file:///opt/zimbra/jetty_base/webapps/service/,STARTING}{/service}
MultiException[javax.servlet.UnavailableException: Tiering Inversion: 'Pool {107374182400 B offheap}' is not smaller than 'Pool {10737418240 B disk(persistent)}', javax.servlet.UnavailableException: Couldn't lock rootDir: /opt/zimbra/data/mailboxd]
	at org.eclipse.jetty.servlet.ServletHandler.initialize(ServletHandler.java:846)
	at org.eclipse.jetty.servlet.ServletContextHandler.startContext(ServletContextHandler.java:346)
	at org.eclipse.jetty.webapp.WebAppContext.startWebapp(WebAppContext.java:1380)
	at org.eclipse.jetty.webapp.WebAppContext.startContext(WebAppContext.java:1342)
	at org.eclipse.jetty.server.handler.ContextHandler.doStart(ContextHandler.java:772)
	at org.eclipse.jetty.servlet.ServletContextHandler.doStart(ServletContextHandler.java:259)
	at org.eclipse.jetty.webapp.WebAppContext.doStart(WebAppContext.java:518)
	at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68)
	at org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:132)
	at org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:114)
	at org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:61)
	at org.eclipse.jetty.server.handler.ContextHandlerCollection.doStart(ContextHandlerCollection.java:161)
	at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68)
	at org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:132)
	at org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:114)
	at org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:61)
	at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68)
	at org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:132)
	at org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:106)
	at org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:61)
	at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68)
	at org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:132)
	at org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:106)
	at org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:61)
	at org.eclipse.jetty.server.handler.gzip.GzipHandler.doStart(GzipHandler.java:212)
	at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68)
	at org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:132)
	at org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:106)
	at org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:61)
	at org.eclipse.jetty.server.handler.DebugHandler.doStart(DebugHandler.java:143)
	at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68)
	at org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:132)
	at org.eclipse.jetty.server.Server.start(Server.java:405)
	at org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:106)
	at org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:61)
	at org.eclipse.jetty.server.Server.doStart(Server.java:372)
	at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68)
	at org.eclipse.jetty.xml.XmlConfiguration$1.run(XmlConfiguration.java:1510)
	at java.security.AccessController.doPrivileged(Native Method)
	at org.eclipse.jetty.xml.XmlConfiguration.main(XmlConfiguration.java:1435)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at org.eclipse.jetty.start.Main.invokeMain(Main.java:214)
	at org.eclipse.jetty.start.Main.start(Main.java:457)
	at org.eclipse.jetty.start.Main.main(Main.java:75)
Suppressed: 
	|javax.servlet.UnavailableException: Couldn't lock rootDir: /opt/zimbra/data/mailboxd
	|	at com.zimbra.soap.SoapServlet.init(SoapServlet.java:134)
	|	at javax.servlet.GenericServlet.init(GenericServlet.java:244)
	|	at org.eclipse.jetty.servlet.ServletHolder.initServlet(ServletHolder.java:626)
	|	at org.eclipse.jetty.servlet.ServletHolder.initialize(ServletHolder.java:405)
	|	at org.eclipse.jetty.servlet.ServletHandler.initialize(ServletHandler.java:875)
	|	at org.eclipse.jetty.servlet.ServletContextHandler.startContext(ServletContextHandler.java:346)
	|	at org.eclipse.jetty.webapp.WebAppContext.startWebapp(WebAppContext.java:1380)
	|	at org.eclipse.jetty.webapp.WebAppContext.startContext(WebAppContext.java:1342)
	|	at org.eclipse.jetty.server.handler.ContextHandler.doStart(ContextHandler.java:772)
	|	at org.eclipse.jetty.servlet.ServletContextHandler.doStart(ServletContextHandler.java:259)
	|	at org.eclipse.jetty.webapp.WebAppContext.doStart(WebAppContext.java:518)
	|	at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68)
	|	at org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:132)
	|	at org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:114)
	|	at org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:61)
	|	at org.eclipse.jetty.server.handler.ContextHandlerCollection.doStart(ContextHandlerCollection.java:161)
	|	at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68)
	|	at org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:132)
	|	at org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:114)
	|	at org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:61)
	|	at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68)
	|	at org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:132)
	|	at org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:106)
	|	at org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:61)
	|	at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68)
	|	at org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:132)
	|	at org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:106)
	|	at org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:61)
	|	at org.eclipse.jetty.server.handler.gzip.GzipHandler.doStart(GzipHandler.java:212)
	|	at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68)
	|	at org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:132)
	|	at org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:106)
	|	at org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:61)
	|	at org.eclipse.jetty.server.handler.DebugHandler.doStart(DebugHandler.java:143)
	|	at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68)
	|	at org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:132)
	|	at org.eclipse.jetty.server.Server.start(Server.java:405)
	|	at org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:106)
	|	at org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:61)
	|	at org.eclipse.jetty.server.Server.doStart(Server.java:372)
	|	at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68)
	|	at org.eclipse.jetty.xml.XmlConfiguration$1.run(XmlConfiguration.java:1510)
	|	at java.security.AccessController.doPrivileged(Native Method)
	|	at org.eclipse.jetty.xml.XmlConfiguration.main(XmlConfiguration.java:1435)
	|	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	|	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	|	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	|	at java.lang.reflect.Method.invoke(Method.java:498)
	|	at org.eclipse.jetty.start.Main.invokeMain(Main.java:214)
	|	at org.eclipse.jetty.start.Main.start(Main.java:457)
	|	at org.eclipse.jetty.start.Main.main(Main.java:75)
Caused by: 
javax.servlet.UnavailableException: Tiering Inversion: 'Pool {107374182400 B offheap}' is not smaller than 'Pool {10737418240 B disk(persistent)}'
	at com.zimbra.soap.SoapServlet.init(SoapServlet.java:134)
	at javax.servlet.GenericServlet.init(GenericServlet.java:244)
	at org.eclipse.jetty.servlet.ServletHolder.initServlet(ServletHolder.java:626)
	at org.eclipse.jetty.servlet.ServletHolder.initialize(ServletHolder.java:405)
	at org.eclipse.jetty.servlet.ServletHandler.initialize(ServletHandler.java:875)
	at org.eclipse.jetty.servlet.ServletContextHandler.startContext(ServletContextHandler.java:346)
	at org.eclipse.jetty.webapp.WebAppContext.startWebapp(WebAppContext.java:1380)
	at org.eclipse.jetty.webapp.WebAppContext.startContext(WebAppContext.java:1342)
	at org.eclipse.jetty.server.handler.ContextHandler.doStart(ContextHandler.java:772)
	at org.eclipse.jetty.servlet.ServletContextHandler.doStart(ServletContextHandler.java:259)
	at org.eclipse.jetty.webapp.WebAppContext.doStart(WebAppContext.java:518)
	at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68)
	at org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:132)
	at org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:114)
	at org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:61)
	at org.eclipse.jetty.server.handler.ContextHandlerCollection.doStart(ContextHandlerCollection.java:161)
	at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68)
	at org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:132)
	at org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:114)
	at org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:61)
	at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68)
	at org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:132)
	at org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:106)
	at org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:61)
	at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68)
	at org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:132)
	at org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:106)
	at org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:61)
	at org.eclipse.jetty.server.handler.gzip.GzipHandler.doStart(GzipHandler.java:212)
	at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68)
	at org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:132)
	at org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:106)
	at org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:61)
	at org.eclipse.jetty.server.handler.DebugHandler.doStart(DebugHandler.java:143)
	at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68)
	at org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:132)
	at org.eclipse.jetty.server.Server.start(Server.java:405)
	at org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:106)
	at org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:61)
	at org.eclipse.jetty.server.Server.doStart(Server.java:372)
	at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68)
	at org.eclipse.jetty.xml.XmlConfiguration$1.run(XmlConfiguration.java:1510)
	at java.security.AccessController.doPrivileged(Native Method)
	at org.eclipse.jetty.xml.XmlConfiguration.main(XmlConfiguration.java:1435)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at org.eclipse.jetty.start.Main.invokeMain(Main.java:214)
	at org.eclipse.jetty.start.Main.start(Main.java:457)
	at org.eclipse.jetty.start.Main.main(Main.java:75)
2018-12-12 00:15:21.190:WARN:oejpw.PlusDescriptorProcessor:main: No value for env-entry-name trustedIPs
2018-12-12 00:15:22.014:WARN:oejs.SecurityHandler:main: ServletContext@o.e.j.w.WebAppContext@4690b489{/zimbra,file:///opt/zimbra/jetty_base/webapps/zimbra/,STARTING}{/zimbra} has uncovered http methods for path: /
2018-12-12 00:15:23.806:INFO:oejsh.ContextHandler:main: Started o.e.j.w.WebAppContext@4690b489{/zimbra,file:///opt/zimbra/jetty_base/webapps/zimbra/,AVAILABLE}{/zimbra}
2018-12-12 00:15:31.003:WARN:oejpw.PlusDescriptorProcessor:main: No value for env-entry-name trustedIPs
2018-12-12 00:15:31.573:WARN:oejs.SecurityHandler:main: ServletContext@o.e.j.w.WebAppContext@79b06cab{/zimbraAdmin,file:///opt/zimbra/jetty_base/webapps/zimbraAdmin/,STARTING}{/zimbraAdmin} has uncovered http methods for path: /
2018-12-12 00:15:33.473:INFO:oejsh.ContextHandler:main: Started o.e.j.w.WebAppContext@79b06cab{/zimbraAdmin,file:///opt/zimbra/jetty_base/webapps/zimbraAdmin/,AVAILABLE}{/zimbraAdmin}
2018-12-12 00:15:34.579:WARN:oejs.SecurityHandler:main: ServletContext@o.e.j.w.WebAppContext@3eb7fc54{/zimlet,[file:///opt/zimbra/jetty_base/webapps/zimlet/, file:///opt/zimbra/zimlets-deployed/],STARTING}{/zimlet} has uncovered http methods for path: /
2018-12-12 00:15:34.581:INFO:oejsh.ContextHandler:main: Started o.e.j.w.WebAppContext@3eb7fc54{/zimlet,[file:///opt/zimbra/jetty_base/webapps/zimlet/, file:///opt/zimbra/zimlets-deployed/],AVAILABLE}{/zimlet}
2018-12-12 00:15:34.592:INFO:oejs.AbstractNCSARequestLog:main: Opened /opt/zimbra/log/access_log.2018-12-12
2018-12-12 00:15:34.602:INFO:oejm.ThreadMonitor:main: Thread Monitor started successfully
2018-12-12 00:15:34.645:INFO:oejs.ServerConnector:main: Started ServerConnector@6e2c9341{HTTP/1.1,[http/1.1]}{0.0.0.0:8080}
2018-12-12 00:15:34.668:INFO:oejus.SslContextFactory:main: x509=X509@350da119(jetty,h=[x-ray.optiplex-networks.com],w=[]) for SslContextFactory@367d34c0(file:///opt/zimbra/jetty_base/etc/keystore,null)
2018-12-12 00:15:34.730:INFO:oejs.ServerConnector:main: Started ServerConnector@12028586{SSL,[ssl, http/1.1]}{0.0.0.0:8443}
2018-12-12 00:15:34.752:INFO:oejs.ServerConnector:main: Started ServerConnector@17776a8{SSL,[ssl, http/1.1]}{0.0.0.0:7071}
2018-12-12 00:15:34.754:INFO:oejs.ServerConnector:main: Started ServerConnector@69a10787{SSL,[ssl, http/1.1]}{0.0.0.0:7073}
2018-12-12 00:15:34.757:INFO:oejs.ServerConnector:main: Started ServerConnector@2d127a61{SSL,[ssl, http/1.1]}{0.0.0.0:7072}
2018-12-12 00:15:34.757:INFO:oejs.Server:main: Started @61216ms
I wonder if this has something to do with it from the above zmmailboxd.out log:

Code: Select all

javax.servlet.UnavailableException: Tiering Inversion: 'Pool {107374182400 B offheap}' is not smaller than 'Pool {10737418240 B disk(persistent)}'
Test to show it has taken the value:

Code: Select all

zmprov -l gs `zmhostname` zimbraImapActiveSessionEhcacheMaxDiskSize
zimbraImapActiveSessionEhcacheMaxDiskSize: 2147483648
User avatar
king0770
Outstanding Member
Outstanding Member
Posts: 242
Joined: Fri Sep 12, 2014 10:44 pm
Contact:

Re: ERROR: service.PROXY_ERROR 503 service unavailable

Post by king0770 »

Hmm...weird. The error is referencing 107374182400, which is 100GB. Do you have anything with that value?

zmprov -l gs `zmhostname` | grep 107374182400

zmprov -l gacf | grep 107374182400
kayasaman
Posts: 20
Joined: Thu Nov 22, 2018 9:05 pm

Re: ERROR: service.PROXY_ERROR 503 service unavailable

Post by kayasaman »

Output:

Code: Select all

  $ zmprov -l gs `zmhostname` | grep 107374182400
zimbraImapInactiveSessionEhcacheMaxDiskSize: 107374182400
zimbraImapInactiveSessionEhcacheSize: 107374182400

Code: Select all

$ zmprov -l gacf | grep 107374182400
zimbraImapActiveSessionEhcacheMaxDiskSize: 107374182400
zimbraImapInactiveSessionEhcacheMaxDiskSize: 107374182400
User avatar
king0770
Outstanding Member
Outstanding Member
Posts: 242
Joined: Fri Sep 12, 2014 10:44 pm
Contact:

Re: ERROR: service.PROXY_ERROR 503 service unavailable

Post by king0770 »

Might want to view the CacheSize settings as well.

zmprov -l gs `zmhostname` | grep -i cachesize

zmprov -l gacf | grep -i cachesize
--
Rick King
kayasaman
Posts: 20
Joined: Thu Nov 22, 2018 9:05 pm

Re: ERROR: service.PROXY_ERROR 503 service unavailable

Post by kayasaman »

Code: Select all

$ zmprov -l gs `zmhostname` | grep -i cachesize
zimbraHttpHeaderCacheSize: 512
zimbraImapInactiveSessionEhcacheSize: 107374182400
zimbraMailFileDescriptorCacheSize: 1000
zimbraMessageCacheSize: 2000
zimbraNotebookFolderCacheSize: 1024
zimbraNotebookPageCacheSize: 10240
zimbraReverseProxySSLSessionCacheSize: 10m
zimbraShortTermAllEffectiveRightsCacheSize: 128
zimbraShortTermGranteeCacheSize: 128

Code: Select all

$ zmprov -l gacf | grep -i cachesize
zimbraHttpHeaderCacheSize: 512
zimbraImapInactiveSessionEhcacheSize: 1048576
zimbraMailFileDescriptorCacheSize: 1000
zimbraMessageCacheSize: 2000
zimbraMessageIdDedupeCacheSize: 3000
zimbraNotebookFolderCacheSize: 1024
zimbraNotebookPageCacheSize: 10240
zimbraReverseProxySSLSessionCacheSize: 10m
zimbraShortTermAllEffectiveRightsCacheSize: 128
zimbraShortTermGranteeCacheSize: 128
Post Reply