ZCS OSE cannot start after upgrade to 8.8.15

rosole · Post by **rosole** » Mon Jul 29, 2019 9:19 pm

Hello.

Once again got problem after Zimbra upgrade to 8.8.15:

zmcontrol status
Host zimbra.domain.pl
amavis Running
antispam Running
antivirus Running
dnscache Running
ldap Running
logger Running
mailbox Stopped
zmmailboxdctl is not running.
memcached Running
mta Running
opendkim Running
proxy Running
service webapp Stopped
zmmailboxdctl is not running.
snmp Running
spell Running
stats Running
zimbra webapp Stopped
zmmailboxdctl is not running.
zimbraAdmin webapp Stopped
zmmailboxdctl is not running.
zimlet webapp Stopped
zmmailboxdctl is not running.
zmconfigd Running

Upgrade was as usually succesfully without errors.

Got a lot of informations like below in /var/log/zimbra.log:

Jul 29 23:13:01 zimbra zmmailboxdmgr[10159]: stale pid 2471 found in /opt/zimbra/log/zmmailboxd_manager.pid: No such process
Jul 29 23:13:01 zimbra zmmailboxdmgr[10159]: assuming no other instance is running
Jul 29 23:13:01 zimbra zmmailboxdmgr[10159]: did not find a number in /opt/zimbra/log/zmmailboxd.pid
Jul 29 23:13:01 zimbra zmmailboxdmgr[10159]: assuming no other instance is running
Jul 29 23:13:01 zimbra zmmailboxdmgr[10159]: no manager process is running

And a lot of WARN in /opt/zimbra/zmmailboxd.out (which i don't understand):

[0.072s][info][gc] Using G1
OpenJDK 64-Bit Server VM warning: .hotspot_compiler file is present but has been ignored. Run with -XX:CompileCommandFile=.hotspot_compiler to load the file.
2019-07-29 23:02:00.529:INFO::main: Logging initialized @1453ms to org.eclipse.jetty.util.log.StdErrLog
JettyMonitor monitoring thread pool QueuedThreadPool[qtp1911152052]@71e9ddb4{STOPPED,10<=0<=250,i=0,r=-1,q=0}[NO_TRY]
[3.299s][info][gc] GC(0) Pause Young (Concurrent Start) (Metadata GC Threshold) 60M->9M(4096M) 42.217ms
[3.299s][info][gc] GC(1) Concurrent Cycle
Zimbra server reserving server socket port=7143 bindaddr=null ssl=false
Zimbra server reserving server socket port=7993 bindaddr=null ssl=false
Zimbra server reserving server socket port=7025 bindaddr=null ssl=false
2019-07-29 23:02:02.413:INFO:oejs.SetUIDListener:main: Setting umask=027[3.370s][info][gc] GC(1) Pause Remark 10M->10M(4096M) 25.828ms
[3.371s][info][gc] GC(1) Pause Cleanup 10M->10M(4096M) 0.380ms

2019-07-29 23:02:02.448:INFO:oejs.SetUIDListener:main: Current rlimit_nofiles (soft=524288, hard=524288)
2019-07-29 23:02:02.454:INFO:oejs.SetUIDListener:main: Set rlimit_nofiles (soft=65536, hard=65536)
2019-07-29 23:02:02.473:INFO:oejs.SetUIDListener:main: Opened ServerConnector@4efc180e{HTTP/1.1,[http/1.1]}{localhost:8080}
2019-07-29 23:02:02.473:INFO:oejs.SetUIDListener:main: Opened ServerConnector@644baf4a{SSL,[ssl, http/1.1]}{0.0.0.0:8443}
2019-07-29 23:02:02.480:INFO:oejs.SetUIDListener:main: Opened ServerConnector@2eae8e6e{SSL,[ssl, http/1.1]}{0.0.0.0:7071}
2019-07-29 23:02:02.480:INFO:oejs.SetUIDListener:main: Opened ServerConnector@2cf3d63b{SSL,[ssl, http/1.1]}{0.0.0.0:7073}
2019-07-29 23:02:02.481:INFO:oejs.SetUIDListener:main: Opened ServerConnector@173ed316{SSL,[ssl, http/1.1]}{0.0.0.0:7072}
2019-07-29 23:02:02.481:INFO:oejs.SetUIDListener:main: Setting GID=999
2019-07-29 23:02:02.497:INFO:oejs.SetUIDListener:main: Setting UID=999
2019-07-29 23:02:02.540:INFO:oejs.Server:main: jetty-9.4.18.v20190429; built: 2019-04-29T20:42:08.989Z; git: e1bc35120a6617ee3df052294e433f3a25ce7097; jvm 11.0.2+9
[3.618s][info][gc] GC(1) Concurrent Cycle 318.800ms
2019-07-29 23:02:03.864:WARN:oeja.AnnotationParser:qtp1911152052-19: org.apache.batik.i18n.ExtendedLocalizable scanned from multiple locations: jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/batik-i18n-1.9.jar!/org/apache/batik/i18n/ExtendedLocalizable.class, jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/batik-util-1.8.jar!/org/apache/batik/i18n/ExtendedLocalizable.class
2019-07-29 23:02:03.864:WARN:oeja.AnnotationParser:qtp1911152052-19: org.apache.batik.i18n.LocaleGroup scanned from multiple locations: jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/batik-i18n-1.9.jar!/org/apache/batik/i18n/LocaleGroup.class, jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/batik-util-1.8.jar!/org/apache/batik/i18n/LocaleGroup.class
2019-07-29 23:02:03.864:WARN:oeja.AnnotationParser:qtp1911152052-19: org.apache.batik.i18n.Localizable scanned from multiple locations: jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/batik-i18n-1.9.jar!/org/apache/batik/i18n/Localizable.class, jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/batik-util-1.8.jar!/org/apache/batik/i18n/Localizable.class
2019-07-29 23:02:03.864:WARN:oeja.AnnotationParser:qtp1911152052-19: org.apache.batik.i18n.LocalizableSupport scanned from multiple locations: jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/batik-i18n-1.9.jar!/org/apache/batik/i18n/LocalizableSupport.class, jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/batik-util-1.8.jar!/org/apache/batik/i18n/LocalizableSupport.class
[5.924s][info][gc] GC(2) Pause Young (Normal) (G1 Evacuation Pause) 203M->20M(4096M) 74.632ms
2019-07-29 23:02:05.507:WARN:oeja.AnnotationParser:qtp1911152052-20: javax.ws.rs.ApplicationPath scanned from multiple locations: jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/javax.ws.rs-api-2.0-m10.jar!/javax/ws/rs/ApplicationPath.class, jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/jersey-core-1.11.jar!/javax/ws/rs/ApplicationPath.class
2019-07-29 23:02:05.517:WARN:oeja.AnnotationParser:qtp1911152052-20: javax.ws.rs.Consumes scanned from multiple locations: jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/javax.ws.rs-api-2.0-m10.jar!/javax/ws/rs/Consumes.class, jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/jersey-core-1.11.jar!/javax/ws/rs/Consumes.class
2019-07-29 23:02:05.517:WARN:oeja.AnnotationParser:qtp1911152052-20: javax.ws.rs.CookieParam scanned from multiple locations: jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/javax.ws.rs-api-2.0-m10.jar!/javax/ws/rs/CookieParam.class, jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/jersey-core-1.11.jar!/javax/ws/rs/CookieParam.class
2019-07-29 23:02:05.518:WARN:oeja.AnnotationParser:qtp1911152052-20: javax.ws.rs.DELETE scanned from multiple locations: jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/javax.ws.rs-api-2.0-m10.jar!/javax/ws/rs/DELETE.class, jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/jersey-core-1.11.jar!/javax/ws/rs/DELETE.class
2019-07-29 23:02:05.518:WARN:oeja.AnnotationParser:qtp1911152052-20: javax.ws.rs.DefaultValue scanned from multiple locations: jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/javax.ws.rs-api-2.0-m10.jar!/javax/ws/rs/DefaultValue.class, jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/jersey-core-1.11.jar!/javax/ws/rs/DefaultValue.class
2019-07-29 23:02:05.519:WARN:oeja.AnnotationParser:qtp1911152052-20: javax.ws.rs.Encoded scanned from multiple locations: jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/javax.ws.rs-api-2.0-m10.jar!/javax/ws/rs/Encoded.class, jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/jersey-core-1.11.jar!/javax/ws/rs/Encoded.class
2019-07-29 23:02:05.519:WARN:oeja.AnnotationParser:qtp1911152052-20: javax.ws.rs.FormParam scanned from multiple locations: jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/javax.ws.rs-api-2.0-m10.jar!/javax/ws/rs/FormParam.class, jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/jersey-core-1.11.jar!/javax/ws/rs/FormParam.class
2019-07-29 23:02:05.519:WARN:oeja.AnnotationParser:qtp1911152052-20: javax.ws.rs.GET scanned from multiple locations: jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/javax.ws.rs-api-2.0-m10.jar!/javax/ws/rs/GET.class, jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/jersey-core-1.11.jar!/javax/ws/rs/GET.class
2019-07-29 23:02:05.521:WARN:oeja.AnnotationParser:qtp1911152052-20: javax.ws.rs.HEAD scanned from multiple locations: jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/javax.ws.rs-api-2.0-m10.jar!/javax/ws/rs/HEAD.class, jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/jersey-core-1.11.jar!/javax/ws/rs/HEAD.class
2019-07-29 23:02:05.521:WARN:oeja.AnnotationParser:qtp1911152052-20: javax.ws.rs.HeaderParam scanned from multiple locations: jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/javax.ws.rs-api-2.0-m10.jar!/javax/ws/rs/HeaderParam.class, jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/jersey-core-1.11.jar!/javax/ws/rs/HeaderParam.class
2019-07-29 23:02:05.522:WARN:oeja.AnnotationParser:qtp1911152052-20: javax.ws.rs.HttpMethod scanned from multiple locations: jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/javax.ws.rs-api-2.0-m10.jar!/javax/ws/rs/HttpMethod.class, jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/jersey-core-1.11.jar!/javax/ws/rs/HttpMethod.class
2019-07-29 23:02:05.522:WARN:oeja.AnnotationParser:qtp1911152052-20: javax.ws.rs.MatrixParam scanned from multiple locations: jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/javax.ws.rs-api-2.0-m10.jar!/javax/ws/rs/MatrixParam.class, jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/jersey-core-1.11.jar!/javax/ws/rs/MatrixParam.class
2019-07-29 23:02:05.522:WARN:oeja.AnnotationParser:qtp1911152052-20: javax.ws.rs.OPTIONS scanned from multiple locations: jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/javax.ws.rs-api-2.0-m10.jar!/javax/ws/rs/OPTIONS.class, jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/jersey-core-1.11.jar!/javax/ws/rs/OPTIONS.class
2019-07-29 23:02:05.523:WARN:oeja.AnnotationParser:qtp1911152052-20: javax.ws.rs.POST scanned from multiple locations: jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/javax.ws.rs-api-2.0-m10.jar!/javax/ws/rs/POST.class, jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/jersey-core-1.11.jar!/javax/ws/rs/POST.class
2019-07-29 23:02:05.525:WARN:oeja.AnnotationParser:qtp1911152052-20: javax.ws.rs.PUT scanned from multiple locations: jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/javax.ws.rs-api-2.0-m10.jar!/javax/ws/rs/PUT.class, jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/jersey-core-1.11.jar!/javax/ws/rs/PUT.class
2019-07-29 23:02:05.526:WARN:oeja.AnnotationParser:qtp1911152052-20: javax.ws.rs.Path scanned from multiple locations: jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/javax.ws.rs-api-2.0-m10.jar!/javax/ws/rs/Path.class, jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/jersey-core-1.11.jar!/javax/ws/rs/Path.class
2019-07-29 23:02:05.526:WARN:oeja.AnnotationParser:qtp1911152052-20: javax.ws.rs.PathParam scanned from multiple locations: jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/javax.ws.rs-api-2.0-m10.jar!/javax/ws/rs/PathParam.class, jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/jersey-core-1.11.jar!/javax/ws/rs/PathParam.class
2019-07-29 23:02:05.526:WARN:oeja.AnnotationParser:qtp1911152052-20: javax.ws.rs.Produces scanned from multiple locations: jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/javax.ws.rs-api-2.0-m10.jar!/javax/ws/rs/Produces.class, jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/jersey-core-1.11.jar!/javax/ws/rs/Produces.class
2019-07-29 23:02:05.529:WARN:oeja.AnnotationParser:qtp1911152052-20: javax.ws.rs.QueryParam scanned from multiple locations: jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/javax.ws.rs-api-2.0-m10.jar!/javax/ws/rs/QueryParam.class, jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/jersey-core-1.11.jar!/javax/ws/rs/QueryParam.class
2019-07-29 23:02:05.533:WARN:oeja.AnnotationParser:qtp1911152052-20: javax.ws.rs.WebApplicationException scanned from multiple locations: jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/javax.ws.rs-api-2.0-m10.jar!/javax/ws/rs/WebApplicationException.class, jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/jersey-core-1.11.jar!/javax/ws/rs/WebApplicationException.class
2019-07-29 23:02:05.533:WARN:oeja.AnnotationParser:qtp1911152052-20: javax.ws.rs.core.Application scanned from multiple locations: jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/javax.ws.rs-api-2.0-m10.jar!/javax/ws/rs/core/Application.class, jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/jersey-core-1.11.jar!/javax/ws/rs/core/Application.class
2019-07-29 23:02:05.534:WARN:oeja.AnnotationParser:qtp1911152052-20: javax.ws.rs.core.CacheControl scanned from multiple locations: jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/javax.ws.rs-api-2.0-m10.jar!/javax/ws/rs/core/CacheControl.class, jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/jersey-core-1.11.jar!/javax/ws/rs/core/CacheControl.class
2019-07-29 23:02:05.534:WARN:oeja.AnnotationParser:qtp1911152052-20: javax.ws.rs.core.Context scanned from multiple locations: jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/javax.ws.rs-api-2.0-m10.jar!/javax/ws/rs/core/Context.class, jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/jersey-core-1.11.jar!/javax/ws/rs/core/Context.class
2019-07-29 23:02:05.534:WARN:oeja.AnnotationParser:qtp1911152052-20: javax.ws.rs.core.Cookie scanned from multiple locations: jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/javax.ws.rs-api-2.0-m10.jar!/javax/ws/rs/core/Cookie.class, jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/jersey-core-1.11.jar!/javax/ws/rs/core/Cookie.class
2019-07-29 23:02:05.535:WARN:oeja.AnnotationParser:qtp1911152052-20: javax.ws.rs.core.EntityTag scanned from multiple locations: jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/javax.ws.rs-api-2.0-m10.jar!/javax/ws/rs/core/EntityTag.class, jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/jersey-core-1.11.jar!/javax/ws/rs/core/EntityTag.class
2019-07-29 23:02:05.535:WARN:oeja.AnnotationParser:qtp1911152052-20: javax.ws.rs.core.GenericEntity scanned from multiple locations: jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/javax.ws.rs-api-2.0-m10.jar!/javax/ws/rs/core/GenericEntity.class, jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/jersey-core-1.11.jar!/javax/ws/rs/core/GenericEntity.class
2019-07-29 23:02:05.535:WARN:oeja.AnnotationParser:qtp1911152052-20: javax.ws.rs.core.HttpHeaders scanned from multiple locations: jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/javax.ws.rs-api-2.0-m10.jar!/javax/ws/rs/core/HttpHeaders.class, jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/jersey-core-1.11.jar!/javax/ws/rs/core/HttpHeaders.class
2019-07-29 23:02:05.536:WARN:oeja.AnnotationParser:qtp1911152052-20: javax.ws.rs.core.MediaType$1 scanned from multiple locations: jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/javax.ws.rs-api-2.0-m10.jar!/javax/ws/rs/core/MediaType$1.class, jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/jersey-core-1.11.jar!/javax/ws/rs/core/MediaType$1.class
2019-07-29 23:02:05.536:WARN:oeja.AnnotationParser:qtp1911152052-20: javax.ws.rs.core.MediaType scanned from multiple locations: jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/javax.ws.rs-api-2.0-m10.jar!/javax/ws/rs/core/MediaType.class, jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/jersey-core-1.11.jar!/javax/ws/rs/core/MediaType.class
2019-07-29 23:02:05.537:WARN:oeja.AnnotationParser:qtp1911152052-20: javax.ws.rs.core.MultivaluedMap scanned from multiple locations: jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/javax.ws.rs-api-2.0-m10.jar!/javax/ws/rs/core/MultivaluedMap.class, jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/jersey-core-1.11.jar!/javax/ws/rs/core/MultivaluedMap.class
2019-07-29 23:02:05.537:WARN:oeja.AnnotationParser:qtp1911152052-20: javax.ws.rs.core.NewCookie scanned from multiple locations: jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/javax.ws.rs-api-2.0-m10.jar!/javax/ws/rs/core/NewCookie.class, jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/jersey-core-1.11.jar!/javax/ws/rs/core/NewCookie.class
2019-07-29 23:02:05.542:WARN:oeja.AnnotationParser:qtp1911152052-20: javax.ws.rs.core.PathSegment scanned from multiple locations: jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/javax.ws.rs-api-2.0-m10.jar!/javax/ws/rs/core/PathSegment.class, jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/jersey-core-1.11.jar!/javax/ws/rs/core/PathSegment.class
2019-07-29 23:02:05.543:WARN:oeja.AnnotationParser:qtp1911152052-20: javax.ws.rs.core.Request scanned from multiple locations: jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/javax.ws.rs-api-2.0-m10.jar!/javax/ws/rs/core/Request.class, jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/jersey-core-1.11.jar!/javax/ws/rs/core/Request.class
2019-07-29 23:02:05.543:WARN:oeja.AnnotationParser:qtp1911152052-20: javax.ws.rs.core.Response$ResponseBuilder scanned from multiple locations: jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/javax.ws.rs-api-2.0-m10.jar!/javax/ws/rs/core/Response$ResponseBuilder.class, jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/jersey-core-1.11.jar!/javax/ws/rs/core/Response$ResponseBuilder.class
2019-07-29 23:02:05.543:WARN:oeja.AnnotationParser:qtp1911152052-20: javax.ws.rs.core.Response$Status$Family scanned from multiple locations: jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/javax.ws.rs-api-2.0-m10.jar!/javax/ws/rs/core/Response$Status$Family.class, jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/jersey-core-1.11.jar!/javax/ws/rs/core/Response$Status$Family.class
2019-07-29 23:02:05.544:WARN:oeja.AnnotationParser:qtp1911152052-20: javax.ws.rs.core.Response$Status scanned from multiple locations: jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/javax.ws.rs-api-2.0-m10.jar!/javax/ws/rs/core/Response$Status.class, jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/jersey-core-1.11.jar!/javax/ws/rs/core/Response$Status.class
2019-07-29 23:02:05.544:WARN:oeja.AnnotationParser:qtp1911152052-20: javax.ws.rs.core.Response$StatusType scanned from multiple locations: jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/javax.ws.rs-api-2.0-m10.jar!/javax/ws/rs/core/Response$StatusType.class, jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/jersey-core-1.11.jar!/javax/ws/rs/core/Response$StatusType.class
2019-07-29 23:02:05.544:WARN:oeja.AnnotationParser:qtp1911152052-20: javax.ws.rs.core.Response scanned from multiple locations: jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/javax.ws.rs-api-2.0-m10.jar!/javax/ws/rs/core/Response.class, jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/jersey-core-1.11.jar!/javax/ws/rs/core/Response.class
2019-07-29 23:02:05.545:WARN:oeja.AnnotationParser:qtp1911152052-20: javax.ws.rs.core.SecurityContext scanned from multiple locations: jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/javax.ws.rs-api-2.0-m10.jar!/javax/ws/rs/core/SecurityContext.class, jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/jersey-core-1.11.jar!/javax/ws/rs/core/SecurityContext.class
2019-07-29 23:02:05.545:WARN:oeja.AnnotationParser:qtp1911152052-20: javax.ws.rs.core.StreamingOutput scanned from multiple locations: jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/javax.ws.rs-api-2.0-m10.jar!/javax/ws/rs/core/StreamingOutput.class, jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/jersey-core-1.11.jar!/javax/ws/rs/core/StreamingOutput.class
2019-07-29 23:02:05.546:WARN:oeja.AnnotationParser:qtp1911152052-20: javax.ws.rs.core.UriBuilder scanned from multiple locations: jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/javax.ws.rs-api-2.0-m10.jar!/javax/ws/rs/core/UriBuilder.class, jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/jersey-core-1.11.jar!/javax/ws/rs/core/UriBuilder.class
2019-07-29 23:02:05.546:WARN:oeja.AnnotationParser:qtp1911152052-20: javax.ws.rs.core.UriBuilderException scanned from multiple locations: jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/javax.ws.rs-api-2.0-m10.jar!/javax/ws/rs/core/UriBuilderException.class, jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/jersey-core-1.11.jar!/javax/ws/rs/core/UriBuilderException.class
2019-07-29 23:02:05.547:WARN:oeja.AnnotationParser:qtp1911152052-20: javax.ws.rs.core.UriInfo scanned from multiple locations: jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/javax.ws.rs-api-2.0-m10.jar!/javax/ws/rs/core/UriInfo.class, jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/jersey-core-1.11.jar!/javax/ws/rs/core/UriInfo.class
2019-07-29 23:02:05.547:WARN:oeja.AnnotationParser:qtp1911152052-20: javax.ws.rs.core.Variant$VariantListBuilder scanned from multiple locations: jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/javax.ws.rs-api-2.0-m10.jar!/javax/ws/rs/core/Variant$VariantListBuilder.class, jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/jersey-core-1.11.jar!/javax/ws/rs/core/Variant$VariantListBuilder.class
2019-07-29 23:02:05.548:WARN:oeja.AnnotationParser:qtp1911152052-20: javax.ws.rs.core.Variant scanned from multiple locations: jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/javax.ws.rs-api-2.0-m10.jar!/javax/ws/rs/core/Variant.class, jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/jersey-core-1.11.jar!/javax/ws/rs/core/Variant.class
2019-07-29 23:02:05.548:WARN:oeja.AnnotationParser:qtp1911152052-20: javax.ws.rs.ext.ContextResolver scanned from multiple locations: jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/javax.ws.rs-api-2.0-m10.jar!/javax/ws/rs/ext/ContextResolver.class, jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/jersey-core-1.11.jar!/javax/ws/rs/ext/ContextResolver.class
2019-07-29 23:02:05.548:WARN:oeja.AnnotationParser:qtp1911152052-20: javax.ws.rs.ext.ExceptionMapper scanned from multiple locations: jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/javax.ws.rs-api-2.0-m10.jar!/javax/ws/rs/ext/ExceptionMapper.class, jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/jersey-core-1.11.jar!/javax/ws/rs/ext/ExceptionMapper.class
2019-07-29 23:02:05.550:WARN:oeja.AnnotationParser:qtp1911152052-20: javax.ws.rs.ext.FactoryFinder$1 scanned from multiple locations: jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/javax.ws.rs-api-2.0-m10.jar!/javax/ws/rs/ext/FactoryFinder$1.class, jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/jersey-core-1.11.jar!/javax/ws/rs/ext/FactoryFinder$1.class
2019-07-29 23:02:05.550:WARN:oeja.AnnotationParser:qtp1911152052-20: javax.ws.rs.ext.FactoryFinder scanned from multiple locations: jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/javax.ws.rs-api-2.0-m10.jar!/javax/ws/rs/ext/FactoryFinder.class, jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/jersey-core-1.11.jar!/javax/ws/rs/ext/FactoryFinder.class
2019-07-29 23:02:05.553:WARN:oeja.AnnotationParser:qtp1911152052-20: javax.ws.rs.ext.MessageBodyReader scanned from multiple locations: jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/javax.ws.rs-api-2.0-m10.jar!/javax/ws/rs/ext/MessageBodyReader.class, jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/jersey-core-1.11.jar!/javax/ws/rs/ext/MessageBodyReader.class
2019-07-29 23:02:05.553:WARN:oeja.AnnotationParser:qtp1911152052-20: javax.ws.rs.ext.MessageBodyWriter scanned from multiple locations: jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/javax.ws.rs-api-2.0-m10.jar!/javax/ws/rs/ext/MessageBodyWriter.class, jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/jersey-core-1.11.jar!/javax/ws/rs/ext/MessageBodyWriter.class
2019-07-29 23:02:05.554:WARN:oeja.AnnotationParser:qtp1911152052-20: javax.ws.rs.ext.Provider scanned from multiple locations: jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/javax.ws.rs-api-2.0-m10.jar!/javax/ws/rs/ext/Provider.class, jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/jersey-core-1.11.jar!/javax/ws/rs/ext/Provider.class
2019-07-29 23:02:05.554:WARN:oeja.AnnotationParser:qtp1911152052-20: javax.ws.rs.ext.Providers scanned from multiple locations: jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/javax.ws.rs-api-2.0-m10.jar!/javax/ws/rs/ext/Providers.class, jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/jersey-core-1.11.jar!/javax/ws/rs/ext/Providers.class
2019-07-29 23:02:05.555:WARN:oeja.AnnotationParser:qtp1911152052-20: javax.ws.rs.ext.RuntimeDelegate$HeaderDelegate scanned from multiple locations: jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/javax.ws.rs-api-2.0-m10.jar!/javax/ws/rs/ext/RuntimeDelegate$HeaderDelegate.class, jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/jersey-core-1.11.jar!/javax/ws/rs/ext/RuntimeDelegate$HeaderDelegate.class
2019-07-29 23:02:05.555:WARN:oeja.AnnotationParser:qtp1911152052-20: javax.ws.rs.ext.RuntimeDelegate scanned from multiple locations: jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/javax.ws.rs-api-2.0-m10.jar!/javax/ws/rs/ext/RuntimeDelegate.class, jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/jersey-core-1.11.jar!/javax/ws/rs/ext/RuntimeDelegate.class
2019-07-29 23:02:05.711:WARN:oeja.AnnotationParser:qtp1911152052-20: javax.ws.rs.ApplicationPath scanned from multiple locations: jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/javax.ws.rs-api-2.0-m10.jar!/javax/ws/rs/ApplicationPath.class, jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/jsr311-api-1.1.1.jar!/javax/ws/rs/ApplicationPath.class
2019-07-29 23:02:05.711:WARN:oeja.AnnotationParser:qtp1911152052-20: javax.ws.rs.Consumes scanned from multiple locations: jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/javax.ws.rs-api-2.0-m10.jar!/javax/ws/rs/Consumes.class, jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/jsr311-api-1.1.1.jar!/javax/ws/rs/Consumes.class
2019-07-29 23:02:05.711:WARN:oeja.AnnotationParser:qtp1911152052-20: javax.ws.rs.CookieParam scanned from multiple locations: jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/javax.ws.rs-api-2.0-m10.jar!/javax/ws/rs/CookieParam.class, jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/jsr311-api-1.1.1.jar!/javax/ws/rs/CookieParam.class
2019-07-29 23:02:05.711:WARN:oeja.AnnotationParser:qtp1911152052-20: javax.ws.rs.DELETE scanned from multiple locations: jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/javax.ws.rs-api-2.0-m10.jar!/javax/ws/rs/DELETE.class, jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/jsr311-api-1.1.1.jar!/javax/ws/rs/DELETE.class
2019-07-29 23:02:05.711:WARN:oeja.AnnotationParser:qtp1911152052-20: javax.ws.rs.DefaultValue scanned from multiple locations: jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/javax.ws.rs-api-2.0-m10.jar!/javax/ws/rs/DefaultValue.class, jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/jsr311-api-1.1.1.jar!/javax/ws/rs/DefaultValue.class
2019-07-29 23:02:05.711:WARN:oeja.AnnotationParser:qtp1911152052-20: javax.ws.rs.Encoded scanned from multiple locations: jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/javax.ws.rs-api-2.0-m10.jar!/javax/ws/rs/Encoded.class, jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/jsr311-api-1.1.1.jar!/javax/ws/rs/Encoded.class
2019-07-29 23:02:05.712:WARN:oeja.AnnotationParser:qtp1911152052-20: javax.ws.rs.FormParam scanned from multiple locations: jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/javax.ws.rs-api-2.0-m10.jar!/javax/ws/rs/FormParam.class, jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/jsr311-api-1.1.1.jar!/javax/ws/rs/FormParam.class
2019-07-29 23:02:05.712:WARN:oeja.AnnotationParser:qtp1911152052-20: javax.ws.rs.GET scanned from multiple locations: jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/javax.ws.rs-api-2.0-m10.jar!/javax/ws/rs/GET.class, jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/jsr311-api-1.1.1.jar!/javax/ws/rs/GET.class
2019-07-29 23:02:05.712:WARN:oeja.AnnotationParser:qtp1911152052-20: javax.ws.rs.HEAD scanned from multiple locations: jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/javax.ws.rs-api-2.0-m10.jar!/javax/ws/rs/HEAD.class, jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/jsr311-api-1.1.1.jar!/javax/ws/rs/HEAD.class
2019-07-29 23:02:05.712:WARN:oeja.AnnotationParser:qtp1911152052-20: javax.ws.rs.HeaderParam scanned from multiple locations: jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/javax.ws.rs-api-2.0-m10.jar!/javax/ws/rs/HeaderParam.class, jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/jsr311-api-1.1.1.jar!/javax/ws/rs/HeaderParam.class
2019-07-29 23:02:05.712:WARN:oeja.AnnotationParser:qtp1911152052-20: javax.ws.rs.HttpMethod scanned from multiple locations: jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/javax.ws.rs-api-2.0-m10.jar!/javax/ws/rs/HttpMethod.class, jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/jsr311-api-1.1.1.jar!/javax/ws/rs/HttpMethod.class
2019-07-29 23:02:05.712:WARN:oeja.AnnotationParser:qtp1911152052-20: javax.ws.rs.MatrixParam scanned from multiple locations: jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/javax.ws.rs-api-2.0-m10.jar!/javax/ws/rs/MatrixParam.class, jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/jsr311-api-1.1.1.jar!/javax/ws/rs/MatrixParam.class
2019-07-29 23:02:05.712:WARN:oeja.AnnotationParser:qtp1911152052-20: javax.ws.rs.OPTIONS scanned from multiple locations: jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/javax.ws.rs-api-2.0-m10.jar!/javax/ws/rs/OPTIONS.class, jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/jsr311-api-1.1.1.jar!/javax/ws/rs/OPTIONS.class
2019-07-29 23:02:05.713:WARN:oeja.AnnotationParser:qtp1911152052-20: javax.ws.rs.POST scanned from multiple locations: jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/javax.ws.rs-api-2.0-m10.jar!/javax/ws/rs/POST.class, jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/jsr311-api-1.1.1.jar!/javax/ws/rs/POST.class
2019-07-29 23:02:05.713:WARN:oeja.AnnotationParser:qtp1911152052-20: javax.ws.rs.PUT scanned from multiple locations: jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/javax.ws.rs-api-2.0-m10.jar!/javax/ws/rs/PUT.class, jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/jsr311-api-1.1.1.jar!/javax/ws/rs/PUT.class
2019-07-29 23:02:05.717:WARN:oeja.AnnotationParser:qtp1911152052-20: javax.ws.rs.Path scanned from multiple locations: jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/javax.ws.rs-api-2.0-m10.jar!/javax/ws/rs/Path.class, jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/jsr311-api-1.1.1.jar!/javax/ws/rs/Path.class
2019-07-29 23:02:05.717:WARN:oeja.AnnotationParser:qtp1911152052-20: javax.ws.rs.PathParam scanned from multiple locations: jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/javax.ws.rs-api-2.0-m10.jar!/javax/ws/rs/PathParam.class, jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/jsr311-api-1.1.1.jar!/javax/ws/rs/PathParam.class
2019-07-29 23:02:05.718:WARN:oeja.AnnotationParser:qtp1911152052-20: javax.ws.rs.Produces scanned from multiple locations: jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/javax.ws.rs-api-2.0-m10.jar!/javax/ws/rs/Produces.class, jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/jsr311-api-1.1.1.jar!/javax/ws/rs/Produces.class
2019-07-29 23:02:05.718:WARN:oeja.AnnotationParser:qtp1911152052-20: javax.ws.rs.QueryParam scanned from multiple locations: jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/javax.ws.rs-api-2.0-m10.jar!/javax/ws/rs/QueryParam.class, jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/jsr311-api-1.1.1.jar!/javax/ws/rs/QueryParam.class
2019-07-29 23:02:05.718:WARN:oeja.AnnotationParser:qtp1911152052-20: javax.ws.rs.WebApplicationException scanned from multiple locations: jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/javax.ws.rs-api-2.0-m10.jar!/javax/ws/rs/WebApplicationException.class, jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/jsr311-api-1.1.1.jar!/javax/ws/rs/WebApplicationException.class
2019-07-29 23:02:05.719:WARN:oeja.AnnotationParser:qtp1911152052-20: javax.ws.rs.core.Application scanned from multiple locations: jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/javax.ws.rs-api-2.0-m10.jar!/javax/ws/rs/core/Application.class, jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/jsr311-api-1.1.1.jar!/javax/ws/rs/core/Application.class
2019-07-29 23:02:05.721:WARN:oeja.AnnotationParser:qtp1911152052-20: javax.ws.rs.core.CacheControl scanned from multiple locations: jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/javax.ws.rs-api-2.0-m10.jar!/javax/ws/rs/core/CacheControl.class, jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/jsr311-api-1.1.1.jar!/javax/ws/rs/core/CacheControl.class
2019-07-29 23:02:05.721:WARN:oeja.AnnotationParser:qtp1911152052-20: javax.ws.rs.core.Context scanned from multiple locations: jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/javax.ws.rs-api-2.0-m10.jar!/javax/ws/rs/core/Context.class, jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/jsr311-api-1.1.1.jar!/javax/ws/rs/core/Context.class
2019-07-29 23:02:05.722:WARN:oeja.AnnotationParser:qtp1911152052-20: javax.ws.rs.core.Cookie scanned from multiple locations: jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/javax.ws.rs-api-2.0-m10.jar!/javax/ws/rs/core/Cookie.class, jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/jsr311-api-1.1.1.jar!/javax/ws/rs/core/Cookie.class
2019-07-29 23:02:05.722:WARN:oeja.AnnotationParser:qtp1911152052-20: javax.ws.rs.core.EntityTag scanned from multiple locations: jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/javax.ws.rs-api-2.0-m10.jar!/javax/ws/rs/core/EntityTag.class, jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/jsr311-api-1.1.1.jar!/javax/ws/rs/core/EntityTag.class
2019-07-29 23:02:05.722:WARN:oeja.AnnotationParser:qtp1911152052-20: javax.ws.rs.core.GenericEntity scanned from multiple locations: jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/javax.ws.rs-api-2.0-m10.jar!/javax/ws/rs/core/GenericEntity.class, jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/jsr311-api-1.1.1.jar!/javax/ws/rs/core/GenericEntity.class
2019-07-29 23:02:05.723:WARN:oeja.AnnotationParser:qtp1911152052-20: javax.ws.rs.core.HttpHeaders scanned from multiple locations: jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/javax.ws.rs-api-2.0-m10.jar!/javax/ws/rs/core/HttpHeaders.class, jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/jsr311-api-1.1.1.jar!/javax/ws/rs/core/HttpHeaders.class
2019-07-29 23:02:05.723:WARN:oeja.AnnotationParser:qtp1911152052-20: javax.ws.rs.core.MediaType$1 scanned from multiple locations: jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/javax.ws.rs-api-2.0-m10.jar!/javax/ws/rs/core/MediaType$1.class, jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/jsr311-api-1.1.1.jar!/javax/ws/rs/core/MediaType$1.class
2019-07-29 23:02:05.723:WARN:oeja.AnnotationParser:qtp1911152052-20: javax.ws.rs.core.MediaType scanned from multiple locations: jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/javax.ws.rs-api-2.0-m10.jar!/javax/ws/rs/core/MediaType.class, jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/jsr311-api-1.1.1.jar!/javax/ws/rs/core/MediaType.class
2019-07-29 23:02:05.725:WARN:oeja.AnnotationParser:qtp1911152052-20: javax.ws.rs.core.MultivaluedMap scanned from multiple locations: jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/javax.ws.rs-api-2.0-m10.jar!/javax/ws/rs/core/MultivaluedMap.class, jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/jsr311-api-1.1.1.jar!/javax/ws/rs/core/MultivaluedMap.class
2019-07-29 23:02:05.725:WARN:oeja.AnnotationParser:qtp1911152052-20: javax.ws.rs.core.NewCookie scanned from multiple locations: jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/javax.ws.rs-api-2.0-m10.jar!/javax/ws/rs/core/NewCookie.class, jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/jsr311-api-1.1.1.jar!/javax/ws/rs/core/NewCookie.class
2019-07-29 23:02:05.726:WARN:oeja.AnnotationParser:qtp1911152052-20: javax.ws.rs.core.PathSegment scanned from multiple locations: jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/javax.ws.rs-api-2.0-m10.jar!/javax/ws/rs/core/PathSegment.class, jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/jsr311-api-1.1.1.jar!/javax/ws/rs/core/PathSegment.class
2019-07-29 23:02:05.726:WARN:oeja.AnnotationParser:qtp1911152052-20: javax.ws.rs.core.Request scanned from multiple locations: jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/javax.ws.rs-api-2.0-m10.jar!/javax/ws/rs/core/Request.class, jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/jsr311-api-1.1.1.jar!/javax/ws/rs/core/Request.class
2019-07-29 23:02:05.729:WARN:oeja.AnnotationParser:qtp1911152052-20: javax.ws.rs.core.Response$ResponseBuilder scanned from multiple locations: jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/javax.ws.rs-api-2.0-m10.jar!/javax/ws/rs/core/Response$ResponseBuilder.class, jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/jsr311-api-1.1.1.jar!/javax/ws/rs/core/Response$ResponseBuilder.class
2019-07-29 23:02:05.737:WARN:oeja.AnnotationParser:qtp1911152052-20: javax.ws.rs.core.Response$Status$Family scanned from multiple locations: jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/javax.ws.rs-api-2.0-m10.jar!/javax/ws/rs/core/Response$Status$Family.class, jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/jsr311-api-1.1.1.jar!/javax/ws/rs/core/Response$Status$Family.class
2019-07-29 23:02:05.737:WARN:oeja.AnnotationParser:qtp1911152052-20: javax.ws.rs.core.Response$Status scanned from multiple locations: jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/javax.ws.rs-api-2.0-m10.jar!/javax/ws/rs/core/Response$Status.class, jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/jsr311-api-1.1.1.jar!/javax/ws/rs/core/Response$Status.class
2019-07-29 23:02:05.738:WARN:oeja.AnnotationParser:qtp1911152052-20: javax.ws.rs.core.Response$StatusType scanned from multiple locations: jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/javax.ws.rs-api-2.0-m10.jar!/javax/ws/rs/core/Response$StatusType.class, jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/jsr311-api-1.1.1.jar!/javax/ws/rs/core/Response$StatusType.class
2019-07-29 23:02:05.738:WARN:oeja.AnnotationParser:qtp1911152052-20: javax.ws.rs.core.Response scanned from multiple locations: jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/javax.ws.rs-api-2.0-m10.jar!/javax/ws/rs/core/Response.class, jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/jsr311-api-1.1.1.jar!/javax/ws/rs/core/Response.class
2019-07-29 23:02:05.739:WARN:oeja.AnnotationParser:qtp1911152052-20: javax.ws.rs.core.SecurityContext scanned from multiple locations: jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/javax.ws.rs-api-2.0-m10.jar!/javax/ws/rs/core/SecurityContext.class, jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/jsr311-api-1.1.1.jar!/javax/ws/rs/core/SecurityContext.class
2019-07-29 23:02:05.739:WARN:oeja.AnnotationParser:qtp1911152052-20: javax.ws.rs.core.StreamingOutput scanned from multiple locations: jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/javax.ws.rs-api-2.0-m10.jar!/javax/ws/rs/core/StreamingOutput.class, jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/jsr311-api-1.1.1.jar!/javax/ws/rs/core/StreamingOutput.class
2019-07-29 23:02:05.739:WARN:oeja.AnnotationParser:qtp1911152052-20: javax.ws.rs.core.UriBuilder scanned from multiple locations: jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/javax.ws.rs-api-2.0-m10.jar!/javax/ws/rs/core/UriBuilder.class, jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/jsr311-api-1.1.1.jar!/javax/ws/rs/core/UriBuilder.class
2019-07-29 23:02:05.740:WARN:oeja.AnnotationParser:qtp1911152052-20: javax.ws.rs.core.UriBuilderException scanned from multiple locations: jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/javax.ws.rs-api-2.0-m10.jar!/javax/ws/rs/core/UriBuilderException.class, jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/jsr311-api-1.1.1.jar!/javax/ws/rs/core/UriBuilderException.class
2019-07-29 23:02:05.740:WARN:oeja.AnnotationParser:qtp1911152052-20: javax.ws.rs.core.UriInfo scanned from multiple locations: jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/javax.ws.rs-api-2.0-m10.jar!/javax/ws/rs/core/UriInfo.class, jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/jsr311-api-1.1.1.jar!/javax/ws/rs/core/UriInfo.class
2019-07-29 23:02:05.740:WARN:oeja.AnnotationParser:qtp1911152052-20: javax.ws.rs.core.Variant$VariantListBuilder scanned from multiple locations: jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/javax.ws.rs-api-2.0-m10.jar!/javax/ws/rs/core/Variant$VariantListBuilder.class, jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/jsr311-api-1.1.1.jar!/javax/ws/rs/core/Variant$VariantListBuilder.class
2019-07-29 23:02:05.741:WARN:oeja.AnnotationParser:qtp1911152052-20: javax.ws.rs.core.Variant scanned from multiple locations: jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/javax.ws.rs-api-2.0-m10.jar!/javax/ws/rs/core/Variant.class, jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/jsr311-api-1.1.1.jar!/javax/ws/rs/core/Variant.class
2019-07-29 23:02:05.749:WARN:oeja.AnnotationParser:qtp1911152052-20: javax.ws.rs.ext.ContextResolver scanned from multiple locations: jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/javax.ws.rs-api-2.0-m10.jar!/javax/ws/rs/ext/ContextResolver.class, jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/jsr311-api-1.1.1.jar!/javax/ws/rs/ext/ContextResolver.class
2019-07-29 23:02:05.749:WARN:oeja.AnnotationParser:qtp1911152052-20: javax.ws.rs.ext.ExceptionMapper scanned from multiple locations: jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/javax.ws.rs-api-2.0-m10.jar!/javax/ws/rs/ext/ExceptionMapper.class, jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/jsr311-api-1.1.1.jar!/javax/ws/rs/ext/ExceptionMapper.class
2019-07-29 23:02:05.750:WARN:oeja.AnnotationParser:qtp1911152052-20: javax.ws.rs.ext.FactoryFinder$1 scanned from multiple locations: jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/javax.ws.rs-api-2.0-m10.jar!/javax/ws/rs/ext/FactoryFinder$1.class, jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/jsr311-api-1.1.1.jar!/javax/ws/rs/ext/FactoryFinder$1.class
2019-07-29 23:02:05.750:WARN:oeja.AnnotationParser:qtp1911152052-20: javax.ws.rs.ext.FactoryFinder scanned from multiple locations: jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/javax.ws.rs-api-2.0-m10.jar!/javax/ws/rs/ext/FactoryFinder.class, jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/jsr311-api-1.1.1.jar!/javax/ws/rs/ext/FactoryFinder.class
2019-07-29 23:02:05.753:WARN:oeja.AnnotationParser:qtp1911152052-20: javax.ws.rs.ext.MessageBodyReader scanned from multiple locations: jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/javax.ws.rs-api-2.0-m10.jar!/javax/ws/rs/ext/MessageBodyReader.class, jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/jsr311-api-1.1.1.jar!/javax/ws/rs/ext/MessageBodyReader.class
2019-07-29 23:02:05.753:WARN:oeja.AnnotationParser:qtp1911152052-20: javax.ws.rs.ext.MessageBodyWriter scanned from multiple locations: jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/javax.ws.rs-api-2.0-m10.jar!/javax/ws/rs/ext/MessageBodyWriter.class, jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/jsr311-api-1.1.1.jar!/javax/ws/rs/ext/MessageBodyWriter.class
2019-07-29 23:02:05.753:WARN:oeja.AnnotationParser:qtp1911152052-20: javax.ws.rs.ext.Provider scanned from multiple locations: jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/javax.ws.rs-api-2.0-m10.jar!/javax/ws/rs/ext/Provider.class, jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/jsr311-api-1.1.1.jar!/javax/ws/rs/ext/Provider.class
2019-07-29 23:02:05.754:WARN:oeja.AnnotationParser:qtp1911152052-20: javax.ws.rs.ext.Providers scanned from multiple locations: jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/javax.ws.rs-api-2.0-m10.jar!/javax/ws/rs/ext/Providers.class, jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/jsr311-api-1.1.1.jar!/javax/ws/rs/ext/Providers.class
2019-07-29 23:02:05.761:WARN:oeja.AnnotationParser:qtp1911152052-20: javax.ws.rs.ext.RuntimeDelegate$HeaderDelegate scanned from multiple locations: jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/javax.ws.rs-api-2.0-m10.jar!/javax/ws/rs/ext/RuntimeDelegate$HeaderDelegate.class, jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/jsr311-api-1.1.1.jar!/javax/ws/rs/ext/RuntimeDelegate$HeaderDelegate.class
2019-07-29 23:02:05.761:WARN:oeja.AnnotationParser:qtp1911152052-20: javax.ws.rs.ext.RuntimeDelegate scanned from multiple locations: jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/javax.ws.rs-api-2.0-m10.jar!/javax/ws/rs/ext/RuntimeDelegate.class, jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/jsr311-api-1.1.1.jar!/javax/ws/rs/ext/RuntimeDelegate.class
[7.247s][info][gc] GC(3) Pause Young (Normal) (G1 Evacuation Pause) 202M->25M(4096M) 76.045ms
2019-07-29 23:02:06.638:WARN:oeja.AnnotationParser:qtp1911152052-18: org.apache.jute.BinaryInputArchive$BinaryIndex scanned from multiple locations: jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/zkclient-0.1.0.jar!/org/apache/jute/BinaryInputArchive$BinaryIndex.class, jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/zookeeper-3.4.5.jar!/org/apache/jute/BinaryInputArchive$BinaryIndex.class
2019-07-29 23:02:06.639:WARN:oeja.AnnotationParser:qtp1911152052-18: org.apache.jute.BinaryInputArchive scanned from multiple locations: jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/zkclient-0.1.0.jar!/org/apache/jute/BinaryInputArchive.class, jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/zookeeper-3.4.5.jar!/org/apache/jute/BinaryInputArchive.class
2019-07-29 23:02:06.639:WARN:oeja.AnnotationParser:qtp1911152052-18: org.apache.jute.BinaryOutputArchive scanned from multiple locations: jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/zkclient-0.1.0.jar!/org/apache/jute/BinaryOutputArchive.class, jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/zookeeper-3.4.5.jar!/org/apache/jute/BinaryOutputArchive.class
2019-07-29 23:02:06.639:WARN:oeja.AnnotationParser:qtp1911152052-18: org.apache.jute.CsvOutputArchive scanned from multiple locations: jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/zkclient-0.1.0.jar!/org/apache/jute/CsvOutputArchive.class, jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/zookeeper-3.4.5.jar!/org/apache/jute/CsvOutputArchive.class
2019-07-29 23:02:06.639:WARN:oeja.AnnotationParser:qtp1911152052-18: org.apache.jute.Index scanned from multiple locations: jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/zkclient-0.1.0.jar!/org/apache/jute/Index.class, jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/zookeeper-3.4.5.jar!/org/apache/jute/Index.class
2019-07-29 23:02:06.640:WARN:oeja.AnnotationParser:qtp1911152052-18: org.apache.jute.InputArchive scanned from multiple locations: jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/zkclient-0.1.0.jar!/org/apache/jute/InputArchive.class, jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/zookeeper-3.4.5.jar!/org/apache/jute/InputArchive.class
2019-07-29 23:02:06.640:WARN:oeja.AnnotationParser:qtp1911152052-18: org.apache.jute.OutputArchive scanned from multiple locations: jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/zkclient-0.1.0.jar!/org/apache/jute/OutputArchive.class, jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/zookeeper-3.4.5.jar!/org/apache/jute/OutputArchive.class
2019-07-29 23:02:06.640:WARN:oeja.AnnotationParser:qtp1911152052-18: org.apache.jute.Record scanned from multiple locations: jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/zkclient-0.1.0.jar!/org/apache/jute/Record.class, jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/zookeeper-3.4.5.jar!/org/apache/jute/Record.class
2019-07-29 23:02:06.640:WARN:oeja.AnnotationParser:qtp1911152052-18: org.apache.jute.Utils scanned from multiple locations: jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/zkclient-0.1.0.jar!/org/apache/jute/Utils.class, jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/zookeeper-3.4.5.jar!/org/apache/jute/Utils.class
2019-07-29 23:02:06.657:WARN:oeja.AnnotationParser:qtp1911152052-18: org.apache.zookeeper.AsyncCallback$ACLCallback scanned from multiple locations: jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/zkclient-0.1.0.jar!/org/apache/zookeeper/AsyncCallback$ACLCallback.class, jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/zookeeper-3.4.5.jar!/org/apache/zookeeper/AsyncCallback$ACLCallback.class
2019-07-29 23:02:06.665:WARN:oeja.AnnotationParser:qtp1911152052-18: org.apache.zookeeper.AsyncCallback$ChildrenCallback scanned from multiple locations: jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/zkclient-0.1.0.jar!/org/apache/zookeeper/AsyncCallback$ChildrenCallback.class, jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/zookeeper-3.4.5.jar!/org/apache/zookeeper/AsyncCallback$ChildrenCallback.class
2019-07-29 23:02:06.669:WARN:oeja.AnnotationParser:qtp1911152052-18: org.apache.zookeeper.AsyncCallback$DataCallback scanned from multiple locations: jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/zkclient-0.1.0.jar!/org/apache/zookeeper/AsyncCallback$DataCallback.class, jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/zookeeper-3.4.5.jar!/org/apache/zookeeper/AsyncCallback$DataCallback.class
2019-07-29 23:02:06.673:WARN:oeja.AnnotationParser:qtp1911152052-18: org.apache.zookeeper.AsyncCallback$StatCallback scanned from multiple locations: jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/zkclient-0.1.0.jar!/org/apache/zookeeper/AsyncCallback$StatCallback.class, jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/zookeeper-3.4.5.jar!/org/apache/zookeeper/AsyncCallback$StatCallback.class
2019-07-29 23:02:06.673:WARN:oeja.AnnotationParser:qtp1911152052-18: org.apache.zookeeper.AsyncCallback$StringCallback scanned from multiple locations: jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/zkclient-0.1.0.jar!/org/apache/zookeeper/AsyncCallback$StringCallback.class, jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/zookeeper-3.4.5.jar!/org/apache/zookeeper/AsyncCallback$StringCallback.class
2019-07-29 23:02:06.674:WARN:oeja.AnnotationParser:qtp1911152052-18: org.apache.zookeeper.AsyncCallback$VoidCallback scanned from multiple locations: jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/zkclient-0.1.0.jar!/org/apache/zookeeper/AsyncCallback$VoidCallback.class, jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/zookeeper-3.4.5.jar!/org/apache/zookeeper/AsyncCallback$VoidCallback.class
2019-07-29 23:02:06.677:WARN:oeja.AnnotationParser:qtp1911152052-18: org.apache.zookeeper.AsyncCallback scanned from multiple locations: jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/zkclient-0.1.0.jar!/org/apache/zookeeper/AsyncCallback.class, jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/zookeeper-3.4.5.jar!/org/apache/zookeeper/AsyncCallback.class
2019-07-29 23:02:06.677:WARN:oeja.AnnotationParser:qtp1911152052-18: org.apache.zookeeper.ClientCnxn$1 scanned from multiple locations: jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/zkclient-0.1.0.jar!/org/apache/zookeeper/ClientCnxn$1.class, jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/zookeeper-3.4.5.jar!/org/apache/zookeeper/ClientCnxn$1.class
2019-07-29 23:02:06.678:WARN:oeja.AnnotationParser:qtp1911152052-18: org.apache.zookeeper.ClientCnxn$2 scanned from multiple locations: jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/zkclient-0.1.0.jar!/org/apache/zookeeper/ClientCnxn$2.class, jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/zookeeper-3.4.5.jar!/org/apache/zookeeper/ClientCnxn$2.class
2019-07-29 23:02:06.681:WARN:oeja.AnnotationParser:qtp1911152052-18: org.apache.zookeeper.ClientCnxn$AuthData scanned from multiple locations: jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/zkclient-0.1.0.jar!/org/apache/zookeeper/ClientCnxn$AuthData.class, jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/zookeeper-3.4.5.jar!/org/apache/zookeeper/ClientCnxn$AuthData.class
2019-07-29 23:02:06.681:WARN:oeja.AnnotationParser:qtp1911152052-18: org.apache.zookeeper.ClientCnxn$EventThread scanned from multiple locations: jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/zkclient-0.1.0.jar!/org/apache/zookeeper/ClientCnxn$EventThread.class, jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/zookeeper-3.4.5.jar!/org/apache/zookeeper/ClientCnxn$EventThread.class
2019-07-29 23:02:06.682:WARN:oeja.AnnotationParser:qtp1911152052-18: org.apache.zookeeper.ClientCnxn$Packet scanned from multiple locations: jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/zkclient-0.1.0.jar!/org/apache/zookeeper/ClientCnxn$Packet.class, jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/zookeeper-3.4.5.jar!/org/apache/zookeeper/ClientCnxn$Packet.class
2019-07-29 23:02:06.685:WARN:oeja.AnnotationParser:qtp1911152052-18: org.apache.zookeeper.ClientCnxn$SendThread scanned from multiple locations: jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/zkclient-0.1.0.jar!/org/apache/zookeeper/ClientCnxn$SendThread.class, jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/zookeeper-3.4.5.jar!/org/apache/zookeeper/ClientCnxn$SendThread.class
2019-07-29 23:02:06.686:WARN:oeja.AnnotationParser:qtp1911152052-18: org.apache.zookeeper.ClientCnxn$WatcherSetEventPair scanned from multiple locations: jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/zkclient-0.1.0.jar!/org/apache/zookeeper/ClientCnxn$WatcherSetEventPair.class, jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/zookeeper-3.4.5.jar!/org/apache/zookeeper/ClientCnxn$WatcherSetEventPair.class
2019-07-29 23:02:06.689:WARN:oeja.AnnotationParser:qtp1911152052-18: org.apache.zookeeper.ClientCnxn scanned from multiple locations: jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/zkclient-0.1.0.jar!/org/apache/zookeeper/ClientCnxn.class, jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/zookeeper-3.4.5.jar!/org/apache/zookeeper/ClientCnxn.class
2019-07-29 23:02:06.690:WARN:oeja.AnnotationParser:qtp1911152052-18: org.apache.zookeeper.ClientWatchManager scanned from multiple locations: jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/zkclient-0.1.0.jar!/org/apache/zookeeper/ClientWatchManager.class, jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/zookeeper-3.4.5.jar!/org/apache/zookeeper/ClientWatchManager.class
2019-07-29 23:02:06.693:WARN:oeja.AnnotationParser:qtp1911152052-18: org.apache.zookeeper.CreateMode scanned from multiple locations: jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/zkclient-0.1.0.jar!/org/apache/zookeeper/CreateMode.class, jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/zookeeper-3.4.5.jar!/org/apache/zookeeper/CreateMode.class
2019-07-29 23:02:06.694:WARN:oeja.AnnotationParser:qtp1911152052-18: org.apache.zookeeper.KeeperException$1 scanned from multiple locations: jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/zkclient-0.1.0.jar!/org/apache/zookeeper/KeeperException$1.class, jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/zookeeper-3.4.5.jar!/org/apache/zookeeper/KeeperException$1.class
2019-07-29 23:02:06.697:WARN:oeja.AnnotationParser:qtp1911152052-18: org.apache.zookeeper.KeeperException$APIErrorException scanned from multiple locations: jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/zkclient-0.1.0.jar!/org/apache/zookeeper/KeeperException$APIErrorException.class, jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/zookeeper-3.4.5.jar!/org/apache/zookeeper/KeeperException$APIErrorException.class
2019-07-29 23:02:06.697:WARN:oeja.AnnotationParser:qtp1911152052-18: org.apache.zookeeper.KeeperException$AuthFailedException scanned from multiple locations: jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/zkclient-0.1.0.jar!/org/apache/zookeeper/KeeperException$AuthFailedException.class, jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/zookeeper-3.4.5.jar!/org/apache/zookeeper/KeeperException$AuthFailedException.class
2019-07-29 23:02:06.698:WARN:oeja.AnnotationParser:qtp1911152052-18: org.apache.zookeeper.KeeperException$BadArgumentsException scanned from multiple locations: jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/zkclient-0.1.0.jar!/org/apache/zookeeper/KeeperException$BadArgumentsException.class, jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/zookeeper-3.4.5.jar!/org/apache/zookeeper/KeeperException$BadArgumentsException.class
2019-07-29 23:02:06.701:WARN:oeja.AnnotationParser:qtp1911152052-18: org.apache.zookeeper.KeeperException$BadVersionException scanned from multiple locations: jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/zkclient-0.1.0.jar!/org/apache/zookeeper/KeeperException$BadVersionException.class, jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/zookeeper-3.4.5.jar!/org/apache/zookeeper/KeeperException$BadVersionException.class
2019-07-29 23:02:06.702:WARN:oeja.AnnotationParser:qtp1911152052-18: org.apache.zookeeper.KeeperException$Code scanned from multiple locations: jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/zkclient-0.1.0.jar!/org/apache/zookeeper/KeeperException$Code.class, jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/zookeeper-3.4.5.jar!/org/apache/zookeeper/KeeperException$Code.class
2019-07-29 23:02:06.705:WARN:oeja.AnnotationParser:qtp1911152052-18: org.apache.zookeeper.KeeperException$CodeDeprecated scanned from multiple locations: jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/zkclient-0.1.0.jar!/org/apache/zookeeper/KeeperException$CodeDeprecated.class, jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/zookeeper-3.4.5.jar!/org/apache/zookeeper/KeeperException$CodeDeprecated.class
2019-07-29 23:02:06.706:WARN:oeja.AnnotationParser:qtp1911152052-18: org.apache.zookeeper.KeeperException$ConnectionLossException scanned from multiple locations: jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/zkclient-0.1.0.jar!/org/apache/zookeeper/KeeperException$ConnectionLossException.class, jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/zookeeper-3.4.5.jar!/org/apache/zookeeper/KeeperException$ConnectionLossException.class
2019-07-29 23:02:06.706:WARN:oeja.AnnotationParser:qtp1911152052-18: org.apache.zookeeper.KeeperException$DataInconsistencyException scanned from multiple locations: jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/zkclient-0.1.0.jar!/org/apache/zookeeper/KeeperException$DataInconsistencyException.class, jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/zookeeper-3.4.5.jar!/org/apache/zookeeper/KeeperException$DataInconsistencyException.class
2019-07-29 23:02:06.709:WARN:oeja.AnnotationParser:qtp1911152052-18: org.apache.zookeeper.KeeperException$InvalidACLException scanned from multiple locations: jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/zkclient-0.1.0.jar!/org/apache/zookeeper/KeeperException$InvalidACLException.class, jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/zookeeper-3.4.5.jar!/org/apache/zookeeper/KeeperException$InvalidACLException.class
2019-07-29 23:02:06.709:WARN:oeja.AnnotationParser:qtp1911152052-18: org.apache.zookeeper.KeeperException$InvalidCallbackException scanned from multiple locations: jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/zkclient-0.1.0.jar!/org/apache/zookeeper/KeeperException$InvalidCallbackException.class, jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/zookeeper-3.4.5.jar!/org/apache/zookeeper/KeeperException$InvalidCallbackException.class
2019-07-29 23:02:06.710:WARN:oeja.AnnotationParser:qtp1911152052-18: org.apache.zookeeper.KeeperException$MarshallingErrorException scanned from multiple locations: jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/zkclient-0.1.0.jar!/org/apache/zookeeper/KeeperException$MarshallingErrorException.class, jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/zookeeper-3.4.5.jar!/org/apache/zookeeper/KeeperException$MarshallingErrorException.class
2019-07-29 23:02:06.713:WARN:oeja.AnnotationParser:qtp1911152052-18: org.apache.zookeeper.KeeperException$NoAuthException scanned from multiple locations: jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/zkclient-0.1.0.jar!/org/apache/zookeeper/KeeperException$NoAuthException.class, jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/zookeeper-3.4.5.jar!/org/apache/zookeeper/KeeperException$NoAuthException.class
2019-07-29 23:02:06.713:WARN:oeja.AnnotationParser:qtp1911152052-18: org.apache.zookeeper.KeeperException$NoChildrenForEphemeralsException scanned from multiple locations: jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/zkclient-0.1.0.jar!/org/apache/zookeeper/KeeperException$NoChildrenForEphemeralsException.class, jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/zookeeper-3.4.5.jar!/org/apache/zookeeper/KeeperException$NoChildrenForEphemeralsException.class
2019-07-29 23:02:06.714:WARN:oeja.AnnotationParser:qtp1911152052-18: org.apache.zookeeper.KeeperException$NoNodeException scanned from multiple locations: jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/zkclient-0.1.0.jar!/org/apache/zookeeper/KeeperException$NoNodeException.class, jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/zookeeper-3.4.5.jar!/org/apache/zookeeper/KeeperException$NoNodeException.class
2019-07-29 23:02:06.714:WARN:oeja.AnnotationParser:qtp1911152052-18: org.apache.zookeeper.KeeperException$NodeExistsException scanned from multiple locations: jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/zkclient-0.1.0.jar!/org/apache/zookeeper/KeeperException$NodeExistsException.class, jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/zookeeper-3.4.5.jar!/org/apache/zookeeper/KeeperException$NodeExistsException.class
2019-07-29 23:02:06.717:WARN:oeja.AnnotationParser:qtp1911152052-18: org.apache.zookeeper.KeeperException$NotEmptyException scanned from multiple locations: jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/zkclient-0.1.0.jar!/org/apache/zookeeper/KeeperException$NotEmptyException.class, jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/zookeeper-3.4.5.jar!/org/apache/zookeeper/KeeperException$NotEmptyException.class
2019-07-29 23:02:06.717:WARN:oeja.AnnotationParser:qtp1911152052-18: org.apache.zookeeper.KeeperException$OperationTimeoutException scanned from multiple locations: jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/zkclient-0.1.0.jar!/org/apache/zookeeper/KeeperException$OperationTimeoutException.class, jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/zookeeper-3.4.5.jar!/org/apache/zookeeper/KeeperException$OperationTimeoutException.class
2019-07-29 23:02:06.718:WARN:oeja.AnnotationParser:qtp1911152052-18: org.apache.zookeeper.KeeperException$RuntimeInconsistencyException scanned from multiple locations: jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/zkclient-0.1.0.jar!/org/apache/zookeeper/KeeperException$RuntimeInconsistencyException.class, jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/zookeeper-3.4.5.jar!/org/apache/zookeeper/KeeperException$RuntimeInconsistencyException.class
2019-07-29 23:02:06.721:WARN:oeja.AnnotationParser:qtp1911152052-18: org.apache.zookeeper.KeeperException$SessionExpiredException scanned from multiple locations: jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/zkclient-0.1.0.jar!/org/apache/zookeeper/KeeperException$SessionExpiredException.class, jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/zookeeper-3.4.5.jar!/org/apache/zookeeper/KeeperException$SessionExpiredException.class
2019-07-29 23:02:06.721:WARN:oeja.AnnotationParser:qtp1911152052-18: org.apache.zookeeper.KeeperException$SessionMovedException scanned from multiple locations: jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/zkclient-0.1.0.jar!/org/apache/zookeeper/KeeperException$SessionMovedException.class, jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/zookeeper-3.4.5.jar!/org/apache/zookeeper/KeeperException$SessionMovedException.class
2019-07-29 23:02:06.722:WARN:oeja.AnnotationParser:qtp1911152052-18: org.apache.zookeeper.KeeperException$SystemErrorException scanned from multiple locations: jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/zkclient-0.1.0.jar!/org/apache/zookeeper/KeeperException$SystemErrorException.class, jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/zookeeper-3.4.5.jar!/org/apache/zookeeper/KeeperException$SystemErrorException.class
2019-07-29 23:02:06.722:WARN:oeja.AnnotationParser:qtp1911152052-18: org.apache.zookeeper.KeeperException$UnimplementedException scanned from multiple locations: jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/zkclient-0.1.0.jar!/org/apache/zookeeper/KeeperException$UnimplementedException.class, jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/zookeeper-3.4.5.jar!/org/apache/zookeeper/KeeperException$UnimplementedException.class
2019-07-29 23:02:06.725:WARN:oeja.AnnotationParser:qtp1911152052-18: org.apache.zookeeper.KeeperException scanned from multiple locations: jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/zkclient-0.1.0.jar!/org/apache/zookeeper/KeeperException.class, jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/zookeeper-3.4.5.jar!/org/apache/zookeeper/KeeperException.class
2019-07-29 23:02:06.728:WARN:oeja.AnnotationParser:qtp1911152052-18: org.apache.zookeeper.WatchedEvent scanned from multiple locations: jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/zkclient-0.1.0.jar!/org/apache/zookeeper/WatchedEvent.class, jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/zookeeper-3.4.5.jar!/org/apache/zookeeper/WatchedEvent.class
2019-07-29 23:02:06.738:WARN:oeja.AnnotationParser:qtp1911152052-18: org.apache.zookeeper.Watcher$Event$EventType scanned from multiple locations: jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/zkclient-0.1.0.jar!/org/apache/zookeeper/Watcher$Event$EventType.class, jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/zookeeper-3.4.5.jar!/org/apache/zookeeper/Watcher$Event$EventType.class
2019-07-29 23:02:06.741:WARN:oeja.AnnotationParser:qtp1911152052-18: org.apache.zookeeper.Watcher$Event$KeeperState scanned from multiple locations: jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/zkclient-0.1.0.jar!/org/apache/zookeeper/Watcher$Event$KeeperState.class, jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/zookeeper-3.4.5.jar!/org/apache/zookeeper/Watcher$Event$KeeperState.class
2019-07-29 23:02:06.742:WARN:oeja.AnnotationParser:qtp1911152052-18: org.apache.zookeeper.Watcher$Event scanned from multiple locations: jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/zkclient-0.1.0.jar!/org/apache/zookeeper/Watcher$Event.class, jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/zookeeper-3.4.5.jar!/org/apache/zookeeper/Watcher$Event.class
2019-07-29 23:02:06.746:WARN:oeja.AnnotationParser:qtp1911152052-18: org.apache.zookeeper.Watcher scanned from multiple locations: jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/zkclient-0.1.0.jar!/org/apache/zookeeper/Watcher.class, jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/zookeeper-3.4.5.jar!/org/apache/zookeeper/Watcher.class
2019-07-29 23:02:06.747:WARN:oeja.AnnotationParser:qtp1911152052-18: org.apache.zookeeper.ZooDefs$Ids scanned from multiple locations: jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/zkclient-0.1.0.jar!/org/apache/zookeeper/ZooDefs$Ids.class, jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/zookeeper-3.4.5.jar!/org/apache/zookeeper/ZooDefs$Ids.class
2019-07-29 23:02:06.749:WARN:oeja.AnnotationParser:qtp1911152052-18: org.apache.zookeeper.ZooDefs$OpCode scanned from multiple locations: jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/zkclient-0.1.0.jar!/org/apache/zookeeper/ZooDefs$OpCode.class, jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/zookeeper-3.4.5.jar!/org/apache/zookeeper/ZooDefs$OpCode.class
2019-07-29 23:02:06.753:WARN:oeja.AnnotationParser:qtp1911152052-18: org.apache.zookeeper.ZooDefs$Perms scanned from multiple locations: jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/zkclient-0.1.0.jar!/org/apache/zookeeper/ZooDefs$Perms.class, jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/zookeeper-3.4.5.jar!/org/apache/zookeeper/ZooDefs$Perms.class
2019-07-29 23:02:06.753:WARN:oeja.AnnotationParser:qtp1911152052-18: org.apache.zookeeper.ZooDefs scanned from multiple locations: jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/zkclient-0.1.0.jar!/org/apache/zookeeper/ZooDefs.class, jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/zookeeper-3.4.5.jar!/org/apache/zookeeper/ZooDefs.class
2019-07-29 23:02:06.757:WARN:oeja.AnnotationParser:qtp1911152052-18: org.apache.zookeeper.ZooKeeper$1 scanned from multiple locations: jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/zkclient-0.1.0.jar!/org/apache/zookeeper/ZooKeeper$1.class, jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/zookeeper-3.4.5.jar!/org/apache/zookeeper/ZooKeeper$1.class
2019-07-29 23:02:06.758:WARN:oeja.AnnotationParser:qtp1911152052-18: org.apache.zookeeper.ZooKeeper$ChildWatchRegistration scanned from multiple locations: jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/zkclient-0.1.0.jar!/org/apache/zookeeper/ZooKeeper$ChildWatchRegistration.class, jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/zookeeper-3.4.5.jar!/org/apache/zookeeper/ZooKeeper$ChildWatchRegistration.class
2019-07-29 23:02:06.769:WARN:oeja.AnnotationParser:qtp1911152052-18: org.apache.zookeeper.ZooKeeper$DataWatchRegistration scanned from multiple locations: jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/zkclient-0.1.0.jar!/org/apache/zookeeper/ZooKeeper$DataWatchRegistration.class, jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/zookeeper-3.4.5.jar!/org/apache/zookeeper/ZooKeeper$DataWatchRegistration.class
2019-07-29 23:02:06.773:WARN:oeja.AnnotationParser:qtp1911152052-18: org.apache.zookeeper.ZooKeeper$ExistsWatchRegistration scanned from multiple locations: jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/zkclient-0.1.0.jar!/org/apache/zookeeper/ZooKeeper$ExistsWatchRegistration.class, jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/zookeeper-3.4.5.jar!/org/apache/zookeeper/ZooKeeper$ExistsWatchRegistration.class
2019-07-29 23:02:06.773:WARN:oeja.AnnotationParser:qtp1911152052-18: org.apache.zookeeper.ZooKeeper$States scanned from multiple locations: jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/zkclient-0.1.0.jar!/org/apache/zookeeper/ZooKeeper$States.class, jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/zookeeper-3.4.5.jar!/org/apache/zookeeper/ZooKeeper$States.class
2019-07-29 23:02:06.774:WARN:oeja.AnnotationParser:qtp1911152052-18: org.apache.zookeeper.ZooKeeper$WatchRegistration scanned from multiple locations: jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/zkclient-0.1.0.jar!/org/apache/zookeeper/ZooKeeper$WatchRegistration.class, jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/zookeeper-3.4.5.jar!/org/apache/zookeeper/ZooKeeper$WatchRegistration.class
2019-07-29 23:02:06.777:WARN:oeja.AnnotationParser:qtp1911152052-18: org.apache.zookeeper.ZooKeeper$ZKWatchManager scanned from multiple locations: jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/zkclient-0.1.0.jar!/org/apache/zookeeper/ZooKeeper$ZKWatchManager.class, jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/zookeeper-3.4.5.jar!/org/apache/zookeeper/ZooKeeper$ZKWatchManager.class
2019-07-29 23:02:06.778:WARN:oeja.AnnotationParser:qtp1911152052-18: org.apache.zookeeper.ZooKeeper scanned from multiple locations: jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/zkclient-0.1.0.jar!/org/apache/zookeeper/ZooKeeper.class, jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/zookeeper-3.4.5.jar!/org/apache/zookeeper/ZooKeeper.class
2019-07-29 23:02:06.787:WARN:oeja.AnnotationParser:qtp1911152052-18: org.apache.zookeeper.data.ACL scanned from multiple locations: jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/zkclient-0.1.0.jar!/org/apache/zookeeper/data/ACL.class, jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/zookeeper-3.4.5.jar!/org/apache/zookeeper/data/ACL.class
2019-07-29 23:02:06.793:WARN:oeja.AnnotationParser:qtp1911152052-18: org.apache.zookeeper.data.Id scanned from multiple locations: jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/zkclient-0.1.0.jar!/org/apache/zookeeper/data/Id.class, jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/zookeeper-3.4.5.jar!/org/apache/zookeeper/data/Id.class
2019-07-29 23:02:06.794:WARN:oeja.AnnotationParser:qtp1911152052-18: org.apache.zookeeper.data.Stat scanned from multiple locations: jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/zkclient-0.1.0.jar!/org/apache/zookeeper/data/Stat.class, jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/zookeeper-3.4.5.jar!/org/apache/zookeeper/data/Stat.class
2019-07-29 23:02:06.798:WARN:oeja.AnnotationParser:qtp1911152052-18: org.apache.zookeeper.proto.AuthPacket scanned from multiple locations: jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/zkclient-0.1.0.jar!/org/apache/zookeeper/proto/AuthPacket.class, jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/zookeeper-3.4.5.jar!/org/apache/zookeeper/proto/AuthPacket.class
2019-07-29 23:02:06.805:WARN:oeja.AnnotationParser:qtp1911152052-18: org.apache.zookeeper.proto.ConnectRequest scanned from multiple locations: jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/zkclient-0.1.0.jar!/org/apache/zookeeper/proto/ConnectRequest.class, jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/zookeeper-3.4.5.jar!/org/apache/zookeeper/proto/ConnectRequest.class
2019-07-29 23:02:06.805:WARN:oeja.AnnotationParser:qtp1911152052-18: org.apache.zookeeper.proto.ConnectResponse scanned from multiple locations: jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/zkclient-0.1.0.jar!/org/apache/zookeeper/proto/ConnectResponse.class, jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/zookeeper-3.4.5.jar!/org/apache/zookeeper/proto/ConnectResponse.class
2019-07-29 23:02:06.805:WARN:oeja.AnnotationParser:qtp1911152052-18: org.apache.zookeeper.proto.CreateRequest scanned from multiple locations: jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/zkclient-0.1.0.jar!/org/apache/zookeeper/proto/CreateRequest.class, jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/zookeeper-3.4.5.jar!/org/apache/zookeeper/proto/CreateRequest.class
2019-07-29 23:02:06.806:WARN:oeja.AnnotationParser:qtp1911152052-18: org.apache.zookeeper.proto.CreateResponse scanned from multiple locations: jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/zkclient-0.1.0.jar!/org/apache/zookeeper/proto/CreateResponse.class, jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/zookeeper-3.4.5.jar!/org/apache/zookeeper/proto/CreateResponse.class
2019-07-29 23:02:06.806:WARN:oeja.AnnotationParser:qtp1911152052-18: org.apache.zookeeper.proto.DeleteRequest scanned from multiple locations: jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/zkclient-0.1.0.jar!/org/apache/zookeeper/proto/DeleteRequest.class, jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/zookeeper-3.4.5.jar!/org/apache/zookeeper/proto/DeleteRequest.class
2019-07-29 23:02:06.807:WARN:oeja.AnnotationParser:qtp1911152052-18: org.apache.zookeeper.proto.ExistsRequest scanned from multiple locations: jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/zkclient-0.1.0.jar!/org/apache/zookeeper/proto/ExistsRequest.class, jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/zookeeper-3.4.5.jar!/org/apache/zookeeper/proto/ExistsRequest.class
2019-07-29 23:02:06.813:WARN:oeja.AnnotationParser:qtp1911152052-18: org.apache.zookeeper.proto.ExistsResponse scanned from multiple locations: jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/zkclient-0.1.0.jar!/org/apache/zookeeper/proto/ExistsResponse.class, jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/zookeeper-3.4.5.jar!/org/apache/zookeeper/proto/ExistsResponse.class
2019-07-29 23:02:06.814:WARN:oeja.AnnotationParser:qtp1911152052-18: org.apache.zookeeper.proto.GetACLRequest scanned from multiple locations: jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/zkclient-0.1.0.jar!/org/apache/zookeeper/proto/GetACLRequest.class, jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/zookeeper-3.4.5.jar!/org/apache/zookeeper/proto/GetACLRequest.class
2019-07-29 23:02:06.814:WARN:oeja.AnnotationParser:qtp1911152052-18: org.apache.zookeeper.proto.GetACLResponse scanned from multiple locations: jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/zkclient-0.1.0.jar!/org/apache/zookeeper/proto/GetACLResponse.class, jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/zookeeper-3.4.5.jar!/org/apache/zookeeper/proto/GetACLResponse.class
2019-07-29 23:02:06.814:WARN:oeja.AnnotationParser:qtp1911152052-18: org.apache.zookeeper.proto.GetChildrenRequest scanned from multiple locations: jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/zkclient-0.1.0.jar!/org/apache/zookeeper/proto/GetChildrenRequest.class, jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/zookeeper-3.4.5.jar!/org/apache/zookeeper/proto/GetChildrenRequest.class
2019-07-29 23:02:06.818:WARN:oeja.AnnotationParser:qtp1911152052-18: org.apache.zookeeper.proto.GetChildrenResponse scanned from multiple locations: jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/zkclient-0.1.0.jar!/org/apache/zookeeper/proto/GetChildrenResponse.class, jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/zookeeper-3.4.5.jar!/org/apache/zookeeper/proto/GetChildrenResponse.class
2019-07-29 23:02:06.818:WARN:oeja.AnnotationParser:qtp1911152052-12: com.zimbra.buildinfo.Version scanned from multiple locations: jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/zm-ews-stub-2.0.jar!/com/zimbra/buildinfo/Version.class, jar:file:///opt/zimbra/lib/jars/zimbrasoap.jar!/com/zimbra/buildinfo/Version.class
2019-07-29 23:02:06.818:WARN:oeja.AnnotationParser:qtp1911152052-18: org.apache.zookeeper.proto.GetDataRequest scanned from multiple locations: jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/zkclient-0.1.0.jar!/org/apache/zookeeper/proto/GetDataRequest.class, jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/zookeeper-3.4.5.jar!/org/apache/zookeeper/proto/GetDataRequest.class
2019-07-29 23:02:06.819:WARN:oeja.AnnotationParser:qtp1911152052-18: org.apache.zookeeper.proto.GetDataResponse scanned from multiple locations: jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/zkclient-0.1.0.jar!/org/apache/zookeeper/proto/GetDataResponse.class, jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/zookeeper-3.4.5.jar!/org/apache/zookeeper/proto/GetDataResponse.class
2019-07-29 23:02:06.819:WARN:oeja.AnnotationParser:qtp1911152052-18: org.apache.zookeeper.proto.ReplyHeader scanned from multiple locations: jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/zkclient-0.1.0.jar!/org/apache/zookeeper/proto/ReplyHeader.class, jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/zookeeper-3.4.5.jar!/org/apache/zookeeper/proto/ReplyHeader.class
2019-07-29 23:02:06.820:WARN:oeja.AnnotationParser:qtp1911152052-18: org.apache.zookeeper.proto.RequestHeader scanned from multiple locations: jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/zkclient-0.1.0.jar!/org/apache/zookeeper/proto/RequestHeader.class, jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/zookeeper-3.4.5.jar!/org/apache/zookeeper/proto/RequestHeader.class
2019-07-29 23:02:06.820:WARN:oeja.AnnotationParser:qtp1911152052-18: org.apache.zookeeper.proto.SetACLRequest scanned from multiple locations: jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/zkclient-0.1.0.jar!/org/apache/zookeeper/proto/SetACLRequest.class, jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/zookeeper-3.4.5.jar!/org/apache/zookeeper/proto/SetACLRequest.class
2019-07-29 23:02:06.821:WARN:oeja.AnnotationParser:qtp1911152052-18: org.apache.zookeeper.proto.SetACLResponse scanned from multiple locations: jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/zkclient-0.1.0.jar!/org/apache/zookeeper/proto/SetACLResponse.class, jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/zookeeper-3.4.5.jar!/org/apache/zookeeper/proto/SetACLResponse.class
2019-07-29 23:02:06.821:WARN:oeja.AnnotationParser:qtp1911152052-18: org.apache.zookeeper.proto.SetDataRequest scanned from multiple locations: jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/zkclient-0.1.0.jar!/org/apache/zookeeper/proto/SetDataRequest.class, jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/zookeeper-3.4.5.jar!/org/apache/zookeeper/proto/SetDataRequest.class
2019-07-29 23:02:06.821:WARN:oeja.AnnotationParser:qtp1911152052-18: org.apache.zookeeper.proto.SetDataResponse scanned from multiple locations: jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/zkclient-0.1.0.jar!/org/apache/zookeeper/proto/SetDataResponse.class, jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/zookeeper-3.4.5.jar!/org/apache/zookeeper/proto/SetDataResponse.class
2019-07-29 23:02:06.822:WARN:oeja.AnnotationParser:qtp1911152052-18: org.apache.zookeeper.proto.SetWatches scanned from multiple locations: jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/zkclient-0.1.0.jar!/org/apache/zookeeper/proto/SetWatches.class, jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/zookeeper-3.4.5.jar!/org/apache/zookeeper/proto/SetWatches.class
2019-07-29 23:02:06.822:WARN:oeja.AnnotationParser:qtp1911152052-18: org.apache.zookeeper.proto.SyncRequest scanned from multiple locations: jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/zkclient-0.1.0.jar!/org/apache/zookeeper/proto/SyncRequest.class, jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/zookeeper-3.4.5.jar!/org/apache/zookeeper/proto/SyncRequest.class
2019-07-29 23:02:06.823:WARN:oeja.AnnotationParser:qtp1911152052-18: org.apache.zookeeper.proto.SyncResponse scanned from multiple locations: jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/zkclient-0.1.0.jar!/org/apache/zookeeper/proto/SyncResponse.class, jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/zookeeper-3.4.5.jar!/org/apache/zookeeper/proto/SyncResponse.class
2019-07-29 23:02:06.823:WARN:oeja.AnnotationParser:qtp1911152052-18: org.apache.zookeeper.proto.WatcherEvent scanned from multiple locations: jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/zkclient-0.1.0.jar!/org/apache/zookeeper/proto/WatcherEvent.class, jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/zookeeper-3.4.5.jar!/org/apache/zookeeper/proto/WatcherEvent.class
2019-07-29 23:02:06.886:WARN:oeja.AnnotationParser:qtp1911152052-14: com.zimbra.buildinfo.Version scanned from multiple locations: jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/zm-ews-stub-2.0.jar!/com/zimbra/buildinfo/Version.class, jar:file:///opt/zimbra/lib/jars/zimbraclient.jar!/com/zimbra/buildinfo/Version.class
2019-07-29 23:02:07.012:WARN:oeja.AnnotationParser:qtp1911152052-17: com.zimbra.buildinfo.Version scanned from multiple locations: jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/zm-ews-stub-2.0.jar!/com/zimbra/buildinfo/Version.class, jar:file:///opt/zimbra/lib/jars/zimbrastore.jar!/com/zimbra/buildinfo/Version.class
[8.609s][info][gc] GC(4) Pause Young (Normal) (G1 Evacuation Pause) 221M->32M(4096M) 109.608ms
2019-07-29 23:02:08.334:INFO:oeja.AnnotationConfiguration:main: Scanning elapsed time=4827ms
2019-07-29 23:02:09.286:INFO:oejs.session:main: DefaultSessionIdManager workerName=node0
2019-07-29 23:02:09.287:INFO:oejs.session:main: No SessionScavenger set, using defaults
2019-07-29 23:02:09.289:INFO:oejs.session:main: node0 Scavenging every 660000ms
2019-07-29 23:02:09.299:WARN:oejs.SecurityHandler:main: ServletContext@o.e.j.w.WebAppContext@48e1f6c7{Zimbra Service,/service,file:///opt/zimbra/jetty_base/webapps/service/,STARTING}{/service} has uncovered http methods for path: /
[11.824s][info][gc] GC(5) Pause Young (Concurrent Start) (Metadata GC Threshold) 185M->38M(4096M) 77.974ms
[11.824s][info][gc] GC(6) Concurrent Cycle
[12.034s][info][gc] GC(6) Pause Remark 40M->40M(4096M) 40.707ms
[12.096s][info][gc] GC(6) Pause Cleanup 40M->40M(4096M) 0.309ms
[12.205s][info][gc] GC(6) Concurrent Cycle 380.883ms
Zimbra server process is running as uid=999 euid=999 gid=999 egid=999

Any help appreciate.

Thank you.

L. Mark Stone · Post by **L. Mark Stone** » Tue Jul 30, 2019 2:18 am

Suggest you execute as zimbra:

Code: Select all

zmcontrol stop

then, become root and execute:

Code: Select all

ps ax | grep zimbra

If you see any Zimbra processes, kill them. Next, try to restart Zimbra:

Code: Select all

zmcontrol start && sleep 20 && zmcontrol status

If you still see "stale pid file" errors in the logs, stop Zimbra, delete the offending pid files and then restart Zimbra.

Keep us posted?

All the best,
Mark

rosole · Post by **rosole** » Tue Jul 30, 2019 10:18 am

Hello Mark.

Thank you for replay.

I did what you suggested:

zmcontrol stop
Host zimbra.domain.pl
Stopping zmconfigd...Done.
Stopping zimlet webapp...Done.
Stopping zimbraAdmin webapp...Done.
Stopping zimbra webapp...Done.
Stopping service webapp...Done.
Stopping stats...Done.
Stopping mta...Done.
Stopping spell...Done.
Stopping snmp...Done.
Stopping cbpolicyd...Done.
Stopping archiving...Done.
Stopping opendkim...Done.
Stopping amavis...Done.
Stopping antivirus...Done.
Stopping antispam...Done.
Stopping proxy...Done.
Stopping memcached...Done.
Stopping mailbox...Done.
Stopping logger...Done.
Stopping dnscache...Done.
Stopping ldap...Done.

ps ax | grep zimbra
11914 pts/0 S 0:00 su zimbra
13344 pts/1 S+ 0:00 grep --color=auto zimbra

cat /opt/zimbra/log/zmmailboxd_java.pid
2581
cat /opt/zimbra/log/zmmailboxd_manager.pid
2471

ps ax| grep 2581
14069 pts/1 S+ 0:00 grep --color=auto 2581

ps ax| grep 2471
14071 pts/1 S+ 0:00 grep --color=auto 2471

zimbra@zimbra:/home/user$ zmcontrol start && sleep 20 && zmcontrol status
Host zimbra.domain.pl
Starting ldap...Done.
Starting zmconfigd...Done.
Starting dnscache...Done.
Starting logger...Done.
Starting mailbox...Done.
Starting memcached...Done.
Starting proxy...Done.
Starting amavis...Done.
Starting antispam...Done.
Starting antivirus...Done.
Starting opendkim...Done.
Starting snmp...Done.
Starting spell...Done.
Starting mta...Done.
Starting stats...Done.
Starting service webapp...Done.
Starting zimbra webapp...Done.
Starting zimbraAdmin webapp...Done.
Starting zimlet webapp...Done.
Host zimbra.domain.pl
amavis Running
antispam Running
antivirus Running
dnscache Running
ldap Running
logger Running
mailbox Stopped
zmmailboxdctl is not running.
memcached Running
mta Running
opendkim Running
proxy Running
service webapp Stopped
zmmailboxdctl is not running.
snmp Running
spell Running
stats Running
zimbra webapp Stopped
zmmailboxdctl is not running.
zimbraAdmin webapp Stopped
zmmailboxdctl is not running.
zimlet webapp Stopped
zmmailboxdctl is not running.
zmconfigd Running

cat /opt/zimbra/log/zmmailboxd_manager.pid
18191
cat /opt/zimbra/log/zmmailboxd_java.pid
19079

ps ax| grep 19079
31881 pts/1 S+ 0:00 grep --color=auto 19079

ps ax| grep 18191
31887 pts/1 S+ 0:00 grep --color=auto 18191

It looks that Zimbra tries to start several times VM but it stops after while and pid exists but processes not.

L. Mark Stone · Post by **L. Mark Stone** » Tue Jul 30, 2019 12:11 pm

Please post logs from when mailboxd is starting, so we can see what is failing.

Let's also look at some basics, so please post the contents of /etc/hosts, /etc/resolv.conf and /etc/hostname.

Mark

rosole · Post by **rosole** » Tue Jul 30, 2019 12:26 pm

Hello Mark.

DNS and MX are correct.

cat /etc/hostname
zimbra

/etc/hosts
127.0.0.1 localhost
10.10.10.11 zimbra.domain.pl zimbra

# The following lines are desirable for IPv6 capable hosts
#::1 localhost ip6-localhost ip6-loopback
#ff02::1 ip6-allnodes
#ff02::2 ip6-allrouters

cat /etc/resolv.conf
# Dynamic resolv.conf(5) file for glibc resolver(3) generated by resolvconf(8)
# DO NOT EDIT THIS FILE BY HAND -- YOUR CHANGES WILL BE OVERWRITTEN
nameserver 127.0.0.1
search domain.pl

Here you have after zmmailboxdctl start:

Jul 30 14:20:12 zimbra zmmailboxdmgr[26218]: stale pid 18191 found in /opt/zimbra/log/zmmailboxd_manager.pid: No such process
Jul 30 14:20:12 zimbra zmmailboxdmgr[26218]: assuming no other instance is running
Jul 30 14:20:12 zimbra zmmailboxdmgr[26218]: did not find a number in /opt/zimbra/log/zmmailboxd.pid
Jul 30 14:20:12 zimbra zmmailboxdmgr[26218]: assuming no other instance is running
Jul 30 14:20:12 zimbra zmmailboxdmgr[26218]: no manager process is running
Jul 30 14:20:12 zimbra /postfix-script[26248]: the Postfix mail system is running: PID: 17491
Jul 30 14:20:12 zimbra zmmailboxdmgr[26317]: stale pid 18191 found in /opt/zimbra/log/zmmailboxd_manager.pid: No such process
Jul 30 14:20:12 zimbra zmmailboxdmgr[26317]: assuming no other instance is running
Jul 30 14:20:12 zimbra zmmailboxdmgr[26317]: did not find a number in /opt/zimbra/log/zmmailboxd.pid
Jul 30 14:20:12 zimbra zmmailboxdmgr[26317]: assuming no other instance is running
Jul 30 14:20:12 zimbra zmmailboxdmgr[26317]: no manager process is running
Jul 30 14:20:14 zimbra zmmailboxdmgr[26445]: stale pid 18191 found in /opt/zimbra/log/zmmailboxd_manager.pid: No such process
Jul 30 14:20:14 zimbra zmmailboxdmgr[26445]: assuming no other instance is running
Jul 30 14:20:14 zimbra zmmailboxdmgr[26445]: did not find a number in /opt/zimbra/log/zmmailboxd.pid
Jul 30 14:20:14 zimbra zmmailboxdmgr[26445]: assuming no other instance is running
Jul 30 14:20:14 zimbra zmmailboxdmgr[26445]: no manager process is running
Jul 30 14:20:14 zimbra zmmailboxdmgr[26462]: stale pid 18191 found in /opt/zimbra/log/zmmailboxd_manager.pid: No such process
Jul 30 14:20:14 zimbra zmmailboxdmgr[26462]: assuming no other instance is running
Jul 30 14:20:14 zimbra zmmailboxdmgr[26462]: did not find a number in /opt/zimbra/log/zmmailboxd.pid
Jul 30 14:20:14 zimbra zmmailboxdmgr[26462]: assuming no other instance is running
Jul 30 14:20:14 zimbra zmmailboxdmgr[26462]: no manager process is running
Jul 30 14:20:14 zimbra zmmailboxdmgr[26479]: stale pid 18191 found in /opt/zimbra/log/zmmailboxd_manager.pid: No such process
Jul 30 14:20:14 zimbra zmmailboxdmgr[26479]: assuming no other instance is running
Jul 30 14:20:14 zimbra zmmailboxdmgr[26479]: did not find a number in /opt/zimbra/log/zmmailboxd.pid
Jul 30 14:20:14 zimbra zmmailboxdmgr[26479]: assuming no other instance is running
Jul 30 14:20:14 zimbra zmmailboxdmgr[26479]: no manager process is running
Jul 30 14:20:16 zimbra zmconfigd[14163]: Processing rewrite request for sasl
Jul 30 14:20:16 zimbra zmconfigd[14163]: Processing rewrite request for webxml
Jul 30 14:20:16 zimbra zmconfigd[14163]: Processing rewrite request for mailbox
Jul 30 14:20:16 zimbra zmconfigd[14163]: Processing rewrite request for service
Jul 30 14:20:16 zimbra zmconfigd[14163]: Processing rewrite request for zimbra
Jul 30 14:20:16 zimbra zmconfigd[14163]: Processing rewrite request for zimbraAdmin
Jul 30 14:20:16 zimbra zmconfigd[14163]: Processing rewrite request for zimlet
Jul 30 14:20:17 zimbra zmconfigd[14163]: Fetching All configs
Jul 30 14:20:17 zimbra zmconfigd[14163]: All configs fetched in 0.05 seconds
Jul 30 14:20:22 zimbra zmmailboxdmgr[26865]: stale pid 18191 found in /opt/zimbra/log/zmmailboxd_manager.pid: No such process
Jul 30 14:20:22 zimbra zmmailboxdmgr[26865]: assuming no other instance is running
Jul 30 14:20:22 zimbra zmmailboxdmgr[26865]: did not find a number in /opt/zimbra/log/zmmailboxd.pid
Jul 30 14:20:22 zimbra zmmailboxdmgr[26865]: assuming no other instance is running
Jul 30 14:20:22 zimbra zmmailboxdmgr[26865]: no manager process is running
Jul 30 14:20:23 zimbra zmmailboxdmgr[26879]: stale pid 18191 found in /opt/zimbra/log/zmmailboxd_manager.pid: No such process
Jul 30 14:20:23 zimbra zmmailboxdmgr[26879]: assuming no other instance is running
Jul 30 14:20:23 zimbra zmmailboxdmgr[26879]: did not find a number in /opt/zimbra/log/zmmailboxd.pid
Jul 30 14:20:23 zimbra zmmailboxdmgr[26879]: assuming no other instance is running
Jul 30 14:20:23 zimbra zmmailboxdmgr[26879]: no manager process is running
Jul 30 14:20:23 zimbra /postfix-script[26911]: the Postfix mail system is running: PID: 17491
Jul 30 14:20:23 zimbra zmmailboxdmgr[26949]: stale pid 18191 found in /opt/zimbra/log/zmmailboxd_manager.pid: No such process
Jul 30 14:20:23 zimbra zmmailboxdmgr[26949]: assuming no other instance is running
Jul 30 14:20:23 zimbra zmmailboxdmgr[26949]: did not find a number in /opt/zimbra/log/zmmailboxd.pid
Jul 30 14:20:23 zimbra zmmailboxdmgr[26949]: assuming no other instance is running
Jul 30 14:20:23 zimbra zmmailboxdmgr[26949]: no manager process is running
Jul 30 14:20:24 zimbra zmmailboxdmgr[27067]: stale pid 18191 found in /opt/zimbra/log/zmmailboxd_manager.pid: No such process
Jul 30 14:20:24 zimbra zmmailboxdmgr[27067]: assuming no other instance is running
Jul 30 14:20:24 zimbra zmmailboxdmgr[27067]: did not find a number in /opt/zimbra/log/zmmailboxd.pid
Jul 30 14:20:24 zimbra zmmailboxdmgr[27067]: assuming no other instance is running
Jul 30 14:20:24 zimbra zmmailboxdmgr[27067]: no manager process is running
Jul 30 14:20:24 zimbra zmmailboxdmgr[27081]: stale pid 18191 found in /opt/zimbra/log/zmmailboxd_manager.pid: No such process
Jul 30 14:20:24 zimbra zmmailboxdmgr[27081]: assuming no other instance is running
Jul 30 14:20:24 zimbra zmmailboxdmgr[27081]: did not find a number in /opt/zimbra/log/zmmailboxd.pid
Jul 30 14:20:24 zimbra zmmailboxdmgr[27081]: assuming no other instance is running
Jul 30 14:20:24 zimbra zmmailboxdmgr[27081]: no manager process is running
Jul 30 14:20:25 zimbra zmmailboxdmgr[27095]: stale pid 18191 found in /opt/zimbra/log/zmmailboxd_manager.pid: No such process
Jul 30 14:20:25 zimbra zmmailboxdmgr[27095]: assuming no other instance is running
Jul 30 14:20:25 zimbra zmmailboxdmgr[27095]: did not find a number in /opt/zimbra/log/zmmailboxd.pid
Jul 30 14:20:25 zimbra zmmailboxdmgr[27095]: assuming no other instance is running
Jul 30 14:20:25 zimbra zmmailboxdmgr[27095]: no manager process is running
Jul 30 14:20:25 zimbra zmconfigd[14163]: Watchdog: service antivirus status is OK.
Jul 30 14:20:25 zimbra zmconfigd[14163]: Rewrote: /opt/zimbra/mailboxd/webapps/service/WEB-INF/web.xml with mode 440 (0.04 sec)
Jul 30 14:20:25 zimbra zmconfigd[14163]: Rewrote: /opt/zimbra/mailboxd/webapps/zimbraAdmin/WEB-INF/jetty-env.xml with mode 440 (0.01 sec)
Jul 30 14:20:25 zimbra zmconfigd[14163]: Rewrote: /opt/zimbra/mailboxd/webapps/zimbra/WEB-INF/jetty-env.xml with mode 440 (0.01 sec)
Jul 30 14:20:25 zimbra zmconfigd[14163]: Rewrote: /opt/zimbra/mailboxd/webapps/zimbraAdmin/WEB-INF/web.xml with mode 440 (0.03 sec)
Jul 30 14:20:25 zimbra zmconfigd[14163]: Rewrote: /opt/zimbra/mailboxd/etc/spnego.conf with mode 440 (0.01 sec)
Jul 30 14:20:25 zimbra zmconfigd[14163]: Rewrote: /opt/zimbra/mailboxd/webapps/zimbra/WEB-INF/web.xml with mode 440 (0.04 sec)
Jul 30 14:20:25 zimbra zmconfigd[14163]: Rewrote: /opt/zimbra/mailboxd/etc/jetty.xml with mode 440 (0.06 sec)
Jul 30 14:20:25 zimbra zmconfigd[14163]: Rewrote: /opt/zimbra/conf/spnego_java_options with mode 440 (0.00 sec)
Jul 30 14:20:25 zimbra zmconfigd[14163]: Rewrote: /opt/zimbra/conf/sasl2/smtpd.conf with mode 440 (0.00 sec)
Jul 30 14:20:25 zimbra zmconfigd[14163]: Rewrote: /opt/zimbra/mailboxd/etc/spnego.properties with mode 440 (0.00 sec)
Jul 30 14:20:25 zimbra zmconfigd[14163]: Rewrote: /opt/zimbra/mailboxd/etc/krb5.ini with mode 440 (0.00 sec)
Jul 30 14:20:25 zimbra zmconfigd[14163]: Rewrote: /opt/zimbra/conf/saslauthd.conf with mode 440 (0.00 sec)
Jul 30 14:20:25 zimbra zmconfigd[14163]: Rewrote: /opt/zimbra/conf/log4j.properties with mode 440 (0.03 sec)
Jul 30 14:20:25 zimbra zmconfigd[14163]: Rewrote: /opt/zimbra/mailboxd/webapps/zimlet/WEB-INF/web.xml with mode 440 (0.01 sec)
Jul 30 14:20:25 zimbra zmconfigd[14163]: Rewrote: /opt/zimbra/mailboxd/modules/setuid.mod with mode 440 (0.01 sec)
Jul 30 14:20:25 zimbra zmconfigd[14163]: Rewrote: /opt/zimbra/mailboxd/start.d/setuid.ini with mode 440 (0.01 sec)
Jul 30 14:20:25 zimbra zmconfigd[14163]: All rewrite threads completed in 0.29 sec
Jul 30 14:20:25 zimbra zmconfigd[14163]: All restarts completed in 0.00 sec
Jul 30 14:20:30 zimbra zmmailboxdmgr[27115]: stale pid 18191 found in /opt/zimbra/log/zmmailboxd_manager.pid: No such process
Jul 30 14:20:30 zimbra zmmailboxdmgr[27115]: assuming no other instance is running
Jul 30 14:20:30 zimbra zmmailboxdmgr[27115]: did not find a number in /opt/zimbra/log/zmmailboxd.pid
Jul 30 14:20:30 zimbra zmmailboxdmgr[27115]: assuming no other instance is running
Jul 30 14:20:30 zimbra zmmailboxdmgr[27115]: no manager process is running
Jul 30 14:20:30 zimbra zmmailboxdmgr[27126]: start requested
Jul 30 14:20:30 zimbra zmmailboxdmgr[27126]: checking if another instance of manager is already running
Jul 30 14:20:30 zimbra zmmailboxdmgr[27126]: stale pid 18191 found in /opt/zimbra/log/zmmailboxd_manager.pid: No such process
Jul 30 14:20:30 zimbra zmmailboxdmgr[27126]: assuming no other instance is running
Jul 30 14:20:30 zimbra zmmailboxdmgr[27126]: did not find a number in /opt/zimbra/log/zmmailboxd.pid
Jul 30 14:20:30 zimbra zmmailboxdmgr[27126]: assuming no other instance is running
Jul 30 14:20:30 zimbra zmmailboxdmgr[27127]: wrote manager pid 27127 to /opt/zimbra/log/zmmailboxd_manager.pid
Jul 30 14:20:30 zimbra zmmailboxdmgr[27127]: manager started mailboxd/JVM with pid 27128
Jul 30 14:20:30 zimbra zmmailboxdmgr[27128]: wrote java pid 27128 to /opt/zimbra/log/zmmailboxd_java.pid
Jul 30 14:20:42 zimbra zmmailboxdmgr[27127]: manager woke up from wait on mailboxd/JVM with pid 27128
Jul 30 14:20:42 zimbra zmmailboxdmgr[27127]: manager started mailboxd/JVM with pid 27237
Jul 30 14:20:42 zimbra zmmailboxdmgr[27237]: wrote java pid 27237 to /opt/zimbra/log/zmmailboxd_java.pid
Jul 30 14:20:52 zimbra zmmailboxdmgr[27127]: manager woke up from wait on mailboxd/JVM with pid 27237
Jul 30 14:20:52 zimbra zmmailboxdmgr[27127]: mailboxd/JVM exited twice in 10 seconds (tolerance=60)

rosole · Post by **rosole** » Tue Jul 30, 2019 12:34 pm

Interesting, that when there is no file zmmailboxd.pid in /opt/zimbra/log/ Zimbra says that this file is missing.
After created file, he's got zero bytes:

ls -l log/
total 141200
-rw-r----- 1 zimbra zimbra 146226 Jun 29 01:50 access_log.2019-06-28
-rw-r----- 1 zimbra zimbra 56734 Jun 30 01:50 access_log.2019-06-29
-rw-r----- 1 zimbra zimbra 142588 Jul 1 01:50 access_log.2019-06-30
-rw-r----- 1 zimbra zimbra 209180 Jul 2 01:55 access_log.2019-07-01
-rw-r----- 1 zimbra zimbra 188771 Jul 3 01:59 access_log.2019-07-02
-rw-r----- 1 zimbra zimbra 232055 Jul 4 01:50 access_log.2019-07-03
-rw-r----- 1 zimbra zimbra 187675 Jul 5 01:58 access_log.2019-07-04
-rw-r----- 1 zimbra zimbra 127881 Jul 6 01:50 access_log.2019-07-05
-rw-r----- 1 zimbra zimbra 58586 Jul 7 01:50 access_log.2019-07-06
-rw-r----- 1 zimbra zimbra 154810 Jul 8 01:50 access_log.2019-07-07
-rw-r----- 1 zimbra zimbra 202922 Jul 9 01:51 access_log.2019-07-08
-rw-r----- 1 zimbra zimbra 273403 Jul 10 01:50 access_log.2019-07-09
-rw-r----- 1 zimbra zimbra 183934 Jul 11 01:50 access_log.2019-07-10
-rw-r----- 1 zimbra zimbra 331808 Jul 12 01:50 access_log.2019-07-11
-rw-r----- 1 zimbra zimbra 160301 Jul 13 01:50 access_log.2019-07-12
-rw-r----- 1 zimbra zimbra 68571 Jul 14 01:59 access_log.2019-07-13
-rw-r----- 1 zimbra zimbra 122450 Jul 15 01:50 access_log.2019-07-14
-rw-r----- 1 zimbra zimbra 105050 Jul 16 01:50 access_log.2019-07-15
-rw-r----- 1 zimbra zimbra 313904 Jul 17 01:50 access_log.2019-07-16
-rw-r----- 1 zimbra zimbra 145381 Jul 18 01:50 access_log.2019-07-17
-rw-r----- 1 zimbra zimbra 303498 Jul 19 01:50 access_log.2019-07-18
-rw-r----- 1 zimbra zimbra 180000 Jul 20 01:59 access_log.2019-07-19
-rw-r----- 1 zimbra zimbra 66071 Jul 21 01:50 access_log.2019-07-20
-rw-r----- 1 zimbra zimbra 120732 Jul 22 01:50 access_log.2019-07-21
-rw-r----- 1 zimbra zimbra 209219 Jul 23 01:54 access_log.2019-07-22
-rw-r----- 1 zimbra zimbra 221674 Jul 24 01:50 access_log.2019-07-23
-rw-r----- 1 zimbra zimbra 185425 Jul 25 01:59 access_log.2019-07-24
-rw-r----- 1 zimbra zimbra 109355 Jul 26 01:50 access_log.2019-07-25
-rw-r----- 1 zimbra zimbra 200274 Jul 27 01:58 access_log.2019-07-26
-rw-r----- 1 zimbra zimbra 135131 Jul 28 01:50 access_log.2019-07-27
-rw-r----- 1 zimbra zimbra 109487 Jul 29 01:50 access_log.2019-07-28
-rw-r----- 1 zimbra zimbra 33972 Jul 29 08:44 access_log.2019-07-29
-rw-r----- 1 zimbra zimbra 0 May 17 2018 activity.log
-rw-r----- 1 zimbra zimbra 6 Jul 30 11:51 amavisd.pid
-rw-r----- 1 zimbra zimbra 6 Jul 30 11:51 amavis-mc.pid
-rw-r----- 1 zimbra zimbra 35233 Jul 29 08:40 audit.log
-rw-rw-r-- 1 zimbra zimbra 6446 Jul 21 02:50 audit.log.2019-07-20.gz
-rw-rw-r-- 1 zimbra zimbra 8090 Jul 22 02:50 audit.log.2019-07-21.gz
-rw-rw-r-- 1 zimbra zimbra 12332 Jul 23 02:50 audit.log.2019-07-22.gz
-rw-rw-r-- 1 zimbra zimbra 9776 Jul 24 02:50 audit.log.2019-07-23.gz
-rw-rw-r-- 1 zimbra zimbra 10689 Jul 25 02:50 audit.log.2019-07-24.gz
-rw-rw-r-- 1 zimbra zimbra 8572 Jul 26 02:50 audit.log.2019-07-25.gz
-rw-rw-r-- 1 zimbra zimbra 9366 Jul 27 02:50 audit.log.2019-07-26.gz
-rw-rw-r-- 1 zimbra zimbra 8885 Jul 28 02:50 audit.log.2019-07-27.gz
-rw-rw-r-- 1 zimbra zimbra 8096 Jul 29 02:50 audit.log.2019-07-28.gz
-rw-r----- 1 zimbra zimbra 3908369 Jul 30 14:21 clamd.log
-rw-rw-r-- 1 zimbra zimbra 6 Jul 30 11:52 clamd.pid
-rw-r----- 1 zimbra zimbra 0 May 17 2018 ews.log
-rw-r----- 1 zimbra zimbra 426770 Jul 30 13:56 freshclam.log
-rw-rw---- 1 zimbra zimbra 6 Jul 30 11:53 freshclam.pid
-rw-r----- 1 root root 965 Jul 30 14:26 gc.log
-rw-r----- 1 zimbra zimbra 966 Jul 30 14:26 gc.log.0
-rw-r----- 1 zimbra zimbra 975 Jul 30 11:53 gc.log.1
-rw-r----- 1 zimbra zimbra 965 Jul 30 11:53 gc.log.2
-rw-r----- 1 zimbra zimbra 964 Jul 30 14:20 gc.log.3
-rw-r----- 1 zimbra zimbra 961 Jul 30 14:20 gc.log.4
-rw-r----- 1 zimbra zimbra 329 Jul 26 10:01 httpd_error.log.2019-07-26
-rw-r----- 1 zimbra zimbra 9733 Jul 29 22:45 httpd_error.log.2019-07-29
-rw-r----- 1 zimbra zimbra 455 Jul 30 11:53 httpd_error.log.2019-07-30
-rw-r----- 1 zimbra zimbra 6 Jul 30 11:53 httpd.pid
-rw-r----- 1 zimbra zimbra 6 Jul 30 11:51 logswatch.pid
-rw-r----- 1 zimbra zimbra 1095086 Jul 30 14:26 mailbox.log
-rw-rw-r-- 1 zimbra zimbra 106528 Jul 21 02:50 mailbox.log.2019-07-20.gz
-rw-rw-r-- 1 zimbra zimbra 151971 Jul 22 02:50 mailbox.log.2019-07-21.gz
-rw-rw-r-- 1 zimbra zimbra 276737 Jul 23 02:50 mailbox.log.2019-07-22.gz
-rw-rw-r-- 1 zimbra zimbra 244698 Jul 24 02:50 mailbox.log.2019-07-23.gz
-rw-rw-r-- 1 zimbra zimbra 230414 Jul 25 02:50 mailbox.log.2019-07-24.gz
-rw-rw-r-- 1 zimbra zimbra 190470 Jul 26 02:50 mailbox.log.2019-07-25.gz
-rw-rw-r-- 1 zimbra zimbra 186902 Jul 27 02:50 mailbox.log.2019-07-26.gz
-rw-rw-r-- 1 zimbra zimbra 171334 Jul 28 02:50 mailbox.log.2019-07-27.gz
-rw-rw-r-- 1 zimbra zimbra 142582 Jul 29 02:50 mailbox.log.2019-07-28.gz
-rw-r--r-- 1 zimbra zimbra 0 Apr 11 21:54 mailbox.pid
-rw-r----- 1 zimbra zimbra 6 Jul 30 11:51 memcached.pid
-rw-rw---- 1 zimbra zimbra 2338 Jul 30 14:26 myslow.log
-rw-rw---- 1 zimbra zimbra 11226 Jul 21 06:24 myslow.log.10.gz
-rw-rw---- 1 zimbra zimbra 3714 Jul 29 23:02 myslow.log.1.gz
-rw-rw---- 1 zimbra zimbra 10620 Jul 29 06:22 myslow.log.2.gz
-rw-rw---- 1 zimbra zimbra 11263 Jul 28 06:25 myslow.log.3.gz
-rw-rw---- 1 zimbra zimbra 15264 Jul 27 06:23 myslow.log.4.gz
-rw-rw---- 1 zimbra zimbra 10527 Jul 26 06:25 myslow.log.5.gz
-rw-rw---- 1 zimbra zimbra 10582 Jul 25 06:24 myslow.log.6.gz
-rw-rw---- 1 zimbra zimbra 10550 Jul 24 06:23 myslow.log.7.gz
-rw-rw---- 1 zimbra zimbra 10815 Jul 23 06:22 myslow.log.8.gz
-rw-rw---- 1 zimbra zimbra 10550 Jul 22 06:25 myslow.log.9.gz
-rw-rw---- 1 zimbra zimbra 475543 Jul 30 11:51 mysql_error.log
-rw-rw---- 1 zimbra zimbra 6 Jul 30 11:51 mysql.pid
-rw-r--r-- 1 zimbra zimbra 9967 Jul 30 14:24 nginx.access.log
-rw-r--r-- 1 zimbra zimbra 67819 Jul 30 05:02 nginx.access.log.1
-rw-r--r-- 1 zimbra zimbra 4276 Jul 29 06:01 nginx.access.log.2.gz
-rw-r--r-- 1 zimbra zimbra 4864 Jul 28 02:48 nginx.access.log.3.gz
-rw-r--r-- 1 zimbra zimbra 10618 Jul 27 05:34 nginx.access.log.4.gz
-rw-r--r-- 1 zimbra zimbra 2795 Jul 26 02:21 nginx.access.log.5.gz
-rw-r--r-- 1 zimbra zimbra 10459 Jul 25 05:14 nginx.access.log.6.gz
-rw-r--r-- 1 zimbra zimbra 12473 Jul 24 06:13 nginx.access.log.7.gz
-rw-r--r-- 1 zimbra zimbra 34169186 Jul 30 14:29 nginx.log
-rw-r--r-- 1 zimbra zimbra 52938278 Jul 30 06:25 nginx.log.1
-rw-r--r-- 1 zimbra zimbra 13515 Jul 29 06:25 nginx.log.2.gz
-rw-r--r-- 1 zimbra zimbra 16216 Jul 28 06:25 nginx.log.3.gz
-rw-r--r-- 1 zimbra zimbra 22170 Jul 27 06:25 nginx.log.4.gz
-rw-r--r-- 1 zimbra zimbra 13714 Jul 26 06:25 nginx.log.5.gz
-rw-r--r-- 1 zimbra zimbra 24457 Jul 25 06:25 nginx.log.6.gz
-rw-r--r-- 1 zimbra zimbra 22043 Jul 24 06:25 nginx.log.7.gz
-rw-r--r-- 1 root root 6 Jul 30 11:51 nginx.pid
-rw-r----- 1 zimbra zimbra 6 Jul 30 11:53 opendkim.pid
-rw-r----- 1 zimbra zimbra 0 May 17 2018 searchstat.log
-rw-rw-r-- 1 zimbra zimbra 0 Jul 30 06:25 spamtrain.log
-rw-rw-r-- 1 zimbra zimbra 1138 Jul 29 23:45 spamtrain.log.1.gz
-rw-rw-r-- 1 zimbra zimbra 278 Jul 28 23:45 spamtrain.log.2.gz
-rw-rw-r-- 1 zimbra zimbra 279 Jul 27 23:45 spamtrain.log.3.gz
-rw-rw-r-- 1 zimbra zimbra 278 Jul 26 23:45 spamtrain.log.4.gz
-rw-rw-r-- 1 zimbra zimbra 277 Jul 25 23:45 spamtrain.log.5.gz
-rw-rw-r-- 1 zimbra zimbra 277 Jul 24 23:45 spamtrain.log.6.gz
-rw-rw-r-- 1 zimbra zimbra 277 Jul 23 23:45 spamtrain.log.7.gz
-rw-r--r-- 1 zimbra zimbra 220 Jul 23 2018 sqlMigration.log
-rw-r----- 1 zimbra zimbra 72 Jul 29 08:45 stacktrace.12335.20190729084500
-rw-r----- 1 zimbra zimbra 72 Jul 26 09:58 stacktrace.15051.20190726095821
-rw-r----- 1 zimbra zimbra 6 Jul 30 11:53 swatch.pid
-rw-r----- 1 zimbra zimbra 0 May 17 2018 sync.log
-rw-r----- 1 zimbra zimbra 0 May 17 2018 syncstate.log
-rw-r----- 1 zimbra zimbra 0 May 17 2018 synctrace.log
-rw-r----- 1 zimbra zimbra 5805991 Jul 18 23:50 trace_log.2019_07_18
-rw-r----- 1 zimbra zimbra 2357679 Jul 19 23:50 trace_log.2019_07_19
-rw-r----- 1 zimbra zimbra 1473455 Jul 20 23:58 trace_log.2019_07_20
-rw-r----- 1 zimbra zimbra 1976411 Jul 21 23:50 trace_log.2019_07_21
-rw-r----- 1 zimbra zimbra 3596915 Jul 22 23:50 trace_log.2019_07_22
-rw-r----- 1 zimbra zimbra 3807058 Jul 23 23:59 trace_log.2019_07_23
-rw-r----- 1 zimbra zimbra 2723130 Jul 24 23:59 trace_log.2019_07_24
-rw-r----- 1 zimbra zimbra 2458404 Jul 25 23:59 trace_log.2019_07_25
-rw-r----- 1 zimbra zimbra 2768249 Jul 26 23:59 trace_log.2019_07_26
-rw-r----- 1 zimbra zimbra 3077314 Jul 27 23:50 trace_log.2019_07_27
-rw-r----- 1 zimbra zimbra 1594495 Jul 28 23:50 trace_log.2019_07_28
-rw-r----- 1 zimbra zimbra 521257 Jul 29 08:45 trace_log.2019_07_29
-rw-r----- 1 root root 0 Jul 30 11:51 trace_log.2019_07_30
-rw-r----- 1 zimbra zimbra 6 Jul 30 11:51 unbound.pid
-rw-r----- 1 zimbra zimbra 0 May 17 2018 wbxml.log
-rw-r----- 1 zimbra zimbra 0 May 17 2018 zmconfigd-audit.log
-rw-r----- 1 zimbra zimbra 15084 Apr 11 06:26 zmconfigd-log4j.log
-rw-r----- 1 zimbra zimbra 6 Jul 30 11:51 zmconfigd.pid
-rw-rw-r-- 1 zimbra zimbra 7 Jul 30 14:20 zmlogprocess.state
-rw-r----- 1 zimbra zimbra 29 Jul 30 11:51 zmlogswatch.out
-rw-r----- 1 zimbra zimbra 409 Jul 29 22:45 zmlogswatch.out.1.gz
-rw-r----- 1 zimbra zimbra 48 Jul 28 06:25 zmlogswatch.out.2.gz
-rw-r----- 1 zimbra zimbra 457 Jul 27 23:50 zmlogswatch.out.3.gz
-rw-r----- 1 zimbra zimbra 48 Jul 26 09:59 zmlogswatch.out.4.gz
-rw-r----- 1 zimbra zimbra 49 Jul 25 06:25 zmlogswatch.out.5.gz
-rw-r--r-- 1 zimbra zimbra 6 Jul 30 14:26 zmmailboxd_java.pid
-rw-r--r-- 1 zimbra zimbra 6 Jul 30 14:26 zmmailboxd_manager.pid
-rw-r----- 1 zimbra zimbra 10624580 Jul 30 14:26 zmmailboxd.out
-rw-r----- 1 zimbra zimbra 1735 Jul 21 22:38 zmmailboxd.out.201907220000
-rw-r----- 1 zimbra zimbra 2731 Jul 22 23:02 zmmailboxd.out.201907230000
-rw-r----- 1 zimbra zimbra 2265 Jul 23 23:50 zmmailboxd.out.201907240000
-rw-r----- 1 zimbra zimbra 2061 Jul 24 23:40 zmmailboxd.out.201907250000
-rw-r----- 1 zimbra zimbra 1957 Jul 25 23:51 zmmailboxd.out.201907260000
-rw-r----- 1 zimbra zimbra 9918 Jul 26 23:46 zmmailboxd.out.201907270000
-rw-r----- 1 zimbra zimbra 1726 Jul 27 22:31 zmmailboxd.out.201907280000
-rw-r----- 1 zimbra zimbra 1632 Jul 28 22:10 zmmailboxd.out.201907290000
-rw-r--r-- 1 zimbra zimbra 0 Jul 29 12:29 zmmailboxd.pid
-rw-r----- 1 zimbra zimbra 6796 May 17 2018 zmmyinit.log
-rw-r----- 1 zimbra zimbra 6 Jul 30 11:51 zmrrdfetch-server.pid
-rw------- 1 zimbra zimbra 79939 May 17 2018 zmsetup.20180517-153027.log
-rw------- 1 zimbra zimbra 86833 Jul 23 2018 zmsetup.20180723-114118.log
-rw------- 1 zimbra zimbra 78173 Aug 22 2018 zmsetup.20180822-184249.log
-rw------- 1 zimbra zimbra 80432 Apr 10 17:44 zmsetup.20190410-172518.log
-rw------- 1 zimbra zimbra 78306 Apr 10 19:41 zmsetup.20190410-192324.log
-rw------- 1 zimbra zimbra 77795 Apr 11 13:13 zmsetup.20190411-125633.log
-rw------- 1 zimbra zimbra 79738 Jul 29 09:18 zmsetup.20190729-084715.log
-rw------- 1 zimbra zimbra 78019 Jul 29 22:35 zmsetup.20190729-222051.log
-rw-r--r-- 1 zimbra zimbra 3284 Jul 30 14:11 zmswatch.out
-rw-r--r-- 1 zimbra zimbra 2035 Jul 30 03:44 zmswatch.out.1.gz
-rw-r--r-- 1 zimbra zimbra 102 Jul 28 06:25 zmswatch.out.2.gz
-rw-r--r-- 1 zimbra zimbra 99 Jul 27 06:25 zmswatch.out.3.gz
-rw-r--r-- 1 zimbra zimbra 244 Jul 26 10:03 zmswatch.out.4.gz
-rw-r--r-- 1 zimbra zimbra 102 Jul 25 06:25 zmswatch.out.5.gz

rosole · Post by **rosole** » Tue Jul 30, 2019 12:45 pm

And before upgrade log file: zmmailboxd.out was like below:

"qtp391914049-27633" #27633 prio=5 os_prio=0 cpu=6.07ms elapsed=18.03s tid=0x00007f722c342000 nid=0x2f3d runnable [0x00007f724c102000]
java.lang.Thread.State: RUNNABLE
at sun.nio.ch.EPoll.wait(java.base@11.0.2/Native Method)
at sun.nio.ch.EPollSelectorImpl.doSelect(java.base@11.0.2/EPollSelectorImpl.java:120)
at sun.nio.ch.SelectorImpl.lockAndDoSelect(java.base@11.0.2/SelectorImpl.java:124)
- locked <0x0000000703d005c0> (a sun.nio.ch.Util$2)
- locked <0x0000000703d00568> (a sun.nio.ch.EPollSelectorImpl)
at sun.nio.ch.SelectorImpl.select(java.base@11.0.2/SelectorImpl.java:141)
at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:232)
at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:180)
at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceAndRun(ExecuteProduceConsume.java:162)
at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.run(ExecuteProduceConsume.java:147)
at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:654)
at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:572)
at java.lang.Thread.run(java.base@11.0.2/Thread.java:834)

"Attach Listener" #27634 daemon prio=9 os_prio=0 cpu=0.17ms elapsed=10.52s tid=0x00007f721800c000 nid=0x3054 runnable [0x0000000000000000]
java.lang.Thread.State: RUNNABLE

"VM Thread" os_prio=0 cpu=132972.93ms elapsed=254726.76s tid=0x00007f7268c11000 nid=0xc14 runnable

"GC Thread#0" os_prio=0 cpu=19105.01ms elapsed=254726.90s tid=0x00007f726803a000 nid=0xc06 runnable

"GC Thread#1" os_prio=0 cpu=18919.63ms elapsed=254719.52s tid=0x00007f722c001000 nid=0xc83 runnable

"G1 Main Marker" os_prio=0 cpu=5.83ms elapsed=254726.89s tid=0x00007f726807a000 nid=0xc08 runnable

"G1 Conc#0" os_prio=0 cpu=640.91ms elapsed=254726.89s tid=0x00007f726807c000 nid=0xc0a runnable

"G1 Refine#0" os_prio=0 cpu=230.13ms elapsed=254726.81s tid=0x00007f7268b46000 nid=0xc10 runnable

"G1 Refine#1" os_prio=0 cpu=95.84ms elapsed=254697.98s tid=0x00007f7230001800 nid=0xea6 runnable

"G1 Young RemSet Sampling" os_prio=0 cpu=23309.39ms elapsed=254726.81s tid=0x00007f7268b48000 nid=0xc12 runnable

But after "successfully" upgrade got now:

OpenJDK 64-Bit Server VM warning: .hotspot_compiler file is present but has been ignored. Run with -XX:CompileCommandFile=.hotspot_compiler to load the file.
2019-07-30 14:26:44.164:INFO::main: Logging initialized @1994ms to org.eclipse.jetty.util.log.StdErrLog
JettyMonitor monitoring thread pool QueuedThreadPool[qtp961409111]@394df057{STOPPED,10<=0<=250,i=0,r=-1,q=0}[NO_TRY]
[4.241s][info][gc] GC(0) Pause Young (Concurrent Start) (Metadata GC Threshold) 60M->9M(4096M) 38.858ms
[4.243s][info][gc] GC(1) Concurrent Cycle
[4.275s][info][gc] GC(1) Pause Remark 10M->10M(4096M) 7.536ms
[4.279s][info][gc] GC(1) Pause Cleanup 10M->10M(4096M) 0.479ms
Zimbra server reserving server socket port=7143 bindaddr=null ssl=false
Zimbra server reserving server socket port=7993 bindaddr=null ssl=false
Zimbra server reserving server socket port=7025 bindaddr=null ssl=false
2019-07-30 14:26:46.516:INFO:oejs.SetUIDListener:main: Setting umask=027
2019-07-30 14:26:46.517:INFO:oejs.SetUIDListener:main: Current rlimit_nofiles (soft=524288, hard=524288)
2019-07-30 14:26:46.518:INFO:oejs.SetUIDListener:main: Set rlimit_nofiles (soft=65536, hard=65536)
[4.378s][info][gc] GC(1) Concurrent Cycle 135.023ms
2019-07-30 14:26:46.550:INFO:oejs.SetUIDListener:main: Opened ServerConnector@bd4dc25{HTTP/1.1,[http/1.1]}{localhost:8080}
2019-07-30 14:26:46.551:INFO:oejs.SetUIDListener:main: Opened ServerConnector@7526515b{SSL,[ssl, http/1.1]}{0.0.0.0:8443}
2019-07-30 14:26:46.551:INFO:oejs.SetUIDListener:main: Opened ServerConnector@8f2ef19{SSL,[ssl, http/1.1]}{0.0.0.0:7071}
2019-07-30 14:26:46.551:INFO:oejs.SetUIDListener:main: Opened ServerConnector@7674f035{SSL,[ssl, http/1.1]}{0.0.0.0:7073}
2019-07-30 14:26:46.552:INFO:oejs.SetUIDListener:main: Opened ServerConnector@25ce9dc4{SSL,[ssl, http/1.1]}{0.0.0.0:7072}
2019-07-30 14:26:46.552:INFO:oejs.SetUIDListener:main: Setting GID=999
2019-07-30 14:26:46.555:INFO:oejs.SetUIDListener:main: Setting UID=999
2019-07-30 14:26:46.567:INFO:oejs.Server:main: jetty-9.4.18.v20190429; built: 2019-04-29T20:42:08.989Z; git: e1bc35120a6617ee3df052294e433f3a25ce7097; jvm 11.0.2+9
2019-07-30 14:26:48.101:WARN:oeja.AnnotationParser:qtp961409111-15: org.apache.batik.i18n.ExtendedLocalizable scanned from multiple locations: jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/batik-i18n-1.9.jar!/org/apache/batik/i18n/ExtendedLocalizable.class, jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/batik-util-1.8.jar!/org/apache/batik/i18n/ExtendedLocalizable.class
2019-07-30 14:26:48.101:WARN:oeja.AnnotationParser:qtp961409111-15: org.apache.batik.i18n.LocaleGroup scanned from multiple locations: jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/batik-i18n-1.9.jar!/org/apache/batik/i18n/LocaleGroup.class, jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/batik-util-1.8.jar!/org/apache/batik/i18n/LocaleGroup.class
2019-07-30 14:26:48.102:WARN:oeja.AnnotationParser:qtp961409111-15: org.apache.batik.i18n.Localizable scanned from multiple locations: jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/batik-i18n-1.9.jar!/org/apache/batik/i18n/Localizable.class, jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/batik-util-1.8.jar!/org/apache/batik/i18n/Localizable.class
2019-07-30 14:26:48.102:WARN:oeja.AnnotationParser:qtp961409111-15: org.apache.batik.i18n.LocalizableSupport scanned from multiple locations: jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/batik-i18n-1.9.jar!/org/apache/batik/i18n/LocalizableSupport.class, jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/batik-util-1.8.jar!/org/apache/batik/i18n/LocalizableSupport.class
[6.899s][info][gc] GC(2) Pause Young (Normal) (G1 Evacuation Pause) 203M->20M(4096M) 43.115ms
2019-07-30 14:26:49.391:WARN:oeja.AnnotationParser:qtp961409111-13: javax.ws.rs.ApplicationPath scanned from multiple locations: jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/javax.ws.rs-api-2.0-m10.jar!/javax/ws/rs/ApplicationPath.class, jar:file:///opt/zimbra/jetty_base/webapps/service/WEB-INF/lib/jersey-core-1.11.jar!/javax/ws/rs/ApplicationPath.class

And i think, this is the issue: WARN:oeja.AnnotationParser:qtp

L. Mark Stone · Post by **L. Mark Stone** » Tue Jul 30, 2019 3:42 pm

So first, let's conform /etc/hosts to Zimbra's requirements:

Code: Select all

127.0.0.1 localhost.localdomain localhost
10.10.10.11 zimbra.domain.pl zimbra

Next, please post the output of (as root):

Code: Select all

ifconfig

And then (as zimbra):

Code: Select all

zmprov gs `zmhostname` zimbraIPMode
host zimbra.domain.pl
host zimbra
host 10.10.10.11

Mark

rosole · Post by **rosole** » Tue Jul 30, 2019 4:57 pm

Hello Mark.
ifconfig
ens160 Link encap:Ethernet HWaddr 00:0c:29:98:00:cd
inet addr:10.10.10.11 Bcast:10.10.10.255 Mask:255.255.255.0
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:2359050 errors:0 dropped:253 overruns:0 frame:0
TX packets:2148137 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:870582634 (870.5 MB) TX bytes:1101573710 (1.1 GB)

zmprov gs `zmhostname` zimbraIPMode
ERROR: zclient.IO_ERROR (invoke Connection refused (Connection refused), server: localhost) (cause: java.net.ConnectException Connection refused (Connection refused))

L. Mark Stone · Post by **L. Mark Stone** » Tue Jul 30, 2019 7:28 pm

Lots of dropped packets there... are you running a firewall on this server, or AppArmor/SELinux?

Also, please rerun the zmprov command with the -l switch:

Code: Select all

zmprov -l gs `zmhostname` zimbraIPMode

Mark

Zimbra Forums

ZCS OSE cannot start after upgrade to 8.8.15

ZCS OSE cannot start after upgrade to 8.8.15

Re: ZCS OSE cannot start after upgrade to 8.8.15

Re: ZCS OSE cannot start after upgrade to 8.8.15

Re: ZCS OSE cannot start after upgrade to 8.8.15

Re: ZCS OSE cannot start after upgrade to 8.8.15

Re: ZCS OSE cannot start after upgrade to 8.8.15

Re: ZCS OSE cannot start after upgrade to 8.8.15

Re: ZCS OSE cannot start after upgrade to 8.8.15

Re: ZCS OSE cannot start after upgrade to 8.8.15

Re: ZCS OSE cannot start after upgrade to 8.8.15