zmprov does not work

Discuss your pilot or production implementation with other Zimbra admins or our engineers.
aek
Posts: 14
Joined: Fri Sep 22, 2017 8:59 am

zmprov does not work

Post by aek »

Code: Select all

$ zmprov -v       
ERROR: service.PROXY_ERROR (error while proxying request to target server: HTTP/1.1 503 Service Unavailable)
com.zimbra.common.service.ServiceException: error while proxying request to target server: HTTP/1.1 503 Service Unavailable
ExceptionId:main:1510652693854:c8cdaa2b514f9e79
Code:service.PROXY_ERROR Arg:(url, STR, "https://localhost:7071/service/admin/soap/AuthRequest")
	at com.zimbra.common.service.ServiceException.PROXY_ERROR(ServiceException.java:375)
	at com.zimbra.common.soap.SoapHttpTransport.invoke(SoapHttpTransport.java:282)
	at com.zimbra.common.soap.SoapHttpTransport.invoke(SoapHttpTransport.java:190)
	at com.zimbra.common.soap.SoapHttpTransport.invoke(SoapHttpTransport.java:183)
	at com.zimbra.common.soap.SoapTransport.invoke(SoapTransport.java:446)
	at com.zimbra.common.soap.SoapTransport.invokeWithoutSession(SoapTransport.java:432)
	at com.zimbra.cs.account.soap.SoapProvisioning.invokeRequest(SoapProvisioning.java:471)
	at com.zimbra.cs.account.soap.SoapProvisioning.invoke(SoapProvisioning.java:516)
	at com.zimbra.cs.account.soap.SoapProvisioning.soapAdminAuthenticate(SoapProvisioning.java:406)
	at com.zimbra.cs.account.soap.SoapProvisioning.soapZimbraAdminAuthenticate(SoapProvisioning.java:435)
	at com.zimbra.cs.account.ProvUtil.initProvisioning(ProvUtil.java:1017)
	at com.zimbra.cs.account.ProvUtil.main(ProvUtil.java:3921)
It seems everything is OK:

Code: Select all

# netstat -anltp | egrep '^tcp' | grep LISTEN | awk '{print $4 " "$7}' | sed -e 's/.*://' | sort -n | uniq
21 2190/xinetd
25 4595/master
53 1836/named
110 11221/nginx
143 11221/nginx
389 2354/slapd
443 11221/nginx
465 4595/master
587 4595/master
953 1836/named
993 11221/nginx
995 11221/nginx
3310 4346/clamd
7025 12160/java
7071 12160/java
7072 12160/java
7073 12160/java
7110 12160/java
7143 12160/java
7171 2374/java
7306 3782/mysqld
7780 4395/httpd
7993 12160/java
7995 12160/java
8080 12160/java
8143 5073/java
8443 12160/java
8465 4366/opendkim
8993 5073/java
10024 4148/amavisd
10025 4595/master
10026 4148/amavisd
10027 4595/master
10028 4595/master
10029 4595/master
10030 4595/master
10032 4148/amavisd
11211 4067/memcached
22222 2239/sshd
23232 4112/perl
23233 4114/perl
phoenix
Ambassador
Ambassador
Posts: 27272
Joined: Fri Sep 12, 2014 9:56 pm
Location: Liverpool, England

Re: zmprov does not work

Post by phoenix »

aek wrote:

Code: Select all

$ zmprov -v       
ERROR: service.PROXY_ERROR (error while proxying request to target server: HTTP/1.1 503 Service Unavailable)[/quote]Well, zmprov does work but the problem is with your configuration and you haven't bothered to give the version of ZCS that's being used nor what exactly happened to cause this problem nor whether this is a new install, an updated install nor whether you've made any changes to your server nor any details of the steps you've taken to resolve or research this issue. I've removed your post in the other thread in which you've recently posted as it's a duplicate of this one, do not post duplices on the forums, it makes it difficult to answer the correct post and follow a specific thread on a topic. Did you follow any of the instructions in the other thread to try and resolve the problem or, perhaps, and of these topics: https://www.startpage.com/do/dsearch?query=%2Bzimbra+%2B%22error+while+proxying+request+to+target+server%22&cat=web&pl=opensearch&language=english
Regards

Bill

Rspamd: A high performance spamassassin replacement

Per ardua ad astra
aek
Posts: 14
Joined: Fri Sep 22, 2017 8:59 am

Re: zmprov does not work

Post by aek »

phoenix wrote:Well, zmprov does work but the problem is with your configuration and you haven't bothered to give the version of ZCS that's being used nor what exactly happened to cause this problem nor whether this is a new install, an updated install nor whether you've made any changes to your server nor any details of the steps you've taken to resolve or research this issue.
This is new installation on the CentOS 7 from sources: https://github.com/Zimbra/zm-build (8.8.4)
phoenix wrote: I've removed your post in the other thread in which you've recently posted as it's a duplicate of this one, do not post duplices on the forums, it makes it difficult to answer the correct post and follow a specific thread on a topic.
OK. I have the same symptoms as here: viewtopic.php?f=15&t=61518#p274765
phoenix wrote:Did you follow any of the instructions in the other thread to try and resolve the problem or, perhaps, and of these topics: https://www.startpage.com/do/dsearch?qu ... ge=english
https://wiki.zimbra.com/wiki/Network_Se ... s_occurred
zmprov does not work

https://wiki.zimbra.com/wiki/Zimbra_Adm ... t_is_blank

Code: Select all

[zimbra@zcs884 ~]$ zmmailbox -z -m account@example.com gaf
ERROR: service.PROXY_ERROR (error while proxying request to target server: HTTP/1.1 503 Service Unavailable)
[zimbra@zcs884 ~]$ zmprov fc all
ERROR: service.PROXY_ERROR (error while proxying request to target server: HTTP/1.1 503 Service Unavailable)
[zimbra@zcs884 ~]$ curl -v http://localhost:80
* Rebuilt URL to: http://localhost:80/
*   Trying 127.0.0.1...
* connect to 127.0.0.1 port 80 failed: Connection refused
* Failed to connect to localhost port 80: Connection refused
* Closing connection 0
curl: (7) Failed to connect to localhost port 80: Connection refused
[zimbra@zcs884 ~]$ netstat -anltp | grep 80
tcp        0      0 127.0.0.1:8080              0.0.0.0:*                   LISTEN      30450/java          
tcp        0      0 :::7780                     :::*                        LISTEN      31377/httpd
[zimbra@zcs884 ~]$ lsof -i :80
[zimbra@zcs884 ~]$
Apparently, the problem is clear: nginx not listening to port 80. Why? It is necessary to look....
aek
Posts: 14
Joined: Fri Sep 22, 2017 8:59 am

Re: zmprov does not work

Post by aek »

aek wrote: Apparently, the problem is clear: nginx not listening to port 80. Why? It is necessary to look....
Though: https://wiki.zimbra.com/wiki/Zimbra_Pro ... _the_Ports
there not is 80 8993/nginx... Then what's the problem?
aek
Posts: 14
Joined: Fri Sep 22, 2017 8:59 am

Re: zmprov does not work

Post by aek »

aek wrote:
aek wrote: Apparently, the problem is clear: nginx not listening to port 80. Why? It is necessary to look....
Though: https://wiki.zimbra.com/wiki/Zimbra_Pro ... _the_Ports
there not is 80 8993/nginx... Then what's the problem?

Code: Select all

[zimbra@zcs884 ~]$ /opt/zimbra/libexec/zmproxyconfig -e -w -o -a 8080:80:8443:443 -x redirect -H `zmhostname`
[zimbra@zcs884 ~]$ zmcontrol restart
....
[zimbra@zcs884 ~]$ curl -v localhost:80
* Rebuilt URL to: localhost:80/
*   Trying 127.0.0.1...
* Connected to localhost (127.0.0.1) port 80 (#0)
> GET / HTTP/1.1
> Host: localhost
> User-Agent: curl/7.49.1
> Accept: */*
> 
< HTTP/1.1 302 Moved Temporarily
< Server: nginx
< Date: Tue, 14 Nov 2017 14:27:34 GMT
< Content-Type: text/html
< Content-Length: 154
< Connection: keep-alive
< Location: https://localhost/
< 
<html>
<head><title>302 Found</title></head>
<body bgcolor="white">
<center><h1>302 Found</h1></center>
<hr><center>nginx</center>
</body>
</html>
* Connection #0 to host localhost left intact
but the problem is not solved.... :(
phoenix
Ambassador
Ambassador
Posts: 27272
Joined: Fri Sep 12, 2014 9:56 pm
Location: Liverpool, England

Re: zmprov does not work

Post by phoenix »

aek wrote:
aek wrote:
aek wrote: [zimbra@zcs884 ~]$ curl -v localhost:80
* Rebuilt URL to: localhost:80/
* Trying 127.0.0.1...
* Connected to localhost (127.0.0.1) port 80 (#0)
> GET / HTTP/1.1
> Host: localhost
> User-Agent: curl/7.49.1
> Accept: */*
>
< HTTP/1.1 302 Moved Temporarily
< Server: nginx
< Date: Tue, 14 Nov 2017 14:27:34 GMT
< Content-Type: text/html
< Content-Length: 154
< Connection: keep-alive
< Location: https://localhost/
<
<html>
<head><title>302 Found</title></head>
<body bgcolor="white">
<center><h1>302 Found</h1></center>
<hr><center>nginx</center>
</body>
</html>
* Connection #0 to host localhost left intact
[/code] but the problem is not solved.... :(
Why do you expect the curl command to give you any response when the Zimbra nginx does not listen on port 80. Why don't you try the following to see if nginx is actually alive:

Code: Select all

netstat -tulnp | grep nginx
Merely the fact you keep saying 'it doesn't work' gives us no information about what is your exact problem, you need to investigate whether all your ZCS services are running and what errors you see in the log files. I also don't understand why you felt it necessary to build ZCS from source when there's a perfectly good rpm build for CentOS.
Regards

Bill

Rspamd: A high performance spamassassin replacement

Per ardua ad astra
aek
Posts: 14
Joined: Fri Sep 22, 2017 8:59 am

Re: zmprov does not work

Post by aek »

phoenix wrote:Why do you expect the curl command to give you any response when the Zimbra nginx does not listen on port 80.
Because I switched zimbraReverseProxyMailMode to redirect. Now the Zimbra nginx does listen on port 80.

Code: Select all

# lsof -i :80
COMMAND   PID   USER   FD   TYPE DEVICE SIZE/OFF NODE NAME
nginx   28709 zimbra   10u  IPv4 444479      0t0  TCP *:http (LISTEN)
nginx   28710 zimbra   10u  IPv4 444479      0t0  TCP *:http (LISTEN)
nginx   28711 zimbra   10u  IPv4 444479      0t0  TCP *:http (LISTEN)
nginx   28712 zimbra   10u  IPv4 444479      0t0  TCP *:http (LISTEN)
phoenix wrote:Why don't you try the following to see if nginx is actually alive:

Code: Select all

netstat -tulnp | grep nginx

Code: Select all

# netstat -tulnp | grep nginx
tcp        0      0 0.0.0.0:443                 0.0.0.0:*                   LISTEN      28708/nginx 
tcp        0      0 0.0.0.0:993                 0.0.0.0:*                   LISTEN      28708/nginx  
tcp        0      0 0.0.0.0:995                 0.0.0.0:*                   LISTEN      28708/nginx   
tcp        0      0 0.0.0.0:110                 0.0.0.0:*                   LISTEN      28708/nginx  
tcp        0      0 0.0.0.0:143                 0.0.0.0:*                   LISTEN      28708/nginx  
tcp        0      0 0.0.0.0:80                  0.0.0.0:*                   LISTEN      28708/nginx
phoenix wrote:Merely the fact you keep saying 'it doesn't work' gives us no information about what is your exact problem, you need to investigate whether all your ZCS services are running and what errors you see in the log files.

Code: Select all

$ zmcontrol status
Host zcs884.mycompany.ru
	amavis                  Running
	antispam                Running
	antivirus               Running
	imapd                   Running
	ldap                    Running
	logger                  Running
	mailbox                 Running
	memcached               Running
	mta                     Running
	opendkim                Running
	proxy                   Running
	service webapp          Running
	snmp                    Running
	spell                   Running
	stats                   Running
	zimbra webapp           Running
	zimbraAdmin webapp      Running
	zimlet webapp           Running
	zmconfigd               Running
elinks https://zcs884.mycompany.ru
At that time: tail /opt/zimbra/log/access_log:

Code: Select all

127.0.0.1 - - [15/Nov/2017:07:28:20 +0000] "POST https://localhost:7071/service/admin/soap/GetDomainInfoRequest HTTP/1.1" 503 286 "-" "ZCS 8.8.4_GA_1015" 0
127.0.0.1 - - [15/Nov/2017:07:28:20 +0000] "POST https://localhost:7071/service/admin/soap/GetDomainInfoRequest HTTP/1.1" 503 286 "-" "ZCS 8.8.4_GA_1015" 0
192.168.233.29 - - [15/Nov/2017:07:28:20 +0000] "GET https://zcs884.mycompany.ru/ HTTP/1.0" 200 12391 "-" "ELinks/0.12pre5 (textmode; Linux; 80x24-2)" 75
After trying to log in (I am presented with an error "A network service error has occurred."):

Code: Select all

192.168.233.29 - - [15/Nov/2017:07:28:20 +0000] "GET https://zcs884.mycompany.ru/css/common,login,zhtml,skin.css?skin=harmony&v=171020171958 HTTP/1.0" 200 59030 "https://zcs884.mycompany.ru/" "ELinks/0.12pre5 (textmode; Linux; 80x24-2)" 29
192.168.233.29 - - [15/Nov/2017:07:28:33 +0000] "POST //localhost:8080/service/soap/AuthRequest HTTP/1.1" 503 271 "-" "zclient 8.8.4_GA_1015" 0
127.0.0.1 - - [15/Nov/2017:07:28:33 +0000] "POST https://localhost:7071/service/admin/soap/GetDomainInfoRequest HTTP/1.1" 503 286 "-" "ZCS 8.8.4_GA_1015" 0
192.168.233.29 - - [15/Nov/2017:07:28:33 +0000] "POST https://zcs884.mycompany.ru/ HTTP/1.0" 200 12640 "https://zcs884.mycompany.ru/" "ELinks/0.12pre5 (textmode; Linux; 80x24-2)" 76
chromium https://192.168.233.29:7071 (the page appears blank):

Code: Select all

10.10.100.100 - - [15/Nov/2017:07:12:24 +0000] "POST https://192.168.233.29:7071/service/admin/soap/GetDomainInfoRequest HTTP/1.1" 503 286 "https://192.168.233.29:7071/zimbraAdmin/" "Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/61.0.3163.100 Safari/537.36" 0
10.10.100.100 - - [15/Nov/2017:07:12:24 +0000] "POST https://192.168.233.29:7071/service/admin/soap/GetDomainInfoRequest HTTP/1.1" 503 286 "https://192.168.233.29:7071/zimbraAdmin/" "Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/61.0.3163.100 Safari/537.36" 0
10.10.100.100 - - [15/Nov/2017:07:12:24 +0000] "POST https://192.168.233.29:7071/service/admin/soap/GetDomainInfoRequest HTTP/1.1" 503 286 "https://192.168.233.29:7071/zimbraAdmin/" "Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/61.0.3163.100 Safari/537.36" 0
10.10.100.100 - - [15/Nov/2017:07:12:24 +0000] "POST https://192.168.233.29:7071/service/admin/soap/GetDomainInfoRequest HTTP/1.1" 503 286 "https://192.168.233.29:7071/zimbraAdmin/" "Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/61.0.3163.100 Safari/537.36" 1
#Repeats until you close the page[/code[b]]$ zmprov[/b]
tail /opt/zimbra/log/access_log:
[code]127.0.0.1 - - [15/Nov/2017:07:25:21 +0000] "POST https://localhost:7071/service/admin/soap/AuthRequest HTTP/1.1" 503 277 "-" "zmprov 8.8.4_GA_1015" 1
tail /opt/zimbra/log/trace_log:

Code: Select all

10:23:33.991:qtp1595953398-6079 OPENED SslConnection@7b9cc1a6{NEED_UNWRAP,eio=-1/-1,di=-1} -> HttpConnection@34f12fb0[DecryptedEndPoint@406ea344{/127.0.0.1:42211<->7071,Open,in,out,-,-,1/0,HttpConnection}->SelectChannelEndPoint@59c6ae0a{/127.0.0.1:42211<->7071,Open,in,out,-,-,1/0,SslConnection}{io=0/0,kio=0,kro=0}][p=HttpParser{s=START,0 of 0},g=HttpGenerator@2c34e3bd{s=START},c=HttpChannelOverHttp@3138966c{r=0,c=false,a=IDLE,uri=null}][b=null]
10:23:33.991:qtp1595953398-6079 OPENED HttpConnection@34f12fb0[DecryptedEndPoint@406ea344{/127.0.0.1:42211<->7071,Open,in,out,-,-,1/0,HttpConnection}->SelectChannelEndPoint@59c6ae0a{/127.0.0.1:42211<->7071,Open,in,out,-,-,1/0,SslConnection}{io=0/0,kio=0,kro=0}][p=HttpParser{s=START,0 of 0},g=HttpGenerator@2c34e3bd{s=START},c=HttpChannelOverHttp@3138966c{r=0,c=false,a=IDLE,uri=null}][b=null]
10:23:34.288:qtp1595953398-6099:https:https://localhost:7071/service/admin/soap/AuthRequest REQUEST 127.0.0.1 POST null; zmprov 8.8.4_GA_1015
10:23:34.289:qtp1595953398-6099:https:https://localhost:7071/service/admin/soap/AuthRequest RESPONSE 503 text/html;charset=iso-8859-1
10:23:34.308:qtp1595953398-6079 CLOSED HttpConnection@34f12fb0[DecryptedEndPoint@406ea344{/127.0.0.1:42211<->7071,CLOSED,ISHUT,OSHUT,-,-,19/0,HttpConnection}->SelectChannelEndPoint@59c6ae0a{/127.0.0.1:42211<->7071,CLOSED,ISHUT,OSHUT,-,-,19/0,SslConnection}{io=0/0,kio=-1,kro=-1}][p=HttpParser{s=CLOSED,0 of -1},g=HttpGenerator@2c34e3bd{s=START},c=HttpChannelOverHttp@3138966c{r=1,c=false,a=IDLE,uri=null}][b=HeapByteBuffer@49721fb9[p=0,l=0,c=17408,r=0]={<<<>>>POST /service/adm...\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00}]
10:23:34.308:qtp1595953398-6079 CLOSED SslConnection@7b9cc1a6{NEED_WRAP,eio=-1/-1,di=-1} -> HttpConnection@34f12fb0[DecryptedEndPoint@406ea344{/127.0.0.1:42211<->7071,CLOSED,ISHUT,OSHUT,-,-,19/0,HttpConnection}->SelectChannelEndPoint@59c6ae0a{/127.0.0.1:42211<->7071,CLOSED,ISHUT,OSHUT,-,-,19/0,SslConnection}{io=0/0,kio=-1,kro=-1}][p=HttpParser{s=CLOSED,0 of -1},g=HttpGenerator@2c34e3bd{s=START},c=HttpChannelOverHttp@3138966c{r=1,c=false,a=IDLE,uri=null}][b=HeapByteBuffer@49721fb9[p=0,l=0,c=17408,r=0]={<<<>>>POST /service/adm...\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00}]
phoenix wrote:I also don't understand why you felt it necessary to build ZCS from source when there's a perfectly good rpm build for CentOS.
Because it's 'not quite' CentOS...
diego.godoi
Posts: 1
Joined: Wed Nov 22, 2017 10:12 am

Re: zmprov does not work

Post by diego.godoi »

Good Morning,

I had a problem identical to yours yesterday. The zmprov service did not upload and did not load the adimin interface with the error (com.zimbra.common.service.ServiceException: error while proxying request to target server: HTTP / 1.1 503 Service Unavailable).
After breaking the head and studying the zimbra LOG a lot, we identified that we were testing and a file was created in the jetty directory with root permission and not with zimbra permission;

And this file did not let the zimbra increase its service;

If you created a file in the / opt / zimbra directory, please validate the directory permission, which may be the problem;

That was the problem we had in yesterday;

See you and good luck.
aek
Posts: 14
Joined: Fri Sep 22, 2017 8:59 am

Re: zmprov does not work

Post by aek »

diego.godoi wrote:Good Morning,

I had a problem identical to yours yesterday. The zmprov service did not upload and did not load the adimin interface with the error (com.zimbra.common.service.ServiceException: error while proxying request to target server: HTTP / 1.1 503 Service Unavailable).
After breaking the head and studying the zimbra LOG a lot, we identified that we were testing and a file was created in the jetty directory with root permission and not with zimbra permission;

And this file did not let the zimbra increase its service;

If you created a file in the / opt / zimbra directory, please validate the directory permission, which may be the problem;

That was the problem we had in yesterday;

See you and good luck.

Code: Select all

$ ls -l /opt/zimbra/jetty/
total 40
drwxr-xr-x 4 zimbra zimbra 4096 Oct 20 17:24 common
drwxr-xr-x 2 zimbra zimbra 4096 Nov 23 13:09 etc
drwxr-xr-x 3 zimbra zimbra 4096 Oct 20 16:48 lib
drwxr-xr-x 2 zimbra zimbra 4096 Oct 20 16:41 logs
drwxr-xr-x 2 zimbra zimbra 4096 Nov 23 13:09 modules
drwxr-x--- 2 zimbra zimbra 4096 Oct 20 16:48 resources
drwxr-xr-x 2 zimbra zimbra 4096 Nov 23 13:09 start.d
drwxr-xr-x 2 zimbra zimbra 4096 Oct 20 17:32 temp
drwxr-xr-x 6 zimbra zimbra 4096 Oct 20 17:24 webapps
drwxr-xr-x 8 zimbra zimbra 4096 Nov 23 13:10 work
asluiter
Posts: 1
Joined: Sun Dec 17, 2017 5:51 am

Re: zmprov does not work

Post by asluiter »

Could you please explain this a little more? I'm having this exact same issue. What file is created? Where is it created? What are the permissions supposed to be set at?
Post Reply