Very Slow Email Delivery

Discuss your pilot or production implementation with other Zimbra admins or our engineers.
combrolegit
Posts: 19
Joined: Tue Dec 31, 2019 7:42 am

Re: Very Slow Email Delivery

Post by combrolegit »

zimico wrote:Hi,
You have to confirm that your server is not compromised before using inplace upgrade.
And if your server is ok, please see: https://wiki.zimbra.com/wiki/Spamming_troubleshooting
Regards,
Minh.
Sorry with this question, but how to identify that my server is compromised since the domain shows valid domain from our own server, please let me know how to identify that my server is compromised, thank you so much.
combrolegit
Posts: 19
Joined: Tue Dec 31, 2019 7:42 am

Re: Very Slow Email Delivery

Post by combrolegit »

I just ran this command:

cat /var/log/zimbra.log | sed -n 's/.*sasl_username=//p' | sort | uniq -c | sort -nr, adn the result as follow:
Capture1.PNG
Capture1.PNG (9 KiB) Viewed 3737 times
Is that an ok value?
User avatar
zimico
Outstanding Member
Outstanding Member
Posts: 225
Joined: Mon Nov 14, 2016 8:03 am
Location: Vietnam
ZCS/ZD Version: 8.8.15 P3
Contact:

Re: Very Slow Email Delivery

Post by zimico »

Dear,
In case you think your server may be compromised, Please investigate the output of:
#su – zimbra

$zmcontrol -v

$grep python-requests /opt/zimbra/log/access_log* $ grep downloads /opt/zimbra/log/access_log* | grep -i jsp

$ ls -lrth /var/tmp/*.sh
$ ls -lrth /opt/zimbra/log/*.sh

$ crontab -l | egrep -i ‘zmmailboxdwatch|zmstorewatch’
$ crontab -l | egrep -i ‘\.sh|\.py’

Best regards,
Minh.
combrolegit
Posts: 19
Joined: Tue Dec 31, 2019 7:42 am

Re: Very Slow Email Delivery

Post by combrolegit »

zimico wrote:Dear,
In case you think your server may be compromised, Please investigate the output of:
#su – zimbra

$zmcontrol -v

$grep python-requests /opt/zimbra/log/access_log* $ grep downloads /opt/zimbra/log/access_log* | grep -i jsp

$ ls -lrth /var/tmp/*.sh
$ ls -lrth /opt/zimbra/log/*.sh

$ crontab -l | egrep -i ‘zmmailboxdwatch|zmstorewatch’
$ crontab -l | egrep -i ‘\.sh|\.py’

Best regards,
Minh.
Hi Minh,

The result might not as expected, here is the result:

zimbra@mail03:/home/zmadmin$ zmcontrol -v
Release 8.8.15.GA.3869.UBUNTU18.64 UBUNTU18_64 FOSS edition, Patch 8.8.15_P6.
zimbra@mail03:/home/zmadmin$ grep python-requests /opt/zimbra/log/access_log* $ grep downloads /opt/zimbra/log/access_log* | grep -i jsp
grep: $: No such file or directory
grep: grep: No such file or directory
grep: downloads: No such file or directory
zimbra@mail03:/home/zmadmin$ ls -lrth /var/tmp/*.sh
ls: cannot access '/var/tmp/*.sh': No such file or directory
zimbra@mail03:/home/zmadmin$ ls -lrth /opt/zimbra/log/*.sh
ls: cannot access '/opt/zimbra/log/*.sh': No such file or directory
zimbra@mail03:/home/zmadmin$ crontab -l | egrep -i ‘zmmailboxdwatch|zmstorewatch’
zmstorewatch’: command not found
zimbra@mail03:/home/zmadmin$ crontab -l | egrep -i ‘\.sh|\.py’
.py’: command not found
zimbra@mail03:/home/zmadmin$
combrolegit
Posts: 19
Joined: Tue Dec 31, 2019 7:42 am

Re: Very Slow Email Delivery

Post by combrolegit »

Well actually this is not an inplace upgrade, I move it to new server.
User avatar
zimico
Outstanding Member
Outstanding Member
Posts: 225
Joined: Mon Nov 14, 2016 8:03 am
Location: Vietnam
ZCS/ZD Version: 8.8.15 P3
Contact:

Re: Very Slow Email Delivery

Post by zimico »

Hi,
Sorry for the cut & past cli. Here it is:

Code: Select all

grep python-requests /opt/zimbra/log/access_log*
grep downloads /opt/zimbra/log/access_log* | grep -i jsp
ls -lrth /var/tmp/*.sh
ls -lrth /opt/zimbra/log/*.sh
crontab -l | egrep -i 'zmmailboxdwatch|zmstorewatch'
crontab -l | egrep -i '\.sh|\.y'
If you don't have 100% CPU. I think you then can focus on spam troublshooting in the wiki which I listed in previous post.
Best regards,
Minh.
combrolegit
Posts: 19
Joined: Tue Dec 31, 2019 7:42 am

Re: Very Slow Email Delivery

Post by combrolegit »

zimico wrote:Hi,
Sorry for the cut & past cli. Here it is:

Code: Select all

grep python-requests /opt/zimbra/log/access_log*
grep downloads /opt/zimbra/log/access_log* | grep -i jsp
ls -lrth /var/tmp/*.sh
ls -lrth /opt/zimbra/log/*.sh
crontab -l | egrep -i 'zmmailboxdwatch|zmstorewatch'
crontab -l | egrep -i '\.sh|\.y'
If you don't have 100% CPU. I think you then can focus on spam troublshooting in the wiki which I listed in previous post.
Best regards,
Minh.
Hi Minh, here's the result, what do you think?

Code: Select all

zimbra@mail03:/home/zmadmin$ grep python-requests /opt/zimbra/log/access_log*
/opt/zimbra/log/access_log.2020-01-30:71.6.199.23 - - [30/Jan/2020:06:01:04 +0000] "GET /favicon.ico HTTP/1.1" 404 1477 "-" "python-requests/2.19.1" 12
/opt/zimbra/log/access_log.2020-01-31:188.165.216.213 - - [31/Jan/2020:05:05:28 +0000] "GET / HTTP/1.0" 200 4833 "-" "python-requests/2.22.0" 38
/opt/zimbra/log/access_log.2020-02-03:77.247.110.73 - - [03/Feb/2020:15:07:20 +0000] "GET //a2billing/customer/templates/default/footer.tpl HTTP/1.0" 404 1477 "-" "python-requests/2.6.0 CPython/2.7.5 Linux/3.10.0-1062.9.1.el7.x86_64" 25
/opt/zimbra/log/access_log.2020-02-03:77.247.110.73 - - [03/Feb/2020:19:42:09 +0000] "GET //a2billing/customer/templates/default/footer.tpl HTTP/1.0" 404 1477 "-" "python-requests/2.6.0 CPython/2.7.5 Linux/3.10.0-1062.9.1.el7.x86_64" 12
/opt/zimbra/log/access_log.2020-02-04:94.102.49.190 - - [04/Feb/2020:16:41:44 +0000] "GET /favicon.ico HTTP/1.0" 404 1477 "-" "python-requests/2.10.0" 10
/opt/zimbra/log/access_log.2020-02-06:176.31.110.135 - - [06/Feb/2020:20:52:59 +0000] "GET / HTTP/1.0" 200 4832 "-" "python-requests/2.22.0" 20
/opt/zimbra/log/access_log.2020-02-08:185.164.72.119 - - [08/Feb/2020:14:29:35 +0000] "POST /Autodiscover/Autodiscover.xml HTTP/1.1" 400 293 "-" "python-requests/2.22.0" 6
/opt/zimbra/log/access_log.2020-02-08:185.164.72.119 - - [08/Feb/2020:15:12:30 +0000] "POST /Autodiscover/Autodiscover.xml HTTP/1.0" 400 293 "-" "python-requests/2.22.0" 5
/opt/zimbra/log/access_log.2020-02-09:71.6.158.166 - - [09/Feb/2020:14:25:47 +0000] "GET /favicon.ico HTTP/1.0" 404 1477 "-" "python-requests/2.10.0" 20
/opt/zimbra/log/access_log.2020-02-09:185.164.72.119 - - [09/Feb/2020:15:07:42 +0000] "POST /Autodiscover/Autodiscover.xml HTTP/1.1" 400 293 "-" "python-requests/2.22.0" 11
/opt/zimbra/log/access_log.2020-02-09:185.164.72.119 - - [09/Feb/2020:23:55:31 +0000] "POST /Autodiscover/Autodiscover.xml HTTP/1.0" 400 293 "-" "python-requests/2.22.0" 53
/opt/zimbra/log/access_log.2020-02-11:80.82.77.139 - - [11/Feb/2020:07:10:29 +0000] "GET /favicon.ico HTTP/1.1" 404 1477 "-" "python-requests/2.13.0" 33
/opt/zimbra/log/access_log.2020-02-11:37.187.74.157 - - [11/Feb/2020:07:10:35 +0000] "GET / HTTP/1.1" 200 4831 "-" "python-requests/2.22.0" 18
/opt/zimbra/log/access_log.2020-02-12:185.164.72.119 - - [12/Feb/2020:09:46:20 +0000] "POST /Autodiscover/Autodiscover.xml HTTP/1.1" 400 293 "-" "python-requests/2.22.0" 6
zimbra@mail03:/home/zmadmin$ grep downloads /opt/zimbra/log/access_log* | grep -i jsp
zimbra@mail03:/home/zmadmin$ ls -lrth /var/tmp/*.sh
ls: cannot access '/var/tmp/*.sh': No such file or directory
zimbra@mail03:/home/zmadmin$ ls -lrth /opt/zimbra/log/*.sh
ls: cannot access '/opt/zimbra/log/*.sh': No such file or directory
zimbra@mail03:/home/zmadmin$ crontab -l | egrep -i 'zmmailboxdwatch|zmstorewatch'
zimbra@mail03:/home/zmadmin$ crontab -l | egrep -i '\.sh|\.y'
zimbra@mail03:/home/zmadmin$

User avatar
zimico
Outstanding Member
Outstanding Member
Posts: 225
Joined: Mon Nov 14, 2016 8:03 am
Location: Vietnam
ZCS/ZD Version: 8.8.15 P3
Contact:

Re: Very Slow Email Delivery

Post by zimico »

Hi,
The output seems to be normal.
Did you check incoming spam issue and outgoing spam issue?
Please show the output of:
as root:

Code: Select all

# cat /var/log/zimbra.log | sed -n 's/.*sasl_username=//p' | sort | uniq -c | sort -nr
# /opt/zimbra/libexec/zmqstat
# cat /var/log/zimbra.log | grep -Eo '[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}'| sort | uniq -c | sort -nr
# netstat -np --protocol=inet | grep ESTABLISHED | grep :smtpd
# htop (or top)
Regards,
Minh.
combrolegit
Posts: 19
Joined: Tue Dec 31, 2019 7:42 am

Re: Very Slow Email Delivery

Post by combrolegit »

zimico wrote:Hi,
The output seems to be normal.
Did you check incoming spam issue and outgoing spam issue?
Please show the output of:
as root:

Code: Select all

# cat /var/log/zimbra.log | sed -n 's/.*sasl_username=//p' | sort | uniq -c | sort -nr
# /opt/zimbra/libexec/zmqstat
# cat /var/log/zimbra.log | grep -Eo '[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}'| sort | uniq -c | sort -nr
# netstat -np --protocol=inet | grep ESTABLISHED | grep :smtpd
# htop (or top)
Regards,
Minh.

Code: Select all

zimbra@mail03:/home/zmadmin$ cat /var/log/zimbra.log | sed -n 's/.*sasl_username=//p' | sort | uniq -c | sort -nr
     75 no-reply@te***up.com
     68 mthu1.11400057@t***m.co.id
     62 mthu1.11400281@t***m.co.id
     54 sogest02.21400046@t***m.co.id
     46 megar1.11400280@t***m.co.id
     43 seilam01.21400052@t***m.co.id
     43 metlam01.21400007@t***m.co.id
     42 mthr1.11400024@t***em.co.id
     39 sogmac01.21400035@t***m.co.id
     38 mthr1.11400281@t***m.co.id
     36 tntpackdev.adm@t***up.com
     36 sogcli01.21400004@t***m.co.id
     36 apjptt.mks@t***p.com
     33 mthu1.11400205@t***m.co.id

Currently mail queue already back to normal, but often happen later...

Code: Select all

root@mail03:/home/zmadmin# /opt/zimbra/libexec/zmqstat
incoming=0
corrupt=0
deferred=0
hold=0
active=2

Code: Select all

root@mail03:/home/zmadmin# cat /var/log/zimbra.log | grep -Eo '[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}'| sort | uniq -c | sort -nr
 486260 127.0.0.1
  75193 10.66.1.16
  21054 10.66.2.22
  18656 192.168.100.20
  16315 10.66.1.20
   7089 10.63.61.4
   6203 92.118.38.41
   5199 202.158.29.162
   4941 10.66.1.14
   4157 117.102.84.140
   1790 129.10.16.21
    735 141.0.0.118

Code: Select all

root@mail03:/home/zmadmin# netstat -np --protocol=inet | grep ESTABLISHED | grep :smtpd
root@mail03:/home/zmadmin#
User avatar
zimico
Outstanding Member
Outstanding Member
Posts: 225
Joined: Mon Nov 14, 2016 8:03 am
Location: Vietnam
ZCS/ZD Version: 8.8.15 P3
Contact:

Re: Very Slow Email Delivery

Post by zimico »

Your system parameters seem to be normal. Is mail flow ok now?
Best regards,
Minh.
Post Reply